Clone track

...and last in this series of blog posts...

With the new version of Trackprofiler you can now "clone" an existing track into a new one and edit it. Simple as that.

GPX 1.1 and track routes

The first version Trackprofiler worked with only one kind of GPS files -- GPX. Not even all versions of GPX, only 1.0. That wasn't a big problem since 1.1 wasn't used very much at the time. With time I added more formats, but as for GPX handling -- Trackprofiler remained for years with 1.0.

Part of this latest rewrite is the ability to handle GPX 1.1.

Related to GPX handling comes another feature. Track files can contain three different kinds of entities:

• Tracks (with segments)
• Routes
• Waypoints

Tracks are a series of geo-locations recorded with a real GPS. Routes are "planning stages" for tracks, they usually contain fewer points (per unit of distance) and are created by the user on the computer without the need of a GPS unit. Routes are usually uploaded to the GPS (not downloaded from like tracks) and are used to help people navigate outdoor.

Trackprofiler is a GPS track editor, but tracks and routes are basically similar, and that's why you can convert track to route on-the-fly when downloading. But there was one thing missing, you couldn't upload a track consisting of only routes.

Now you can, on the upload page there is a checkbox "Convert routes". If checked routes will be converted to tracks and then you can edit them as with any other track on Trackprofiler.

Track snapshots and multistep undo

...so, let us start with the changes introduced with yesterday upgrade. The first I want to highlight is the "Undo" link when you are editing the track:

From now on, every change you make on a track is stored in a temporary memory with "snapshots" of the track. Snapshots are stored for 10-15 minutes from the moment you made the edit. When you are not happy with the changes you made -- you can reset the track to any of the snapshots (or, of course, reset it to original any time).

Faster Trackprofiler

After 4 weeks in beta testing the new version of Trackprofiler is deployed on the main site. This version brings many new features. I'll mention only one here (and leave the rest for another blogpost)...

In the last 3 months the entire code for GPX/track manipulation is rewritten from Python to Go(lang). The result is that all the track operations, graph and thumbnail reloading and map refresh time is now much faster than before.

More about this update soon...

Private tags and edit quicklinks

A new version is just deployed with a bunch of new features. 

First, you can now browse your own tracks by tags. When you visit your profile page you'll see your own tag cloud in the right column:

By clicking on any tag from the cloud, only your tracks tagged with that tag will be filtered. If you are careful to choose the right tags when uploading the track -- this is a good way to intuitively organize and quickly find tracks.

Next feature is track edit quick links. On any page where a list of tracks is shown, on your tracks you'll see edit icons/links:

  

The title and description can be changed without moving from the page. The icons below the map thumbnail are quick links to the most used edit options.

And the last option... Premium users can now decide for uploaded tracks to be private by default. Just select the checkbox in your profile edit page.

Moving to Leaflet and Openstreetmap

Until a few weeks ago, all embedded maps on Trackprofiler were based on Google maps, but I've made a switch to Leaflet powered maps with MapBox powered layers based on Openstreetmap data and Bing maps layer for aerial imagery. The functionality is more-or-less the same as before.

Why the switch? Google maps is still a great solution which thought me a lot about mapping and online geographical programming, but there are some important reasons why I decided to rewrite a big part of Trackprofiler:

• I believe that Openstreetmap is the future of mapping. It is a great community doing great efforts to create the best map data ever,
• Leaflet.js is a great mapping Javascript library, it is fast and easy to work with,
• Too restrictive terms of service,
• This allowed me to rewrite legacy parts of the application and this, in turn, will help to implement even more new features...

More open source projects

Regular readers of this blog already know, Trackprofiler is not only built on open source components but is actively releasing new open source projects.

Continuing with this trend, here are two new projects for all users of the Leaflet.js mapping framework:

• Leaflet editable polyline: point-by-point editor for (very) big polylines
• Leaflet clicks: a small plugin for custom events where you need to differentiate between single and double clicks. 

Simplify GPS tracks

If you happen to record a GPS track driving on the North Dakota State Highway 46, which...

...is a 123 mile (198 km) road in eastern North Dakota, crossing the Red River Valley between Streeter and Oxbow. It parallels and runs about twenty miles (32 km) south of Interstate 94, and is the longest stretch of straight road in North America. (Wikipedia)

...your track may have tens of thousands of points. Of course, since all of them present a straight line -- only two are important: The starting point and the ending point.

In that case you may use the new feature that is just deployed on Trackprofiler today. The "Simplify track" option will remove points where the track resembles a straight line. You can still decide how many points to choose (by selecting "More points" or "Less points" on the page).

Note that by removing intermediate points you remove the metadata, too. Most importantly the time (needed to compute the speed) and elevation.  But, sometimes those are not that important and you need your tracks to contain only the shape of the path taken. If that's the case -- enjoy simplifying your tracks!

Happy new 2014

Happy new 2014 year!

As you can see from posts on this blog -- 2013 was a busy year for Trackprofiler. This year will probably see some more features. In the last few weeks I was compiling a list of features from users, and here it is. This is what you can expect this year:

• Simplify tracks by removing too many points.
• OpenStreetMap and Leaflet integration instead of Google Maps.
• Easier handling of multiple tracks (bulk rename, change tags, delete, ...).
• Support for the newer GPX version of: GPX 1.1 (and possibly more track formats, as well).
• Show multiple tracks on one page. 
• Multistep undo. At the moment the only way to revert an edit is to use "Revert to original track". With multistep undo You'll be able to undo only single edits.
• Login with Mozilla persona.
• Better editing tools for tags, automatic detection of tags.
• Share waypoints between multiple tracks.
• Public Trackprofiler API.
• Bulk download of tracks on Dropbox.
• User-defined colors for track segments.
• Microsites (more on this later...).

In italic are the features planned for the first 3-4 months of this year.

This is not a definitive list, maybe some new and important features will be added, but I hope that at least two-thirds of this list will be ready this year.

Split and move GPS track segments

Some GPS devices record tracks with random errors like this:

I followed a smooth path here, the actual GPS track looks pretty irregular. To fix it you need to move every point, or just use a real GPS (instead of a smartphone "GPS" :-).

Other times your track may look something like this:

Note that the first part of the track follows the road but then it "decides" to leave the road (in reality, I never went off-road there) and then, later, it returns back on the road. To fix a problem like this one until now you had to use the "Edit points" option and move points one by one.

This is a problem people from CadenceVR had. For their product (a virtual reality cycling/training application), they need to have reliable tracks files. But when the GPS make an error like the above -- they need to fix many (thousands!) points by moving them one by one, and every move by the same few meters in the same direction.

After a few days of emailing back and forth with ideas on how to fix problems like this one, a new feature of Trackprofiler is born. You can now use the Segment editor to split segments and move them around:



How to do it...

• Click on the track to split the segment into two new (you can merge them back later)
• Drag the marker with the number at the start of the segment. The entire segment will move accordingly. 
• When the segment is moved where it should have been you can merge the segments again.

Track segment editor

Let's suppose you recorded a track like this:



Note that the path is "broken" in 4 smaller parts. Those parts are called track segments. Every track can have multiple track segments and GPS devices will usually create a new segment every time it lost GPS satellite visibility (for example if you entered in a tunnel or  a deep canyon) or you temporarily switched it off.

But we have another problem here. When we reached the peak the batteries of our GPS discharged and that's why we don't have the return of our trip recorded. BTW, the return was the same path.

This is a typical scenario where you would use the new "Track segment editor" on Trackprofiler. When you click on the "segment editor" tab you will see the map with numbered segments and on the right the list with all possible operations on single segments:

By selecting the drop-down with the number you can change the segment order in your GPS file. Use  to reverse the order of points inside that segment. With  you can duplicate the segment or delete it with .

Use  to join two segments.

Now back to our problematic track. We need to:

• Join all our segments using ,
• copy the resulting segment with ,
• reverse the points on the second segment (the returning path) with ,
• join them together with .

...and the result will be the complete path without "holes":

The Track segment editor is online now. 

Feedback is welcome!

Geoelevations for Ruby

WGS84 Undulations map of the Earth

Just released another open source project related to Trackprofiler. Geoelevations.rb is a SRTM and EGM2008 undulations parser library for Ruby:

Geoelevations.rb is a SRTM and EGM2008 undulations parser library for ruby. Geoelevation.rb allows you to retrieve the elevation for any point in Earth (if present in the SRTM dataset) and the Geoid undulation (the difference between the WGS84 ellipsoid and the actual Earth size). 

SRTM: "The Shuttle Radar Topography Mission (SRTM) obtained elevation data on a near-global scale to generate the most complete high-resolution digital topographic database of Earth." 

EGM2008: "The official Earth Gravitational Model EGM2008". Part of this dataset is a "geoid undulation values with respect to WGS 84".

Basically it is simillar to the existing Python SRTM library (used in Trackprofiler to add elevations to GPS tracks) with two differences:

• It is written in Ruby,
• It contains also a EGM2008 undulations data (remember the difference between smartphone elevation and the actual elevation? That's the reason).

Geoelevations.rb is hosted here: github.com/tkrajina/geoelevations

GPS, Navigation and our Brains

This sounds true to me:

How GPS affects our natural ability to navigate is a question that has, in recent years, begun to attract the attention of researchers around the world. What they are finding suggests that (...) I was one of millions of people for whom technology is disrupting something the human brain is supposed to do well. When we use GPS, the research indicates, we remember less about the places we go, and put less work into generating our own internal picture of the world.

Often referred to as mental maps, these schematics tell us where things are in relation to each other and allow us to navigate among them. They are as powerful as they are mysterious, even to specialists who have devoted their careers to studying how they work. “They are very individual,” said Julia Frankenstein, a researcher at the Center for Cognitive Science at the University of Freiburg in Germany. “The things which matter to you might be completely different to those that matter to your wife or your children.”

With the option to use GPS to do our wayfinding for us, it might seem like we don’t have much need for mental maps anymore. But according to Veronique Bohbot, a neuroscientist affiliated with McGill University and the Douglas Institute who studies spatial memory and navigation, the process of generating mental maps also plays a role in activities that have nothing to do with getting to work. Becoming overly reliant on GPS and letting that skill atrophy, she and others suggest, might actually be bad for us.

Later in the article it says that things maybe aren't that bad. But anyway, nobody is sure how bad things are.

Let's suppose things are bad. My experience tells me that is true not only for GPS navigation but even when people go hiking but somebody else is reading the map and telling them where to turn. I think that the quoted article is true. But if true, what to do? If every one of us becomes overly reliant to GPS navigation? What then?

I think I have a solution. Use the GPS to record the path you took but not to navigate. When outside do experiment, try alternative paths, study paper maps, ... Only when everything else fails -- look at the GPS map. I may be a little extreme but I still have my 8-years-old Garmin GPS60 without maps.

Because the quoted research says nothing about GPS. It says that the navigation without thinking about the surroundings is something that might atrophy our skills. But we can use GPS outside and still think about our surroundings.

PS. I think that Trackprofiler users are typically the kind of people (mountainbikers, hikers, paragliders, ...) who do think about surroundings and don't rely too much on car navigation.

Better GPS track elevation data

Few weeks ago we made a nice little hiking trip near Pazin. I forgot my Garmin and recorded the trip with my Android smartphone. The resulting elevation graph was pretty strange. Look at the black line:

The blue line is the elevation computed from SRTM (Shuttle Radar Topography Mission) data.

Is it obvious why the Android elevations were strange? The first thing is that the graph was cca 40 meters above the actual elevation lines. But this isn't something that bothers me too much. Because if it were only this -- the actual elevation gain should be the same (but the maximum/minimum elevations would be wrong).

But the second thing is what was really strange. Android cooked up a hill in the first 700 meters of our trip. A big hill, almost 420m high. But in reality there was no hill there.

This is why I added an option "Add GPS track elevation" on Trackprofiler. It is online almost two months now. But the actual data used is raw-SRTM data. And that's why the resulting graph was sometimes not a smooth line but a succession of horizontal lines (Look at the blue line in the graph).

I've been experimenting with few different algorithms to make a better elevation graphs from NASA's elevation data. And I think I finally found a good approximation. The result is the red line:

This new (and better) algorithm for elevation detection will be deployed is on Trackprofiler soon now.

PS. I'm not sure but the fact that my Android elevations are cca 40 meters higher may be because of the deviation between the ideal geoid and the actual Earth surface. 

Add missing time on GPS tracks

It is easy to add new points on an existing track. Just click on the track and move the points marked with (or move the if you need to move existing points):

Those new points don't have the same metadata as the old ones. For example, new points don't have elevations. That's why the elevation graph looks like this:

There is a "empty hole" on the part of the track where the new points are. This is easy to solve: Just go to "Add elevation" and click on "Add missing elevations" and the result is:

By the way, you could use the "Add elevations" option instead of "Add missing elevations" if you want to override even the existing elevation data. 

But there still is one thing missing. Look at the analytics page, the speed graph:

Here, too, there is the same "empty hole", and that's because the new points have no time(stamps) saved

For situations like this I'm finishing/testing a new feature on Trackprofiler. When choosing it, the application will try to interpolate times for all points where the time data is missing. The interpolation is based on distances between points.

The "Add missing time" feature is almost ready and will be online soon is deployed now.  (BTW. Watch us on Trackprofiler on Twitter, Facebook or Google+ for notifications about the new versions deployed.)

Print only selected track elements

From now on, you can decide which elements to include when printing the track:

Feedback is welcome.

New layout for GPS track analytics page

There was a few deploys with loads of new features on trackprofiler in the last few months. This is the first of a series of blog posts covering new features introduced lately.

If you remember, the original "Track analytics" page was just a series of graphs. Yesterday a new version was rolled out with the new version:

For every graph you can see:

• Elevation graph
• Speed graph
• Steepness graph

For every graph by moving over it -- you can see exactly on the map where that point was recorder. 

See an example here.

GPS elevation graphs

This blog post is a short history on the evolution of GPS elevation graphs on trackprofiler.

When starting the service I created a simple javascript based graph library and used it for GPS tracks elevation graphs:

...but over time, some problems arose. First of all it was slow and rendered in the browser. That means that for slower computers the graphs slowed the rendering of the entire page by more than a second.

And, there was another thing I wanted to show on the graph -- the speed graph. So, a month ago a new version of TrackProfiler was deployed. The graph is now a PNG image created on the server -- no client's CPU is used. The graph, then, looked like:

The green graph is speed, the gray one is elevation.

But, there still was one thing in the evolution of the graph I wanted to make. For example on the GPS track shown -- it started on elevation 197m and never went below. So, it don't make sense to show the vertical  part of the graph from zero. It only makes the uphill and downhill slopes flatter.

After another iteration (online for the last 3 days) -- the vertical part of the graph now starts from the minimum elevation found on the graph. And the uphill/downhill slopes are better visible:

See an example here, or try it yourself on TrackProfiler.

Gpxpy version 0.8.0

Gpxpy is an open source python library for parsing and manipulating GPX files. It is one of the core parts of the TrackProfiler web site, and one of the open sourced parts of it. In the last few months a lot of work is done. Two new important features:

• gpxpy can now use lxml (if available) to parse GPX files (contribution from Isaev Igor)
• Python 3 support (contribution from Robert Smallshire)

...and many more small bug fixes and unit tests to make it faster and more stable.

It is licensed under Apache 2.0 and there should be no problems using it even with your own closed-source commercial apps.

PS. The original trackprofiler utility app and the library used to show analytics graphs (cartesius) are also open source.

Even more GPS editing tools

Even more GPS editing tools are available in the newest version of TrackProfiler. Those are:

• Remove time metadata from the track,
• Remove elevation metadata from the track,
• Elevation graphs are not shown if the track has no elevation data,
• Move graphs,
• Add/remove to elevation.

Plus, few bug fixes:

• Problem with embeding tracks whose subject contain non-ASCII characters,
• Default background for embeded tracks,
• Better handling for inbound emails for email track uploading.

Enjoy!