Thursday, December 4, 2014

Mapping My Way

Due to the need for specialized medical treatment, my older son, Michael, and his wife have been living with my wife and I for the past several weeks. Michael happens to be an exceptional computer programmer and more generally, has knack for getting computers to do what he wants them to. He has recently become interested in a company called Mapbox, which provides a more sophisticated alternative to Google Maps for individuals and companies who want to develop custom mapping products. Both to show me the potential of this company as well as to become familiar with their latest programming interface, he undertook the task of taking all the accumulated GPS data from my Garmin 500 cycle computer and displaying it on a map. This is what he came up with:

The above graphic is dynamic; all the controls on this map work. Most importantly, you can zoom in and zoom out using the + and - buttons and can move around on the map with your mouse. Feel free to play around with it, at any time you can go back to the starting version of the picture by reloading this page in your browser. However, for purposes of this blog post, I have created static versions below so that what I am talking about corresponds to what you are looking at.

I have posted maps of my rides before by using the Garmin supplied software, "Garmin Training Center", to "view" a ride in Google Earth. Garmin Training Center has its own built-in map, but it is rather crude. Here is what the Tour de Pink looks like on the built-in Garmin map:

If you view that same ride in Google Earth, you have a lot more options in terms of what you see and what you don't see. I pick a set of options to look as much like what Michael did with Mapbox as possible, and that looks like the following:

So what can Mapbox provide that Google Earth cannot?
  1. Mapbox allows me to display ALL of my rides on one map. All three maps show the same Tour de Pink route with its characteristic arrowhead shape, but the Mapbox version also shows the overlap between that ride and my 2013 brevet, and where they all are relative to my regular training rides. This was our original motivation for undertaking this project.
  2. Mapbox provides a much greater range of options than Google Earth, both in terms of the map background and in terms of how the ride data is displayed on that background. Michael tried several backgrounds before selecting the one he used, a nighttime-looking satellite image. What he and I like about this background is that it is subtle enough to allow one to clearly see my rides while still allowing one to see the geographical context of where those rides took place. 
  3. Also possible because of the flexibility of Mapbox,  Michael was able to include an effect wherein as the same route is ridden again and again, the map displays the route with brighter and brighter lines.

This map is centered just west of the City of Houston, Texas, where I live. The "loop on a stick" image to the left of the picture represent my most recent 200K brevet (the "stick") and the Tour de Pink (the "loop".) They have similarly bright lines because each has been ridden twice, the brevet all in one day, as it is an out and back ride, and the Tour de Pink in two successive years (2013 and 2014.)  If you look closely, you will notice that the stretch where the Tour de Pink and brevet overlap is brighter than the rest of either of these rides. Much more obvious is how bright some of the rides to the right are. The two separated sets of rides represent Terry Hershey/George Bush parks in the middle and the White Oak and Braes Bayou trails to the right. If you zoom in on the active map at the top of the post, you can identify the Rice Track as an extremely bright oval. In addition to these very bright sections, there are many fainter sections representing rides done one or two times. If you zoom all the way in on the bright sections, you can see that they are made up of many close together lines representing individual rides along the same route.

To make the point about "all the rides on the same map" most dramatically, I zoomed out on the map until it included all of the United States:

I have been riding mostly in Texas, but when I would visit family in Washington DC or California, I would oftentimes be able to squeeze in a bike ride. I circled the rides in these three areas so you can see them more easily. If I then zoom in on the rides around my Dad's house in California, you can see my rides there:

Everything up until this point, Michael did without programming (except for some optional data cleanup.) You do not need to be a programmer to do any of this, any computer savvy person could create their own customized display and directions for doing so are given at the end of the post. But Michael wanted to do more. Rather than just layering all the rides together, he wanted to be able to look at single rides, subsets of rides, and what the pattern of my rides looked like over time. This required some programming. The result was the following three part graphic:

The top part of the graphic is a map with all my rides layered on it, similar to the first graphic at the beginning of this post. Just below that is another representation of my rides, as a bar chart with the horizontal (X) axis representing the date the ride was done and the vertical (Y) axis representing the length of that ride. These two parts are connected. If you hold your mouse over a ride on the map, all the days that route was ridden are highlighted in pink on the middle bar chart, and vice versa. At the bottom of the bar chart are two sliders. Only the rides ridden on the dates between the two sliders are shown on the map. This allows one to control which rides are shown.

The third and final part of the graphics are a series of controls on the bottom. By clicking on the words "Houston" or "West University" or "Antioch" (a.k.a. California) or "Washington DC" zooms in on that part of the map. (Because the graphic is limited to a small size in this blog post, it sometimes is necessary to zoom out with the "-" control on the map to see the rides.) Clicking on the words "Last 30 Days", "Last 90 Days", "2013", "2014", and "All Time" display the rides done during those times. (The last one, as you might guess, shows all the rides.) This has the same effect as the sliders below the bar graph. The last two controls, "Build the Map" and "Sliding 30 Day Window"are rather clever animations. The first adds rides to the map, one at a time, in the order they were ridden. The newest addition is highlighted in pink. The second shows all the rides ridden during a 30 day window, and the window slides forward in time. Honestly, playing with this graphic is the best way to understand it.

The above tiny window really does my son's program a disservice. If you have a large screen (and a fairly fast computer), you should really take a moment and play with it all on its own, in all its glory.

How to Put Garmin Data onto a Mapbox Map
- by Michael Steffen

This blog post from Mapbox explains the basics of importing GPX data into Mapbox Studio and styling it.

This project required a few additional steps:

  1. Garmin Training Center exports data as a TCX file, a format that Mapbox Studio doesn't support.  So I converted the Garmin TCX file into the more standard GPX format with the excellent free tool GPSBabel.
  2. I wrote a little python script to clean the GPX data by (i) eliminating user-uploaded routes that aren't rides actually ridden -- Garmin mixes these in with the rides data; and (ii) removing segments that were clearly where the GPS had been left on after a ride for the car ride home (identified, in this case, as segments at the end of a ride where the average speed was >30 mph). The script is available here.  It uses Tomo Krajina's very nice gpxpy library. If we had wanted a completely programming-free version, this step could have been skipped.
  3. I customized the Mapbox styles in Mapbox Studio, using the Mapbox satellite layer, the Mapbox streets layer, and the rides data. The styles I used are available here. The key to the rides style is that each ride is actually three lines -- a base line in semi-transparent blue, a "heat" overlay in a more transparent white, and then a wide, blurred "glow" overlay, in a yet more transparent white.  As the rides layer on top of each other, they build up the glow effect.

The inspiration for the final interactive graphic came from the "Urban Layers" website; I borrowed both concepts and some code. All the code used by the final graphic is available on Github.