Visualizing an entire city's buildings live with runtime styling

I’m using runtime styling to make a new mobile native version of Justin Palmer’s Portland buildings map, which shows the age of every building in the city going back to the 1840s. Justin’s work from a few years ago was cutting edge and fast – this was before we had vector tiles and as a result we needed to pre-render large visualizations.

Now that we have runtime styling, which lets us render and dynamically style thousands of features in realtime, I wanted to load the 650,000 buildings and their metadata into the map on the fly while allowing interaction with the map, letting you zoom in and tap each building to query the specifics.

The app centers on my home city, Portland, and shows every building cataloged by the city government. Each building has a data point for its construction date. I can show all of the buildings color-coded by age. Instantly we can see the oldest buildings (ranging to the 1840s) in blue and the newest in red, with a range of values and colors in between.

Some of the buildings don’t have data from the city. You can quickly highlight these buildings with a menu option.

You can enable a query mode to stop moving the map and instead run your finger over the buildings to see a popup of the actual build year. All of the data is there, in the map, already downloaded and being rendered on the fly. And runtime styling highlights the building under your finger (and adjacent buildings) for clarity.

I can explore the data using runtime styling’s filtering ability. Go into sibling mode to explore and see all buildings in the city built the same year, rapidly moving between years. Again, all of the data is already client-side; the style changes in code, on the fly, based on user interaction.

With all of this ability to change the map on the fly, as well as an enormous dataset, I thought it would be fun to create a timeline animation that shows Portland’s history in buildings. Again, runtime styling makes it easy to incrementally style the map in this manner for a dramatic visualization.

Preparing the data

All of the building data is uploaded to Mapbox and turned into vector tiles. The vector tiles are automatically optimized when the data is uploaded, so the map efficiently renders all of the source data (for example, the year built) that I want for the app. When working with large datasets such as this (nearly 650,000 polygons with metadata), reducing the data as much as possible makes your app fast and performant.

I obtained the data in Shapefile format from a local government GIS data portal, Portland Maps. Once unzipped, it’s roughly 1.5GB of data on disk! This clearly won’t do for mobile or dynamic styling use, but I can pare it down in a couple ways since I only need two components of the data—building outlines and the construction year.

Convert to GeoJSON

Ordinarily, you could drag and drop zipped Shapefiles into Mapbox Studio’s upload interface to have them converted automatically to vector tiles. But this dataset is too large for that, so I can use Tippecanoe (installed via Homebrew on my Mac with brew install tippecanoe), which allows both large dataset use as well as custom refinements to shave data size and complexity.

Tippecanoe takes large GeoJSON, though, so let’s first convert the Shapefile to that. The ogr2ogr tool from the GDAL project (installed with brew install gdal) is our friend here, and Ben Balter’s guide got me nearly all of the way there.

$ ogr2ogr -f GeoJSON -t_srs crs:84 buildings_raw.geojson Buildings.shp

That turned 1.7GB of unzipped Shapefile data into 915MB of GeoJSON—still the same, full GIS data, just in a verbose textual format and with no metadata stripped yet.

Massage field values into numbers

One downside to working with GeoJSON is JavaScript’s loose typing when it comes to field values. In this case, the data I care about, the building construction year, is formatted as a string (e.g. "YEAR_BUILT": "1948"). I can use a quick shell script to remove the quotes around the values to make these numeric so that later in the mobile code, I can do comparisons between numbers using operators like < and >.

$ cat buildings_raw.geojson | perl -pe 's/"YEAR_BUILT": "(\w+)",/"YEAR_BUILT": \1,/g'> buildings.geojson

Note that we’re working on building this ability right into Tippecanoe in order to avoid this step.

Convert to vector tiles

Next up, I use Tippecanoe to strip and simplify the GeoJSON into vector tiles specifically for this app. I can throw away dozens of metadata fields such as building height, roof type, and construction type.

The command I used to convert the Shapefile into an MBTiles full of vector tiles was the following:

$ tippecanoe -ac -an -y YEAR_BUILT -l buildings -A "© 2016 City of Portland, Oregon" -o buildings.mbtiles buildings.geojson

I’ll highlight three key options that I used in order to save space:

1. -y to remove all metadata except for the YEAR_BUILT field.

2. -ac to coalesce adjacent features with the same property (in this case, matching only YEAR_BUILT since that’s all we keep). Coalescing will have the effect of only allowing identical styling between these coalesced features, but since I just want styling based on building age, this doesn’t really matter.

3. -an to drop the physically smallest features in order to keep tile sizes within reasonable limits. For lower-zoom tiles which show many buildings at once, this won’t be noticeable visibly.

Together, these really increase the compression potential of the data since many unique properties of the features are dropped and the geometries are greatly simplified without resulting in any visual artifacts.

Once Tippecanoe works its magic, I have a 32MB MBTiles file of vector tiles. This is a huge reduction in data size! I still have all buildings visible at the required zoom levels, along with metadata for the year built for each of them. I can upload this to Mapbox Studio’s Tilesets, where in my case I get a Map ID of justin.0079auf9 (which you are welcome to use directly if you want a similar Portland buildings dataset).

Now I’m ready to combine a default Mapbox map with this custom data and start having fun with the mobile SDK.

Using runtime styling

There are three basic parts to enabling the runtime styling examples discussed above. Whether it’s coloring by age, highlighting queries, showing siblings, or playing the animation, the same basic concepts are involved.

Map style and customizing the default data

First, I’ll load up a default map style, Mapbox Dark, and turn off its default, OpenStreetMap-based buildings, since I want to style a custom dataset which has correlated metadata.

The basic steps for this are to set the overall map style (done in this app’s viewDidLoad()):

map.styleURL=MGLStyle.darkStyleURL(withVersion:9)

And to dynamically turn off the default buildings layer at runtime (encapsulated in hideDefaultBuildings()):

ifletdefaultBuildings=map.style.layer(withIdentifier:"building"){defaultBuildings.isVisible=false}

Note that as with all other runtime styling functionality, I branch from the mapView(_:didFinishLoading:) method to ensure that everything has been rendered at the start.

Adding data with sources

Next, I add sources to the map style as needed to introduce new data. The SDK supports vector sources like the one I created above and based on vector tiles, raster sources like satellite imagery or legacy map data, or even on the fly data in the form of shape sources.

To use our Tippecanoe-generated custom buildings source, I add it to the style with an identifier that I can use in styling later (encapsulated in addCustomBuildingsSource()):

letsourceURL=URL(string:"mapbox://justin.0079auf9")!letsource=MGLVectorSource(identifier:"buildings",configurationURL:sourceURL)map.style.addSource(source)

Manipulating style layers

Later on, you can make as many layers as needed that use this source data, including multiple layers that use it in different configurations, possibly only styling certain features.

For example, to add a layer that styles all buildings in the source built in 1945 as green, I can do the following:

ifletsource=map.style.source(withIdentifier:"buildings"){letlayer=MGLFillStyleLayer(identifier:"buildings-1945",source:source)layer.sourceLayerIdentifier="buildings"layer.predicate=NSPredicate(format:"YEAR_BUILT == 1945")layer.fillColor=MGLStyleValue(rawValue:.green)layer.fillOpacity=MGLStyleValue(rawValue:0.75)map.style.addLayer(layer)}

Note the use of sourceLayerIdentifier, specifying which among potentially several data layers in the source I want to actually render. This data source only has one layer, buildings (which I specified during the Tippecanoe step with the -l flag), but I still need to specify it in order to properly target the data.

Putting it all together

These three parts—altering the default map, adding data sources, and styling map layers—are the building blocks for everything accomplished in the app, in combination with user gestures; creative use of multiple layers; changing style attributes, filtering predicate, and layer visibility, and some basic app user interface for popups and other interactive elements.

Check out the app source code and have some fun! I’ve also made the source data Shapefile, ogr2ogr-generated GeoJSON, and final MBTiles available in case you want to experiment with the GDAL and Tippecanoe tools yourself to see how they work. Note that the runtime styling portions of this app require a beta version of our iOS SDK, but the same capabilities are present in Android SDK v4.2.

Special thanks to my colleague Eric Fischer for some great advice on the best use of Tippecanoe to really slim down the data for this app.

Ping me on Twitter @incanus77 if you have questions.