By: Aaron Ng
Downtown LA. My July 4th weekend FourSquare checkins visualized in AR. (ARKit + Unity + Mapbox + Swarm) https://t.co/1Gddt3i6kP
Last week, we saw this tweet from Aaron Ng showing how he used the Mapbox Unity SDK, Apple’s ARKit, and Swarm to visualize his check-ins on Foursquare over the Fourth of July weekend. We had to know more, so we’re excited to share Aaron’s behind the scenes look at how he created it.
1| Can you explain what we’re looking at?
This is a map of Downtown LA with my July 4th weekend checkins visualized using a combination of ARKit, Unity, and Mapbox.
2| Where did you get the idea for this prototype?
I’m really interested in logging stuff and zooming out to see what patterns emerge– whether it’s my blood glucose levels, my heart rate, how I spend my time, or where I’ve visited in a city. Taking a zoomed out look at these patterns can be interesting, revealing, bring new insight, or just be cool.
So, I thought visualizing check-ins on an AR map would be interesting: maybe the closest thing to a holographic table we can emulate today. Seeing my check-ins clustered around this miniaturized Downtown LA and being able to walk and duck around it brought a new dimension and perspective to something that we’re otherwise relatively used to.
3| Could you explain how you put this together? Anything that would be helpful for people to start tinkering?
I took my check-ins over July 4th weekend and added them to a Mapbox Dataset and used that to create a custom Tileset. From there it was as simple as spinning up an ARKit scene in Unity, loading in a map with the Mapbox Unity SDK, tweaking the layers to show what I want, and styling it a bit. Then I built it in the Xcode 9 beta and ran it on an iPad Pro running the iOS11 beta.
The example Scenes in the Mapbox Unity SDK are really helpful for getting a grip on the SDK. Pick the examples apart– they’re organized well and after a bit of tinkering, you’re pretty much ready to get started. There are layers included for buildings, roads, points of interest, topology– it’s great. Combine this with the samples and tutorials for the Unity SDK and ARKit, and you’re good to go.
4| How did you hear about the Mapbox Unity SDK?
I’ve used Mapbox for embedding custom maps onto websites and iOS apps before– I actually didn’t know that Mapbox had a Unity SDK, but I was looking on Google for Unity mapping plugins and saw the Mapbox plugin. Had a great time using your tools in the past, so I was pleasantly surprised.
5| What makes you most excited about the Mapbox Unity SDK?
It makes it surprisingly easy to get started with mapping and visualization in ARKit.
6| With your prototype, you’re basically turning your camera into the interface to interact with a visualization. Where do you see exciting opportunities for these types of interactions?
I think most of the interesting opportunities for interaction are actually going to be in the nebulous zone between phone interactions and real-world interactions. Screen interactions will still exist, but imagine moving through a virtual haunted house environment rooted in your actual home, pointing your camera at the Great Pyramid of Giza with the original construction overlaid, or gesturing at the Golden Gate Bridge and annotating the original site where the suspension cables were spun on site — that’s where I see the exciting opportunities.
7| Where do you expect to see the first break-through AR use cases in the next year?
I think the use cases around entertainment, visualization, and sharing are going to be huge. AR is going to be the glue between the digital world and ours, and that’ll add an entire new dimension to how we interact with our phones. Want your own show? Point the camera at yourself and wield superpowers for an audience. Play games that interact with your living room through your camera. Visualize your favorite sci-fi worlds in your back yard. It’s going to change everything.
I think something particularly exciting to me is the combination of computer-vision and AR to query and tag the world around you: Like that chair? Point your camera at it, find out everything about it, and stick a virtual model of it in your home. Don’t know what species a flower is? Point your camera at it, see all the parts tagged in AR, and get a closer look at a 3d model on your phone.
8| What will be your next demo? ;-)
You’re gonna have to follow me @aaronykng on twitter to find out! :)
How I built it: Visualizing Foursquare check-ins with Mapbox + ARKit was originally published in Points of interest on Medium, where people are continuing the conversation by highlighting and responding to this story.