Latest ODK Collect release adds new features and maps
By: Marena Brinkhurst
Open Data Kit Collect is one of the most widely used mobile-based data collection tools for nonprofits, governments, and researchers working on humanitarian issues at the biggest scales. Users include the World Food Program, the Jane Goodall Institute, World Health Organization and Centers for Disease Control.
This month, ODK released a new version that expands what you can do with location data within the app. The update adds Mapbox basemaps with the Maps SDK for Android, making the maps faster and easier to use especially in offline and limited connectivity environments. I caught up with leading contributors Yaw Anokwa and Hélène Martin to learn more about the app’s new geospatial features.
What is ODK and why has it become so widely used?
Open Data Kit is a free and open-source set of mobile data collection tools used for social good efforts ranging from polio elimination to election monitoring to rainforest conservation. ODK is widely used because it helps social good projects, regardless of size, collect critical data in the most challenging environments — even in space believe it or not!
ODK Collect is the most popular component in the ODK suite. The Android app has millions of users, is running on 12,000 different devices, and has been translated into 50 languages. Collect is an offline-friendly replacement for the paper forms that are typically used for field data collection. Unlike paper, Collect allows for a wide range of question types including points, lines, and polygons and support flexible skip logic, robust data validation, and end-to-end encryption.
What’s an example of how ODK Collect is being used?
In January of last year, the Somalia Ministry of Health, WHO, and UNICEF launched a campaign to vaccinate more than 726,000 children from polio. The campaign was supported by some 3,500 personnel, and staff used Android devices running Collect to provide quick and accurate reports on vaccine coverage.
And while global health is a big part of our user base, ODK is unique in that it’s used across a wide variety of sectors:
- In Western Australia, the Department of Biodiversity, Conservation & Attractions uses ODK to record 30,000 turtle tracks and nests.
- In Albania, the Coalition of Domestic Observers deployed some 2,000 observers with ODK to monitor Assembly Elections.
- In Yemen, the World Food Program used ODK to monitor the distribution of food to 26,000 families.
- In Tanzania, the Jane Goodall Institute uses ODK to manage land use and forest reserves.
- In Italy, volunteers use ODK to fight illegal fishing controlled by organized crime.
- In Portland, Trimet uses ODK for on-board rider surveys to produce data in support of transit planning.
How does location factor into ODK Collect?
One of the most important things Collect does is enable easy collection of location data. The primary use cases for collecting location data are ground truthing, improving data quality, and wayfinding. For example, mapping clinics to plan an outbreak response, or ensuring an election monitor was actually on site, or locating a rural household that was surveyed last year.
Recently, we’ve been putting a lot of effort into improving the geospatial functionality and map experience in the app. In our latest release, we’ve added the Mapbox SDK which means we can now provide much faster, more responsive maps — and increased usability like smooth rotation and zooming that previously were not possible. This means that data collectors can more easily, quickly and accurately collect point, line, and polygon data.
What else are you excited about in the new release?
Collect has supported offline raster tiles for some time, but adding the Mapbox SDK means we can now support offline vector MBTiles. Vector tiles are much smaller and faster to load. This is important because our users work in areas with little or expensive cell coverage and so every megabyte counts.
It was also very exciting to see how our lead developer, Ka-Ping Yee, and a volunteer mobile engineer from Mapbox, Langston Smith, collaborated on these new features. Thanks to the hours they invested into integrating the Mapbox SDK and vector tiles, a whole new roadmap of geospatial feature development has opened up.
What are some technical hurdles the team had to work through?
One challenge was how exactly to load offline layers from the file system on the device. The Mapbox SDK can only load tiles from a URL, so we built a way to run a small background web-server inside Collect to serve the tiles from the filesystem. It’s been working great so far. The Mapbox Community team also has support available for organizations that need special permissions to create and side-load offline map tiles between devices, which is very helpful for users in that situation.
We also had to build some workarounds to keep the app size small. The Mapbox Maps SDK for Android has native binaries for four architectures and including the SDK as-is would have increased our APK to 22 MB. The recommended workarounds like ABI splits or App Bundles required too much maintenance for our small team. Instead, we use zero-byte stubs for the architectures we wanted to exclude. This approach enables us to pass the Play Store requirements for native code, continue shipping non-Mapbox related features to all architectures with a single APK, and keep Collect APK to 10 MB.
We wanted to preserve the option for users to choose between different basemap providers, but this means that the app has various interfaces so that any mapping library code can hook into the core code. For example, an interface makes sure that pressing the device location button will work the same no matter the basemap. Building the Mapbox SDK into these location-related interfaces was also an interesting and challenging process, but Ping and Langston did a great job navigating that.
What else are you working on?
We don’t yet support styles for vector MBTiles, so for now colors are selected by Collect to differentiate lines from different layers. We are exploring options to work with the broader geo community to define a common standard to package styles with tiles.
In addition to styling, we will soon enable form designers to bundle MBTiles and GeoJSON as part of their Collect form and specify which questions show which layers. Together, these features will make it easier for projects to deploy very tailored experiences for large-scale data collection campaigns.
Longterm, we hope to add more “map-centric” data collection workflows to Collect. For example, we’d like to make it possible for users to select a geo feature from an offline layer and collect data about that feature by launching a form specific to that type of feature. You’ll also be able to view previously-collected data about those features to provide even more context.
What motivates you to continue contributing to an open-source project like ODK?
It’s all about helping the helpers! We had the opportunity to travel to Somalia last year to help train vaccinators on how to better use ODK in their work. These are people working in conflict zones and who put their lives on the line to ensure children are vaccinated against polio. They were amazed when we told them that ODK is a community-owned open-source project and some of them have started contributing translations to the project. The potential for everyone to be involved and continuously improve the tools is incredibly powerful.
What sets ODK apart from other data collection tools is that it’s a community-owned public good. Nowhere is this community more visible than on our forum where some 10,000 users, implementers, and developers from around the globe help each other collect data. We strongly believe that ODK only gets better if everyone participates.
To anyone who loves geo: We need your skills and your passion to help design, build, and test more functionality in Collect. The skills and ideas from the wider geo community are invaluable to the development of this tool, no matter your background, talent, or skill, you can help make our community and tools better.
You can help build ODK Collect! Join the forum at http://forum.opendatakit.org and orient yourself to the code with the readme.md at https://github.com/opendatakit/collect. And learn more about working with the Mapbox SDKs for mobile. If you need support with offline map tiles for mobile-based data collection, contact the Mapbox Community team.
Yaw Anokwa co-founded the ODK project. He’s currently CEO at Nafundi, a technology consulting company that leads the software development on ODK. Nafundi designs, builds, deploys and supports data collection systems that work well in challenging environments.
Hélène Martin is CTO at Nafundi and a lead maintainer on the ODK project. In her free time, she sews and builds Cora, an iOS app that helps sewists keep track of their fabric stash.
Marena Brinkhurst - Community Team Program Manager - Mapbox | LinkedIn
Taking data collection offline with Open Data Kit and Mapbox was originally published in Points of interest on Medium, where people are continuing the conversation by highlighting and responding to this story.