01 Oct 2019
Josh
Planning for Exist in 2020
We've come a long way since we started building Exist back in 2013. The landscape has changed a lot. When Exist first launched, Fitbit and Jawbone UP were our most popular integrations, and there was no iPhone app, no Android app, and no way to enter manual data. Now Jawbone UP is long dead, Apple Health is our most popular source of data, and the vast majority of our users track mood and custom tags manually.
A lot of the assumptions and compromises we made at the outset no longer make sense. We thought people wanted to sync their data automatically, hands-off, so we prioritised syncing data from our integrations. Because we had only one developer, and my background was web development, we launched with just a web app. To best understand and give insights into the types of data users were tracking, we maintained a whitelist of allowed attributes.
But today all of these things have changed. We took on the extra work of building mobile apps, and Belle even learned to program from scratch in order to build our iOS app, which started off as a mood rating client but has grown into a fully-fledged Apple Health integration. Our web app, though still more feature-rich, languishes as a client in favour of the mobile apps, reflecting a broader shift in consumption. And we've relaxed that whitelist of attributes to allow user-defined custom tags. But those tags have only ever been a stop-gap solution, a way of carving out a little patch that users can manage themselves, and they're an uneasy fit with how Exist was initially designed. The further we go down the path of manually tracking data, the less our initial assumptions make sense and the more constrained we become by the setup we have now.
A different approach
So in 2020, we're going to break with convention by making some sweeping changes to how Exist works, in order to provide some long asked-for additions — things we and users have both wanted but haven't been feasible.
This year we'll still send out our annual user survey, but we won't be using survey answers to prioritise what to work on for the next year as we normally do. Instead, we'll be building a new version of Exist, with one big focus — making manual tracking possible.
What we'll be working on
- We want to make it possible to track anything you like manually, whether it's a simple yes/no, a quantity, a duration, a time of day, or a scale like mood.
- We'll be expanding the mood rating scale to 1–9 from its current scale of 1–5, and making that scale available to other attributes, so users can track things like energy and stress in the same way.
- We'll add emoji support to mood notes and increase the supported note length.
- All of this will be added to our API along the way, so developers have the same opportunities to track manual data in Exist programmatically.
Manual tracking opens up the power of Exist's correlations and insights to anything users want to track, however they want to track it. Users can add individual medication amounts for each day, for example, and see how these correlate with symptoms they rate from 1–9. Developers can automate sending "+1 coffee" each time they enter a café, or get their home's thermostat to send the temperature to Exist to see how it affects their sleep.
Importantly, we'll still have a whitelist of attributes we understand, and we'll still support automatic syncing of data from our integrations. But the way we'll handle attributes will change dramatically.
Currently, we allow user-defined attributes as long as they're a special type of attribute, a custom tag. This divides attributes into those where we understand what you're tracking, like steps and productive time, and things we don't understand, like tags for "gary", "washing", and "pain", which could be used to mean whatever users like. Because of this split, users can currently only add a tag if its name doesn't clash with any of our whitelisted, understood attributes.
As part of the planned changes, we're going to remove this split, allowing users to track anything they like, calling it whatever they want, without worrying if it clashes with anything. This means you could manually track something called "steps" that's actually the amount of time you spent following the steps for assembling IKEA furniture, if you thought that was a good idea.
But our understanding of certain attributes isn't going away, just becoming opt-in, so you can still choose to manually track "steps" as a regular count of steps you took during a day, and we could still provide insights about how far you walked. And over time we'll be able to expand that whitelist based on what's popular, so we can show more specific insights and trends based on our understanding of those attributes.
While we're poking around in the guts of Exist we'll look into adding user-defined subgroups, so users can group tags and manual attributes however they like, and related enhancements like custom tag colours. But these will be lower priority and may not happen.
All of these changes to how we store and understand attributes will require big changes to how we show and let users enter that data too, so we'll need to spend just as much time making new manual tracking interfaces across all of the Exist apps. We'll focus on redesigning the mobile apps ahead of the web app, though, making sure we can create an intuitive and low-effort flow for manually adding things.
Side note: losing integrations
Unfortunately we'll be losing our Gmail integration on 1 January 2020, due to our inability to pay "up to [US] $75,000" for a Google-mandated security audit. Because Google doesn't make email statistics available directly, our current integration needs access to your inbox, even though we don't request any email content. While it is important to make sure services don't expose your private email data, we can't justify the cost of the Google approval process just so we can provide email counts in Exist, so we'll instead be losing access and turning off this integration.
Instagram plans to turn off their programmatic access to user data sometime in early 2020, restricting their integrations to services for businesses and brand accounts, so we'll be denied access there too. But we're hoping these new features we'll be adding will more than make up for what will be taken away.
The approach
After we've sent out the survey and finished this year's priorities, we'll get started. We'll first update the back-end to support that new data, and then redesign the mobile and web apps to match, after which they can go through beta testing and then out to everyone else.
It means we're going to be pretty quiet for a lot of the year, and you won't see much external progress — but we hope you agree the end result will be worth it!