The product team launched studies to understand the motivations of new signups and to identify high intent users. Aligning the acquisition and product onboarding flows for the high intent user cohort improved D1 retention from 20% to 35%.
Key learnings
- Optimizing acquisition campaigns on high intent users than only on CAC and D1 retention.
- Better roadmap prioritization using feedback of high intent users.
Context
A Series A edtech company building a suite of mobile applications to provide high-quality content access to the Tier II to Tier IV towns in India launched a new exam preparation mobile app in Feb 2022. This app provided modules for preparing for the highly coveted UPSC civil services exam.
The app had 20% D1 retention. A far cry from Lenny Rachitsky’s retention benchmarks of Good: 40% Great: 60% for month 6 retention for a consumer transaction app. The product teams being aware of this had a list of ideas to fix D1 retention including optimizations and features. Together accounting for over 3 months of engineering effort.
“Our original prioritization hinged on copying features from competitors’ apps and on the collective intuition of the team. In the hindsight, we would have been way off course if we had built according to the original plan” - a product manager on the team
By conducting a series of Blitzllama micro surveys, the team discovered quick wins by first identifying the high intent user cohort and then deep-diving into the cohort’s experience.
The exploration
The first Blitzllama study was to explore the motivations of the D0 users: why did they download and what were their expectations. This micro survey was launched by clicking the app exit button for the first 500 users in a day.
Question1
Question2
Question3
In the study over 1200 users responded to the second question. Exam preparation - the core use case of the app was the intent of only 42% of D0 users. The other users, the majority, were looking for something that the app was not offering.
Amongst users looking for exam prep support, only 63% met their expectations on D0 of using the app. A cue to iterate on the onboarding experience.
Themes related to exam preparation only totaled 42%. Source: Blitzllama dashboard.
Themes related to exam preparation only totaled 42%. Source: Blitzllama dashboard
The team further conducted follow-up surveys to further build the personas of the high intent users and to deeper understand their expectations and experience.
Impact of the studies
The biggest takeaway from studying the new users was identifying the high intent users. Each team incorporated the high intent users into their workflows:
- The product team doubled down on their feedback to redesign the app home screen and onboarding flows.
- The growth team used this cohort’s users as input to optimize paid acquisition campaigns and create targeted campaign messaging.
- A small set of D1 users are sampled daily to gauge intent. This is used for the new app health KPIs of % high intent users and D7/D15 retention of high intent users.
