Original article
Full case study
A 10% drop in DAU. What would you investigate first?
I recently went through a product analysis case involving a food-delivery app where DAU dropped by 10% over a weekend.My first thought wasn't to rush to find a fix.It was to look at the metric closely. DAU is a number so just seeing the number doesn't show where the issue really is.I started by looking at geography.If the drop only happened in some cities there could be reasons. Like weather, events, road problems or something happening in those areas.
From there I came up with ideas to check:→ Did more orders get cancelled?
→ Did it take longer to deliver?
→ Was there a problem with the app or payment?
→ Did fewer couriers show up?
→ Was there something happening outside that affected orders?
→ Did other companies deal with it better?One idea was weather.
But I didn’t want to stop at:“The weather caused the DAU drop.”If other companies had the weather the more important question is:“Did we handle the problem worse than they did?" That’s when comparing with competitors becomes helpful. By just looking at one competitor, I would compare several things. How many orders got cancelled how well orders were handled how reliable deliveries were, how many customers stayed and how people felt about the service, to see where we were different. This changed how I thought about the answer.At first, I wanted to focus on keeping customers and offer them discounts.
That was too quick.We should not decide on a fix before understanding the problem.If the real issue is delivery, a discount won’t help.If the issue is not talking to customers, an apology won’t fix that.If the issue is a mistake in how things work, the fix has to deal with that mistake.
So, my better way became:
DAU drop → Look at groups → Make guesses → Get proof → Find the main reason → Make a fix → See if it works.