Product Analytics: Find and Fix User Drop-Off

A team can celebrate rising sign-ups while the product itself is going quiet. People create accounts, visit once or twice, and never reach the task they came to do. A dashboard showing more visitors will not tell the team where progress stopped. A single retention percentage will not tell it whether the problem began during setup, appeared after the first successful use, or came from counting the wrong people.

product analytics: a large centered funnel holding a phone, tablet showing an abstract event chart, clock, target with dart, closed notebook, pen, potted plant

Product analytics examines what people do inside a digital product and how that behavior relates to outcomes such as continued use or payment. The useful question is narrower than “How is the product doing?” It is “Which people reached a meaningful result, which did not, and at what step did their paths diverge?” That is the practical use of behavioral analysis described in Amplitude’s guide to product analytics. My starting choice would be to define the meaningful result before adding another chart. Once the result is clear, the team can count the route to it and decide what to change.

Begin with a user outcome, not a dashboard metric

Consider an illustrative reporting app. A customer creates an account, connects a data source, builds a report, shares it with a colleague, and returns when the next reporting cycle begins. A visit to the pricing page may matter to marketing, but it does not show that the customer has obtained a report. A login shows access, but it does not show that the report was useful. For this product, publishing a report that can be opened again is a more defensible sign of initial value. If the product’s actual promise were simply to view existing reports, the meaningful action would be different. The product promise, not a standard dashboard template, determines the choice.

This is why product analytics should begin with a decision. If the team must improve onboarding, it needs to know how many new accounts get from sign-up to their first useful report, how long that takes, and where they stop. If the concern is renewal, it needs to know whether the same accounts continue using reports over the following cycles. If the concern is revenue, it needs to relate product use to paid accounts without treating usage as payment. Acquisition, early success, recurring activity, and money are connected, but each answers a different question.

The underlying records are often events: a person or system did something at a particular time, with details about the action and the actor. Amplitude’s explanation of event-based analytics distinguishes events, event properties, users, user properties, and sessions. In the reporting app, “Report Published” is an event; the report type is an event property; an account’s plan may be a user or account attribute; and a session groups a period of use. These are different units of information. A report with 10,000 “Report Opened” events has not necessarily been useful to 10,000 people, because one person can open the same report repeatedly.

The line between product and website analytics is less about a vendor name than about the decision the team needs to make. Google Analytics describes GA4 events as interactions or occurrences on a site or app and lets a team mark a business-important action as a key event. That can capture a product action. The question is whether the implementation can follow the right actor through the relevant steps and time period. A beautifully configured purchase event still leaves the onboarding question unanswered if the app never records what happened between sign-up and purchase.

Decide who counts before deciding what counts

A metric begins with an entity. “Active users” could mean individual visitors, signed-in people, paying accounts, or organizations with at least one active member. Each may be a useful number, but they cannot be substituted for one another. For the illustrative reporting app, I would use accounts to judge whether a customer organization reaches a useful report and people to judge which parts of the interface individuals use. If three employees from one company publish reports, the product has three active people and one active account. If the product charges by account, treating those three people as three customers would overstate the commercial result.

Identity also has to survive ordinary behavior. Someone may explore anonymously, sign up on a laptop, and return on a phone. Two people may use a shared device. Amplitude’s unique-user documentation explains how device IDs, user IDs, and its own IDs can be combined, and shows cases where anonymous use and multiple devices change the count until identities are connected. That documentation describes Amplitude’s handling, not a universal rule every tool follows. The broader lesson is to know what identifier is sent at each stage and what happens when a visitor signs in or out. Otherwise, a change in identity handling can look like a change in growth or retention.

A session is a different unit again. A person can have several sessions, and a session can contain several events. Session counts can answer whether use is frequent or brief, but they cannot stand in for unique people or completed tasks. The app might record many short sessions because its reports load quickly, or because people cannot finish setup; duration alone does not decide which interpretation is right. I would define “active” through a meaningful action, then use sessions to add context where the task genuinely depends on visits or time.

The time window belongs in the metric definition too. “Accounts that published a report within seven days of creation” is a different measure from “accounts that have ever published.” The first can reveal whether onboarding gets people to a result promptly; the second includes late successes and older accounts. Before comparing two periods, I would write down the entity, qualifying event, start point, time window, and eligibility rule. That short sentence prevents a great deal of debate over a percentage whose denominator nobody remembers.

Instrument the shortest useful route through the product

The first tracking plan should be small enough that someone can inspect it against real product behavior. Amplitude’s implementation guidance suggests starting with two important events and expanding after they are tracked successfully. In a reporting app, “Account Created” and “Report Published” could establish the first question: do new accounts reach an initial result? “Data Source Connected” would be the next addition if the team needs to locate the break between account creation and publication. Recording every click at the start is a poor trade if the team cannot trust the few events that define success.

Event names need operational definitions. Does “Report Published” fire when someone presses a button, when the server accepts the request, or when a usable report becomes available? If a failed request fires the event, the funnel will count a success that the customer never experienced. If autosave fires it every few seconds, event volume will grow without more customers succeeding. I would count the completed outcome once, keep separate records for attempts and failures when those are relevant, and use a stable account identifier on the events that answer account-level questions. These are design choices for the illustrative app, not a claim that one event taxonomy fits every product.

Properties should capture the context the team might need to compare later. For a published report, useful candidates might include the data-source type, whether the report was created from a template, and whether it was the account’s first publication. The plan should say which fields are required, which values are allowed, and what to do when a value is unknown. A property that alternates among “template,” “Template,” and “from_template” creates three groups where the product has only one. If a product release changes the meaning of an event, the team needs to mark that change before it compares old and new cohorts.

Some context must be attached when the action happens. Mixpanel’s user-profile documentation says its events can join to a user’s latest profile state, even for earlier events, while an event property preserves context on the event. Suppose an account published a report while on a free plan and upgraded later. A chart that joins old events to the current plan can make that earlier publication appear to belong to a paid account. If the team needs to ask what free accounts did before upgrading, it should record plan status at the time of the action or use a documented historical-property method. The distinction changes the story the chart tells.

Good instrumentation includes a simple reality check. Create an account, connect a source, publish a report, and confirm that the expected events arrive once, in the expected order, with the right account and event-time details. Repeat a failed publication and check that it does not count as success. Then try sign-out and sign-in on another device if the analysis depends on that transition. This is less glamorous than building a dashboard, but it is the shortest path to numbers the team can use.

Read the five metric families as different stages of progress

Amplitude groups product measures into acquisition, activation, engagement, retention, and monetization. The categories are useful because each asks what happened at a different point in the relationship. I would choose one or two measures per current decision, not fill a dashboard with every available member of every category.

Acquisition describes entry. The reporting app might count new accounts and the channels that brought them. That tells the team whether people are arriving and from where. It does not tell the team whether those accounts contain intended customers or whether anyone produced a report. If sign-ups climb while the number of accounts publishing stays flat, I would first separate the change in incoming audience from the change in the product experience. Improving the sign-up form could increase the top line without increasing value delivered.

Activation should describe a first meaningful outcome. In the illustrative app, “published a usable report within seven days of account creation” is a candidate. Suppose 1,000 newly created accounts have all had a full seven days to act and 410 publish. The activation rate under that definition is 410 divided by 1,000, or 41%. The count is illustrative; the denominator includes every eligible new account, not just those that connected a data source. If a product needs a colleague to use the report before the customer experiences value, publication alone is too early, and the team should move the activation event to the first shared or reopened report. The stricter definition will usually yield a different number, but it can better match the product promise.

Engagement asks whether and how people use the product after that first result. Daily, weekly, or monthly active users are common labels, yet “active” should be tied to the use that matters. For a reporting app with a weekly workflow, weekly accounts that open, refresh, or publish a report might be more informative than daily logins. Feature usage can reveal whether a new report template is being tried, but heavy use of one feature does not by itself mean the overall product is healthy. People may use a workaround repeatedly because another step is difficult. The count tells the team where to look.

Retention asks whether a starting group returns in a later interval. A reporting app may have strong activation and weak retention if customers can make a first report but cannot keep its data current. A team should not call every absence “churn” without considering the product’s natural rhythm. A tax tool and a daily workplace tool invite different return intervals. The honest choice is a return action and interval that correspond to the product’s expected use, stated beside the rate. Retention deserves its own cohort analysis because a rising total of active accounts can hide a stream of new accounts replacing old ones.

Monetization asks how product use becomes revenue. Paid accounts, conversion from a free plan, recurring revenue, or revenue retained across a customer group may be relevant, depending on the business model. A team should keep the unit consistent: an individual’s report activity is not account revenue, and a paid account is not necessarily an engaged account. If 90 of the illustrative 1,000 new accounts eventually pay, “9% paid” is only meaningful after specifying the observation period and whether every account had enough time to reach a payment decision. The timing of payment may lag useful product activity; I would compare both outcomes before declaring the onboarding change successful.

Use a funnel to locate the stalled step

A funnel is valuable when the task has an ordered route. Extend the illustrative cohort of 1,000 eligible new accounts: 620 connect a data source within seven days, and 410 publish a report within those seven days after connecting. The account-creation-to-connection rate is 62%. The connection-to-publication rate is about 66% of the 620 connected accounts. The end-to-end publication rate is 41% of the original 1,000. Saying “the funnel converts at 66%” without naming the step and denominator would mislead a reader who assumes the percentage begins at sign-up.

The next question is practical: what prevents the 210 connected accounts from publishing? The event sequence can identify that group; it cannot explain their intent. Some may find the report editor confusing. Others may connect a source merely to inspect the product, discover that the source lacks the data they need, or leave because their reporting cycle has not arrived. Before redesigning the editor, I would look at the failure events and the actual product flow for this group, then seek direct feedback from a few affected users. The number tells us where to spend attention, not which solution to ship.

A funnel also needs an explicit conversion window and ordering rule. An account might connect a source on day eight and publish on day nine; a seven-day activation chart excludes that success even though the customer eventually gets value. Another account might publish a template-based report without connecting an external source; a rigid connection-first funnel would omit a valid route. I would keep the narrow seven-day measure for onboarding speed but examine late and alternative routes before changing the product. When the product has several legitimate ways to succeed, the analysis should reflect those paths instead of forcing every account through one ideal sequence.

Use cohorts to learn whether value lasts

Retention is easiest to misread when the denominator changes quietly. Pendo’s retention documentation separates new-user from all-user retention, lets reports use visitors or accounts, and specifies activity, cohort size, and date range. Its report also excludes anonymous visitors. Those choices are part of the result. A figure labeled “monthly retention” cannot be compared fairly with another figure until the reader knows what started the cohort, what counts as return activity, and whether both groups had enough time to return.

For the reporting app, an illustrative definition could be: of accounts created during one complete week, what share publishes at least one report during days 30 through 59 after creation? If the cohort contains 1,000 accounts, all have reached day 60, and 270 qualify, the rate is 27%. The same accounts might have a much higher rate for “any login,” because logging in is easier than producing a report. Neither rate is intrinsically correct; the first measures recurring report production, while the second measures access. The choice should match the decision about whether the product delivers repeat value.

The complete-week and full-follow-up conditions matter. If half the cohort was created yesterday, those accounts cannot yet appear in a day-30 return period. Counting them in the denominator would depress retention for reasons unrelated to product quality. Different tools may also define “month” differently; Pendo describes its monthly retention increments as 30-day periods tied to each visitor or account. I would state whether my own report uses those rolling periods or calendar months. A chart should not require readers to guess what a month means.

Comparing cohorts over time can show whether newer accounts behave differently from older ones. It still requires care. A better-looking cohort may contain more established customers, a different acquisition channel, or a different share of accounts whose users consented to tracking. If the team changed its user IDs during the same period, the apparent retention shift could be partly a counting shift. I would first compare like groups under a stable event definition, then investigate any large difference. The cohort chart tells the team that behavior changed; the product history and customer context help explain why.

Compare segments without turning correlation into a story of cause

Suppose the illustrative app finds that accounts publishing a report in their first week are more likely to pay later than accounts that do not. That relationship is useful: early publication is a candidate signal of value. It does not show that forcing everyone through a publishing tutorial will create the same paid outcome. Customers with clearer needs may be more likely both to publish quickly and to pay. Bigger accounts may have more staff available to build reports. A marketing campaign may bring customers whose problems already fit the product. Product analytics can locate these differences, but the chart cannot remove them by itself.

I would use segments to sharpen the question, not to manufacture certainty. Compare accounts from the same sign-up period and acquisition route; separate plans or account sizes when those are relevant; and inspect whether a change persists. For a feature launched only to some users, distinguish who was eligible to see it from who chose to use it. Comparing adopters with nonadopters without that distinction makes adoption look like a treatment when it is partly a sign of motivation. If the decision is consequential and the product can assign a change fairly, a controlled test is stronger than a before-and-after chart. If it cannot, describe the result as an association and keep the alternative explanations visible.

Property timing can undermine even a careful segment. A report that groups last month’s behavior by current paid status answers “What did today’s paid accounts do last month?” It does not answer “What did accounts that were paid last month do?” Mixpanel’s description of latest-state profile joins is a concrete example of that distinction. For any comparison involving plan, region, account size, or eligibility that can change, I would ask whether the chart uses the value at event time or the value now. The answer may reverse the apparent result.

Use outside benchmarks only after matching definitions

A benchmark can help a team notice that a number deserves investigation, but it cannot supply a product’s target by itself. Pendo’s product benchmark page reports a “best-in-class” user-retention figure of 57% and says its retention calculation follows new visitors in monthly cohorts over the subsequent three months. The page describes its benchmark population as aggregated, anonymized data from more than 6,800 applications across 2,500 Pendo customers, with results that can be viewed by region, company size, and industry. Its 57% is a result under Pendo’s definitions and population, not a universal threshold for a reporting app.

Our illustrative 27% is account-level repeat publication during days 30–59. Pendo’s displayed figure concerns visitors returning over a different period. The two percentages should not be put side by side as if the lower one proves failure. If the team wants an external comparison, it should first match the entity, action, time period, and peer group as closely as possible. If that is not possible, I would use the benchmark as a prompt for a question, then judge improvement against the product’s own consistent cohorts and customer outcomes. This costs the comfort of a simple league table but gives the team a number it can actually act on.

The same principle applies when choosing software. GA4 can record events and mark key events; Amplitude documents event, property, user, and session analysis; Mixpanel documents the consequences of joining profiles to events; and Pendo exposes particular retention and benchmark definitions. A purchase decision should begin with the questions the team needs to answer, the identities it can lawfully and reliably use, and the reports it can maintain. A tool with more charts will not rescue a product that has no clear definition of success or events that fire at the wrong moment.

Keep the data trustworthy and proportionate

The strongest chart is only as good as the events allowed into it. A product release can rename an event, double-fire it, or stop sending an account ID. An app that filters out failed attempts may make a new error look like a decline in demand rather than a broken workflow. I would keep a small set of named event definitions with owners in the product team, check the main route after releases, and annotate metric breaks when tracking changes. A sudden rise in “new users” deserves an identity check before it becomes a growth claim; a sudden fall in publication deserves a check that the event still fires after success.

Privacy limits are part of measurement, not a footnote. Google Analytics’ guidance on personally identifiable information warns that URLs and page titles can carry recognizable details, that people may enter such details in forms or search boxes, and that custom dimensions or event data need care. In the reporting app, a report title might contain a customer’s name or private business information. I would avoid sending raw titles or entered text as analytics properties merely because they are convenient for debugging. A coarse report type can answer the adoption question with less unnecessary data.

Consent and collection choices also define the observed population. If some visitors are not represented, the dashboard describes the people and actions recorded under those settings. It should not quietly be described as a census of everyone who used the product. This is especially important when comparing acquisition sources, devices, or periods in which consent behavior or tracking settings differ. State what is included and keep definitions stable before interpreting a small movement as a product effect.

There is a cost to restraint. A lean event plan will leave some unexpected questions unanswered until new events are added. A narrow definition of activation may show a lower rate than a generous “logged in” measure. Avoiding raw user-entered text can make individual debugging less convenient. I would accept those costs in exchange for a few measures that correspond to real customer outcomes and can be explained without a glossary. Expand tracking when a specific decision needs it, and record the new event definition before comparing it with older data.

Turn a finding into a change worth testing

The reporting app’s 210 connected accounts that did not publish within seven days give the team a place to start. I would first confirm that the publication event represents a usable report and that these accounts had enough time to finish. Then I would inspect failed steps and speak with affected customers. If many attempts fail at data mapping, the next change may be better mapping defaults or clearer error messages. If accounts never open the editor after connecting, the next change may be a clearer route from connection to a first report. Those are hypotheses tied to different observed behaviors, and each calls for a different product change.

Before shipping, state what improvement would look like. For example, the team might seek a larger share of eligible new accounts publishing within seven days, without an increase in failed reports or a decline in later repeat publication. After release, compare cohorts under the same entity, event, and time-window definitions; where feasible, compare groups assigned to different experiences. A rise in button clicks alone would be weak evidence of success if usable reports and returns did not rise with it. The decision remains tied to the customer result, not to the easiest event to increase.

That is the working value of product analytics. It connects product actions to the people or accounts taking them, shows where a defined route loses momentum, and follows cohorts long enough to see whether value lasts. It does not reveal a customer’s motive or prove that one feature caused a later payment. When identities, events, and denominators are explicit, the team can see the next sensible change and recognize when the data still leaves a question for customers or a test to answer.

One person. A whole marketing team.

Invite only