The Failed Test That Changes Everything
Most growth teams celebrate wins and bury failures. Phil Carter does the opposite. When he interviews growth PMs, he asks them to walk through an A/B test that completely bombed—but led to their biggest breakthrough.
Oftentimes the biggest wins come right on the heels of a failed A/B test. That's not what I was expecting to see, but based on what I am seeing now, I've learned something new about the customer. So now let's go run a second A/B test on the basis of that learning, and then suddenly you sort of unlock a new opportunity.
— Phil Carter
The pattern repeats across Quizlet, Faire, and the consumer subscription companies Carter advises: the hypothesis was wrong, the data revealed something unexpected about user psychology, and the follow-up test unlocked step-function growth. This isn't about pivoting. It's about building on negative signal.
Your North Star Is Lying to You
ARR looks great. Subscriber count is climbing. The board is happy. And your growth team is optimizing the wrong thing.
Carter draws a hard line between outputs (ARR, MRR, subscribers) and inputs (signup rate, trial start rate, cost per install). VCs care about outputs. Growth teams die when they optimize for them. The real work happens upstream—connecting individual initiatives to input metrics like trial conversion rate or install-to-subscriber conversion—and then pressure-testing which inputs have the most leverage.
At Quizlet, Carter's team didn't obsess over total subscribers. They mapped the conversion funnel, identified bottlenecks (great activation, terrible trial starts), and went five whys deep on the constraint. The art isn't picking a vanity metric. It's knowing which input metric, when moved, cascades into the output the CEO actually needs.
The $60 ARPU Problem Nobody Talks About
Consumer subscription apps are easy to launch. No sales team. No two-sided marketplace complexity. High gross margins. Global distribution through the App Stores. Carter has watched dozens sprint to $1-10M ARR, then flatline.
The average consumer subscription app is losing more than 50% of its annual subscribers in the first year and more than 50% of its monthly subscribers in the first 3 months.
— Phil Carter
The trap is structural. ARPU averages $10/month or $60-80/year—a fraction of B2B contract sizes. Churn rates are brutal. And unlike SaaS, there's no land-and-expand motion. Most apps have one tier. You can't offset churn by upselling retained subscribers into enterprise plans.
Layer in App Store fees (15-30%), subscription fatigue, and the fact that paid channels like Facebook are saturated with competing apps, and the unit economics break. LTV shrinks. CAC climbs. The growth engine stalls out right when it should be compounding.
CACs Go Up, Not Down—Even With Brand
Brand recognition should drive CACs down, right? More word-of-mouth, more organic installs, lower blended acquisition costs. Carter says that's true for exactly three types of companies: Duolingo, Tinder, Strava. Maybe ChatGPT. For everyone else, CACs climb over time.
CACs almost by definition will go up over time. As you expand beyond your core and as you tap out your highest-intent early adopters, you are just going to have a harder and harder time converting eyeballs into subscribers.
— Phil Carter
The math is unforgiving. Early adopters have the highest intent. They convert easily. As you scale, you're reaching lower-intent users. Cost per install rises. Signup, activation, and trial conversion rates all degrade. On paid channels, throwing more budget at Facebook or Google pushes you into lower-quality ad inventory. The system breaks down.
Carter's advice for early-stage teams spending $5,000-10,000/month (not enough to train the algorithms): grow organically as long as possible. Test paid channels on the margin, but force yourself to find product-market fit strong enough to sustain word-of-mouth or SEO. It's like training at altitude—you get stronger before you need performance-enhancing spend.
One Channel, Then Watch the Inputs
Peter Thiel's power law of distribution applies to growth channels. For most fast-growing companies, 70% or more of early acquisition comes through a single channel. Carter has seen too many teams spread thin across Facebook, TikTok, Google, and SEO simultaneously. They optimize nothing.
The playbook: test early to find your dominant channel. Go all-in on efficiency. Then obsessively monitor input metrics through the funnel—not output metrics like revenue. When you spot bottlenecks (trial starts plateauing, install-to-subscriber conversion compressing), you have two options: fix the product or ad creative to unlock the next growth leg, or diversify to a second channel before saturation hits.
Timing matters. Don't wait until the 11th hour. If you can connect the dots to saturation 6-12 months out, start testing your next channel now. The companies that die are the ones that ride one channel to exhaustion, then scramble to find a replacement with no runway left.
Source Episode
7 Core Levers for Consumer Subscription
20Growth (20VC) · 79 min
Related Insights
Elena Verna on Why $100M ARR Doesn't Mean You Have Product-Market Fit
Elena Verna
Lucas Vargas on Building Nomad: Why a VIP Lounge Beats a Business Model
Lucas Vargas
Kate Syuma on Why Product Quality Kills More PLG than Bad Tactics
Kate Syuma
Casey Winters on Why Marketplace Founders Play the Wrong Game Early On
Casey Winters