Learning Velocity: Why Speed of Learning Beats Speed of Shipping
AI made building faster. It did not make knowing what to build faster. Learning velocity, the speed from question to evidence-backed decision, is the advantage that compounds.
Share
Learning velocity is how fast a product team turns an open question into a decision backed by evidence: from "we think customers want X" to "we tested it, and here is what we will do". It matters more than shipping speed because most product ideas fail, so the team that discards bad ideas and confirms good ones fastest compounds its advantage.
Key takeaways
Most ideas don't work. At Microsoft, only about one-third of ideas tested in controlled experiments improved the metric they were designed to improve, and about a third made it worse (Kohavi et al., 2009).
Building got faster. In GitHub's 2022 controlled study of 95 developers, those using Copilot finished a task 55% faster (GitHub, 2022). Learning did not speed up by itself.
Delivery metrics stop at deployment. DORA's change lead time runs from commit to production (DORA); nothing in it measures whether the change worked.
High-learning companies test at volume: Booking.com runs some 25,000 tests a year (HBR, 2020).
You can measure it with four numbers: learning cycle time, decisions per month, share of shipped bets reviewed, and weekly customer touchpoints.
What is learning velocity?
It is the rate at which a team produces validated decisions: choices to build, change, or stop something, based on evidence gathered for that purpose. It is the product version of Eric Ries's idea that "the unit of progress for Lean Startups is validated learning" (The Lean Startup). Ries argues every startup process should speed up the Build-Measure-Learn loop.
Two things make it different from shipping speed:
It counts decisions, not releases. A feature shipped and never measured adds zero learning. A two-day test that kills a bad idea adds a lot.
It includes the slow part after launch. The loop is not closed until someone checks whether the bet worked and changes the plan.
How fast questions become evidence-backed decisions
Typical metric
DORA change lead time, deployment frequency
Learning cycle time, validated decisions per month
Clock starts
Code committed
Assumption written down
Clock stops
Change deployed
Decision made on the result
Failure mode
Slow, risky releases
Fast releases of things nobody needed
Both matter. But a team can be elite on delivery and still learn slowly, which our post on the product velocity illusion explores in more depth.
Why is speed of learning a competitive advantage in 2026?
Three facts combine.
1. Most ideas fail, so speed of discarding them matters. Kohavi's team at Microsoft found that only about a third of ideas improved their target metric, a third were flat and about a third hurt it (Kohavi et al., 2009). They note Amazon's success rate was also below 50%. If two out of three bets don't pay off, the team that finds out in a week beats the team that finds out in a quarter.
2. The winners are large and hard to predict. One Bing experiment, a change to how ad headlines were displayed, raised revenue by 12%, worth more than $100 million a year in the US alone (HBR, 2017). It needed only a few days of engineering, yet it sat unbuilt for more than six months because it was marked low priority. You find winners like that by testing more ideas, not by guessing better.
3. Building is no longer the bottleneck. Coding assistants cut task time sharply in controlled studies; GitHub's showed 1 hour 11 minutes with Copilot against 2 hours 41 minutes without. When building gets cheaper, shipping unvalidated features also gets cheaper, and Pendo's finding that 80% of features in the average product are rarely or never used (Pendo, 2019) becomes easier to repeat. What stays scarce is knowing which features deserve to exist.
Competitors can copy a feature in weeks. They cannot copy the accumulated record of what you tested, what your customers did, and why you decided what you decided.
How do you measure learning velocity?
No single industry-standard formula exists. These four metrics are what we recommend; together they resist gaming.
Metric
Definition
How to collect
Healthy direction
Learning cycle time
Median days from an assumption being written down to a decision on it
Assumption log or decision log with two dates
Down
Validated decisions per month
Decisions to build, change or stop that cite evidence gathered for that question
Count decision log entries with evidence attached
Up
Shipped bets reviewed
Share of launches with a recorded result against their target metric within 6 weeks
Compare launch list to post-launch reviews
Up, aim for 80%+
Customer touchpoints per week
Conversations, tests or observations involving the team that builds the product
Calendar or research log
At least 1, per Torres
The last one comes from Teresa Torres, who defines continuous discovery as weekly touchpoints with customers by the team building the product, in pursuit of a desired outcome (Product Talk).
Worked example: what learning three times faster is worth (illustrative)
Say a 5-person product team makes 3 validated decisions a month, with a median learning cycle time of 41 days. Use Microsoft's base rate: roughly 1 in 3 tested ideas improves its target metric.
Before: 3 decisions × 1/3 success = about 1 winning change per month, or 12 a year.
After cutting cycle time to 12 days with smaller tests and a weekly review: 9 decisions a month × 1/3 = about 3 winning changes per month, or 36 a year.
The team also stops roughly 6 losing ideas a month earlier, before they consume engineering weeks.
The ratio is the point: the team is no better at guessing than before. It is three times faster at finding out.
How do you help a product team learn faster? A 7-step method
Write the question before the solution. For each bet, record the assumption, the decision it will inform, and the metric that would settle it. An untested assumption with no decision attached is just an opinion.
Pick the cheapest test that could change your mind. Interviews, fake doors, concierge tests and prototypes often answer in days what a full build answers in months. Our guide to choosing a product experiment matches 55 methods to the question you need answered.
Set the decision rule in advance. "If fewer than 15% of trial users click the import button in 2 weeks, we drop it." Pre-committing stops results being reinterpreted to fit the plan.
Talk to customers every week. Torres's weekly cadence keeps a steady stream of evidence, so tests start from real problems. Our post on continuous discovery covers how to schedule it without it taking over the roadmap.
Move fast on reversible decisions. Jeff Bezos argues most decisions should be made with about 70% of the information you wish you had, and calls reversible ones two-way doors that deserve a light process (Amazon, 2017). His goal: "high-quality, high-velocity decisions". Save slow, heavy review for decisions you cannot undo.
Close the loop after launch. Book the review when you ship: check the target metric at 2 and 6 weeks, record the result, and decide to keep, iterate or remove.
Make learning shared. Write each decision, its evidence and its result into a product decision log. Learning in one person's head does not compound when they leave or forget.
What slows a product team's learning down?
Most teams are slow for structural reasons, not effort:
Big batches. A quarter-long project returns one answer per quarter. Splitting it into 3-4 testable slices returns 3-4 answers.
Roadmaps that promise features by date. A dated feature becomes a commitment, so evidence against it is ignored. An outcome-based roadmap keeps the outcome fixed and lets the bets change.
No agreed success metric. If nobody agreed what "worked" means, every result is debatable. A clear north star with input metrics settles it upfront.
Evidence scattered across tools. Interviews in one place, tickets in another and analytics in a third means each decision starts with an archaeology project.
No strategy to test against. Without a stated bet, tests are random. A one-page strategy canvas names the riskiest assumptions worth testing first.
Common mistakes and limits
Counting experiments instead of decisions. A team running 40 tests a month that changes nothing has activity, not learning. Track decisions made on results.
Speed without rigour. Kohavi's paper warns against sequential, uncontrolled experiments, such as comparing this week with last week instead of running a control group at the same time, because they very often mislead. Fast and wrong is worse than slow and right.
Assuming you need A/B tests. Controlled experiments need traffic. A product with 200 weekly users cannot detect small effects. Below that scale, fast learning comes from interviews, usability tests, fake doors and concierge offers, not statistics.
Learning theatre. Research that never changes a decision is theatre. If the last 10 studies changed nothing on the roadmap, ask whether the roadmap was already fixed.
Ignoring irreversible decisions. Pricing changes, data migrations and public commitments are one-way doors. They deserve the slow path, even in a fast-learning team.
Where Specky fits
The slowest parts of the loop are usually gathering evidence and closing it after launch, and that is where Specky helps. It reads customer calls, feedback, Slack, Jira and analytics into a Product Graph, so a new question starts from what you already know. Alex, its AI research agent, runs customer interviews from a shareable link, so research does not wait on calendars. After a launch, it checks whether the bet moved its target, and the Decision Log keeps the record. For larger organisations, the same loop across many squads is part of the product operating model for scaling teams.
Frequently asked questions
What is learning velocity in product management?
Learning velocity is the speed at which a product team turns assumptions into decisions backed by evidence. It is measured from the moment a question is written down to the moment a build, change or stop decision is made on the result. It differs from delivery velocity, which measures how fast code reaches production, regardless of whether it works.
How do you measure learning velocity?
Track four numbers: median learning cycle time (days from assumption to decision), validated decisions per month, the share of shipped bets reviewed against their target metric within about 6 weeks, and customer touchpoints per week. Use them together, because any one alone can be gamed, for example by running many trivial experiments.
Is learning speed the same as experiment velocity?
No. Experiment velocity counts tests run; learning speed counts decisions changed or confirmed by evidence. A team can run many experiments and learn little if the tests are trivial or the results never affect the roadmap. Experiment velocity is a useful input, but the output that matters is better decisions made sooner.
Can a small startup learn faster without A/B testing?
Yes. Most early-stage learning comes from qualitative and low-traffic methods: customer interviews, usability tests, fake door tests, concierge offers and pricing conversations. These can answer a question in 2-10 days. A/B tests only become useful once you have enough traffic to detect the effect sizes you care about, which most products under a few thousand weekly users do not.
Why does speed of learning matter more now that AI writes code?
Because AI lowers the cost of building, but not the cost of building the wrong thing. GitHub's 2022 study found developers finished a task 55% faster with Copilot. When features are cheap to build, the scarce skill becomes knowing which ones to build, and the team that validates ideas fastest wastes the least of its new capacity.
GDPR-aligned. EU workspace data is stored in Frankfurt, Germany. GDPR and EU data Built in Vienna, Austria. A product of influence solutions fm GmbH (FN 675374y). Imprint 14-day trial. Card required, and nothing is charged until day 14. 30-day refund after subscribing. See pricing