Feature Factory: 10 Warning Signs Your Team Ships Without Evidence
A feature factory ships a lot and learns little. Ten warning signs, a scorecard you can run in five minutes, and the changes that move a team from output to outcomes.
Share
A feature factory is a product team that measures success by how much it ships rather than by whether the work changed anything for customers or the business. John Cutler named the pattern in 2016. The ten warning signs below cover culture, process and follow-through. If four or more describe your team, you are probably shipping without evidence.
Key takeaways
John Cutler's 2016 post listed 12 signs of a feature factory, starting with "no measurement" and "celebrating launches" (Cutler, 2016).
Marty Cagan draws the same line between feature teams, handed a roadmap and judged on output, and empowered product teams, handed problems and judged on outcomes (SVPG, 2019).
Pendo's 2019 analysis found 80% of features in the average software product are rarely or never used, representing up to $29.5 billion of development spend at public cloud companies (Pendo, 2019).
Opinion-led prioritisation misses value both ways: one Bing idea sat in the backlog for over six months, then raised revenue 12% when finally tested (HBR, 2017).
The fastest way out is small: define the metric before building, and review it after launch.
What is a feature factory?
The term comes from John Cutler, whose November 2016 post "12 Signs You're Working in a Feature Factory" described teams that ship constantly, measure little and rarely ask whether anything worked (Cutler, 2016). The phrase stuck because it named something many teams felt but couldn't describe.
Two other frameworks describe the same failure from different angles:
Feature teams vs product teams. Cagan's 2019 essay contrasts feature teams, which are given a prioritised roadmap of features by stakeholders and judged on what they ship, with empowered product teams, which are given problems and business outcomes and work out the solution through discovery (SVPG, 2019).
The build trap. Melissa Perri describes organisations "cranking out features to meet their schedule rather than the customer's needs", where output becomes the goal instead of the value it should create ().
All three describe one root cause: the team's success metric is output, so evidence becomes optional.
What does building without evidence cost?
The most cited number comes from Pendo's 2019 Feature Adoption Report: across 615 Pendo subscriptions, 80% of features in the average product were rarely or never used, and Pendo estimated that public cloud software companies had invested up to $29.5 billion building them (Pendo, 2019). It is one vendor's customer data, not a universal law, but the direction matches what most PMs see in their own analytics. We dig into that figure in why 80% of features are never used.
Unused features aren't free after launch either. Each one adds interface surface, support questions, test cases and code to maintain.
The opposite cost is just as real: good ideas that never get built because prioritisation runs on opinion. Ron Kohavi and Stefan Thomke describe a Bing employee's idea to change how ad headlines were displayed. It needed only a few days of engineering, but program managers ranked it low among hundreds of proposals, and it sat in the backlog for more than six months. When an engineer finally ran a simple A/B test, it raised revenue 12% without hurting user-experience metrics, which the authors estimate at over $100 million a year in the US alone (HBR, 2017). A feature factory builds the wrong things and skips the right ones.
What are the 10 warning signs of a feature factory?
Culture signs
1. The roadmap is set by the highest-paid opinion. Priorities change after a leadership meeting, not after new customer evidence. Quick test: for your top 3 roadmap items, can anyone name the customer evidence behind each one?
2. Velocity is the headline metric. Story points, tickets closed and release counts are reported every sprint; adoption, retention and conversion are reported rarely or never. Cutler lists judging PMs by output and velocity as a sign in its own right (Cutler, 2016).
3. Launches are celebrated; results are not discussed. The all-hands slide says "shipped". Nobody follows up 30 days later with "and here's what changed".
Process signs
4. Work goes straight from idea to ticket. No problem statement, no customer conversations, no prototype. Discovery is something "we'll do when we have time". Our product discovery process guide shows what the missing step looks like.
5. One loud customer can redirect the quarter. A single high-value client's email triggers a major build, with no check on whether other customers share the problem. Cutler's list includes building features to close deals with weak financial justification.
6. The roadmap lists features, not outcomes. Every row is a deliverable with a date. None says what customer behaviour or business number it is meant to change. See outcome-based roadmaps for the alternative.
7. Builders can't explain why their work matters. Ask an engineer or designer how their current task affects churn, activation or revenue. If the answer is "the PM asked for it", the strategy isn't reaching the team.
Follow-through signs
8. No one checks results after launch. The team moves to the next ticket the moment code reaches production. Shipping is treated as the end of the process rather than the start of learning.
9. Nothing is ever removed. Features are added but almost never killed or reworked based on data. Cutler calls this out directly: teams rarely scrap work, and often lack the safety to admit a bet failed.
10. No iteration after version one. Once a feature ships, it is "done", even if early usage shows people struggling with it. Feedback never makes it back into a second pass.
How do you score your team? A five-minute feature factory scorecard
Score each sign 0 (not true), 1 (sometimes true) or 2 (usually true). Be honest; the score is for you.
#
Warning sign
Score (0-2)
1
Roadmap set by the highest-paid opinion
2
Velocity is the headline metric
3
Launches celebrated, results not discussed
4
Ideas go straight to tickets
5
One loud customer redirects the quarter
6
Roadmap lists features, not outcomes
7
Builders can't explain why their work matters
8
No post-launch check
9
Nothing is ever removed
10
No iteration after version one
Total (max 20)
How to read it:
0-5: You are mostly outcome-driven. Keep the habits that got you here.
6-11: Mixed. Pick the two highest-scoring signs and fix those first.
12-20: Feature factory. Start with sign 8: a post-launch check is the cheapest change and makes every other problem visible.
These thresholds are a rule of thumb for starting a conversation, not a benchmark.
Is shipping fast a bad thing?
No. Speed is an advantage when you are moving in the right direction. When the roadmap is a wishlist of features rather than a list of outcomes, faster delivery just gets you to the wrong place sooner. That gap has grown as AI coding tools cut build time; we look at it in the product velocity illusion.
What protects speed is a steady flow of evidence. Teresa Torres defines continuous discovery as weekly touchpoints with customers by the team building the product (Product Talk). Teams that talk to customers every week can move fast because they find out early when they are wrong. Without that habit, every experiment becomes a must-succeed project, and teams that can't fail stop experimenting.
How do you get out of a feature factory?
Define the success metric before writing code. For every item above a few days of work, write down the number that should move and by how much. If you can't name one, you're not ready to build it.
Prototype early. Test the riskiest assumption with the cheapest artefact: a sketch, a clickable prototype or a fake door. If a feature isn't worth prototyping, it's rarely worth building. Our breakdown of what counts as evidence for each product risk helps you pick the right test.
Redefine "done". Change it from "code in production" to "code in production and the metric reviewed".
Release to a slice first. Use feature flags to ship to a small share of users, compare against the rest, then widen. You gather evidence before the whole user base pays for a mistake.
Rewrite the roadmap as outcomes. Each row becomes a problem and a target metric; features become candidate solutions underneath.
Hold a quarterly kill review. List features with low usage and decide whether to improve, keep or remove each one.
Worked example: one quarter, measured
This example is illustrative. A 12-engineer SaaS team ships 30 features in a quarter and reports "30 shipped" to the board. The PM runs the scorecard and gets 13.
The team goes back through the 30 launches:
8 had a success metric written down before build.
Of those 8, 3 moved their metric by the target amount.
The other 22 had no metric, so nobody can say whether they worked.
Next quarter the team commits to 15 bets instead of 30, each with a metric and a 30-day review date. At the end of the quarter, 6 of the 15 hit their target, 5 are inconclusive and get a second iteration, and 4 failed and are rolled back. The board slide now says "6 outcomes moved, 4 ideas retired" instead of "30 shipped". Fewer launches, twice as many proven wins, and a clear list of what to stop maintaining.
When is "feature factory" the wrong diagnosis?
Very early startups. Before product-market fit, shipping many small things quickly can be the research method, as long as each release is treated as a test and someone looks at the result.
Contractual or regulatory work. Compliance features and committed customer deliverables don't need a growth metric. Label them as commitments so they don't distort the rest of the roadmap.
Platform and debt work. Refactoring rarely has a customer metric. Cutler's own list warns against ignoring it in favour of shiny features.
Sales-led features with real economics. Building for a deal can be the right call when the revenue justifies the long-term complexity. The sign is doing it by default, without that maths.
Where Specky fits
Specky helps teams leave the factory by keeping evidence attached to the work. It reads calls, feedback, Slack, Jira and analytics into a Product Graph, drafts specs and tickets with customer evidence cited inline, and checks after launch whether the bet worked, so sign 8 stops being the default.
Frequently asked questions
What is a feature factory in product management?
A feature factory is a team or company that treats shipping features as the goal. Success is measured in output, such as features released or story points closed, rather than in outcomes like adoption, retention or revenue. John Cutler popularised the term in a 2016 post listing 12 signs, including no measurement, celebrating launches and rarely killing work.
Who coined the term feature factory?
John Cutler, a product coach who later worked at Amplitude, published "12 Signs You're Working in a Feature Factory" on his blog in November 2016. He wrote a follow-up in late 2019. The idea overlaps with Marty Cagan's distinction between feature teams and product teams and with Melissa Perri's build trap.
What is the difference between a feature team and a product team?
In Marty Cagan's framing, a feature team receives a prioritised roadmap of features from stakeholders and is judged on delivering them. An empowered product team receives a problem and a business outcome, works out the solution through discovery, and is judged on whether the outcome moved. The people can be identical; the difference is what they're accountable for.
How do you measure outcomes instead of output?
Before building, write down the customer behaviour or business number the work should change, the target and the review date. After launch, check it and record the result next to the original decision. Track the share of shipped work that had a metric and the share that hit it. Those two numbers say more than velocity.
Can a fast-moving startup avoid becoming a feature factory?
Yes, if speed is used for learning. Ship small, define what each release should prove, and look at the result before deciding the next step. Weekly customer contact, as in Teresa Torres's continuous discovery, keeps the evidence flowing. The risk grows as teams scale and start reporting output upwards because it's easier to count.
GDPR-aligned. EU workspace data is stored in Frankfurt, Germany. GDPR and EU data Built in Vienna, Austria. A product of influence solutions fm GmbH (FN 675374y). Imprint 14-day trial. Card required, and nothing is charged until day 14. 30-day refund after subscribing. See pricing