Boomerang Test Template: Watch Customers Use the Competition
Someone has already built a working prototype of your idea: your competitor. Watch customers use it, and see what you have to beat.
Share
A boomerang test watches customers use a competitor's product for a real task, so you can see where it falls short and what your offer has to beat. You build nothing: the competitor has already built a working prototype of your idea. In five one-hour sessions you learn what "good enough" looks like today, and which gap people actually care about.
Key takeaways
Recruit five people from your target segment: two who use the competitor today and three who do not.
Strategyzer's experiment library for Testing Business Ideas rates the boomerang 2 out of 5 for cost, setup and run time, with evidence strength from 1 to 3 out of 5 (Strategyzer poster).
Name the gap before the first session. Pass signal: at least 3 of 5 hit the same gap, and it's one you plan to close.
Note what the competitor does well. Nielsen Norman Group's advice on competitive testing: beat the competition, don't copy it (NN/g, 2024).
A gap shows a problem exists. It does not show people would switch.
What is a boomerang test?
A boomerang test is a moderated session in which a customer does a real job with an existing product, usually a competitor's, while you watch and they think aloud. You note where they stall, the workarounds they reach for, and the parts that work well.
The name comes from David J. Bland and Alexander Osterwalder's Testing Business Ideas, a library of 43 experiment types sorted by cost, time and strength of evidence (Strategyzer). The boomerang sits in the discovery group of "discussion prototypes", next to the paper prototype, the storyboard and the pretend-to-own test. Strategyzer's poster for the library scores it like this (Strategyzer poster):
The low end of the evidence range is what people say about the product. The high end is what you watch them do with it.
UX researchers run a close cousin called competitive usability testing, where people complete the same tasks on two or more designs (NN/g, 2024). The boomerang borrows that method but asks a product question: is there a gap worth building for?
Which risk does it test, and how strong is the evidence?
This tests desirability: where do existing options leave a gap people care about? The evidence is behaviour, because you watch real attempts at a real job. It tells you that a gap exists, and not that people would switch to fill it. See our guide to choosing a product experiment for how to take it further.
When should you run a boomerang test, and when not?
Use it when alternatives already exist, and you want to know what you would actually have to do better. You get a working product to test without building anything, and you learn what "good enough" looks like.
NN/g lists the same benefits for competitive testing: competitors have often already run their own rounds of testing, so you can learn from their strengths and weaknesses without repeating the work, and you can compare approaches before doing much design (NN/g, 2024).
Do not use it if there is nothing comparable, or if using the rival's product would break its terms or require someone else's login. Do not go in looking for flaws so you can feel good. Do not use it as evidence that people want your version, since everybody finds some fault in everything. To check that, follow up with a fake door.
Boomerang test template (copy and use)
Open the image at full size, or copy the text version below.
Who to recruit and how many
Five customers from your target segment. A mix is best: two who use the rival today and three who do not. Each participant should use their own free or paid account, or a public trial.
Step-by-step plan
Pick one real task that matters.
Choose the alternative your customers would most likely use.
Set your pass signal, including the gap you plan to close.
Ask each person to do the task while you watch.
Stay quiet, and take notes on workarounds.
Ask what they would do next, and whether they would keep using it.
Compare gaps across people.
Script
"Please do this the way you normally would, using this product: [task]. I didn't build it, so you can't offend me. Please think out loud."
Afterwards:
"What was the most annoying part?"
"What did you do to get around it?"
"If this disappeared tomorrow, what would you do?"
"What did it do better than you expected?"
What to measure
Whether they finished. Where they got stuck. Workarounds. How many people hit the same gap. What the product did well.
Pass signal to set before you start
Specky's starting point: at least 3 of 5 hit the same gap, and it's one you plan to close. This is a starting point, not a benchmark. Name the gap before you start.
Results log
Person
Current user?
Task finished?
Gap hit
Workaround
What it did well
Would they keep it?
How do you run a boomerang session step by step?
Book 60 minutes per person and keep the structure the same each time:
Minutes 0-5: set up. Confirm consent to record, and ask them to use test data rather than real client records.
Minutes 5-10: context. Ask how they did this job last time, and with what. Past behaviour beats predictions.
Minutes 10-35: the task. Read it aloud, then stay quiet. Jakob Nielsen's recipe for thinking aloud is representative users, representative tasks, and then let them talk; he warns that interruptions can easily change behaviour (NN/g, 2012).
Minutes 35-50: debrief. Ask the four script questions. Note the strengths as carefully as the gaps.
Minutes 50-60: write up. Log the gap, workaround and strength while it is fresh.
Should you test more than one competitor?
You can. NN/g suggests focusing on 2 to 4 competitors, having each participant use 2 or 3 products, and no more than 3, since more can overwhelm people. Randomise which product each person uses first, so learning from the first does not flatter the second. End by asking participants to compare what they used (NN/g, 2024).
How many participants do you need for a boomerang test?
Five is a practical first round. Jakob Nielsen's model estimates that five users find about 85% of usability problems in one design, and recommends three rounds of five over one round of 15 (NN/g, 2000). With two distinct groups, such as current users and non-users, he suggests three to four people per group.
Using the same formula, here is the chance that a gap shows up at least once in five sessions, depending on how common it really is (our calculation):
Share of customers who hit the gap
Chance it appears in 5 sessions
50%
97%
30%
83%
10%
41%
Five sessions will almost always surface a gap that half your customers hit. They will often miss a gap that only 1 in 10 hits, which is one more reason not to build for a gap that appears once.
Boomerang test vs usability test: which should you use?
Both watch people do a task in silence. The difference is whose product they use, and what you are deciding.
Run a boomerang first to choose what to build. Run a usability test once you have built it. Some teams then repeat the boomerang task on their own prototype, with the same five-person structure, as a head-to-head.
What does a boomerang look like in practice?
This example is made up to show the method. A team wants to build a simple accounting tool for freelancers, believing current tools are too complicated. They watch five freelancers use a popular option to send an invoice and record an expense.
Everyone finishes. The invoice is simple and all five praise it. But four of five get stuck categorising an expense, and two give up and write "other." One says she exports it to a spreadsheet.
Four people hit the same gap, and it was the one the team named, so the signal passes. They also learn something they had not planned to: invoicing is already good, and does not need rebuilding.
Extending the example: five one-hour sessions cost the team about one working day plus write-up. Their next step is a fake door offering "expenses sorted automatically" to freelancers, to see whether the 4-of-5 gap turns into clicks from people who would switch.
What are the most common boomerang mistakes?
Looking for faults to feel better. Note what is good, too. That is what you have to match.
Testing a task nobody does. Use a job people really care about.
Copying the rival's feature list. You are looking for gaps, not a checklist.
Assuming a gap means switching. People put up with a lot.
Breaking the rules. Use public trials and people's own accounts.
Explaining the product. If you help, you hide the gap. NN/g adds a related warning: observe what people do, not only what they say.
When it doesn't work: for brand-new categories with no real alternative, for products people use rarely, and when the true competitor is a spreadsheet or a habit. In that case, watch the spreadsheet: it is the alternative you have to beat.
Is it legal and fair to test a competitor's product?
Generally, watching someone use a product they have legitimate access to is ordinary research, but terms differ. This is not legal advice. Read the product's terms first. HubSpot's, for example, give each user their own login and prohibit reverse engineering the service (HubSpot terms). So use each participant's own account or a public trial, never a shared login, and do not try to take the product apart.
Treat participants fairly too. Ask before recording, use test data so their clients' details never appear on screen, and do not post clips of a competitor's product failing.
How to run it in Specky
On the Assumptions page, click the note and choose Pick a test, then Boomerang (competitor testing). Specky fills in the steps, the pass signal, and a one-week timeline.
You run this one. Specky keeps the plan and the result. Watch the sessions yourself.
Click Start running. Paste in the gaps, workarounds, and strengths, and record the result.
A pass shows on the board as Supported, because watching real attempts is behavior. What it supports is that the gap exists. It does not show that people would switch.
What should you do with each outcome?
Supported: You know what to beat. Offer exactly that with a fake door, and test whether people want it enough to act.
Unclear: The gaps were different for each person. You may be mixing different kinds of customers. Narrow the group, or change the task.
Contradicted: The existing option is good enough. Do not compete there. Look for something the product does badly elsewhere, or a group it does not serve. Any bet resting on this note now shows as Contradicted.
Frequently asked questions
What is a boomerang test?
A boomerang test has customers use a competitor's product for a real task while you watch and they think aloud. It shows where the existing option falls short, and what it does well, without you having to build anything. The method comes from the experiment library in Bland and Osterwalder's Testing Business Ideas.
How many people do you need for a boomerang test?
Five is a practical first round: two who use the competitor today and three who do not. Nielsen's model estimates five users find about 85% of usability problems. Five sessions will almost always surface a gap that half your customers hit, but can miss rarer ones, so test again before building for a gap seen once.
Is it legal to test a competitor's product with customers?
Generally you can observe people using a product they have access to, but check the product's terms and avoid sharing accounts. Use public trials and each person's own login, and do not reverse engineer the product. Terms and laws vary, so read the agreement and take legal advice if in doubt.
What should I do if the competitor's product works well?
That is a useful result. It means you should not try to compete on that task. Look for a different job, or a customer group the product serves poorly. Note exactly what it does well, because any product you build will be compared with it.
What is the difference between a boomerang test and a usability test?
A boomerang test watches people use a competitor's product to find a gap worth building for. A usability test watches people use your own product to check they can finish a task. Run the boomerang before you design, and the usability test once you have something to put in front of people.