AI-Moderated Interviews: When They Beat Live Calls, and When Not
An AI interviewer can talk to customers without anyone booking a call, but not every question suits it. What the research says, where AI interviews fall short, how to handle consent, and how Alex runs them.
Share
AI-moderated interviews are customer or user interviews in which an AI interviewer asks the questions, follows up on the answers and summarises what it heard, usually by text chat through a link or an in-product widget. They beat live calls for short, focused questions asked of many people on their own schedule. They fall short in open-ended discovery, where a human moderator's judgment still matters.
Key takeaways
In a January 2026 Nielsen Norman Group study of two AI interviewers, only 3 of 10 experienced researchers found the conversation natural; NN/g recommends AI interviewers as a complement to human moderation, not a replacement (NN/g, 2026).
Adoption is still early: 11% of 521 UX practitioners surveyed in 2026 had used AI interviewers, and 61% had never used AI to interview users (NN/g, 2026).
Conversation beats forms for open questions: in a study of about 600 people, a chatbot drew more informative, relevant, specific and clear answers than a standard online survey (Xiao et al.).
AI interviews talk to real people; synthetic users are AI pretending to be people. Only the first is customer evidence.
In the EU, people must be told they are talking to an AI from 2 August 2026 (EU AI Act, Article 50), and recording needs consent you can prove.
What are AI-moderated interviews?
The AI replaces the human moderator, not the participant. You write an interview guide; the AI opens the conversation, asks one question at a time, follows up when an answer is thin ("What happened next?"), and closes with a transcript and a summary. Participants usually take part by chat, from a link sent by email or from a widget inside your product, whenever it suits them.
Three things get confused under the "AI research" label:
Nielsen Norman Group defines synthetic users as AI-generated profiles that try to mimic a user group, and found their answers overly optimistic and shallow next to real research (NN/g, 2024). An AI interviewer is a different thing: the machine asks, but a real person answers.
When do AI-moderated interviews beat live interviews?
When the question is clear, the answers are short and the bottleneck is calendars. NN/g's 2026 evaluation points to four good uses (NN/g, 2026):
Product feedback, especially after a launch or during a beta, as a richer alternative to a survey.
Teams with no researchers, as a step up from doing no user research at all.
Interviews in other languages, where nobody on the team speaks the participant's language. In NN/g's study, one participant switched to Spanish mid-interview and the tool carried on in Spanish.
Recruitment screening, to check eligibility before a live session.
Two more follow from how the format works:
In-the-moment questions. A widget can ask why someone is cancelling on the cancel page, or what they were trying to do during their first session, while the memory is fresh. A call booked a week later gets a reconstruction.
Consistency across many people. Every participant gets the same guide, which helps when you want to compare segments.
The research on conversational formats supports the second point. Xiao and colleagues ran a field study with about 600 participants: half took a standard online survey, half answered the same open-ended questions through an AI chatbot. Across more than 5,200 free-text responses, the chatbot drew significantly higher engagement and answers rated more informative, relevant, specific and clear (Xiao et al., ACM TOCHI).
When should you still run a live interview?
NN/g lists four situations where AI interviewers are the wrong choice: topics that need deep domain knowledge or jargon, exploring an unfamiliar problem space, studies where each participant's path will differ a lot, and sessions where people need to share a screen or demonstrate a workflow (NN/g, 2026).
Its study, with 10 research leaders from 8 countries, explains why:
Weak probing. The tools stuck to the script. They didn't chase unexpected answers, skip irrelevant questions or notice when a topic was exhausted.
No sense of time. Interviews ran anywhere from 13 to 56 minutes because neither tool could aim for a target length.
Thin rapport. Only 3 of 10 participants agreed the conversation felt natural, and 5 of 10 felt comfortable.
Sycophancy. Over-the-top praise felt fake to many participants.
Trust. Sessions that started abruptly, without a proper introduction, made some people hold back.
NN/g's verdict: AI interviewers are "not yet suitable for semistructured or in-depth interviews". So keep a human in the room for the first conversations in a new problem space, and use AI interviews to widen and repeat what you learn there. Our customer interview template covers the live version.
Question you need answered
Better format
Why did people cancel last month?
AI-moderated, via the cancellation email or page
What were new users trying to do in their first session?
AI-moderated, in-product
Is this problem real in a market we don't know yet?
Live interview
How does a finance team close the books today?
Live interview with screen share
Which of three onboarding messages is clearest?
Survey or usability test
Do people want this enough to pay?
A behaviour test, not an interview
Are AI-moderated interviews ethical? Consent, disclosure and privacy
They can be, if participants know what they are talking to and agree to it. Four rules cover most of it.
1. Say it's an AI, up front. Under Article 50(1) of the EU AI Act, providers must design AI systems that interact directly with people so that those people are informed they are dealing with an AI, unless that is obvious. The obligation applies from 2 August 2026 (EU AI Act, Article 50). Outside the EU it is simply good practice: NN/g saw participants hold back when sessions began without a proper introduction.
2. Get consent before anything is recorded. If you rely on consent under GDPR, you must be able to show the person gave it, and they can withdraw it at any time (GDPR Article 7). Record a consent timestamp per session and make withdrawal easy.
3. Collect only what you need. Don't require an account, don't ask for names you won't use, and set a retention period. If the interview runs inside your product, don't record screen content when a list of actions will do.
4. Don't let the AI sell. An interviewer that pitches, praises or leads turns research into marketing. The guide should forbid it, and you should read transcripts to check.
A short checklist before launch:
The opening message names the AI and the company behind it
Consent is asked before the first question, and logged
Participants can stop at any time and ask for deletion
The guide avoids sensitive topics you don't need (health, finances, politics)
You know where transcripts are stored and for how long
Someone on the team reads a sample of transcripts every week
How do you write a guide an AI interviewer can run well?
AI interviewers follow the script closely, so the script carries more weight than in a live call.
Ask for a story, not an opinion. "Tell me about the last time you exported a report" beats "Do you like our reports?"
One question per message. Stacked questions get half-answered.
Keep it to five to seven questions, plus permission for one follow-up on each. Short guides finish; long ones get abandoned.
Scope it tightly. "The last time you onboarded a new teammate", not "your experience with our product".
Ban pitching and praise in the instructions, explicitly.
Pilot it on yourself and two colleagues before sending it to customers.
For a ready-made set of story-based questions, start from the free JTBD interview template and cut it down to the five that matter.
How many to run? The same rule as live interviews: start with five or six per type of customer and stop when new answers add nothing; in one landmark study, 34 of 36 frequent themes had appeared after six interviews (NN/g, 2021). Because AI interviews cost less time, it is tempting to run 200. More transcripts don't fix a vague question, and someone still has to read them. For turning answers into findings, see how to keep AI research synthesis honest.
How does Alex run AI-moderated interviews?
Alex is Specky's AI research agent. It runs these interviews in three ways:
Link interviews (available now). Share a link by email or anywhere else. Participants need no account, and the chat takes about two minutes. It ends with a 0–10 recommend question and the reason.
In-product studies (available now). One script tag adds a widget that asks a sample of users (5% of visitors by default, each asked once) for consent, gives them a task, then asks what they were trying to do. With consent, it notes which parts of the product they clicked, by name only, never what is on screen.
Visitor conversations (early access). A short chat asking signed-out visitors what brought them.
Alex's instructions follow the rules above: one question at a time, mirror the participant's words, follow threads with "What happened next?", never pitch or defend the product, keep each turn to two to four sentences. No session starts until the participant opts in, and each in-product session stores a consent timestamp. Every completed interview returns a transcript, a short summary, sentiment and themes; studies add an action log and whether the task was completed, failed or abandoned. Results land in your Product Graph next to calls, tickets and feedback. See pricing for plan details.
What does an AI interview programme look like in practice?
The following is illustrative, not a real company.
Say a SaaS company sees about 120 cancellations a month and wants to know why. It adds an AI interview link to its cancellation email, with one scoped question: "Tell me about the moment you decided to cancel."
Weeks 1–2: 19 of roughly 60 people who cancelled complete the interview (about 32%).
Themes: 8 mention a missing integration, 6 a price rise at renewal, 3 a team change, 2 unclear.
Follow-up: the team books four live calls with people from the integration group to understand what the integration needed to do.
Decision: the integration moves up the roadmap, and the price-rise group gets a retention offer test.
The AI interviews widened the evidence; the live calls deepened it. Neither alone would have shown both the pattern and the detail.
What mistakes do teams make with AI interviews?
Using them for open-ended discovery in a new market. Start with live interviews; scale with AI later.
Treating a summary as the finding. Read the transcripts behind any theme you act on.
Counting themes as prevalence. Twelve interviews that mention a problem don't tell you what share of customers have it.
Hiding the AI. It breaks trust, and in the EU it can break the law.
Running interviews nobody reads. Book the synthesis time when you launch the campaign.
Confusing AI interviews with synthetic users. One talks to customers; the other imagines them.
Are AI-moderated interviews as good as human interviews?
Not for everything. They match or beat surveys and quick calls for short, structured questions, and they remove scheduling. Nielsen Norman Group's 2026 study found AI interviewers weak at probing unexpected answers and building rapport, and judged them not yet suitable for in-depth interviews. Use them alongside live interviews, not instead of them.
Do participants need to know they are talking to an AI?
Yes. It is the honest choice, it improves trust, and in the EU it is a legal duty from 2 August 2026: Article 50 of the AI Act requires that people are informed when they interact directly with an AI system unless it is obvious. Say so in the first message and before you ask for consent.
What is the difference between AI interviews and synthetic users?
In an AI interview, a real person answers questions asked by an AI. With synthetic users, an AI model generates the answers, imitating a user group. AI interviews produce customer evidence; synthetic users produce hypotheses at best. Nielsen Norman Group found synthetic users overly optimistic and shallow compared with real research participants.
How many AI interviews do you need?
Plan as you would for live interviews: five or six per type of customer for one narrow question, stopping when new answers add nothing. Running hundreds is easy but rarely helpful unless you are comparing many segments. If the answers are all over the place, sharpen the question or narrow the audience before you collect more.
How long should an AI interview be?
Short. Five to seven questions, finishing in a few minutes, is enough for one focused question and keeps completion high. NN/g saw AI interviews run from 13 to 56 minutes when tools had no target length, so set one. Topics that need an hour of exploration belong in a live interview.
GDPR-aligned. EU workspace data is stored in Frankfurt, Germany. GDPR and EU data Built in Vienna, Austria. A product of influence solutions fm GmbH (FN 675374y). Imprint 14-day trial. Card required, and nothing is charged until day 14. 30-day refund after subscribing. See pricing