Interview Length and Drop-Off in AI-Moderated Interviews
In an AI interview nobody is on the call to keep the participant there. Set the length from the goal, put the key questions first and find the question where people stop.
Overview
When a person moderates, social pressure keeps most participants on the call. In an AI interview, the participant can close the tab. Length and question order matter more than in a scheduled call.
In User Evaluation, an AI interview can be 15, 30 or 45 minutes. Participants must answer at least half of the questions before they can press Done. They can press End at any time, but an interview that stops early is not paid. The invite page tells them about how many questions to expect.
The recording uploads in short pieces while the interview runs. If a participant leaves early, you still have what they said up to that point.
Rules
- 01
Set the length from the research goal, then cut questions to fit.
A guide grows when each stakeholder adds a question. Write the goal first. Remove each question that does not serve it.
Do notFourteen questions from four teams in a 15-minute interview.
DoFive questions about the goal "why trial users do not pay" in a 30-minute interview.
- 02
Put the most important questions in the first half.
Attention falls as a session continues, and the last questions get the shortest answers. Participants also reach the minimum for Done after half the questions.
- 03
Give the true length in the invite.
A participant who expects 10 minutes and gets 30 leaves before the end. An honest length filters for people who have the time.
- 04
Remove questions that the screener already answered.
Asking for facts twice wastes time and tells the participant that nobody read their answers.
Do notHow many people work at your company? (also in the screener)
DoRemove the question and use the screener answer in the analysis.
- 05
Place on-screen ratings after the open question on the same topic.
A rating first gives the participant a number to defend. An open question first gets the story, and the rating then summarizes it.
Do notRate the onboarding from 1 to 5. Then: tell me about your first day.
DoTell me about your first day with the product. Then: rate the onboarding from 1 to 5.
- 06
Find the question where people stop.
Read the sessions that ended early and note the last question. If many stop at the same question, that question is the cause. Rewrite it, move it or remove it.
- 07
Keep follow-ups from using up the time.
Two follow-ups on every question can double the number of turns. Turn off follow-ups on context questions so the time goes to the main questions.
- 08
Plan a thank-you for your own customers.
In User Evaluation, panel participants are paid for each completed interview. People you invite with your own link are not paid by User Evaluation. Arrange a fair thank-you for the length you ask for.
Checklist before launch
- The research goal is written in one sentence above the guide.
- Every question serves that goal.
- The key questions are in the first half of the guide.
- The invite states the true length.
- No question repeats a screener question.
- On-screen ratings come after the open question on the same topic.
- A timed pilot ended inside the length you set.
- You have a thank-you plan for your own customers.
Questions
Long enough for your main questions and their follow-ups, and no longer. A focused guide of about five strong questions can fill 30 minutes.
In User Evaluation, the recording uploads in pieces during the interview, so you keep what they said. An interview stopped early is not paid.
Look at the last question in each session that ended early. If many stop at the same point, that question or the item shown there is the likely cause.
Ask real people your question
The AI moderator interviews screened people by voice or text. Each claim in the study report links to its clip. The first 3 interviews are free.