Follow-Up Depth: How Many Follow-Up Questions to Allow in an AI Interview
Follow-up depth is the number of follow-ups after each answer. Too few gives labels without stories. Too many tires the participant and leaves no time for later questions.
Overview
Each follow-up adds a turn to the interview. A guide with eight questions and two follow-ups on each has up to 24 turns. Eight questions with no follow-ups is a different interview. It is shorter and asks less effort of the participant.
In User Evaluation, the probe depth setting has three levels. Light asks no follow-ups, Standard asks one and Deep asks two. The interview length can be 15, 30 or 45 minutes. The help docs say that five strong questions can fill a 30-minute interview.
Depth is a study setting, but your probe notes control each question. Use the setting for the default, and use the notes to turn follow-ups off where they add nothing.
Rules
- 01
Count the turns before you choose the depth.
The number of turns is the number of questions multiplied by one plus the follow-ups. Compare this number with the length you set. Then time a pilot to learn the real minutes for each turn in your study.
Do notTwelve questions, Deep probe depth, 15-minute interview.
DoFive questions, Deep probe depth, 30-minute interview. The pilot took 26 minutes.
- 02
Turn off follow-ups on context questions with the probe note.
Questions about role, tenure or team size only give context. A follow-up on them uses time that the main questions need.
Do notHow long have you worked in this role? (no note, Deep depth)
DoHow long have you worked in this role? Note: do not ask a follow-up. Record the answer as context.
- 03
Choose the depth by the type of answer you need.
A decision story, such as a cancellation, needs two follow-ups to get the event and the people around it. A quick reaction to a name or a headline loses value when you ask for it twice.
- 04
Make the second follow-up ask about something different from the first.
A second follow-up that repeats the first in new words tells the participant that the first answer was wrong. It adds no data. Write the note so that the second follow-up moves to a new detail.
Do notModerator: "Can you tell me more about that?" Then: "What else can you tell me about that?"
DoModerator: "What happened right before that?" Then: "Who else was involved when it happened?"
- 05
Stop when the participant cannot remember.
Memory of small events fades fast. When a participant does not remember, more follow-ups push them to guess. In the transcript, a guess looks like a fact.
Do notNote: if they do not remember, ask them to try to think back.
DoNote: if they do not remember, accept it. Ask what they usually do instead, then move on.
- 06
Use fewer follow-ups in text than in voice.
Typing takes more effort than talking. Each follow-up in a text study costs the participant more, and answers get shorter as the session continues.
- 07
Rewrite a question that always needs two follow-ups.
If every participant needs two follow-ups to give an event, the question is the problem. A better question gets the event in the first answer and frees a follow-up for something new.
Do notWhat do you think about reporting?
DoTell me about the last report you made for your manager. What did you do first?
- 08
Use live steering for exceptions, not as the plan.
In User Evaluation you can listen in and click "Push deeper" to ask one more follow-up. Steering needs the Team or Scale plan, and you cannot attend every session. The guide must work without you.
Checklist before launch
- You counted the turns and they fit the interview length.
- Context questions have a note that turns follow-ups off.
- The main decision questions allow two follow-ups.
- Second follow-ups ask about a new detail, not the same one again.
- Each note says what to do when the participant does not remember.
- Text studies use fewer follow-ups than voice studies.
- A timed pilot ended inside the length you set.
Questions
Ask one or two follow-ups on the questions that carry your research goal. Ask none on context questions. More than two follow-ups on one answer usually repeat themselves.
Probe depth is the number of follow-ups that the AI moderator asks after each answer. In User Evaluation it is Light (none), Standard (one) or Deep (two).
Only up to a point. The first follow-up often gives the most specific detail. Later follow-ups give less, and they tire the participant before the later questions.
Ask real people your question
The AI moderator interviews screened people by voice or text. Each claim in the study report links to its clip. The first 3 interviews are free.