The first 3 interviews are free · Start free or see the prices

AI Feature Trust Interview: Guide and Questions

Use this guide with people who use an AI feature in your product. It finds which tasks they give to the AI and how they check results. It also finds what a wrong result did to their trust.

Length
30 min
Questions
13
People
10 to 15
Sections
5
Run this study

What this guide finds out

  • Which tasks users give to the AI feature, and which they keep for themselves.
  • How users check AI results, and when they use a result without a check.
  • What wrong results did to trust and use.
  • How much control users want, and what concerns they have about data.

When to use it

  • Many people try your AI feature once, but few use it again.
  • Users edit most AI results before they use them, and you want to know why.
  • You plan to let the AI take actions without approval.

Who to talk to

People who used the AI feature three or more times in the last 30 days. Include some who tried it once and stopped. Talk to 10 to 15 people. Each interview takes about 30 minutes.

The discussion guide

Each question has a probe note. The note tells the moderator, a person or an AI, what to do with the answer.

01

Use of the AI feature

5 min

Learn what the person uses the AI feature for.

  1. When did you last use the AI feature, and what did you ask it to do?

    ProbeAsk for the exact request, in their words. Ask what result they got.

  2. Which tasks do you give to the AI feature, and which tasks do you keep for yourself?

    ProbeAsk why they draw the line there. Ask if the line has moved since they started.

02

Checking results

9 min

Learn how the person decides to trust a result.

  1. After the AI gives you a result, what do you do with it?

    ProbeAsk if they use it as is, edit it or check it. Ask how long the check takes.

  2. How do you decide if a result is correct?

    ProbeAsk what they look at first. Ask for an example of a result they did not accept.

  3. For which tasks would you use the result without checking it?

    ProbeAsk what makes those tasks safe. If there are none, ask what would need to change.

  4. What does the feature show you that helps you trust a result?

    ProbeAsk about sources, explanations and confidence labels. Ask which ones they read and which they ignore.

03

Wrong results

7 min

Learn what a wrong result does to trust.

  1. Tell me about a time when the AI feature gave a wrong or strange result.

    ProbeAsk how they noticed and what the result could have caused. If they never noticed one, ask how they would know.

  2. How did your use of the feature change after that?

    ProbeAsk if they check more now, use it for fewer tasks or stopped. Ask how long the effect lasted.

  3. What should the feature do when it is not sure of an answer?

    ProbeAsk them to choose: say so, ask a question, give the best guess or give no answer. Ask why.

04

Control and data

5 min

Learn the level of control the person wants and their concerns.

  1. How much do you want the AI to do on its own, and what should it ask you about first?

    ProbeAsk for one action they would let the AI do alone, and one they would not.

  2. What concerns do you have about the data the AI feature uses?

    ProbeDo not reassure them and do not explain the policy. Record the concern and where it comes from.

05

Close

4 min

Find what would increase use and trust.

  1. Would you be comfortable if a colleague or customer knew you used AI for this work? Why?

    ProbeThis shows social risk. If they would not tell people, ask what worries them.

  2. What would make you rely on the feature more?

    ProbeAsk for one change. Thank them and end the interview.

Example finding

ExampleOne wrong number breaks trust

Users who found one wrong figure in an AI summary checked every summary after that, which removed the time saving.

“It said revenue went up when it went down. Now I read the whole report anyway, so what is the point?”
In a User Evaluation study report, each claim links to the clip where a participant says it.

Mistakes to avoid

  • Do not ask "Do you trust the AI?" Trust depends on the task. Ask about specific tasks and specific results.
  • Do not explain how the AI works during the interview. Users act on what they believe, so record what they believe.
  • Do not talk only to fans of the feature. People who tried it once and stopped show the trust problem most clearly.
  • Do not treat more trust as the goal for every task. For risky tasks, the right result is that users check.

Analyze the answers

Analysis templateThematic Analysis Codebook: Starter Codes for InterviewsUse this codebook as the first pass on interview transcripts about a task or a workflow. The 10 codes describe what people do and why, so themes come from behavior, not from opinions.

Questions

Look at behavior, not opinion. Ask which tasks people give to the AI, how they check results and what changed after a wrong result.

Yes. Include some people who tried it once or twice and stopped. Their reasons often explain low adoption.

Tell people at the start that the moderator is an AI. Some people are more open about AI problems with an AI moderator, because they do not fear judgment.

Let the AI moderator ask these questions

The AI moderator interviews screened people by voice or text. Each claim in the study report links to its clip. The first 3 interviews are free.

Run this study