The first 3 interviews are free · Start free or see the prices

New Feature Feedback Interview: Questions to Ask After a Release

Use this guide 2 to 6 weeks after a release. Talk to users who tried the new feature and to users who did not. It shows how people found the feature, what job they gave it and what stops repeat use.

Length
25 min
Questions
14
People
8 to 12
Sections
5
Run this study

What this guide finds out

  • How users found the feature, or why they did not see it.
  • Which task users gave the feature, compared with the task you designed it for.
  • Where the first use failed or confused users.
  • If the feature replaced an old workaround, or if users went back to the workaround.

When to use it

  • Usage data shows many first uses but few repeat uses.
  • You must decide to improve, keep or remove a feature after a release.
  • Support gets questions about the feature and you do not know which users are affected.

Who to talk to

Active users of the product, split into users who tried the new feature and users who did not. Talk to 8 to 12 people. Each interview takes about 25 minutes.

The discussion guide

Each question has a probe note. The note tells the moderator, a person or an AI, what to do with the answer.

01

Warm-up

3 min

Learn how the participant uses the product now.

  1. What do you use the product for in a typical week?

    ProbeAsk for the one task they do most often. Record it as context for the rest of the interview.

  2. Before the last few weeks, how did you do the task that the new feature is for?

    ProbeDescribe the task in neutral words, not with the feature name. Ask them to show or describe their old method step by step.

02

Discovery

4 min

Find out how users found the feature, or why they did not.

  1. Have you noticed anything new in the product recently?

    ProbeDo not name the feature. If they do not mention it, ask if they saw it, and record that they did not find it alone.

  2. Where did you first see it, and what did you think it was for?

    ProbeAsk what words or image made them think that. Compare their idea with the real purpose, but do not correct them.

03

First use

7 min

Rebuild the first use and find where it failed.

  1. Tell me about the first time you tried it. What were you trying to do?

    ProbeAsk for the real task and the date. If they did not try it, skip to the next section.

  2. What happened, step by step?

    ProbeAsk what they expected at each step. Stop at the first step where the result was different from what they expected.

  3. Was there a moment when you were not sure what to do next?

    ProbeAsk what they did at that moment: guess, read help, ask a colleague or stop.

  4. How did the result compare with your old way of doing the task?

    ProbeAsk for one thing that was better and one thing that was worse. Ask about time, effort and quality.

04

Repeat use

6 min

Learn if the feature became part of the routine.

  1. When did you last use it, and what for?

    ProbeIf the last use was the first use, ask what made them not use it again.

  2. Do you still use your old way for this task?

    ProbeIf yes, ask in which cases they choose the old way. Record each case.

  3. Who else on your team uses it, if anyone?

    ProbeAsk if they told anyone about the feature. Ask what they said.

05

Non-users and close

5 min

Learn what stops non-users, and get one priority change.

  1. If you have not tried it, what would make you try it?

    ProbeAsk this only to people who did not try it. Accept "I do not need it" and ask how they do the task now.

  2. What is one change that would make you use it more often?

    ProbeAsk them to name one change. Ask what it would let them do that they cannot do now.

  3. Is there anything else about the feature that you want to tell me?

    ProbeThank them and end the interview.

Example finding

ExampleOld workaround wins

Users who tried bulk edit once often went back to a spreadsheet, because the spreadsheet let them undo.

“It worked, but I could not see what I changed. In Excel I can at least press undo, so I just went back to that.”
In a User Evaluation study report, each claim links to the clip where a participant says it.

Mistakes to avoid

  • Do not talk only to users who tried the feature. Non-users tell you about discovery and fit, which users cannot.
  • Do not name the feature in the first questions. A name tells participants what you want to hear about.
  • Do not ask "Do you like it?". Liking does not predict repeat use. Ask about the last real use.
  • Do not interview in the first week after release. Users need time to try the feature on a real task.

Analyze the answers

Analysis templateAffinity Mapping Template: How to Do an Affinity DiagramUse this template to sort interview notes into clusters from the bottom up. The rules say what goes on a note, when notes belong together and how to write a useful label.

Questions

Wait 2 to 6 weeks after the release. Users then have time to try the feature on real work. After that, they forget the first use.

Yes. Put about one third of the sample in this group. They show if the problem is discovery, fit or trust.

A usability test watches people do a set task. This interview asks about real use over several weeks, including repeat use and the old workaround.

Show it only at the end, after the discovery questions. If you show it earlier, you cannot learn if users find it alone.

Let the AI moderator ask these questions

The AI moderator interviews screened people by voice or text. Each claim in the study report links to its clip. The first 3 interviews are free.

Run this study