AI Respondents Matched a UX Survey’s Main Finding but Gave Fewer Distinct Answers

On October 16, 2026, Nielsen Norman Group published a comparison of how people and AI respondents answered an open-ended question about what to call work in the UX field. According to the group, both samples led to the same conclusion: UX remains the most common term. But the synthetic respondents offered far fewer alternatives. If rare answers or differences between professional groups matter to a study, it is better to ask people: matching the main finding does not mean AI captured the full range of views.

How the responses were compared

The researchers revisited a question from an earlier survey of 604 technology and design professionals. Participants were asked to name one professional field for work in which someone studies people’s needs and experiences, then turns those insights into recommendations for developing a product or service. The same question was put to 120 synthetic respondents created in February 2026 using the Synthetic Users platform.

The platform received the study plan, including the questions and selection criteria. The synthetic participants were meant to represent professionals in different roles and countries. Synthetic Users assigned each one a profile that included age, location, profession, cultural context, and other characteristics. The researchers analyzed both samples in the same way. If a person or AI respondent named several fields, they recorded every term used rather than assigning the response to just one category.

The conditions under which they answered were different, however. People completed a survey, while the platform simulated interviews, so synthetic participants often gave longer answers. Synthetic Users sometimes changed the wording of the question, adding a reference to people’s behavior, for example. Even a small change like that could affect word choice, and the researchers do not rule out its influence. This is a limitation of the comparison itself: the researchers did not control every aspect of how the platform asked the question.

The shared conclusion hid a difference in the answers

UX-related terms dominated both samples. But people used more than 50 distinct terms, while synthetic respondents used about a dozen. Human responses included rare, specific terms related, for example, to studying consumer behavior or managing the customer journey. The AI responses contained no such unusual terms. And brevity was not the reason: the interviews gave synthetic participants more opportunity to list related concepts.

UX-related terms appeared 14 percentage points more often in the synthetic sample than in the human sample. The researchers found the difference statistically significant, although it does not change the survey’s practical takeaway. The human sample may already have been particularly familiar with the term UX: some participants were recruited through Nielsen Norman Group channels, and 67% of the people surveyed were UX professionals. Among the synthetic respondents, 20% were UX professionals—the proportions of different professions were set deliberately when the sample was created. That makes the AI’s even stronger tendency to return to UX all the more notable.

Profession affected word choice for both people and synthetic participants, but differences between roles were less pronounced in the AI responses. For example, 93% of synthetic product managers used UX terminology, compared with 44% of product managers in the human sample. The researchers compare this pattern to mode collapse: generation gravitates toward common options and fails to reproduce many rare ones. The comparison itself cannot establish exactly why the synthetic responses turned out this way.

What this means in practice

If you need an initial indication of the most common answer, synthetic respondents can be useful. But a website owner, marketer, or designer may need to hear how different groups describe work, products, or services in their own words. In an open-ended question, an uncommon term is not necessarily just noise. It may reveal a distinction hidden by the overall result.

You can ask a generator for more options, but the number it produces says nothing by itself about their reliability. What matters is whether real people use those terms and how often they use them. That is hard to verify without human responses. AI responses are useful for an initial check of an idea; conclusions about users’ views need to be confirmed through research with people. This is especially true when a decision depends on unusual wording or differences between audiences.

My take

As a designer just starting out, this result reminds me that a tidy overall conclusion does not tell the whole story. When working on a design, it is easy to pick the most common term and decide the wording is settled. But one person’s uncommon answer might reveal where the team and part of the audience understand the same work differently.

I would use synthetic responses to prepare questions and spot a likely general pattern. But before choosing words for an interface and handing the decision over to a developer, I would want to hear from people. Otherwise, an important detail could get lost before it even makes it into the design.

Sources

Where the news comes from. The text is a retelling in the author’s own words; the facts come from the source, the opinion is the author’s.

1. Synthetic Users vs. Human Respondents: Same Conclusion, Less Variation www.nngroup.com

About the author

Tomáš F.

Web design student

I’m studying web design in Brno and building a portfolio from class projects. I’m especially interested in how to hand a design off to a developer without losing the important details.

All posts by the author