This site reviews published research on AI and audiences, for insights and marketing communications leaders. It does not run new research. It reads the studies, grades them and states what they mean for practice. It has three parts.
Can I trust AI to help me understand my audience?
Yes for coding text and ranking options, with a human check.
Only after validation for whole-population estimates.
No for subgroups. Synthetic respondents are not yet reliable for subgroup analysis.
How will my audience respond when they know AI was involved?
Trust goes down in almost every study.
The drop is largest for emotional messages, and when people assume AI did all the work.
A specific statement of what AI did makes the drop smaller. Exposure by someone else makes it larger.
Has research been through a change like this before?
Yes, twice: the telephone, then the internet.
Each time, most work moved to the cheaper method. The old method became the benchmark for high-stakes work.
AI adds something new. It can generate the answers, not only collect them.
Each study has an evidence grade. Each use case has a verdict. Each verdict names the evidence that would change it. The Study library lists all 30 studies. The Method page explains how we grade them.
Each figure uses a different measure. We do not average them. The Method page explains why.
I am a researcher. For about 15 years, I have worked in audience research, measurement and analytics. I founded Zeta Impact Advisory, a research and measurement practice. Before that, I was a senior program officer for measurement and insights at the Gates Foundation. Earlier, I worked across technology, media and telecommunications, at T-Mobile, Strategy Analytics, eMarketer, The New York Times and France 24.
AI has made many knowledge workers anxious. Market and consumer research is one of the fields most affected, and it is my field. Every week brings new claims. Some say AI will replace surveys and interviews. Others say AI research is worthless. Both kinds of claim are loud. Few of them show their evidence.
A large, vague worry is hard to manage. It becomes smaller when you divide it into specific questions and check the evidence for each one. That is how this site works. Which research tasks can AI do well today? Which can it not do yet? How do audiences react when they know AI was involved? What happened the last time technology changed our industry?
This is a personal project. I built it to learn how my industry, and the areas I have worked in, are changing with AI. I also built it for people who are working through the same questions. No organization funds this site. I do not sell synthetic respondents, survey panels or AI research tools. The verdicts follow the evidence, and each one says what would change it.
Do you know a study that I missed? Do you disagree with a verdict, or with how I graded a study? Send me a message. I will update the site when the evidence changes.
Message Zhaowen Wu on LinkedIn ↗linkedin.com/in/zhaowenwu
Assessment The answer depends on the level of detail you need. AI does well when it codes text into defined categories. It does reasonably well when it ranks options. It is less reliable for whole-population estimates. It is not reliable for the spread of opinion or for subgroups. Many audience decisions depend on subgroups.
Open-ended survey answers, annotation, interview coding with a codebook
AI was more accurate than crowd workers on defined categories. Agreement with human experts was moderate for interpretive codes. Check a human-coded sample.Gilardi 2023 · Zhang 2025 · Arora 2025
Which message, concept or treatment is stronger
AI-predicted effects correlated with actual effects (r = 0.85, preprint figure). The AI overestimated effect sizes. Use AI to shorten a list. Test the finalists with real people.Ashokkumar/Hewitt 2026 · Maier 2025 · Li 2024
What share of adults think or do something
Error was about 3 points on common political questions and about 20 points on coffee questions. Some willingness-to-pay estimates had the wrong sign.Verasight 2025–26 · Bisbee 2024 · Brand 2025
Variance, intensity, minority views
Synthetic answers had less variance than real answers. With answer order randomized, answers moved toward random. One new method narrowed the gap in one product category.Bisbee 2024 · Dominguez-Olmedo 2024 · Wang 2025 · Maier 2025
Crosstabs, segment sizes, driver and regression analysis
Error increased as groups got smaller. Models missed the same groups again and again, for example older and religious people and non-Western populations. Regression results differed from real data.Verasight · Santurkar 2023 · Atari 2023 · Bisbee 2024 · Wang 2025
Mean absolute error between synthetic respondents and a real survey, in percentage points. Source: Verasight Synthetic Sampling Reports I, III and IV (2025–26). “≈10” is the average subgroup error stated in the reports. “Up to 30” is the largest subgroup error they report. It is not an average. The publisher sells human survey panels.
We tested this position against the 15 studies. It holds.
Evidence for this verdict
Strongest evidence against. Park et al. (2024) built simulated respondents from two-hour interviews with 1,052 real people. This reduced accuracy gaps between racial and ideological groups. But each simulated respondent depends on a real person's interview. The method copies people you already studied. It does not replace a sample.
What would change this verdict. Independent, preregistered tests that meet all three conditions:
Assessment Disclosure reduced trust in almost every controlled study in this review. In both field experiments, it also reduced purchases or clicks. The size of the drop depends on the message and on what the audience assumes AI did. Marketing communications teams can control both.
The source can be the sender, a platform label or a third party. Exposure by a third party caused the largest drop (Schilke 2025).
Without details, people assumed AI did all the work (Altay 2024).
Emotional messages lost the most (Kirk 2025; Rubin 2025).
Stated intentions changed little. Observed clicks and purchases changed a lot (Wittenberg 2025; Luo 2019; Lee 2025).
| Study | Design | Measure | Effect of disclosure | Grade |
|---|---|---|---|---|
| Luo et al. 2019 | Field experiment, 6,200+ customers | Purchase rate | −79.7% | A |
| Lee et al. 2025 | Field test, 105,999 ad impressions | Click-through rate | −31.5%1 | B |
| Schilke & Reimann 2025 | 13 preregistered experiments | Trust in the person or firm | −16% to −20%2 | A |
| Altay & Gilardi 2024 | 2 preregistered experiments, 4,976 people | Perceived accuracy, sharing intent | ⅓ of a “false” label | A |
| Wittenberg et al. 2025 | 2 preregistered experiments, 7,579 people | Belief; stated engagement | Belief ↓; engagement ≈ 0 | A |
1 Figure from news coverage of the working paper. 2 Figures from the authors' university press release.
Assessment In a survey, people said an AI label would not change their sharing much (Wittenberg). In field tests, clicks and purchases fell sharply (Luo; Lee). The studies used different settings, so this comparison is indirect. It suggests that intention surveys underestimate the disclosure penalty.
| Factor | Larger penalty | Smaller penalty | Studies |
|---|---|---|---|
| Message type | Emotional messages: thank-you notes, condolences, care | Factual messages: updates, service information | Kirk 2025; Rubin 2025; Yin 2024 |
| Stated role of AI | No detail, or “AI-generated” | A specific, smaller role: “edited with AI” | Altay 2024; Kirk 2025 |
| Evidence of process | A label only | A label with the list of sources | Toff 2025 |
| Who led the work | AI led, with “human oversight” | A person led, with AI help | Reuters 2025; Reis 2024; Brüns 2024 |
| How the audience learned | A third party exposed it | The sender disclosed it | Schilke 2025 |
| Audience | High trust in news; more knowledge of journalism | Positive attitudes toward AI | Toff 2025; Schilke 2025 |
| Norms | A mixed setting, where only some people use AI | A setting where everyone uses AI | Jakesch 2019 |
Hiding AI use is a risk. In one set of experiments, exposure by a third party reduced trust more than self-disclosure did (Schilke 2025).
A specific statement works better than a general label. People assumed full automation when a label gave no detail (Altay 2024). In one news study, a list of sources removed the trust drop (Toff 2025).
Human leadership helped more than human oversight. Comfort was 21% for news made mostly by AI with human oversight. It was 43% for news made mostly by people with AI help (Reuters 2025). Advice from “a physician with AI” was still rated less reliable than advice from a physician alone (Reis 2024). If a person led the work, say so. If AI led the work, oversight wording will probably not restore trust.
Assessment Yes, twice. The telephone and then the internet each made research cheaper and faster. Each time, methodologists warned about a specific loss of quality. Each time, most of the work moved to the new method anyway. The old method did not disappear. It moved to the work that needs its quality. AI looks like a third change of this kind, with one important difference: AI can generate the answers, not only collect them.
| Framework | Question it answers | Source |
|---|---|---|
| Three eras of survey research | What changed in how researchers selected and reached people? | Groves, Public Opinion Quarterly, 2011 |
| Total survey error | Which kind of error did each new method add or reduce? The framework splits error into coverage, sampling, nonresponse and measurement error. | Groves & Lyberg, Public Opinion Quarterly, 2010 |
| Disruptive innovation | Why does a cheaper, lower-quality method take most of the market? Disrupters start with customers who need only a “good enough” product, then improve and move up. | Christensen, Raynor & McDonald, Harvard Business Review, 2015 |
| Task-based model of automation | What happens to the work? Automation removes tasks from people (displacement). New technology also creates new tasks for people (reinstatement). | Acemoglu & Restrepo, Journal of Economic Perspectives, 2019 |
Why these four: the first two come from survey methodology and describe this industry directly. The last two are established, widely tested theories of markets and work. We do not use a generic “technology adoption arc,” because no single arc is an established law.
| Era | How researchers selected and reached people | What became cheaper or faster | Main quality risk | What happened to the old method |
|---|---|---|---|---|
| 1 · Invention 1930–1960 |
Area and quota samples. Face-to-face and mail interviews. | Scientific sampling replaced very large straw polls. In 1936 the Literary Digest received 2.4 million ballots and predicted the wrong winner. Gallup used a much smaller sample and predicted the right one. | Sample design, not sample size. Quota samples also failed in 1948, and probability sampling replaced them. | Straw polls lost credibility. The Literary Digest went bankrupt. |
| 2 · Expansion 1960–1990s |
Random-digit dialing. Telephone interviews, later computer-assisted (CATI). | Telephone became an accepted method in the 1970s, when about 90% of US households had a telephone. It then passed face-to-face interviewing as the most common method. | Coverage at first (homes with no phone). Later nonresponse: Pew's telephone response rate fell from 36% in 1997 to 6% in 2018. | Face-to-face interviewing continued, but it became the less common method. |
| 3 · Online 1990s–2020s |
Opt-in online panels. Self-completed web surveys. | Online research grew fast. About 56% of national US polls before the 2016 election used online methods; in 2020 the share was 84%. | Coverage and selection. In 2010, AAPOR advised researchers not to use opt-in panels to estimate population values. Pew later found that opt-in samples had about twice the error of probability-based panels. | Live phone became the only method for 10% of national pollsters in 2022. Probability-based online panels grew: from 7 to 23 national pollsters between 2016 and 2022. |
| 4 · AI 2023– · Emerging |
AI across the workflow: survey design, AI-moderated interviews, AI coding, synthetic respondents, research agents. | Speed and scale. Vendors report that AI moderators can run hundreds of interviews in a day. Vendor surveys report fast adoption. | A new kind of error. Synthetic respondents have no sampling frame. Their error comes from the model. The total survey error framework does not yet name it. | Too early to know. See the predictions below. |
Sources: Groves 2011; course and encyclopedia summaries of the 1936 and 1948 polls; SAGE Encyclopedia of Survey Research Methods; Pew Research Center 2012, 2019 and 2023; AAPOR Report on Online Panels 2010. Full list at the end of this page. Era dates follow Groves (2011). Some authors date the end of the second era at 2000.
Response rate of a typical Pew Research Center telephone survey. Source: Pew Research Center (2012 study table; 2018 figure as reported in 2019). Pew moved most of its work online.
Global insights industry turnover, 2024, preliminary. Source: ESOMAR Global Top Insights Companies 2025. ESOMAR later reported the total as US$153 billion. ESOMAR also reports that about 48% of research projects are now done inside client organizations. Software and in-house teams already do much of the work that agencies once did.
Telephone replaced travel. Online replaced interviewers.
Phone: coverage, then nonresponse. Online: selection bias (AAPOR 2010).
It starts with work where “good enough” is enough. This matches disruptive innovation.
Efficient random-digit dialing (1978). Weighting. Probability-based online panels.
It becomes the benchmark that checks the new method.
| Framework | Prediction | Evidence so far | Signal that would confirm or refute it |
|---|---|---|---|
| Disruptive innovation | AI methods take volume first in low-stakes, exploratory work: early concept screens, message pre-tests, and teams that did little research before. They move to high-stakes work only as quality improves. | The verdicts in AI in Audience Research: AI is reliable for ranking options, not for subgroups. About 48% of projects are already done in-house (ESOMAR). | The share of final decisions, not early screens, that rest on synthetic data. Track it in industry surveys that are not run by vendors. |
| Total survey error | The field will name and measure a new error source: model error. Professional bodies will issue guidance, as AAPOR did for online panels in 2010. | The UK Market Research Society's Delphi Group published a report on synthetic participants in 2024. No accepted error framework exists yet. | Published standards and benchmark tests from AAPOR, ESOMAR or MRS. |
| Old method as benchmark | Data from real people becomes more valuable per unit. It will be used less often, but as the reference that calibrates AI methods. | Fine-tuning on past human surveys improved AI results (Brand 2025). Interview-based simulations needed real interviews (Park 2024). Probability panels grew from 7 to 23 national pollsters (Pew). | Growth or decline of probability panels and high-quality human samples. |
| Task-based model | Displacement: routine fielding, routine interviews, coding, tables and first-draft reports. Reinstatement: validation design, benchmark upkeep, model audits, synthesis across sources, and framing the business decision. | Vendor tests found AI moderators weaker than expert humans on rapport and on probing beyond the script (Ipsos 2025). AI coding beats crowd workers on defined categories (Gilardi 2023). | Job postings and team structures: fewer fieldwork and coding roles; more roles in validation, data science and research strategy. |
What history cannot tell us: whether total employment in research will grow or shrink. In the task-based model, the net effect depends on the balance between displacement and new tasks. History shows both forces, not which one will be larger this time.
For comparison: in 2010 the AAPOR task force recommended against using opt-in panels to estimate population values. A decade later, most national polls were online. Expert warnings did not stop the shift. They shaped the methods that made it work.
Filter by collection, grade or study design. Each card shows what the study found and our assessment of it.
We use one rubric for both collections. If you disagree with a verdict, you can see which studies it depends on.
A median across studies is meaningful only when the studies use the same measure. These studies do not. Disclosure studies measure purchase rates, click-through rates, trust scales and belief ratings. Synthetic-respondent studies measure correlations, mean absolute error, agreement scores and distribution similarity. An average across these measures would look precise but would have no meaning. We show each figure with its measure. Verdicts depend on the direction and consistency of results across study designs.
| Criterion | Increases the grade | Decreases the grade |
|---|---|---|
| Design | Randomized field experiment. Comparison with high-quality human data that the model did not see. | One scenario, one dataset, or a case study rated by experts. |
| Measure | Observed behavior. Established survey benchmarks such as ANES, GSS, Pew or Census data. | Stated intentions only. |
| Scale and replication | Several preregistered studies. Many models or many questions. | One study. A small or convenience sample. |
| Independence | Academic authors with no commercial interest in the result. | Authors from a firm that sells synthetic respondents or human survey samples. |
| Review status | Peer-reviewed journal or major conference. | Preprint, working paper or white paper. |
| Currency | Recent models. Tests on data after the model's training cutoff. | Older models, or a period before 2022. We note this; it does not exclude a study. |