The record for this issue
The poll with no one in it.
79 claims in the draft, each tested by a second agent against the page it cites. 77 held; 2 did not and were fixed or cut before this went out.
-
01
titleconfirmed
The poll with no one in it (synthetic poll with no real respondents)
The page says “there is growing interest in using it to stand in for real respondents in public opinion surveys”
-
02
dekwrong
On Sept. 30, Pew tested whether an AI model could answer its surveys
The page says “Pew published Sept. 30, 2026; the replications ran March 9-12, March 30-April 2, April 7-10 and April 27-May 1, 2026”
-
03
dekconfirmed
It missed by 12 points on average
The page says “by an average of 12 percentage points”
-
04
dekconfirmed
it turned varied groups into uniform ones
The page says “portrayed them as nearly ubiquitous”
-
05
summaryconfirmed
what Pew's test found; a different model gives a different public
The page says “each synthetic sample painted a very different picture of the American public on multiple dimensions”
-
06
art alt / art svg 'NOT SURE'confirmed
Questionnaire with a 'Not sure' box left empty (illustrates AI rarely choosing not sure)
The page says “human panelists were around four times as likely as the AI model to choose it”
-
07
The essay, section 1confirmed
Pew published the test on Sept. 30
The page says “Published September 30, 2026”
-
08
The essay, section 1confirmed
Pew calls the results synthetic samples
The page says “we wanted to understand the data these "synthetic samples" produce”
-
09
The essay, section 1confirmed
researchers note growing interest in using them to stand in for real respondents
The page says “there is growing interest in using it to stand in for real respondents in public opinion surveys”
-
10
The essay, section 1confirmed
used real panel members' profiles; three of Pew's 2026 surveys
The page says “three different ATP surveys conducted in the first half of 2026”
-
11
The essay, section 1confirmed
compared model answers with what the real people said
The page says “Average absolute error by group (across three waves) comparing synthetic and human results”
-
12
The essay, section 1confirmed
nearly 300 questions
The page says “Across nearly 300 individual survey questions”
-
13
The essay, section 1confirmed
average 12 percentage points
The page says “by an average of 12 percentage points”
-
14
The essay, section 1confirmed
around 28% of questions missed by more than 15 points
The page says “The average difference exceeded 15 points on around 28% of questions we asked”
-
15
The essay, section 1confirmed
AI models are not an adequate replacement for traditional polling on topics of broad public importance
The page says “AI models are not an adequate replacement for traditional polling on topics of broad public importance”
-
16
The essay, section 1confirmed
same figures reported by AI Weekly
The page says “off by an average of 12 percentage points ... 28% of questions missed by more than 15 points”
-
17
The essay, section 1confirmed
synthetic respondents are AI-generated personas meant to predict how real consumers would answer
The page says “Synthetic respondents are AI-generated personas designed to predict how real consumers would respond to research questions”
-
18
The essay, section 1confirmed
used to compare product ideas, pricing or messaging
The page says “Evaluate multiple product ideas...; compare pricing, messaging, and positioning options across simulated audiences”
-
19
The essay, section 2confirmed
2023 paper (labelled as background)
The page says “Published online by Cambridge University Press: 21 February 2023”
-
20
The essay, section 2confirmed
'silicon samples' by conditioning GPT-3 on thousands of sociodemographic backstories
The page says “create 'silicon samples' by conditioning the model on thousands of sociodemographic backstories from real human participants”
-
21
The essay, section 2confirmed
property called algorithmic fidelity
The page says “We term this property algorithmic fidelity and explore its extent in GPT-3”
-
22
The essay, section 2confirmed
Pew used a 'digital twins' approach
The page says “We used a "digital twins" approach for this study”
-
23
The essay, section 2confirmed
ATP is a multimode, probability-based panel of roughly 10,000 adults
The page says “multimode, probability-based survey panel ... roughly 10,000 adults”
-
24
The essay, section 2confirmed
probability-based means members are picked at random rather than volunteering
The page says “select people at random to join the American Trends Panel”
-
25
The essay, section 2confirmed
multimode means more than one way to take part
The page says “either online ... or over the phone with a live interviewer”
-
26
The essay, section 2confirmed
model given demographics and 2025 political typology answers
The page says “their self-reported demographic information ... their responses to questions on our 2025 political typology survey”
-
27
The essay, section 2confirmed
three surveys in the first half of 2026, same order and same instructions
The page says “given each question on the survey in order from start to finish ... exactly the same instructions and survey programming that the real humans who took it received”
-
28
The essay, section 2confirmed
Pew tested several models; main results use Claude Opus 4.6, best performance
The page says “had the best performance of several we evaluated”
-
29
The essay, section 2confirmed
low reasoning, extended profile information, expert reflection
The page says “models set to low reasoning with extended profile information and expert reflection”
-
30
The essay, section 2confirmed
worked example 20, 5, 15 -> 13.3 points; per-question average absolute error
The page says “we calculated the average absolute percentage point error for each survey question ... (20+5+15)/3 = 13.3”
-
31
The essay, section 2confirmed
overall average 12.4 points
The page says “Total ... Average 12.4”
-
32
The essay, section 2confirmed
model reads demographics and earlier answers, predicts new answers
The page says “self-reported demographic information ... responses to questions on our 2025 political typology survey”
-
33
The essay, section 2confirmed
Profile in, answer out; label 'Persona'
The page says “AI-generated personas (Greenbook's term; Pew says 'profile'/'digital twins'); descriptive label only”
-
34
The essay, section 3confirmed
97% of Hispanic adults at least somewhat likely to follow the World Cup
The page says “97% of Hispanic adults are at least somewhat likely to follow the World Cup”
-
35
The essay, section 3confirmed
actual share 43%
The page says “(while the actual share is 43%) -- also 43% in AI Weekly. NOTE: Pew's detailed-findings page gives 42% ('fewer than half of real Hispanic adults (42%)'); the cited overview and AI Weekly say 43%. Writer may want to…”
-
36
The essay, section 3confirmed
AI tended to portray a group's view as near-universal
The page says “portrayed them as nearly ubiquitous”
-
37
The essay, section 3confirmed
nearly half the questions had at least one answer choice no AI respondent picked
The page says “Nearly half the questions we asked had at least one answer choice that was not selected by a single AI-generated respondent”
-
38
The essay, section 3confirmed
humans about four times as likely to choose 'not sure'
The page says “human panelists were around four times as likely as the AI model to choose it”
-
39
The essay, section 3confirmed
AI poll estimated nearly every member of the public knows the answers to knowledge questions
The page says “our AI poll estimated that nearly every member of the public knows the answers”
-
40
The essay, section 3unsupported
A language model predicts likely text; asked for a typical group member it reaches for the most typical answer
-
41
The essay, section 3confirmed
errors larger for some groups
The page says “especially error-prone for some subgroups, including Republicans and Black adults”
-
42
The essay, section 3confirmed
Republicans and leaners 16.1 points
The page says “Rep/Lean Rep ... Average 16.1”
-
43
The essay, section 3confirmed
Black adults 15.1 points
The page says “Black ... Average 15.1”
- 44 confirmed
-
45
The essay, section 3confirmed
AI missed badly on a variety of questions about data centers
The page says “It missed badly on a variety of questions about data centers, from general awareness to views about their impact”
-
46
The essay, section 3confirmed
errors highly subject to factors like unforeseen real-world events
The page says “they are highly subject to factors like unforeseen real-world events or the choice of model used”
-
47
The essay, section 3confirmed
bars: All adults 12.4, Republicans and leaners 16.1, Black adults 15.1
The page says “Total 12.4; Rep/Lean Rep 16.1; Black 15.1”
-
48
The essay, section 3confirmed
Average error, percentage points, Pew's three 2026 surveys
The page says “Average absolute error by group (across three waves)”
-
49
The essay, section 3confirmed
overall average 12.4; some groups further off
The page says “Total 12.4; none had an average error of less than 12 points”
-
50
The essay, section 4confirmed
separate test on a subset of questions comparing GPT-5.1 with Opus 4.6
The page says “used both OpenAI's GPT-5.1 and Anthropic's Claude Opus 4.6 to simulate 6,700 real respondents (one January 2026 questionnaire; 119 questions tested, vs nearly 300 overall)”
-
51
The essay, section 4confirmed
given the same profiles
The page says “otherwise built in exactly the same way”
-
52
The essay, section 4confirmed
89 of 119 questions with ordered answer choices
The page says “89 out of 119 questions tested”
-
53
The essay, section 4confirmed
Opus 11.4, GPT-5.1 13.3
The page says “Opus performed slightly better, with an average absolute error of 11.4 percentage points (vs 13.3 for GPT)”
-
54
The essay, section 4confirmed
humans chose middle option 45% of the time
The page says “Human respondents chose a 'middle' option 45% of the time, on average”
-
55
The essay, section 4confirmed
GPT-5.1 31%
The page says “GPT respondents did so 31% of the time”
-
56
The essay, section 4confirmed
Opus 56%
The page says “Opus respondents did so 56% of the time”
-
57
The essay, section 4confirmed
GPT-5.1 more extreme, Opus more middle-of-the-road
The page says “GPT tended to depict Americans as having more extreme opinions than they actually do”
-
58
The essay, section 4confirmed
GPT-5.1 estimated 100% dissatisfied
The page says “estimating that 100% of Americans are dissatisfied with the way things are going in the country today”
-
59
The essay, section 4confirmed
each synthetic sample painted a very different picture of the public
The page says “each synthetic sample painted a very different picture of the American public on multiple dimensions”
-
60
The essay, section 4confirmed
footnote: GPT-5 nano, lighter-weight, very different estimates from GPT-5.1 and Opus 4.6
The page says “a 'lighter weight' model within the GPT-5 family ... produced very different estimates from both GPT-5.1 and Opus 4.6”
-
61
The essay, section 4confirmed
bars: Real people 45, GPT-5.1 31, Claude Opus 4.6 56 (%)
The page says “45% ... 31% ... 56%”
-
62
The essay, section 4confirmed
Share of answers on Pew's ordered-choice questions
The page says “questions with ordered response categories”
-
63
The essay, section 4confirmed
one model pushed to edges, other to center
The page says “more extreme opinions ... more middle-of-the-road”
-
64
The essay, section 5confirmed
around 28% of questions missed by over 15 points
The page says “The average difference exceeded 15 points on around 28% of questions”
-
65
The essay, section 5confirmed
results highly subject to the choice of model
The page says “highly subject to factors like unforeseen real-world events or the choice of model used”
-
66
The essay, section 5confirmed
the two models erred in opposite directions
The page says “GPT ... more extreme ... Opus ... more middle-of-the-road”
-
67
The essay, section 5confirmed
changes in how data is generated, or future model updates, could produce different patterns
The page says “future updates to the GPT and Opus models themselves could lead to different patterns (preceded by note on changes to how synthetic data is generated)”
-
68
The essay, section 5confirmed
several groups had larger errors than the overall average
The page says “Hispanic 13.9, Black 15.1, Rep/Lean Rep 16.1, Dem/Lean Dem 13.6 vs Total 12.4”
-
69
The essay, section 5confirmed
AI can categorize real open-ended responses or write analysis code
The page says “AI can be used to categorize real responses to open-ended questions or write the code used to analyze the survey results”
-
70
The essay, section 5confirmed
Pew has no current or future plans to use AI models to generate survey results
The page says “has no current or future plans to use AI models to generate survey results”
-
71
The essay, section 6confirmed
terms: synthetic sample, synthetic respondents, digital twins, silicon samples
The page says “"synthetic samples" / "digital twins" (Pew); synthetic respondents (Greenbook); silicon samples (Argyle et al.)”
-
72
The essay, section 6confirmed
four checks each grounded in Pew findings (real-survey comparison, not sure, unpicked options, uniform subgroups, second model)
The page says “nearly ubiquitous ... four times as likely ... not selected by a single AI-generated respondent ... choice of model used”
-
73
The essay, section 6confirmed
use AI where Pew does: on real answers (categorize open-ended, write tabulation code)
The page says “AI can be used to categorize real responses to open-ended questions or write the code used to analyze the survey results”
-
74
The essay, section 6confirmed
checks target uniform groups, missing uncertainty, model-dependence
The page says “each synthetic sample painted a very different picture”
-
75
The essayconfirmed
Pull quote is from the essay's own text (section 3, paragraph 2)
The page says “verbatim in essay; underlying finding: portrayed them as nearly ubiquitous”
-
76
Further reading, item 1confirmed
Further reading: Pew overview title and URL
The page says “Can AI Stand In for Human Survey-Takers? Not Really”
-
77
Further reading, item 2confirmed
Further reading: detailed findings, tables by survey and group
The page says “Average absolute error by group (across three waves) table”
-
78
Further reading, item 3confirmed
Further reading: two-model comparison
The page says “How synthetic polling results change based on the AI model”
-
79
Further reading, item 4confirmed
Further reading: Argyle et al., background from 2023
The page says “Published online 21 February 2023”
The checker's notes
- Window: D=2026-10-09, 45-day window from 2026-08-25. All sources dated on/before D (Pew 9/30, AI Weekly 10/2, Greenbook 8/25 = day 45). Argyle et al. 2023 is labelled background in text and further reading. No 'ahead' block. No compliance, quote-length or first-person issues found.
- Source discrepancy: World Cup actual share is 43% on the cited Pew overview and AI Weekly, 42% on Pew's detailed-findings page.