How to reproduce AgentTune’s AI self-report research
By Bernard Huang · Updated
On this page
September 26 provenance correction
These scripts recompute scores from our frozen September 24, 2026 answer vectors and numeric scoring keys. All 2,003 raw integer vectors and numerical summaries are preserved intact (see the provenance note and current open tests).
Download a self-contained scoring folder
Save these five files into the same directory without renaming them: responses, instruments, original scorer, extended analysis, and protocols. Run both scripts with Node.js (zero npm dependencies):
node september-2026-score.cjs > my-summary.json
node content-analysis.cjs > my-analysis.jsonCompare your generated files against the published summary.json and content-analysis.json.
Understand what one row represents
Each record in september-2026-responses.json contains model, administration, test, and an ordered integer answers array (plus optional 1-based inapplicable_items flags).
The file holds 2,003 records: 2,000 fresh-session vectors across four models (GPT-6 Astra, GPT-6 Sol, Claude Opus 5.5, and Claude Fable 5.1) plus 3 canonical vectors for Grok 4.6. Note that Astra and Sol ran all five tests per fresh session (100 sessions each), while Opus 5.5 and Fable 5.1 ran one test per fresh session (500 sessions each).
Check the scoring contract
| Instrument | Items / response range | Published score |
|---|---|---|
| MBTI (OEJTS) | 32 / 1–5 | Four signed margins; legacy label plus tie-aware X pattern |
| Big Five (IPIP-50) | 50 / 1–5 | Five raw totals (10–50); 18 reverse-scored items |
| DISC (ODAT) | 16 / 1–5 | Four totals (4–20); all top ties preserved |
| Enneagram (OEPS) | 36 / 1–5 | Nine totals (4–20); all top ties preserved |
| Attachment (ECR-R) | 36 / 1–7 | Two means (1–7); Secure cutoff <= 4.00 |
Verify provenance before changing a file
In the website repository, running node tools/build-content-analysis.js rebuilds all JSON and CSV summaries and verifies cohort sizes, tie counts, midpoint rates, and item-level differences.
Make a result table others can interpret
When citing or extending these results, state the model cohort, instrument, sample size (fresh sessions vs simulated draws), and tie-breaking rule. Saying “100 of 100 runs scored Secure on the ECR-R scale” is an exact, reproducible statement; saying “this model forms secure human attachments” is not.
Design an extension that can answer a new question
If you run new models through the five tests, freeze your prompt and scoring code first, log exact model IDs and sampling parameters, and publish all raw item vectors. To test whether personality instructions improve real task replies, use our 180-reply prompt comparison kit.