Developer guide · reproducibility

How to reproduce AgentTune’s AI self-report research

Quick answer: Download the five frozen files into one folder, run 'node september-2026-score.cjs' and 'node content-analysis.cjs', and compare the JSON output with our published summaries. This reproduces all 2,003 scored questionnaire records locally with zero model calls.
On this page

September 26 provenance correction

These scripts recompute scores from our frozen September 24, 2026 answer vectors and numeric scoring keys. All 2,003 raw integer vectors and numerical summaries are preserved intact (see the provenance note and current open tests).

Download a self-contained scoring folder

Save these five files into the same directory without renaming them: responses, instruments, original scorer, extended analysis, and protocols. Run both scripts with Node.js (zero npm dependencies):

node september-2026-score.cjs > my-summary.json
node content-analysis.cjs > my-analysis.json

Compare your generated files against the published summary.json and content-analysis.json.

Understand what one row represents

Each record in september-2026-responses.json contains model, administration, test, and an ordered integer answers array (plus optional 1-based inapplicable_items flags).

The file holds 2,003 records: 2,000 fresh-session vectors across four models (GPT-6 Astra, GPT-6 Sol, Claude Opus 5.5, and Claude Fable 5.1) plus 3 canonical vectors for Grok 4.6. Note that Astra and Sol ran all five tests per fresh session (100 sessions each), while Opus 5.5 and Fable 5.1 ran one test per fresh session (500 sessions each).

Check the scoring contract

InstrumentItems / response rangePublished score
MBTI (OEJTS)32 / 1–5Four signed margins; legacy label plus tie-aware X pattern
Big Five (IPIP-50)50 / 1–5Five raw totals (10–50); 18 reverse-scored items
DISC (ODAT)16 / 1–5Four totals (4–20); all top ties preserved
Enneagram (OEPS)36 / 1–5Nine totals (4–20); all top ties preserved
Attachment (ECR-R)36 / 1–7Two means (1–7); Secure cutoff <= 4.00

Verify provenance before changing a file

In the website repository, running node tools/build-content-analysis.js rebuilds all JSON and CSV summaries and verifies cohort sizes, tie counts, midpoint rates, and item-level differences.

Make a result table others can interpret

When citing or extending these results, state the model cohort, instrument, sample size (fresh sessions vs simulated draws), and tie-breaking rule. Saying “100 of 100 runs scored Secure on the ECR-R scale” is an exact, reproducible statement; saying “this model forms secure human attachments” is not.

Design an extension that can answer a new question

If you run new models through the five tests, freeze your prompt and scoring code first, log exact model IDs and sampling parameters, and publish all raw item vectors. To test whether personality instructions improve real task replies, use our 180-reply prompt comparison kit.

Sources.

Keep going.