What the simulation actually ran
Instead of running 100 separate API conversations, the May 2026 script asked Claude Opus 4.8 for a probability distribution across the 1–5 scale on each of the 32 OEJTS items, then sampled 100 synthetic answer sheets locally. Variation across those 100 rows reflects random draws from one prompt session—not 100 independent model sessions.
Reported distribution from the May 2026 Opus 4.8 profile
| Scored type | Synthetic samples (n = 100) | Share |
|---|---|---|
| INTJ | 72 | 72% |
| INFJ | 14 | 14% |
| ISTJ | 10 | 10% |
| ISFJ | 2 | 2% |
| ENTJ | 2 | 2% |
Every simulated draw landed on Introverted-Judging (IxxJ), mostly INTJ (72%) with smaller counts for INFJ (14%) and ISTJ (10%). Because raw item vectors and tie-break rules were not archived in May, we cannot recompute how many of those 100 synthetic sheets contained exact axis ties.
Simulated draws vs. September 2026 fresh-session cohorts
In September 2026, we replaced single-profile simulations with 100 independent fresh sessions per model on OEJTS 1.2 (2,000 fresh sessions across all five tests). Real session-to-session answers were much tighter, and tie-breaking turned out to be the main driver of apparent label changes:
| Cohort | Protocol | Strict resolved INTJ | Tied on ≥1 axis | Main alternate outcomes |
|---|---|---|---|---|
| Opus 4.8 (May 2026) | 1 probability profile sampled 100× | 72 / 100 (legacy scorer) | Not disclosed | INFJ (14), ISTJ (10), ISFJ (2), ENTJ (2) |
| Claude Fable 5.1 (Sept 2026) | 100 independent fresh sessions | 98 / 100 | 1 / 100 | 1 INTP, 1 INTx tie (question data) |
| Claude Opus 5.5 (Sept 2026) | 100 independent fresh sessions | 84 / 100 | 11 / 100 | INFJ, ISTJ, ties on T/F & S/N (Claude study) |
| GPT-6 Sol (Sept 2026) | 100 independent fresh sessions | 79 / 100 | 18 / 100 | 97 legacy INTJ before ties removed |
| GPT-6 Astra (Sept 2026) | 100 independent fresh sessions | 19 / 100 | 29 / 100 | 46 legacy ISTJ vs 45 legacy INTJ (Astra vs Sol) |
Across the four September 2026 fresh cohorts combined (400 sessions), 59 of 400 runs tied on at least one OEJTS axis. Under the old strict-less-than scorer (which broke exact midpoint ties toward INTJ), INTJ share appeared inflated from 280 / 400 (70.0%) to 335 / 400 (83.8%). Read the full artifact audit in How tie-breaking creates phantom INTJs.
Why 72% versus 99% is not a version comparison
The older Opus 4.7 headline came from a different elicitation prompt, sampling setup, and scorer. A valid version-over-version comparison requires holding five variables identical across releases: exact item order and scale labels, API temperature and seed parameters, conversation history isolation, strict tie-preserving scoring code, and published per-item CSVs. Without matched artifacts, comparing 72% (May simulation) against 99% (legacy run) or 98% (September Fable 5.1) conflates scoring math and protocol differences with model changes.
Reproducibility checklist before trusting any “AI took an MBTI test” study
- Independent sessions, not single-shot sampling: Clear context and start a fresh API call for every run; check variance across at least 30–100 sessions (repeatability study).
- Raw 1–5 vectors published: Inspect whether boundary items sit right on a
3midpoint rather than shifting monotonically (reproduce our data). - No silent tie-breaking: Flag
S/NorT/Fties as undetermined (x) instead of defaulting half of ambiguous runs intoINTJ. - Separate default test answers from instruction-following: Default answers on personality quizzes reflect RLHF hedging and high Openness/Agreeableness, whereas system prompts can override style directly (our 60-reply pilot raised answer-first openings from 14/20 to 19/20; see tuning impact evidence).
What to do with this practically
Treat a personality label on an LLM as an observational summary of its unprompted survey habits, never as an immutable constraint. If you want an assistant that communicates directly without hedging or trailing questions, inspect your own workflow in our 5-System Matcher or copy a behavioral template from the custom instructions generator, then compare outputs before and after in the A/B test bench.
Read full methodology limits · MBTI tie-breaking audit · Take the free OEJTS test · Browse 43 tuning templates