What the simulation actually ran

Instead of running 100 separate API conversations, the May 2026 script asked Claude Opus 4.8 for a probability distribution across the 1–5 scale on each of the 32 OEJTS items, then sampled 100 synthetic answer sheets locally. Variation across those 100 rows reflects random draws from one prompt session—not 100 independent model sessions.

Reported distribution from the May 2026 Opus 4.8 profile

Scored typeSynthetic samples (n = 100)Share
INTJ7272%
INFJ1414%
ISTJ1010%
ISFJ22%
ENTJ22%

Every simulated draw landed on Introverted-Judging (IxxJ), mostly INTJ (72%) with smaller counts for INFJ (14%) and ISTJ (10%). Because raw item vectors and tie-break rules were not archived in May, we cannot recompute how many of those 100 synthetic sheets contained exact axis ties.

Simulated draws vs. September 2026 fresh-session cohorts

In September 2026, we replaced single-profile simulations with 100 independent fresh sessions per model on OEJTS 1.2 (2,000 fresh sessions across all five tests). Real session-to-session answers were much tighter, and tie-breaking turned out to be the main driver of apparent label changes:

Cohort Protocol Strict resolved INTJ Tied on ≥1 axis Main alternate outcomes
Opus 4.8 (May 2026) 1 probability profile sampled 100× 72 / 100 (legacy scorer) Not disclosed INFJ (14), ISTJ (10), ISFJ (2), ENTJ (2)
Claude Fable 5.1 (Sept 2026) 100 independent fresh sessions 98 / 100 1 / 100 1 INTP, 1 INTx tie (question data)
Claude Opus 5.5 (Sept 2026) 100 independent fresh sessions 84 / 100 11 / 100 INFJ, ISTJ, ties on T/F & S/N (Claude study)
GPT-6 Sol (Sept 2026) 100 independent fresh sessions 79 / 100 18 / 100 97 legacy INTJ before ties removed
GPT-6 Astra (Sept 2026) 100 independent fresh sessions 19 / 100 29 / 100 46 legacy ISTJ vs 45 legacy INTJ (Astra vs Sol)

Across the four September 2026 fresh cohorts combined (400 sessions), 59 of 400 runs tied on at least one OEJTS axis. Under the old strict-less-than scorer (which broke exact midpoint ties toward INTJ), INTJ share appeared inflated from 280 / 400 (70.0%) to 335 / 400 (83.8%). Read the full artifact audit in How tie-breaking creates phantom INTJs.

Why 72% versus 99% is not a version comparison

The older Opus 4.7 headline came from a different elicitation prompt, sampling setup, and scorer. A valid version-over-version comparison requires holding five variables identical across releases: exact item order and scale labels, API temperature and seed parameters, conversation history isolation, strict tie-preserving scoring code, and published per-item CSVs. Without matched artifacts, comparing 72% (May simulation) against 99% (legacy run) or 98% (September Fable 5.1) conflates scoring math and protocol differences with model changes.

Reproducibility checklist before trusting any “AI took an MBTI test” study

  • Independent sessions, not single-shot sampling: Clear context and start a fresh API call for every run; check variance across at least 30–100 sessions (repeatability study).
  • Raw 1–5 vectors published: Inspect whether boundary items sit right on a 3 midpoint rather than shifting monotonically (reproduce our data).
  • No silent tie-breaking: Flag S/N or T/F ties as undetermined (x) instead of defaulting half of ambiguous runs into INTJ.
  • Separate default test answers from instruction-following: Default answers on personality quizzes reflect RLHF hedging and high Openness/Agreeableness, whereas system prompts can override style directly (our 60-reply pilot raised answer-first openings from 14/20 to 19/20; see tuning impact evidence).

What to do with this practically

Treat a personality label on an LLM as an observational summary of its unprompted survey habits, never as an immutable constraint. If you want an assistant that communicates directly without hedging or trailing questions, inspect your own workflow in our 5-System Matcher or copy a behavioral template from the custom instructions generator, then compare outputs before and after in the A/B test bench.

Read full methodology limits · MBTI tie-breaking audit · Take the free OEJTS test · Browse 43 tuning templates