01
Similarity to human raters
We calculate a 95% confidence interval which indicates an acceptable margin of error around the aggregate friends-and-family rating then test whether RichieBot’s score falls within that interval.
Testing the portrait
How "Richie" is RichieBot?
We are using validated psychological assessments to measure how closely RichieBot’s personality matches the Richie his friends and family knew and loved.
- Friends and family informant reports
- 12
- Validated psychological assessments
- 3
A shared test
Three conditions, assessed with psychological measures
Friends and family fill out psychological assessments about Richie. Both RichieBot and a naïve LLM with no information about Richie complete the same measures. We examine the correspondence between each instance of measurement.
What counts as similar?
For every survey measure, we ask two questions.
02
Closer than the blank control
We compare the distance between each model’s score and the ground truth, then test whether RichieBot is closer than the uninformed control persona.
Results
Assessment-by-assessment findings
Results will be added here when the analysis is complete.
01
Big Five Personality Traits
Personality traits
02
Primal World Beliefs
Beliefs about the world
03
Schwartz Values
Personal values