RichieBot

From evidence to evaluation

Development

Descriptions from friends and family and personal materials form the evidence used to build RichieBot’s personality. Quantitative survey data is used to evaluate how closely the resulting personality aligns with Richie.

The evidence base

What kind of data goes into RichieBot’s personality

Two kinds of source material provide the stories, language, ideas, and patterns used to construct the personality. Surveys provide a separate measure for evaluating it.

01

Descriptions from friends and family

Interviews and written descriptions from Richie’s friends and family.

02

Personal archive

Articles, notes, and other materials Richie chose to keep over time.

The full arc

From source material to tested model

A sequence of language-model passes makes the sources usable, interprets them, finds recurring patterns, and turns those patterns into behavior. The survey comparison comes last.

Prepare

Make the sources legible

  1. Structure the sources

    Convert images and PDFs into structured text that language models can work with.

  2. Summarize each source

    Use a language model to capture the meaning of each interview, description, article, note, or document.

Interpret & synthesize

Build a coherent portrait

  1. Apply psychological lens

    Interpret source data through the lens of each framework.

  2. Find recurring patterns

    Aggregate across sources to identify themes and characteristics that recur.

  3. Assemble the soul file

    Turn those themes into instructions for how a language model should behave like Richie.

Evaluate

Test the resulting model

  1. Compare survey profiles

    Have RichieBot complete the same surveys as friends and family, then compare how similar the profiles are.

Friends & family descriptions + archiveSoul fileSurvey alignment