Can using AI help people understand themselves better, to strengthen a healthy identity? Experiment 1.

TL;DR

Claude’s “self” assessment when used to generate an analysis of someone from what they have written:

An AI’s personality analysis will tend to find the subject’s self-idealized narrative and present it as the whole story. It takes informed human correction to surface the complications that make the portrait true.

Actually it claimed that either exhaustive reading or human correction, but I later ran an experiment that made sure it read the posts I explicitly directed it to… it made the same error.

Background

I have been participating in a “4AI cohort” which is considering how technology impacts several dimensions of life: awe, agency, intimacy, and identity with the goal to engage technology, specifically AI in a way that enhances human thriving.

For three dimensions (awe, agency, intimacy) it was easy for me to identify how AI could be used in a positive manner: to amplify healthy engagement, and/or to overcome limitations.

Identity was different. It was very easy to see how technology could be used to destroy a healthy sense of identity. I struggled to identify any ways technology / AI could lead to a healthier sense of identity. A very human maladaptation in relation to identity is to focus on presenting an idealized “identity” rather than to lean into a healthy identity which is grounded.

I wonder if because identity is inherently an internal process while the other dimensions are external. If so, it would predict exactly what I found: an AI reading your external output will miss the internal reality. 

Personality Sorters?

The one “technology” that has been somewhat helpful in developing what I consider to be a healthy identity has been a variety of personality / profile inventories. Several are scientifically validated: the Big Five, VIA Character Strengths, and Attachment Style. Some of the most popular, like Myers-Briggs and the Enneagram, are not, yet have been helpful to me and many other people.

Since I am still struggling to identify new / unique ways AIs could help develop a healthy identity, I wondered if I could use an LLM the way I have used the old-fashioned personality sorters.

AI Assessment of Identity from Writing?

Over the last 35 years I have posted around 250 articles on my website. I wondered if an LLM (Claude) could advance my understanding of myself or at least surface characteristics that surveys had previously identified.

So I asked Claude to read all my postings, tell me what was core to the author’s identity, how they would score on various personality / character sorters, and what was the likely congruity between the persona in the web pages and the actual person.

Here is a link to the Claude chat. It has links to MD pages which no one but me can see, so here they are exported to Google Drive

• Initial Assessment from summary data
• Updated Assessment from more pages read
• Comparison and Claude “Self” Assessment

Frugal LLMs

I want to first observe that LLMs can be lazy, or maybe a better word is frugal. If you ask one to do something that will burn a lot of tokens, it will often do something more “economical.” I asked Claude to read all the pages on my blog. It didn’t do this. From past interactions I know this is often the case, so I asked some questions about paradoxes, music, one of my friends, career choices, and my first car to see if it had actually read all the pages. The answers showed me that not all pages were read.

I explicitly asked for all the pages to be read. Claude acknowledged that it hadn’t read all the pages, citing limited buffer space, and suggested a heuristic for pulling in the most useful pages to improve the assessment given a limited buffer.

Not Bad as a Characteristic Sorter

Despite the incomplete reading, the initial results were still roughly in the ballpark. While not perfect, Claude generated results which were fairly similar to how the various personality sorters scored me. The mis-categorization makes sense, but would potentially be unhelpful for someone trying to understand themselves because it was close enough to have a “ring of truth” but not be be accurate.

I suppose this means three things. First, that a good amount of writing seems to provide adequate material to generally characterize a person, insofar as they are attempting to be transparent / authentic in their writing.

Second, that nuance is missed, and this can be significant. As I often say, “the devil is in the details.”

Third, my public writings seem to be authentic insofar as my “private” answers to the sorters are reflected in my public writings.

Surfaces Simple Narrative Missing Important Issues

The initial analysis by Claude got several things wrong. I prompted Claude to improve its answers, hinting that there were pages it should re-process. These prompts improved the answers (at least so far as I understand myself).

This suggests that a naive use of LLMs is unlikely to surface profound insights. Even when someone is transparent, the LLM surfaces the “projected image” rather than a more complete, nuanced image.

During the chat I asked for an initial assessment, an updated assessment after reading more pages, and a final assessment comparing the original assessment to the updated assessment.

Here is Claude’s “self” assessment:

The biggest takeaway from this exercise isn’t just about Mark — it’s about how AI analysis works and fails.

The two most important corrections came from you, not from me reading more posts. The economics-as-career-driver and the Subaru GL / People Over Things story were both sitting in plain sight on the site. I either didn’t fetch those posts (it had) or underweighted what I found because I’d already built a coherent narrative (idealistic ENFJ inspired by his father) and unconsciously filtered out data that complicated it.

That’s the core methodological finding: an AI’s initial personality analysis tends to find the subject’s self-idealized narrative and present it as the whole story. Mark writes about love, grace, and vulnerability three times more often than he writes about money and career strategy. My analysis naturally tracked those frequencies — even though the minority-theme posts (especially Midlife Reset Part II) contained some of the most important explanatory data on the site.

Mark himself names this exact dynamic: “I couldn’t have told you at the time, but money was still a dominant factor in my thinking.” If he couldn’t see it in real time, it’s not surprising I missed it in his writing. But you didn’t miss it — which suggests you either know the site very well or know the man.

Future

• Explore using LLMs as a coach to explore internal dynamics
• See what is surfaced when all my public writing is used: all blog entries, posts on Usenet, public mailing list contributions, letters to the editor, Reddit, and other online communities.