Beating the Voight-Kampff Test and Becoming Authentic Through the Process

I used Claude to help write the LLM explanation posts.

Some of it was research, most posts had at least a quick spelling and grammar check, and more technical piece on layers had a kind of ghostwritten first draft resulting from close discussion, direction and iteration.

The landscape metaphor is my own conception, but I drilled into it over a few conversations to develop the finer details (like the layers and attention head part) and make sure I was being accurate and not misleading - I thought it would run out of steam relatively quickly. I didn't actually expect the metaphor to map quite as well as it does.

The main intention was to visualize the mathematical operation of an LLM chat as exploring the solution space of language relations (represented in 3d terrain as horizontal concept space and frequency of concept height) and returning the findings. I wanted to tilt towards the idea that some - or even substantial parts - of "intelligence" are built into the structure of langugae itself. It's a demonstration of cognitive offloading as described by Andy Clark and David Chalmers extended mind theory.

Without Claude, this would have required weeks of research into mathematics I'd long forgotten. In addition, it allowed me to get my head around whats going on with layers and the differences between models through discussion. I set Claude up with a Socratic dialogue prompt and iterated through multiple checks for accuracy.

All the text (for all posts) only took a few hours, from initial “Hey Claude, what if you think of LLMs in this way, does it make sense?” to a draft good enough for the original purpose (extended notes for film studies students). I then spent almost as long cleaning up repetition and language from fast typing.

Now, for sure there is an issue that Claude may be leading me up the garden path and I have unwittingly misunderstood parts. But I suspect not. I tried to push Claude to explain the extended metaphors problems across multiple instances and it seemed relatively happy. I re-read some of the old A.I. philosophy and theoretical bits on the maths. I think the risk of me being substantially incorrect is lower than if I asked a human research assistant to help provide me with material - unless they were already an expert.

I also feel insulated a little by the fact that this whole exercise is a vast simplification to provide a mental framework for something that is extremely hard to get your head into. It’s not an academic paper, it’s a way of trying to get people to think about the tool differently. Mathematical operations are nothing like Mars explorer robots except the are notionally both “exploring” something. That said, if someone had told me the rover story as an undergraduate, I’d probably have been more interested in the mathematics.

The marginally unsettling part is that I'm struggling to cleanly separate which of the later technical descriptions are mine and which were Claude’s. Claude needed pushing to keep things simple, on the technical elements I was ask Claude to summarise the mathematics and the tech, then provide a metaphor and I would streamline the metaphor or change it entirely.

For example, Claude suggested new rovers with new instruments for each layer. But visually this is weird, why would you drop in the same rovers in the same place. So I changed it to different type of robots get dropped into the same location. This helps visualise why you might do this (different locomotion ability), and nudges you to see that each layer is looking at a different type of exploration of linguistic feature.

Clark and Chalmers point, however, is that trying to make the separation between Claude and myself is inherently problematic. The “Mind” is inside and outside the body - it’s not confined to the brain - it’s the interaction that counts.

Never the less, out of curiosity I asked Claude to assess the amount of AI use in the articles. In particular, Claude flagged my organizational structure as suspicious. Ironically that was entirely mine. Claude told me the first draft was confusing and offered and alternative which wasn't very good. Claude is a better critic than a writer. So I restructured myself. Mostly by adding better headings and doing a little shuffle. Claude went from "this is terrible" to "this is brilliant" (over praise is part of the Anthropic house style, you do need to be careful about thinking your insights are great when you're just kind of average - it's hard to prompt it out).

And here is the paradox: structure and precision in writing look like AI fingerprints. Yet Claude told me that the deep technical details section of the extended Mars rover were definitely human. Nope. Some of that technical stuff is lightly edited Claude.

That said, I think the technical details are more authentic and accurate precisely because Claude helped write them. Claude can explore all the technical data in a way no human could (or at least very few).

I read a lot of Claude's output which does lead to the uncanny thought that Claude is likely to shape how I use language. Especially since it is used at the speed of an instant message conversation rather than a novel. We incorporate what we read into ourselves, text is not so wildly different from food. The most visible version is in the way we pick up accents -- or ingest and remix ideas from friends, mass media and collaborators. However, you do have power over your information intake, and it’s a good idea to curate high quality sources and ignore everything else. The boundary between reading writing speaking and thinking has always been permeable: more human than human as they say.