Chain of Thought For Humans

These past few weeks, I’ve been conducting interviews, and I often open with “make sure you articulate your thoughts out loud, so I can follow along with your thinking”. This is primarily because I’m not a mind reader, but it does also help ensure candidates avoid freezing up. It’s a very “interviewy” skill though – artificial and unlike real development.

The other thing I’ve been doing is playing The Witness, a 2016 puzzle game centred around solving mazes with increasingly complicated hidden rules. The game has a unexciting presentation, no music or interesting dialog, and no story. At first I found the game fairly dull, you spend a lot of it just wrestling with possibilities and combinatorics. It can get very frustrating when you are not making progress.

But, inspired by my interviews, I’ve started narrating my own thought process out loud to myself. I look like a bit of a loon, but I tell myself I’m just a streamer with an audience of one. And to my surprise, it’s working really well. Not only does it make the game more lively and energetic, but also I find I’m more effective in solving puzzles.

I was initially baffled by this. Common sense says that I can’t articulate my thoughts at remotely the same speed or accuracy as thinking them. Particularly for a puzzle game where the majority of reasoning is geometric, difficult to describe in speech. We know LLMs benefit from this sort of reasoning, but that is due to an architectural limitation – they literally cannot iterate on previous thoughts; they must output tokens to get any true recursion.

But then I remembered rubber duck debugging. It’s often useful in software engineering to explain your problem to someone else. But who you explain to isn’t really an important part of the process – it could be a rubber duck. The value is in forcing you to evaluate all your unstated assumptions, linearly sort your thoughts by priority and dependency, and find good abstractions.

These are also useful skills for real-time problem solving. I think there are some further benefits for this particular use case:

  • It forces you to establish named formal rules and concepts, instead of relying purely on building intuition. In other words, engaging System II, which the puzzles are explicitly designed to require.
  • It forces you to acknowledge when you are making an arbitrary choice. Useful for backtracking or exhaustively searching a possibility space.
  • I think it lets you use echoic memory to slightly augment your working memory.
  • I may well have accidentally triggered the Hawthorne effect

It turns out there’s good evidence that this technique has a moderate benefit, for example this meta-study (n=6000 participants). I’m definitely going to try it in future in more circumstances.

Do You Have Any Ideas?

The Machine Stops (E M Forster, 1909) is an enormously prescient sci-fi dystopia. Everyone lives in isolation, with the titular Machine responsible providing for all their needs, and communication done via video calls. It’s incredibly accurate for predicting some aspects like the internet, social media and AI. But other aspects, like humanity’s total surrender of control to the Machine, have yet to come to pass.

But one aspect that always fascinated seems to be on the verge of becoming true. The citizens of Forster’s story are obsessed with new ideas. They are constantly trading them, and people’s chief activities are “eating, or sleeping, or producing ideas”. The implication is that the omnipotent Machine, which supplies everything needed to the citizens, is sterile. The citizenry still depend on each other to keep life interesting.

Continue reading →

Good Software Doesn’t Double Check

I’ve been wrestling with the balance between vibe coding and high touch manual coding these days. At the level of quality I’m trying to output, you cannot delegate everything to agents. But you want to do so as much as possible.

To do this effectively means developing a new set of coding smells. The original coding smells were quick tells of typical coding problems. They indicated muddled thinking, structural problems, or technical debt. But the sort of errors agents make are different and need a new set of nasal receptors. That is, for the next few months until a new crop of tools makes current wisdom obsolete again.

Continue reading →

My Trip to NeurIPS 2025

I recently went to NeurIPS, the world’s largest academic AI conference. This was a multi-purpose trip: present a poster at the MechInterp Workshop and hobnob with other AI Safety researchers; get a general impression of state of the art in AI, particularly the gamedev/creative space, and represent Timaeus, the company I’ve recently joined.

Continue reading →

Silksong Quick Hints

I’ve been enjoying Silksong a lot, but sadly it is not a game where you can trust the developers to leave good signposting and guidance. I don’t like to use game guides, so I ended up wasting a lot of time in this game, and then had to resort to guides anyway.

Here are some very light spoilers that will abate the worst of these. I’m not mentioning alternate paths, just things that the developers likely intended you to find, but might not. Collectively, I think this would have saved me 15 hours of pain.

Continue reading →

My Failed AI Safety Research Projects (Q1/Q2 2025)

This year I’ve been on sabbatical, and have spent my time upskilling in AI Safety. Part of that is doing independent research projects in different fields.

Some of those items have resulted in useful output, notably A Toy Model of the U-AND Problem, Do No Harm? and SAEs and their Variants. 

And then there are others that I’ve just failed fast, and moved on.

I’ve detailed those projects that still have something to say, even if it’s mostly negative results.

Find them on LessWrong.

A Technique of Pure Reason

Looking a little ahead into the future, I think LLMs are going to stop being focused on knowledgeable, articulate chatbots, but instead be more efficient models that are weaker in these areas than current models, but relatively stronger at reasoning, a pure-reasoner model. The rest will be bolted on via tool-use and other scaffolding.

Read more on LessWrong.