Posts that defy neater categorisation
Everyone loves the Townscaper grid. It’s got a a nice organic look, while still being quadrilateral tiles, somewhat regular, and reasonably easy to implement. But it has some annoyances. I’ve finally managed to find my own design of grid that has a very similar look, but fixes these problems.
Continue reading
I got a good comment on my previous article about implementing the Game of Life in CUDA pointing out that I was leaving a lot of performance at the table by only considering a single step at once.
Their point was that my implementations were bound by the speed of DRAM. An A40 can send 696 GB/s from DRAM memory to the cores, and my setup required sending at least one bit in each direction per-cell per-step, which worked out at 1.4ms.
But DRAM is the slowest memory on a graphics card. The L1 cache is hosted inside each Streaming Multiprocessor, much closer to where the calculations can occur. It’s tiny, but has an incredible bandwidth – potentialy 67 TB/s. Fully utilizing this is nigh impossible, but even with mediocre utilization we’d do far better than before.
Continue readingA couple of people have mentioned to me: “we need more fiction examples of positive AI superintelligence – utopias like the Culture novels”. And they’re right, AI can be tremendously positive, and some beacons lit into the future could help make that come around.
But one of my hobbies is “oppositional reading” – deliberately interpreting novels counter to the obvious / intended reading. And it’s not so clear to me that the Culture is all it is cracked up to be.
Continue reading
Let’s look at implementing Conway’s Game of Life using a graphics card. I want to experiment with different libraries and techniques, to see how to get the best performance. I’m going to start simple, and get increasingly complex as we dive in.
The Game Of Life is a simple cellular automata, so should be really amenable to GPU acceleration. The rules are simple: Each cell in the 2d grid is either alive or dead. At each step, count the alive neighbours of the cell (including diagonals). If the cell is alive, it remains alive if 2 or 3 neighbours are alive. Otherwise it dies. If the cell is dead, it comes ot life if exactly 3 neighbours are alive. These simple rules cause an amazing amount of emergent complexity which has been written about copiously elsewhere.
For simplicity, I’ll only consider N×N grids, and skip calculations on the boundary. I ran everything with an A40, and I’ll benchmark performance at N=216 . For now, we’ll store each cell as 1 byte so this array is which equates to 4 GB of data.
All code is shared in the GitHub repo.
Continue reading
In procedural generation, the absolute simplest, most common technique is randomly picking an item from a list. More often than not, it is a weighted random choice, where each item is selected with a frequency proportional to its “weight”, a numerical value that can be tweaked by the designer.
def random_choice(items: list, weights: list[float]):
total_weight = sum(weights)
random_value = random.random() * total_weight
# Find the item that corresponds to the random number
for i, weight in enumerate(weights):
random_value -= weight
if random_value <= 0:
return items[i]I wanted to share a technique from the Machine Learning community that is simple enhancement to this routine that gives a lot of convenience over tweaking weights directly, called temperature.
Continue reading
Last week, I saw a talk on Vuntra City, a procedurally generated city with a fully explorable city. Developer Larissa Davidova explained that she settled on using Recursive Subdivision for the city blocks, as she wanted some level of organicness, while still only having to deal with rectangles. But she didn’t like having indefinitely long roads that cause implausible sightlines.
One way Vuntra City handles this is by subdividing a rectangle into 5 blocks, a pattern I called “whirl” in my previous article on recursive subdivision. You can see that it has no internal roads that stretch across the entire map.
But Larissa’s talk got me thinking. The whirl pattern is interesting because it cannot be made from simple cuts. What other ways of subdividing a rectangle into smaller rectangles1, are out there?
Continue readingI’ve been working on some AI Safety research. It’s kinda dense for a blog, so I’m hosting elsewhere.
It’s investigation into how ML models do boolean at the most fundamental level. Under an assumption of feature sparsity, which is common for large models, certain patterns appear.
A user requested adding wrapping hex grids to Sylves. That is, a grid that is a collection of hexagons, and the collection itself is also hexagonal shaped. When you exit from one side, you re-enter on the opposite side.
The maths for this turned out to be surprisingly fiddly, and not listed by RedBlobGames, so I thought I’d share it here.
Continue reading
For many years, I always gifted people books for birthdays. It took a long time to realize that most people see them more as obligations than anything. I guess I was trying to recapture some of the magic they have always brought me.
The following books are not necessarily the best ones I’ve read or even the ones that have absorbed deepest into my personality. But they’ve had a distinct impact. Let’s begin.
Continue reading
I went to some lectures on the future of science in games recently, and the keynote speaker was Tommy Thompson, an well-known AI expert in the game dev space.
Of course, by AI, he didn’t mean the modern sort that dominates the news. His focus is AI for games, which is algorithmic and rarely involves any ML component. Still, he spoke about the challenges the industry faces regarding Image Generators, LLMs and so on. He specifically called LLMs “stochastic parrots”, which I found disappointing. Imho it’s an incredibly misleading model of what LLMs are capable of and is usually deployed to downplay their abilities and belittle them. But it’s a common view, particularly in creative industries.
So what is a better model? It’s clear that they are not that smart in most ways we consider important, but they do have some interesting capabilities. Here’s model I use that I feel give a better intuition for what they can and cannot do.
Continue reading