Is it possible to write good code without reading it? Right now a debate is raging between Andrew Kelley and Jarred Sumner about whether tests are "sufficient to catch bugs in 1 million lines of unreviewed slop."
The standard view in the community is that "yes, you can do it if you test it well." But I think that view vastly oversimplifies the challenge. Testing a million-line codebase is harder than it seems. So in this post we will crack open the problem with some specifics. We will take a look at some AI-written code to see one of the core challenges facing AI-driven development.
On this post: unlike the bun port, here you can play along! Enter the NetHack Teleport Coding Challenge by pointing your coding agent at the repo and trying to get it to score some points. The contest, open to anybody, asks coding agents to port the recently-released version of NetHack from C to JavaScript. We will be looking at two specific contestants currently in the challenge.
NetHack 5.0 is 442,901 lines of C and Lua accumulated over more than four decades. A serious port will contain more code than any one person can carefully review. Is it possible to do high-quality software engineering without reading the code?
The lack of code review is an emblematic AI interpretability problem.
In my research on neural-network internals, the interpretability problem arises because a network’s calculations are too numerous to inspect or understand directly. Interpretability seems like it should be easier in a regular programming task: here there is no neural network inside the finished program, and every line is ordinary readable JavaScript, the world's most widely known programming language. Yet in the Teleport contest, traditional code becomes opaque for the same reason as a big neural network does: there is simply too much of it.
To understand the problem it is instructive to look at a single game of the contest and dive into two of the top contestants' play.
Continue reading "Hunting Zombies"