I post trained a model on Demoscenes

Post training a model to explore Demoscenes On the grammar of demoscenes and the possibilities of unfinished code Lately, I have been fascinated by the world of demoscening and sizecoding and eventually I came across HellMood’s remarkable MS DOS demo “Memories”. Whilst reading through his writeup, I became interested in how the author had arrived at some of its visual effects. I was struggling to create any of my own, so I hoped to squeeze out a little bit of inspiration out of that. I noticed that these effects could be traced back to earlier experiments consisting of alternative implementations and several ideas borrowed from previous demos. I began wondering how many other directions the same underlying ideas might have taken. I decided to start with a few existing demos, the simplest ones of all, and modify them character by character to observe how those changes can affect the visual feel. I came across Dwitter.net, which is like Twitter but for 140 character JavaScript demos. They already supported “remix”-ing any dweet so that we can build on top of them. Co incidentally, the most liked Dweet on the site shared the same effect as the parallax checkerboard effect from Memories! Digging a little deeper, I noticed some remixes shared substantial portions of their source code, differing by only a few characters, whilst others achieved similar visual effects through entirely different implementations. Here is an example, I traced back d/36555 from d/367547 which in turn is derived from d/36467. The gray underlined substrings show what is common across all of them. Blue shows what’s common in between the first two, and green shows for the next couple. In linguistics, language is productive i.e. a finite vocabulary and a finite set of grammatical rules permit an enormous number of different expressions. Programming languages share this property too. Now let’s consider the aforementioned Dweets. They contain overlapping substrings, but their rendered results differ. Suppose you take a 140 character program, and remove its final 30 characters. Even with a hypothetical alphabet of just 64 character, there are almost 1.5 x 10^54 possible completion strings that can be formed. Almost all will be syntactically invalid or produce uninteresting results. Nevertheless, the number of possibilities is astronomical. It’s not humanly possible to enumerate all of them exhaustively and it would be foolish to to generate and test all possible strings. However, the continuations of a partial program are not equally plausible. Given a partial program, certain characters and expressions are considerably more likely to follow than others. This is pretty much how autoregressive language models work. I wondered whether I could use such a model to sample plausible continuations of unfinished Dweets and visualize the different directions each program could have taken. And then, I could surgically delete substrings or mix-and-match them to my own aesthetic taste. I had also grown rather frustrated with my own ability to “finish” a demo after starting one, so having an autocomplete that understood the peculiar language of Dweeting rather looked like an appealing prospect to me. Around the same time, I came across Mithil Vakde’s tweet (you might have seen him on HackerNews recently). That gave me another excuse to experiment - could I inject my own aesthetic taste as a prior into my autocomplete model? That’d be really amazing. I understand that no one can write a function that determines whether a demo is beautiful, but people can express their preferences by comparing examples, right? With that in mind, I decided to put my GitHub Sponsors payout towards renting a GPU and post training a model of my own. The first hurdle was finding a suitable dataset. I initially experimented with generating synthetic Dweets using Claude Sonnet 5.5 but found them not at all aesthetically pleasant. Fortunately, Dwitter.net provides an API, which I used to build a modest dataset of 25k Dweets to be used for training along with ~1.3k more as a validation set. I used Qwen2.5-Code-7B as the base model. It struggled to complete most of the Dweets I tested it on, which was understandable given how peculiar their syntax is. Dweets make extensive use of code golfing tricks and mathematical shorthand, which are rather unusual compared with the code a general purpose coding model typically encounters. I decided to try continued pretraining (CPT) on the corpus, hoping to shift the model’s probability distribution towards the pecular language of dweeting. To see whether it had worked, I measured the surprisal of individual tokens before and after training. Surprisal is simply the negative log probability a model assigns to a token given everything preceding it. The less likely a token is, the more surprising it is to the model. In the figure below, darker red indicates greater surprisal. After CPT, much of the code becomes noticeably less surprising (particularly the unfamiliar programming idioms). Across 30 held out dweets, perplexity fell from 28.63 to 8.57, which means that the model had become substantially better at predicting the structure of these tiny weird programs. Although the generated programs were more consistent, they still lacked a certain “charm”. Fortunately, Dwitter has an “awesomeness” score for every Dweet. I decided to use it to construct a curriculum. I ranked the top 5k dweets by their “awesomeness” scores and constructed a curriculum that gradually favoured the more appreciated programs. Initially, every dweet had an equal probability of being sampled. Over 400 training steps, I progressively increased the weight assigned to higher ranked programs, until the top 1000 accounted for roughly 67% of the sampling probability. | Rank bucket | Step 1 | Step 100 | Step 200 | Step 300 | Step 400 | |---|---|---|---|---|---| | 1-1000 | 20.00% | 35.89% | 48.74% | 59.01% | 67.23% | | 1001-2000 | 20.00% | 27.97% | 29.60% | 28.01% | 24.99% | | 2001-3000 | 20.00% | 20.03% | 15.23% | 10.41% | 6.75% | | 3001-4000 | 20.00% | 12.06% | 5.63% | 2.41% | 0.99% | | 4001-5000 | 20.00% | 4.05% | 0.81% | 0.16% | 0.03% | This turned out to be a breakthrough! The generated continuations became much more visually interesting. Interestingly, the improvements in token likelihood extended even to Dweets that had never been sampled during curriculum training. Perplexity fell from 7.42 to 5.65, although the gains were spread fairly evenly across awesomeness ranks. Across 7000 Dweets, curriculum training improved token likelihood across all awesomeness ranks, including programs never sampled during training. The gains were particularly pronounced for programs the original model found difficult. I now had a model that could produce rather beautiful continuations. I wondered whether I could take this further by incorporating my own aesthetic preferences. In mid-September, I came across Surya’s article on Training AI to Paint with Code, where they had used RL to produce watercolor paintings using p5js. They experimented with different reward functions before settling on pairwise comparsions against a collection of hand rated paintings. I remembered that idea and thought of trying something similar with Demoscener (it doesn’t feel alright to call it “the model” repeatedly). I opted to do RLHF using GRPO, with myself acting as the reward function. For each unfinished Dweet, I sampled a group of eight completions, rendered them side by side and selected the one I found most aesthetically pleasing. If none appealed to me, I skipped the group. I used these preferences to update Demoscener’s policy, then sampled fresh continuations and repeated the process over roughly 9–10 rounds of 500 groups each. Altogether, I manually inspected around 40,000 animations. Somewhere around the 10,000th animation, I began to appreciate the existence of VLMs and HPSv3. Sadly, it was too late. After several rounds of training, I wanted to see how much of my taste had actually rubbed off on Demoscener. I enlisted a rather unwilling friend to judge 400 sets of demos from a few of the checkpoints (I picked 4 randomly), without knowing which model had produced which. The results showed that checkpoint 4 was the most favorable, so I used that to run inference on my Mac. Now coming to the end of the side quest, I built a little interface where I could generate several continuations from the same prefix, render them side by side and branch off from whichever one caught my eye. I could then tweak the code, generate more continuations, and keep exploring. Each branch represents a possible continuation, which I could compare visually, tweak by hand, and use as the starting point for further exploration. There is something quite lovely about being able to wander through these possibilities. I think I'll be spending a little more time doing just that. You can find Demoscener-7B and the dataset on HuggingFace.