RL Trains Qwen 3.5 to Paint Editable Watercolours in JavaScript
- A Qwen 3.5 35B model writes complete p5.brush JavaScript sketches, which Puppeteer renders to PNGs, so the generated painting remains editable as code rather than requiring another image prompt.
- The team replaced nine reward signals with a four-part pairwise reward: compile and p5.brush use at 0.05, length at 0.05, HPSv3 at 0.30, and comparisons against hand-picked paintings at 0.60.
- The old quality and prompt-adherence judges had correlations of 0.85 to 0.95, while code length accounted for roughly one-third of reward and stopped providing training gradient after step 30.
- The new rubric reached the old 0.65 reward plateau three times faster, continued past it, and reduced generated code from about 13,500 tokens to under 2,000 tokens.
- The comparison pool contains 581 hand-rated paintings drawn from 1,664 generations, including 117 love-tier examples that seeded the reference comparisons.
Hacker News opinions
I wonder whether current LLMs can write Logo. That could produce some interesting designs.
I think this might be one of the better ways to teach people to use AI. It pushes prompting, constraints, and expressing a desired outcome.
I liked the presentation's point that image generators lock us into a context and turn us into spectators. I used p5.js years ago and may try these ideas, though the RL part is beyond me.
I am curious how SVG-generating image models work. Are they trained like this, or is an LLM conditioned on images or diffusion latents that emits SVG-compatible output?
I was reminded of a 2018 paper on learning brush strokes that combined RL and GANs. Pixel-based GAN work was common then, and I still find descriptive scene generation more interesting than pure pixels; this is a good modern use of that idea.
I have been building something similar for voxels. It generates fairly good models from Python code calling a custom Rust native module, though it is still imperfect.
I have been thinking about LLMs for generative art in the older sense of the term. I like seeing code generation used for that approach.
I see this as genetic programming via an LLM, and I think that is cool.