Where a tool should stop, and ask

I spent a session building a sprite pipeline for my 2D engine. The shading tool worked. The importer worked. The animation tool only worked once I let it stop being clever and ask the agent driving it a question.

I have a game engine called ressort, written in machin, my machine-first language. It has one unusual property: every sprite in it is text. Rows of characters indexing a palette. No PNG, no atlas, no asset loader, no binary file in the repository at all. The art diffs in a pull request the way the code does.

That constraint is the whole reason the rest of this happened.

Derive the expensive two thirds

Authoring a sprite has three costs and they are wildly uneven. The silhouette is cheap, and it is the creative part. The shading — outline, light ramp, rim light, occlusion — is about 70% of the labour. The variants and the animation frames multiply everything above.

So the engine now automates the second and the third. You draw a flat mass of #, name a material, and it computes the rest. A recolour is a character map, because the draw call already took the palette as an argument. A walk cycle is three deformations of one pose. All of it headless, all of it checksummed the same way the simulation checksums the world — so generated art shows up in review as a digest that changed.

Then a character arrives that no shader can invent

A detailed soldier — chest rig, sling, camo, 100×144 pixels. Every assumption above breaks at that resolution. A light model that means depth below the top surface has nothing to say about a pouch. At that size, the shading is the drawing.

What did not break was the format, which genuinely surprised me. Quantised down to the 32 palette entries the text format can address, the error against 64 colours is 7.28 versus 5.43 — indistinguishable at 1×. Even 16 colours still reads as the same man. The source only used 274 distinct colours to begin with. The palette cap was never the constraint.

So the right tool was not a generator. It was a transcriber: read the image, cut the colours, write out characters. raylib decodes the PNG — LoadImage is CPU-side, so the command needs no display at all — and the output is source code. The image is never committed. The artifact is the transcription, which is text, which diffs, which you own.

It costs something, and the number is worth saying out loud: 14,400 characters per pose against 216 for a SNES-scale sprite. That still versions. Nobody reviews it as a diff.

The line

Then I tried to animate him, and hit the thing this post is actually about.

To swing a leg you must first know which pixels are the leg.

That is not a measurement. There is no filter, no threshold, no clever segmentation that turns a grid of colours into this region is a thigh and it pivots at the hip. It is an act of understanding, and my engine has no way to perform one. I could have written six heuristics that work on this soldier and fail on the next character. Instead the tool stops.

It does what a tool can do — measure the sprite, state the format, check the answer, run the deformation deterministically — and asks the agent driving it for the one thing only reasoning supplies:

$ ressort sprite brief spetsnaz.spr --for walk
{"task":"rig","sprite":"spetsnaz","size":"64x92","anchor":"32,92","for":"walk"}

THE SPRITE as tone — one character per pixel, dark to light:
  31      .:::.---==+++++--:---.:-.-:-=::::-
  32      .-:.:--==-++++=-:-:-::.:.:----:.::
  …
YOUR TASK — write a .rig file naming the moving parts of this sprite.
  part <name> x=<x0>,<x1> y=<y0>,<y1> pivot=<px>,<py> z=<order>

One detail there took two attempts and is the reason the whole loop works: the sprite is printed as tone, not as its own palette characters. A quantised palette is in median-cut order, so its characters are noise laid out on a grid — completely unreadable. Rendered as brightness it is a picture, and you can point at an arm.

The agent replies with a file. The tool grades it, because a plausible answer and a correct one are not the same thing — every opaque pixel must belong to some part, or it silently vanishes the moment the sprite is posed:

$ ressort sprite rig spetsnaz.spr --rig spetsnaz.rig
{"ok":true,"parts":9,"covered":2327,"opaque":2327,"coverage_pct":100,"first_uncovered_row":-1}
$ echo $?
0

Then it poses him. Eight frames of a walk cycle, from one drawing, by rotating nine rectangles about their pivots and compositing them back to front. The case study has the frames, plus the before and after of the mistake below.

The failure was the useful part

My first rig gave the near arm one rectangle. The rifle is held diagonally across the chest, so that box swallowed the vest — and the chest tore open on every stride. Both versions passed the coverage check at 100%. Only one was right.

The fix was not a smarter tool. It was a better answer: several boxes sharing a role prefix and a pivot move as one rigid part. One command each way, and the lesson now lives in the brief the tool prints, so the next agent gets it for free.

Why this is more than a prompt

Because the answer is a file with an exit code. Same rig in, same frames out, same checksums. An agent-written rig is not a plausible-sounding artifact you have to trust — it is text you can diff, with a coverage number attached, producing output that reproduces byte for byte.

That is the same contract I keep arriving at for agent-first tools generally. The tool states the task in a form the agent can act on. The agent answers in a form the tool can check. Neither pretends to do the other's job. The interesting engineering is in drawing that line in the right place — and the right place is almost always further back than it feels like it should be.

What was not working, and the fix

The honest scorecard on the day I wrote this: walk was solid, idle and aim were serviceable and generic, and recoil and death did not hold up. Both need the body to move as a chain — hips leading, shoulders arriving late — and a rig was a flat list of parts with no notion of a parent joint. Every part rotated about its own pivot in isolation and the pose came apart.

So I added the parent field the next day. A child now inherits everything done to its parent and states only what it adds, which means anything body-wide — the walk bob, the recoil shove, the drop of a fall — applies to roots only. A rig with no parents is all roots, so walk poses byte-identically to before.

part torso            x=4,44  y=18,56 pivot=26,56 z=3
part head             x=18,40 y=0,20  pivot=29,20 z=4 parent=torso
part arm_front_hand   x=17,31 y=38,52 pivot=24,41 z=6 parent=torso
part leg_front        x=26,50 y=52,92 pivot=33,54 z=5     # legs stay roots

The legs stay roots deliberately — parented to the torso they lift off the ground every time the body leans. That is now stated in the brief the tool prints, so the next agent does not have to find it out.

Recoil fell out immediately: a shove that travels up the body and settles, near arm first, then far arm, then head. Death did not, and the reason is the better lesson. A body going over backwards is not a chain at all. It rotates entirely, about a point on the ground, and no per-part pivot expresses that. So the poser gained one transform outside every chain — a virtual root the whole sprite hangs from — and the parts now supply only what the body does while falling. It reads as a topple. The frames are in the case study.

What I want to underline is where those fixes landed. Both are geometry. Neither moved the line: the tool still cannot tell a leg from a rifle, and the agent still cannot write a pixel. The vocabulary the agent answers in got one word richer, and two motions that were impossible to state became possible to state. That is what a good split buys — the ceiling rises without the contract changing.

Getting it back out

An animation nobody can load is a demo, not a tool. Exporting turned out to rest on one unglamorous thing that had to come first: posed frames are not the same size as each other. Deleting a row moves the anchor, a topple pads the canvas, a walk does neither. Pack those into a strip naively and the character jitters against the ground in whatever engine loads it.

So every export is built from one box — the union of every frame aligned by its anchor — with a single anchor in the metadata. Then three tiers: a 352-byte recipe (.anm: sprite, rig, kind, fps/loop, a checksum per frame), a PNG strip plus Aseprite-shaped JSON that Godot, Unity, Phaser and LÖVE already read, and the frames as MFL or as a multi-frame text sprite for machin consumers.

352 bytes against 97 kB of baked frames is the whole argument for shipping the recipe rather than the pixels. And because the recipe carries a digest per frame, an animation proves itself the way a playthrough already does:

$ ressort sprite verify walk.anm
{"verified":true,"anim":"spetsnaz","kind":"walk","frames":8,"box":"84x112"}
$ echo $?
0

Change the rig, the sprite or the poser and it exits 90 with the first divergent frame. The PNG is written by poking an RGBA buffer and handing raylib an Image{ptr,w,h,1,7} — CPU-side, no window, so the export runs in CI next to the tests. And timing turned out to belong to the motion rather than the rig: a walk loops, a death holds its last frame.

The two leftovers, and the bug under them

idle and aim were the ones I had left as "serviceable and generic", and both were wrong the same way: they moved the body in whole pixels. A one-pixel square wave on the entire upper body is not breathing, it is a flicker. Rotation is continuous where a pixel offset is not — so idle is now 0.02 radians of lean with the head arriving late and the arms later still, and aim is a curve that dips before it lifts, crosses the mark and settles onto it.

Rebuilding them surfaced something that had been wrong in every transition from the start. A cycle divides its span by the frame count, because frame n would be frame 0 again. A transition must not — it has to arrive — and dividing by n meant its last frame only reached (n−1)/n of the way there. That is why the topple stopped three quarters over and never landed, and why a recoil's last three frames were dead air.

Note how that one hid: every frame was individually plausible, every checksum reproduced, the tests passed. It was visible only as an absence — a motion that never quite arrives — which is exactly what a coverage check and a digest cannot catch, and a pair of eyes on a contact sheet can. It is pinned as a property now: a four-frame aim and a nine-frame aim end on the same pose.

The binary was wrong, too

All of this shipped inside demoman — the Demolition Man binary — which was a mistake of habit rather than design. ressort is the engine; Demolition Man is one POC built on it, and a stage of a game has no business shipping an art pipeline.

So the toolchain is its own program now: bin/ressort, which knows about .spr, .sil, .rig and .anm files and nothing whatsoever about any particular game's art table. The game keeps a small demoman art for its own sprites. Everything both needed — flag parsing, the JSON printers, the palette the material presets are written against — moved into the engine, where it belonged in the first place.

Worth saying plainly because it is the same lesson as the rest of this post: the interesting work was not adding anything. It was noticing which side of a line something had been sitting on.

All of it is on GitHub: github.com/javimosch/machin-ressort — the engine, the sprite tool, the headless test suite, and a Demolition Man stage to prove the engine still plays. The full case study with the images is here.