cho.sh
Agent Runtime Server Setup

Agent Runtime Server Setup

Running agents evergreen on a server came with its own problems. RAM is fixed. If one runaway program eats all of it, Linux panics and starts killing processes almost at random (the OOM killer). Sometimes it kills the wrong thing, like your SSH connection, and the whole box goes down. The setup should have layered defenses. One greedy process should slow itself down, not crash everyone else.

Info

This is about agent runtime setup, not the interface or inference. Those are three different things. With Claude, inference runs on Anthropic's server and the interface can be your phone, but the runtime should be a filesystem the local Claude binary can reach (the program, not the model). Runtime and interface can be separate.

Layers

1. A big overflow area (swap)

RAM is the desk, swap is the filing cabinet next to it. When the desk fills, rarely used papers go in the cabinet instead of on the floor.

I put a 128GB swap file on the SSD. Slower than RAM, but slow beats crashed. One gotcha. The file must be created with dd, which writes the bytes directly, not fallocate, which leaves holes Linux refuses to use as swap.

2. A compression buffer in front of swap (zswap)

Before pages go to the slow cabinet, Linux compresses them with zstd and keeps them in a corner of RAM, like vacuum packing clothes before boxing them. Most of the time, the swapped data never actually hits the disk, so things stay fast.

I picked zswap over its cousin zram because zswap bills usage to the process that caused it. That matters in Layer 4.

3. A polite bouncer (earlyoom).

The kernel OOM killer only acts at the very last second, when the system is already unresponsive. earlyoom is a small daemon that watches memory and steps in earlier. When RAM and swap are both under 5% free, it politely asks the fattest process to quit (SIGTERM). Under 2%, it forces it (SIGKILL). A do-not-kill list protects what has to survive (systemd, sshd, tailscaled, docker/containerd, and earlyoom itself).

4. Straitjackets for known-heavy jobs (cgroup caps).

Know a job is hungry? Wrap it first.

text
sudo systemd-run --scope -p MemoryMax=8G -p MemorySwapMax=0 -- <your command>

That puts the command in a memory-limited box. Past 8GB the kernel kills only that box and the rest of the server never notices. Best layer in the stack, since the blast radius is exactly one job.

5. Bodyguards for the way in

The SSH and Tailscale daemons get an OOM score of -500, which tells the kernel, "If you must kill someone, pick almost anyone else first." You can always get back in and fix things.

6. Plus tuning knobs (sysctls)

Swap eagerly into the compressed tier (swappiness=100), start background cleanup earlier so the box never stalls waiting for memory (watermark_scale_factor=125), and keep a bigger emergency reserve (min_free_kbytes).

Backlinks1