Author Burns 4B Tokens Testing Astra, Gets No Usable Python Work
- The author gave GPT-6 Astra about 4 billion tokens and 35 hours to build a Python with virtual threads and lexical scoping, but says the autonomous "software factory" delivered nothing of value.
- The experiment let Astra manage its own context, keep records in an agent-notes directory, and spawn subagents, so the author says the failure was not merely a result of close human supervision.
- The author suspects Astra's training rewards long-horizon completion more than code quality: it can persist on complex tasks and computer use, but may generate poor engineering work while pursuing a result.
- Astra repeatedly used dense Python string manipulation to edit C source files instead of the patch tool, while Codex similarly relies on Bash commands such as sed that its harness may hide from users.
Hacker News opinions
I've seen the same thing: new models run obscene Bash commands or Python scripts packed with flags. They're harder to review than regex.
I added a reminder to Claude Code's system prompt to use standard read and write tools, but its default auto-mode seems to push Bash-only workflows like sed and Python, with plenty of bad side effects.
Why not tell the model exactly what is unreadable and ask for a rewrite? For hobby scripts, I often need several prompts before ChatGPT produces something both readable and efficient.
GPT-6 Astra at medium feels less rigorous and thoughtful than GPT-5.6 Sol at its default setting. I certainly do not get any sense that this is AGI.
That might be a benefit. Sol has a reputation for overengineering, and most software does not need moon-mission standards.
Astra is a smarter development agent for many of my tasks, but its GPU shader code can look like Brainfuck. It often makes no attempt to format the code for humans.
I got unclear shader code from GPT-5.6, then spent about an hour working through the math and naming variables sensibly. Once I could explain the model in my own terms, it regenerated code I could read.
I have one Astra session that has been running for 4 days, 13 hours, and 10 minutes, and it is still going.
Astra burned through my Fable usage by launching full-codebase reviews for one-line changes. I do not need it to write a tmux emulator in Bash just to test a change.
I have seen it hit a missing sudo password for objdump, then decide it should write a compiler, binutils, and a disassembly framework instead of asking for help.
This criticism feels premature while these models are still in R&D. For bug search, analysis, small PRs with a good harness, or code nobody reads, the method matters much less.
The methods and generated code may be ugly, but I have seen work that took weeks or months finish in minutes or hours, with testing, reasoning, and experimentation. I will take that speed gain.
I do not see evidence that code quality will improve with time. Newer models seem to produce worse code, only significantly faster.
LLMs let non-programmers get farther, which also makes low-quality software easier to create. Professional products still need a professional or a deeply technical amateur to judge and maintain them.
I have seen Opus and Fable forget they can edit files and use Python scripts as patching tools instead.
If Python is available, I would rather use a deterministic script than an LLM edit tool. Agents should write executable code for operations where determinism matters.
The harness can drive this behavior and save tokens. Bulk scripts avoid repeated context costs, and with a 300K context each command may incur cache reads comparable to about 30K input tokens.
In software engineering, Astra at high is inconsistent and oddly eager to take responsibility for its mistakes. When I ask for fixes, the proposed steps are often incomplete or outside the task scope.