Historian uses Opus 5.5 to surface a 1615 Dutch eyewitness report of dodo hunting
- Historian Benjamin Breen used Opus 5.5 agents with semantic search over the GLOBALISE archive of Dutch East India Company records to surface what appears to be a previously unnoticed 1615 Dutch report of dodo hunting.
- The pipeline: start from specialist knowledge to define a research question, pick a large well-edited corpus, embed it for semantic search, have frontier models rank candidate passages for human review, then iterate on whatever answers the question.
- Agents like Opus 5.5 spawn dozens of copies of themselves to read sources in multiple languages, and given an API key they run their own embedding searches on new sources that emerge mid-research.
- Breen calls the pattern epistemological weirdness: frontier models now produce novel historical findings while staying bad at judging the historical significance of what they find.
- In a parallel case, security researcher Carter Church said he used GPT-6 Astra over six hours to break a Napoleonic-era cipher that had resisted all earlier decryption attempts.
Hacker News opinions
What a great read. I usually associate Substack with verbose, low quality content, but that's definitely not the case here.
The epistemological weirdness thing is real. LLMs are also notably bad at judging the historical significance of what they find.
I use LLMs outside the usual use cases, like 3D design for printing, and the errors are so unlike human errors that they're hard to anticipate. It builds perfect snap catches but rotates the two pieces they connect 90 degrees from where they should be. Dumb error, but the same model does very hard things perfectly.
'Seven chord groups' sounds a lot more like Opus 5.0 than Opus 5.5 to me. Wonder if that was an earlier investigation, since 5.5 improved that kind of language a lot.
Maybe the model isn't intelligence in any form, except as an imperfect reflection of the intelligence of its training data.
Sounds like a Chinese room situation. Its whole understanding of the world comes through text messages, so imagine a human who never saw or touched the object and built their ontology from descriptions alone. Off by 90 degrees is a believable mistake.
LLMs are just weak at spatial reasoning and it's still unsolved. Human language is imprecise about space and people think about spatial problems visually. Would having an image model check the 3D output give a better result?
AI capabilities are spiky: far ahead in some dimensions, falling short in others seemingly at random. That 'thus spoke compute' musical is an absolute banger with impressive graphics and writing, but some metaphors make no sense and the train animation at 2:35 runs backwards. A person who could make the rest of that video would never make those mistakes.
Unfortunately there are reports that they've already dumbed down Opus 5.5.
I use llama with OpenSCAD for 3D modelling and see the same odd arrangements, but it's also great at the parts I'd find boring or onerous. I can picture a part, see it in my mind, rotate it, place it in context, and have an intuition about it.
Good read! How many pages of text were in scope? If 1615 and 1629 were the total, narrowing it down to just 3000 pages would be more impressive than finding the dodo mention among them.
Aren't those years, not page counts? The article says millions of records but I'm not sure how big a record can be.
My dad died and left tons of notebooks of his journals in very hard-to-read handwriting. Is it worth scanning all of them so I can feed them in and get to work? There's so much minutia out there ready to be meta-understood.
I set up an OCR flow with local models on my own journals going back 30 years. It's about 80% accurate, which means it misses enough keywords to make a lot of it uselessly unintelligible. I can compare the images against grep results, but retraining on my handwriting would mean manually revising several hundred pages.
I didn't realize dodos only lived on Mauritius. I thought they were everywhere for some reason. Little meat footballs that can't fly were never going to colonize the planet.