Lemmalog uses Datalog to retract stale LLM research conclusions
- Lemmalog splits investigation state between an LLM, which converts source code, debugger output, and natural language into structured facts, and a Datalog engine that derives their deterministic consequences.
- When an input fact is retracted, Lemmalog can identify and invalidate every derived fact that depended on it, rather than retrieving contradictory notes and asking the LLM to reason through them again.
- The author built the system for long-running vulnerability research, where agents otherwise repeat rejected approaches, retain false assumptions, or continue from observations that later changed.
- The model can translate an observation such as an LLDB report of a freed object being reused as a write destination into facts like
freed(object_a)andreused_as(object_a, write_target), while the database applies rules. - The Hacker News discussion compares the approach with CodeQL, Graph RAG, answer set programming, Cyc, and other neuro-symbolic designs; several commenters report the same failure to propagate invalidated conclusions in long agent sessions.
Hacker News opinions
I have been exploring local agentic coding, and this pairing of LLMs with Datalog feels like a return to symbolic AI. Heuristic search and logic programming may be useful for planning agents.
I wonder whether this is Graph RAG from another direction, or whether there is something materially new here.
It feels more like a hand-rolled CodeQL to me.
I hit the same issue while diagnosing a hardware failure from logs. The model kept reviving explanations we had already ruled out, so I had to repeatedly correct it while gathering supporting reports.
I have seen a similar system turn articles into entity-relation facts and events, then query them with ordinary graph queries. That worked especially well for timeline questions, and provenance could isolate an error to a bad fact or traversal.
I think this helps evidence grounding, but in most cases it will not produce a large improvement in memory or reasoning.
My experience with long Claude research projects matches this exactly. Old disproven claims contaminate the current state because Claude records them widely and later treats them as facts.
This sounds like non-monotonic logic. Answer set programming is built around retracting or "unlearning" conclusions, and some implementations extend Datalog.
I handle this with a decision log. My CLAUDE.md tells the agent to record each decision, its date, and its context, which keeps it from losing the thread and gives teammates project history.
For malware analysis, I have had better results with short-lived agent workflows, symbols as function contracts, explicit decision state, CLI linters, and knowledge bases. Linters that describe the expected document or code structure improve agent output far more than token-level errors.
This is classic AI territory: an LLM emits facts into an is_a-style representation, then the system eventually needs quantifiers and exceptions. Cyc already traveled much of that path.
I expect major LLM providers to move toward neuro-symbolic systems. Pure scaling has hit diminishing returns, even if providers do not frame their work that way.