Project DANSO \U0001FA88: Document Abstract Notation for Semantic Operations

DANSO is a specialized markdown parser designed to enhance the performance of LLMs when processing markdown. It addresses the limitations of naive approaches and provides a targeted solution to improve quality, coherence, and fidelity to the original.

Key Features

• Shallow Parsing -- Sorts markdown into frontmatter, MDX/HTML tags, code blocks, and paragraphs, so the LLM can better understand the structure and context.
• Targeted Logic -- Each chunk gets corresponding processing, ensuring only the necessary parts are processed while preserving structure and functionality.
• Frontmatter Handling -- Identifies and processes only the string values, like title and description, keeping the frontmatter structure intact.
• MDX/HTML Tag Preservation -- Recognizes tags and processes only the inner strings, preserving the tags and their functionality.
• Code Block Extraction -- Pulls code blocks out whole, with basic metadata. DANSO can optionally process code comments while keeping the code itself unmodified.
• Paragraph Context -- Gives the raw text of each paragraph whole, allowing LLMs to understand the context and generate more coherent outputs.

Why DANSO?

Traditional document processing often faces challenges when dealing with markdown.

• Unintentional processing of MDX tags and code blocks, leading to syntax errors and altered functionality.
• Limited context from parsing at HTML-tag level, resulting in suboptimal outputs and difficulty reordering sentences.
• Improper handling of proper nouns, causing confusion and altering the intended meaning.

DANSO parses each part the way that part needs, preserves important elements, and maintains context. This leads to improved quality, coherence, and fidelity to the original.

Usage

1. Install it.
   text

   npm install danso

2. Import the parser.
   javascript

   import { parse } from "danso";

3. Parse your markdown.
   javascript

   const parsed = parse(markdownString);

4. Do what you need with the parsed chunks.

Contributing

Contributions welcome. If you encounter any issues or have suggestions for improvements, open an issue on GitHub. Feel free to submit pull requests with bug fixes, enhancements, or new features.

License

MIT. You are free to use, modify, and distribute it in accordance with the terms of the license.

Acknowledgements

We would like to thank the contributors and the open-source community for their valuable input and support in developing DANSO.

Contact

Questions go to hey@cho.sh

Happy parsing and processing with DANSO 🪈!

Inspired & Modified from wobsoriano/bun-lib-starter: My minimal bun library starter