Streams Markdown and markup into hierarchical semantic events, enabling real-time transformations and rendering to HTML, Markdown, XML, or custom formats while preserving nested structure.
Incremental Markdown parser that emits streams of semantic events, plus tools to manipulate them — designed for real-time rendering of streamed LLM output.
val markdown = """
# Hello
A *streaming* parser, <b>live</b>.
"""
flowOf(markdown).parse().collect {
println(it)
}
Will print:
{"type":"mark","name":"h1"}
{"type":"text","text":"Hello"}
{"type":"unmark","name":"h1"}
{"type":"mark","name":"p"}
{"type":"text","text":"A"}
{"type":"text","text":" "}
{"type":"mark","name":"em"}
{"type":"text","text":"streaming"}
{"type":"unmark","name":"em"}
{"type":"text","text":" parser,"}
{"type":"text","text":" "}
{"type":"mark","name":"b","tagged":true}
{"type":"text","text":"live"}
{"type":"unmark","name":"b","tagged":true}
{"type":"text","text":"."}
{"type":"unmark","name":"p"}
The stream is append-only: each event is emitted as soon as the parser commits to it, so <h1> opens before arrives and opens the moment the resolves. The Markdown and the literal tag produce the same kind of / events — the only difference is , flagging the as real HTML in the source rather than Markdown-derived (the , being Markdown, omits it).
Backed by the first test in
MarkanywhereParserTest—should parse the README parsing example.
println(flowOf(markdown).parse().render())
Will print:
<h1>
Hello
</h1>
<p>
A <em>streaming</em> parser, <b>live</b>.
</p>
Backed by the first test in
SemanticEventsRenderingTest—should render the README HTML example.
Because the same SemanticEvent stream can carry pure HTML (everything isTagged = true), the pipeline runs in reverse too: feed a real page's DOM — presentational wrappers, icon fonts, tracking scripts and all — and get clean Markdown back, ideal for handing a web page to an LLM.
val page = semanticEvents(tagged = true) {
"body" {
"h1" { +"Weather" }
"p" {
"i"("class" to "fa-solid fa-sun") { }
+
{ + }
+
( to ) { + }
+
}
{ + }
}
}
println(page.transformHtmlToMarkdown().renderMarkdown())
Will print:
# Weather
☀️ Sunny and **warm** today — see the [forecast](https://example.com/forecast).
(in ) is a fixed chain of stream operators: maps icon-font glyphs to emoji ( → ☀️), unwraps the presentational and drops the noise while keeping semantic tags and link s, blank inline formatting and structural whitespace are dropped, and runs last; then serializes the result. Each operator is a plain extension, so you can compose your own subset — the module also ships (honour the browser's hidden-subtree and layout-table verdicts) for when you start from a raw capture.
The input here is built by hand, but in practice it comes from a captured page: markanywhere-dump injects into any browser, and drives a real Chrome over CDP to produce a . That capture stamps every actionable element with a reference, and the final step turns those into -style links (a no-op here, since the hand-built input carries no refs) — so an agent can read the Markdown and then click an element back on the live page by its . See .
Backed by the first test in
HtmlToMarkdownTest—should convert the README HTML to Markdown example.
val markdownFlow = flowOf(markdown)
document.body!!.appendSemanticEvents(
markdownFlow.parse()
)
Renders equivalent HTML into the browser's DOM tree.
Note: Typically markdownFlow: Flow<String> represents a Markdown text stream, for example from LLM inference.
Backed by the first test in
AppendSemanticEventsTest—should append the README DOM example.
Each reusable rule set is an extension on TransformerBuilder — think of one as an XSLT stylesheet you can compose with others:
fun TransformerBuilder.demoteHeadings() {
match("h1") { "h2" { children() } } // re-emit <h1> as <h2>, keeping its content
}
fun TransformerBuilder.emphasizeToStrong() {
match("em") { "strong" { children() } } // re-emit <em> as <strong>
}
println(
flowOf(markdown)
.parse()
.transform {
demoteHeadings()
emphasizeToStrong()
passthrough()
}
.render()
)
Will print:
<h2>
Hello
</h2>
<p>
A <strong>streaming</strong> parser, <b>live</b>.
</p>
Each is like an XSLT template: it selects marks by name (), by the wildcard, or by a predicate over the mark's attributes (), and emits replacement marks and text in their place. mirrors — it descends into the matched subtree, and only there: an unmatched mark stops traversal unless a rule opts into transparent descent. Raw text survives at exactly one place — a rule — so alone never leaks text. Rules are tried in registration order with the first match winning, which is why the specific rules above sit ahead of , the catch-all that copies every remaining mark (preserving its origin) and its text verbatim — here the literal flows through untouched while the Markdown is rewritten to .
Because the rule sets are plain extension functions, they compose — transform { demoteHeadings(); emphasizeToStrong(); passthrough() }. The block also runs per collection, so stateful rule sets stay correct when a flow is collected more than once: a sequence counter, text captured for an afterClose { … } summary emitted once the subtree closes, or a mode-scoped sub-pipeline (children(mode = "capture") routing a subtree to its own set of rules). The same machinery normalizes HTML down to Markdown (simplifyHtml() in markanywhere-html), redacts spans, or routes a tagged block to a separate sink.
Backed by the first test in
TransformerTest—should transform the README example. The remaining tests there exercise each DSL feature mentioned above.
The same semanticEvents { } builder used to feed the renderer also describes an expected stream, so a parser/transformer test reads like the output it asserts:
@Test
fun `should parse the greeting`() = runTest {
// given
val markdownFlow = flowOf(markdown)
// when
val events = markdownFlow.parse()
// then
events.mergeAdjacentText() sameAs semanticEvents {
{ + }
{
+
{ + }
+
tag() { + }
+
}
}
}
semanticEvents and mergeAdjacentText ship in markanywhere-flow; sameAs ships in markanywhere-test:
testImplementation("com.xemantic.markanywhere:markanywhere-test:0.1.3")
This is the exact pattern behind every "Backed by …" test linked above.
GFM is the dialect LLMs were trained on, which is why we take it as the baseline — not as a conformance target. The parser diverges where spec-correct behaviour would force buffering past the next emitted event (which would break streaming, the whole point of the library), and those divergent shapes are ones LLMs effectively never emit. The long-term aim is a separate spec, anchored on this parser, defined to fit the streaming model rather than the document model.
Namespaced tags let you embed arbitrary custom markup in a Markdown document. Any <namespace:tagname …> / </namespace:tagname> pair passes through as Mark/Unmark events with isTagged = true and parsed attributes — your renderer handles them however it wants. This covers use cases like custom card components, alert boxes, or any domain-specific block type without requiring new parser syntax.
See markanywhere-parse/README.md for a full list of divergences and their rationale.
You can depend only on markanywhere-parse and consume the Flow<SemanticEvent> with your own renderer — the API surface is a single three-variant sealed class. The markanywhere-transform module additionally lets you intercept and rewrite events before they reach any renderer.
We use language to convey meaning, and we use text to express language. The document-whether scroll, codex, or book-established a paradigm for how text is preserved as a packaged unit. Documents also introduced formatting: visual and structural conventions that signal the intent behind particular fragments of text within a larger context.
When we built machines to process text, we formalized this into "document formats". These formats naturally inherited the hierarchical structure of books-parts, chapters, sections, paragraphs-and the software we built assumed that documents exist as complete artifacts to be parsed, transformed, and rendered.
But something new has emerged. We started texting each other, and text became a stream of information: received, comprehended, and often discarded in the moment of reception. This is also the communication paradigm between humans and LLMs. The text is not a document to be opened and read-it is an unfolding stream, with alternating modalities, comprehended while being generated.
Structured documents are not the right abstraction here. What we need instead is an ontology of expressive meaning as a stream of events: each event signaling either an incremental fragment of text or a transition between modalities of linguistic expression (from prose to code, from paragraph to heading, from plain text to emphasis). markanywhere inverts the traditional document processing flow. Rather than consuming complete documents and producing structure, it consumes streaming tokens and emits semantic events in real-time. These events can then be transformed-also as a stream-into various output formats: HTML, Markdown, XML, or whatever the receiving context requires.
The SemanticEvent can be a:
Text: a chunk of charactersMark (e.g. <em> tag, with optional attributes)Unmark (e.g. </div>, indicating that previously opened mark is closed)and carry an flag distinguishing the origin of the event: when it comes from an actual HTML/XML tag in the source, when it is derived from Markdown syntax (e.g. yields an mark with , while yields ). The same stream can therefore represent pure Markdown, pure HTML/XML (everything ), or HTML embedded in Markdown — with the distinction preserved end-to-end so downstream renderers can treat each origin appropriately.
See the SemanticEvent definition.
In build.gradle.kts add:
dependencies {
implementation("com.xemantic.markanywhere:markanywhere:0.1.3")
}
devBuildThis is a Kotlin Multiplatform project published for JVM, JS, Wasm, and Native.
Building every target on every local run is slow, so the markanywhere.convention
plugin exposes a devBuild Gradle property that selects a minimal,
fast-to-build target set for local development:
devBuild defaults to true, so a bare ./gradlew build only touches the
dev targets each module declares — JVM, plus browser-JS for the JS modules and
the chain they depend on.-PdevBuild=false to build the full published set.The convention plugin declares no Kotlin target itself. Instead each module
reads the flag via val devBuild: Boolean by extra and branches its own
kotlin { } target declarations on it, using two helpers the convention adds to
KotlinMultiplatformExtension — allTargets() (the full published set, honoring
the targetGroup flag) and jsTarget() (a configured browser+nodejs JS target):
import con.xemantic.markanywhere.buildlogic.allTargets
val devBuild: Boolean by extra
kotlin {
if (devBuild) jvm() else allTargets() // most modules
sourceSets { /* … */ }
}
Variations:
if (devBuild) { jvm(); js { browser() } } else allTargets(), or for a JS-only
module .To build (or just configure) the complete multiplatform set locally, run any task
with -PdevBuild=false.
renderDumpFixtures taskThe end-to-end HTML→Markdown tests run against captured DOM dumps, not raw
HTML. Each fixture in markanywhere-html/src/commonTest/dumps/*.json is a
SemanticEventDump:
the semantic event stream of a real page's rendered DOM tree, plus the url it
was captured from and the dumpedAt instant of the capture. The events — not any
original HTML — are the source of truth, so the bloated source HTML is not kept
in the repository.
A raw event stream is hard to read, so to regenerate the human-readable HTML a dump represents (for example to see what input produced a given Markdown output):
./gradlew :markanywhere-html:renderDumpFixtures
This renders every dump back to pretty-printed HTML under
markanywhere-html/build/renderedDumps/<name>.html.
Hello<em>**streaming*<b>markunmark"tagged":truebemtransformHtmlToMarkdown()markanywhere-htmlresolveIcons()fa-sunsimplifyHtml()<body><script>hrefencodeActionableRefs()renderMarkdown()Flow<SemanticEvent>applyAccessibility()window.markanywhere.dump()SemanticEventDumpencodeActionableRefs()[forecast](ref:7:https://…)refmatch"em""*"match({ name == "p" && this["role"] == "note" })children()<xsl:apply-templates/>"*"matchText { +it }children()passthrough()isTagged<b>*streaming*<strong>semanticEvents { } builds a Flow<SemanticEvent> from a tree-shaped DSL: "name" { … } opens a Markdown-derived mark, tag("name") { … } opens an HTML-tagged one (isTagged = true), and +"…" emits text.sameAs is a suspend infix that compares two event flows by serializing each to JSON lines — a mismatch prints a readable line diff."A", " ", " parser," events in the parsing output above), so mergeAdjacentText() coalesces adjacent text events first, keeping the expectation concise.| Feature | Syntax |
|---|
| ATX headings | # H1 … ###### H6 |
| Paragraphs | plain text blocks |
| Hard line break | two trailing spaces or \ before newline |
| Thematic break | --- / *** / ___ |
| Fenced code block | ```lang … ``` |
| Indented code block | 4-space indent |
| Block quote | > text |
| Unordered list | - item / * item / + item |
| Ordered list | 1. item |
| Task list | - [x] done / - [ ] todo |
| Table | | col | col | with separator row |
| Inline code | `code` |
| Strong | **bold** or __bold__ |
| Emphasis | *italic* or _italic_ |
| Strikethrough | ~~text~~ |
| Inline link | [text](url) / [text](url "title") |
| Inline image |  |
| Autolink | <https://example.com> / <user@example.com> |
| Extended autolink | www.example.com / https://… (GFM §6.9) |
| Raw HTML (block) | HTML type 1–7 blocks pass through |
| Raw HTML (inline) | <tag attr="val">…</tag> |
| Entity references | & / * / * |
| Backslash escapes | \* → literal * |
| Link reference definitions | [label]: url "title" (back-references only — see divergences) |
| Feature | Syntax | Output element |
|---|
| Highlight | ==text== | <mark> |
| Superscript | ^text^ | <sup> |
| Inline math | $E=mc^2$ | <math> |
| Display math | $$…$$ on own lines | <math display="block"> |
| Front matter | ---/+++ fence at document start | <frontmatter format="yaml|toml"> |
| Namespaced tags | <ns:tag attr="val"> | Mark(name="ns:tag", isTagged=true) |
| DOCTYPE declaration | <!DOCTYPE html> (case-insensitive) | Mark(name="doctype", isTagged=true) |
| Feature | Reason |
|---|
Setext headings (=== / --- underline) | Requires one-line look-ahead to distinguish from paragraph + thematic break |
Forward reference links ([text][label] before [label]: url) | Definition must precede usage — the append-only stream cannot revisit emitted events |
| Tight vs. loose lists | Tight/loose can only be decided after the full list closes |
| Mid-paragraph tables | Tables only start at a fresh block boundary |
| Multi-line inline constructs | flushInline force-closes inline state (code/em/strong/etc.) at every line/block boundary |
Image inside link ([](url)) | Nested inline constructs require speculative recursive parsing |
Nested inline links ([foo [bar](/u)](/u)) | Inner link is treated as label content; spec requires parser unwinding |
| Multi-line link parsing | Link destination, title, or label spanning newlines is not supported |
| Module | Purpose |
|---|
markanywhere-api | SemanticEvent sealed type — the only interface between modules |
markanywhere-parse | Streaming parser: Flow<String> → Flow<SemanticEvent> |
markanywhere-render | HTML renderer: Flow<SemanticEvent> → HTML string |
markanywhere-transform | DSL for rewriting event streams on the fly |
markanywhere-flow | Utilities for composing and splitting event flows |
markanywhere-extract | Utilities for extracting structured data from event streams |
markanywhere-js | Kotlin/JS DOM renderer |
markanywhere-dump | Injectable browser bundle exposing window.markanywhere.dump() — captures a live page's DOM as SemanticEventDump JSON |
markanywhere-browse | Drives real Chrome over CDP (via kdriver) to capture a live page as a SemanticEventDump and act on it by element reference |
markanywhere-html | HTML→Markdown pipeline transformHtmlToMarkdown (resolveIcons, simplifyHtml, dropBlankInlineFormatting, whitespace normalization, encodeActionableRefs), plus applyAccessibility |
markanywhere-test | Test helpers: sameAs for asserting Flow<SemanticEvent> equality |
MarkUnmarkisTaggedtruefalse*text*emisTagged = false<em>text</em>isTagged = trueSemanticEventisTagged = trueif (devBuild) js { browser() } else allTargets()allTargets(). For example markanywhere-browse
depends on kdriver (dev.kdriver:core), which publishes only JVM, JS, and the
desktop-native triples — no Wasm, Apple-mobile, or android-native — so it
declares exactly that intersection in its else branch.Surfaced from shared tags and platforms — no rankings paid for.