v0.42.0 · What a composition can express

All releases

Since v0.41.0, anyone can write a composed rune. This milestone widens what a definition can say. Templates may be indented, and refrakt format indents nested runes for you. Content may be split on --- into named zones, a sections model may treat content with no headings as one section, and each can walk the items of a plain list. An itemModel declared in YAML now extracts its fields instead of leaving every one empty (, ).

Changes

  • Composed runes can now say more in their content model (SPEC-169):
  • type: delimited splits content on --- into fixed, named zones. The slots are all the zones' fields together, and primary names the zone that receives content written with no ---. A field name used in two zones, a delimiter other than hr, and dynamicZones are each construction errors (zone-field-twice, delimiter-unsupported, dynamic-zones).
  • implicitSection on a sections model makes content with no section heading one item of sections rather than preamble. Its attributes supply $each and must give exactly the keys of emitAttributes (implicit-section-mismatch).
  • each over a list field iterates the items of a non-greedy field that matches one list. Each item's content is the bare {% slot /%}, and $each binds nothing. each over a greedy list field is an error (each-on-list-ambiguous).
  • Fix (BUG-038): an itemModel declared in YAML now compiles its pattern strings into regular expressions at every nesting level. Before this, every $each field it fed was silently empty. An invalid pattern is reported as content-model-invalid, naming the field.
  • refrakt inspect shows a composed rune's content model and template slots (composition in --json), and refrakt reference lists its template slots and marks a delimited model's primary zone.
  • Markdoc content may now be indented. A line indented by a tab or by four or more spaces is no longer an indented code block, so the content of a rune can be indented to show its nesting:
  • Before, those lines silently became a code block. That broke a tag whose closing line was indented, with no error. Every surface parses this way: the content loader, refrakt validate, the language server, the editor preview, composed-rune templates and plan files. They all go through one new shared function, parse from @refrakt-md/runes, which tokenizes with Markdoc's allowIndentation.
  • Behaviour change: a code sample written as an indented code block, with four spaces or a tab and no fence, now renders as ordinary text. Use a fenced block (```) instead. Fenced code is unaffected, including its own indentation. Every tracked Markdown file in this repository parses to the same tree as before.
  • Markdoc is upgraded from 0.4.0 to 0.5.10, pinned exactly in every package. New sites get ^0.5.10. An existing site that lists @markdoc/markdoc itself should move it to ^0.5.10, so that it does not install a second copy.
  • Every tracked .md file in this repository parses to the same AST as before, and rendered output, contracts and structured data are unchanged. What authors may notice:
  • {% if %} and {% else %} conditions are type-checked. A string or number written as a condition, such as {% if "yes" %}, is now an attribute-type-invalid error. Before, any value was accepted. Use a variable, a function such as equals(…), or true/false.
  • {% table %} is stricter. Content between table rows that is not part of a row is now a critical table-syntax error, where it was silently dropped before.
  • Validation errors point at their own line. An error that a rune's validator raises with a specific location is now reported at that line, not at the first line of the rune.
  • Fewer formatting round-trip bugs. An image URL with several parentheses no longer loses everything after the first (. A list or code fence inside a blockquote no longer falls out of the quote. Both affected content that refrakt formats and re-parses internally, such as collection item bodies.
  • New refrakt format [paths] re-indents nested runes. A rune's content is indented one tab only when the rune holds another block rune. A rune holding only prose stays flat, and inline runes do not count. Only leading whitespace changes, and fenced code moves as a block with its own indentation kept. Before writing, the formatter checks that the file still parses to the same tree; if it would not, the file is left alone and reported with its line.
  • refrakt format --check writes nothing and exits 1 if any file would change, for CI. --format json gives a machine-readable report.
  • The language server provides document formatting with the same rule, so editors can format on save.
  • The MCP server adds a refrakt.format tool.
  • formatIndentation is exported from @refrakt-md/runes for other tools.
  • RuneConfig.sequence can now be read from a modifier: { fromModifier, valueMap?, default? }, the form sequenceDirection already has. valueMap translates a modifier value into a sequence style, and default applies when the value is unset or unmapped. The literal form is unchanged.
  • plot uses it (sequence: { fromModifier: 'structure', valueMap: { linear: 'connected' } }) in place of its postTransform. Its rendered HTML is unchanged for every structure. refrakt inspect --audit-dimensions counts a modifier-sourced rune under each style its map or default can produce.
  • Indented Markdoc now displays correctly in code samples.
  • Highlighting: a markdoc code block whose lines are indented with tabs is highlighted the way its flat form would be. The tabs are kept as written. Before, Markdown's grammar read every tab-indented line as an indented code block, so indented runes, headings and emphasis showed as plain text. Lines indented with four spaces at least get their tags highlighted.
  • preview with source=true: the source it shows is now dedented to the preview's own content, keeping the nesting inside it. Before, the first line lost its indentation and the rest kept theirs.
  • Code blocks show blank lines again, and render tabs two columns wide.
  • Blank lines: a code block lays its lines out as a grid, and an empty line had no height, so every blank line in a code sample disappeared. An empty line now takes a normal line's height. A trailing newline still adds no extra row.
  • Tab width: pre now sets tab-size from a new custom property, --rf-code-tab-size, which defaults to 2. Rune content is indented with tabs, and the browser default of eight columns pushed indented samples off narrow screens. Set it in a theme or site stylesheet to change it.
  • Rune tags inside an indented code fence are no longer processed as real tags. refrakt escapes {% … %} inside fences before parsing, but it only recognised a fence that started at the beginning of a line. A fence inside a list item, or inside a rune whose content is indented, had its tags parsed: for example, an md example showing {% hero %} became a real hero. refrakt format now also checks its rewrites against the tree the site actually builds from, with tags inside fences escaped.

What the composition parser must accept, and what belongs elsewhere

SPEC-169accepted

What the composition parser must accept, and what belongs elsewhere

runescompositioncontent-modelarchitecture

Summary

SPEC-168 D7 recorded four gaps between what native runes declare and what a composition can express. They block more runes than any reuse mechanism would unlock. Each was examined against the runes it blocks:

GapRunesOutcome
No delimited content modelhero, feature, step, recipe, playlistReal. Accepted, with fixed zones only (D1)
No conditional (when) content modelsteps, itinerary, symbolNot needed. Each has a declarative alternative (D2–D5), so the existing rejection stands
No implicitSectionitineraryReal, and small. Accepted (D3)
Generated config lacks checklist, sequence, mediaSlots, chrome defaultsplan runes, timeline, itinerary, plot, …Not a parser gap. checklist is not a gap at all; each other feature has a home in another spec (D6)

Two parser additions, two small template additions, and a routing table. No new content-model type is invented.

What the parser accepts today

  • content.type must be sequence or sections (packages/runes/src/lib/composition.ts:1340-1352). The sections keys are type, sectionHeading, fields, preamble, sectionModel, emitTag, emitAttributes, headingExtract and knownSections (:1329-1339).
  • A conditional model is rejected with a stated reason: "the template, not the model, is where an assembly varies" (:262-266).
  • composedRuneConfig produces only modifiers, metaFields, blocks and provides (:1428-1455).
  • each iterates a greedy field, a field with an itemModel and an emitTag, or sections (site/content/extend/rune-authoring/composed-runes.md, "each and $each").

Decisions

D1 — compositions accept a delimited model with fixed zones

The DelimitedModel type exists (packages/types/src/content-model.ts:169-188): a delimiter node, named zones each with a sequence of fields, an optional primary zone for input with no delimiter, and dynamicZones.

content:
  type: delimited
  delimiter: hr
  zones:
    - name: media
      fields:
        media: { match: any, optional: true, greedy: true }
    - name: content
      primary: true
      fields:
        eyebrow:  { match: paragraph, optional: true }
        headline: { match: heading }
        blurb:    { match: paragraph, optional: true }
        actions:  { match: list, optional: true, greedy: true }
{% section forward="split" %}
{% slot name="media" /%}

---

{% slot name="eyebrow" /%}
{% slot name="headline" /%}
{% slot name="blurb" /%}

{% actions %}{% slot name="actions" /%}{% /actions %}
{% /section %}
  1. The slots are the union of the zones' fields. A field name used in two zones is an error at construction.
  2. Only hr is accepted as the delimiter, which is all five runes use.
  3. dynamicZones is rejected. Its zones have no fixed names, so a template has nothing to place. No measured rune uses it.
  4. The composition consumes the delimiters; the template re-emits them where a placed primitive should split, as the composed realm already does with card. The --- in the template is unconditional, per SPEC-145 D17.

Why not let section split the content itself? It can: SPEC-160 D2 gives section --- zones. A composition that only needs the zones passes everything through one slot. A delimited model is for compositions that must address a field inside a zone: hero wraps its trailing list in {% actions %}, recipe places its ingredient and step lists, playlist extracts tracks. Those are the five measured runes.

This corrects SPEC-145 D18, which said recipe's "delimited/zoned model … [is] declarable in frontmatter". It was not: the parser rejected it. This decision makes the statement true.

D2 — when stays rejected; each conditional rune has a declarative form

The rejection's reason holds up. In all three runes that use a conditional model, the condition either can be absorbed into one model or is really a choice of assembly:

RuneCondition todayDeclarative form
itineraryhas an h2 child (plugins/places/src/tags/itinerary.ts:171-201)Both branches are already sections at heading:2; the default only adds implicitSection. One model with implicitSection covers both (D3)
stepshas a heading child (plugins/marketing/src/tags/steps.ts:109-131)One sections model with an optional ordered list in the preamble; the template places both lists, and one is empty (D4)
symbolkind is class, interface or module (plugins/docs/src/tags/symbol.ts:153-179; a function-form model, not when)One sections model at heading:3; the assembly varies by kind, which is a SPEC-145 D15 template choice (D5)

A function-form content model (contentModel: (attrs) => …) is evaluated with no attributes when a composition reads it (composition.ts:261), so it silently takes one branch. A composition definition cannot declare one, but the rule should be stated: a composition's content model does not depend on attribute values.

D3 — compositions accept implicitSection

implicitSection already exists on the sections model (packages/types/src/content-model.ts:160-166): when no section heading is found, all children become one section with the given attributes instead of preamble.

In a composition there is no emitTag, so the implicit section is an ordinary item of the sections slot, and its attributes supply the $each values:

content:
  type: sections
  sectionHeading: heading:2
  emitAttributes: { label: $heading }
  implicitSection: { attributes: { label: "" } }

The keys in implicitSection.attributes must be exactly the keys of emitAttributes, so $each is bound the same way for a real and an implicit section. This replaces itinerary's conditional model.

D4 — each iterates the items of a list field

Today each needs an itemModel with an emitTag to iterate a list's items. A list whose items need no extraction, where each item's content is placed as it is, has no way in.

content:
  type: sections
  sectionHeading: heading
  emitAttributes: { heading: $heading }
  preamble:
    header: { match: paragraph, optional: true, greedy: true }
    items:  { match: "list:ordered", optional: true }
{% group name="steps" layout="ladder" %}
{% slot name="items" each %}
{% section %}{% slot /%}{% /section %}
{% /slot %}
{% slot name="sections" each %}
{% section %}
### {% $each.heading %}

{% slot /%}
{% /section %}
{% /slot %}
{% /group %}
  • A non-greedy field matching one list node may be placed with each. Each list item is one item. The bare {% slot /%} is the item's content, and $each exposes nothing. Naming $each.x on such a slot is the existing each-field-not-emitted error.
  • steps then needs no conditional. An author writes either headings or an ordered list, and the template places both slots. The one not written is empty and renders nothing.
  • An author who writes an ordered list and headings gets both, in source order. That is what they wrote, and steps today would have read only the headings.

D5 — a D15 template may cover several values

SPEC-145 D15 constraint 2 required templates and the selecting attribute's matches values to correspond one to one. symbol's kind has eight values and two assemblies. A template may therefore list several values:

{% template kind="class, interface, module" %}
	…groups and members…
{% /template %}

{% template kind="function, enum, type, hook, component" %}
	…a flat symbol…
{% /template %}

The constraint becomes: every value is covered by exactly one template, and every listed value is in matches. A value covered twice, or not at all, is the same error as before.

symbol then has one sections model at heading:3. In the grouped template each section is a group, and its members are a nested each at heading:4 (SPEC-165 D6). In the flat template the sections render as ordinary titled sections.

D6 — the generated config stays narrow; each missing feature has another home

composedRuneConfig emitting little is consistent with ADR-042 and ADR-043: a composed rune's output is primitives and vocabulary, so behaviour belongs to what it places. Each feature the measurement listed:

FeatureNative runesHome for a composition
checklistwork, bugNot a gap. Checkbox markers are detected on every <li> in the engine (packages/transform/src/engine.ts:173-175, detectCheckboxMarker). The flag only documents intent (packages/transform/src/types.ts:464-469)
sequence, sequenceDirectionsteps, timeline, itinerary, plot, howto, recipeSPEC-156's ladder: {% group layout="ladder" %} with its marker, placed by the template
mediaSlotsstorytelling, recipe, playlistSPEC-160 D2: section's media attribute
Chrome defaults (defaultDensity, width, elevation)hero, cta, feature, hintSPEC-145 D14 (the chrome carrier and its defaults) and SPEC-159's composite intents
staggerItemsbento, feature, steps, pricing, playlistThe placed container's own staggerItems, once the universal stagger attribute lands on it (SPEC-145 D14)
editHintsmanyThe placed primitives' own hints. A composition adds none

Nothing is added to composedRuneConfig by this spec.

Runes this unblocks, and what each still waits on

RuneUnblocked byStill waits on
heroD1SPEC-160 D6 (actions), SPEC-145 D14
featureD1SPEC-161 D3 (deflist term shapes), SPEC-156 arrangements
step, stepsD1, D4SPEC-160 D2, SPEC-156 ladder
recipeD1SPEC-146 (structured data across section)
playlistD1SPEC-145 D27 (per-variant item defaults), SPEC-146
itineraryD3SPEC-156 ladder; requiresParent children become each items
symbolD5SPEC-165 D6 (nested each), SPEC-146; byte compatibility with refrakt docs extract

Not in scope

  • custom content models (map, bento, storyboard, today's comparison). They stay native or move to primitives, as their own specs decide (SPEC-147 D3, SPEC-167).
  • Structured data crossing a placed primitive (SPEC-146).
  • Attribute presets and forward (SPEC-168).

Open questions

  • D4's list-and-headings case. Rendering both is faithful to the source, but it is a change for an author who relied on steps ignoring the list. Whether to report it as a warning is open.
  • Function-form models in native runes. D2's rule could also be a warning on native runes whose model depends on attributes, since such a rune cannot later become a composition. That is a policy question for ADR-044's tiers, not decided here.

Scheduling

  • v0.42.0: D1 (WORK-646), D3 (WORK-647) and D4 (WORK-648). D1's composed hero fixture needs section's split and the actions primitive from SPEC-160, so it moves to WORK-654 in v0.43.0. WORK-646 proves the model over card instead. D4's steps fixture uses a plain container until SPEC-156's ladder exists.
  • D4's example renders correctly only after ADR-046. It places each step with {% section %}{% slot /%}{% /section %}. Under section's current grammar, a step's text is a paragraph with no heading, so it renders as an eyebrow. ADR-046 anchors the grammar to the headline, so it becomes body (WORK-659, v0.43.0).
  • Deferred: D5. Templates covering several values need templates as tags (SPEC-168 D9), which are not implemented. Its criterion stays open, and this spec can reach implemented only when D5 lands or is split out.

Acceptance Criteria

  • A composition accepts type: delimited with delimiter: hr and fixed zones, honours primary for input with no delimiter, and places each zone field as a slot
  • A field name in two zones, a delimiter other than hr, and dynamicZones are each construction errors with their own message
  • A composed hero over section with a delimited model renders its media and content zones and wraps its actions list, asserted against a fixture
  • A composition accepts implicitSection on a sections model; its attribute keys must equal emitAttributes' keys; with no section heading, all content becomes one item of the sections slot
  • each on a non-greedy field matching one list iterates its items, with the bare slot as each item's content and no $each bindings
  • A composed steps renders the same steps from headings and from an ordered list, with one model and no conditional
  • A D15 {% template %} may list several values; every value is covered exactly once, reported both ways as before
  • The error for a conditional content model names this spec's alternatives (implicit sections, list each, D15 templates)
  • SPEC-145 D18's statement about recipe's delimited model and SPEC-145 D15's constraint 2 carry notes pointing here

References

  • SPEC-168 D7: where these gaps were found.
  • SPEC-145: D15 templates, D17 conditionals and delimiters, D18 recipe, D26 each, D27 item defaults.
  • SPEC-156: the ladder, the home for sequence.
  • SPEC-160: section zones (D2) and actions (D6).
  • SPEC-165 D6: nested each, which symbol's members need.
  • ADR-042, ADR-043: why the generated config stays narrow.

Markdoc sources may be indented

SPEC-170implemented

Markdoc sources may be indented

markdocparsingauthoringcompositiontooling

Summary

Composition templates nest runes several levels deep, and without indentation they are hard to read. Today, indenting the content of a tag by four spaces or a tab silently breaks it. Markdoc 0.4.0, the version installed, has an allowIndentation tokenizer option that fixes this. Measured across every Markdown file in the repository, turning it on changes no existing parse. This spec turns it on everywhere refrakt parses Markdoc, through one shared parse function.

What happens today

A template indented with tabs, which is this repository's indentation convention:

{% card %}
	{% slot name="portrait" /%}

	---

	# {% $attrs.name %}

	- item one
	- item two

	{% details summary="x" %}
		Inner text
	{% /details %}
{% /card %}

Parsed with the default tokenizer:

  • Two spaces work. CommonMark lets a block start up to three spaces in, so ---, headings and lists are recognised.
  • Four spaces or a tab do not, and nothing reports it. ---, # Name and - item become literal paragraph text. An indented closing tag becomes inline text, so tags nest wrongly and the following content ends up inside the wrong rune.

Markdoc disables CommonMark's indented code blocks (the tokenizer disables the code rule, node_modules/@markdoc/markdoc/src/tokenizer/index.ts), so the indentation means nothing; the text just falls through into paragraphs.

What allowIndentation does

new Markdoc.Tokenizer({ allowIndentation: true }), with Markdoc.parse(tokenizer.tokenize(source)). The tokenizer source only declares the option; the behaviour is in Markdoc's bundled copy of markdown-it, which skips its "four or more spaces is not a block" check in each block rule when the option is set (node_modules/@markdoc/markdoc/dist/index.js:3363 and seven similar sites).

Measured on the template above:

  • with four spaces or tabs, hr, heading, list and nested tags parse exactly as they do unindented;
  • a fenced code block inside an indented tag keeps its relative indentation: a line written with one more tab than the fence keeps that tab, and the shared indentation is removed.

It is a parser-wide setting, not limited to the inside of tags.

Measured: no existing content changes

Every tracked .md file in the repository except changelogs (1,366 files: site content, plan, fixtures, package docs) was parsed with and without the option, and the ASTs compared. No file changed. Nothing indents inside tags today, because it breaks, so nothing relies on the old behaviour.

Decisions

D1 — one shared parse function, used by every parse site

Markdoc.parse is called directly in about 20 source files, among them:

  • the content loader and layouts (packages/content/src/site.ts, layout.ts);
  • content validation (packages/content/src/validate-content.ts);
  • composition templates (packages/runes/src/lib/composition.ts:425);
  • collections and deferred bodies (packages/runes/src/collection-*.ts, deferred-body.ts, expand-pipeline.ts);
  • the language server (packages/language-server/src/parser/);
  • the editor's preview and layout resolver (packages/editor/);
  • the CLI's inspect and gallery;
  • the plan scanner and renderer (plugins/plan/src/).

A core package exports one parse(source) that creates the tokenizer with allowIndentation: true, and every site calls it. Setting the option at each call site instead would let a page render correctly and fail validation, or show differently in the editor, as soon as one site was missed.

A lint check (or a test that greps the source) reports a direct Markdoc.parse( call outside the shared function, so a new parse site cannot skip it.

D2 — content gets it too, not only templates

Templates need it most, and they are new, so nothing depends on the old behaviour. Content gets the same parser because:

  • the measurement found no change to any existing file;
  • one parser for templates and another for pages would make a snippet behave differently depending on where it is pasted;
  • authors nesting runes in content have the same readability problem.

There is no opt-out setting. If a third-party site turns out to depend on the old behaviour, one can be added then.

D3 — round trips through Markdoc.format stay consistent

Markdoc.format is used for in-memory round trips (packages/runes/src/deferred-body.ts:38, collection-helpers.ts:49, collection-resolve.ts:120, :133, packages/content/src/site.ts:547): a node is serialised and parsed again later. Its default output is unindented, which both parsers read the same way, so these need no change. No tool in the repository writes source files back through Markdoc.format. A future one (the editor writing Markdoc, for example) passes allowIndentation: true to the formatter, so an author's indentation is kept.

D4 — documentation and examples use indentation

  • The composed-runes guide (site/content/extend/rune-authoring/composed-runes.md) states that tag content may be indented with tabs or spaces, and its examples are indented.
  • The composed-rune fixtures (packages/content/test/fixtures/composed-storytelling/runes/) are re-indented, as the first real use. Their rendered output must not change.
  • The changeset notes the one behaviour change for content: a line indented four or more spaces that used to continue a paragraph as text can now start a block.

D5 — indent a tag's content only when it holds another block tag

Indentation is allowed everywhere. It is expected only where it shows structure:

  • A block tag whose content includes another block tag is indented by one tab per level. Composed-rune templates, _layout.md regions, and tabs, hero or grid holding other runes are the cases that gain.
  • A block tag holding only prose and inline tags stays flat. A {% hint %} with paragraphs, or a plan entity with prose and {% ref /%} tags, is not indented. So plan files, nearly all single-level, are unchanged by the rule.
  • Fences keep their own relative indentation and move as a block.

Why not indent everything. Every other Markdown renderer follows CommonMark, where a line indented a tab or four spaces is a code block. GitHub, Prettier and most editor previews would render an indented single-level file, such as every plan entity, as code. Where indentation shows no structure, that cost buys nothing. The rule has one cost of its own: adding the first nested tag inside a flat tag re-indents its body. The formatter makes that mechanical.

Applied by a tool, not by hand. refrakt format (WORK-657) re-indents by this rule, changes only leading whitespace, and refuses any rewrite that would change the AST. Its --check mode runs in CI. Markdoc.format cannot do it: its indent is two spaces, it indents every tag, and it reprints the whole document (see D3).

Acceptance Criteria

  • A core package exports a parse function that tokenizes with allowIndentation: true, and every direct Markdoc.parse call in packages and plugins uses it
  • A check reports any new direct Markdoc.parse call outside the shared function
  • A template indented with tabs and one indented with four spaces produce the same AST as the unindented template, including ---, headings, lists, nested tags and a fence with relative indentation, asserted in a test
  • Parsing every tracked .md file with the shared function gives the same AST as before, asserted once when the change lands
  • refrakt validate, the language server and the editor preview parse an indented page the same way the content loader does
  • The composed-runes guide documents indentation, and the composed fixtures are indented with byte-identical rendered output
  • The changeset describes the behaviour change for lines indented four or more spaces
  • refrakt format applies D5, changes only leading whitespace and refuses any rewrite that changes the AST; --check runs in CI and the language server formats with it
  • site/content/ follows D5, migrated by the formatter with byte-identical rendered output

References

  • SPEC-145: composed-rune templates, the main consumer.
  • SPEC-169: what compositions can express; indentation is a readability fix, independent of it.
  • WORK-640: the Markdoc 0.5.10 upgrade, which lands first so the shared parse() is built on the current version.