Grammar observability

Parséman has two opt-in grammar-observability modes. Interpreter coverage uses the selected start-rule closure; a coverage-enabled macro grammar map carries the stable IDs emitted for that compiled map, including final composed winners.

• Coverage answers “which rules, choice arms, dispatch arms, and labels succeeded?”
• Trace answers “what did this parse try, select, fail, and backtrack through?”

Neither mode changes ordinary interpreter parsing or ordinary macro output.

Coverage

Use coverage when a test should prove that a grammar exercised a particular semantic branch. A run returns the normal RunResult plus an immutable coverage snapshot.

import { choice, literal, runWithGrammarCoverage } from 'parseman'

const parser = choice(literal('yes'), literal('no'))
const { result, coverage } = runWithGrammarCoverage(parser, 'no')

console.log(result.ok)       // true
console.log(coverage.hits)   // ['choice:entry/arm:1']
console.log(coverage.unhit)  // ['choice:entry/arm:0']

Pass one collector explicitly to merge several inputs. CI thresholds should use the boolean hit set and an explicit required-ID list.

Vitest with a macro grammar

Enable instrumentation only in the test build, then give run() the collector and (optionally) the trace sink. Coverage-enabled macro grammar maps carry their own immutable definition list; this is what makes the reported percentage truthful for the generated grammar rather than a count of whichever events a test happened to observe.

// vitest.config.ts
import parseman from 'parseman/plugin'

export default {
  plugins: [parseman.vite({ grammarCoverage: true })],
}

import {
  compiledGrammarCoverageDefinitions,
  createGrammarCoverageCollector,
  createGrammarInstrumentationContext,
  run,
} from 'parseman'
import { grammar } from '../src/grammar.js'

const collector = createGrammarCoverageCollector(
  compiledGrammarCoverageDefinitions(grammar),
)

for (const source of fixtures) {
  const result = run(grammar.Stylesheet, source, {
    trivia: grammar.whitespace,
    instrumentation: createGrammarInstrumentationContext({ collector }),
  })
  expect(result.ok && result.unconsumedFrom === null).toBe(true)
}

const coverage = collector.snapshot()
expect(coverage.ratio).toBe(1) // 100% of structural definitions in this corpus

ratio is hits.length / definitions.length: it counts successful named rules, choice arms, dispatch arms, and labels in the coverage-enabled generated grammar map. It is not V8 line coverage, statement coverage, or a claim that every invalid input has been tested. Macro output without grammarCoverage: true intentionally has no definition metadata or hooks; production parsing stays unchanged.

Trace

Trace is intentionally more verbose. It records lifecycle events with the same IDs: rule entry/success/failure, choice-arm attempt/failure/backtrack/selection, dispatch-arm attempt/selection/success/failure, and successful labels.

import { createGrammarTraceSink, runWithGrammarCoverage } from 'parseman'

const trace = createGrammarTraceSink({ capacity: 200 })
runWithGrammarCoverage(parser, 'no', { trace })

console.log(trace.snapshot().events)

The sink retains the first capacity events. It detaches when full, when a stream callback returns false, or when that callback throws. Its snapshot reports truncated and dropped; detachment never changes parse results.

dispatch traces only the selected route. Arms excluded by the returned string do not emit attempts or backtracks; they were never parsed. If a selected tail fails, the dispatch arm emits failure and the parse failure is committed. Branches that use routed() still report the dispatch start offset, so the trace stays about the grammar route rather than the branch's internal ownership mechanics.

Macro mode

Static combinators, ref() entries, and rules(...) maps can also emit instrumentation. This includes a terminal composeLeaf(...), which uses its post-compose winner plan rather than imported-piece identities. Enable it only in a test/debug build:

import parseman from 'parseman/plugin'

export default {
  plugins: [parseman.vite({ grammarCoverage: true })],
}

With this option off, the macro emits its normal parser source: no collector, trace sink, helper, or observability identifier is present. With it on, the generated parser reads the dedicated coverage/trace context supplied by its test harness. Use the typed helper rather than constructing internal context fields:

import {
  createGrammarCoverageCollector,
  createGrammarInstrumentationContext,
  createGrammarTraceSink,
  compiledGrammarCoverageDefinitions,
  run,
} from 'parseman'

const collector = createGrammarCoverageCollector(compiledGrammarCoverageDefinitions(grammar))
const trace = createGrammarTraceSink({ capacity: 200 })
const context = createGrammarInstrumentationContext({ collector, trace })

run(grammar.Entry, 'yes', { instrumentation: context })

CI artifacts

The canonical result is the in-memory snapshot. A CI job may serialize that snapshot as stable JSON after its tests finish, sorted by grammar ID, and compare required IDs plus a per-grammar minimum ratio. Keep this separate from line coverage: grammar coverage changes when branch topology changes, while V8 line coverage answers a different question.

For composed grammars always select an explicit start rule. IDs come from the final composed winner graph; an overridden rule does not retain a second ID.