7  DTD Derivation Playbook

Agent-driven derivation from printed front matter, with worked examples

7.1 The iron rule

WarningIron Rule

A schema invented for an edition that never declared its own record form is a fabrication. The locate_conventions stage must stop and report when no front-matter declaration exists. The subsequent stages must not run.

Every field in record_schema.yaml must carry:

  • generated_from — the printed page, column, and line where the rule appears
  • status: front-matter (authoritative) or status: facsimile-only (provisional)

Fields that appear on no checked page must not appear in the schema YAML. Use optional: true, not omission, when a front-matter section is absent from some body pages.

7.2 Running the agent

# Via the Conductor CLI
cd ~/ai-mac-studio/development/ai-agent-orchestrator
uv run python -m orchestrator.cli run schema-derive-agent \
  --input collection=medieval-sec \
  --input vol=bischoff-1 \
  --input front_matter_pages=[1,2,3,4,5,6,7,8]

The agent writes:

  • config/catalogue_schema.yaml
  • config/page.dtd
  • config/catalogue.dtd

and returns a summary JSON with element counts and evidence grades.

7.3 Worked example: medieval-sec (Bischoff Katalog)

Front matter source: “Aufbau der Beschreibung”, pp. 5–6.

Bischoff’s Katalog explicitly declares the order of fields in each catalogue entry:

Ort · Bibliothek · Signatur — Inhalt — Datierung — Herkunft — Schrift — Literatur

Stage 1 extracts this declaration:

record_sections:
  - name: Ort
    order: 1
    xml_tag: ort
    content_type: text
    optional: false
    generated_from: "p. 5 col. 1, first field"
    status: front-matter
  - name: Bibliothek
    order: 2
    xml_tag: bibliothek
    content_type: text
    optional: false
    generated_from: "p. 5 col. 1"
    status: front-matter
  # ... 6 more sections

Stage 3 (verify-against-facsimile) found that Herkunft is absent from 2/4 checked pages — it was marked optional: true.

Stage 4 generated:

<!-- config/page.dtd — medieval-sec
     Base: PDAI canonical page DTD v3.0
     Extensions: catalogue apparatus
     Generated: 2026-08-21 by schema_derivation_agent 1.0
     Source: Aufbau der Beschreibung, pp. 5-6 -->

<!ELEMENT ort (#PCDATA | hi)*>
<!ELEMENT bibliothek (#PCDATA | hi)*>
<!ELEMENT signatur (#PCDATA | hi | ref)*>
<!ELEMENT inhalt (#PCDATA | hi)*>
<!ELEMENT datierung (#PCDATA | hi)*>
<!ATTLIST datierung certainty (certain|uncertain|ca) #IMPLIED>
<!ELEMENT herkunft (#PCDATA | hi)*>
<!ELEMENT schrift (#PCDATA | hi)*>
<!ELEMENT literatur (#PCDATA | hi | ref)*>

7.4 Worked example: diaries (ADART Astronomical Diaries)

Front matter source: Introduction to ADART vol. I, pp. xxi–xxv.

The front matter describes the tablet structure (obverse/reverse, column structure, transliteration conventions). The template was derived accordingly.

Note that the diaries collection added repeat_penalty: 1.15 to the OCR options (via DIARIES_DELTA in diaries_ocr_run.py) because early transcriptions showed excessive repetition of Babylonian month names.

7.5 Worked example: kepler-prim (KGW)

Front matter source: KGW editorial introduction, and the Briefverzeichnis.

Kepler’s Gesammelte Werke is a mixed-content scholarly edition combining Latin and Early Modern German. The letter-head apparatus was not declared in one place but inferred from the consistent pattern across all letter pages. Fields derived from body observation rather than front matter carry status: facsimile-only.

The table ban documented in earlier KEpler collection READMEs was reversed by the 2025 schema-first approach: <table> is now required for all mathematical ephemerides and trigonometry grids.

7.6 CSS scaffolding after DTD generation

After emit-dtds completes, run css_coverage.py --scaffold to generate a starter page.css from the new DTD:

python3 pdai/pipeline/schema-audit/src/schema_audit/css_coverage.py \
  --dtd collections/<name>/config/page.dtd \
  --scaffold > collections/<name>/config/page.css

This produces a valid CSS with placeholder display rules for every declared element. Refine the visual rules against the facsimile using the visual-layout-audit skill.