The Famulus universe

Layers, components and machines from import folder to landing page

Published

October 14, 2026

This page lists every layer between a source folder and a published landing page, with the component, path, port and role of each layer. Concepts are on Famulus reference. Storage and operation are on DuckLake.

Layer map

flowchart TD
  src["Sources / import folders<br/>scholarly/research/&lt;project&gt;/import/"]
  intake["pdai-collection-intake skill<br/>pdai/skills/pipeline/pdai-collection-intake"]
  coll["PDAI collection on PDAI-1<br/>page XML, DTDs, index.duckdb, sidecars"]
  fdb["&lt;coll&gt;_famulus.duckdb<br/>schema famulus-reference/1"]
  lake["DuckLake<br/>_lake/famulus.ducklake<br/>+ read replica famulus.read.ducklake"]
  mcp["famulus-reference MCP :5135<br/>fr_* tools, /oai, /jsonld, /iiif, /peers"]
  proof["famulus-proof<br/>evidence, proofs, fr: resolver"]
  nb["famulus-notebook :5118"]
  lek["ai-lektorat"]
  orch["ai-agent-orchestrator<br/>(Partituren)"]
  query["ai-query :5113"]
  land["landing pages<br/>landings2026"]
  harv["Catalog harvesters<br/>(libraries, archives, data stores)"]

  src --> intake --> coll
  coll -->|"famulus-reference build"| fdb
  fdb -->|"famulus-reference export [STUDIO]"| lake
  coll -->|"famulus-reference migrate [STUDIO]"| lake
  lake -->|"READ_ONLY"| mcp
  lake -->|"FAMULUS_LAKE=1"| proof
  mcp --> orch
  mcp --> harv
  proof --> nb
  proof --> lek
  orch --> nb
  lake -.->|"planned adapter"| query
  lake -.->|"planned lake: prefix"| land

Data flow of the Famulus universe. Solid edges are implemented; dotted edges are planned.

Components

Layer	Component	Path	Port	Role
Sources	import folder	`scholarly/research/<project>/import/` (+ `MANIFEST.yaml`)	—	PDFs, images, metadata files, databases as delivered; never modified
Intake	skill `pdai-collection-intake`	`pdai/skills/pipeline/pdai-collection-intake/`	—	eight stages: inventory, profile, DTD draft, segment plan, review gate, build, famulus DB, lake export. See Skills
Collection	PDAI collection	`/Volumes/PDAI-1/pdai-collections/<coll>/`	—	page XML, page images, `biblio.xml`, per-document `index.duckdb`, legacy sidecars (`_fql`, `_stanza`, …). See PDAI Collections, Document Structure
Collection code	DTD, segment plan	`pdai/collections/<coll>/config/`	—	`famulus.yaml`, `page.dtd`, `segments.yaml`. See XML Schemas
Famulus DB	`<coll>_famulus.duckdb`	`/Volumes/PDAI-1/pdai-collections/<coll>/`	—	complete famulus reference data of one collection, schema `famulus-reference/1`; rebuildable
DuckLake	catalog + Parquet	catalog `pdai-collections/_lake/famulus.ducklake`, Parquet `<coll>/_lake/`, core `_lake/core/`	—	authoritative store of all collections; snapshots, releases. See DuckLake
Read replica	`famulus.read.ducklake`	next to the catalog	—	copy published after every write session; all readers attach it
Query trace	`famulus_reference_traces.duckdb`	`pdai-collections/_lake/`	—	`fr_run`, `fr_run_step`; written only by the MCP server
API	famulus-reference MCP server	`pdai/famulus-reference/`	5135	`fr_*` tools (streamable HTTP `/mcp`), harvest routes `/oai`, `/jsonld/<coll>`, `/iiif/<coll>/<doc>/manifest.json`, `/peers`. See MCP Servers
Proxy	ai-platform-mcp	`platform/mcp-server/`	stdio	lists and proxies the `fr_*` tools. See MCP Orchestra
Proof	famulus-proof	`development/famulus-proof/`	—	evidence, proofs; lake mode `FAMULUS_LAKE=1`; `fr:` resolver (`famulus_proof/lake.py`) for `Clause.address`; probe lab (`scripts/satz_labor.py`: normalise, filter, classify), described in the paper The Famulus Probe Lab
Notebook	famulus-notebook	`development/famulus-notebook/`	5118	notebooks; guard resolves `fr:` citations. See Famulus Notebook
Lektorat	ai-lektorat	`publication/ai-lektorat/`	—	text review; uses famulus-proof evidence
Orchestration	ai-agent-orchestrator	Partituren in skill folders (`partitur.yaml`)	—	executes Partituren, e.g. `pdai-collection-intake` (goal path `pipeline/intake/collection`)
Query	ai-query	`pdai/agents/ai-query/`	5113	query resolution; lake adapter planned
Landings	landings2026	`scholarly/projects/landings2026/`	—	static landing pages; reading through the lake planned. See Landing backends

Machine topology

flowchart LR
  subgraph studio ["mac-studio (M1 Ultra), ~/ai-mac-studio"]
    pdai1["PDAI-1 drive<br/>pdai-collections, _lake/"]
    writer["lake writer<br/>export, migrate, maintain"]
    srv["famulus-reference MCP :5135"]
    runner["syncthink runner"]
  end
  subgraph client ["client, e.g. openscience, ~/ai-openscience"]
    mirror["Syncthing mirror of the code tree"]
    smb["PDAI-1 via SMB<br/>(read)"]
    queue["~/syncthink/queue/"]
  end
  mirror <-->|"Syncthing"| studio
  smb -.->|"SMB"| pdai1
  queue -->|"Syncthing"| runner --> writer
  writer --> pdai1
  srv --> pdai1
  client -->|"tailnet ‹mac-im-tailnet›:5135"| srv

mac-studio holds data, lake and server; clients mirror code and mount PDAI-1.

Machine	Tree	Role
mac-studio	`~/ai-mac-studio`	central machine: PDAI-1 drive with all PDAI data, DuckLake (single writer), famulus-reference MCP server, all Famulus development, syncthink runner
client (e.g. openscience)	`~/ai-openscience` (Syncthing mirror of `~/ai-mac-studio`)	code edits; PDAI-1 mounted via SMB (read); studio-only steps queued in `~/syncthink/queue/` (allow-list `platform/config/syncthink_jobs.yaml`) or run by the owner on mac-studio

Steps marked [STUDIO] (lake export, migration, maintenance, the PDAI-1 space audit platform/scripts/pdai1_space_audit.py) run only on mac-studio. On every other host a write to the production catalog raises WriteRefused.

Status on 2026-10-14

PDAI-1 was 100 % full (24 GiB free). The migration of existing DuckDBs into the lake waits for the report-only space audit and the owner’s approval of its deletion list. Route /famulus-reference is registered with deploy: planned.