PDAI Document Standard

Author

Gerd Graßhoff

Published

December 30, 2025

Overview

The PDAI (PDF AI) standard defines a structured, portable format for AI-processed PDF documents. PDAI documents are generated by AI Scholar and viewed with AI Viewer.

This specification covers:

Page-centric directory structure
File naming conventions
Manifest format (from segmentation)
Two-level processing (pages and segments)

PDAI Document Structure

All PDAI documents follow a page-centric structure where all files for a page live together:

pdai/DocumentName/
├── DocumentName.pdf              # Original PDF
├── manifest.json                 # Document metadata + segment mappings
├── ai-orchestrator.yml           # Agent configuration
├── css/                          # Display styles
│   └── default.css
├── templates/                    # Document-specific Jinja2 templates
│   └── translation.j2
├── pages/                        # ALL page-level content
│   ├── page_000/                 # Folder per page
│   │   ├── page_000.png          # Page image
│   │   ├── page_000.xml          # OCR XML
│   │   ├── page_000.html         # HTML conversion
│   │   ├── page_000_translation_english.xml   # Agent output
│   │   ├── page_000_translation_german.xml    # Agent output
│   │   └── page_000_fig_1.png    # Extracted figure (if any)
│   ├── page_001/
│   │   ├── page_001.png
│   │   ├── page_001.xml
│   │   └── page_001.html
│   └── ...
└── segments/                     # Document segments (optional)
    └── letters/                  # Segment type folder
        ├── letter_000/           # Folder per segment instance
        │   ├── letter_000.xml
        │   ├── letter_000.html
        │   └── letter_000_translation_english.xml
        └── letter_001/
            └── ...

Design Principles

Page-centric: All derivatives of a page (image, xml, html, translations, figures) share the same stem and live together in pages/page_NNN/
Consistent naming: page_NNN[_agent].ext pattern with underscores only (no hyphens in filenames)
Three-digit padding: page_000, page_001, …, page_999
Segments mirror pages: Same subfolder pattern for logical document segments

File Naming Conventions

Page Files

pages/page_{num:03d}/page_{num:03d}.{ext}
pages/page_{num:03d}/page_{num:03d}_{agent}.xml
pages/page_{num:03d}/page_{num:03d}_fig_{n}.png

Examples:

pages/page_000/page_000.png - First page image
pages/page_042/page_042.xml - Page 42 OCR result
pages/page_005/page_005_translation_english.xml - English translation of page 5
pages/page_012/page_012_fig_1.png - First figure on page 12

Segment Files

segments/{type}/{type}_{num:03d}/{type}_{num:03d}.xml
segments/{type}/{type}_{num:03d}/{type}_{num:03d}_{agent}.xml

Examples:

segments/letters/letter_000/letter_000.xml - First letter
segments/letters/letter_000/letter_000_translation_english.xml - Translation
segments/chapters/chapter_003/chapter_003.xml - Third chapter

Agent Name Normalisation

Agent names use underscores only in filenames:

Agent ID	Filename
`translation-english`	`page_000_translation_english.xml`
`translation-german`	`page_000_translation_german.xml`
`annotate-entities`	`page_000_annotate_entities.xml`

manifest.json

The manifest is generated by ai-scholar segment and contains:

{
  "document": {
    "name": "Goldbach",
    "source": "Goldbach.pdf",
    "page_count": 245,
    "created": "2025-01-15T10:30:00Z"
  },
  "segments": {
    "letters": {
      "display_name": "Letters",
      "numbering": "correspondence",
      "start": 1,
      "items": [
        {
          "number": 1,
          "start_page": 3,
          "end_page": 5,
          "sender": "Goldbach",
          "recipient": "Euler",
          "date": "1729-12-01"
        }
      ]
    }
  },
  "_assessment": {
    "page_coverage": 95.5,
    "sequence_assessments": {
      "letters": {
        "completeness": 100,
        "gaps": [],
        "duplicates": [],
        "order_violations": []
      }
    }
  }
}

Segment Types

Type	Numbering	Use Case
`chapters`	`hierarchical`	Books, treatises
`paragraphs`	`numbered`	Scholarly articles
`letters`	`correspondence`	Letter editions
`documents`	`diplomatic`	Historical acts
`sections`	`numbered`	Journal articles

Two Processing Levels

PDAI supports processing at two levels:

1. Page-Level (Default)

Agents operate on individual pages:

pages/page_000/page_000.xml → pages/page_000/page_000_translation_english.xml

2. Segment-Level

After segmentation, agents can operate on logical segments:

segments/letters/letter_000/letter_000.xml → segments/letters/letter_000/letter_000_summary.xml

Processing Workflow

flowchart TD
    subgraph Ingestion["1. Ingestion"]
        PDF[PDF Document]
        PAGES["pages/page_*/page_*.png"]
    end
    
    subgraph OCR["2. OCR Processing"]
        XML["pages/page_*/page_*.xml"]
    end
    
    subgraph Segmentation["3. Segmentation"]
        MANIFEST["manifest.json"]
    end
    
    subgraph Materialise["4. Materialisation"]
        SEGMENTS["segments/{type}/{type}_*/"]
    end
    
    subgraph Agents["5. Agent Processing"]
        AI_PAGE["pages/page_*/page_*_{agent}.xml"]
        AI_SEG["segments/{type}/{type}_*/{type}_*_{agent}.xml"]
    end
    
    subgraph View["6. Viewing"]
        HTML["pages/page_*/page_*.html"]
        VIEWER["AI Viewer"]
    end
    
    PDF --> PAGES
    PAGES --> XML
    XML --> MANIFEST
    XML --> HTML
    MANIFEST --> SEGMENTS
    XML --> AI_PAGE
    SEGMENTS --> AI_SEG
    HTML --> VIEWER
    PAGES --> VIEWER

CLI Integration

Generate PDAI with AI Scholar

# Process a PDF
ai-scholar process document.pdf

# Run segmentation
ai-scholar segment ../pdai/DocumentName

# Assess segment validity
ai-scholar assess ../pdai/DocumentName

# Run an agent
ai-scholar agent ../pdai/DocumentName translation-english --pages 0-50

View with AI Viewer

# Start server from project root
cd /path/to/2026project
python -m http.server 4100

# Open viewer
open http://localhost:4100/ai-viewer/viewer.html?doc=pdai/DocumentName

Validation Requirements

A valid PDAI document must have:

Required: pages/ directory with at least one page_NNN/page_NNN.png
Recommended: manifest.json for segment navigation
Optional: segments/, templates/, css/

Structural Validation

# Validate with ai-scholar
ai-scholar assess ../pdai/DocumentName

Assessment Output

Document: Goldbach
Pages: 245
Segments: letters (142 items)

Sequence Assessment:
  letters:
    Completeness: 100%
    Gaps: none
    Duplicates: none
    Order: valid

PDAI standard specification - December 2025

--- title: "PDAI Document Standard" author: "Gerd Graßhoff" date: today format: html: theme: sandstone toc: true toc-depth: 3 code-fold: false code-tools: true --- ## Overview The PDAI (PDF AI) standard defines a structured, portable format for AI-processed PDF documents. PDAI documents are generated by [AI Scholar](https://github.com/grasshoff/ai-scholar) and viewed with AI Viewer. This specification covers: - Page-centric directory structure - File naming conventions - Manifest format (from segmentation) - Two-level processing (pages and segments) ## PDAI Document Structure All PDAI documents follow a **page-centric structure** where all files for a page live together: ``` pdai/DocumentName/ ├── DocumentName.pdf # Original PDF ├── manifest.json # Document metadata + segment mappings ├── ai-orchestrator.yml # Agent configuration ├── css/ # Display styles │ └── default.css ├── templates/ # Document-specific Jinja2 templates │ └── translation.j2 ├── pages/ # ALL page-level content │ ├── page_000/ # Folder per page │ │ ├── page_000.png # Page image │ │ ├── page_000.xml # OCR XML │ │ ├── page_000.html # HTML conversion │ │ ├── page_000_translation_english.xml # Agent output │ │ ├── page_000_translation_german.xml # Agent output │ │ └── page_000_fig_1.png # Extracted figure (if any) │ ├── page_001/ │ │ ├── page_001.png │ │ ├── page_001.xml │ │ └── page_001.html │ └── ... └── segments/ # Document segments (optional) └── letters/ # Segment type folder ├── letter_000/ # Folder per segment instance │ ├── letter_000.xml │ ├── letter_000.html │ └── letter_000_translation_english.xml └── letter_001/ └── ... ``` ### Design Principles 1. **Page-centric**: All derivatives of a page (image, xml, html, translations, figures) share the same stem and live together in `pages/page_NNN/` 2. **Consistent naming**: `page_NNN[_agent].ext` pattern with **underscores only** (no hyphens in filenames) 3. **Three-digit padding**: `page_000`, `page_001`, ..., `page_999` 4. **Segments mirror pages**: Same subfolder pattern for logical document segments ## File Naming Conventions ### Page Files ``` pages/page_{num:03d}/page_{num:03d}.{ext} pages/page_{num:03d}/page_{num:03d}_{agent}.xml pages/page_{num:03d}/page_{num:03d}_fig_{n}.png ``` Examples: - `pages/page_000/page_000.png` - First page image - `pages/page_042/page_042.xml` - Page 42 OCR result - `pages/page_005/page_005_translation_english.xml` - English translation of page 5 - `pages/page_012/page_012_fig_1.png` - First figure on page 12 ### Segment Files ``` segments/{type}/{type}_{num:03d}/{type}_{num:03d}.xml segments/{type}/{type}_{num:03d}/{type}_{num:03d}_{agent}.xml ``` Examples: - `segments/letters/letter_000/letter_000.xml` - First letter - `segments/letters/letter_000/letter_000_translation_english.xml` - Translation - `segments/chapters/chapter_003/chapter_003.xml` - Third chapter ### Agent Name Normalisation Agent names use **underscores only** in filenames: | Agent ID | Filename | |----------|----------| | `translation-english` | `page_000_translation_english.xml` | | `translation-german` | `page_000_translation_german.xml` | | `annotate-entities` | `page_000_annotate_entities.xml` | ## manifest.json The manifest is generated by `ai-scholar segment` and contains: ```json { "document": { "name": "Goldbach", "source": "Goldbach.pdf", "page_count": 245, "created": "2025-01-15T10:30:00Z" }, "segments": { "letters": { "display_name": "Letters", "numbering": "correspondence", "start": 1, "items": [ { "number": 1, "start_page": 3, "end_page": 5, "sender": "Goldbach", "recipient": "Euler", "date": "1729-12-01" } ] } }, "_assessment": { "page_coverage": 95.5, "sequence_assessments": { "letters": { "completeness": 100, "gaps": [], "duplicates": [], "order_violations": [] } } } } ``` ### Segment Types | Type | Numbering | Use Case | |------|-----------|----------| | `chapters` | `hierarchical` | Books, treatises | | `paragraphs` | `numbered` | Scholarly articles | | `letters` | `correspondence` | Letter editions | | `documents` | `diplomatic` | Historical acts | | `sections` | `numbered` | Journal articles | ## Two Processing Levels PDAI supports processing at two levels: ### 1. Page-Level (Default) Agents operate on individual pages: ``` pages/page_000/page_000.xml → pages/page_000/page_000_translation_english.xml ``` ### 2. Segment-Level After segmentation, agents can operate on logical segments: ``` segments/letters/letter_000/letter_000.xml → segments/letters/letter_000/letter_000_summary.xml ``` ## Processing Workflow ```mermaid flowchart TD subgraph Ingestion["1. Ingestion"] PDF[PDF Document] PAGES["pages/page_*/page_*.png"] end subgraph OCR["2. OCR Processing"] XML["pages/page_*/page_*.xml"] end subgraph Segmentation["3. Segmentation"] MANIFEST["manifest.json"] end subgraph Materialise["4. Materialisation"] SEGMENTS["segments/{type}/{type}_*/"] end subgraph Agents["5. Agent Processing"] AI_PAGE["pages/page_*/page_*_{agent}.xml"] AI_SEG["segments/{type}/{type}_*/{type}_*_{agent}.xml"] end subgraph View["6. Viewing"] HTML["pages/page_*/page_*.html"] VIEWER["AI Viewer"] end PDF --> PAGES PAGES --> XML XML --> MANIFEST XML --> HTML MANIFEST --> SEGMENTS XML --> AI_PAGE SEGMENTS --> AI_SEG HTML --> VIEWER PAGES --> VIEWER ``` ## CLI Integration ### Generate PDAI with AI Scholar ```bash # Process a PDF ai-scholar process document.pdf # Run segmentation ai-scholar segment ../pdai/DocumentName # Assess segment validity ai-scholar assess ../pdai/DocumentName # Run an agent ai-scholar agent ../pdai/DocumentName translation-english --pages 0-50 ``` ### View with AI Viewer ```bash # Start server from project root cd /path/to/2026project python -m http.server 4100 # Open viewer open http://localhost:4100/ai-viewer/viewer.html?doc=pdai/DocumentName ``` ## Validation Requirements A valid PDAI document must have: 1. **Required**: `pages/` directory with at least one `page_NNN/page_NNN.png` 2. **Recommended**: `manifest.json` for segment navigation 3. **Optional**: `segments/`, `templates/`, `css/` ### Structural Validation ```bash # Validate with ai-scholar ai-scholar assess ../pdai/DocumentName ``` ### Assessment Output ``` Document: Goldbach Pages: 245 Segments: letters (142 items) Sequence Assessment: letters: Completeness: 100% Gaps: none Duplicates: none Order: valid ``` --- *PDAI standard specification - December 2025*