Three-Package Integration

Complete workflow across ai-scholar, pdai, and ai-viewer

Overview

The AI document processing ecosystem consists of three interconnected packages, each with a distinct responsibility:

Package	Role	Primary Function
ai-scholar	Processing	OCR, segmentation, translation, annotation
pdai	Data	Structured document storage (page-centric)
ai-viewer	Presentation	Interactive web-based viewing

This guide explains how they work together to transform PDFs into interactive, navigable documents.

Architecture

flowchart TD
    subgraph Input["Source Documents"]
        PDF[PDF File]
        SCANS[Scanned Images]
    end
    
    subgraph Scholar["ai-scholar Package"]
        INGEST[Ingestion]
        OCR[OCR Agent]
        SEG[Segmenter Agent]
        TRANS[Translation Agent]
        ANNOT[Annotation Agent]
    end
    
    subgraph PDAI["pdai Package"]
        PAGES["pages/page_NNN/"]
        MANIFEST["manifest.json"]
        SEGMENTS["segments/"]
    end
    
    subgraph Viewer["ai-viewer Package"]
        HTML[viewer.html]
        UI[Interactive Viewer]
    end
    
    PDF --> INGEST
    SCANS --> INGEST
    INGEST --> PAGES
    PAGES --> OCR
    OCR --> PAGES
    PAGES --> SEG
    SEG --> MANIFEST
    PAGES --> TRANS
    PAGES --> ANNOT
    TRANS --> PAGES
    ANNOT --> PAGES
    
    PAGES --> HTML
    MANIFEST --> HTML
    HTML --> UI

Data flow through the three-package architecture

Package Responsibilities

ai-scholar: The Processing Engine

AI-Scholar transforms raw documents into structured, annotated content:

# Core processing commands
ai-scholar process document.pdf         # PDF → pages/page_NNN/
ai-scholar segment ../pdai/DocName      # → manifest.json
ai-scholar agent ../pdai/DocName translation-english  # → pages/page_NNN/page_NNN_translation_english.xml

Generates:

pages/page_NNN/page_NNN.png — High-resolution page images
pages/page_NNN/page_NNN.xml — Structured OCR output (DTD-validated)
pages/page_NNN/page_NNN.html — Viewer-compatible HTML
pages/page_NNN/page_NNN_fig_M.png — Extracted figures
pages/page_NNN/page_NNN_{agent}.xml — Agent processing outputs
manifest.json — Document structure and segment mappings

pdai: The Data Layer

PDAI provides a standardised, page-centric directory structure:

pdai/DocumentName/
├── DocumentName.pdf              # Original source
├── manifest.json                 # Structure metadata
├── ai-orchestrator.yml           # Agent configuration
├── css/                          # Display styles
├── templates/                    # Document-specific templates
├── pages/                        # ALL page-level content
│   ├── page_000/
│   │   ├── page_000.png
│   │   ├── page_000.xml
│   │   ├── page_000.html
│   │   ├── page_000_translation_english.xml
│   │   └── page_000_fig_1.png
│   └── page_001/
│       └── ...
└── segments/                     # Materialised segments
    └── letters/
        ├── letter_000/
        └── letter_001/

Key principle: All files for a page live together in pages/page_NNN/. No scattered directories.

ai-viewer: The Presentation Layer

AI-Viewer serves PDAI documents through an interactive web interface:

# Start server from project root
cd /path/to/2026project
python -m http.server 4100

# Open viewer
open "http://localhost:4100/ai-viewer/viewer.html?doc=pdai/DocumentName"

Provides:

Interactive page navigation (Document mode)
Agent output viewing (Agent mode)
Segment-based browsing (letters, chapters, paragraphs)
MathJax rendering for mathematics
Translation layer selection

Complete Workflow

Step 1: Ingestion

Start with a PDF or folder of scanned images:

# From PDF
ai-scholar process ../pdai/E57/E57.pdf

# From scans folder
ai-scholar process ../scans/historical_doc/

This creates the basic PDAI structure with page images in pages/page_NNN/.

Step 2: OCR Processing

Extract text from page images:

ai-scholar process ../pdai/E57 --model gemini-3-flash-preview

Generates pages/page_NNN/page_NNN.xml with structured content:

<page pidx="001">
  <meta model="gemini-3-flash-preview" processed_date="2025-12-28"/>
  <main>
    <p><parno>1.</parno> Omnem mutationem quae...</p>
    <p><parno>2.</parno> Huius generis effectus...</p>
  </main>
  <footnotes>
    <footnote data-fn="1">Reference note...</footnote>
  </footnotes>
</page>

Step 3: Segmentation

Identify logical document structure:

ai-scholar segment ../pdai/E57

Creates manifest.json with segment mappings:

{
  "document": {
    "name": "E57",
    "title": "Inquisitio Physica...",
    "page_count": 105,
  },
  "segments": {
    "chapters": {
      "items": [
        {"id": "1", "number": 1, "title": "CAPUT PRIMUM", "pages": [1, 2, 3]}
      ]
    },
    "paragraphs": {
      "items": [
        {"id": "1", "number": 1, "pages": [1], "start_text": "Omnem mutationem..."}
      ]
    }
  }
}

Step 4: Agent Processing

Run translation, annotation, or custom agents:

# Translate to English
ai-scholar agent ../pdai/E57 translation-english --pages 0-50

# Annotate entities
ai-scholar agent ../pdai/E57 annotate-entities --pages 0-50

Outputs go to pages/page_NNN/page_NNN_{agent}.xml.

Step 5: Viewing

Serve the document with a simple HTTP server:

# Start from project root
cd /path/to/2026project
python -m http.server 4100

# Open browser
open "http://localhost:4100/ai-viewer/viewer.html?doc=pdai/E57"

Navigate to explore:

Browse pages with images and HTML side-by-side (Document mode)
View agent outputs in three-pane layout (Agent mode)
Navigate by segment (chapter, paragraph)
Switch between translation layers

Segment Navigation

The viewer uses manifest.json to enable segment-based navigation.

How It Works

Viewer loads manifest — Reads segment definitions on document load
Populates dropdowns — Creates navigation controls for each segment type
Jump to segment — Selecting a segment navigates to its start page
Display metadata — Shows segment-specific information (dates, authors)

Segment Types

Type	Use Case	Metadata
`chapters`	Scholarly treatises	Title, number, pages
`paragraphs`	Numbered arguments	Number, start text, pages
`letters`	Correspondence editions	Sender, recipient, date, place
`documents`	Diplomatic acts	Parties, date, reference

Agent Output Layers

The viewer can display different processing results.

Available Layers

After running agents, page folders contain multiple outputs:

pages/page_000/
├── page_000.png
├── page_000.xml
├── page_000.html
├── page_000_translation_english.xml
├── page_000_translation_german.xml
└── page_000_annotate_entities.xml

Viewing Layers

Document Mode:

Use language dropdown to switch between Original, Deutsch, English

Agent Mode:

Select agent from navigator to view output
Compare source input with agent output side-by-side

Comparing Outputs

Original vs Translation — See how text was interpreted
Multiple translations — Compare English and German versions
Annotation overlays — View entity markup on original text

Cross-Package Commands

From ai-scholar

# Full processing pipeline
ai-scholar process document.pdf
ai-scholar segment ../pdai/DocName
ai-scholar agent ../pdai/DocName translation-english

# Migrate legacy documents
ai-scholar migrate ../pdai/DocName
ai-scholar migrate-all ../pdai

# Assessment
ai-scholar assess ../pdai/DocName

From Project Root

# Start server
cd /path/to/2026project
python -m http.server 4100

# View documents
open "http://localhost:4100/ai-viewer/viewer.html"           # Home catalog
open "http://localhost:4100/ai-viewer/viewer.html?doc=pdai/E57"  # Specific document

URL Structure

Viewer URLs

URL	Description
`/ai-viewer/viewer.html`	Home catalog
`/ai-viewer/viewer.html?doc=pdai/E57`	Document viewer
`/ai-viewer/viewer.html?doc=pdai/E57&page=10`	Specific page
`/ai-viewer/viewer.html?doc=pdai/E57&mode=agent`	Agent mode

Content URLs

URL	Description
`/pdai/E57/pages/page_000/page_000.png`	Page image
`/pdai/E57/pages/page_000/page_000.html`	Page HTML
`/pdai/E57/pages/page_000/page_000_translation_english.xml`	Translation
`/pdai/E57/manifest.json`	Document manifest

Document Discovery

Primary Source: pdai/index.json

The canonical document list:

{
  "documents": [
    {"name": "E57", "title": "Euler Tidal Theory", "pages": 105},
    {"name": "Goldbach", "title": "Euler-Goldbach Letters", "pages": 483}
  ]
}

Fallback: pdai-documents.json

Cached copy at project root for convenience.

Adding Documents

Process document with ai-scholar process
Add entry to pdai/index.json
Refresh viewer — document appears in home catalog

Best Practices

Document Preparation

Process documents — Use ai-scholar process for new documents
Migrate old documents — Use ai-scholar migrate for legacy documents
Segment early — Generate manifest.json for navigation
Validate structure — Use ai-scholar assess to check integrity

Viewing Workflow

Start simple — View original HTML first
Check segments — Verify navigation works correctly
Add layers — Run translation/annotation agents as needed
Compare outputs — Switch layers to compare processing results

Troubleshooting

Document Not Loading

# Check structure
ls -la ../pdai/DocumentName/pages/page_000/

# Verify required files
ls ../pdai/DocumentName/pages/page_000/page_000.png
ls ../pdai/DocumentName/pages/page_000/page_000.html

Missing Segments

# Run segmentation
ai-scholar segment ../pdai/DocumentName

# Check manifest
cat ../pdai/DocumentName/manifest.json | jq '.segments'

Agent Outputs Not Showing

# Check page folder
ls ../pdai/DocumentName/pages/page_000/

# Run agent if missing
ai-scholar agent ../pdai/DocumentName translation-english

Server 404 Errors

# Ensure server runs from project root
cd /path/to/2026project  # Parent of ai-viewer and pdai
python -m http.server 4100

Summary

The three-package architecture provides clean separation of concerns:

ai-scholar handles all AI processing
pdai stores data in page-centric format
ai-viewer presents content interactively via static HTTP

Data flows from processing through storage to presentation, with all page-related files organised together in pages/page_NNN/ folders.

--- title: "Three-Package Integration" subtitle: "Complete workflow across ai-scholar, pdai, and ai-viewer" --- ## Overview The AI document processing ecosystem consists of three interconnected packages, each with a distinct responsibility: | Package | Role | Primary Function | |---------|------|------------------| | **ai-scholar** | Processing | OCR, segmentation, translation, annotation | | **pdai** | Data | Structured document storage (page-centric) | | **ai-viewer** | Presentation | Interactive web-based viewing | This guide explains how they work together to transform PDFs into interactive, navigable documents. ## Architecture ```{mermaid} %%| fig-cap: "Data flow through the three-package architecture" flowchart TD subgraph Input["Source Documents"] PDF[PDF File] SCANS[Scanned Images] end subgraph Scholar["ai-scholar Package"] INGEST[Ingestion] OCR[OCR Agent] SEG[Segmenter Agent] TRANS[Translation Agent] ANNOT[Annotation Agent] end subgraph PDAI["pdai Package"] PAGES["pages/page_NNN/"] MANIFEST["manifest.json"] SEGMENTS["segments/"] end subgraph Viewer["ai-viewer Package"] HTML[viewer.html] UI[Interactive Viewer] end PDF --> INGEST SCANS --> INGEST INGEST --> PAGES PAGES --> OCR OCR --> PAGES PAGES --> SEG SEG --> MANIFEST PAGES --> TRANS PAGES --> ANNOT TRANS --> PAGES ANNOT --> PAGES PAGES --> HTML MANIFEST --> HTML HTML --> UI ``` ## Package Responsibilities ### ai-scholar: The Processing Engine AI-Scholar transforms raw documents into structured, annotated content: ```bash # Core processing commands ai-scholar process document.pdf # PDF → pages/page_NNN/ ai-scholar segment ../pdai/DocName # → manifest.json ai-scholar agent ../pdai/DocName translation-english # → pages/page_NNN/page_NNN_translation_english.xml ``` **Generates:** - `pages/page_NNN/page_NNN.png` — High-resolution page images - `pages/page_NNN/page_NNN.xml` — Structured OCR output (DTD-validated) - `pages/page_NNN/page_NNN.html` — Viewer-compatible HTML - `pages/page_NNN/page_NNN_fig_M.png` — Extracted figures - `pages/page_NNN/page_NNN_{agent}.xml` — Agent processing outputs - `manifest.json` — Document structure and segment mappings ### pdai: The Data Layer PDAI provides a standardised, page-centric directory structure: ``` pdai/DocumentName/ ├── DocumentName.pdf # Original source ├── manifest.json # Structure metadata ├── ai-orchestrator.yml # Agent configuration ├── css/ # Display styles ├── templates/ # Document-specific templates ├── pages/ # ALL page-level content │ ├── page_000/ │ │ ├── page_000.png │ │ ├── page_000.xml │ │ ├── page_000.html │ │ ├── page_000_translation_english.xml │ │ └── page_000_fig_1.png │ └── page_001/ │ └── ... └── segments/ # Materialised segments └── letters/ ├── letter_000/ └── letter_001/ ``` **Key principle:** All files for a page live together in `pages/page_NNN/`. No scattered directories. ### ai-viewer: The Presentation Layer AI-Viewer serves PDAI documents through an interactive web interface: ```bash # Start server from project root cd /path/to/2026project python -m http.server 4100 # Open viewer open "http://localhost:4100/ai-viewer/viewer.html?doc=pdai/DocumentName" ``` **Provides:** - Interactive page navigation (Document mode) - Agent output viewing (Agent mode) - Segment-based browsing (letters, chapters, paragraphs) - MathJax rendering for mathematics - Translation layer selection ## Complete Workflow ### Step 1: Ingestion Start with a PDF or folder of scanned images: ```bash # From PDF ai-scholar process ../pdai/E57/E57.pdf # From scans folder ai-scholar process ../scans/historical_doc/ ``` This creates the basic PDAI structure with page images in `pages/page_NNN/`. ### Step 2: OCR Processing Extract text from page images: ```bash ai-scholar process ../pdai/E57 --model gemini-3-flash-preview ``` Generates `pages/page_NNN/page_NNN.xml` with structured content: ```xml <page pidx="001"> <meta model="gemini-3-flash-preview" processed_date="2025-12-28"/> <main> <p><parno>1.</parno> Omnem mutationem quae...</p> <p><parno>2.</parno> Huius generis effectus...</p> </main> <footnotes> <footnote data-fn="1">Reference note...</footnote> </footnotes> </page> ``` ### Step 3: Segmentation Identify logical document structure: ```bash ai-scholar segment ../pdai/E57 ``` Creates `manifest.json` with segment mappings: ```json { "document": { "name": "E57", "title": "Inquisitio Physica...", "page_count": 105, }, "segments": { "chapters": { "items": [ {"id": "1", "number": 1, "title": "CAPUT PRIMUM", "pages": [1, 2, 3]} ] }, "paragraphs": { "items": [ {"id": "1", "number": 1, "pages": [1], "start_text": "Omnem mutationem..."} ] } } } ``` ### Step 4: Agent Processing Run translation, annotation, or custom agents: ```bash # Translate to English ai-scholar agent ../pdai/E57 translation-english --pages 0-50 # Annotate entities ai-scholar agent ../pdai/E57 annotate-entities --pages 0-50 ``` Outputs go to `pages/page_NNN/page_NNN_{agent}.xml`. ### Step 5: Viewing Serve the document with a simple HTTP server: ```bash # Start from project root cd /path/to/2026project python -m http.server 4100 # Open browser open "http://localhost:4100/ai-viewer/viewer.html?doc=pdai/E57" ``` Navigate to explore: - Browse pages with images and HTML side-by-side (Document mode) - View agent outputs in three-pane layout (Agent mode) - Navigate by segment (chapter, paragraph) - Switch between translation layers ## Segment Navigation The viewer uses `manifest.json` to enable segment-based navigation. ### How It Works 1. **Viewer loads manifest** — Reads segment definitions on document load 2. **Populates dropdowns** — Creates navigation controls for each segment type 3. **Jump to segment** — Selecting a segment navigates to its start page 4. **Display metadata** — Shows segment-specific information (dates, authors) ### Segment Types | Type | Use Case | Metadata | |------|----------|----------| | `chapters` | Scholarly treatises | Title, number, pages | | `paragraphs` | Numbered arguments | Number, start text, pages | | `letters` | Correspondence editions | Sender, recipient, date, place | | `documents` | Diplomatic acts | Parties, date, reference | ## Agent Output Layers The viewer can display different processing results. ### Available Layers After running agents, page folders contain multiple outputs: ``` pages/page_000/ ├── page_000.png ├── page_000.xml ├── page_000.html ├── page_000_translation_english.xml ├── page_000_translation_german.xml └── page_000_annotate_entities.xml ``` ### Viewing Layers **Document Mode:** - Use language dropdown to switch between Original, Deutsch, English **Agent Mode:** - Select agent from navigator to view output - Compare source input with agent output side-by-side ### Comparing Outputs 1. **Original vs Translation** — See how text was interpreted 2. **Multiple translations** — Compare English and German versions 3. **Annotation overlays** — View entity markup on original text ## Cross-Package Commands ### From ai-scholar ```bash # Full processing pipeline ai-scholar process document.pdf ai-scholar segment ../pdai/DocName ai-scholar agent ../pdai/DocName translation-english # Migrate legacy documents ai-scholar migrate ../pdai/DocName ai-scholar migrate-all ../pdai # Assessment ai-scholar assess ../pdai/DocName ``` ### From Project Root ```bash # Start server cd /path/to/2026project python -m http.server 4100 # View documents open "http://localhost:4100/ai-viewer/viewer.html" # Home catalog open "http://localhost:4100/ai-viewer/viewer.html?doc=pdai/E57" # Specific document ``` ## URL Structure ### Viewer URLs | URL | Description | |-----|-------------| | `/ai-viewer/viewer.html` | Home catalog | | `/ai-viewer/viewer.html?doc=pdai/E57` | Document viewer | | `/ai-viewer/viewer.html?doc=pdai/E57&page=10` | Specific page | | `/ai-viewer/viewer.html?doc=pdai/E57&mode=agent` | Agent mode | ### Content URLs | URL | Description | |-----|-------------| | `/pdai/E57/pages/page_000/page_000.png` | Page image | | `/pdai/E57/pages/page_000/page_000.html` | Page HTML | | `/pdai/E57/pages/page_000/page_000_translation_english.xml` | Translation | | `/pdai/E57/manifest.json` | Document manifest | ## Document Discovery ### Primary Source: pdai/index.json The canonical document list: ```json { "documents": [ {"name": "E57", "title": "Euler Tidal Theory", "pages": 105}, {"name": "Goldbach", "title": "Euler-Goldbach Letters", "pages": 483} ] } ``` ### Fallback: pdai-documents.json Cached copy at project root for convenience. ### Adding Documents 1. Process document with `ai-scholar process` 2. Add entry to `pdai/index.json` 3. Refresh viewer — document appears in home catalog ## Best Practices ### Document Preparation 1. **Process documents** — Use `ai-scholar process` for new documents 2. **Migrate old documents** — Use `ai-scholar migrate` for legacy documents 3. **Segment early** — Generate manifest.json for navigation 4. **Validate structure** — Use `ai-scholar assess` to check integrity ### Viewing Workflow 1. **Start simple** — View original HTML first 2. **Check segments** — Verify navigation works correctly 3. **Add layers** — Run translation/annotation agents as needed 4. **Compare outputs** — Switch layers to compare processing results ## Troubleshooting ### Document Not Loading ```bash # Check structure ls -la ../pdai/DocumentName/pages/page_000/ # Verify required files ls ../pdai/DocumentName/pages/page_000/page_000.png ls ../pdai/DocumentName/pages/page_000/page_000.html ``` ### Missing Segments ```bash # Run segmentation ai-scholar segment ../pdai/DocumentName # Check manifest cat ../pdai/DocumentName/manifest.json | jq '.segments' ``` ### Agent Outputs Not Showing ```bash # Check page folder ls ../pdai/DocumentName/pages/page_000/ # Run agent if missing ai-scholar agent ../pdai/DocumentName translation-english ``` ### Server 404 Errors ```bash # Ensure server runs from project root cd /path/to/2026project # Parent of ai-viewer and pdai python -m http.server 4100 ``` ## Summary The three-package architecture provides clean separation of concerns: - **ai-scholar** handles all AI processing - **pdai** stores data in page-centric format - **ai-viewer** presents content interactively via static HTTP Data flows from processing through storage to presentation, with all page-related files organised together in `pages/page_NNN/` folders.