Architecture

Ekklesia is structured as three independently deployable services connected through well-defined interfaces. The browser talks only to the frontend service; the frontend proxies API calls through nginx; the API orchestrates the agent pipeline against the database.

System diagram

graph TB
    subgraph Browser
        UI[React App - Vite / Tailwind]
    end

    subgraph Frontend [Frontend Service - nginx]
        NGINX[nginx reverse proxy]
    end

    subgraph API [API Service - FastAPI]
        APP[FastAPI app.py]
        HEALTH[GET /health]
        SERMONS[POST /sermons - SSE]
        REVISE[POST /sermons/revise]
        LOOKUP[GET /lookup and /passage]
        ORCH[Orchestrator]
    end

    subgraph Agents [Agent Pipeline - Pydantic AI]
        PLAN[Planner]
        RES[Research]
        EXE[Exegesis]
        STR[Structure]
        CRIT[Critique]
        REV[Revision]
    end

    subgraph Retrieval
        HYBRID[Hybrid Search]
        EMBED[Embedding]
        PASSAGE[Passage Lookup]
        LEX[Lexicon Lookup]
        XREF[Cross-Reference]
    end

    subgraph Storage [Storage - PostgreSQL + pgvector]
        PG[(PostgreSQL)]
        BIBLE[bible_passages / bible_verses]
        COMM[commentary_chunks / commentary_parents]
        CULT[cultural_entries]
        LXDB[lexicon_entries]
        XRDB[cross_references]
    end

    subgraph External
        GEMINI[Google Gemini API]
        LOGFIRE[Logfire]
    end

    UI -->|POST /sermons| NGINX
    UI -->|POST /sermons/revise| NGINX
    UI -->|GET /passage| NGINX
    UI -->|GET /lookup| NGINX
    NGINX -->|proxy_pass| APP
    APP --> HEALTH
    APP --> SERMONS
    APP --> REVISE
    APP --> LOOKUP
    SERMONS --> ORCH
    REVISE --> REV
    ORCH --> PLAN
    ORCH --> RES
    ORCH --> EXE
    ORCH --> STR
    ORCH --> CRIT
    REV -->|LLM| GEMINI
    RES --> HYBRID
    EXE --> HYBRID
    HYBRID --> EMBED
    HYBRID --> PG
    EMBED -->|HTTP| GEMINI
    PLAN -->|LLM| GEMINI
    RES -->|LLM| GEMINI
    EXE -->|LLM| GEMINI
    STR -->|LLM| GEMINI
    CRIT -->|LLM| GEMINI
    LOOKUP --> PASSAGE
    LOOKUP --> LEX
    LOOKUP --> XREF
    PASSAGE --> PG
    LEX --> PG
    XREF --> PG
    PG --> BIBLE
    PG --> COMM
    PG --> CULT
    PG --> LXDB
    PG --> XRDB
    APP -->|spans| LOGFIRE

Components

API layer (src/ekklesia/api/)

File	Responsibility
app.py	Application factory; Logfire bootstrap; CORS middleware; router registration
deps.py	get_session FastAPI dependency — yields an async SQLAlchemy session per request
routers/health.py	GET /health — returns {"status": "ok"}
routers/sermons.py	POST /sermons — SSE stream via EventSourceResponse; POST /sermons/revise — applies critique revisions via RevisionResult
routers/lookup.py	/lookup/strongs/{id} and /passage/{reference:path} — thin HTTP façades over retrieval functions

Agent pipeline (src/ekklesia/agents/)

File	Responsibility
orchestrator.py	Runs all five agents in sequence; emits StageEvent objects; Logfire root span
planner.py	Produces a SermonPlan — passage reference, audience, main question, research/exegesis/structure goals
research.py	Calls hybrid search tools; produces ResearchFindings with passage summary, themes, cross-refs, commentary highlights
exegesis.py	Deep analysis of key scriptures; produces ExegesisNotes with word studies and theological moves
structure.py	Produces a SermonOutline — sections, transitions, illustrative content
critique.py	Reviews the full draft; produces CritiqueReport with strengths and suggestions
revision.py	Applies CritiqueReport.recommended_revisions to the final markdown; produces RevisionResult — invoked on-demand via POST /sermons/revise, not as part of the pipeline
deps.py	AgentDeps dataclass — holds SermonDraft state + DB session injected into each agent
state.py	SermonDraft — mutable accumulator updated by each stage
events.py	StageEvent Pydantic model; StageName and StageStatus literals
tools/	Reusable Pydantic AI tools: lexicon_tool, cross_reference_tool

Retrieval layer (src/ekklesia/retrieval/)

File	Responsibility
hybrid.py	Runs dense + sparse queries per source type in parallel; fuses with RRF
embedding.py	embed_query() — async call to Gemini gemini-embedding-001; 768-dim output
passage.py	Parses a reference string (e.g. Romans 8:28) and fetches verses from bible_verses
lexicon.py	lookup_strongs() — point lookup on lexicon_entries by Strong's number
cross_reference.py	Returns related verse IDs from cross_references
reference.py	Reference string normalisation utilities
verse_id.py	Converts book/chapter/verse tuples to canonical verse IDs

Data layer (src/ekklesia/)

File	Responsibility
db.py	SQLAlchemy Table definitions; HNSW and GIN indexes; create_engine()
config.py	Settings (pydantic-settings); reads .env; normalises DATABASE_URL scheme
models/	Shared Pydantic models: RetrievalResult, LexiconEntry, SourceType, primitives
exceptions.py	Domain exceptions: InvalidReference, PassageNotFound, LexiconEntryNotFound

Ingestion scripts (scripts/)

Script	Purpose
ingest_bible.py	Loads ESV OSIS XML → bible_passages + bible_verses; calls Gemini batch embedding API
ingest_commentary.py	Loads commentary JSON → commentary_chunks + commentary_parents
ingest_lexicon.py	Loads Strong's lexicon JSON → lexicon_entries
ingest_cross_references.py	Loads TSV cross-reference data → cross_references

Key design decisions

Why SSE instead of WebSockets? SSE is unidirectional (server → client) and maps naturally to a one-way streaming pipeline. It works over plain HTTP/1.1, requires no upgrade handshake, and reconnects automatically in the browser. WebSockets add complexity with no benefit for this use case.

Why Pydantic AI? It enforces structured outputs at the type level — each agent returns a validated Pydantic model, not a raw string. This makes agent outputs composable (the planner's output is the next agent's input) and keeps mypy happy end-to-end.

Why pgvector over a dedicated vector database? The corpus is medium-sized (tens of thousands of passages) and shares a relational schema with lexicon, cross-reference, and verse data. Keeping everything in one PostgreSQL instance simplifies transactions, backup, and deployment. HNSW indexes give sub-millisecond ANN search at this scale.

Why Gemini embeddings over local sentence-transformers? The gemini-embedding-001 model produces 768-dim embeddings with significantly better semantic coverage for biblical/theological language than all-MiniLM-L6-v2 (384-dim). The latency cost at ingestion time is acceptable; query-time latency is ~100ms per embed call.

Why hybrid (dense + sparse) retrieval? Dense search finds semantically similar passages even when the exact words differ. Sparse search (PostgreSQL full-text search with websearch_to_tsquery) finds exact word matches and proper nouns (book names, people, places). Neither alone is sufficient for theological research; RRF fusion captures the best of both.

Why Logfire? Logfire is OpenTelemetry-native and integrates directly with Pydantic AI. A single logfire.instrument_pydantic_ai() call captures every LLM call with token counts, tool use, and latency — without any manual instrumentation inside agent code.