Architecture

Service-by-service map of what is actually running today (post wave 5 + 6 cutover, 2026-05-19).

Services (with reference hashes)

Use these 4-hex hashes when filing tickets or design notes — deterministic from the service name and disambiguate across repos.

The canonical view is the SVG diagram embedded at the top of README.html. The text representation that follows is preserved for terminal / accessibility reading.

Two pipelines, one DB

There is one source-of-truth corpus (corpus.db, on node-eighteen, mounted at /var/lib/fraud-data/corpus.db). Writer pipelines flow into it:

1. Crawler pipeline — crawler/ writes new documents.
2. Queue enrichment harness — fraud-publisher sweeps pending rows and publishes {doc_id, workflow} envelopes to fraud.rows; fraud-enricher-v2 (20 replicas) consumes them and decorates documents. 18 workflows. See enrichment.md.
3. Config-loop enrichment harness — fraud-enrich-worker runs the ENRICH_CONFIGS array in an in-process loop (CIK resolution, CourtListener captions, matter grouping, scheme classification). Self-selects via each config's selectorSql — no publisher. See enrichment-configs.md.

The legacy per-workflow stack-enrich-*.yml services (old in-process loops) are scaled to 0 as the queue harness took over their work — but note that fields they once filled (e.g. statute_refs, outcome) only continue to advance if the matching workflow is being published and consumed.

Every write goes through fraud-db-broker (Bun HTTP service on port 3100). It owns the single rw SQLite handle. Everyone else is read-only.

Enrichment coverage

The live /stats page renders the per-workflow populated/done/skipped table at the top. Conceptually it belongs here — the table is "how much of the architecture diagram has actually run end-to-end across the corpus". Each row maps one of the enrichment passes to a column on documents and a row class in workflow_runs. A small mirror of this still lives at the bottom of /stats as a progress-bar summary so the UI carries a "this view is X% populated" hint without forcing readers to leave the page.

The numbers update every time fraud-publisher flushes envelopes and fraud-enricher-v2 ACKs them. Inputs:

• documents.outcome IS NOT NULL → outcome-detect coverage
• documents.scheme IS NOT NULL → scheme-classify coverage
• documents.title_formatted IS NOT NULL → format-title coverage
• entities row count grouped by kind → entity-kind coverage
• workflow_runs.status per workflow → done / skipped totals

Data flow (current)

                  EDGAR  ─────────────────────┐
                  fraud-scan (node-eighteen)  │
                                              ▼
                                  flagged/*.ndjson  (separate corpus)

  DOJ / SEC sites ──▶ fraud-crawler-* ──▶ fraud-db-broker ──▶ corpus.db
                              (writes)        (single rw)        │
                                                                 │ readonly
                                                                 ▼
                            ┌─────────────────────────────────────────────────┐
                            │ fraud-publisher    (wave 5, node-eighteen)      │
                            │   sweeps documents where                        │
                            │     workflow.selectSql() matches                │
                            │     AND no terminal workflow_runs row exists    │
                            │   publishes {doc_id, workflow} envelope to      │
                            │     federation.work / fraud.rows                │
                            │   flips queued_at / queued_workflow / envelope_id│
                            └─────────────────────────────────────────────────┘
                                                 │
                                                 ▼ AMQP (federation broker on node-eleven)
                                       gpumon-rabbitmq (vhost: federation)
                                                 │  fraud.rows  ──▶ fraud.dlq
                                                 ▼
                            ┌─────────────────────────────────────────────────┐
                            │ fraud-enricher-v2  (wave 6, node-eighteen)      │
                            │   prefetch=1, max_attempts=5                    │
                            │   loads WorkflowDef, calls process(doc)         │
                            │   POSTs patch to broker /update or              │
                            │     /apply/<endpoint>                           │
                            │   writes workflow_runs row (status, attempts)   │
                            │   LLM/OCR calls go to gpumon-ingress:4001       │
                            │     (NIM → LAN GPU overflow → failover;         │
                            │      no direct Ollama/Haiku calls)              │
                            └─────────────────────────────────────────────────┘
                                                 │
                                                 ▼ rw via broker
                                              corpus.db

  search-ui  (Cloudflare → postcrime.atsignhandle.xyz) ──▶ corpus.db (RO)
  stats-miner  (node-eighteen :3200)                ──▶ corpus.db (RO)
  watchdog                                          ──▶ corpus.db (RO)

Running services (node-eleven swarm, 2026-05-19)

Live list:

ssh rooot@node-eleven 'docker service ls | grep -E "fraud|edgar"'

The federation queue (waves 5–6)

Why node-eighteen

corpus.db lives on node-eighteen's local SSD. Every primary fraud-* service pins to node.hostname == node-eighteen. Do not trust node.labels.fraud_corpus == true — that label doesn't actually exist on any swarm node despite some stack files referring to it (see _overnight-questions.md Q6). The canonical answer is hostname pinning.

Exception: fraud-processor-triples pins to node-eleven because that's where the four pool-fraud-triples GPU backends live with line-of-sight; fraud-tesseract-sidecar fans out to any fraud-ocr=true node.

Image distribution

There is no Docker registry. After rebuilding a fraud service:

docker save fraud-enricher-v2:latest | ssh rooot@node-eighteen docker load
ssh rooot@node-eleven 'docker service update --force fraud_fraud-enricher-v2'

You must docker save | ssh ... docker load to every node the task can be scheduled on, then --force the service update. See docker/README.md and _overnight-questions.md Q10 ("image digests aren't enough — verify routes after broker rebuild").

Public routing

https://postcrime.atsignhandle.xyz → Cloudflare → node-eighteen:7850 (fraud_fraud-search).

GAP: there is no public surface for fraud-enricher-v2 health or queue depth. RabbitMQ management UI is reachable inside the cluster but not exposed via Cloudflare.

Licensing & Consulting

This platform — the crawler/enrichment pipeline, the anomaly-detection heuristics, the scheme-detection rule banks, and the enriched corpus — is research software under active development. It is provided as-is, with no warranty and no claim of fitness for any legal, financial, or investment purpose. Nothing surfaced here is advice; outputs are probabilistic signals, not findings of wrongdoing.

Licensing and consulting are available for serious inquiries. Typical engagements:

• Source / data licensing — access to the enriched corpus, the heuristic and scheme-rule definitions, or the per-source crawlers under a commercial license.
• Detection consulting — applying the precrime heuristics to a specific filer universe, custom rule banks, or integration with an existing surveillance/compliance stack.
• Pipeline build-out — standing up the SEC/DOJ ingest → enrichment → anomaly-scoring stack against another jurisdiction or document corpus.

Serious inquiries only. Reach out through the contact channel listed on the landing page; please include the use case, the data scope, and whether the engagement is licensing or consulting.