Forge — LLM Batch & Realtime Orchestration
Two cooperating Node daemons coordinate LLM batch and realtime jobs through Postgres only. Most AI apps stop at calling an API; Forge handles the operational layer — batching, provider routing, leases, retries, crash recovery, quota recovery, validation, notifications, and observability.
At a glance
By the numbers
- ~40/s
- jobs per replica
- ~50%
- lower token cost
- 2
- LLM providers
- 9
- worker loops
- 0
- queue services
≈3.4M per day
provider Batch APIs
OpenAI + Anthropic
two cooperating daemons
Postgres-only coordination
What we were solving
Context & problem
LLM workloads need both batch and realtime execution, provider flexibility, retry safety, cost awareness, validation, and operational visibility. A queue-based architecture adds infrastructure and coordination complexity.
Forge explores a Postgres-centered architecture where workers coordinate through the database alone — no queue, no IPC, no shared worker processes.
