Obelisk argues SQLite and Litestream offer superior durable execution for AI agents

A recent blog post from Obelisk argues that SQLite, augmented by Litestream for asynchronous replication to object storage, provides a sufficient and cost-effective foundation for durable execution in AI agent and experimentation workflows. The author extends a previous argument made by DBOS, which advocated for PostgreSQL as the primary requirement for durable systems, by suggesting that SQLite is the optimal choice for a large class of durable architectures.

The proposed model involves running an Obelisk server with a local SQLite database, using Litestream to stream changes asynchronously to S3-compatible object storage. This approach is positioned as particularly suitable for AI agents and AI-generated workflows, which are described as bursty and experimental. The architecture allows for local replay, debugging, and inspection of agent actions using the same database file, offering better fault isolation and lower operational complexity than shared databases.

Durable execution typically involves persisting workflow state so that compute resources can remain cheap and disposable, with workflows replaying from persisted history. The Obelisk architecture leverages this by keeping workflow progress in an execution log. By using a local database file, the system avoids the network hops, extra control planes, and additional operational surface area associated with separate database services.

The author acknowledges that Litestream replication is asynchronous, meaning a restore can miss the newest local writes if the SQLite volume disappears before they are copied to object storage. While this poses a risk for highly available shared databases, it is considered acceptable for many AI and experimentation workflows where the tolerance for minor data loss during replication windows is higher.

Despite the recommendation for SQLite, the author notes that PostgreSQL remains the correct choice for requirements involving higher availability, broader shared scalability, or synchronous durability models. Obelisk supports both databases, allowing developers to select the appropriate tool based on whether they need a network database or a local file-based system.

For a significant number of deployments, the combination of a local SQLite database and Litestream backup to S3 provides a durable system with minimal infrastructure. This model is especially attractive for AI agents, which benefit from small, self-contained units of state. A fleet of tiny servers in micro VMs or containers, each with its own SQLite database and object storage backup, is often simpler and cheaper than maintaining one large always-on shared system.

The article concludes that many workflow systems do not require complex infrastructure on day one and should avoid starting with more resources than their state actually demands. For the world of AI agents, the local SQLite plus Litestream model may represent the most sensible default, offering durability without the overhead of traditional enterprise database architectures.