# SQLite + Lua Cursor Machine

Status: **SHIPPED** deterministic SQLite scheduler model, restricted-Lua compiler boundary, batched worker RPC, Linda tuples, transactional outbox/receipts, inspection views, metrics, watch-party demo, official SQLite WASM Worker with memory/OPFS/SAH-pool backends, bounded read-only authorizer, local `ATTACH`, semantic Durable Object sync admission, and a live cockpit proof flight. **EXPERIMENTAL** Node SQLite host, generated relational projections, SQLite Session envelope shape, and the Nelua/C/Wasm portable ABI. **PLANNED** embedded general Lua VM, measured Lua worker pool, native virtual tables, Cap'n Web effect adapter, and remote partition integration.

SQLite and Lua are one **Cursor Body runtime**, but they have deliberately different jobs. SQLite is the scheduling, state, query, lease, coordination, and commit authority. Lua supplies coarse-grained program meaning. Cursors sits above both and turns explicit Lua execution boundaries into capability-scoped, serializable, resumable continuations.

```mermaid
flowchart LR
  UI["UI / cockpit"] -->|"one batched MessagePort RPC"| SW["SQLite scheduler Worker"]
  SW --> DB["SQLite queues + tuples + checkpoints + outbox"]
  SW -->|"immutable task packets"| LW["restricted executor / future Lua VM worker(s)"]
  LW -->|"batched semantic intents"| SW
  DB --> FX["post-commit effect executor"]
  FX -->|"Cap'n Web pipeline / mailbox sync"| REM["remote body or DO SQLite"]
  REM -->|"stable receipt"| DB
```

The UI thread never opens the writable database or resumes Lua. The shipped browser slice uses a dedicated WebWorker owning official SQLite WASM, its selected local VFS, and restricted-Lua compilation. The complete scheduler model currently uses a coarse executor adapter; an embedded general Lua VM remains planned. A native body uses an owning thread or process. Mobile uses a background-capable task with the same bounded request protocol. Cloudflare is a semantic adapter: Durable Object SQLite stores mailbox/rendezvous/anti-entropy records, but cannot impersonate the page's embedded SQLite C API or replicate its pages.

## What runs today

`src/cursor-machine-schema.mjs` defines indexed strict tables for programs, continuations, workers, body capabilities, tuple assertions/interests, effects, receipts, checkpoints, and instrumentation. Generated tuple columns (`kind` and `room`) make the watch-party match path SQL-indexable instead of a JavaScript scan. `cursor_machine_state` and `live_tuples` are cockpit/devtools inspection views.

`src/cursor-machine.mjs` installs source plus compiled IR, claims runnable continuations in an indexed SQL batch under `BEGIN IMMEDIATE`, requires a live worker heartbeat, assigns task leases, and commits each returned intent batch atomically. A failed intent rolls back the checkpoint and continuation advancement. Expired local leases are reconciled explicitly; no global-clock ordering is claimed.

The runnable query expands each program's declared capability list with `json_each` and excludes bodies without a matching unexpired grant. A deterministic, direct-only `cursor_stable_id` function is registered when the host SQLite API supports application functions. `attachLocal(path, schema)` exposes SQLite `ATTACH` for explicitly selected local databases after validating the schema identifier; it is never available to Lua source. Generated tuple projections provide the current vtable-like indexed query surface without per-row Lua callbacks. `installRestrictedQueryAuthorizer` configures a separate query connection to allow only `SELECT`, deterministic functions, and reads from capability-selected tables; mutations, schema operations, and `ATTACH` are denied. The scheduler connection itself retains those operations.

The execution crossing is deliberately coarse:

1. SQLite selects and claims up to `N` ready continuations.
2. The scheduler sends immutable task packets grouped by the selected batch.
3. Lua advances each task until an explicit Cursors primitive.
4. Lua returns intents such as assertion, interest, checkpoint, sleep, join, or effect requirement.
5. SQLite applies the intents and successor state in a transaction.
6. Only after commit may an effect executor claim a leased outbox batch and perform network or device work.
7. A stable effect ID deduplicates byte-equivalent retries and rejects conflicting semantic reuse; a receipt requires the live outbox claim. Receipt persistence and continuation wakeup commit together, and the continuation wakes only after all of its effects are received.

This avoids `SQLite row → Lua → SQLite row` callback thrash. SQL does relational selection and batching; Lua sees complete continuation tasks.

## Restricted Lua and portable continuations

Original Lua remains a durable artifact with its hash, profile, compiler version, required capabilities, and compiled IR. The current compiler boundary rejects ambient loaders and native/opaque escape hatches (`load`, `require`, `package`, `debug`, `io`, `os`, and FFI) and records recognized `cursors.*` yield primitives. It is intentionally a front-end boundary, not yet a hardened Lua parser or sandbox.

Durable state is `{ point, serializable locals, cursor, capabilities }`, never a Lua heap or C stack snapshot. The planned embedded VM may optimize between yield points, but recovery always begins from the explicit continuation state. Untrusted mailbox text is data; it is never passed to `eval` or installed as a program.

The portable Nelua core validates and executes the fixed 32-byte scheduler task header through a C-friendly ABI. JS produces the same frozen fixtures and expected result packets. Nelua still compiles to C; Emscripten produces scalar and SIMD Wasm artifacts, and CI executes both before accepting them. The ongoing Rust comparison in issue #50 can implement the same bytes and fixtures without blocking this path.

## Workers and promise pipelines

`src/cursor-machine-worker.mjs` accepts an array of calls in one MessagePort-shaped request. Results may reference earlier results in the same batch, so `compile(source) → install(program) → spawn(args) → tick()` does not require four UI round trips. This is the local equivalent of Cap'n Web promise pipelining. Live RPC accelerates a body; losing the port loses no accepted continuation because SQLite remains authoritative.

Start with one scheduler Worker and one coarse executor. A safe pool keeps the SQLite connection in the scheduler Worker, sends immutable packets to `N` Lua Workers, then serializes returned commits. Worker leases and cursor/effect idempotency make duplicate execution observable and safe. Pooling should ship only after measurements find a break-even point; one SQLite connection must not be concurrently driven by unrelated Workers.

Platform mapping:

| Host | SQLite/Lua owner | Durable storage | Wake mechanism |
| --- | --- | --- | --- |
| Browser | dedicated WebWorker | official SQLite WASM + pluggable OPFS VFS | MessagePort/BroadcastChannel hint |
| Native | thread or isolated process | embedded SQLite file/WAL | local async queue |
| Mobile | background-capable worker/task | app-owned SQLite | OS background wake, best effort |
| Cloudflare | Durable Object/Worker semantic adapter | DO SQLite mailbox/rendezvous records | request, alarm, or WebSocket event |

No browser VFS synchronously pages over a network. Official SQLite WASM is the baseline; Roy Hashimoto's `wa-sqlite` VFS work remains an important benchmark source. VFS choice is a measured local storage decision, not a protocol identity.

## Linda, effects, and Cloudflare sync

Assertions and interests are attributed leased tuples. Hot match fields can become generated columns or materialized projections; application predicates should compile to SQL where practical rather than call Lua once per row. Registered deterministic SQLite functions, authorizers, update hooks/session changesets, `ATTACH`, virtual tables, FTS, and incremental blobs remain available host tools, but each needs a concrete workload and security boundary before being enabled.

Local-to-DO synchronization moves semantic records, never pages:

- cursor/checkpoint records and causal high-water marks;
- tuple assertions plus lease/reconciliation metadata;
- mailbox envelopes, tombstones, and bounded anti-entropy ranges;
- effect intents and stable receipts.

Every record has a stable ID. Reconnect replays a bounded journal; duplicates return prior receipts. DO SQLite remains the remote mailbox/rendezvous body, not page replication, and this design adds no paid Cloudflare product.

## Watch party and measurements

Run `node examples/watch-party-cursor-machine.mjs`. The restricted source asserts leased presence, waits on a room playback interest, and checkpoints explicit playback state. The demo uses an in-memory SQLite host so it is deterministic and dependency-free. In the cockpit, open **Cursor Apps → Lua + SQLite Machine Lab** and press **Run Proof Flight**; `/cursor-machine-proof.html` runs the identical component without the Open MCT shell. See [the lab guide](cursor-machine-cockpit-lab.md).

Run `npm run benchmark:cursor-machine -- 2000 64 1` for the scheduler microbenchmark; the last argument simulates the Lua worker-pool width. It reports throughput, batch crossings, and SQLite transaction counts. Pool comparisons remain scheduler simulations until embedded Lua workers are available. One earlier local sweep over 2,000 checkpointing continuations measured batch 1 at **2,051 resumes/s and 2 crossings/continuation**, batch 32 at **9,020 resumes/s and 0.063 crossings/continuation**, batch 64 at **10,109 resumes/s and 0.032 crossings/continuation**, and batch 128 at **10,968 resumes/s and 0.016 crossings/continuation**. These are development-machine, in-memory Node SQLite numbers—not browser/OPFS, embedded-Lua, network, or production claims.

The benchmark records claim time in `machine_metrics`. The cockpit-facing model exposes runnable/sleeping/leased/executing-or-awaiting states, continuation depth, worker occupancy, and effect status. Planned timers split SQL, Lua, and network time and add transaction rate, lease-renew cost, local outbox latency, Cap'n Web pipeline versus sequential REST, VFS comparison, and single-worker versus pool break-even.

## Cursor Worlds

A future Cursor World is an inspectable horizon of source artifacts, compiled IR, SQLite databases, continuation graph, capabilities, and tuple/object projections. A Squeak/Smalltalk-like browser can edit Lua, inspect tables and live bodies, and navigate causal checkpoints. Edits create new source artifacts and cursors; they do not invisibly mutate a suspended VM heap. A native or GNU Smalltalk-like host/editor could present the same records later without becoming a runtime dependency.

## Safety and current limits

- **SHIPPED:** deterministic tests cover racing claims, capability-gated scheduling, duplicate claims/results, continuation and effect lease expiry, worker death, explicit-state resume, rollback, duplicate receipt, and transactional receipt wakeup semantics.
- **EXPERIMENTAL:** the restricted compiler is a conservative recognizer, not a complete Lua AST validator; the complete scheduler host currently uses Node's real SQLite API, while the browser Worker supplies the portable transactional body slice.
- **PLANNED:** embedded general Lua, effect adapters, enabled Session changesets, native vtables, a measured worker pool, and remote partition integration tests.

Issue #44 owns the SQLite+Lua Cursor Machine direction. Issue #50 independently compares Nelua and Rust portable implementations. Issue #42 remains complementary work on Cap'n Web/worker pipelining rather than a dependency for durable execution.