Sync

Overview

The sync subsystem is the bridge between a local nikcli process and the optional remote hub (such as https://s.nikcli-ai.dev). It turns the ephemeral bus events a session produces into a durable, ordered, replayable log, then ships that log over HTTP when (and only when) a remote is configured.

Sync is opt-in. With no NIKCLI_REMOTE_URL and no sync block in the global config file, the local event log and projections still exist, but no outbound transport runs. Nothing about the local CLI, TUI, or server depends on the hub being reachable.

What gets synced

Anything published on GlobalBus is fair game, but the persisted log filters that stream down to the aggregates that are useful to replay on another machine. The reducer in src/sync/reducer.ts maps each event type to its aggregate and project id.

Event log

src/sync/sync.sql.ts defines two SQLite tables under the data root: syncEvent (one row per event) and syncSequence (last-seen seq per project + aggregate). The storage layer in sync/index.ts exposes load/save helpers and applies two practical limits:

Compaction rewrites the table transactionally: it deletes all rows for a project and re-inserts the surviving window in one transaction, so a crash mid-compaction cannot leave a partial log. saveEvents and saveSequence are the only writers outside projection tests.

Projections and snapshots

Projections turn the log into typed state. There are two complementary paths:

1. Live projection in src/sync/projector.ts and src/sync/projection.ts: a per-aggregate reducer updates in-memory state on every event. TUI routes and the SDK sync provider subscribe to it.
2. Cold-start snapshot in src/sync/snapshot.ts: when the SDK reconnects (cold start, server restart, app foreground), it asks the server for a single typed snapshot rather than replaying thousands of events. Snapshots are produced from the same projection state, so live and cold-start paths converge.

# Cold-start endpoint used by SDK clients
curl "http://127.0.0.1:4096/sync/snapshot?directory=$PWD"

Outbox and idempotency

The outbox (src/sync/outbox.ts) is the only writer that talks to the hub. It exists so a crash between "event written" and "hub acknowledged" never produces a duplicate or a missed event.

nikcli sync status
# target:        https://s.nikcli-ai.dev (config file)
# outbox pending: 12
# outbox failed:  0
# outbox total:   1024

Remote transport

The remote client (src/sync/remote-client.ts) batches confirmed outbox entries and POSTs them to the hub's /sync/push endpoint. The remote sync loop (src/sync/remote-sync.ts) wires that client to the outbox and projection:

• Reads the local projection state at startup to catch up after a crash.
• Subscribes to the live event stream for new entries.
• Reconciles after transient failures using stored seq counters.

CLI command

nikcli sync is the user-facing entrypoint. It exposes status, an explicit connect/disconnect flow, and a token issuer for new CLI machines.

# Set the target hub for this shell
export NIKCLI_REMOTE_URL=https://s.nikcli-ai.dev
export NIKCLI_REMOTE_TOKEN=<cli-sync token>

nikcli sync status              # outbox state + source of truth
nikcli sync connect             # force a connection (Ctrl-C to disconnect)
nikcli sync disconnect          # how to end the connect session
nikcli sync token create --name laptop --expiry-days 90

TUI sync dialog

The TUI exposes sync through dialog-sync.tsx and the remote-sync context. From the command palette the user can:

• Save URL + token to the global config (written through POST /sync/config).
• Watch live stats polled from /sync/stats every two seconds.
• Toggle the autostart flag so nikcli connects on bootstrap.
• Inspect outbox pending / failed counts and trigger a reconnect.

SyncConfig.invalidate() is called after every config write so the next resolve is fresh, even though the read path caches values for five seconds.

Auth, scopes, rate limits

Tokens used for hub push are minted by the host's mobile auth flow. There are two distinct scopes:

Hub endpoints enforce scope at the route group level and apply a per-token rate limit (sliding window) and an audit log entry per push. The audit log records tokenID, aggregate count, and seq range so the CLI side can correlate its own outbox state with what the hub saw.

Bootstrapping

The bootstrap flow decides whether to autostart sync. Two flags are honored, in priority order:

1. NIKCLI_REMOTE_AUTOSTART=1 — force autostart regardless of config.
2. config.sync.autostart: true — autostart when both URL and token are present.

With either set and both URL + token resolved, RemoteSync.start runs once per nikcli bootstrap. The CLI keeps the sync loop running alongside the TUI or session; --print-logs surfaces retry / failure messages from the hub client.

Failure modes

Source anchors