# Sync

How nikcli journals local events, projects them into typed snapshots, ships them through an idempotent outbox, and replays them onto an optional remote hub.

## Overview

The sync subsystem is the bridge between a local nikcli process and the optional **remote hub** (such as `https://s.nikcli-ai.dev`). It turns the ephemeral bus events a session produces into a durable, ordered, replayable log, then ships that log over HTTP when (and only when) a remote is configured.

Sync is **opt-in**. With no `NIKCLI_REMOTE_URL` and no `sync` block in the global config file, the local event log and projections still exist, but no outbound transport runs. Nothing about the local CLI, TUI, or server depends on the hub being reachable.

Durable

### Local event log

Append-only per-project event store backed by SQLite (`syncEvent`, `syncSequence`).

Typed

### Projections

Per-aggregate reducers rebuild typed state (`session`, `message`, `permission`, …) from the log.

Reliable

### Outbox

Outgoing events are kept locally until the hub confirms them, so retries are idempotent and crash-safe.

Optional

### Hub transport

Batched push of confirmed events with token-scope checks, rate limiting, and audit logs.

Sync is layered, not coupled

The event log, projections, outbox, and remote transport are independent. You can disable only the hub transport by clearing `sync.url` / `NIKCLI_REMOTE_URL` while keeping local journaling for fast cold-starts and snapshot restores.

## What gets synced

Anything published on `GlobalBus` is fair game, but the persisted log filters that stream down to the aggregates that are useful to replay on another machine. The reducer in `src/sync/reducer.ts` maps each event type to its aggregate and project id.

| Aggregate | Examples | Used for |
| --- | --- | --- |
| `session` | Session created, updated, archived, deleted, parent changed. | Project-wide session list, mobile cold-start. |
| `message` | User / assistant / tool parts streamed, edited, or removed. | Transcript replay and snapshot restore. |
| `permission` | Ask, reply, auto-allowed rules. | Hydrating pending approval bars after reconnect. |
| `question` | Ask, reply. | Hydrating open questions across reconnects. |
| `todo` | Todo list updates and notifications. | Cross-device task state. |
| `workspace` | Workspace connection status and lifecycle events. | Remote workspace dashboards. |

Per-project scope

Events carry `projectID` and (when applicable) `workspaceID`. The local log and the hub both key on these, so two projects in the same data root never see each other's events.

## Event log

`src/sync/sync.sql.ts` defines two SQLite tables under the data root: `syncEvent` (one row per event) and `syncSequence` (last-seen seq per project + aggregate). The storage layer in `sync/index.ts` exposes load/save helpers and applies two practical limits:

| Setting | Value | Why |
| --- | --- | --- |
| `MAX_EVENTS_PER_AGGREGATE` | `1000` | Caps per-aggregate growth between compactions. |
| `COMPACTION_TRIM_TO` | `500` | After compaction, keep 500 most recent events per aggregate. |

Compaction rewrites the table transactionally: it deletes all rows for a project and re-inserts the surviving window in one transaction, so a crash mid-compaction cannot leave a partial log. `saveEvents` and `saveSequence` are the only writers outside projection tests.

## Projections and snapshots

Projections turn the log into typed state. There are two complementary paths:

1.  **Live projection** in `src/sync/projector.ts` and `src/sync/projection.ts`: a per-aggregate reducer updates in-memory state on every event. TUI routes and the SDK `sync` provider subscribe to it.
2.  **Cold-start snapshot** in `src/sync/snapshot.ts`: when the SDK reconnects (cold start, server restart, app foreground), it asks the server for a single typed snapshot rather than replaying thousands of events. Snapshots are produced from the same projection state, so live and cold-start paths converge.

Terminal

bash 

```
# Cold-start endpoint used by SDK clients
curl "http://127.0.0.1:4096/sync/snapshot?directory=$PWD"
```

$ shell · paste into your terminal

Hot reload aware

Projections are reset on `SIGUSR2` worker reloads — the next event simply rebuilds state from the surviving event log, and snapshot consumers refresh on reconnect.

## Outbox and idempotency

The outbox (`src/sync/outbox.ts`) is the only writer that talks to the hub. It exists so a crash between "event written" and "hub acknowledged" never produces a duplicate or a missed event.

| Concept | Implementation | Guarantee |
| --- | --- | --- |
| Idempotency | Each event carries a stable `id` plus `origin` / `originSeq`. | The hub can dedup replays on the same key. |
| Per-aggregate seq | `syncSequence` tracks the last seq the hub confirmed per aggregate. | Outbox only pushes newer events after each ack. |
| Retry | Failed batches stay in the outbox with backoff metadata; retries are bounded by scope and rate-limit windows. | Process crashes, network blips, and hub restarts do not lose events. |

Terminal

bash 

```
nikcli sync status
# target:        https://s.nikcli-ai.dev (config file)
# outbox pending: 12
# outbox failed:  0
# outbox total:   1024
```

$ shell · paste into your terminal

## Remote transport

The remote client (`src/sync/remote-client.ts`) batches confirmed outbox entries and POSTs them to the hub's `/sync/push` endpoint. The remote sync loop (`src/sync/remote-sync.ts`) wires that client to the outbox and projection:

-   Reads the local projection state at startup to catch up after a crash.
-   Subscribes to the live event stream for new entries.
-   Reconciles after transient failures using stored seq counters.

Hub is optional

Without a configured hub, the transport loop never starts. Local journaling, projections, and snapshot endpoints continue to work — they just don't leave the machine.

## CLI command

`nikcli sync` is the user-facing entrypoint. It exposes status, an explicit connect/disconnect flow, and a token issuer for new CLI machines.

Terminal

bash 

```
# Set the target hub for this shell
export NIKCLI_REMOTE_URL=https://s.nikcli-ai.dev
export NIKCLI_REMOTE_TOKEN=<cli-sync token>

nikcli sync status              # outbox state + source of truth
nikcli sync connect             # force a connection (Ctrl-C to disconnect)
nikcli sync disconnect          # how to end the connect session
nikcli sync token create --name laptop --expiry-days 90
```

$ shell · paste into your terminal

| Subcommand | Behavior |
| --- | --- |
| `status` | Prints resolved URL/token source, plus pending / failed / total outbox counts. |
| `connect` | Boots the remote sync loop and keeps it running until interrupted. |
| `disconnect` | There is no separate stop API — ending the connect process is the way to disconnect. |
| `token create` | Issues a `cli-sync` scoped mobile token for a new CLI machine. |

## TUI sync dialog

The TUI exposes sync through `dialog-sync.tsx` and the `remote-sync` context. From the command palette the user can:

-   Save URL + token to the global config (written through `POST /sync/config`).
-   Watch live stats polled from `/sync/stats` every two seconds.
-   Toggle the `autostart` flag so `nikcli` connects on bootstrap.
-   Inspect outbox pending / failed counts and trigger a reconnect.

`SyncConfig.invalidate()` is called after every config write so the next resolve is fresh, even though the read path caches values for five seconds.

## Auth, scopes, rate limits

Tokens used for hub push are minted by the host's mobile auth flow. There are two distinct scopes:

| Scope | Issued by | Permits |
| --- | --- | --- |
| `mobile` | `nikcli mobile pair` / `serve --pair` | Full phone workflow: bootstrap, sessions, repos, git, terminals, approvals, routines. |
| `cli-sync` | `nikcli sync token create` | Hub push only. Cannot open sessions or read messages. |

Hub endpoints enforce scope at the route group level and apply a per-token rate limit (sliding window) and an audit log entry per push. The audit log records `tokenID`, aggregate count, and seq range so the CLI side can correlate its own outbox state with what the hub saw.

Use the narrowest scope

Use `cli-sync` tokens on headless machines. `mobile` tokens are full-power and should stay on phones or paired devices only.

## Bootstrapping

The bootstrap flow decides whether to autostart sync. Two flags are honored, in priority order:

1.  `NIKCLI_REMOTE_AUTOSTART=1` — force autostart regardless of config.
2.  `config.sync.autostart: true` — autostart when both URL and token are present.

With either set and both URL + token resolved, `RemoteSync.start` runs once per `nikcli` bootstrap. The CLI keeps the sync loop running alongside the TUI or session; `--print-logs` surfaces retry / failure messages from the hub client.

## Failure modes

| Symptom | Likely cause | Fix |
| --- | --- | --- |
| `remote sync not configured` | Neither env vars nor `sync` block are populated. | Set `NIKCLI_REMOTE_URL` + `NIKCLI_REMOTE_TOKEN`, or use the TUI /sync dialog. |
| `outbox failed` grows on every push | Hub rejected the token or exceeded the rate limit. | Check token scope, rotate via `nikcli sync token create`, then reconnect. |
| TUI stats seem stale | Config cache TTL is five seconds; recent writes may not have flushed. | Reopen the dialog or call `SyncConfig.invalidate()` after a write. |
| Cold start missing sessions | Hub snapshot returned an older revision than the local log. | Verify `projectID` matches; the local log is still the source of truth. |
| Compaction truncated events I needed | An aggregate exceeded `MAX_EVENTS_PER_AGGREGATE`. | Take a snapshot first; the next compaction keeps the latest 500 events plus the snapshot. |
| Hot reload drops live state | Worker reload rebuilds projection state from the event log. | Wait for the next event; reconnect SDK clients to refresh from a snapshot. |

## Source anchors

| Need | File |
| --- | --- |
| Event log schema | `packages/nikcli/src/sync/sync.sql.ts` |
| Local storage helpers | `packages/nikcli/src/sync/index.ts` |
| Live projection | `packages/nikcli/src/sync/projector.ts`, `projection.ts` |
| Cold-start snapshot | `packages/nikcli/src/sync/snapshot.ts` |
| Event reducer | `packages/nikcli/src/sync/reducer.ts` |
| Outbox | `packages/nikcli/src/sync/outbox.ts` |
| Hub transport | `packages/nikcli/src/sync/remote-client.ts`, `remote-sync.ts` |
| Hub sync CLI | `packages/nikcli/src/cli/cmd/sync.ts` |
| TUI sync dialog | `packages/tui/src/component/dialog-sync.tsx` |
| TUI sync context | `packages/tui/src/context/remote-sync.tsx` |
| Sync config | `packages/nikcli/src/sync/sync-config.ts` |

---

Source: https://nikcli-ai.dev/docs/sync