# Sync Feature — Implementation Notes

## DB: better-sqlite3 (migrated from @libsql/client)

**Why migrated:** `@libsql/client` caused SIGSEGV (exit code 139) on Azure App Service due to Rust native module incompatibility with seccomp profile.

**Key constraints of better-sqlite3:**
- All operations are **synchronous** — no `await` on db calls
- `db.transaction()` callback **must be synchronous** — `async (tx) => {}` throws `TypeError: Transaction function cannot return a promise`
- Drizzle ORM for better-sqlite3: `drizzle-orm/better-sqlite3` + `drizzle-orm/better-sqlite3/migrator`
- Result object uses `.changes` (not `.rowsAffected`) → use `(result as any).changes ?? (result as any).rowsAffected ?? 0`

**Config:**
- `drizzle.config.ts`: `url` without `file:` prefix
- Docker base: `node:22-slim` (prebuilt binaries available)
- No build tools needed in Dockerfile (prebuilts used on Linux)

**Migration:** `migrate(db, { migrationsFolder })` called synchronously at startup in `connection.ts`.

## sync-service.ts

`pushChanges` processes entities in FK order: contexts → topics → historyEntries → ratings → imageBlobs → pages → notebooks → pageNotebooks.

`upsertEntity` logic: insert if not exists, update if `clientVersion > serverVersion`, else record conflict.

**No transaction wrapper** — the old `db.transaction(async ...)` was dead code and was removed. Each entity is upserted individually.

## API Keys (M2M Auth)

Added `api_keys` table in schema.ts. Format: `ka_<base64url-32bytes>`. Only SHA-256 hash stored in DB.

Auth middleware (`auth.ts`) detects `ka_...` Bearer tokens **before** JWT path, looks up by hash, fires `lastUsedAt` update async.

Routes: `GET /api/api-keys`, `POST /api/api-keys`, `DELETE /api/api-keys/:id` (soft-delete).

Raw key returned **once** on creation — not recoverable after that.

## Browser API Key (client-side)

`browserApiKey` store in `authStore.ts` backed by `localStorage['ka-note-browser-key']`.

`getAccessToken()` returns browser key first, then falls back to MSAL.

`isAuthenticated` derived from `account OR browserApiKey`.

UI in Settings → "Browser-Authentifizierung" section.

Use case: avoids MSAL silent refresh failures (Azure AD returning HTTP 400 on refresh token).

**401 auto-invalidation:** `authFetch` in `apiClient.ts` detects 401 responses when a browser key was used and calls `setBrowserApiKey(null)` — clears `localStorage` and the store. `isAuthenticated` becomes `false`, restoring the MSAL login UI. Prevents the stuck state where a revoked/expired key blocks both sync and manual login.

## Deploy

`deploy.ps1` uses normal `docker build` with layer caching. Only changed layers are pushed to ACR.

`COPY server/ server/` triggers a cache miss whenever server source changes → tsc reruns automatically. `--no-cache` is not needed and was removed (it caused all layers to be rebuilt and pushed on every deploy, >40 MB).

## Delta Push (client-side filtering)

`pushAll(since)` only sends entities with `updatedAt > since.toISOString()`. A read-only client sends 0 entities and skips the HTTP request entirely.

`fullSync()` passes `since=null` → pushes everything (used after DB reset or first sync).

**Root cause for all-conflicts:** `upsertEntity` accepts only `clientVersion > serverVersion`. After a pull, client has `version == serverVersion` → equal-version entities are never pushed (filtered out by delta push). Without delta push, a read-only client would produce 700+ conflicts on every sync cycle.

## Client identification (`X-Client-Id` header)

Each browser instance generates a stable random ID in `localStorage['ka-clientId']`. The header value is `Browser·OS·id` (e.g. `Edge·Win·a3f2`, `Safari·iOS·x9k2`).

Server logs `client:` field in `[sync/push]` and `[sync/pull]` entries.

## ECONNRESET on POST /api/sync/push

`Error: aborted / ECONNRESET` logged as 500 is a **client-side disconnect**, not a server error. Hono logs 500 because response couldn't be sent. Causes: tab backgrounded on iOS, Azure idle timeout, browser request timeout.

Not actionable on server side unless response time is consistently >10s (indicates oversized payload, e.g. first full sync after DB reset).

## Paginated Pull & Push (since v1.2.6)

**Root cause:** Single pull response was 6.25 MB (all tables at once). iOS Safari aborted both push and pull on full sync (e.g. after local DB reset).

### Pull — table-by-table pagination

`GET /api/sync/pull?since=&table=historyEntries&limit=500&offset=0`

- `table` param routes to `pullTable()` in `sync-service.ts`
- `limit+1` trick detects `hasMore` without extra COUNT query
- **Stable sort**: `ORDER BY updatedAt ASC, id ASC` — mandatory for correct offset pagination (ties on `updatedAt` alone cause skipped/duplicate rows)
- Response: same `SyncPullResponse` shape + optional `hasMore` / `nextOffset`
- Legacy path (no `?table=`): `pullChanges()` unchanged, returns all at once

**Pull order** (FK-safe, imageBlobs excluded — lazy loaded):
```
contexts → notebooks → topics → pageNotebooks → pages → ratings → historyEntries → tasks
```

Client loops per table until `hasMore === false`, merges each page into Dexie in a **small per-page transaction** (previously one big transaction across all 9 tables).

`lastSyncAt` is set only after all tables complete (last page of last table).

### Push — table-by-table batches

Client sends one table at a time in FK order, 500 entities per batch (10 for imageBlobs). Server `pushChanges()` already handles sparse changes (empty arrays skipped) — no server-side changes needed.

**Push order** (FK-safe, imageBlobs last):
```
contexts → topics → historyEntries → ratings → pages → notebooks → pageNotebooks → tasks → imageBlobs
```

### imageBlobs — lazy on-demand

imageBlobs are **excluded from the pull loop** (1 blob = 1–3 MB base64, pagination alone doesn't scale).

- `GET /api/sync/blob/:id` — fetches single blob by ID, returns same `ImageBlob` format
- `getImageUrl(id)` in `imageStore.ts`: checks Dexie → if miss, fetches from server → stores in Dexie → returns objectURL
- Push remains eager (blobs must reach server so other devices can lazy-fetch)
- Offline: missing blob → `getImageUrl()` returns `null` → `ka-img:ID` stays unreplaced in markdown → no crash

### Concurrency guard

`_syncInProgress` flag in `syncService.ts` prevents overlapping sync runs (30s auto-interval + visibilitychange + manual can fire simultaneously; with pagination a sync takes much longer).