Resilience & Operations
This section explains how the system remains available and correct under failure, and how we observe behavior without centralizing the read path. Resilience is achieved through client‑side fallbacks, idempotent on‑chain designs, and conservative rate controls.
Availability strategy
- Multi‑path content retrieval: CDN primary, Walrus aggregator fallback with hash verification.
- Identity redundancy: wallet‑only flows remain available when zkLogin IdPs are degraded.
- On‑chain as source of truth: read paths do not depend on mutable backend state; clients re‑derive from chain and manifests.
Idempotency and retries
- On‑chain settlement (bounties, payouts) is single‑shot; retries that would duplicate effects are rejected by Move invariants.
- Reads use bounded retries with exponential backoff; fetch ordering prefers fast‑fail to preserve snappy UX.
Rate limits and quotas
- Sponsored transaction quotas per principal and per time window; deny‑list known abuse addresses.
- Content fetch throttling under repeated integrity mismatches; exponential backoff reduces load.
Client telemetry & health
- Telemetry emits path choices (CDN vs Walrus), fetch timings, and verification outcomes without exposing PII.
- Error budgets and alerting tie to user‑visible metrics (failed renders after retries, slow sponsored settlement).
Disaster considerations
- CDN outage: clients fall back to Walrus; banner messaging can inform users of degraded mode.
- Aggregator congestion: prefer alternate aggregators; clients cache a vetted list and rotate on errors.
- Key compromise (sponsor): revoke and rotate sponsorship keys; require wallet‑only flows temporarily.
Operational checklists
- Content release: publish to Walrus, validate hashes, warm CDN, update on‑chain site references.
- Policy change: update Kiosk policy and publish new package version; version gates in clients.
- Storage planning: estimate monthly asset growth; maintain storage resources ahead of demand.