Skip to main content

Troubleshooting

Common failures, what they mean, and how to fix them. Setup steps live in Getting Started; every environment variable referenced here is defined in the Configuration Reference.

Install And Build Failures​

  • pnpm install fails or picks the wrong pnpm. The repo pins pnpm@10.x through the packageManager field. Use Corepack instead of a globally installed pnpm: corepack enable, then corepack pnpm install.
  • Engine errors during install. OpenEve requires Node >=22.19.0 (declared in engines). Check node --version and upgrade before reinstalling.
  • pnpm openeve fails with a module-not-found error. The CLI runs from built output; run pnpm build first. This also applies after pulling changes that touch any packages/*/src.
  • Typecheck or tests fail on a fresh clone. Run in order: pnpm install, pnpm build, pnpm check, pnpm test.

CLI Not Found​

  • openeve: command not found. The packages are not yet published to npm, so there is no global openeve binary to install. Inside this repo, run pnpm openeve <command> — the root workspace depends on @openeve/cli, so pnpm resolves the built workspace binary.
  • pnpm openeve prints nothing useful. Run pnpm openeve help for the full command list or pnpm openeve help <command> (or pnpm openeve <command> --help) for per-command flags.
  • Unknown command: <cmd>. Run "openeve help" for usage. Check the spelling against pnpm openeve help; commands are init, add, validate, manifest, build, dev, run, serve, channels, deploy, models, and help.
  • Agent root not found: <path>. The positional argument (or --root) must point at an agent folder. Run openeve init <path> to scaffold one.

Model And Provider Key Errors​

  • A model run fails before the first response. The model prefix in agent.ts decides the required env var: openai/* needs OPENAI_API_KEY, anthropic/* needs ANTHROPIC_API_KEY, and so on (full table in Runtime And Deployment). The CLI prints the same hint when a provider auth error reaches the top level: Hint: the model prefix in agent.ts decides the required env var (e.g. openai/* needs OPENAI_API_KEY).
  • validate warns unknown-model. The model in agent.ts is not in the local model catalog. It is a warning, never an error — custom providers still work at runtime — so first check for typos. pnpm openeve models lists the catalog (--provider openai filters); --no-model-check skips the check entirely.
  • openeve models prints Model catalog unavailable. The catalog comes from @openeve/pi; make sure the workspace is built (pnpm build). When the catalog cannot load, validate silently skips the model check rather than failing.

Build Artifacts And .openeve Churn​

  • A command fails while rebuilding an example's .openeve/ directory. Point the artifact somewhere disposable with --out /private/tmp/openeve-minimal (supported by build, run, dev, serve, and deploy), or delete the generated .openeve/ directory and rebuild.
  • Example artifacts pile up. pnpm clean:artifacts removes ignored examples/**/.openeve directories; pnpm clean removes package dist/ output; pnpm clean:tmp removes openeve-* temp directories.
  • Generated files show up in git status. .openeve/, dist/, node_modules/, and env files are local build output and should stay out of commits.
  • A rebuild did not restart dev --watch. Restarts only happen when the agent revision changes; the CLI prints Rebuilt: no manifest change; keeping the running server. for cosmetic edits. While the folder is invalid, the previous server keeps serving and the CLI prints the validation issues until the folder is valid again.

Serve And Control-Plane Auth​

  • Production boot fails with Production OpenEve Node runtime requires OPENEVE_ADMIN_TOKEN or a host-provided auth policy to protect control-plane routes. Set OPENEVE_ADMIN_TOKEN (or pass an auth policy from host code). Local openeve serve runs in dev mode, where inspection endpoints are open; a deployed artifact boots in production mode.
  • 401 admin_auth_required / 403 admin_auth_invalid. /manifest, /routes, and the /runs* inspection endpoints require Authorization: Bearer <OPENEVE_ADMIN_TOKEN> in production.
  • 403 API run creation is disabled. Production POST /runs is off by default. Set OPENEVE_ENABLE_API_RUNS=true and authenticate with the admin token; leave it off unless API-triggered runs are intended.
  • Load balancer keeps routing during deploys. Point readiness at GET /readyz — it returns 503 { "draining": true } during graceful shutdown — and liveness at /health or /healthz, which stay 200. Deliver SIGTERM so in-flight runs drain within OPENEVE_SHUTDOWN_TIMEOUT_MS (default 30s); a second signal exits immediately.

Connection Store Secrets​

File-backed connection credential stores are encrypted, and production boot validates the secret:

  • File-backed connection credential stores require OPENEVE_CONNECTION_STORE_SECRET or OPENEVE_SECRET outside dev mode. Set one of the two, or use Postgres state (the Postgres adapter implements the grant stores directly and needs no file secret).
  • ... must be at least 32 characters outside dev mode. The secret has a hard 32-character minimum in production.
  • ... cannot use the local development secret outside dev mode. The built-in dev fallback value is rejected in production; generate a real secret.

Rotating the secret makes previously encrypted grant files unreadable — plan rotation as a re-authorization event.

Deploy Preflight Failures​

  • deploy --dry-run reports missing setup. Each preflight item names a required env var or provider setup step inferred from gateway.ts, channels, connections, and the model prefix. Satisfy every required item before publishing.
  • Railway: requires RAILWAY_TOKEN (or an authenticated Railway CLI) and a linked project/service or explicit --railway-project and --railway-service.
  • Docker (preview): requires Docker locally; the image tag comes from --docker-image or OPENEVE_DOCKER_IMAGE.
  • Fly (preview): requires FLY_API_TOKEN and --fly-app or FLY_APP_NAME (Fly deploys require --fly-app or FLY_APP_NAME.).
  • <bin> not found. Install it, or pass the matching --*-bin flag. The deploy path shells out to railway, docker, or flyctl; install the binary or point --railway-bin/--docker-bin/--fly-bin at it.
  • Migration files are present, but no migration runner was configured. The artifact contains .openeve/migrations; pass --migration-command or set OPENEVE_MIGRATION_COMMAND.
  • Third-party providers: a provider-package-unresolved validation warning means the compiler could not import the packageName given to adapter(kind, opts, { package }) from the agent root, so preflight falls back to a generic requirement. Install the package where the agent builds. At boot, the Node host re-resolves it and fails with a specific error when the package is missing (Failed to load provider package ...), does not export openeveProvider, or has no registration for the role/kind. See Authoring Adapters.

Scheduler And Channel Ingress​

  • 403 scheduler_unauthorized on /openeve/scheduler/tick. Set OPENEVE_SCHEDULER_SECRET and send it as Authorization: Bearer <secret> or x-openeve-scheduler-secret. Without a configured secret the endpoint only accepts dev-mode requests.
  • Production boot fails with Production provider ingress requires webhook authentication: .... A channel declared ingress secrets (ingress.requiredSecretEnv, any-of groups) and none of its groups is fully set — for example Telegram needs TELEGRAM_WEBHOOK_SECRET, and Photon needs a signing secret or a bearer token. Dev mode logs a warning instead of failing. Set the secrets named in the error.
  • Webhooks return 401 even though boot succeeded. Boot checks that the secrets exist; each request is still verified by the channel module (for example Telegram compares x-telegram-bot-api-secret-token constant-time). Make sure the provider-side webhook config sends the same secret.
  • 429 { "error": "Rate limited." }. Ingress rate limiting is enabled (OPENEVE_INGRESS_RATE_LIMIT / OPENEVE_RUNS_RATE_LIMIT or the host rateLimit option). The Retry-After header says when to retry.
  • 429 Run capacity exhausted. Retry later. The run concurrency cap (OPENEVE_MAX_CONCURRENT_RUNS, production default 16) rejected a brand-new run. Capacity is checked before the event's idempotency key is consumed, so provider retries succeed once load drops.

Durability Workers And Recovery​

  • Deliveries sit in pending and never send. The delivery worker drains the durable queue; check that OPENEVE_DELIVERY_WORKER is not set to false/0 and that the host called startBackgroundWorkers() (listenNodeRuntime does this automatically).
  • A run completed but the user got no message yet. A retryable send failure defers the delivery instead of failing the run: the run event log shows delivery.deferred with the error, attempt count, and nextAttemptAt, and the worker retries with backoff up to OPENEVE_DELIVERY_QUEUE_MAX_ATTEMPTS (default 5) before a terminal delivery.failed.
  • Runs stuck in running after a crash or redeploy. Orphan recovery sweeps runs stale past max(5min, 4x heartbeat): already-delivered runs complete, runs with a continuation checkpoint get one resume attempt, runs with a model response get a real pending delivery, and everything else is marked failed. It runs at boot and every OPENEVE_RUN_RECOVERY_INTERVAL_MS; the kill-switch is OPENEVE_RUN_RECOVERY=false.
  • Sandbox writes (memory, skills, /workspace files) not persisting. The sandbox-sync worker (OPENEVE_SANDBOX_SYNC_WORKER) performs the writeback; use sandboxSyncDiagnostics() and inspectSandboxSyncJob(jobId) to see due/leased/expired/blocked jobs, and retrySandboxSyncJob(jobId) after fixing a blocked one.
  • Kill-switches for debugging: OPENEVE_DELIVERY_WORKER, OPENEVE_SANDBOX_SYNC_WORKER, and OPENEVE_RUN_RECOVERY each accept false/0. Every background behavior has one; see the tables in Runtime And Deployment.

Still Stuck?​

Inspect the durable record: .openeve/manifest.json, .openeve/preflight.json, and .openeve/route-table.json for compile-time surprises; GET /runs/:id/events (or the state file in dev) for runtime behavior — every model step, tool call, pause, delivery attempt, and recovery action is recorded as an event.