OpenEve Framework Guide
OpenEve is a filesystem-first framework for durable AI agents. An agent folder declares what the agent is; the compiler turns that folder into a manifest and runtime artifact; the runtime executes runs durably through pluggable adapters.
Agent Folder Convention
Only instructions.md and agent.ts are required.
agent/
instructions.md
agent.ts
context.ts
gateway.ts
skills/
tools/
channels/
schedules/
triggers/
connections/
sandbox/
subagents/
instrumentation.ts
lib/ and playbooks/ are not reserved OpenEve conventions. App helpers can live wherever the app normally keeps source code.
agent.ts
agent.ts exports defineAgent({ ... }).
import { defineAgent } from "@openeve/core";
export default defineAgent({
model: "openai/gpt-5.4-mini",
reasoning: "medium",
description: "A portable OpenEve agent.",
defaultTools: ["echo"]
});
The compiler statically extracts the model, reasoning level, description, default tools, and context declaration. Tool implementations, secrets, databases, deployment providers, and blob stores do not belong in agent.ts.
reasoning is optional and accepts "off", "minimal", "low", "medium", "high", or "xhigh". OpenEve defaults to "medium" for Pi model runs when this field is omitted.
OpenEve reads manifest-facing config through the TypeScript AST. This applies
to agent.ts, gateway.ts, tools, channels, schedules, trigger handlers,
connections, sandbox definitions, and subagent agent.ts files.
Manifest-affecting values must be statically readable: a default-exported
define*({ ... }) helper call, an exported identifier that resolves to that
call or object, top-level const indirection for literal strings, arrays, and
objects, shorthand properties, imported helper aliases, as const, and
satisfies are supported. The compiler does not execute arbitrary config code
during manifest generation. Legacy source-text fallbacks are kept only for
older dynamic patterns and validation compatibility; new framework API behavior
should be expressed through the static config contract.
Agent Engine
Pi is the canonical primary engine. The model loop is
pi — open source, minimal, and proven —
adapted in @openeve/pi and constructed directly by the runtime. Agents never
select an engine: agent.ts has no harness: slot (declaring one fails
validation with harness-not-configurable); subagents may still declare
harness: in subagents/<name>/agent.ts for delegate engines such as codex,
claude-code, or livekit.
The runtime still talks to pi only through the internal AgentHarness
contract in @openeve/core. This seam is not a public extension point, but it
is deliberately engine-shaped and load-bearing: the runtime imports no pi
types, continuation state is opaque JSON owned by the engine adapter (so
persisted runs never depend on pi internals), and all durability (tool
records, approvals, checkpoints, events, usage, spans) stays in the runtime.
Embedders and tests can replace the engine for every run through
RuntimeOptions.agentHarness — the durability test suite drives scripted
engines through exactly this seam. Subagents reuse it too: a subagent with
harness: adapter("pi") (the default) runs through the primary-loop machinery
including follow-ups via persisted continuations, and
runtime.agentHarnessAsSubagentHarness(harness) exposes that bridge for
custom wiring.
Tool batches run in parallel by default: when the model emits several
tool calls in one response, parallel-safe tools execute concurrently.
Pause-capable tools — approval-gated tools, ask_question, connection tools,
and the deferred tool_call bridge — carry execution: "sequential" on
their descriptors, which serializes any batch that contains one. A pause
stops the batch wherever it happens: later sequential calls are skipped with
an explicit { skipped: true } result instead of executing after the run
parked, and a dynamic pause inside a parallel batch (a custom tool calling
ctx.askQuestion) still ends the turn without another model request.
The engine also reports token deltas (response_delta events). They are
ephemeral: the runtime fans them out to live run-stream subscribers
(runtime.subscribeRunStream(runId, listener), or the Node host's
GET /runs/:id/stream SSE endpoint) and never writes them to the durable
event log.
context.ts
Agents do not need context.ts. When it is absent, OpenEve uses defaultContext().
defaultContext() builds a Flue-shaped context bundle with trusted instructions, active event, bounded recent history, memory filesystem shape, file manifest, current attachments, a compact skill and capability catalog, always-on tool summaries, channel metadata, and trust boundaries. Deferred tool schemas and skill bodies are not injected up front. Files, screenshots, webpages, tool output, search results, memory, and attachments are context, not instructions.
Every runtime model prompt also receives a small stable OpenEve filesystem contract immediately after instructions.md. It names logical resource paths and their mutability: /memory is durable memory accessed directly through memory tools by default, /workspace is writable workspace for generated artifacts and modified copies when a sandbox exists, /history is read-only conversation history, and /files is read-only input context whose manifest.json should be inspected before reading file contents. These paths are projected into a sandbox only when sandbox-backed tools need them; the contract does not inject memory or file contents into every turn.
Custom context can extend the default:
import { defaultContext, defineContext } from "@openeve/core";
export const customContext = defineContext({
kind: "custom",
name: "customContext",
extends: defaultContext({ recentHistory: { maxMessages: 5 } })
});
Context policy is trusted app-runtime code and is recorded in the manifest with source attribution.
gateway.ts
gateway.ts is the portable stack declaration. It is tiny and declarative:
import { adapter, defineGateway } from "@openeve/core";
export default defineGateway({
deploy: adapter("railway"),
runtime: adapter("node"),
state: adapter("postgres"),
blob: adapter("r2"),
sandbox: adapter("daytona"),
scheduler: adapter("gateway")
});
Runtime host, state, blobs, sandbox, scheduler, connections, and observability are independent choices. Deploy adapters host the runtime process; they do not force a state/blob/sandbox provider.
Like agent.ts, gateway.ts is parsed as TypeScript syntax rather than executed. Use statically readable defineGateway({ ... }) declarations, adapter("kind"), or known provider helper calls such as railwayDeploy(), neonPostgres(), r2Blob(), and dockerSandbox(). Dynamic expressions are ignored unless they resolve to top-level literals the compiler can validate.
Scheduler choices are explicit. adapter("local") starts an in-process polling loop for development or single-process hosts. adapter("gateway") does not start a loop; a cloud scheduler, platform cron, or gateway worker calls the runtime tick endpoint or runDueSchedules(). adapter("postgres") starts the same polling loop but expects Postgres state so multiple workers coordinate through shared idempotency and dynamic-schedule leases.
Production state is Postgres. Neon remains the default hosted path, Supabase and local/custom Postgres are presets, and the adapter remains adapter("postgres") because each provider exposes standard Postgres. Blob storage is S3-compatible, with R2 preserved as a first-class preset/wrapper. Railway is the supported hosted deploy target through @openeve/railway; Docker and Fly deploy paths have adapter-owned publishers and remain preview until live smoke coverage exists. Deploy choice does not imply a state/blob/sandbox vendor.
See Adapters for the current adapter list, helper functions, and provider environment variables.
Tools
Each file in tools/ becomes one model-facing tool. The filename is the tool name. Tool descriptors include name, description, input schema, optional output schema, approval policy, and model-output projection.
Tool code runs in the trusted app runtime by default. It only uses the sandbox through ctx.getSandbox().
OpenEve starts model runs with exactly the core harness tools: read, write, edit, delete, list, grep, bash, ask_question, spawn_subagent, tool_search, tool_describe, and tool_call. Authored tools, plugin tools, MCP tools, and connection tools are deferred behind tool_search -> tool_describe -> tool_call; the runtime still logs, approves, traces, and executes the underlying tool name. bash is enabled by default in every runtime mode unless OPENEVE_BASH_TOOL_MODE=approval or OPENEVE_BASH_TOOL_MODE=disabled is set, or the host passes the equivalent core tool policy.
Tools can optionally declare capability metadata, but most apps should rely on inference:
capability: {
visibility: "deferred",
execution: "direct",
namespace: "billing",
tags: ["invoice", "customer"],
aliases: ["receivables"]
}
visibility can be always, deferred, skill, or hidden; execution can be direct, sandbox, or both. These fields feed deferred discovery metadata. Core harness tools are not configurable by agent authors.
All file, memory, skill, workspace, and artifact work goes through the sandbox filesystem. /history and /files are read-only; /memory, /skills, and /workspace are writable according to policy. The core file tools lazily acquire and hydrate the sandbox for requested paths.
import { approvalRequired, defineTool } from "@openeve/core";
export default defineTool({
description: "Record a note after approval.",
inputSchema: { type: "object", properties: { note: { type: "string" } }, required: ["note"] },
needsApproval: approvalRequired("Recording a note is a durable side effect."),
async execute(input, ctx) {
await ctx.emit("note.recorded", { idempotencyKey: ctx.idempotencyKey("note") });
return { recorded: true, note: input.note };
}
});
toModelOutput can expose a bounded, safe projection while the rich result remains available in the durable tool log.
Skills
Skills live under skills/<name>/SKILL.md. OpenEve parses skill frontmatter and builds compact skill capabilities in the manifest. Runs see skill names and descriptions up front; the body is loaded on demand through the core model-facing load_skill tool or the host-driven loadSkill(runId, skillName) API. Loading a skill reads from the durable skill store, returns instructions, and does not reveal additional tools. If a sandbox already exists, the selected skill may also be projected into /skills/<name>/SKILL.md for filesystem inspection.
Self-improvement
Self-improvement means one thing: the agent authoring and editing its own skills as it learns how to do things. It is configured by selfImprovement in agent.ts and is on by default. Runtime scheduling and connections are deliberately separate concerns with their own config blocks (dynamicSchedules, dynamicConnections) and their own clearly-named tool APIs — they are not part of "self-improvement." Setting every block off restores fully static, manifest-only behavior.
export default defineAgent({
// Stable logical identity for durable learned state across revisions/deploys:
id: "travel-concierge",
model: "openai/gpt-5.4-mini",
// The agent learning by writing/editing skills:
selfImprovement: { writable: true, writeApproval: false },
// Separate, clearly-named concerns:
dynamicSchedules: { dynamic: true, approval: false },
dynamicConnections: { dynamic: false, approval: true, allowedHosts: [] }
});
Skills are a durable folder (the self-improvement surface). On boot, compiled skills/ seed a durable SkillStore, scoped by stable agent.id when configured and tracked by a content hash so redeploys upgrade pristine seeded skills and preserve anything the agent or user changed. Each run puts only the skill index in the prompt: name, description, and path. Full bodies live at /skills/<name>/SKILL.md and are read, edited, created, or deleted with the normal file tools. When selfImprovement.writable is off, /skills writeback is blocked.
Trigger handlers. Static schedules and runtime-created dynamic schedules can reference triggers/<name>.ts through trigger.handler. The runtime reserves or leases the schedule fire, creates the durable run with the schedule's user/conversation/project scope, calls prepare(ctx) in trusted app code, invokes the returned skill/playbook/agent target with promptContext in the active event, then calls finalize(ctx, result) for both success and failure paths. Dynamic schedules may only reference compiled trigger handlers by name; they cannot inject handler code at runtime. Handler context exposes memory, resources, blob, schedule metadata, trigger metadata, and routine-run state APIs without requiring model-visible tools.
Dynamic schedules (a separate concern). With dynamicSchedules.dynamic on, a clearly-named schedule-management tool (calling ctx.scheduleManager, not ctx.selfImprovement) creates rows in a ScheduleStore. A built-in minute dispatcher atomically leases due rows and starts one durable run per row; recurring rows recompute their next run, one-shot rows disable themselves. Delivery is at least once, so scheduled work should be idempotent. Owner/tenant scope comes from the run session, never from model input. These dynamic rows are distinct from the static schedules/ folder.
Dynamic connections (a separate, gated concern). With dynamicConnections.dynamic on (off by default), a tool calling ctx.connectionManager can persist a new MCP/HTTP/OpenAPI connection into a ConnectionDefinitionStore; stored definitions merge into the connection registry and become discoverable through tool_search. Credentials route to the existing encrypted grant store through host APIs or authorization flows, never model-visible tool input, the agent folder, sandbox, or model context. Saving is approval-gated and restricted to dynamicConnections.allowedHosts; remote tool descriptions are treated as untrusted data. Agents never author trusted tool code -- a CLI a user hands the agent is run in the sandbox and wrapped in a skill; net-new typed tools remain a reviewed source change.
Tools reach these through three distinct context APIs — ctx.selfImprovement (skills), ctx.scheduleManager, and ctx.connectionManager — and the runtime exposes saveSkill, listSkills, createDynamicSchedule, runDueSchedules, saveConnectionDefinition, and related methods for host-driven use. Durable stores are provided by the state adapter (Postgres) or fall back to local JSON files under the artifact root. See examples/self-improving-agent and Customizing Agents.
Channels
Channels normalize platform entrypoints and delivery behavior. HTTP-capable channels declare a route and methods; the compiler emits a route table.
import { defineChannel } from "@openeve/core";
export default defineChannel({
transport: "http",
route: "/message",
methods: ["POST"]
});
The default raw HTTP message fallback is dev-only unless the Node host has
authenticated the request. Production provider routes should use a helper or
custom normalizeHttp() that verifies the provider request before accepting a
turn.
Channel files own platform event semantics, not durable state schema or sandbox lifecycle.
OpenEve ships one-line helpers for Slack, Discord, Telegram, Microsoft Teams, Photon-style, and GitHub repo-event agent communication channels. Provider helpers preserve the same channel contract: verify the incoming event, normalize to ChannelTurn, use provider delivery IDs for idempotency where available, and send replies through provider APIs. The GitHub channel (preview) maps signed repository webhooks into turns and replies through issue/PR comments; @openeve/github also remains available as a connection package for GitHub App capabilities.
For retried webhook providers, channel modules can return kind: "accepted" to acknowledge the
HTTP request before the model turn completes. Use the provider's stable delivery id as the
idempotency key. For Slack Events API channels, verify the request signature, normalize the event,
return a 2xx response immediately, and set idempotencyKey to Slack's event_id so retries do not
start duplicate turns.
Channel modules can also augment context after ACK and before default context bundle construction. Slack uses this hook to bridge a bounded set of recent OpenEve Slack conversations for the same user, so a user can follow up from a thread in a DM without merging every Slack surface into one transcript or fetching workspace-wide Slack context.
Channel modules can also export startIngress(ctx, emit) for long-lived
provider listeners. The Node host starts these listeners beside the scheduler,
restarts provider-owned listeners through adapter code, and stops them on server
shutdown. emit.accepted({ turn, idempotencyKey, idempotencyScope }) feeds
Gateway-style events into the same durable, idempotent run path used by accepted
HTTP webhooks. Discord uses this for Gateway DMs, mentions, and thread messages.
Connections
Connections declare required external capabilities, scopes, subject mapping, and whether they are required. Raw secrets and refresh tokens stay outside the agent folder, model context, and sandbox filesystem. Live MCP, OpenAPI, and HTTP connection tools are searched through tool_search; concrete remote tool schemas are not visible until tool_describe selects them. Tools and channels consume connection handles from runtime context.
Schedules
Schedules declare a cron expression, timezone, and idempotency key. The compiler emits scheduler registration metadata. Long-running execution and retries belong to the scheduler/runtime adapter, not the schedule file.
The runtime uses one dispatch contract for static and dynamic schedules: runDueSchedules({ now, lookbackMs }) registers manifest schedules, computes due static cron occurrences, reserves idempotency keys in state, leases due dynamic rows from the ScheduleStore, and then dispatches either the plain message path or the trigger lifecycle path. Trigger lifecycle events are recorded on the run as trigger.prepare_*, trigger.finalize_*, routine.run_*, and related target/resource events, with matching telemetry spans for prepare/finalize. The Node host exposes the same operation at GET or POST /openeve/scheduler/tick; production callers should set OPENEVE_SCHEDULER_SECRET and pass it as Authorization: Bearer <secret> or x-openeve-scheduler-secret.
Sandbox
Sandbox files declare the agent computer selection. The core contract supports file reads/writes, shell execution, and optional provider snapshots. The local adapter is for trusted dev/test work; Docker is the supported local isolation baseline; Daytona, E2B, and Modal are supported hosted sandbox choices. Sandboxes are acquired lazily when a sandbox-backed tool or capability asks for one.
Sandbox paths are OpenEve logical paths rooted under the adapter workspace.
Providers must reject traversal and return virtual absolute paths such as
/workspace/report.txt from listings. Local sandbox execution is trusted
developer or self-managed execution only; use Docker, Daytona, E2B, or Modal
when untrusted code needs an isolation boundary.
Snapshots are a scarce infrastructure checkpoint, not the normal turn persistence mechanism. Production runs should use OpenEve state/blob sync for memory, messages, tool traces, and /workspace artifacts, and keep fresh run sandboxes ephemeral. The default snapshot policy is never, so a hosted sandbox run does not create a remote snapshot unless the agent explicitly opts in.
Sandbox files may declare a snapshot policy:
export default defineSandbox({
adapter: "daytona",
image: "node:22",
snapshot: {
mode: "manual",
retainLast: 3,
reason: "operator-requested checkpoint"
}
});
Supported modes are never, manual, on_failure, and always. manual only captures when run metadata includes an explicit sandbox snapshot request. always is intended for short-lived debugging or controlled checkpoint jobs, not chat turns. OPENEVE_SANDBOX_SNAPSHOT_MODE, OPENEVE_SANDBOX_SNAPSHOT_RETAIN_LAST, and OPENEVE_SANDBOX_SNAPSHOT_REASON can override policy at runtime.
Compiler Output
openeve build emits:
.openeve/
manifest.json
agent-revision.json
Dockerfile
server/
boot.json
boot.js
sources.json
source-metadata.json
source-map.json
assets/
migrations/
preflight.json
route-table.json
schedules.json
The generated server/boot.js starts the real @openeve/node host, not a health-only stub. It loads manifest.json and server/sources.json, constructs production runtime adapters from gateway.ts, and serves /health, /healthz, /readyz, route-table HTTP channels, scheduler ticks, connection callbacks, authenticated Agent Runs endpoints under /runs, and authenticated control-plane endpoints. In production, /manifest, /routes, Agent Runs inspection endpoints under /runs, and opt-in API run creation require a host auth policy or OPENEVE_ADMIN_TOKEN; POST /runs also requires OPENEVE_ENABLE_API_RUNS=true. Generic route-table HTTP channels also require host auth unless they are first-party provider helper routes that verify ingress themselves.
agentRevision is a deterministic hash of source paths, source hashes, and relevant config. Rebuilding unchanged source produces the same revision; changing source changes it.
The manifest contains instructions, agent definition, context policy, gateway config, tools, capabilities, skills, channels, schedules, trigger handlers, connections, sandbox declarations, subagents, instrumentation source, route table, schedule metadata, preflight requirements, validation results, package versions, and source hashes.
CLI Commands
openeve initscaffolds a minimal agent folder.openeve validatechecks required files, names, schemas, routes, schedules, triggers, connections, subagents, and skill tool references.openeve manifestprints or writes the compiled manifest.openeve buildemits the.openeve/artifact.openeve devvalidates, builds, and runs one local dev check. With--watchit serves the agent over local HTTP (likeserve), watches the agent folder, and on each change re-validates (keeping the old server alive while the agent is invalid), rebuilds, and restarts on the same port when the agent revision changes.openeve runexecutes a local run from a message and optional tool/input.openeve modelslistsprovider/modelspecs from the local model catalog.validatewarns (never errors) whenagent.tsnames a model outside that catalog;--no-model-checkskips the check.openeve serveboots the compiled runtime as a local HTTP server.--onceperforms a socket-free boot check for CI and restricted sandboxes.openeve deploy --dry-runbuilds and reports deploy steps plus missing setup.openeve deploy --target localprepares a local deploy path.openeve deployuses the deploy target fromgateway.ts; Railway is supported, while Docker and Fly deploy targets are preview.
Hosted publishing requires provider credentials and provider setup. Railway deploys require RAILWAY_TOKEN, the Railway CLI, and either a linked project/service or explicit --railway-project and --railway-service flags. Preview Docker deploys require Docker locally and can build or run the compiled artifact image. Preview Fly deploys require FLY_API_TOKEN, flyctl, and an app name from --fly-app or FLY_APP_NAME. Hosted adapter("postgres") state requires DATABASE_URL; runtime boot runs OpenEve migrations automatically unless OPENEVE_AUTO_MIGRATE=false. If the compiled artifact contains migration files under .openeve/migrations, hosted deploys run --migration-command or OPENEVE_MIGRATION_COMMAND before publishing, with OPENEVE_ARTIFACT_ROOT, OPENEVE_AGENT_REVISION, OPENEVE_DEPLOY_ENV, and OPENEVE_MIGRATION_FILES in the migration process environment.
Deploy Adapter Contract
Deploy publishers receive the build artifact, gateway config, preflight requirements, and target environment. @openeve/railway wraps railway up --ci; preview @openeve/docker builds and can locally run the artifact image; preview @openeve/fly writes a minimal fly.toml before flyctl deploy. Each path records deployment.json in the build artifact. Local deploy is available for developer machines and long-lived VMs.
Runtime Guarantees
The runtime persists the run before execution starts. It records context bundle creation, sandbox acquisition, harness/model events, tool requests, approval pauses, tool execution, sandbox events, memory sync, delivery obligations, delivery sends, and final status. Replay reconstructs a run from durable events, checkpoints, tool calls, and delivery records.
Human approval and ask_question pause states are represented durably, and resuming them re-enters the model loop. resumeApproval(runId) executes the approved tool and feeds its result back to the harness as the pending tool result, so the model continues reasoning from where it paused; resumeInput(runId, answer) does the same with the human's answer, so the agent that asked the question actually sees it. If no usable continuation checkpoint exists (pre-upgrade runs, or a harness that returned none), the resume degrades to completing the run directly and records a harness.resume_degraded event so the fallback is observable. Every resume path (approval, input, and connection-authorization callbacks) first claims a per-pause idempotency key (resume scope, keyed by tool call and pause generation), so a double-clicked approval, a double-submitted answer, or a replayed OAuth callback can never execute the gated tool twice — across replicas included. Final delivery is an idempotent delivery obligation.
A delivery only reports success when a real channel sender ran (or when the channel legitimately has no sender: dev-no-sender in dev mode, local-no-sender for local channels that declare no required env). If a channel module fails to load at delivery time, the delivery fails as retryable instead of being silently marked sent, and the failure is logged.
Final delivery is backed by a durable queue, mirroring sandbox sync. On queue-capable state adapters the completion path creates the obligation already leased by the inline sender (status sending with a lease token), so a delivery worker on this or any other replica can never lease and send the same obligation concurrently; success settles through the token-enforced completeDeliverySend, and a crash mid-send leaves the row leased until expiry, after which the worker retries it. The in-process fast path retries transient failures first (OPENEVE_DELIVERY_RETRY_ATTEMPTS, default 2). If it still fails retryably and the state adapter supports the queue (in-memory, file, and Postgres all do), the obligation is deferred instead of terminally failed: status back to pending with exponential backoff (nextAttemptAt, attemptCount, lastError), a delivery.deferred event, and the run still completes with deliveryDeferred metadata. A delivery worker (runDueDeliveries() / startDeliveryWorker()) recovers expired sending leases, leases due obligations (Postgres uses for update skip locked, so multiple replicas are safe), re-sends through the channel module using the persisted payload (channels fall back to payload.delivery when the original turn is gone), and settles each attempt with delivery.sent, delivery.retrying, or — after maxAttempts — a terminal delivery.failed with failedAt. Non-retryable failures go terminal immediately. Knobs: OPENEVE_DELIVERY_QUEUE_LEASE_MS, OPENEVE_DELIVERY_QUEUE_BATCH_SIZE, OPENEVE_DELIVERY_QUEUE_MAX_ATTEMPTS, OPENEVE_DELIVERY_QUEUE_INTERVAL_MS.
Orphaned-run recovery is staleness-guarded and multi-replica safe. Every executing run has a heartbeat (default 30s, OPENEVE_RUN_HEARTBEAT_MS) that bumps updatedAt through the state adapter's touchRun — a guarded, column-scoped write that never touches status and never resurrects a terminal run, so a heartbeat racing a completion cannot clobber it. recoverIncompleteRuns() only touches runs stuck in created/running longer than max(5min, 4x heartbeat) (override with staleAfterMs), and claims each candidate through a run:recovery idempotency key before acting. Recovery policy, in order: runs with delivery.sent complete without re-sending; tool calls requested but never started are cancelled before side effects; tools that were mid-execution at crash time (tool.execution_started without a settle event) are marked failed-as-interrupted (tool.execution_interrupted) and, when the continuation is parked on that tool, the resume hands the model an explicit { interrupted: true } tool result instead of silently re-driving it into re-execution; runs with a persisted harness.continuation (or legacy pi.continuation) checkpoint — written after every model iteration by default — get one in-place resume attempt through the normal harness resume path (run.recovery_resume_attempted, capped, then degrade); runs that reached a model response but not delivery first adopt any existing delivery obligation before creating one (using the same final-delivery idempotency key the original attempt would have used), so crash recovery can never double-send (deliver-after-model-response); everything else is marked failed with a run.failed event rather than being misreported as completed. The sweep runs once at host boot and periodically via startBackgroundWorkers() (default every 60s, OPENEVE_RUN_RECOVERY_INTERVAL_MS).
Resume is checkpoint-restart, not deterministic event replay, and tool execution is therefore at-least-once: a crash in the window between a tool's side effect and the next persisted continuation can lead the model to request the tool again on resume. The runtime narrows this window (per-iteration checkpoints, the interrupted-tool guard above) but cannot close it for arbitrary external side effects. Authored tools that perform non-idempotent external writes should use the ctx.idempotencyKey(...) helper (or their own dedup key derived from the tool-call id) so a re-executed call is a no-op at the destination. Crashed in-flight model requests are simply re-issued from the last continuation; the cost is extra tokens, never lost state.
runtime.startBackgroundWorkers() starts the delivery worker, the sandbox-sync worker, and the periodic orphan sweep, and returns a controller with stop(). The Node host (listenNodeRuntime) wires all of this automatically and each worker has an env kill-switch: OPENEVE_DELIVERY_WORKER=false, OPENEVE_SANDBOX_SYNC_WORKER=false, OPENEVE_RUN_RECOVERY=false.
OpenEve does not acquire a sandbox before every turn. Channel lifecycle events and final text delivery run without sandbox hydration. The first core file or shell tool lazily acquires the sandbox, then hydrates only requested paths or bounded candidate sets:
/memory
/history
/files
/workspace
The local adapter materializes projected paths as normal files under the sandbox root. /memory, /skills, and /workspace remain writable; /history and /files are marked read-only after hydration. Minimal hydration writes indexes, bounded history, file manifests, runtime context, and selected resources instead of every skill or memory document. Agents should write generated artifacts, transformed files, reports, scripts, and modified copies of input files under /workspace.
Sandbox acquisition is provider-neutral and ordered: connect to the current live sandbox for the agent/conversation/project key, wake the warm sandbox recorded in Postgres, then create a fresh sandbox. The foreground path creates a durable session row and sync obligation before mutating side effects, but final delivery and the next message never wait for filesystem scanning or canonical writeback.
Sandbox sync is a durable queue, not just in-process cleanup. Mutating sandbox tools enqueue a sandbox_sync_job before the side effect. The worker leases jobs, retries with backoff, recovers expired leases and dirty sessions through recoverIncompleteRuns(), writes /memory/** to durable memory documents, writes/deletes /skills/*/SKILL.md through the durable skill store, stores /workspace/** blobs and file catalog records, and records blocked failures as blocked_requires_operator without undoing final delivery. Dirty sandboxes are retained or paused until writeback completes. Operators can call sandboxSyncDiagnostics(), inspectSandboxSyncJob(jobId), and retrySandboxSyncJob(jobId) for counts, due/leased/expired/blocked status, manifests, attempts, and manual retry. Relevant knobs are OPENEVE_SANDBOX_SYNC_INLINE, OPENEVE_SANDBOX_SYNC_INTERVAL_MS, OPENEVE_SANDBOX_SYNC_BATCH_SIZE, OPENEVE_SANDBOX_SYNC_MAX_ATTEMPTS, and OPENEVE_SANDBOX_SYNC_LEASE_MS.
Postgres memory search uses indexed full-text search for keyword/exact/hybrid modes. Semantic search is optional: provide a MemoryEmbeddingProvider, set OPENEVE_MEMORY_EMBEDDINGS_ENABLED=true, apply the gated pgvector migration, and run runMemoryEmbeddingBackfill() for stale or missing embeddings. Deployments that do not enable embeddings keep deterministic full-text and portable lexical fallback behavior.
Postgres migrations are recorded in openeve_schema_migrations with id, checksum, description, package version, and applied time. PostgresStateAdapter.migrate() is idempotent and rejects checksum drift; planMigrations() reports pending/applied/skipped-optional/checksum-mismatch state without applying SQL. Existing memory file indexes can be promoted into state-backed memory documents with backfillMemoryDocumentsFromFileIndexes() when the blob adapter can read the indexed blob keys.
Security boundaries are intentionally conservative. Memory, history, files, webpages, search results, and tool output are untrusted context, not instructions. /files and /history are read-only projections; write generated or transformed outputs under /workspace. Skills are trusted instructions from the durable skill store and are loaded one selected skill at a time. Sandboxes remain isolated execution environments, and projection is an allowlisted copy of selected resources, not ambient host filesystem access.
Observability
Agent Runs-style observability is available without instrumentation.ts through the run, event, tool, checkpoint, delivery, usage, subagent, and grouped run query contracts. The Node host exposes GET /runs, GET /runs/:id, GET /runs/:id/events, and GET /runs/:id/timeline so a dashboard or CLI can inspect persisted runs without replaying provider calls.
Optional OpenTelemetry-shaped sinks receive an Eve-style span hierarchy. Each completed turn emits ai.openeve.turn as the parent span, with child spans for model steps (ai.streamText), tool calls (ai.toolCall), subagents, sandbox commands, memory sync, and delivery sends. Existing openeve.run and openeve.tool span names remain for compatibility. Spans carry trace/span IDs plus agent revision, run, session/conversation, turn, channel, model, tool, sandbox, delivery, status, usage (input/output/cached tokens, cost), and error attributes where available.
Telemetry is configured in agent/instrumentation.ts — the single home, auto-discovered and run once at startup. @openeve/otlp provides createOtlpSink / createOtlpSinkFromEnv to export these spans over OTLP/HTTP as OpenTelemetry GenAI semantic conventions to any OTLP backend (Langfuse, Phoenix, Grafana, Honeycomb, ...); see Customizing Agents → Observability.
If instrumentation.ts exports defineInstrumentation({ setup }), the runtime runs setup({ agentName, manifest, env }) before the first agent turn. recordInputs, recordOutputs, captureContent, and functionId describe capture/export preferences; OpenEve defaults to usage-level capture (token/cost/model metadata, no message bodies) and bounded previews rather than full payloads. If setup() returns a telemetry sink, the runtime uses it unless a host supplied telemetry programmatically. Exported telemetry remains supported as the compatibility fallback.
State And Blob Adapters
The state contract stores runs, run events, checkpoints, tool calls, delivery obligations, and replay data. The Postgres package provides migrations plus a driver-neutral query(sql, params) adapter. The Node production host wires that adapter to DATABASE_URL connections through pg; Neon, Supabase, local Postgres, and custom Postgres use the same schema. The schema covers agents, revisions, conversations, messages, runs, run events, checkpoints, tool calls, approval gates, delivery obligations, schedules, schedule runs, memory file indexes, file catalog records, sandbox leases, sandbox snapshots, usage records, run queue entries, and idempotency keys.
The blob contract stores context bundles, attachments, extracted text, generated artifacts, and sandbox sync bundles. The S3 package implements the blob contract against S3-compatible storage and ships R2/AWS/MinIO-style helpers. The R2 package remains a compatibility wrapper and in-memory test bucket.
Package Boundaries
@openeve/core: definitions, contracts, manifest types, anddefine*helpers.@openeve/compiler: folder discovery, validation, manifests, revisions, artifacts, routes, schedules, and deploy plans.@openeve/cli: command-line developer path.openeve: public CLI/meta package that re-exports the core framework APIs.@openeve/runtime: durable lifecycle, context bundles, local adapters, engine-neutral harness loop, tool execution, approvals, human input, replay, and delivery obligations.@openeve/pi: the canonical pi engine (multi-provider model loop) behind the internalAgentHarnesscontract.@openeve/node: Railway-friendly HTTP runtime host.@openeve/codex,@openeve/claude-code: external coding subagent harnesses.@openeve/railway: Railway deploy adapter helper and publisher.@openeve/postgres: Postgres state adapter migrations, query-client implementation, and provider presets.@openeve/s3: S3-compatible blob adapter plus AWS, MinIO, and R2 helpers.@openeve/r2: R2 compatibility wrapper.@openeve/otlp: OTLP/HTTP telemetry sink forinstrumentation.ts.@openeve/daytona: Daytona sandbox adapter boundary.@openeve/docker: Docker deploy and sandbox adapters.@openeve/e2b: E2B sandbox adapter boundary.@openeve/modal: Modal sandbox adapter boundary.@openeve/fly: Fly deploy adapter helper and publisher.@openeve/slack,@openeve/photon,@openeve/discord,@openeve/telegram,@openeve/teams: agent communication channel helpers.@openeve/github: GitHub App connection helper and preview repo-event webhook channel (signed deliveries in, issue/PR comments out).@openeve/livekit: preview voice/telephony capability package for LiveKit dispatch, SIP calls, and LiveKit-backed subagents.
OpenEve core does not import product app code. Provider support remains optional and adapter-backed.