Web API#

oxo-flow includes a built-in REST API server for building, validating, running, and monitoring bioinformatics workflows. The server is built with axum and follows a domain-driven modular monolith architecture.

API Design Conventions#

Envelope: success responses are bare JSON objects/arrays; errors are { code, message, detail?, suggestion? }
Errors: { code: "E001", message, detail?, suggestion? }
Lists: GET /api/runs returns a cursor-paginated envelope { items, next_cursor, total } (limit ≤ 500, status/q filters); other list endpoints return bare arrays (≤ 100 items)
Versioning: /api/ prefix for all endpoints
Authentication: in team/hpc mode, protected endpoints accept an Authorization: Bearer <token> session token or an X-API-Key header. The generated OpenAPI spec at GET /api/openapi.json declares both bearerAuth and apiKey security schemes; public endpoints (health, login, license, openapi.json, etc.) require no auth.
Self-discoverable: OpenAPI 3.1 spec at GET /api/openapi.json. The spec is code-generated via utoipa from the #[utoipa::path] annotations on every route handler — there is no hand-maintained static file. crates/oxo-flow-web/tests/openapi_gate.rs is the drift gate: it asserts every route in the router appears in the generated spec, so a new route without an annotation fails CI.

Structured Error Format#

All errors return a unified JSON format:

{
  "code": "AUTH_REQUIRED",
  "message": "Authentication is required for this endpoint",
  "detail": "The request did not include a valid session token or Bearer token",
  "suggestion": "Please login at POST /api/auth/login to obtain a session token"
}

Rate limiting follows the same contract: over-limit requests get 429 {"code":"RATE_LIMITED", "message":"Rate limit exceeded", "detail":"retry in Ns", ...} with a Retry-After header (sliding window, 100 requests / 60 s per client IP by default).

POST /api/auth/login adds a dedicated brute-force guard: 10 failed attempts per 5 minutes per (account, source IP) trip a lockout that answers the same structured 429 RATE_LIMITED (the countdown is in detail; the login handler's error tuple cannot attach a Retry-After header) before any credential check runs. Per-account keying means a sprayed username cannot lock out other accounts; a successful login resets the counter; OXO_FLOW_DISABLE_RATE_LIMIT=1 disables both limiters (tests/dev only).

Malformed input is answered in the same envelope, never axum's bare-text rejection: a query string that does not fit the endpoint's parameters gets 400 INVALID_QUERY, and a JSON body that does not deserialize against the endpoint's schema gets 400 INVALID_BODY — both carry axum's parse message in detail and a pointer to GET /api/openapi.json in suggestion.

Starting the Server#

# Mode 1: Personal (default) — SQLite, no auth, localhost
oxo-flow serve

# Mode 2: Team — auth enabled, network-facing
oxo-flow serve --mode team

# Mode 3: HPC — cluster-aware
oxo-flow serve --mode hpc

# Or via the standalone binary:
oxo-flow-web --mode personal -p 3000

System & Monitoring#

Health Check#

GET /api/health

Returns status, version, mode, uptime, component health (database, filesystem, scheduler, AI provider), resource usage, and license info.

components.ai_key_storage is "encrypted" when OXO_FLOW_MASTER_KEY is set, "plaintext" otherwise (issue #205).

components.engine (issue #579) reports the engine CLI the server spawns for runs, resolved once at startup by running <binary> --version:

{
  "status": "ok",
  "components": {
    "engine": {
      "version": "0.25.0",
      "path": "/usr/local/bin/oxo-flow",
      "version_mismatch": false
    }
  }
}

version_mismatch is true when the engine's major.minor differs from the server's — runs will execute under the engine's semantics, and the server log carries a drift warning with a hint to pin OXO_FLOW_BIN. The field is null when the probe failed (binary missing or unparseable --version output); the run log header remains the version record in that case.

System Info#

GET /api/system

Returns OS, architecture, PID, uptime, and version. Team/hpc modes require authentication (the endpoint left the anonymous whitelist in the v0.11 hardening).

The response also carries engine — the same engine-CLI info as /api/health's components.engine (issue #579).

Runtime Metrics#

GET /api/metrics

Returns real-time resource metrics: CPU%, memory (used/total/swap), active workflows, total requests, CPU count. Team/hpc modes require authentication.

Audit Logs#

GET /api/audit?days=7&page=1&per_page=50

Returns paginated structured audit entries:

{
  "entries": [{ "timestamp", "user", "action", "resource", "result" }],
  "days": 7,
  "page": 1,
  "per_page": 50,
  "total": 128
}

page defaults to 1 and per_page defaults to 50 (max 200). Team/hpc modes: admin-only (the trail spans every user's actions; personal mode keeps the localhost trust model).

Server-Sent Events#

GET /api/events
Accept: text/event-stream

SSE stream for real-time workflow execution events: terminal events (run_completed, run_failed, run_cancelled) plus per-rule events (rule_started, rule_completed, rule_failed, rule_skipped — parsed live from the engine's execution log). Keepalive is axum's 15-second comment ping; if the client falls behind the broadcast buffer, the stream carries a synthetic {"type":"lagged","data":{"missed":N}} event — refetch run state when you see it.

Team/hpc modes require a one-time ?ticket= minted at POST /api/events/ticket under normal bearer auth (EventSource cannot set an Authorization header; the ticket is single-use, expires in 30 s, and replaces the session token that used to travel — and leak — in the URL,

522). `POST /api/events/ticket` responds#

{"ticket": "...", "expires_in_secs": 30}. The stream is filtered to the subscriber's own runs — admins see everything. Events carry a user field (the owning user id, or null for system-wide events). The run_started event's data.username field predates the canonical user model and also carries the canonical user id (kept for wire compatibility — read the envelope user field instead).

run_completed carries a summary field: the CLI's invalidation summary extracted from the execution log — config changes, edited rule definitions, and input-set changes that invalidated checkpoint records this run (null when the run had no invalidation activity).

The stream ends when the server shuts down (SIGTERM/Ctrl+C): the response is closed as part of the graceful drain, so an EventSource sees its error/close at shutdown rather than a hung connection (#572). Clients should reconnect with backoff after an unexpected close.

Authentication & Authorization#

POST /api/auth/login
Content-Type: application/json

{"username": "admin", "password": "admin"}

Returns session token, username, and role.

Credentials are environment-driven, not hardcoded. The example above only works when OXO_FLOW_DEV_MODE=1 (accepts password == username) or when matching env passwords are set: OXO_FLOW_ADMIN_PASSWORD, OXO_FLOW_USER_PASSWORD, and OXO_FLOW_VIEWER_PASSWORD define the admin/user/viewer accounts. Without one of these the login returns 401 — set the env vars when launching oxo-flow serve (see Web System Architecture).

Identity is pinned server-side (#516): the shared user/viewer passwords never mint or adopt the admin identity (or any existing API-created account) — admin signs in only with OXO_FLOW_ADMIN_PASSWORD, and a username that belongs to a managed account requires that account's own password. OAuth identities are namespaced (<provider>:<name>) and can never collide with local usernames. Sessions store the canonical users.id resolved at login; role lookups match by id only and a session whose user row is gone (deleted user, stale pre-upgrade session) is rejected with 401.

Check Session#

GET /api/auth/me
Authorization: Bearer <token>

Returns {"authenticated": true, "username": "admin", "role": "admin"} or {"authenticated": false}.

Sign Out#

POST /api/auth/logout
Authorization: Bearer <token>

Revokes the presented session server-side (#520) — the token stops authenticating immediately instead of living out its 24-hour TTL. The client should drop its local copy (logged_out: true on success). Deleting a user (DELETE /api/users/{id}) cascades: the deleted user's sessions and API keys are removed with the account.

License Status#

GET /api/license

Returns license type, validity, commercial use flag, and contact info.

Upload License#

POST /api/license/upload

Admin-only outside personal mode; the payload is bounded (64 KiB). The raw blob is not stored — only its size and SHA-256 land in the audit trail (the endpoint does not yet activate licenses) (#519).

Users (admin)#

GET    /api/users          # List users (admin only)
POST   /api/users          # Create user
DELETE /api/users/{id}     # Delete user

Pipeline Lifecycle (v0.8 `/api/pipelines/*`)#

The pre-v0.8 /api/workflows/* endpoints no longer exist in the source — the running server exposes only the /api/pipelines/* API below.

Parse#

POST /api/pipelines/parse
Content-Type: application/json

{"toml_content": "<workflow TOML>", "format_version": "1.0"}

Returns structured pipeline: pipeline_id, name, version, rules (with summaries), dag (nodes + edges), stats. Pure function, zero side effects.

Validate#

POST /api/pipelines/validate
Content-Type: application/json

{"pipeline_id": "...", "toml_content": "<TOML>", "base_dir": "inputs"}

Returns { valid, errors: [{ code, message, rule, suggestion }] }. base_dir (optional, for missing-input checks) must be relative (no ..) and resolves inside the acting user's workspace (#521).

Prepare#

POST /api/pipelines/prepare
Content-Type: application/json

{"toml_content": "<TOML>", "resolve_wildcards": true, "apply_defaults": true}

Expands wildcards, resolves environments. Returns expanded_rules_count, wildcard_combinations, environment_setup_cmds.

Build DAG#

POST /api/pipelines/dag
Content-Type: application/json

{"pipeline_id": "...", "toml_content": "<TOML>"}

Returns { nodes, edges, parallel_groups, critical_path, metrics } as structured JSON.

Format#

POST /api/pipelines/format
Content-Type: application/json

{"toml_content": "<TOML>"}

Returns canonical TOML formatting.

Lint#

POST /api/pipelines/lint
Content-Type: application/json

{"toml_content": "<TOML>"}

Returns { valid, errors: [{ code, message, rule, suggestion }] } — validation errors plus lint-level findings (e.g. missing description).

Stats#

POST /api/pipelines/stats
Content-Type: application/json

{"toml_content": "<TOML>"}

Returns aggregate pipeline statistics.

Diff#

POST /api/pipelines/diff
Content-Type: application/json

{"toml_a": "<TOML A>", "toml_b": "<TOML B>"}

Returns structured diffs: { diffs: [{ path, category, description, severity }] }.

Export#

POST /api/pipelines/export
Content-Type: application/json

{"toml_content": "<TOML>", "format": "docker|singularity"}

Generates Dockerfile or Singularity definition.

List / Save / Get / Update / Delete#

GET    /api/pipelines              # List pipelines (most recent first, up to 100)
POST   /api/pipelines              # Save new pipeline
GET    /api/pipelines/{id}         # Get pipeline with TOML content
PUT    /api/pipelines/{id}         # Update pipeline
DELETE /api/pipelines/{id}         # Delete pipeline
POST   /api/pipelines/search       # Search by name, tags, content

The pipeline's version is always derived from the stored TOML's [workflow] version key — the TOML is the source of truth (#907 P2-5). An explicit version field on POST/PUT is ignored (never stored), so a save whose content carries version = "1.2.3" reports 1.2.3 on the next GET, and updating content without touching [workflow] version bumps the reported version to match. This mirrors the CLI, where the file on disk defines the version.

Execution & Runs#

Create Run#

PostgreSQL deployments: every /api/runs* endpoint returns 503 {"code": "RUNS_REQUIRE_SQLITE", "message", "detail", "suggestion"} — run execution requires a SQLite-backed server. Library/AI/auth domains remain available.

POST /api/runs
Content-Type: application/json

{"toml_content": "<workflow TOML>", "max_jobs": 4, "dry_run": false, "keep_going": false, "pipeline_id": "<uuid>", "cluster_id": "<cluster-id>", "samples": ["S1", "S2"], "targets": ["rule_name"], "include_files": {"envs/clinical/pixi.toml": "[env] ..."}, "config_overrides": ["min_quality=30"], "allow_protected_drift": false}

toml_content is the workflow source (required — the run is created from it, not from a saved pipeline); max_jobs, dry_run, keep_going, pipeline_id, cluster_id, samples, targets, include_files, config_overrides, and allow_protected_drift are top-level fields (not nested under a config object). Run-mutating endpoints (create/cancel/pause/resume/retry/clean/resume-checkpoint) require the user role or above — the viewer role is read-only — and a pipeline_id must be readable by the acting user (#519). This flat shape is the typed CreateRunRequest schema — the OpenAPI spec declares it as the endpoint's requestBody, and the handler parses exactly that type, so the spec, the server, and the TypeScript client stay in lockstep. Returns { run_id, status: "queued", estimated_resources, execution_plan }.

Before spawning the executor the run stages the acting user's uploaded files (POST /api/files → workspace/users/<user>/inputs/) into the run's working directory, so metadata_file and relative input paths resolve the same way they do on the CLI. Files already present in the workdir are never overwritten.

pipeline_id (optional) only associates the run with a saved pipeline — the TOML still comes from toml_content. It makes the run execute in the pipeline's persistent working directory (workspace/users/<user>/pipelines/<id>), so the checkpoint survives across re-runs. Re-running with a changed config rebuilds exactly the rules referencing the changed keys (plus their DAG downstream) — the rest keep their checkpoint records and are skipped. Runs without pipeline_id get a fresh per-run sandbox and execute everything. Malformed or unknown pipeline ids are rejected (400 INVALID_PIPELINE_ID / 404 PIPELINE_NOT_FOUND).

Execution flags are forwarded to the CLI executor:

dry_run: true spawns the preview subcommand (oxo-flow dry-run) — nothing executes; the log shows the would-be plan.
max_jobs defaults to 4 at this endpoint (the CLI flag itself defaults to 1): an omitted max_jobs still spawns the CLI with -j 4, which is also the value the resource estimate assumes.
keep_going: true maps to --keep-going (the CLI also accepts -k).
samples (array of strings, optional) maps to --samples (comma-joined).
targets (array of strings, optional) maps to -t per entry.
include_files (object, optional) stages [[include]] source files alongside the workflow: keys are workflow-relative paths (e.g. envs/clinical/pixi.toml), values are the file contents. Keys are validated as relative workspace paths — absolute paths, .. components, and degenerate keys (empty, .-only, trailing slash) are rejected (400 INVALID_INCLUDE_PATH), and the key workflow.oxoflow is rejected because it would overwrite the main workflow. Files are written beside workflow.oxoflow in the run directory, where the CLI's resolve_includes resolves them unchanged — the same machinery local oxo-flow run uses. Note: pipeline_snapshot in the run row stores only the host TOML, so status/DAG endpoints re-parse it without includes; the run itself executes correctly because the spawned CLI re-parses from disk. There is no per-key content cap beyond the request-body limit (2 MiB by default, body_limit in the server config).
config_overrides (array of strings, optional) passes CLI config overrides through to the spawned run as --arg=KEY=VALUE entries. Each entry must be a non-empty KEY=VALUE string (400 INVALID_CONFIG_OVERRIDE otherwise); full override semantics (canonicalization, unknown keys, declared-key value checks) are enforced by the CLI — it warns on unknown keys (they change nothing) and hard-fails on an invalid value for a declared key (e.g. config 'min_quality' expects an integer). A typo in an override key is therefore only a warning, not a run failure. Cluster (cluster_id) runs carry the same overrides into the remote wrapper command. Overrides and include-file keys are validated before the run row is persisted, so a 400 never leaves a ghost queued run behind (PR #930 NIT-3).
allow_protected_drift (boolean, optional, default false) forwards --allow-protected-drift to the spawned run (issue #935): the CLI's run gate refuses to execute when files under a protected_data_trees tree changed since the recorded baseline; this flag is the web escape hatch to proceed anyway (the CLI still narrates the drift into the run log). Cluster (cluster_id) runs carry the flag into the remote wrapper command the same way. Web retry keeps the gate — only the original create request may relax it, mirroring the CLI's own resume. The flag only affects actual runs; with dry_run: true it is ignored (there is no gate to bypass in preview). Preview the drift first with GET /api/runs/{id}/protected-drift (see below) before overriding.
provenance (boolean, optional, default false) forwards --provenance to the spawned run: the CLI records sha256 checksums of every rule output into the checkpoint so the run can later be audited with GET /api/runs/{id}/provenance-verify (see below) or the CLI's provenance verify. Run-only — ignored for dry_run: true (the CLI rejects the flag there). Cluster (cluster_id) runs carry the flag into the remote wrapper command the same way. Web retry does not re-apply it (checksums are recorded by the original create request only).
cluster_id (optional) names a configured SSH cluster connection: the workdir is staged to that host and executed there (see Cluster Connections).

List Runs#

GET /api/runs?limit=100&cursor=<created_at>&status=<status>&q=<search>

Cursor pagination, not page numbers: pass the created_at of the last row of the previous page as cursor (created_at < cursor). limit defaults to 100 and is capped at 500. The response is the envelope { items, next_cursor, total }, where next_cursor: null means the last page. Each item carries dry_run (#907 P3-14) — see Run Detail.

Run Status#

GET /api/runs/{id}/status

Real-time status: { status, phase, nodes: [{ rule, status, started_at, duration_ms, exit_code }], timeline, resources }. A rule that did not run — when-gated off (when_verdicts == false) or abort-cancelled (rule_runs status cancelled, which records no set membership) — is surfaced as skipped, never as a forever-pending node (issues #739, #767).

DAG Status#

GET /api/runs/{id}/dag-status

DAG JSON with per-node live status. Color-coded: green=completed, blue=running, red=failed, gray=skipped. metrics partitions the node set exactly: completed_nodes counts successful rules only, and skipped_nodes (when-gated off or abort-cancelled, issues #739/#767, surfaced in the metric since #907) keeps completed + failed + running + pending + skipped == total_nodes true at every instant. The ETA average is computed over successful nodes' durations — failures and skips never inflate it (#907 P1-2).

Diagnostics#

GET /api/runs/{id}/diagnostics

Deterministic error analysis:

{ failed_nodes: [{ rule, error_pattern, likely_cause, suggestions, auto_fixable, fix_action, relevant_log_lines }], warnings, resource_bottlenecks }

. Uses 30+ deterministic error patterns — zero AI in this endpoint. Diagnosis reads only the newest 256 KiB of execution.log (seek-based tail, torn first line dropped), never the whole file (issue #710).

Run Detail#

GET /api/runs/{id}

The full run row: { id, user_id, pipeline_id, pipeline_snapshot, status, phase, dry_run, workdir, started_at, finished_at, created_at } — the identity other run endpoints key off. (pipeline_id/pipeline_snapshot may be null for legacy rows; no pid — the host process id stays engine-internal.) dry_run mirrors the creation request (true for preview runs spawned by dry_run: true) and appears in both the list and detail responses (#907 P3-14) — legacy rows created before the column existed read as false.

Run Preview#

GET /api/runs/{id}/preview

The instance-level dry-run plan (checkpoint_preview + execution order), persisted by the executor as dry-run-preview.json when a dry-run completes. Returns 404 NO_PREVIEW for runs that never produced one — or when the workdir path is not a regular file (issue #735: a rule-writable workdir may contain special files). Files above the 16 MiB server-written maximum return 413 PREVIEW_TOO_LARGE.

Protected-Tree Drift Preview#

GET /api/runs/{id}/protected-drift

Read-only drift audit of the run's [engine] protected_data_trees (issue #935 slice 3). The response mirrors the CLI dry-run's protected_tree_drift object:

{
  "drifted_files": ["raw/S1.fq: 1024 B @2026-01-01 → 0 B @2026-01-02 (TRUNCATED)"],
  "new_trees": 0,
  "warnings": []
}

drifted_files — one human-readable line per drifted file (removed / added / changed, with (TRUNCATED) and sha256: labels — the same ManifestChange::describe rendering as the CLI dry-run and the run-gate refusal).
new_trees — trees declared under [engine] protected_data_trees that exist but have no baseline yet. The next real oxo-flow run adopts them; this preview never does.
warnings — per-tree problems that did not block the audit (unreadable tree, ignored empty path values).

The preview is read-only: unlike oxo-flow run, which persists new baselines into the checkpoint, nothing is recorded here — repeated calls are stable, and the checkpoint file is never created or modified. A run that is clean but not yet baselined reports empty drifted_files with a non-zero new_trees. The audit reads the tree as it exists now: files can still change between this call and the next run, which is exactly the window allow_protected_drift on Create Run covers. Returns 404 for unknown or foreign runs; 503 when runs are not available on this backend.

Provenance Verify#

GET /api/runs/{id}/provenance-verify

Audit the output-file checksums a run recorded into its checkpoint (CLI→web parity, oxo-flow --json provenance verify <checkpoint>). The endpoint shells out to the CLI binary against the run's workdir/.oxo-flow/checkpoint.json and forwards the verdict:

{
  "checkpoint": "/abs/workdir/.oxo-flow/checkpoint.json",
  "matched": 1,
  "mismatched": 0,
  "missing": 1,
  "cleaned": 0,
  "ok": false,
  "entries": [
    {"file": "gone.txt", "status": "missing"},
    {"file": "out.txt", "status": "matched", "expected": "sha256:…", "actual": "sha256:…"}
  ]
}

entries — one per audited file, flat (the CLI nests them under verify in its --json document); status is matched, mismatched, missing (vanished output), or cleaned (temporary output swept by the end-of-run gate). Checksums use the CLI's sha256:<hex> form.
ok — true only when mismatched == 0 && missing == 0 (mirrors the CLI's exit 1 = drift policy; drift is a 200 finding here, not an HTTP error).
note — present only when the checkpoint carries no stored checksums (the run was created without provenance: true): the CLI's hint to re-run with --provenance, plus matched: 0 and empty entries (ok: true).

Checksum tracking is opt-in: pass "provenance": true on Create Run (forwarded as the CLI's --provenance) so the run records sha256 checksums of its outputs into the checkpoint. The endpoint is read-only — verification never mutates the checkpoint. Errors: 404 NOT_FOUND (unknown/foreign run), 404 NO_WORKDIR (run has no workdir), 404 NO_CHECKPOINT (no checkpoint under the workdir), 400 INVALID_CHECKPOINT (the CLI rejected the checkpoint as malformed), 500 PROVENANCE_VERIFY_ERROR (CLI output unparseable — usually a stale binary; set OXO_FLOW_BIN to the current one), 503 when runs are not available on this backend.

AI Status#

GET /api/runs/{id}/ai-status

Node-level execution statuses pulled from the run's checkpoint — the input the AI endpoints (explain/interpret) consume.

Run Report#

GET /api/runs/{id}/report

The deterministic report object the report page renders: run identity, node statuses with timings, output file tree. Zero AI — the same object answers report/ask and feeds report/visualize. Like the diagnostics endpoint, the report summarizes only the newest 256 KiB of execution.log (issue #710); the full log stays on GET /api/runs/{id}/logs. The report is failure-aware (issue #759): a failed run's narrative headline names the failed rules and exit codes, and carries run_status + failed_rules: [{rule, exit_code, stderr_tail, stdout_tail}] from the checkpoint's structured failure data (stdout captured since #691, surfaced since #765).

Report Q&A#

POST /api/runs/{id}/report/ask
Content-Type: application/json

{"question": "which rules failed and why"}

Answers from the deterministic report (pattern matching over failed nodes, durations, file tree) — not an LLM call. Returns the answer string. On a failed run, EVERY question answers from the failure data (rule, exit code, bounded stderr excerpt — falling back to the stdout tail when stderr is empty) — never a completion claim (issues #759, #765).

Run Status exit codes and failure tails#

GET /api/runs/{id}/status, /dag-status, and /instances populate exit_code from the checkpoint's rule_runs records (issue #758): completed rules report 0, failed rules their recorded code — plus bounded stderr_tail and stdout_tail on failed nodes (issue #765; some tools print their root cause on stdout). The AI monitor (/ai-status) receives the same tails and classifies failures through the shared DiagnosticsEngine pattern table before falling back to exit-code-only signatures (#907 P1-3); its alerts carry a stable alert_id for cross-poll dedup (#907 P1-8). The AI chat get_run_status tool carries the same fields.

Report Visualization#

POST /api/runs/{id}/report/visualize
Content-Type: application/json

{"type": "files" | "durations" | "volcano"}

Chart data built from real sources: files maps the run's file tree ({name, size_bytes}), everything else maps per-rule durations from the checkpoint. No canned rows.

Smart Retry#

POST /api/runs/{id}/retry
Content-Type: application/json

{"skip_succeeded": true}

The retry spawns a real run, so it pays the same #213 runs-per-minute limiter and the quota pre-flight as run creation — a retry that exceeds the budget answers 429 RUN_RATE_LIMITED / QUOTA_EXCEEDED (#519). from_rule is not supported: the engine has no "re-run from rule X downstream" mode (--target selects the upstream closure), so the field is rejected with 400 UNSUPPORTED_FIELD instead of being silently ignored.

The retry really executes: the returned new_run_id is a real run in the database (same workdir, same owner), spawned with --resume-failed --rerun so the failed rules re-execute despite their existing outputs and the checkpoint's cascade invalidation re-runs their downstream dependents. The retry replays the original run's config_overrides (PR #930 NIT-8): the new row carries the stored overrides forward and the spawned CLI receives the same --arg=KEY=VALUE entries, so a retry re-executes with the configuration it claims to redo instead of silently falling back to defaults. Returns { new_run_id, will_rerun: [...], will_skip: [...] } — plus an optional note when the plan is empty on a previously failed run (it died before executing any rule, e.g. a configuration error at spawn; the retry re-fails identically unless the error is fixed — issue #760).

Cancel#

POST /api/runs/{id}/cancel

Cancels a running/pending run.

Pause / Resume#

POST /api/runs/{id}/pause
POST /api/runs/{id}/resume

Pauses a running run ({"reason": "..."} optional; other body fields are ignored) and resumes it. On resume, a from_rule field is rejected with 400 UNSUPPORTED_FIELD — a paused run continues in place, so re-running from a specific rule has no meaning (see Smart Retry above: cancel the run and retry it instead). A terminal run in either endpoint returns 409 RUN_NOT_ACTIVE.

Logs#

GET /api/runs/{id}/logs

Returns the execution log text — bounded to the newest 8 MiB with an explicit [log truncated …] marker line when the log is larger (issue #734; buffering a multi-GB log per request was a memory-amplification surface). Read the full file from the run workdir when the marker appears.

Results#

GET /api/runs/{id}/results

Returns output file tree with sizes and types.

Files — download / preview / zip#

GET /api/runs/{id}/files?path=<relative-path>
GET /api/runs/{id}/files?path=<relative-path>&preview=true

The read-only result-delivery layer:

file → bytes with ETag, Content-Disposition: attachment, and single-range support (Range: bytes=a-b → 206; malformed/multi-range requests degrade to the full body per RFC 9110)
directory → a streaming STORE-mode zip (no temporary archive)
preview=true → truncated JSON for text-ish formats (100 KB cap) or inline image bytes; other types return 415 NO_PREVIEW
paths are sandboxed to the run's workdir (traversal rejected); sensitive filenames (.env, keys, credentials) are never served

Instances#

GET /api/runs/{id}/instances

The sample×rule instance table: every expanded instance the checkpoint knows about (qc_auto-discovered_S1 → rule qc, group auto-discovered, sample S1) with status, duration, and exit code — answers "which sample under which rule failed".

Upload & list user inputs#

POST /api/files        # multipart: field "path" (optional subdir) + file parts
GET  /api/files        # list the acting user's uploaded inputs

Uploads land in workspace/users/<user>/inputs/ (chunked to disk, 8 GiB per-file cap).

Data Discovery#

Analyze Data#

POST /api/data/analyze
Content-Type: application/json

{"paths": ["/data/*.fastq.gz", "/data/*.bam"], "max_depth": 2}

Deterministic file scanning + format inference + pipeline recommendation. Returns { files: [{ path, size, format, format_confidence, paired_with?, sample_name? }], summary, suggested_workflow }. Format detection uses filename extension matching — not AI.

Reference Discovery#

POST /api/data/reference
Content-Type: application/json

{"genome": "hg38", "components": ["fasta", "gtf", "star_index"]}

Finds installed reference genome components and reports missing ones with download commands.

Reference Status#

GET /api/data/reference/status

Which common reference genomes have their expected components present at the standard /data/references/ locations — the deterministic counterpart to the POST discovery above. Probed components per genome: hg38 and mm10 check genome.fa, star (STAR index directory) and genes.gtf; mm39 checks only genome.fa and star (the [[references]] auto-build needs no annotation to construct an index). A genome counts as installed only when at least one probed component exists; builds with nothing on disk are reported under missing_genomes. Response shape (components is a legacy alias of found — both carry the absolute paths that exist on disk):

{
  "installed": [
    { "genome": "hg38", "components": ["/data/references/hg38/genome.fa", "..."],
      "found": ["..."], "missing_components": ["..."] }
  ],
  "missing_genomes": ["mm39"],
  "missing": ["/data/references/mm39/genome.fa"],
  "download_commands": ["# mm39: stage the expected files under /data/references/mm39 (genome.fa, STAR index) ..."]
}

download_commands carries staging guidance, not fetchable URLs: oxo-flow hosts no reference mirror. Point users at reference_dir + [[references]] auto-build in the workflow format for index construction.

Data Perception#

POST /api/data/perceive
Content-Type: application/json

{"paths": ["/data/"], "description": "RNA-seq paired-end human samples"}

Data-profiling report built from paths (the same scanner as /api/data/analyze) or from a natural-language description. Powers the "what data do I have" panel.

Samplesheet Parsing#

POST /api/data/samplesheet/parse
Content-Type: application/json

{"content": "sample,fastq_r1,fastq_r2,condition\nS1,a_R1.fastq.gz,a_R2.fastq.gz,ctrl"}

Parses pasted CSV/TSV samplesheet content into the standard column structure (sample, fastq_r1, fastq_r2, condition). Returns { format, samples: [...] } — the bridge from a spreadsheet to [[sample_groups]].

Verify Data#

POST /api/data/verify
Content-Type: application/json

{"tree": "inputs/delivery-2026-10", "md5_content": "d41d8cd98f00b204e9800998ecf8427e  raw/S1.fq\nd41d8cd98f00b204e9800998ecf8427e  raw/S2.fq\n"}

Audits a delivered data tree against a vendor md5 sidecar — the web slice of the CLI's oxo-flow verify-data command (issue #935 P2). tree is workspace-relative (uploads live under inputs/, run workdirs under runs/<id>/; absolute paths and .. are rejected). The sidecar arrives inline in the standard md5sum text format and is staged server-side; the CLI subprocess streams an md5 over every file.

Response is HTTP 200 even when drift is found — the counts ARE the finding:

{
  "tree": "inputs/delivery-2026-10",
  "matched": 12, "mismatched": 1, "missing": 0, "extra": 1,
  "entries": [
    {"file": "raw/S1.fq", "status": "matched", "expected": "d41d...", "actual": "d41d..."},
    {"file": "raw/S2.fq", "status": "mismatched", "expected": "aa...", "actual": "bb..."},
    {"file": "raw/unlisted.txt", "status": "extra", "actual": "cc..."}
  ]
}

status vocabulary: matched, mismatched (includes unreadable files, with a note), missing, extra (present but unlisted — always non-compliant). An extra entry may carry a note instead of an actual checksum in two cases: the file could not be read for checksumming, or a subdirectory of the tree could not be traversed at all (tree walk failed note — the audit is incomplete, the named path may be a directory, and the verdict has already failed with exit-equivalent extra > 0). An empty sidecar returns all-zero counts with "note": "The md5 sidecar lists no files." and no extras scan. A sidecar the CLI hard-rejects (malformed line, duplicate path, traversal) yields 400 INVALID_SIDECAR with the CLI's stderr as detail. HTTP errors are otherwise reserved for infrastructure problems: 400 INVALID_TREE (unknown path or traversal), 500 VERIFY_ERROR (spawn failure or unparseable CLI output). Errors use the structured {code, message, detail, suggestion} shape. The inline sidecar counts against the server's default ~2 MiB JSON body limit (~50k entries); larger sidecars should be split by directory.

Templates#

GET    /api/templates
POST   /api/templates
GET    /api/templates/{id}
DELETE /api/templates/{id}

Built-in and user-created pipeline templates. System templates are read-only. The list endpoint does not accept filter query parameters.

Plugins#

Validate Plugin#

POST /api/plugins/validate
Content-Type: application/json

{"manifest": {"name": "...", "version": "1.0", "plugin_type": "rule"}, "trusted_keys": {"key1": "hex..."}}

Validates a plugin manifest and optionally verifies its HMAC signature against trusted keys. Returns { valid, name, version, plugin_type, signature_valid, errors }.

AI (Phase 2 — calls deterministic APIs above)#

POST /api/ai/translate          # NL intent → validated .oxoflow (JSON response)
POST /api/ai/translate/stream   # Same, streamed over SSE (progress → done events)
POST /api/ai/explain            # Explain run failure + suggest fix (run owner or admin)
POST /api/ai/interpret          # Interpret results with caveats (run owner or admin)
POST /api/ai/optimize           # Optimize pipeline parameters (pipeline reader via id; owner-supplied TOML unaffected)
GET  /api/ai/config             # Get AI provider configuration (public)
POST /api/ai/config             # Update the shared provider (admin-only outside personal mode)
POST /api/ai/test               # Test the provider (admin-only outside personal mode)
GET  /api/ai/config/user        # The acting user's private provider config
PUT  /api/ai/config/user        # Set the acting user's private provider config
GET  /api/ai/config/server      # Server-level provider config (admin-only outside personal mode)
PUT  /api/ai/config/server      # Set the server-level provider config
GET  /api/ai/config/effective   # The config a run would use: user → server → env fallback chain
GET  /api/knowledge/tools       # Tool catalog the AI can call
GET  /api/knowledge/skills      # Skill catalog the AI can follow

Both knowledge endpoints accept ?q=<query>&limit=<1-50> (default 20). /api/knowledge/tools treats an empty q as a browse: it returns the first limit Bioconda entries. /api/knowledge/skills tokenizes the query and ranks matches (rare terms first); an empty or unmatched q returns an empty skills array — total is always the full catalog size, not the hit count.

The translate endpoints take { "intent": "<natural-language description>", "context": { "data_analysis_id": "<uuid>" } } — intent is REQUIRED (sending only a prompt field fails deserialization); context is optional.

The config endpoints form a three-level resolution chain: a user's private provider overrides the server-wide one, which overrides environment variables. GET /api/ai/config/effective shows the resolved result and which level supplied each field. API keys are stored encrypted at rest (#205) and never echoed back — responses mark key_set instead.

See AI Translation Layer for details.

Chat (v0.8 AI Companion)#

POST /api/chat/send        # Streamed chat over SSE
POST /api/chat/send/json   # Same, single JSON response
GET  /api/chat/sessions    # The acting user's chat sessions

OAuth#

POST /api/auth/oauth/authorize   # Begin the OAuth flow → provider authorize URL
POST /api/auth/oauth/callback    # Provider callback → session token

The redirect URI is OXO_FLOW_OAUTH_REDIRECT_URI when set, with a fixed fallback of http://localhost:3000/api/auth/oauth/callback (see the environment table in AGENTS.md).

The local user record is idempotent on the provider identity: the canonical users row is keyed by the namespaced id oauth:{provider}:{provider_user_id} (e.g. the ORCID iD), so re-login reuses the same row. Display names are NOT unique across provider accounts — two ORCID accounts can share a given name — so when a display name is already taken the row is persisted under a fallback username derived from the unique provider id ({provider}:{provider_user_id}); the login still succeeds and the session always resolves to a real user row. When the user row cannot be ensured (database unavailable or insert fails), the callback answers 503 {code:"DB_ERROR"} instead of minting a session that would fail closed on every subsequent request.

Collaboration (Phase 3)#

POST /api/pipelines/{id}/fork    # Fork into workspace (owner = session user)
POST /api/pipelines/{id}/share   # Share (link/public only; URL follows the actual transport)
POST /api/pipelines/import       # Import from oxo+http(s):// share URL
GET  /api/share/{token}          # PUBLIC landing payload (no session required)

GET /api/share/{token} powers the share landing page: pipeline identity, DAG rule order, TOML, owner, expiry, and the most recent terminal run — the token itself is the authorization. Expired links return 410.

Share URLs minted by POST .../share use oxo+http:// by default (the server has no native TLS) and oxo+https:// only when OXO_FLOW_PUBLIC_URL declares a public HTTPS origin; the import endpoint accepts both schemes. An override that carries its own mount path (e.g. https://flows.example.com/oxo-flow behind a sub-path proxy) replaces the local --base-path prefix instead of being appended to it.

See Collaboration for details.

Version History#

GET  /api/pipelines/{id}/revisions        # Snapshot list (newest first, ≤ 50)
GET  /api/pipelines/{id}/revisions/{rev}  # One snapshot's full TOML
POST /api/pipelines/{id}/rollback         # {"revision_id": ...} — restore

Every save/update snapshots the previous content; rollback preserves the current version as a new revision (nothing is lost).

Run Administration#

POST /api/runs/{id}/clean              # CLI clean on the run's workdir
POST /api/runs/{id}/resume-checkpoint  # CLI resume from .oxo-flow/checkpoint.json

resume-checkpoint continues an unfinished run in place as a NEW run row ({"max_jobs": 2} optional). Both are ownership-checked like every other run endpoint.

Webhooks#

GET /api/webhook   # admin-only outside personal mode (#519) — { enabled, url, secret_set, events, signature_scheme }, secret never echoed
PUT /api/webhook   # admin-only outside personal mode — {"enabled": true, "url": "https://...", "secret": "...", "events": [...], "signature_scheme": "..."}

Runs POST a signed payload to the configured URL on terminal states. The signature scheme is configurable:

sha256-keyed (default) — sha256(secret‖body), the pre-v0.12 format. The default keeps existing webhook consumers working across upgrades.
hmac-sha256 — RFC 2104 HMAC (X-OxoFlow-Signature: hmac-sha256=<hex>), the recommended opt-in for new deployments.

Validation on PUT (issue #907 N5): url must parse as an absolute http:// or https:// URL with a host (400 INVALID_URL otherwise — a bare https:// prefix no longer passes). Enabling delivery requires a usable signing secret: either supply secret in the request or leave a previously stored one in place (400 WEBHOOK_SECRET_REQUIRED otherwise) — a webhook URL is a bearer credential, and unsigned delivery lets anyone who learns the URL forge events. Disabling (enabled: false) never demands a secret. A stored secret that fails to decrypt (e.g. the master key was rotated) reads as absent: enable is refused until a fresh secret is supplied (#907, review NIT-1) — silently signing with an empty key would produce forgeable deliveries.

Admin-only outside personal mode — the endpoint is shared infrastructure.

API Keys#

POST   /api/auth/keys       # {"name": "ci-bot"} → { id, name, key } (shown once)
GET    /api/auth/keys       # The acting user's keys (hashes only)
DELETE /api/auth/keys/{id}  # Revoke immediately

Machine credentials: send X-API-Key: oxo_... instead of a Bearer session. Keys resolve to the same ownership context (a key's requests see exactly what its owner sees), are stored as SHA-256 hashes, and revocation is immediate.

Quota#

Runs pre-flight the quota tracker with the workflow's declared threads and memory; over-limit requests get 429 QUOTA_EXCEEDED with the violation list.

GET  /api/quota           # current limits and usage
PUT  /api/quota           # update limits (admin-only; personal mode bypasses admin checks)

PUT /api/quota accepts { max_concurrent_runs, max_total_threads, max_total_memory_mb, max_runs_per_day } — all four fields are required (no defaults applied server-side). Usage is visible at GET /api/quota ({ enabled, limits: {…}, usage: { active_runs, used_threads, used_memory_mb, runs_today } }).

Cluster Connections & Remote Execution#

GET    /api/clusters                  # configured SSH connections
POST   /api/clusters                  # upsert (admin-only outside personal mode)
DELETE /api/clusters/{id}
POST   /api/clusters/{id}/probe       # SSH connectivity + scheduler detection (admin-only outside personal mode)

The stored ssh_key is a secret (#517): it is never returned by any response (including the upsert echo) — clients see

ssh_key_set: true |
false

instead. When OXO_FLOW_MASTER_KEY is set, the key is sealed at rest with the same AES-256-GCM scheme as AI provider keys. POST /api/runs accepts cluster_id: the run then stages its workdir to the remote host (tar over stdio — no rsync), executes under a per-run nohup wrapper, and pulls the results back on completion so every downstream endpoint (logs, files, report) works unchanged. See Run on a cluster.

HPC#

GET /api/hpc

Returns scheduler status (SLURM, PBS/Torque, LSF, SGE), available queues, and node count. This route is only mounted when the server runs in hpc mode.

DAG Editing#

POST /api/pipeline/{id}/command   # Apply an edit command (add/remove rule, connect, ...)
POST /api/pipeline/{id}/undo      # Revert the last edit (JSON body: {"toml_content": ...})
POST /api/pipeline/{id}/redo      # Re-apply the last undone edit (JSON body: {"toml_content": ...})

undo/redo require a JSON body carrying the caller's current TOML; the stacked edit is applied only when it matches that state (otherwise 404). See DAG Edit API for the full command reference.