Skip to content

DAG Edit API#

The DAG Edit API enables programmatic manipulation of workflow DAGs — adding and removing rules, connecting and disconnecting dependencies, and full undo/redo support. This API powers the Web UI's interactive DAG editor.


Overview#

The DAG Edit domain (crates/oxo-flow-web/src/domains/dag/service.rs) provides a command-queue architecture with automatic TOML round-tripping and validation on every edit. Each edit is:

  1. Parsed — the current TOML content is parsed as a WorkflowConfig (gate: it must parse before any mutation)
  2. Applied — the command mutates the toml_edit document in place, preserving comments and formatting
  3. Validated — the edited document is re-parsed and validated through the workflow validation pipeline

Edits that fail validation — error-severity findings such as a depends_on reference to an unknown rule — roll back atomically: the response reports success: false with validation_errors populated and toml_content echoes the caller's unchanged input, so a client that adopts the returned TOML never holds a document the engine would reject. Edits that break parsing of the edited document — invalid TOML syntax, unknown fields, or duplicate rule names — are rejected with HTTP 400 (DAG_EDIT_ERROR) and the previous state is kept. Rejected edits never enter the undo/redo history.

Endpoint#

All edit commands are sent to a single endpoint, with the pipeline ID in the URL path:

POST /api/pipeline/{id}/command

The pipeline ID is a logical key for the undo/redo stacks (paired with the acting identity — see History scope, caps, and lifetime) — it does not need to reference a saved pipeline.

Commands#

All commands share a common request envelope. The request body must include the current TOML content (toml_content) plus the command.

The commands below reflect the v0.11+ extended API: add_rule accepts a complete rule table, update_rule patches any rule fields through a TOML table (with null meaning "remove this key"), rename_rule renames a rule and rewrites every reference to it, and update_workflow replaces top-level sections. update_params remains as a legacy alias of update_rule.

{
  "toml_content": "<current workflow TOML>",
  "source": "dag_editor",
  "operation": "<command>",
  "payload": { ... }
}

source is one of dag_editor (the interactive editor), chat, or proposal. operation is one of add_rule, remove_rule, connect, disconnect, rename_rule, update_rule/update_params, or update_workflow.

add_rule#

Add a new rule to the workflow.

Payload:

Field Type Required Description
rule Table Yes (full form) Complete rule table — any rule field from the workflow format (input, output, shell, environment, resources, envvars, when, retries, tags, …)
name / shell String No (legacy form) Legacy minimal shape, kept for compatibility

Example (full form):

{
  "toml_content": "<current workflow TOML>",
  "source": "dag_editor",
  "operation": "add_rule",
  "payload": {
    "rule": {
      "name": "fastp_trim",
      "description": "Trim adapters",
      "input": ["raw/{sample}_R1.fastq.gz", "raw/{sample}_R2.fastq.gz"],
      "output": ["trimmed/{sample}_R1.fastq.gz"],
      "shell": "fastp --in1 {input[0]} --out1 {output[0]}",
      "environment": {"conda": "envs/fastp.yaml"},
      "resources": {"threads": 8, "memory": "16G"}
    }
  }
}

Result: The new rule is appended to the rule list and core parsing validates the field set (unknown fields are rejected with a parse error).


remove_rule#

Remove a rule and all references to it.

Payload:

Field Type Required Description
name String Yes Name of the rule to remove

Example:

{
  "toml_content": "<current workflow TOML>",
  "source": "dag_editor",
  "operation": "remove_rule",
  "payload": { "name": "obsolete_step" }
}

Result: The rule is removed from the rule list. All depends_on entries in other rules that reference this rule are cleaned up automatically.


connect#

Add an explicit dependency edge between two rules.

Payload:

Field Type Required Description
from String Yes Name of the upstream rule (runs first)
to String Yes Name of the downstream rule (runs after from)

Example:

{
  "toml_content": "<current workflow TOML>",
  "source": "dag_editor",
  "operation": "connect",
  "payload": { "from": "fastqc", "to": "trim_reads" }
}

Result: "fastqc" is added to trim_reads's depends_on list. If the dependency already exists, the command is idempotent (no duplicate). Returns an error if the target rule doesn't exist, if from equals to, or if the edge would close a cycle — including cycles formed with file-inferred edges (an output→input match plus the drawn edge). Self-connections and cyclic connections are rejected before the document is touched, so the workflow stays valid.


disconnect#

Remove an explicit dependency edge between two rules.

Payload:

Field Type Required Description
from String Yes Name of the upstream rule
to String Yes Name of the downstream rule

Example:

{
  "toml_content": "<current workflow TOML>",
  "source": "dag_editor",
  "operation": "disconnect",
  "payload": { "from": "fastqc", "to": "trim_reads" }
}

Result: "fastqc" is removed from trim_reads's depends_on list. File-based dependencies (inferred from input/output matching) are not affected — only explicit depends_on entries are managed by connect/disconnect.


rename_rule#

Rename a rule and rewrite every reference to it.

Payload:

Field Type Required Description
name String Yes Current name of the rule
new_name String Yes New rule name (must stay unique; the usual rule-name charset applies)

Example:

{
  "toml_content": "<current workflow TOML>",
  "source": "dag_editor",
  "operation": "rename_rule",
  "payload": { "name": "fastqc", "new_name": "fastqc_qc" }
}

Result: The rule's name key is rewritten in place (inline comments on that line are preserved), and every depends_on entry, extends reference, and scatter.gather target that pointed at the old name now points at the new one. Renaming to an existing rule's name fails with a parse error; an unknown name fails with Rule '<name>' not found.


update_rule#

Update any fields on an existing rule.

Payload:

Field Type Required Description
name String Yes Name of the rule to update
patch Table Yes Rule fields to replace. Nested tables (resources, environment, envvars) replace wholesale — send complete sub-objects. A null value removes the key (e.g. dropping an environment)

Example:

{
  "toml_content": "<current workflow TOML>",
  "source": "dag_editor",
  "operation": "update_rule",
  "payload": {
    "name": "bwa_align",
    "patch": {
      "shell": "bwa-mem2 mem -t {threads} {config.reference} {input} | samtools sort -o {output}",
      "input": ["raw/{sample}_R1.fastq.gz", "raw/{sample}_R2.fastq.gz"],
      "resources": {"threads": 32, "memory": "64G"},
      "retries": null
    }
  }
}

Result: Patch keys replace the rule's fields; all unpatched fields are preserved. update_params ({name, threads, shell}) remains as a legacy alias.

update_workflow#

Patch top-level TOML sections (workflow, config, defaults, …). null removes a key; an object patch on a key that is already a table is merged into that table (comments preserved); everything else is replaced wholesale.

{
  "toml_content": "<current workflow TOML>",
  "source": "dag_editor",
  "operation": "update_workflow",
  "payload": {
    "patch": {"workflow": {"name": "renamed", "version": "0.32.0", "description": "d"}}
  }
}

Undo & Redo#

The DAG Edit API maintains one undo/redo stack per (acting identity, pipeline id) pair, with documented depth and pipeline caps (see History scope, caps, and lifetime).

undo#

Reverts the last edit, restoring the previous TOML state.

POST /api/pipeline/{id}/undo
Content-Type: application/json

{"toml_content": "<the client's current TOML>"}

The request requires a JSON body with toml_content set to the caller's current TOML state. The undo stack's top transition is applied only when its resulting state matches that content exactly.

Failure codes (all fail closed — no edit is produced and no history is consumed):

Status Code Meaning
409 EDIT_HISTORY_UNAVAILABLE No history exists for this identity/pipeline — the first edit has not happened, or the stack was lost to a server restart or LRU eviction. suggestion tells the caller to re-apply the change.
409 EDIT_HISTORY_STALE History exists, but the supplied toml_content is not the tip of the recorded history (stale or racing client).
404 NO_UNDO History exists but every recorded edit is already undone.

Returns { "toml_content": "<previous TOML>" } — i.e. the state before the last edit.

redo#

Re-applies the last undone edit.

POST /api/pipeline/{id}/redo
Content-Type: application/json

{"toml_content": "<the client's current TOML>"}

Same contract as undo: the JSON body is required, and toml_content must match the current state (the result of the last undo). The failure codes are the same: 409 EDIT_HISTORY_UNAVAILABLE, 409 EDIT_HISTORY_STALE, and 404 NO_REDO when history exists but nothing was undone.

Note: Performing a new edit after an undo clears the redo stack (standard undo/redo semantics).

History scope, caps, and lifetime#

Defined in crates/oxo-flow-web/src/domains/dag/service.rs (issue #1137):

  • Scope — a stack is keyed by the acting identity and the pipeline id. The identity is the users.id the auth middleware resolves for the request (session or API key); personal mode has no auth middleware, so every request falls back to the default pseudo-user. Two identities editing the same pipeline id get independent stacks: neither can observe or undo the other's edits.
  • Depth cap — MAX_EDIT_HISTORY_DEPTH = 50 transitions per stack. Pushing past the cap drops the oldest transition, so undo can walk back at most 50 edits.
  • Pipeline cap — MAX_EDIT_HISTORY_PIPELINES = 256 stacks process-wide. Recording an edit for a new (identity, pipeline) pair evicts the least-recently-used stack.
  • Lifetime — history lives in RAM only; it is not persisted. A server restart (or LRU eviction) empties a stack, and a subsequent undo/redo fails closed with 409 EDIT_HISTORY_UNAVAILABLE and a remediation suggestion. It never silently no-ops, and it never applies an edit to a document the server no longer holds history for.

Response Format#

All edit commands return a DagEditResponse:

{
  "success": true,
  "toml_content": "[workflow]\nname = \"...\"\n...",
  "validation_errors": []
}
Field Type Description
success Boolean true if validation passed (no hard errors)
toml_content String The complete workflow TOML after the edit
validation_errors Array of String Validation error-severity messages (lint warnings such as a missing description are collected separately and are not part of this response)

If validation fails, success is false, validation_errors is populated, and the edit is rolled back atomically: toml_content echoes the caller's unchanged input (see Overview). Rejected edits never enter the undo/redo history.

Malformed commands (unknown operation, missing required payload fields such as the rule name, an update_rule target rule that does not exist, or a connect/disconnect/update_params target rule that does not exist) are rejected with HTTP 400 and code DAG_EDIT_ERROR; the edit is not applied. The same applies to edits whose resulting document no longer parses as a workflow (invalid TOML syntax, unknown fields, duplicate rule names).


Important Notes#

  • File-based edges are rewired through update_rule, not connect/disconnect. The connect/disconnect commands only manage depends_on entries. File-based dependencies (inferred from input/output matching) are controlled by the input and output fields of rules: update_rule patches those fields like any other, so editing a rule's input (or output) list is how the edit API adds or removes an implicit edge. There is no dedicated edge command for them — an edit that changes what a rule reads changes the inferred edges with it.
  • All edits are validated. The edit API runs the full workflow validation pipeline after every command. Validation findings (e.g., a depends_on entry naming an unknown rule) are returned in validation_errors with success: false, and the edit is rolled back atomically — toml_content echoes the caller's unchanged input and the undo/redo history is untouched. Edits that break parsing — invalid TOML syntax, unknown fields, or duplicate rule names — are rejected with HTTP 400 instead. Cycle detection is part of the edit validation path: validate_format builds the DAG with parsed config values on every edit, so a cycle introduced via depends_on edits surfaces immediately as success: false with the DAG error: (E006) message in validation_errors (the edit is rolled back atomically). See DAG Engine.

  • TOML round-tripping preserves formatting. Edits mutate the parsed TOML document in place (toml_edit), so the author's comments and formatting survive every edit; the document is never re-serialized through the canonical format::format_workflow.

  • Undo/redo is in-memory, identity-scoped, and bounded. Stacks are keyed by (acting identity, pipeline id), capped at 50 transitions each and 256 stacks process-wide (LRU), and live only for the duration of the server process. They do not persist across server restarts; undo/redo after a restart fails closed with 409 EDIT_HISTORY_UNAVAILABLE (see History scope, caps, and lifetime).

See Also#