DAG Edit API#
The DAG Edit API enables programmatic manipulation of workflow DAGs — adding and removing rules, connecting and disconnecting dependencies, and full undo/redo support. This API powers the Web UI's interactive DAG editor.
Overview#
The DAG Edit domain (crates/oxo-flow-web/src/domains/dag/service.rs) provides a command-queue architecture with automatic TOML round-tripping and validation on every edit. Each edit is:
- Parsed — the current TOML content is parsed as a
WorkflowConfig(gate: it must parse before any mutation) - Applied — the command mutates the
toml_editdocument in place, preserving comments and formatting - Validated — the edited document is re-parsed and validated through the workflow validation pipeline
Edits that fail validation — error-severity findings such as a
depends_on reference to an unknown rule — roll back atomically: the
response reports success: false with validation_errors populated and
toml_content echoes the caller's unchanged input, so a client that
adopts the returned TOML never holds a document the engine would reject.
Edits that break parsing of the edited document — invalid TOML syntax,
unknown fields, or duplicate rule names — are rejected with HTTP 400
(DAG_EDIT_ERROR) and the previous state is kept. Rejected edits never
enter the undo/redo history.
Endpoint#
All edit commands are sent to a single endpoint, with the pipeline ID in the URL path:
The pipeline ID is a logical key for the undo/redo stacks (paired with the acting identity — see History scope, caps, and lifetime) — it does not need to reference a saved pipeline.
Commands#
All commands share a common request envelope. The request body must include the
current TOML content (toml_content) plus the command.
The commands below reflect the v0.11+ extended API:
add_ruleaccepts a complete rule table,update_rulepatches any rule fields through a TOML table (withnullmeaning "remove this key"),rename_rulerenames a rule and rewrites every reference to it, andupdate_workflowreplaces top-level sections.update_paramsremains as a legacy alias ofupdate_rule.
{
"toml_content": "<current workflow TOML>",
"source": "dag_editor",
"operation": "<command>",
"payload": { ... }
}
source is one of dag_editor (the interactive editor), chat, or proposal.
operation is one of add_rule, remove_rule, connect, disconnect,
rename_rule, update_rule/update_params, or update_workflow.
add_rule#
Add a new rule to the workflow.
Payload:
| Field | Type | Required | Description |
|---|---|---|---|
rule |
Table | Yes (full form) | Complete rule table — any rule field from the workflow format (input, output, shell, environment, resources, envvars, when, retries, tags, …) |
name / shell |
String | No (legacy form) | Legacy minimal shape, kept for compatibility |
Example (full form):
{
"toml_content": "<current workflow TOML>",
"source": "dag_editor",
"operation": "add_rule",
"payload": {
"rule": {
"name": "fastp_trim",
"description": "Trim adapters",
"input": ["raw/{sample}_R1.fastq.gz", "raw/{sample}_R2.fastq.gz"],
"output": ["trimmed/{sample}_R1.fastq.gz"],
"shell": "fastp --in1 {input[0]} --out1 {output[0]}",
"environment": {"conda": "envs/fastp.yaml"},
"resources": {"threads": 8, "memory": "16G"}
}
}
}
Result: The new rule is appended to the rule list and core parsing validates the field set (unknown fields are rejected with a parse error).
remove_rule#
Remove a rule and all references to it.
Payload:
| Field | Type | Required | Description |
|---|---|---|---|
name |
String | Yes | Name of the rule to remove |
Example:
{
"toml_content": "<current workflow TOML>",
"source": "dag_editor",
"operation": "remove_rule",
"payload": { "name": "obsolete_step" }
}
Result: The rule is removed from the rule list. All depends_on entries in other rules that reference this rule are cleaned up automatically.
connect#
Add an explicit dependency edge between two rules.
Payload:
| Field | Type | Required | Description |
|---|---|---|---|
from |
String | Yes | Name of the upstream rule (runs first) |
to |
String | Yes | Name of the downstream rule (runs after from) |
Example:
{
"toml_content": "<current workflow TOML>",
"source": "dag_editor",
"operation": "connect",
"payload": { "from": "fastqc", "to": "trim_reads" }
}
Result: "fastqc" is added to trim_reads's depends_on list. If the dependency already exists, the command is idempotent (no duplicate). Returns an error if the target rule doesn't exist, if from equals to, or if the edge would close a cycle — including cycles formed with file-inferred edges (an output→input match plus the drawn edge). Self-connections and cyclic connections are rejected before the document is touched, so the workflow stays valid.
disconnect#
Remove an explicit dependency edge between two rules.
Payload:
| Field | Type | Required | Description |
|---|---|---|---|
from |
String | Yes | Name of the upstream rule |
to |
String | Yes | Name of the downstream rule |
Example:
{
"toml_content": "<current workflow TOML>",
"source": "dag_editor",
"operation": "disconnect",
"payload": { "from": "fastqc", "to": "trim_reads" }
}
Result: "fastqc" is removed from trim_reads's depends_on list. File-based dependencies (inferred from input/output matching) are not affected — only explicit depends_on entries are managed by connect/disconnect.
rename_rule#
Rename a rule and rewrite every reference to it.
Payload:
| Field | Type | Required | Description |
|---|---|---|---|
name |
String | Yes | Current name of the rule |
new_name |
String | Yes | New rule name (must stay unique; the usual rule-name charset applies) |
Example:
{
"toml_content": "<current workflow TOML>",
"source": "dag_editor",
"operation": "rename_rule",
"payload": { "name": "fastqc", "new_name": "fastqc_qc" }
}
Result: The rule's name key is rewritten in place (inline comments on
that line are preserved), and every depends_on entry, extends reference,
and scatter.gather target that pointed at the old name now points at the
new one. Renaming to an existing rule's name fails with a parse error; an
unknown name fails with Rule '<name>' not found.
update_rule#
Update any fields on an existing rule.
Payload:
| Field | Type | Required | Description |
|---|---|---|---|
name |
String | Yes | Name of the rule to update |
patch |
Table | Yes | Rule fields to replace. Nested tables (resources, environment, envvars) replace wholesale — send complete sub-objects. A null value removes the key (e.g. dropping an environment) |
Example:
{
"toml_content": "<current workflow TOML>",
"source": "dag_editor",
"operation": "update_rule",
"payload": {
"name": "bwa_align",
"patch": {
"shell": "bwa-mem2 mem -t {threads} {config.reference} {input} | samtools sort -o {output}",
"input": ["raw/{sample}_R1.fastq.gz", "raw/{sample}_R2.fastq.gz"],
"resources": {"threads": 32, "memory": "64G"},
"retries": null
}
}
}
Result: Patch keys replace the rule's fields; all unpatched fields are
preserved. update_params ({name, threads, shell}) remains as a legacy
alias.
update_workflow#
Patch top-level TOML sections (workflow, config, defaults, …). null
removes a key; an object patch on a key that is already a table is merged
into that table (comments preserved); everything else is replaced wholesale.
{
"toml_content": "<current workflow TOML>",
"source": "dag_editor",
"operation": "update_workflow",
"payload": {
"patch": {"workflow": {"name": "renamed", "version": "0.32.0", "description": "d"}}
}
}
Undo & Redo#
The DAG Edit API maintains one undo/redo stack per (acting identity, pipeline id) pair, with documented depth and pipeline caps (see History scope, caps, and lifetime).
undo#
Reverts the last edit, restoring the previous TOML state.
POST /api/pipeline/{id}/undo
Content-Type: application/json
{"toml_content": "<the client's current TOML>"}
The request requires a JSON body with toml_content set to the caller's
current TOML state. The undo stack's top transition is applied only when its
resulting state matches that content exactly.
Failure codes (all fail closed — no edit is produced and no history is consumed):
| Status | Code | Meaning |
|---|---|---|
| 409 | EDIT_HISTORY_UNAVAILABLE |
No history exists for this identity/pipeline — the first edit has not happened, or the stack was lost to a server restart or LRU eviction. suggestion tells the caller to re-apply the change. |
| 409 | EDIT_HISTORY_STALE |
History exists, but the supplied toml_content is not the tip of the recorded history (stale or racing client). |
| 404 | NO_UNDO |
History exists but every recorded edit is already undone. |
Returns { "toml_content": "<previous TOML>" } — i.e. the state before the
last edit.
redo#
Re-applies the last undone edit.
POST /api/pipeline/{id}/redo
Content-Type: application/json
{"toml_content": "<the client's current TOML>"}
Same contract as undo: the JSON body is required, and toml_content must
match the current state (the result of the last undo). The failure codes are
the same: 409 EDIT_HISTORY_UNAVAILABLE, 409 EDIT_HISTORY_STALE, and
404 NO_REDO when history exists but nothing was undone.
Note: Performing a new edit after an undo clears the redo stack (standard undo/redo semantics).
History scope, caps, and lifetime#
Defined in crates/oxo-flow-web/src/domains/dag/service.rs (issue #1137):
- Scope — a stack is keyed by the acting identity and the pipeline id.
The identity is the
users.idthe auth middleware resolves for the request (session or API key); personal mode has no auth middleware, so every request falls back to thedefaultpseudo-user. Two identities editing the same pipeline id get independent stacks: neither can observe or undo the other's edits. - Depth cap —
MAX_EDIT_HISTORY_DEPTH= 50 transitions per stack. Pushing past the cap drops the oldest transition, so undo can walk back at most 50 edits. - Pipeline cap —
MAX_EDIT_HISTORY_PIPELINES= 256 stacks process-wide. Recording an edit for a new (identity, pipeline) pair evicts the least-recently-used stack. - Lifetime — history lives in RAM only; it is not persisted. A server
restart (or LRU eviction) empties a stack, and a subsequent
undo/redofails closed with409 EDIT_HISTORY_UNAVAILABLEand a remediation suggestion. It never silently no-ops, and it never applies an edit to a document the server no longer holds history for.
Response Format#
All edit commands return a DagEditResponse:
| Field | Type | Description |
|---|---|---|
success |
Boolean | true if validation passed (no hard errors) |
toml_content |
String | The complete workflow TOML after the edit |
validation_errors |
Array of String | Validation error-severity messages (lint warnings such as a missing description are collected separately and are not part of this response) |
If validation fails, success is false, validation_errors is populated, and the edit is rolled back atomically: toml_content echoes the caller's unchanged input (see Overview). Rejected edits never enter the undo/redo history.
Malformed commands (unknown operation, missing required payload fields such as the rule name, an update_rule target rule that does not exist, or a connect/disconnect/update_params target rule that does not exist) are rejected with HTTP 400 and code DAG_EDIT_ERROR; the edit is not applied. The same applies to edits whose resulting document no longer parses as a workflow (invalid TOML syntax, unknown fields, duplicate rule names).
Important Notes#
- File-based edges are rewired through
update_rule, notconnect/disconnect. Theconnect/disconnectcommands only managedepends_onentries. File-based dependencies (inferred from input/output matching) are controlled by theinputandoutputfields of rules:update_rulepatches those fields like any other, so editing a rule'sinput(oroutput) list is how the edit API adds or removes an implicit edge. There is no dedicated edge command for them — an edit that changes what a rule reads changes the inferred edges with it. -
All edits are validated. The edit API runs the full workflow validation pipeline after every command. Validation findings (e.g., a
depends_onentry naming an unknown rule) are returned invalidation_errorswithsuccess: false, and the edit is rolled back atomically —toml_contentechoes the caller's unchanged input and the undo/redo history is untouched. Edits that break parsing — invalid TOML syntax, unknown fields, or duplicate rule names — are rejected with HTTP400instead. Cycle detection is part of the edit validation path:validate_formatbuilds the DAG with parsed config values on every edit, so a cycle introduced viadepends_onedits surfaces immediately assuccess: falsewith theDAG error:(E006) message invalidation_errors(the edit is rolled back atomically). See DAG Engine. -
TOML round-tripping preserves formatting. Edits mutate the parsed TOML document in place (
toml_edit), so the author's comments and formatting survive every edit; the document is never re-serialized through the canonicalformat::format_workflow. - Undo/redo is in-memory, identity-scoped, and bounded. Stacks are keyed by (acting identity, pipeline id), capped at 50 transitions each and 256 stacks process-wide (LRU), and live only for the duration of the server process. They do not persist across server restarts; undo/redo after a restart fails closed with
409 EDIT_HISTORY_UNAVAILABLE(see History scope, caps, and lifetime).
See Also#
- DAG Engine — how the DAG is built and validated
- Workflow Format — rule field reference
- Web API — REST endpoint documentation
- Web System Architecture — how the edit API fits into the Web UI