oxo-flow batch#
Execute a command template in parallel across multiple items.
Usage#
Arguments#
| Argument | Description |
|---|---|
<TEMPLATE> |
Command template with placeholders ({item}, {stem}, etc.) |
[ITEMS...] |
Items to process (files, globs, or from stdin/file) |
Options#
| Option | Short | Default | Description |
|---|---|---|---|
--jobs |
-j |
1 | Number of parallel workers |
--stop-on-error |
-x |
— | Stop after first failure |
--file |
-f |
— | Read items from file |
--json-output |
— | — | Output results as formatted JSON (legacy alias — the global --json now drives the same output). With --dry-run it emits the previewed commands as JSON on stdout |
--dry-run |
-n |
— | Preview without executing |
--workdir |
-d |
. | Working directory |
--environment |
-e |
— | Environment spec |
--generate-workflow |
— | — | Generate .oxoflow file |
--output |
-o |
batch.oxoflow | Output workflow file |
Placeholders#
| Placeholder | Example | Description |
|---|---|---|
{} / {item} |
s1.bam |
Current item |
{nr} |
1, 2, 3... | 1-based index |
{basename} |
s1.bam |
Filename (no path) |
{dir} |
data/ |
Directory |
{stem} |
s1 |
Filename without extension |
{ext} |
bam |
Extension |
Examples#
Basic batch execution#
Parallel with 8 workers#
Stop on first error#
From stdin (pipeline)#
With environment#
The conda: form wraps through the engine's own conda-family command builder
(see Environment System):
each item runs inside the SAME environment an oxo-flow run rule would use —
<yaml name|stem>-<hash8> for a file-backed spec — with the environment's
bin/ first on PATH, so an environment the engine already created is
reused rather than looked up under a hand-derived name. A relative spec path
resolves against --workdir (the same anchor run uses), and a bare
environment name (-e bwa_env, no kind: prefix) is used verbatim — a
dotted name such as r-4.4 is not truncated. mamba: is accepted and uses
the same shared builder; pixi:/venv:/modules: are rejected rather than
silently running the item on the host PATH.
The item shell (sh -c <template>) runs under the same cleared-environment
allowlist as an oxo-flow run rule shell (issue #1233, RB-15): it starts
from an empty environment plus PATH, HOME, LANG, every LC_*
variable, the OXO_FLOW_ENV_PASSTHROUGH names, and the
OXO_FLOW_TMPDIR-derived TMPDIR/TMP/TEMP triple — never the
operator's ambient environment. A docker:/singularity: item is an
exception in one direction only: because the wrapper runs docker run /
singularity exec from that very shell, the container-daemon variables
(DOCKER_HOST, DOCKER_TLS_VERIFY, DOCKER_CERT_PATH, DOCKER_CONFIG,
APPTAINER_*, SINGULARITY_*) are re-admitted for it, exactly as the rule
executor admits them for a container rule (issue #1173). See
Rule execution environment.
Generate workflow#
JSON output#
Dry-run preview#
Dry-run JSON preview#
With --json-output (or the global --json), the dry-run preview is also
written to stdout as JSON — the preview IS the result in dry-run, so
scripts can consume the exact commands that would run without executing
anything. The human-readable preview stays on stderr either
way, so redirecting stdout always yields pure JSON:
{
"tool": "oxo-flow batch",
"dry_run": true,
"template": "samtools flagstat {item}",
"total": 2,
"failed": 0,
"success": true,
"results": [
{ "item": "a.bam", "command": "samtools flagstat a.bam", "exit_code": null, "success": true, "error": null },
{ "item": "b.bam", "command": "samtools flagstat b.bam", "exit_code": null, "success": true, "error": null }
]
}
The results entries share the executed-mode shape — exit_code and
error are null and success is true because nothing ran.
Notes#
- Items can be files, globs (
*.bam), or plain strings - Input file format: one item per line, blank lines and
#comments ignored - When no items provided and stdin has data, reads from pipe
{}is rush-compatible shorthand for{item}- Environment spec format:
conda: env.yaml,mamba: env.yaml,docker: image:tag, orsingularity: image.sif(a bare-e <name>selects a conda environment by name) -
In dry-run, the human-readable preview goes to stderr; with
--json-output/--jsonthe same preview is emitted as JSON on stdout -
Generated workflows create one rule per item (
batch_item_1,batch_item_2, ...)
Exit Codes#
- 0 — all items succeeded.
- 1 — any item failed. The per-item failures are still listed in
the output and in
--json-output'sresultsarray.--stop-on-error(-x) additionally aborts the batch on the FIRST failed item instead of running the rest.
The shared CLI exit-code contract (what 0/1/2 mean across commands)
is documented on run.