Skip to content

oxo-flow batch#

Execute a command template in parallel across multiple items.


Usage#

oxo-flow batch [OPTIONS] <TEMPLATE> [ITEMS...]

Arguments#

Argument Description
<TEMPLATE> Command template with placeholders ({item}, {stem}, etc.)
[ITEMS...] Items to process (files, globs, or from stdin/file)

Options#

Option Short Default Description
--jobs -j 1 Number of parallel workers
--stop-on-error -x — Stop after first failure
--file -f — Read items from file
--json-output — — Output results as formatted JSON (legacy alias — the global --json now drives the same output). With --dry-run it emits the previewed commands as JSON on stdout (issue #864)
--dry-run -n — Preview without executing
--workdir -d . Working directory
--environment -e — Environment spec
--generate-workflow — — Generate .oxoflow file
--output -o batch.oxoflow Output workflow file

Placeholders#

Placeholder Example Description
{} / {item} s1.bam Current item
{nr} 1, 2, 3... 1-based index
{basename} s1.bam Filename (no path)
{dir} data/ Directory
{stem} s1 Filename without extension
{ext} bam Extension

Examples#

Basic batch execution#

oxo-flow batch "samtools flagstat {item}" *.bam

Parallel with 8 workers#

oxo-flow batch -j 8 "fastqc {item}" *.fastq.gz

Stop on first error#

oxo-flow batch -x "bwa mem ref.fa {item}" samples.txt

From stdin (pipeline)#

ls *.bam | oxo-flow batch "samtools flagstat {item}"

With environment#

oxo-flow batch -e "conda: bwa_env" "bwa mem ref.fa {item}" *.fastq.gz

Generate workflow#

oxo-flow batch --generate-workflow "bwa mem ref.fa {item}" *.bam -o align.oxoflow

JSON output#

oxo-flow batch --json-output "samtools flagstat {item}" *.bam > results.json

Dry-run preview#

oxo-flow batch -n "fastqc {item}" *.fastq.gz

Dry-run JSON preview#

With --json-output (or the global --json), the dry-run preview is also written to stdout as JSON — the preview IS the result in dry-run, so scripts can consume the exact commands that would run without executing anything (issue #864). The human-readable preview stays on stderr either way, so redirecting stdout always yields pure JSON:

oxo-flow batch -n --json-output "samtools flagstat {item}" *.bam > preview.json
{
  "tool": "oxo-flow batch",
  "dry_run": true,
  "template": "samtools flagstat {item}",
  "total": 2,
  "failed": 0,
  "success": true,
  "results": [
    { "item": "a.bam", "command": "samtools flagstat a.bam", "exit_code": null, "success": true, "error": null },
    { "item": "b.bam", "command": "samtools flagstat b.bam", "exit_code": null, "success": true, "error": null }
  ]
}

The results entries share the executed-mode shape — exit_code and error are null and success is true because nothing ran.


Notes#

  • Items can be files, globs (*.bam), or plain strings
  • Input file format: one item per line, blank lines and # comments ignored
  • When no items provided and stdin has data, reads from pipe
  • {} is rush-compatible shorthand for {item}
  • Environment spec format: conda: env.yaml or docker: image:tag
  • In dry-run, the human-readable preview goes to stderr; with --json-output/--json the same preview is emitted as JSON on stdout (issue #864)

  • Generated workflows create one rule per item (batch_item_1, batch_item_2, ...)

Exit Codes#

  • 0 — all items succeeded.
  • 1 — any item failed (#540). The per-item failures are still listed in the output and in --json-output's results array. --stop-on-error (-x) additionally aborts the batch on the FIRST failed item instead of running the rest.