Skip to content

oxo-flow env#

Manage software environments. Provides subcommands for listing available backends, checking workflow environment requirements, and maintaining the shared environment store.


Usage#

oxo-flow env <SUBCOMMAND>

Subcommands#

env list#

List available environment backends, or list environments defined in a workflow.

oxo-flow env list [WORKFLOW]

When called without arguments, lists all backends detected on the system. When given a workflow file, lists the environment each rule will actually run in. Resolution follows the run's own precedence — env_group → the rule's environment → [defaults] environment — so a rule that inherits the defaults is reported with the backend the run will bootstrap, not as system. run and dry-run fold that same resolved spec onto every rule before scheduling (issue #1210), so the listing and the executed backend cannot disagree — including a rule that declares both an env_group and a conflicting inline environment: the group wins on both surfaces. A rule that resolves to no environment is reported as system.

Output:

oxo-flow v0.32.0 — Rust-native bioinformatics pipeline engine
https://github.com/Traitome/oxo-flow
Available environment backends:
  ✓ conda
  ✓ docker
  ✓ singularity
  ✓ venv

The two banner lines print only at an interactive terminal. All of this output goes to stderr — capture it with 2> or 2>&1; stdout stays empty.

env check#

Check that all environments declared in a workflow file are valid and their backends are available. Each rule's environment is resolved with the same precedence the run uses (env_group → rule environment → [defaults] environment), so a rule that inherits the defaults is validated against the backend that will actually execute.

oxo-flow env check [WORKFLOW]
Argument Description
[WORKFLOW] Optional path to the .oxoflow workflow file. If omitted, checks system-wide backend availability instead.

Output (all valid):

  ✓ align (conda)
  ✓ call_variants (docker)
  ✓ annotate (singularity)

Without a workflow, each available backend also names the binary that a run would resolve — the engine prefers mamba → micromamba → conda, so on a machine with several conda-family installs the resolved binary is what actually gets invoked:

Environment backend availability:
  ✓ mamba (resolved: micromamba)
  ✓ conda
  ✗ docker (not found)

env create#

Create a new environment from a specification file.

oxo-flow env create [OPTIONS] <SPEC>
Argument Description
<SPEC> Path to the environment specification file (.yaml/.yml, .toml, .lock)
Option Short Description
--name -n Custom name for the created environment (default: derived from the spec filename)
--ai — Generate the environment spec from a natural-language description (SPEC is the description)
--backend — Environment backend to generate: conda (YAML) or pixi (TOML). Default: inferred from the spec's extension, or conda with --ai

Output (missing backend):

  ✓ align (conda)
  ✗ call_variants — environment error (docker): docker is not installed or not in PATH

env gc#

Inspect and clean up the opt-in shared environment store.

The store is disabled by default: a workflow is self-contained and its pixi/venv environments materialize inside the workflow tree exactly like any other artifact — the tree stays copyable, movable, and deletable on its own.

Setting OXO_FLOW_ENV_STORE to a non-empty path activates the store (location IS activation). When enabled, pixi and venv environments materialize once per content key under the store root; the in-tree location the engine already uses (.pixi next to the manifest, the venv dir) becomes a symlink into the store, so wraps like pixi run --manifest-path and source .venv/bin/activate keep working unchanged. Repeated runs across many pipeline checkouts reuse the same environment instead of re-downloading and re-building it per tree. A broken link (e.g. the store dir was deleted) is detected automatically on the next run: the cached entry is invalidated, the dangling in-tree symlink is detached, and the environment is rebuilt, no manual intervention needed.

oxo-flow env gc [--min-age-days N] [--dry-run]
Option Default Description
--min-age-days 90 Minimum age (mtime) before an unreferenced store dir may be deleted
--dry-run off Report what would be deleted without touching anything

With the store disabled (OXO_FLOW_ENV_STORE unset), env gc prints a note and exits — there is nothing to collect.

Where environments live (store enabled):

  • Store root: exactly the path OXO_FLOW_ENV_STORE points at — there is no implicit default; the environment variable is per-process, so on a multi-user server each user who opts in gets their own store unless an operator deliberately points everyone at a shared filesystem path
  • Each environment: <root>/<backend>-<sha256-16hex of its content key> (e.g. pixi-3f2a…)
  • Workflow trees: a symlink (.pixi for pixi rules, the venv path for venv rules) pointing into the store

What env gc deletes:

  • Only store dirs not referenced by any discovered run checkpoint (env_store_refs in .oxo-flow/checkpoint.json), AND older than --min-age-days
  • Checkpoint discovery covers the CURRENT workflow directory only — runs in other worktrees/workdirs sharing the store are invisible to GC and are protected solely by the age gate. The 90-day default is sized for that: keep it well above the longest interval between runs in any workdir sharing the store, and always check --dry-run output before a real collection
  • Referenced dirs are always kept — a running or resumable run keeps its environments
  • Young dirs are kept even when torn/incomplete: a fresh torn dir may be a concurrent run mid-setup, so the age gate protects those too
  • Deleting a still-needed env is recoverable: the next run detects the broken link and rebuilds
# Preview
OXO_FLOW_ENV_STORE=~/.cache/oxo-flow/envs oxo-flow env gc --dry-run

Shared environment store: /home/user/.cache/oxo-flow/envs (3 env dirs)
  pixi-3f2a91b0c4d5e6f7 [pixi] referenced by a checkpoint — kept
      1.2 GiB /home/user/.cache/oxo-flow/envs/pixi-3f2a91b0c4d5e6f7
  venv-8b7c6d5e4f3a2b1c [venv] unreferenced and old enough — eligible
      310.4 MiB /home/user/.cache/oxo-flow/envs/venv-8b7c6d5e4f3a2b1c

Summary: 1 store dir(s) would be deleted, freeing ~310.4 MiB (dry run — nothing touched)

Notes:

  • GC discovery covers the checkpoint of the workdir you run it from (.oxo-flow/checkpoint.json); runs in other trees are protected by the age gate. Check --dry-run output before a real run when multiple worktrees share one store
  • oxo-flow env gc only ever deletes inside the store root — workflow-tree symlinks are never followed or removed, and store contents are never modified, only whole dirs deleted
  • Store re-homing is best-effort and non-destructive: if the env cannot be moved into the store (cross-device, permissions), the run keeps the plain in-tree environment and simply misses the dedup

Options#

Option Short Description
--verbose -v Enable debug-level logging

Examples#

# List available backends
oxo-flow env list

# Check a specific workflow
oxo-flow env check pipeline.oxoflow

# Opt into the shared env store for this shell session, then preview GC
export OXO_FLOW_ENV_STORE=~/.cache/oxo-flow/envs
oxo-flow env gc --dry-run

Notes#

  • env check exits with code 1 if any environment validation fails
  • The check verifies backend availability, not that specific conda environments or Docker images exist — it confirms that the required type of environment manager is installed
  • Run env check before submitting to a cluster to catch environment issues early
  • To check that environment definition files exist (conda YAML, venv directories, pixi.toml), use oxo-flow test --deep (finding D002)
  • Only pixi and venv environments participate in the shared store — by design, not an omission. Conda/mamba name-based environments already live in a shared central prefix (the engine's store could not deduplicate them further), conda prefix environments are not relocatable (absolute paths are baked into the env, so moving breaks activation), Docker and Singularity pull images into their own registries, and HPC modules are provided by the cluster. See env gc for store maintenance