14 — Paired Experiment-Control (Single Pair)#
Somatic variant calling for one tumor/control pair with hardcoded sample names — the readable version of the [[pairs]]-driven pattern in Paired Experiment-Control (Multiple Pairs).
What It Demonstrates#
- Both tumor and control flow through identical QC → align → dedup → BQSR stages in parallel branches
- Mutect2
-normal CTRL_01resolves via the read-group SM tag set at alignment - BQSR table and application run in one rule (intermediate table kept inside the shell)
Pipeline Overview#
graph TD
FE[fastp_experiment] --> BE[bwa_mem2_experiment]
FC[fastp_control] --> BC[bwa_mem2_control]
BE --> ME[markdup_experiment]
BC --> MC[markdup_control]
ME --> RE[bqsr_experiment]
MC --> RC[bqsr_control]
RC --> H[haplotype_caller]
RC --> MU[mutect2]
RE --> MU
MU --> FM[filter_mutect_calls]
FM --> AV[annotate_variants]
AV --> CR[clinical_report]
Two mirror-image branches process tumor (EXP_01) and control (CTRL_01) independently through QC, alignment, deduplication, and BQSR. They converge in mutect2, which takes both recalibrated BAMs as input; the somatic chain then runs straight through filtering, annotation, and the clinical report.
Workflow Definition#
# examples/gallery/14_paired_experiment_control.oxoflow
# 14 — Paired Experiment-Control (Single Pair)
# Somatic variant calling for one tumor/control pair, with hardcoded
# sample names for readability: fastp → BWA-MEM2 → MarkDuplicates →
# BQSR → Mutect2 → FilterMutectCalls → VEP → clinical report.
[workflow]
name = "paired-experiment-control"
version = "1.0.0"
description = "Paired experiment-control variant calling for general diagnostics"
author = "oxo-flow examples"
[config]
reference_fasta = "/data/references/GRCh38/genome.fa"
known_sites = "/data/references/GRCh38/dbsnp_146.hg38.vcf.gz"
genome_build = "GRCh38"
[defaults]
threads = 8
memory = "16G"
[report]
# template: reserved — supply a real Tera template file path to use one
# template = "report/clinical_template.html"
# Section IDs are the built-in generator names (oxo-flow report --list-sections).
# The output format is chosen on the CLI with -f (html, json, md, pdf).
sections = [
"universal",
"workflow-info",
"execution-status",
"commands",
"metrics",
"sample-matrix",
"software-versions",
"provenance",
]
[[rules]]
name = "fastp_experiment"
input = [
"raw/EXP_01_R1.fq.gz",
"raw/EXP_01_R2.fq.gz",
]
output = [
"trimmed/EXP_01_R1.fq.gz",
"trimmed/EXP_01_R2.fq.gz",
"qc/EXP_01_fastp.json",
]
shell = "fastp -i {input[0]} -I {input[1]} -o {output[0]} -O {output[1]} --json {output[2]} --thread {threads}"
[rules.resources]
threads = 8
[rules.environment]
conda = "envs/fastp.yaml"
[[rules]]
name = "fastp_control"
input = [
"raw/CTRL_01_R1.fq.gz",
"raw/CTRL_01_R2.fq.gz",
]
output = [
"trimmed/CTRL_01_R1.fq.gz",
"trimmed/CTRL_01_R2.fq.gz",
"qc/CTRL_01_fastp.json",
]
shell = "fastp -i {input[0]} -I {input[1]} -o {output[0]} -O {output[1]} --json {output[2]} --thread {threads}"
[rules.resources]
threads = 8
[rules.environment]
conda = "envs/fastp.yaml"
[[rules]]
name = "bwa_mem2_experiment"
input = [
"trimmed/EXP_01_R1.fq.gz",
"trimmed/EXP_01_R2.fq.gz",
]
output = ["aligned/EXP_01.sorted.bam"]
shell = """
rg=$(printf '@RG\tID:%s\tSM:%s\tPL:ILLUMINA' EXP_01 EXP_01)
bwa-mem2 mem -t {threads} -R "$rg" {config.reference_fasta} {input[0]} {input[1]} | samtools sort -@ 4 -o {output[0]}
"""
[rules.resources]
threads = 16
memory = "32G"
[rules.environment]
conda = "envs/bwa_mem2.yaml"
[[rules]]
name = "bwa_mem2_control"
input = [
"trimmed/CTRL_01_R1.fq.gz",
"trimmed/CTRL_01_R2.fq.gz",
]
output = ["aligned/CTRL_01.sorted.bam"]
shell = """
rg=$(printf '@RG\tID:%s\tSM:%s\tPL:ILLUMINA' CTRL_01 CTRL_01)
bwa-mem2 mem -t {threads} -R "$rg" {config.reference_fasta} {input[0]} {input[1]} | samtools sort -@ 4 -o {output[0]}
"""
[rules.resources]
threads = 16
memory = "32G"
[rules.environment]
conda = "envs/bwa_mem2.yaml"
[[rules]]
name = "markdup_experiment"
input = ["aligned/EXP_01.sorted.bam"]
output = [
"dedup/EXP_01.dedup.bam",
"dedup/EXP_01.metrics.txt",
]
shell = "gatk MarkDuplicates --VALIDATION_STRINGENCY SILENT -I {input[0]} -O {output[0]} -M {output[1]}"
[rules.resources]
threads = 4
memory = "16G"
[rules.environment]
conda = "envs/gatk.yaml"
[[rules]]
name = "markdup_control"
input = ["aligned/CTRL_01.sorted.bam"]
output = [
"dedup/CTRL_01.dedup.bam",
"dedup/CTRL_01.metrics.txt",
]
shell = "gatk MarkDuplicates --VALIDATION_STRINGENCY SILENT -I {input[0]} -O {output[0]} -M {output[1]}"
[rules.resources]
threads = 4
memory = "16G"
[rules.environment]
conda = "envs/gatk.yaml"
[[rules]]
name = "bqsr_experiment"
input = ["dedup/EXP_01.dedup.bam"]
output = ["recal/EXP_01.recal.bam"]
shell = "gatk BaseRecalibrator --read-validation-stringency SILENT -I {input[0]} -R {config.reference_fasta} --known-sites {config.known_sites} -O recal/EXP_01.recal.table && gatk ApplyBQSR -I {input[0]} -R {config.reference_fasta} --bqsr-recal-file recal/EXP_01.recal.table -O {output[0]}"
[rules.resources]
threads = 4
memory = "16G"
[rules.environment]
conda = "envs/gatk.yaml"
[[rules]]
name = "bqsr_control"
input = ["dedup/CTRL_01.dedup.bam"]
output = ["recal/CTRL_01.recal.bam"]
shell = "gatk BaseRecalibrator --read-validation-stringency SILENT -I {input[0]} -R {config.reference_fasta} --known-sites {config.known_sites} -O recal/CTRL_01.recal.table && gatk ApplyBQSR -I {input[0]} -R {config.reference_fasta} --bqsr-recal-file recal/CTRL_01.recal.table -O {output[0]}"
[rules.resources]
threads = 4
memory = "16G"
[rules.environment]
conda = "envs/gatk.yaml"
[[rules]]
name = "mutect2"
input = [
"recal/EXP_01.recal.bam",
"recal/CTRL_01.recal.bam",
]
output = ["variants/EXP_01.mutect2.vcf.gz"]
shell = "gatk Mutect2 --read-validation-stringency SILENT -I {input[0]} -I {input[1]} -normal CTRL_01 -R {config.reference_fasta} -O {output[0]}"
[rules.resources]
threads = 4
memory = "16G"
[rules.environment]
conda = "envs/gatk.yaml"
[[rules]]
name = "filter_mutect_calls"
input = ["variants/EXP_01.mutect2.vcf.gz"]
output = ["variants/EXP_01.mutect2.filtered.vcf.gz"]
shell = "gatk FilterMutectCalls -V {input[0]} -R {config.reference_fasta} -O {output[0]}"
description = "Filter raw Mutect2 calls"
[rules.resources]
threads = 2
memory = "8G"
[rules.environment]
conda = "envs/gatk.yaml"
[[rules]]
name = "haplotype_caller"
input = ["recal/CTRL_01.recal.bam"]
output = ["variants/CTRL_01.g.vcf.gz"]
shell = "gatk HaplotypeCaller --read-validation-stringency SILENT -I {input[0]} -R {config.reference_fasta} -O {output[0]} -ERC GVCF"
description = "Variant calling for control sample"
[rules.resources]
threads = 4
memory = "16G"
[rules.environment]
conda = "envs/gatk.yaml"
[[rules]]
name = "annotate_variants"
input = ["variants/EXP_01.mutect2.filtered.vcf.gz"]
output = ["annotated/EXP_01.annotated.vcf.gz"]
shell = "vep --input_file {input[0]} --output_file {output[0]} --format vcf --vcf --offline --cache"
[rules.resources]
threads = 4
memory = "8G"
[rules.environment]
conda = "envs/vep.yaml"
[[rules]]
name = "clinical_report"
input = ["annotated/EXP_01.annotated.vcf.gz"]
output = ["reports/EXP_01_report.html"]
shell = "python scripts/generate_report.py --input {input[0]} --output {output[0]} --sample EXP_01"
description = "Generate variant report"
[rules.resources]
threads = 1
memory = "4G"
[rules.environment]
conda = "envs/report.yaml"
Running the Workflow#
Validate#
$ oxo-flow validate examples/gallery/14_paired_experiment_control.oxoflow
✓ examples/gallery/14_paired_experiment_control.oxoflow — 13 rules, 12 dependencies
⚠ Warning: The following input files do not exist:
- raw/EXP_01_R1.fq.gz
- raw/EXP_01_R2.fq.gz
- raw/CTRL_01_R1.fq.gz
- raw/CTRL_01_R2.fq.gz
The ✓ line is what matters — validate exits 0 and the workflow is well-formed. The warnings are expected here: the raw/ FASTQs don't exist until you stage real data (see the note at the bottom of this page).
Dry-Run#
$ oxo-flow dry-run examples/gallery/14_paired_experiment_control.oxoflow
Plan: would run: 13 | skip: 0 | completed: 0 (DAG size: 13)
...
10. mutect2 [run: never completed]
outputs: ["variants/EXP_01.mutect2.vcf.gz"]
command: gatk Mutect2 ... -I recal/EXP_01.recal.bam -I recal/CTRL_01.recal.bam -normal CTRL_01 -R /data/references/GRCh38/genome.fa -O variants/EXP_01.mutect2.vcf.gz
Summary: 13 rules, total 86 threads declared, max 16 threads/rule
13 rules with memory requirements
To execute: oxo-flow run examples/gallery/14_paired_experiment_control.oxoflow -j 2
Because both sample names are hardcoded, the plan equals the template — 13 rules, no expansion, and no sample-readiness block. The mutect2 command is the teaching moment: both branches' recalibrated BAMs appear as -I inputs, and -normal CTRL_01 names the control by the SM tag that bwa_mem2_control wrote into the BAM header via its read-group string. haplotype_caller runs only on the control branch (variants/CTRL_01.g.vcf.gz) — a common strategy to profile the normal's germline background.
DAG Structure#
$ oxo-flow graph examples/gallery/14_paired_experiment_control.oxoflow
Workflow DAG: 13 rules, 12 dependencies
Depth: 8, Width: 2, Critical path: 8 steps
Critical path: fastp_control → bwa_mem2_control → markdup_control → bqsr_control → mutect2 → filter_mutect_calls → annotate_variants → clinical_report
graph shows the template DAG (before expansion — see the gallery index note). Through the first four levels every dependency stays within its own branch (bwa_mem2_control [depends: fastp_control]) — the two branches only meet at mutect2.
Expected Outputs#
| Path | Producer |
|---|---|
trimmed/EXP_01_R1.fq.gz, trimmed/EXP_01_R2.fq.gz (+ CTRL_01) |
fastp_experiment / fastp_control |
aligned/EXP_01.sorted.bam, aligned/CTRL_01.sorted.bam |
bwa_mem2_experiment / bwa_mem2_control |
dedup/{EXP_01,CTRL_01}.dedup.bam + metrics |
markdup_experiment / markdup_control |
recal/{EXP_01,CTRL_01}.recal.bam |
bqsr_experiment / bqsr_control |
variants/CTRL_01.g.vcf.gz |
haplotype_caller (control only) |
variants/EXP_01.mutect2.vcf.gz |
mutect2 |
variants/EXP_01.mutect2.filtered.vcf.gz |
filter_mutect_calls |
annotated/EXP_01.annotated.vcf.gz |
annotate_variants |
reports/EXP_01_report.html |
clinical_report |
What's Next?#
- Scale it up: Paired Experiment-Control (Multiple Pairs) — the same chain driven by
[[pairs]]. - How-to guide: Experiment-Control Pairing — pairing syntax and naming conventions.
Input data and environments
Input paths under /data/references/... and raw/ are placeholders —
replace them with your own data. The validate warnings about missing
raw/ files are expected until real data is staged. The referenced
envs/*.yaml specs ship in
examples/envs/.