Skip to content

14 — Paired Experiment-Control (Single Pair)#

Somatic variant calling for one tumor/control pair with hardcoded sample names — the readable version of the [[pairs]]-driven pattern in Paired Experiment-Control (Multiple Pairs).

What It Demonstrates#

  • Both tumor and control flow through identical QC → align → dedup → BQSR stages in parallel branches
  • Mutect2 -normal CTRL_01 resolves via the read-group SM tag set at alignment
  • BQSR table and application run in one rule (intermediate table kept inside the shell)

Pipeline Overview#

graph TD
    FE[fastp_experiment] --> BE[bwa_mem2_experiment]
    FC[fastp_control] --> BC[bwa_mem2_control]
    BE --> ME[markdup_experiment]
    BC --> MC[markdup_control]
    ME --> RE[bqsr_experiment]
    MC --> RC[bqsr_control]
    RC --> H[haplotype_caller]
    RC --> MU[mutect2]
    RE --> MU
    MU --> FM[filter_mutect_calls]
    FM --> AV[annotate_variants]
    AV --> CR[clinical_report]

Two mirror-image branches process tumor (EXP_01) and control (CTRL_01) independently through QC, alignment, deduplication, and BQSR. They converge in mutect2, which takes both recalibrated BAMs as input; the somatic chain then runs straight through filtering, annotation, and the clinical report.

Workflow Definition#

# examples/gallery/14_paired_experiment_control.oxoflow
# 14 — Paired Experiment-Control (Single Pair)
# Somatic variant calling for one tumor/control pair, with hardcoded
# sample names for readability: fastp → BWA-MEM2 → MarkDuplicates →
# BQSR → Mutect2 → FilterMutectCalls → VEP → clinical report.

[workflow]
name = "paired-experiment-control"
version = "1.0.0"
description = "Paired experiment-control variant calling for general diagnostics"
author = "oxo-flow examples"

[config]
reference_fasta = "/data/references/GRCh38/genome.fa"
known_sites = "/data/references/GRCh38/dbsnp_146.hg38.vcf.gz"
genome_build = "GRCh38"

[defaults]
threads = 8
memory = "16G"

[report]
# template: reserved — supply a real Tera template file path to use one
# template = "report/clinical_template.html"
# Section IDs are the built-in generator names (oxo-flow report --list-sections).
# The output format is chosen on the CLI with -f (html, json, md, pdf).
sections = [
    "universal",
    "workflow-info",
    "execution-status",
    "commands",
    "metrics",
    "sample-matrix",
    "software-versions",
    "provenance",
]

[[rules]]
name = "fastp_experiment"
input = [
    "raw/EXP_01_R1.fq.gz",
    "raw/EXP_01_R2.fq.gz",
]
output = [
    "trimmed/EXP_01_R1.fq.gz",
    "trimmed/EXP_01_R2.fq.gz",
    "qc/EXP_01_fastp.json",
]
shell = "fastp -i {input[0]} -I {input[1]} -o {output[0]} -O {output[1]} --json {output[2]} --thread {threads}"

[rules.resources]
threads = 8

[rules.environment]
conda = "envs/fastp.yaml"

[[rules]]
name = "fastp_control"
input = [
    "raw/CTRL_01_R1.fq.gz",
    "raw/CTRL_01_R2.fq.gz",
]
output = [
    "trimmed/CTRL_01_R1.fq.gz",
    "trimmed/CTRL_01_R2.fq.gz",
    "qc/CTRL_01_fastp.json",
]
shell = "fastp -i {input[0]} -I {input[1]} -o {output[0]} -O {output[1]} --json {output[2]} --thread {threads}"

[rules.resources]
threads = 8

[rules.environment]
conda = "envs/fastp.yaml"

[[rules]]
name = "bwa_mem2_experiment"
input = [
    "trimmed/EXP_01_R1.fq.gz",
    "trimmed/EXP_01_R2.fq.gz",
]
output = ["aligned/EXP_01.sorted.bam"]
shell = """
rg=$(printf '@RG\tID:%s\tSM:%s\tPL:ILLUMINA' EXP_01 EXP_01)
bwa-mem2 mem -t {threads} -R "$rg" {config.reference_fasta} {input[0]} {input[1]} | samtools sort -@ 4 -o {output[0]}
"""

[rules.resources]
threads = 16
memory = "32G"

[rules.environment]
conda = "envs/bwa_mem2.yaml"

[[rules]]
name = "bwa_mem2_control"
input = [
    "trimmed/CTRL_01_R1.fq.gz",
    "trimmed/CTRL_01_R2.fq.gz",
]
output = ["aligned/CTRL_01.sorted.bam"]
shell = """
rg=$(printf '@RG\tID:%s\tSM:%s\tPL:ILLUMINA' CTRL_01 CTRL_01)
bwa-mem2 mem -t {threads} -R "$rg" {config.reference_fasta} {input[0]} {input[1]} | samtools sort -@ 4 -o {output[0]}
"""

[rules.resources]
threads = 16
memory = "32G"

[rules.environment]
conda = "envs/bwa_mem2.yaml"

[[rules]]
name = "markdup_experiment"
input = ["aligned/EXP_01.sorted.bam"]
output = [
    "dedup/EXP_01.dedup.bam",
    "dedup/EXP_01.metrics.txt",
]
shell = "gatk MarkDuplicates --VALIDATION_STRINGENCY SILENT -I {input[0]} -O {output[0]} -M {output[1]}"

[rules.resources]
threads = 4
memory = "16G"

[rules.environment]
conda = "envs/gatk.yaml"

[[rules]]
name = "markdup_control"
input = ["aligned/CTRL_01.sorted.bam"]
output = [
    "dedup/CTRL_01.dedup.bam",
    "dedup/CTRL_01.metrics.txt",
]
shell = "gatk MarkDuplicates --VALIDATION_STRINGENCY SILENT -I {input[0]} -O {output[0]} -M {output[1]}"

[rules.resources]
threads = 4
memory = "16G"

[rules.environment]
conda = "envs/gatk.yaml"

[[rules]]
name = "bqsr_experiment"
input = ["dedup/EXP_01.dedup.bam"]
output = ["recal/EXP_01.recal.bam"]
shell = "gatk BaseRecalibrator --read-validation-stringency SILENT -I {input[0]} -R {config.reference_fasta} --known-sites {config.known_sites} -O recal/EXP_01.recal.table && gatk ApplyBQSR -I {input[0]} -R {config.reference_fasta} --bqsr-recal-file recal/EXP_01.recal.table -O {output[0]}"

[rules.resources]
threads = 4
memory = "16G"

[rules.environment]
conda = "envs/gatk.yaml"

[[rules]]
name = "bqsr_control"
input = ["dedup/CTRL_01.dedup.bam"]
output = ["recal/CTRL_01.recal.bam"]
shell = "gatk BaseRecalibrator --read-validation-stringency SILENT -I {input[0]} -R {config.reference_fasta} --known-sites {config.known_sites} -O recal/CTRL_01.recal.table && gatk ApplyBQSR -I {input[0]} -R {config.reference_fasta} --bqsr-recal-file recal/CTRL_01.recal.table -O {output[0]}"

[rules.resources]
threads = 4
memory = "16G"

[rules.environment]
conda = "envs/gatk.yaml"

[[rules]]
name = "mutect2"
input = [
    "recal/EXP_01.recal.bam",
    "recal/CTRL_01.recal.bam",
]
output = ["variants/EXP_01.mutect2.vcf.gz"]
shell = "gatk Mutect2 --read-validation-stringency SILENT -I {input[0]} -I {input[1]} -normal CTRL_01 -R {config.reference_fasta} -O {output[0]}"

[rules.resources]
threads = 4
memory = "16G"

[rules.environment]
conda = "envs/gatk.yaml"

[[rules]]
name = "filter_mutect_calls"
input = ["variants/EXP_01.mutect2.vcf.gz"]
output = ["variants/EXP_01.mutect2.filtered.vcf.gz"]
shell = "gatk FilterMutectCalls -V {input[0]} -R {config.reference_fasta} -O {output[0]}"
description = "Filter raw Mutect2 calls"

[rules.resources]
threads = 2
memory = "8G"

[rules.environment]
conda = "envs/gatk.yaml"

[[rules]]
name = "haplotype_caller"
input = ["recal/CTRL_01.recal.bam"]
output = ["variants/CTRL_01.g.vcf.gz"]
shell = "gatk HaplotypeCaller --read-validation-stringency SILENT -I {input[0]} -R {config.reference_fasta} -O {output[0]} -ERC GVCF"
description = "Variant calling for control sample"

[rules.resources]
threads = 4
memory = "16G"

[rules.environment]
conda = "envs/gatk.yaml"

[[rules]]
name = "annotate_variants"
input = ["variants/EXP_01.mutect2.filtered.vcf.gz"]
output = ["annotated/EXP_01.annotated.vcf.gz"]
shell = "vep --input_file {input[0]} --output_file {output[0]} --format vcf --vcf --offline --cache"

[rules.resources]
threads = 4
memory = "8G"

[rules.environment]
conda = "envs/vep.yaml"

[[rules]]
name = "clinical_report"
input = ["annotated/EXP_01.annotated.vcf.gz"]
output = ["reports/EXP_01_report.html"]
shell = "python scripts/generate_report.py --input {input[0]} --output {output[0]} --sample EXP_01"
description = "Generate variant report"

[rules.resources]
threads = 1
memory = "4G"

[rules.environment]
conda = "envs/report.yaml"

Running the Workflow#

Validate#

$ oxo-flow validate examples/gallery/14_paired_experiment_control.oxoflow
✓ examples/gallery/14_paired_experiment_control.oxoflow — 13 rules, 12 dependencies

  ⚠ Warning: The following input files do not exist:
    - raw/EXP_01_R1.fq.gz
    - raw/EXP_01_R2.fq.gz
    - raw/CTRL_01_R1.fq.gz
    - raw/CTRL_01_R2.fq.gz

The ✓ line is what matters — validate exits 0 and the workflow is well-formed. The warnings are expected here: the raw/ FASTQs don't exist until you stage real data (see the note at the bottom of this page).

Dry-Run#

$ oxo-flow dry-run examples/gallery/14_paired_experiment_control.oxoflow
Plan: would run: 13 | skip: 0 | completed: 0 (DAG size: 13)
  ...

10. mutect2  [run: never completed]
     outputs: ["variants/EXP_01.mutect2.vcf.gz"]
     command: gatk Mutect2 ... -I recal/EXP_01.recal.bam -I recal/CTRL_01.recal.bam -normal CTRL_01 -R /data/references/GRCh38/genome.fa -O variants/EXP_01.mutect2.vcf.gz

Summary: 13 rules, total 86 threads declared, max 16 threads/rule
         13 rules with memory requirements

To execute:  oxo-flow run examples/gallery/14_paired_experiment_control.oxoflow -j 2

Because both sample names are hardcoded, the plan equals the template — 13 rules, no expansion, and no sample-readiness block. The mutect2 command is the teaching moment: both branches' recalibrated BAMs appear as -I inputs, and -normal CTRL_01 names the control by the SM tag that bwa_mem2_control wrote into the BAM header via its read-group string. haplotype_caller runs only on the control branch (variants/CTRL_01.g.vcf.gz) — a common strategy to profile the normal's germline background.

DAG Structure#

$ oxo-flow graph examples/gallery/14_paired_experiment_control.oxoflow
Workflow DAG: 13 rules, 12 dependencies
Depth: 8, Width: 2, Critical path: 8 steps

Critical path: fastp_control → bwa_mem2_control → markdup_control → bqsr_control → mutect2 → filter_mutect_calls → annotate_variants → clinical_report

graph shows the template DAG (before expansion — see the gallery index note). Through the first four levels every dependency stays within its own branch (bwa_mem2_control [depends: fastp_control]) — the two branches only meet at mutect2.

Expected Outputs#

Path Producer
trimmed/EXP_01_R1.fq.gz, trimmed/EXP_01_R2.fq.gz (+ CTRL_01) fastp_experiment / fastp_control
aligned/EXP_01.sorted.bam, aligned/CTRL_01.sorted.bam bwa_mem2_experiment / bwa_mem2_control
dedup/{EXP_01,CTRL_01}.dedup.bam + metrics markdup_experiment / markdup_control
recal/{EXP_01,CTRL_01}.recal.bam bqsr_experiment / bqsr_control
variants/CTRL_01.g.vcf.gz haplotype_caller (control only)
variants/EXP_01.mutect2.vcf.gz mutect2
variants/EXP_01.mutect2.filtered.vcf.gz filter_mutect_calls
annotated/EXP_01.annotated.vcf.gz annotate_variants
reports/EXP_01_report.html clinical_report

What's Next?#

Input data and environments

Input paths under /data/references/... and raw/ are placeholders — replace them with your own data. The validate warnings about missing raw/ files are expected until real data is staged. The referenced envs/*.yaml specs ship in examples/envs/.