Skip to content

13 — Germline Variant Calling (Simple Chain)#

The per-sample GATK germline chain: FastQC + fastp QC, BWA-MEM2 alignment with read groups, MarkDuplicates, BQSR, and HaplotypeCaller GVCF — the simpler sibling of WGS Germline Variant Calling, which adds joint calling, VQSR, and annotation.

What It Demonstrates#

  • Read-group string built with printf '@RG\tID:%s\tSM:%s\tPL:ILLUMINA' renders real TAB separators
  • Each stage declares its own environment (conda for QC tools, singularity for GATK)
  • Mix of environment backends per rule is first-class

Pipeline Overview#

graph TD
    F[fastp] --> A[bwa_align]
    A --> MD[mark_duplicates]
    MD --> BR[base_recalibrator]
    BR --> AB[apply_bqsr]
    AB --> HC[haplotype_caller]
    Q[fastqc]

Three samples (NA12878, NA12891, NA12892) expand every rule into three parallel instances. The backbone is a strict six-stage chain ending in HaplotypeCaller; fastqc is a standalone QC rule with no downstream consumers — it runs independently of the chain.

Workflow Definition#

# examples/gallery/13_simple_variant_calling.oxoflow
# 13 — Germline Variant Calling (Simple Chain)
# Per-sample germline best practices: FastQC/fastp QC → BWA-MEM2 →
# MarkDuplicates → BQSR → HaplotypeCaller GVCF. The simpler sibling of
# gallery 07 (which adds joint calling, VQSR, and annotation).

[workflow]
name = "simple-variant-calling"
version = "1.0.0"
description = "A simple variant calling pipeline demonstrating oxo-flow features"
author = "Traitome"

[config]
reference = "/data/reference/GRCh38.fa"
known_sites = "/data/reference/dbsnp_146.hg38.vcf.gz"

[defaults]
threads = 4
memory = "8G"

[report]
# Optional HTML template override — "report.html" is the built-in default;
# any other value is a Tera template path resolved next to the workflow file.
# template = "report.html"
#
# Section IDs are the built-in generator names (oxo-flow report --list-sections).
# The output format is chosen on the CLI with -f (html, json, md, pdf).
sections = [
    "universal",
    "workflow-info",
    "execution-status",
    "metrics",
    "provenance",
]

[[rules]]
name = "fastqc"
input = [
    "raw/{sample}_R1.fastq.gz",
    "raw/{sample}_R2.fastq.gz",
]
output = [
    "qc/{sample}_R1_fastqc.html",
    "qc/{sample}_R2_fastqc.html",
]
shell = "fastqc {input} -o qc/"
description = "Quality control check on raw sequencing reads"

[rules.resources]
threads = 2

[rules.environment]
conda = "envs/qc.yaml"

[[rules]]
name = "fastp"
input = [
    "raw/{sample}_R1.fastq.gz",
    "raw/{sample}_R2.fastq.gz",
]
output = [
    "trimmed/{sample}_R1.fastq.gz",
    "trimmed/{sample}_R2.fastq.gz",
    "qc/{sample}_fastp.json",
]
shell = """
fastp -i {input[0]} -I {input[1]} -o {output[0]} -O {output[1]} --json {output[2]} --thread {threads}
"""
description = "Adapter trimming and quality filtering"

[rules.resources]
threads = 8

[rules.environment]
conda = "envs/fastp.yaml"

[[rules]]
name = "bwa_align"
input = [
    "trimmed/{sample}_R1.fastq.gz",
    "trimmed/{sample}_R2.fastq.gz",
]
output = ["aligned/{sample}.sorted.bam"]
shell = '''
rg=$(printf '@RG\tID:%s\tSM:%s\tPL:ILLUMINA' '{sample}' '{sample}')
bwa-mem2 mem -t {threads} -R "$rg" {config.reference} {input[0]} {input[1]} | samtools sort -@ 4 -o {output[0]}
samtools index {output[0]}
'''
description = "Read alignment to reference genome"

[rules.resources]
threads = 16
memory = "32G"

# The pipe needs BOTH bwa-mem2 and samtools; the single-tool docker image
# cannot run it — use a conda environment that ships the pair.
[rules.environment]
conda = "envs/alignment.yaml"

[[rules]]
name = "mark_duplicates"
input = ["aligned/{sample}.sorted.bam"]
output = [
    "dedup/{sample}.dedup.bam",
    "dedup/{sample}.metrics.txt",
]
shell = """
gatk MarkDuplicates --VALIDATION_STRINGENCY SILENT -I {input[0]} -O {output[0]} -M {output[1]} --CREATE_INDEX true
"""
description = "Mark PCR duplicate reads"

[rules.resources]
threads = 4
memory = "16G"

[rules.environment]
singularity = "docker://broadinstitute/gatk:4.5.0.0"

[[rules]]
name = "base_recalibrator"
input = ["dedup/{sample}.dedup.bam"]
output = ["recal/{sample}.recal_data.table"]
shell = """
gatk BaseRecalibrator --read-validation-stringency SILENT -I {input[0]} -R {config.reference} --known-sites {config.known_sites} -O {output[0]}
"""
description = "Base quality score recalibration table"

[rules.resources]
threads = 4
memory = "16G"

[rules.environment]
singularity = "docker://broadinstitute/gatk:4.5.0.0"

[[rules]]
name = "apply_bqsr"
input = [
    "dedup/{sample}.dedup.bam",
    "recal/{sample}.recal_data.table",
]
output = ["recal/{sample}.recal.bam"]
shell = """
gatk ApplyBQSR --read-validation-stringency SILENT -I {input[0]} -R {config.reference} --bqsr-recal-file {input[1]} -O {output[0]}
"""
description = "Apply base quality score recalibration"

[rules.resources]
threads = 4
memory = "16G"

[rules.environment]
singularity = "docker://broadinstitute/gatk:4.5.0.0"

[[rules]]
name = "haplotype_caller"
input = ["recal/{sample}.recal.bam"]
output = ["variants/{sample}.g.vcf.gz"]
shell = """
gatk HaplotypeCaller --read-validation-stringency SILENT -R {config.reference} -I {input[0]} -O {output[0]} -ERC GVCF --native-pair-hmm-threads {threads}
"""
description = "Germline variant calling"

[rules.resources]
threads = 4
memory = "16G"

[rules.environment]
singularity = "docker://broadinstitute/gatk:4.5.0.0"

[[sample_groups]]
name = "samples"
samples = [
    "NA12878",
    "NA12891",
    "NA12892",
]

[sample_groups.metadata]

Running the Workflow#

Validate#

$ oxo-flow validate examples/gallery/13_simple_variant_calling.oxoflow
✓ examples/gallery/13_simple_variant_calling.oxoflow — 7 rules, 6 dependencies

  ⚠ Warning: The following input files do not exist:
    - raw/NA12878_R1.fastq.gz
    - raw/NA12878_R2.fastq.gz
    ...

The ✓ line is what matters — validate exits 0 and the workflow is well-formed. The warnings are expected here: the raw/ FASTQs don't exist until you stage real data (see the note at the bottom of this page).

Dry-Run#

$ oxo-flow dry-run examples/gallery/13_simple_variant_calling.oxoflow
Plan: would run: 21 | skip: 0 | completed: 0 (DAG size: 21)
Sample readiness: 0/3 complete, 3 waiting
    ⏳ NA12878 (missing: raw/NA12878_R1.fastq.gz +1 more)
    ⏳ NA12891 (missing: raw/NA12891_R1.fastq.gz +1 more)
    ⏳ NA12892 (missing: raw/NA12892_R1.fastq.gz +1 more)
  ...

1. fastp_samples_NA12878  [run: never completed]
     threads=8
     env=conda
     ...
7. bwa_align_samples_NA12878  [run: never completed]
     threads=16
     env=conda
     ...
10. mark_duplicates_samples_NA12878  [run: never completed]
     env=singularity

Summary: 21 rules, total 126 threads declared, max 16 threads/rule
         21 rules with memory requirements
         1 sample group, 0 pairs

To execute:  oxo-flow run examples/gallery/13_simple_variant_calling.oxoflow -j 2

The 7 template rules expand to 21 tasks (7 rules × 3 samples); instance names are suffixed _samples_<sample>. The env mix is visible directly in the dry-run output: the QC and alignment stages report env=conda, while from mark_duplicates onward every GATK stage reports env=singularity (docker://broadinstitute/gatk:4.5.0.0). BQSR is split into two rules — base_recalibrator emits the recalibration table, and apply_bqsr consumes it together with the deduplicated BAM.

DAG Structure#

$ oxo-flow graph examples/gallery/13_simple_variant_calling.oxoflow
Workflow DAG: 7 rules, 6 dependencies
Depth: 6, Width: 2, Critical path: 6 steps

Critical path: fastp → bwa_align → mark_duplicates → base_recalibrator → apply_bqsr → haplotype_caller

graph shows the template DAG (before expansion — see the gallery index note). apply_bqsr depends on both base_recalibrator (the recal table) and mark_duplicates (the dedup BAM).

Expected Outputs#

Path Producer
qc/{sample}_R1_fastqc.html, qc/{sample}_R2_fastqc.html fastqc
trimmed/{sample}_R1.fastq.gz, trimmed/{sample}_R2.fastq.gz, qc/{sample}_fastp.json fastp
aligned/{sample}.sorted.bam bwa_align
dedup/{sample}.dedup.bam, dedup/{sample}.metrics.txt mark_duplicates
recal/{sample}.recal_data.table base_recalibrator
recal/{sample}.recal.bam apply_bqsr
variants/{sample}.g.vcf.gz haplotype_caller

What's Next?#

  • The full pipeline: WGS Germline Variant Calling — adds cohort joint genotyping, VQSR, and VEP annotation on top of this chain.
  • Field reference: Rules — environment backends and per-rule resources.

Input data and environments

Input paths under /data/references/... and raw/ are placeholders — replace them with your own data. The validate warnings about missing raw/ files are expected until real data is staged. The referenced envs/*.yaml specs ship in examples/envs/.