Getting Started¶
GRiD is driven by a single YAML configuration file. This page walks through every section — what each parameter does, why it is needed, and how to obtain or prepare the required input files.
Copy the bundled example and edit it for your cohort:
cp grid/example_config.yaml my_config.yaml
# edit my_config.yaml, then:
grid wgs my_config.yaml
Global Settings¶
These fields apply to every pipeline step.
samples_file: "path/to/sample_ids.txt"
directory_loc: "path/to/crams/"
reference_genome: "path/to/hg38.fa"
output_dir: "path/to/output/"
threads: 8
file_type: "cram"
chrom: "chr6"
start_bp: 160605062
end_bp: 160647661
output_file_type: "tsv"
Parameter |
Type |
Description |
|---|---|---|
|
path |
Plain-text file with one sample ID per line. IDs must match the file stems
of the CRAM/BAM files in |
|
path |
Directory containing the CRAM (or BAM) files for your cohort. |
|
path |
Reference genome FASTA (hg19 or hg38). Must be indexed ( |
|
path |
Directory where all pipeline outputs are written. Created if it does not exist. |
|
int |
Number of CPU threads shared across steps. Match to your SLURM
|
|
str |
Sequencing file format: |
|
str |
Target chromosome in the same notation as your CRAM headers
(e.g. |
|
int |
Start coordinate (0-based) of the VNTR region of interest.
For LPA KIV-2 in hg38: |
|
int |
End coordinate of the VNTR region of interest.
For LPA KIV-2 in hg38: |
|
str |
Output table format: |
Note
Supported VNTR loci. Mukamel et al. (2021) Science (doi:10.1126/science.abg8289) catalogued 734 coding VNTRs with elevated IBD2 rates across the human genome, identifying loci where copy number is likely to vary and be heritable. Their full code is available at Zenodo 4776804.
The same file is bundled with GRiD at
734_possible_coding_vntr_regions.IBD2R_gt_0.25.uniq.txt.
To run GRiD on a different locus, simply update the chrom, start_bp, and
end_bp fields in your config file to match the coordinates of your target region.
Step 1 — index¶
Verifies or creates .crai / .bai index files for all CRAM/BAM files in
directory_loc. Only needed if your files are not already indexed.
index:
run: False
output_file_prefix: "index_file_results"
Parameter |
Description |
|---|---|
|
Set |
|
Prefix for the log output file. |
Step 2 — count_reads¶
Counts properly paired reads overlapping the target VNTR region from each CRAM/BAM file. The resulting per-sample read counts are used downstream as a proxy for locus-specific sequencing depth.
count_reads:
run: True
output_file_prefix: "read_counts"
min_mapq: 1
flags:
- 83 # proper pair, read reverse strand
- 147 # proper pair, mate reverse strand
- 81 # read reverse strand
- 145 # mate reverse strand
Parameter |
Description |
|---|---|
|
Set |
|
Prefix for the output read-count file. |
|
Minimum mapping quality. Reads below this threshold are ignored.
|
|
List of SAM bitwise flags to include. Only reads whose flag exactly matches one of the listed values are counted. The defaults capture properly paired reads on both strands. See SAM flag documentation for flag definitions. |
Step 3 — mosdepth¶
Runs mosdepth in windowed mode to compute binned read depth across the full genome for each sample. The resulting depth profiles are used for normalization and neighbor finding.
mosdepth must be available on your PATH:
conda install -c bioconda mosdepth
# or: module load mosdepth (HPC)
mosdepth:
run: True
output_file_prefix: "mosdepth_results"
bin_size: 1000
mode: "fast"
region_name: "LPA"
work_dir: "path/to/mosdepth_work/"
remove_intermediate: True
Parameter |
Description |
|---|---|
|
Set |
|
Prefix for the aggregated depth output file. |
|
Genomic bin size in base pairs. |
|
|
|
Label applied to the VNTR region in output files (e.g. |
|
Scratch directory for per-sample mosdepth intermediate files. Should be on fast local or scratch storage. |
|
If |
Step 3a — mosdepth.normalize¶
Normalizes the depth matrix: filters low/high-coverage bins, applies a repeat-mask, and z-scores across individuals. See Coverage Normalization for the full method.
mosdepth:
normalize:
run: True
min_depth: 20
max_depth: 100
top_frac: 0.1
output_file_prefix: "mosdepth_results_normalized"
repeat_mask_file: "path/to/repeat_mask.bed"
Parameter |
Description |
|---|---|
|
Set |
|
Bins with median depth below this value are excluded. |
|
Bins with median depth above this value are excluded. |
|
Fraction of bins with the highest variance to retain for normalization
(e.g. |
|
Prefix for the normalized depth matrix output. |
|
BED file of repetitive regions to exclude before normalization. Download UCSC RepeatMasker tracks from the UCSC Table Browser (group: Repeats, track: RepeatMasker). |
Step 3b — mosdepth.neighbors¶
Finds read depth-matched nearest neighbors in the normalized depth space
using scikit-learn NearestNeighbors. See Nearest-Neighbor Identification.
mosdepth:
neighbors:
run: True
output_file_prefix: "neighbor_coverage"
num_neighbors: 5
zmax: 2.0
sigma2_max: 1000
Parameter |
Description |
|---|---|
|
Set |
|
Prefix for the neighbor assignment output file. |
|
Number of nearest neighbors to retain per sample. |
|
Clip normalized depth values to \(\pm\) |
|
Maximum allowed neighbor variance. Samples whose nearest-neighbor set has variance above this threshold are flagged. Increase if many samples are being excluded in diverse cohorts. |
Step 4 — compute_diploid_genotypes¶
Estimates diploid copy number for each sample using the neighbor-normalized depth ratio. For each sample, its locus depth is scaled by the mean depth of its nearest neighbors at the same locus. See Diploid Copy Number Estimation.
compute_diploid_genotypes:
run: True
output_file_prefix: "diploid_genotypes"
n_nbr: 300
Parameter |
Description |
|---|---|
|
Set |
|
Prefix for the diploid CN output file. |
|
Number of nearest neighbors to use when computing each sample’s diploid CN
estimate. Default: |
Step 5 — compute_haploid_genotypes¶
Decomposes diploid CN estimates into haplotype-specific copy numbers using a neighbor-based phasing algorithm. Two neighbor sources (IBS or IBD) and five phasing algorithms are supported. See Haplotype Inference for full algorithm details and Computing IBS/IBD Neighbors for how to generate the neighbors file.
IBS (original method):
compute_haploid_genotypes:
run: True
output_file_prefix: "haploid_genotypes"
method: "ibs"
phasing_method: "iterative"
ibs_output: "path/to/ibs_neighbors_chr6.tsv.gz"
min_neighbors: 1
max_neighbors: 10
n_iters: 20
IBD (recommended):
compute_haploid_genotypes:
run: True
output_file_prefix: "haploid_genotypes"
method: "ibd"
phasing_method: "graph_prop"
ibd_output: "path/to/output.match"
alpha: 0.7
n_iters: 20
max_neighbors: 20
min_length: 2.0
min_match: 0.95
Parameter |
Description |
|---|---|
|
Set |
|
Prefix for the haploid CN output file. |
|
Neighbor source: |
|
Phasing algorithm. Options: |
|
(graph_prop / graph_prop_weighted) Mixing weight between the
individual’s own diploid signal and the neighbor aggregate. Range (0, 1);
higher values trust the individual more. Default: |
|
Neighbor aggregation function. Options: |
|
Number of phasing iterations. Ignored by |
|
Minimum neighbors required on both haplotypes to phase a sample.
Samples below this threshold receive mean-imputed values. Default: |
|
Maximum neighbors loaded per haplotype. Default: |
|
(IBS method) Path to IBS neighbors file from |
|
(IBD method) Path to iLASH |
|
(IBD method) Minimum IBD segment length in cM. Default: |
|
(IBD method) Minimum iLASH match score; lenient floor for long
segments when |
|
(IBD method, optional) Strict match score required at |
|
(IBD method) If |
|
(IBD method, weighted) \(\lambda\) in bp for the Lorentzian
distance weight function. Default: |
Note
Either ibs_output or ibd_output must be provided depending on the
chosen method. See Computing IBS/IBD Neighbors for file format details and how to
generate each neighbors file.
Full Example Config¶
1samples_file: "path/to/sample_ids.txt"
2directory_loc: "path/to/crams/"
3reference_genome: "path/to/hg38.fa"
4output_dir: "path/to/output/"
5threads: 4
6file_type: "cram"
7
8# VNTR region of interest
9chrom: "chr6"
10start_bp: 160605062
11end_bp: 160647661
12
13output_file_type: "tsv"
14
15index:
16 run: False
17 output_file_prefix: "index_file_results"
18
19count_reads:
20 run: True
21 output_file_prefix: "read_counts"
22 min_mapq: 1
23 flags:
24 - 83 # proper pair, reverse strand
25 - 99 # mate forward, reverse strand (unpaired-ish)
26 - 147 # proper pair, mate reverse
27 - 163 # mate forward, reverse strand (unpaired-ish)
28 - 81 # reverse, mate forward (unpaired-ish)
29 - 97 # forward, mate reverse (unpaired-ish)
30 - 145 # mate reverse
31 - 161 # mate forward
32
33mosdepth:
34 run: True
35 output_file_prefix: "mosdepth_results"
36 bin_size: 1000
37 work_dir: "path/to/mosdepth_work/"
38 remove_intermediate: True
39 example_bed: null
40
41 normalize:
42 run: True
43 output_file_prefix: "ID_scale_zdepths"
44 combined_bins_gz: null # optional, defaults to output_dir/mosdepth_bins.txt.gz
45 repeat_mask_file: "path/to/repeat_mask_list.hg38.ucsc_bed"
46
47 neighbors:
48 run: True
49 output_file_prefix: "neighbor_coverage"
50 num_neighbors: 5
51 zmax: 2.0
52 sigma2_max: 1000
53
54 gc_normalize:
55 run: False
56
57compute_diploid_genotypes:
58 run: True
59 output_file_prefix: "diploid_genotypes"
60 n_nbr: 300
61
62compute_haploid_genotypes:
63 run: True
64 output_file_prefix: "haploid_genotypes"
65 method: "ibs" # "ibs" or "ibd"
66 min_neighbors: 1
67 max_neighbors: 10
68 n_iters: 100
69
70 # IBS method: provide computeIBSpbwt output
71 ibs_output: "path/to/ibs_neighbors_chr6.tsv.gz"
72
73 # IBD method: provide iLASH output and filtering thresholds
74 # ibd_output: "path/to/ilash_output.txt"
75 # weighted: False # True = Lorentzian distance+match weighting
76 # weight_scale: 1000000 # bp half-width for distance weight
77 # min_length: 0.5 # minimum IBD segment length (cM)
78 # min_match: 0.70 # minimum IBD match score