Coverage Normalization

Normalizes mosdepth coverage in two steps: within-individual (divide by sample mean) and across-individual (z-score transformation). High-variance regions are then selected to reduce noise before neighbor computation.

grid.utils.normalize_mosdepth.build_depth_matrix(regions_to_extract)

Build depth matrix from extracted regions.

Parameters:

regions_to_extract (dict[str, list[tuple[int, int, float]]]) – Dictionary of sample IDs to region lists

Returns:

{(start,end): depth}}

Return type:

Dictionary mapping {sample

grid.utils.normalize_mosdepth.build_matrix_from_regions(regions_to_extract, individuals_order=None)

Build numpy matrix from regions_to_extract.

Parameters:

  • regions_to_extract – dict of {individual_id: [(start,end,depth),…]}

  • individuals_order – optional list of individual IDs to order rows

Returns:

list of individual IDs in order regions_list: list of (start,end) tuples in order mat: numpy array of shape (n_individuals, n_regions) with depths

Return type:

individuals_order

grid.utils.normalize_mosdepth.compute_population_mean_depths(individuals, mosdepth_dir, chromosome, start, end, excluded, threads=1, console=None)

Compute the population mean depth for every region across all individuals.

This mirrors the C++ first pass (batches 0–9) that computes mean_depths[] and uses it as the depth filter threshold — so the region mask is universal (same regions included/excluded for every sample) rather than per-sample.

Parameters:

  • individuals (dict[str, Path]) – {sample_id: bed_gz_path}

  • mosdepth_dir (str) – directory with mosdepth output (passed through to reader)

  • chromosome (str) – target chromosome (or None for all)

  • start/end – target region bounds (or None for whole chromosome)

  • excluded (dict[str, set[int]]) – repeat-mask kb bins {chrom: set(kb)}

  • threads (int) – worker threads for parallel reading

  • console – Rich console for logging

  • start (int)

  • end (int)

Returns:

population_mean_depth}

Return type:

{(region_start, region_end)

grid.utils.normalize_mosdepth.filter_empty_samples(regions_to_extract, console=None)

Remove samples that have zero regions.

Parameters:

  • regions_to_extract (dict[str, list[tuple[int, int, float]]]) – dict of {sample_id: [(start, end, depth), …]}

  • console – optional Rich console for logging

Returns:

Filtered dictionary with only samples that have ≥1 region

grid.utils.normalize_mosdepth.find_bed_gz_for_individual(individual_id, mosdepth_dir)

Find the .regions.bed.gz file for a given individual in the mosdepth directory.

Parameters:

  • individual_id (str) – Sample ID to find

  • mosdepth_dir (str) – Directory containing mosdepth output files

Returns:

Path to the individual’s .regions.bed.gz file, or Path to non-existent file if not found

Return type:

Path

grid.utils.normalize_mosdepth.load_repeat_mask(repeat_bed)

Load repeat regions into {chrom: set(kb_bins)}.

Parameters:

repeat_bed (str) – Path to repeat mask BED file

Returns:

Dictionary mapping chromosomes to sets of excluded kb bins

Return type:

dict[str, set[int]]

grid.utils.normalize_mosdepth.map_mosdepth_files_to_samples(mosdepth_dir, samples)

Map sample IDs to their mosdepth.global.dist files.

Parameters:

  • mosdepth_dir (str) – Directory containing mosdepth output files

  • samples (list[str]) – List of sample IDs

Returns:

Dictionary mapping sample IDs to their global.dist file paths

Return type:

dict[str, Path]

grid.utils.normalize_mosdepth.norm_chrom(chrom)

Normalise a chromosome name to ‘chrN’ form. ‘6’ -> ‘chr6’, ‘chr6’ -> ‘chr6’, ‘chrX’ -> ‘chrX’

Parameters:

chrom (str)

Return type:

str

grid.utils.normalize_mosdepth.normalize_matrix(mat)

Normalize the depth matrix to match C++ normalize_mosdepth_inflow logic.

Steps:

  1. Row-wise (within-individual): divide each row by its mean depth.

  2. Column-wise (across-individual): for each region compute mu and sigma2, then transform x -> (x - mu) / sqrt(mu).

  3. Compute variance ratios (100 * sigma2 / mu) for every region.

  4. Median-based rescaling: multiply all values by 1/sqrt(median_ratio/100) so the output approximates true z-scores (matches C++ scale factor).

Parameters:

mat – numpy array shape (n_individuals, n_regions) with raw depths.

Returns:

numpy array same shape, rescaled z-scores. variance_ratios: dict {region_index: 100*sigma2/mu} for non-NaN regions.

Return type:

normalized_mat

grid.utils.normalize_mosdepth.normalize_mosdepth(config, console)

Normalizes mosdepth coverage for all samples in the config file.

Parameters:

  • config (dict) – The configuration data loaded from the config file.

  • console (Console) – The Rich console object for logging.

grid.utils.normalize_mosdepth.process_one_individual(individual_id, mosdepth_dir, chromosome, start, end, valid_regions, excluded)

This runs in its own process. Returns (individual_id, [(start,end,depth),…]) args_tuple contains:

individual_id, mosdepth_dir, chromosome, start, end, min_depth, max_depth, excluded

grid.utils.normalize_mosdepth.select_high_variance_regions(variance_ratios, top_frac=0.9)

Select top fraction of regions by variance ratio.

Parameters:

  • variance_ratios (dict[int, float]) – {region_index: variance_ratio}

  • top_frac (float) – fraction of top regions to retain

Returns:

List of selected region indices

Return type:

list[int]

grid.utils.normalize_mosdepth.write_normalized_output(mat, individuals_order, selected_indices, output_file, col_means, col_vars, individual_raw_means, ratio_mult=100.0)

Write normalised matrix in the C++-compatible format.

File layout

Line 0 : N <tab> Rwant <tab> mu_1 <tab> mu_2 … Line 1 : N <tab> Rwant <tab> varRatio_1 <tab> varRatio_2 … Line 2+ : ID <tab> indiv_scale <tab> z_1 <tab> z_2 …

param mat:

normalised + rescaled matrix (n_individuals × n_regions).

param individuals_order:

list of sample IDs.

param regions_list:

list of (start, end) tuples for all regions.

param selected_indices:

indices of high-variance regions to keep.

param output_file:

Path for the output .tsv.gz file.

param col_means:

per-region means (length = n_regions).

param col_vars:

per-region variances ddof=1 (length = n_regions).

param individual_raw_means:

per-sample mean depth before any normalisation (used to write the scale column; matches C++ 0.01f*scales[n] where scales[n] was in units of 0.01x, so the written value is in 1x units).

param ratio_mult:

multiplier used for variance ratios (default 100).

Parameters:

  • mat (ndarray)

  • individuals_order (list)

  • selected_indices (list)

  • output_file (Path)

  • col_means (ndarray)

  • col_vars (ndarray)

  • individual_raw_means (ndarray)

  • ratio_mult (float)