Nearest-Neighbor Identification

Identifies the top-N nearest neighbors per individual in normalized depth space using Euclidean distance (scikit-learn). Neighbors serve as an ancestry-matched reference panel for diploid copy number normalization.

grid.utils.find_neighbors.filter_regions_by_variance(sigma2ratios, frac_r=1.0, sigma2_max=1000.0, console=None)

Select region indices whose sigma2 ratio passes the [sigma2min, sigma2max] filter, exactly mirroring the C++ logic.

Parameters:

  • sigma2ratios (ndarray) – per-region variance ratios from the file header (length R)

  • frac_r (float) – fraction of regions to retain by lower bound (C++ = 1.0)

  • sigma2_max (float) – upper bound on variance ratio (C++ = 1000)

  • console – Rich console for logging

Returns:

np.ndarray of int indices that pass the filter R_use : number of retained regions

Return type:

valid_indices

grid.utils.find_neighbors.find_neighbors(config, console)

Find nearest neighbors from normalized coverage data for all samples.

Mirrors the C++ find_neighbors.cpp logic:

  1. Read N, R, per-region means (header row 0) — means unused here but read to advance the file pointer.

  2. Read per-region sigma2 ratios (header row 1) — used to filter regions.

  3. Clip z-scores to [-zmax, zmax] and replace NaN → 0.

  4. Filter regions: keep those with sigma2_min <= ratio <= sigma2_max.

  5. Compute pairwise Euclidean distances on the filtered z-score matrix.

  6. For each individual output the top N_OUTPUT nearest neighbors (excluding self), with normalised distance = raw_dist / (2 * R_use).

Parameters:

  • config – configuration dictionary

  • console – Rich console for logging

grid.utils.find_neighbors.find_neighbors_sklearn(data_matrix, individuals, n_neighbors=500)

Find nearest neighbors using scikit-learn’s NearestNeighbors.

The C++ computes raw squared Euclidean distances then writes dist / (2 * R_use). sklearn returns Euclidean (not squared), so we square the distances before normalising to match — the normalisation itself happens in save_neighbors.

Clipping and NaN-filling must be done BEFORE calling this function (handled in find_neighbors).

Parameters:

  • data_matrix (ndarray) – np.ndarray [N x R_use], already clipped and NaN-filled

  • individuals (list[str]) – list of N sample IDs

  • n_neighbors (int) – number of neighbors to return per individual (C++ = 500)

Returns:

[(neighbor_id, squared_euclidean_distance), …]} Length of each list = min(n_neighbors, N-1).

Return type:

dict {individual_id

grid.utils.find_neighbors.read_normalized_data(input_file)

Parse the gzipped normalized depth file produced by normalize_mosdepth.

File layout (matches write_normalized_output):

Line 0 : N <tab> Rwant <tab> mu_1 … mu_Rwant Line 1 : N <tab> Rwant <tab> varRatio_1 … varRatio_Rwant Line 2+: ID <tab> scale <tab> z_1 … z_Rwant

Parameters:

input_file (Path) – Path to the .tsv.gz output of normalize_mosdepth

Returns:

list of sample IDs (length N) sigma2ratios : np.ndarray of per-region variance ratios (length Rwant) data_matrix : np.ndarray shape [N x Rwant] of z-scores scales : dict {sample_id: scale_value}

Return type:

individuals

grid.utils.find_neighbors.save_neighbors(neighbors_dict, scales, output_file, zmax, R_use)

Write neighbors to a gzipped text file.

Output format per line (matches C++):

ID scale nb1_ID nb1_scale nb1_norm_dist nb2_ID …

where norm_dist = squared_euclidean_distance / (2 * R_use), exactly as in C++: fout << dists[n*Nbatch+i] / (2*Ruse)

NOTE: the original Python constructed the output filename here again, causing a double-suffix bug (.zMax2.0.txt.gz.zMax2.0.txt.gz). The caller now passes the fully-resolved Path directly.

Parameters:

  • neighbors_dict (dict[str, list[tuple[str, float]]]) – {id: [(neighbor_id, squared_dist), …]}

  • scales (dict[str, float]) – {id: scale_value}

  • output_file (Path) – fully-resolved output Path (no filename construction here)

  • zmax (float) – z-score clip value (unused in output, kept for logging)

  • R_use (int) – number of regions used, for distance normalisation

Return type:

None