amazon-science/uniqsketch v1.6.1

v1.6.1 — fix BloomFilter data race in index construction

Repository: amazon-science/uniqsketch

Tag: v1.6.1

Published: 2026-09-03T17:17:07Z

Prerelease: no

Release notes:

Fixes

• Data race in `BloomFilter::insert_make_change` that made index construction non-deterministic.

Each hash bit was written atomically, but the composite "was this k-mer newly inserted?" answer was not: two threads inserting the same k-mer could each win a subset of the bits and both report success. uniqsketch builds its distinct and solid Bloom filters in parallel across references and relies on exactly one caller observing success, so that a k-mer occurring in more than one reference is recorded as a repeat and excluded from every reference's unique set. When the race hid that repeat, the k-mer remained a signature for both references, and reads from one inflated the reported abundance of the other. Signature selection and querysketch --solid depend on the same contract.

The fix serialises the test-then-insert sequence on a lock striped by the leading hash value, so inserts of the same k-mer are ordered while unrelated k-mers proceed in parallel. The lock is only taken when the k-mer appears absent, so its cost is per distinct k-mer rather than per occurrence and index build time is unchanged.

Who is affected

Reference sets containing near-identical genomes (for example engineered variants of the same strain). Sets of well-separated references are unaffected in practice. All releases from v1.2.0 onward carry the bug.

On an 8-genome test set with four near-identical members, counting index builds that report a reference absent from the sample at above 1% abundance:

| configuration | before | after | |---|---|---| | default | 3 of 6 builds (worst 11.2%) | 0 of 6 | | --min-margin=2 | 2 of 6 (worst 15.3%) | 0 of 6 | | --min-margin=2 --strict-margin | 2 of 6 (worst 4.1%) | 0 of 6 | | --cluster=2000 | 0 of 6 | 0 of 6 |

Across a 280-sample binary mixture sweep, flat-mode main-reference mean absolute error improved from 0.44 to 0.08 percentage points, and the worst false positive fell from 3.08% to 0.074%. Index builds were also non-reproducible: three consecutive builds previously produced three different indexes, while six consecutive builds are now byte-identical.

If you worked around this with --cluster, that is no longer necessary. Existing indexes remain readable, but rebuilding is recommended if your reference set contains near-identical genomes.

Tests

• Added a concurrency regression test for insert_make_change: sixteen threads released from a spin

barrier insert the same k-mer, and exactly one must observe success. Verified to fail against the previous implementation and pass against the fixed one. bloomfiltertest now links pthreads (-pthread under make, Threads::Threads under CMake).

No API, CLI, or output-format changes.

Full changelog: https://github.com/amazon-science/uniqsketch/blob/main/CHANGELOG.md