Implement filter search for disk path using AdpativeL#1173
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds an AdaptiveL-enabled, inline label-filtered graph-search path for disk index search, wiring it through DiskIndexSearcher while keeping existing behavior unchanged when AdaptiveL is not provided.
Changes:
- Add
adaptive_l: Option<AdaptiveL>toDiskIndexSearcher::search/search_internaland dispatch to anInlineFilterSearchpath when present. - Introduce a small
PredicateLabelProvideradapter to bridge the existing&dyn Fn(&u32) -> boolfilter predicate intoQueryLabelProvider<u32>. - Add disk-provider tests covering (1) no-op equivalence vs plain
Knnand (2) selective predicates returning only matching IDs; update callers to passNoneuntil CLI/JSON exposes AdaptiveL.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| diskann-tools/src/utils/search_disk_index.rs | Passes None for the new adaptive_l argument (not yet exposed via CLI). |
| diskann-disk/src/search/provider/disk_provider.rs | Implements filter_search via InlineFilterSearch + AdaptiveL, adds adapter + tests, and extends search APIs with adaptive_l. |
| diskann-disk/src/build/builder/core.rs | Updates test call sites to include the new adaptive_l parameter. |
| diskann-benchmark/src/backend/disk_index/search.rs | Passes None for the new adaptive_l argument (not yet exposed via benchmark JSON). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
You may want to take a look at the changes that are coming to filters with #1141 and make sure your changes are compatible |
| @@ -8,8 +8,12 @@ use std::{collections::HashSet, sync::atomic::AtomicBool, time::Instant}; | |||
| use diskann::utils::IntoUsize; | |||
There was a problem hiding this comment.
Worth noting this whole module diskann-tools/src/utils/search_disk_index.r looks dead. pub fn search_disk_index / SearchDiskIndexParameters aren't referenced from any binary.
I recommend deleting search_disk_index.rs (and its pub mod / pub use in utils/mod.rs) in a follow-up rather than carry the filter search migration.
| /// at visit time (not just during rerank). `adaptive_l = Some(_)` grows the | ||
| /// beam mid-search if the observed match specificity is low. | ||
|
|
||
| pub enum SearchPlan<'a> { |
There was a problem hiding this comment.
nit: Plan tends to read as a query/execution plan: something the engine generates and optimize. Consdier naming SearchMode or SearchKind which would line up more directly with what the variants express
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1173 +/- ##
==========================================
+ Coverage 89.46% 89.47% +0.01%
==========================================
Files 487 488 +1
Lines 92170 92363 +193
==========================================
+ Hits 82460 82642 +182
- Misses 9710 9721 +11
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
| } else { | ||
| Some(Box::new(move |vid: &u32| vf.contains(vid)) | ||
| as Box<dyn Fn(&u32) -> bool + Send + Sync>) | ||
| // Construct the mode from the JSON-driven |
There was a problem hiding this comment.
Can we expose adaptive L by creating a json deserializable struct here that has a function returning the SearchMode object? It's a bit awkward to just construct it from params here. If we had a dedicated struct it would be much easier to build on what's already here.
|
This looks fine to me pending a couple of things:
|
This PR is to add filter search to disk path using AdpativeL filter algorithm.
Reference PR: #1131
The main changes: