fix(analysis): serialize background analyzer behind scans + batch writes#320
Open
InstaZDLL wants to merge 1 commit into
Open
fix(analysis): serialize background analyzer behind scans + batch writes#320InstaZDLL wants to merge 1 commit into
InstaZDLL wants to merge 1 commit into
Conversation
The library-wide audio analyzer was a second SQLite writer running concurrently with the scanner, which caused two real bugs on large libraries: - Lost results: a per-row `INSERT INTO track_analysis` racing a concurrent scan hit `database is locked` after the 5 s busy-timeout and the freshly-decoded BPM / loudness was silently dropped (`rows_affected=0`), wasting the expensive Symphonia decode. - Inflated scan wall-clock: the analyzer's CPU-heavy decode + write bursts contended for every core and the single writer, turning a clean ~34 s AAC scan into a minute-plus. Fix: - Park the analyzer while a scan is walking. `scan_folder_inner` (the one path every scan entry point routes through) bumps a process-wide `SCANS_IN_FLIGHT` counter via an RAII guard; `run_analyze_library` waits it out at the top of each iteration, cancel-aware. - Batch `track_analysis` writes 16-at-a-time in one transaction and retry the whole batch on `SQLITE_BUSY` / `SQLITE_LOCKED` (low-byte 5/6) with exponential backoff, so a transient lock never drops a decoded result. The buffer is flushed on cancel + at run end, so a stopped run still saves everything it had decoded. Adds unit tests for the busy-code classifier. Docs synced (library.md + the single-writer rule in CLAUDE.md).
InstaZDLL
added
scope: backend
📝 WalkthroughWalkthroughLe scan bibliothèque expose désormais un état partagé “en cours”, et l’analyse en arrière-plan attend la fin du scan avant de tamponner puis persister les résultats ChangesSynchronisation scan et analyse
Sequence Diagram(s)sequenceDiagram
participant scan_folder_inner
participant run_analyze_library
participant scan_in_flight
participant SQLite
scan_folder_inner->>scan_folder_inner: incrémente SCANS_IN_FLIGHT
run_analyze_library->>scan_in_flight: lit l’état du scan
loop tant que scan_in_flight() est vrai
run_analyze_library->>run_analyze_library: attend `wait_out_active_scan()`
end
run_analyze_library->>SQLite: flush `track_analysis` par lots
alt SQLITE_BUSY / SQLITE_LOCKED
run_analyze_library->>run_analyze_library: backoff exponentiel
run_analyze_library->>SQLite: réessaie le lot
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src-tauri/crates/app/src/commands/analysis.rs`:
- Around line 329-335: Le flux de persistance dans persist_batch_once et ses
flush via flush_analysis_batch peut encore écrire pendant un scan, car l’attente
n’est faite qu’avant le décodage. Ajoute une vérification d’attente du scan
juste avant chaque flush DB, y compris dans les chemins qui déclenchent le flush
par taille de batch et en fin de traitement, afin que batch,
flush_analysis_batch et persist_batch_once ne continuent pas tant qu’un scan est
actif.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 23196ae2-478f-4cc0-bdc1-6b97cb217960
📒 Files selected for processing (4)
CLAUDE.mddocs/features/library.mdsrc-tauri/crates/app/src/commands/analysis.rssrc-tauri/crates/app/src/commands/scan.rs
Comment on lines
+329
to
+335
| batch.push(PendingAnalysis { | ||
| track_id, | ||
| result, | ||
| analyzed_at: Utc::now().timestamp_millis(), | ||
| }); | ||
| if batch.len() >= ANALYSIS_BATCH_SIZE { | ||
| failed += flush_analysis_batch(pool, &mut batch).await; |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Attends aussi le scan avant chaque flush DB.
L’attente actuelle est seulement avant le décodage. Si un scan démarre entre la ligne 308 et un flush ligne 335 ou 363, persist_batch_once() peut encore écrire pendant le scan et finir par dropper tout le batch après retries.
Correctif proposé
async fn flush_analysis_batch(pool: &SqlitePool, batch: &mut Vec<PendingAnalysis>) -> u32 {
if batch.is_empty() {
return 0;
}
const MAX_ATTEMPTS: usize = 6;
let mut backoff = Duration::from_millis(50);
for attempt in 1..=MAX_ATTEMPTS {
+ while crate::commands::scan::scan_in_flight() {
+ tokio::time::sleep(SCAN_WAIT_POLL).await;
+ }
match persist_batch_once(pool, batch).await {
Ok(()) => {
batch.clear();
return 0;
}Also applies to: 360-363, 473-489
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src-tauri/crates/app/src/commands/analysis.rs` around lines 329 - 335, Le
flux de persistance dans persist_batch_once et ses flush via
flush_analysis_batch peut encore écrire pendant un scan, car l’attente n’est
faite qu’avant le décodage. Ajoute une vérification d’attente du scan juste
avant chaque flush DB, y compris dans les chemins qui déclenchent le flush par
taille de batch et en fin de traitement, afin que batch, flush_analysis_batch et
persist_batch_once ne continuent pas tant qu’un scan est actif.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problème
L'analyseur audio (BPM / loudness, sweep bibliothèque) tournait comme second writer SQLite en concurrence du scanner, ce qui provoquait deux bugs réels sur les grosses bibliothèques (mesuré sur ~900 fichiers AAC) :
INSERT INTO track_analysisligne-par-ligne en course avec un scan tombait surdatabase is lockedaprès le busy-timeout de 5 s →rows_affected=0, le BPM/loudness fraîchement décodé était jeté silencieusement (décode Symphonia coûteux gaspillé).Fix
scan_folder_inner(chemin commun àscan_folder/rescan_library/import_paths/ fs-watcher / rescan de démarrage) incrémente un compteur process-wideSCANS_IN_FLIGHTvia un guard RAII.run_analyze_libraryle wait-out en tête de chaque itération, cancel-aware.track_analysisgroupées par 16 dans une transaction, avec retry du batch entier surSQLITE_BUSY/SQLITE_LOCKED(octet bas 5/6) en backoff exponentiel → plus aucune perte sur lock transitoire. Le buffer est flush sur cancel + en fin de run → un run stoppé sauve quand même tout ce qu'il avait décodé.Tests
code_is_busy(masquage des codes étendus SQLite).cargo check+cargo clippy --all-targetsverts. (Tests app-crate non exécutables localement sur Windows — DLL Tauri — valent via CI.)Docs
docs/features/library.mdsection Audio analysis.CLAUDE.md.Summary by CodeRabbit
Nouvelles fonctionnalités
Corrections de bugs