fix(liquidity): bound external balance fetches with timeouts#3905
Open
TaprootFreak wants to merge 1 commit into
Open
fix(liquidity): bound external balance fetches with timeouts#3905TaprootFreak wants to merge 1 commit into
TaprootFreak wants to merge 1 commit into
Conversation
A single hung integration.getBalances() froze the entire liquidity balance refresh (Promise.allSettled waits for all), leaving every balance stale for hours. Add two layers: a default 60s timeout on the central HttpService so raw HTTP calls can no longer hang indefinitely, and a 120s per-integration backstop in refreshBalances for non-HTTP hangs. Slow backoffice/cron calls (App Insights KQL, Sumsub, Dilisense, Yapeal statements) get explicit generous timeouts so the new default does not truncate them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A single hung
integration.getBalances()froze the entire liquidity-balance refresh:refreshBalancesawaits all integrations viaPromise.allSettled, which never resolves if one promise never settles. The centralHttpServiceset no axios timeout, so a raw HTTP call (e.g. a bank API holding the connection open without responding) could hang indefinitely → all liquidity balances stayed frozen for hours, and the financial-data-log total ran on stale balances (plus side frozen while order-derived minus kept moving →valid=false, overstated total).Fix (two layers)
HttpModule.register({ timeout: 60_000 }). Every central-HttpServicecall is now bounded; per-requestconfig.timeoutstill overrides. ccxt exchanges are unaffected (own 30s timeout).refreshBalances: eachgetBalances()is wrapped inUtil.timeout(..., 120_000)(deliberately longer than the 60s HTTP timeout, so HTTP hangs are caught fine-grained at the source and only genuine non-HTTP hangs hit the backstop). A timed-out integration rejects →Promise.allSettleddrops only it → the other integrations still save.Avoiding regressions
Slow backoffice/cron HTTP calls that legitimately may exceed 60s get an explicit generous
timeout: 180_000so the new global default cannot truncate them: App Insights KQL, Sumsub, Dilisense, Yapeal statement/CAMT retrievals.Validation
type-check,lint,format:checkall pass.