feat(server): auto-refresh provider model catalog and push change events#1207
feat(server): auto-refresh provider model catalog and push change events#1207sailist wants to merge 4 commits into
Conversation
- add scheduled provider-model refresh in the daemon (configurable
interval + refresh-on-start) plus manual endpoints:
POST /providers:refresh and POST /providers/{id}:refresh
- publish global event.model_catalog.changed when a refresh changes
the catalog so connected clients can resync
- extract the refresh orchestrator into @moonshot-ai/kimi-code-oauth so
the CLI and server share managed/open-platform/custom-registry logic
- wire the web daemon client to the new refresh endpoints
🦋 Changeset detectedLatest commit: 35dd35c The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
commit: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0a0e3a5ce2
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const intervalMs = resolveIntervalMs(config.modelCatalog?.refreshIntervalMs); | ||
| const refreshOnStart = resolveRefreshOnStart(config.modelCatalog?.refreshOnStart); |
There was a problem hiding this comment.
When users configure the new scheduler in config.toml (for example [model_catalog] refresh_interval_ms = 0), this read still sees config.modelCatalog as undefined: transformTomlData only preserves object sections it explicitly handles (providers, models, background, etc.) and there is no model_catalog writer in configToTomlData. As a result the advertised config knobs are ignored and the daemon still refreshes on startup and every 6h unless env vars are used.
Useful? React with 👍 / 👎.
| case 'modelCatalogChanged': | ||
| break; |
There was a problem hiding this comment.
When the daemon's scheduled refresh adds or removes models while the web UI is open, this case only advances the websocket sequence; the models/providers refs used by the composer quick-switch and provider manager are only updated by loadModels()/loadProviders() or manual refresh. That leaves connected clients showing stale model entries after the pushed catalog-change event, including aliases the server has just removed, until some unrelated explicit reload happens.
Useful? React with 👍 / 👎.
When the daemon's scheduled refresh changes the catalog, the pushed event.model_catalog.changed only advanced the websocket sequence, leaving the web composer's model/provider refs stale until an unrelated reload. Reload both caches when the event arrives.
sailist
force-pushed
the
feat/auto-refresh-provider-models
branch
from
3d6422e to
35dd35c
Compare
1 participant
Related Issue
No linked issue — the problem is described below.
Problem
When
kimi serverruns as a long-lived daemon, remote provider model metadata (the/modelslists for managed OAuth, open platforms, and custom registries) was only fetched at startup, on login, or on a manual/model. A daemon that starts once and runs for days never re-fetched, so the catalog went stale. There was also no server-side API to refresh all providers (only an OAuth-only endpoint), and a refresh did not notify connected clients of the change.What changed
1. Automatic background refresh
A scheduler in the daemon refreshes the provider model catalog once on start and then on a configurable interval (default 6h). Configurable via
[modelCatalog]inconfig.toml(refreshIntervalMs,refreshOnStart) or env overrides. Refreshes are serialized with manual ones so they never race on writing config.2. Manual refresh API
POST /api/v1/providers:refresh— refresh every refreshable provider.POST /api/v1/providers/{id}:refresh— refresh a single provider.POST /api/v1/providers:refresh_oauth— unchanged (OAuth only).All return
{ changed, unchanged, failed }.3. Change event pushed to clients
New global
event.model_catalog.changedcarrying the per-provider diff (added/removed/failed), published when a refresh actually changes the catalog so connected clients can resync.4. Shared refresh logic
The refresh orchestration moved into the shared OAuth package so the CLI and the server use the same managed / open-platform / custom-registry logic instead of diverging.
Checklist
gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.