feat(server): add proactive channel probes and load-aware stable routing / 服务端新增主动探活与负载感知稳定路由 (#330)

* wip: preserve local changes before upstream merge

* Improve proxy routing stability and add stable-first strategy

* Sync explicit-group routing strategy to eligible source routes

* Fix proxy retry and runtime health review issues

* fix: make proxy channel retry attempts configurable

* fix: harden proxy retry recovery and bookkeeping

* fix: address pr 257 review comments

* feat(proxy): add sticky session channel coordination

* feat(server): add proactive channel probes and load-aware stable routing

* test(server): stabilize session load routing selection assertion

* fix(proxy): harden codex forwarding stability

* fix(router): rotate stable-first channels by healthy order

* fix(server): address probe review feedback

* fix(server): address probe review follow-ups

* fix(ci): address CodeQL and core test failures

* feat(server): refine stable-first routing recovery and observation

* Finish PR 330 settings and review fixes

* Fix chat site endpoint CI mock

* Address latest PR 330 review comments

---------

Co-authored-by: Cita <juricek.chen@gmail.com>
This commit is contained in:
Bainily
2026-04-02 09:49:04 +08:00
committed by GitHub
parent 1133846998
commit 596d1b93cb
75 changed files with 6365 additions and 507 deletions
+19
View File
@@ -128,6 +128,25 @@ Metapi 的路由引擎按多因子加权选择最优通道。
|--------|------|--------|------|
| `TOKEN_ROUTER_CACHE_TTL_MS` | Token 路由缓存 TTL(毫秒) | `1500` | >= 100 |
### 模型弹活探测
用于异步校正“上游 `/models` 能看到,但实际调用不可用”的假阳性模型。
| 变量名 | 说明 | 默认值 | 范围 |
|--------|------|--------|------|
| `MODEL_AVAILABILITY_PROBE_ENABLED` | 启用后台模型可用性探测 | `false` | `true` / `false` |
| `MODEL_AVAILABILITY_PROBE_INTERVAL_MS` | 后台定时探测间隔(毫秒) | `1800000` | >= 60000 |
| `MODEL_AVAILABILITY_PROBE_TIMEOUT_MS` | 单次探测超时(毫秒) | `15000` | >= 3000 |
| `MODEL_AVAILABILITY_PROBE_CONCURRENCY` | 单账号模型探测并发数,默认串行避免对上游形成批量测活 | `1` | 1 ~ 16 |
说明:
- 探测任务在后台异步执行,不阻塞前台请求。
- 只对看起来像对话/推理模型的条目发最小化探测请求,明显的 embedding、rerank、moderation、tts、image/video 等模型会先跳过。
- 只有“明确模型不存在 / 不支持 / 无权限”的错误才会写回不可用;超时、5xx、网络抖动会归为“不确定”,不会误杀现有模型。
- 为了避免对上游形成意外的批量测活,默认关闭;只有显式开启后才会按间隔执行,且不会在服务启动后立刻扫全量模型。
- 同一批探测任务带去重,上一轮未结束时不会并发堆积;默认并发为 `1`,建议只在确认上游允许主动探测时启用。
### 路由预设建议
| 场景 | COST_WEIGHT | BALANCE_WEIGHT | USAGE_WEIGHT | 说明 |