forked from boos/BoosAPI
Merge pull request #114 from cita-777/fix/gemini-native-route-regressions
fix: address gemini native and route coverage regressions
This commit is contained in:
@@ -349,6 +349,7 @@ docker run -d --name metapi \
|
|||||||
> [!TIP]
|
> [!TIP]
|
||||||
> 初始管理员令牌即启动时配置的 `AUTH_TOKEN`。
|
> 初始管理员令牌即启动时配置的 `AUTH_TOKEN`。
|
||||||
> 若在 Compose 外运行且未显式设置 `AUTH_TOKEN`,默认为 `change-me-admin-token`(仅用于本地调试)。
|
> 若在 Compose 外运行且未显式设置 `AUTH_TOKEN`,默认为 `change-me-admin-token`(仅用于本地调试)。
|
||||||
|
> 桌面安装包首次启动也属于这类场景:如果你没有额外注入 `AUTH_TOKEN`,默认管理员令牌同样是 `change-me-admin-token`。
|
||||||
> 如果在「设置」面板中修改了管理员令牌,后续登录请使用新令牌。
|
> 如果在「设置」面板中修改了管理员令牌,后续登录请使用新令牌。
|
||||||
|
|
||||||
Docker Compose、桌面安装包、反向代理、升级与数据库选项等详见 [部署指南](https://metapi.cita777.me/deployment)。
|
Docker Compose、桌面安装包、反向代理、升级与数据库选项等详见 [部署指南](https://metapi.cita777.me/deployment)。
|
||||||
|
|||||||
@@ -350,6 +350,7 @@ After starting, visit `http://localhost:4000` and log in with your `AUTH_TOKEN`!
|
|||||||
> [!TIP]
|
> [!TIP]
|
||||||
> The initial admin token is the `AUTH_TOKEN` configured at startup.
|
> The initial admin token is the `AUTH_TOKEN` configured at startup.
|
||||||
> If running outside Compose without explicitly setting `AUTH_TOKEN`, the default is `change-me-admin-token` (for local debugging only).
|
> If running outside Compose without explicitly setting `AUTH_TOKEN`, the default is `change-me-admin-token` (for local debugging only).
|
||||||
|
> The desktop installer falls into this category on first launch too: if you do not inject `AUTH_TOKEN`, the default admin token is also `change-me-admin-token`.
|
||||||
> If you change the admin token in the Settings panel, use the new token for subsequent logins.
|
> If you change the admin token in the Settings panel, use the new token for subsequent logins.
|
||||||
|
|
||||||
For Docker Compose, desktop installers, reverse proxy, upgrades, and database options, see [Deployment Guide](docs/deployment.md).
|
For Docker Compose, desktop installers, reverse proxy, upgrades, and database options, see [Deployment Guide](docs/deployment.md).
|
||||||
|
|||||||
+6
-2
@@ -10,7 +10,7 @@
|
|||||||
|------|----------|--------------|----------|
|
|------|----------|--------------|----------|
|
||||||
| 云服务器 / NAS / 家用主机长期运行 | Docker / Docker Compose / Zeabur | 固定服务地址,例如 `http://your-host:4000` 或反向代理域名 | 你挂载的 `DATA_DIR` / 持久化卷 |
|
| 云服务器 / NAS / 家用主机长期运行 | Docker / Docker Compose / Zeabur | 固定服务地址,例如 `http://your-host:4000` 或反向代理域名 | 你挂载的 `DATA_DIR` / 持久化卷 |
|
||||||
| 免费云部署(24h 在线) | Render + TiDB + UptimeRobot | Render 分配的 `.onrender.com` 域名或自定义域名 | TiDB Serverless(外部 MySQL 数据库) |
|
| 免费云部署(24h 在线) | Render + TiDB + UptimeRobot | Render 分配的 `.onrender.com` 域名或自定义域名 | TiDB Serverless(外部 MySQL 数据库) |
|
||||||
| 个人电脑本地使用 | 桌面版安装包 | 桌面窗口;如需本机客户端直连,使用日志中打印的 `http://127.0.0.1:<port>` | Electron `app.getPath('userData')/data` |
|
| 个人电脑本地使用 | 桌面版安装包 | 桌面窗口;默认本机客户端直连使用 `http://127.0.0.1:4000`,局域网客户端可使用当前机器 IP + `4000`;如有覆盖则按 `METAPI_DESKTOP_SERVER_PORT` 的实际端口访问 | Electron `app.getPath('userData')/data` |
|
||||||
| 二次开发 / 调试 | 本地开发 | 前端 `http://localhost:5173`,后端默认 `http://localhost:4000` | 仓库内 `./data` 或自定义 `DATA_DIR` |
|
| 二次开发 / 调试 | 本地开发 | 前端 `http://localhost:5173`,后端默认 `http://localhost:4000` | 仓库内 `./data` 或自定义 `DATA_DIR` |
|
||||||
|
|
||||||
> [!NOTE]
|
> [!NOTE]
|
||||||
@@ -191,6 +191,10 @@ docker run -d --name metapi \
|
|||||||
- 托盘菜单支持重新打开窗口、重启后端、开机自启
|
- 托盘菜单支持重新打开窗口、重启后端、开机自启
|
||||||
- 支持基于 GitHub Releases 的应用内更新检查
|
- 支持基于 GitHub Releases 的应用内更新检查
|
||||||
|
|
||||||
|
> [!IMPORTANT]
|
||||||
|
> 桌面版首次启动时,如果没有显式注入 `AUTH_TOKEN`,管理员登录令牌默认是 `change-me-admin-token`。
|
||||||
|
> 这只适合本机初始调试使用,首次登录后应立即修改。
|
||||||
|
|
||||||
> [!NOTE]
|
> [!NOTE]
|
||||||
> 服务器部署不再提供裸 Node.js Release 压缩包,统一推荐 Docker / Docker Compose。
|
> 服务器部署不再提供裸 Node.js Release 压缩包,统一推荐 Docker / Docker Compose。
|
||||||
|
|
||||||
@@ -211,7 +215,7 @@ docker run -d --name metapi \
|
|||||||
|
|
||||||
## 反向代理
|
## 反向代理
|
||||||
|
|
||||||
以下反向代理配置面向 Docker / 服务器模式。桌面版内置后端默认只绑定本机 `127.0.0.1`,通常不作为公网服务直接暴露。
|
以下反向代理配置面向 Docker / 服务器模式。桌面版内置后端默认监听 `0.0.0.0:4000`,但通常仍作为单机桌面应用使用;如果要给局域网或公网客户端访问,请自行配置防火墙、反向代理和认证边界。若你显式设置了 `METAPI_DESKTOP_SERVER_PORT`,请把示例里的 `4000` 改成对应端口。
|
||||||
|
|
||||||
### Nginx
|
### Nginx
|
||||||
|
|
||||||
|
|||||||
+18
-12
@@ -80,18 +80,22 @@ docker compose up -d
|
|||||||
| 项目 | 说明 |
|
| 项目 | 说明 |
|
||||||
|------|------|
|
|------|------|
|
||||||
| 管理界面 | 应用启动后会直接打开桌面窗口,不需要假设固定的 `http://localhost:4000` |
|
| 管理界面 | 应用启动后会直接打开桌面窗口,不需要假设固定的 `http://localhost:4000` |
|
||||||
| 本地后端地址 | 桌面版把内置服务绑定到 `127.0.0.1`,默认会在 `4310..4399` 中挑选空闲端口;只有显式设置 `METAPI_DESKTOP_SERVER_PORT` 时才会固定 |
|
| 本地后端地址 | 桌面版内置服务默认监听 `0.0.0.0:4000`;桌面窗口和本机 curl 可继续使用 `http://127.0.0.1:4000`,局域网其他设备请使用当前机器的实际 IP + `4000`;如需改端口,可显式设置 `METAPI_DESKTOP_SERVER_PORT` |
|
||||||
| 数据目录 | 保存在 `app.getPath('userData')/data`,不是仓库里的 `./data` |
|
| 数据目录 | 保存在 `app.getPath('userData')/data`,不是仓库里的 `./data` |
|
||||||
| 日志目录 | 保存在 `app.getPath('userData')/logs`;托盘菜单提供 `Open Logs Folder` |
|
| 日志目录 | 保存在 `app.getPath('userData')/logs`;托盘菜单提供 `Open Logs Folder` |
|
||||||
|
|
||||||
|
> [!IMPORTANT]
|
||||||
|
> 桌面版首次启动时,如果你没有额外注入 `AUTH_TOKEN`,默认管理员令牌就是 `change-me-admin-token`。
|
||||||
|
> 首次登录后建议立即到「设置」里改成你自己的强密码令牌。
|
||||||
|
|
||||||
> [!TIP]
|
> [!TIP]
|
||||||
> - Windows 下常见路径是 `%APPDATA%\Metapi\data` 和 `%APPDATA%\Metapi\logs`。
|
> - Windows 下常见路径是 `%APPDATA%\Metapi\data` 和 `%APPDATA%\Metapi\logs`。
|
||||||
> - 如果你要把本机其他客户端接到桌面版内置后端,先到日志里查当前端口,不要写死 `4000`。
|
> - 如果没有额外覆盖端口,本机其他客户端可以直接连接 `http://127.0.0.1:4000`。
|
||||||
|
|
||||||
> [!WARNING]
|
> [!WARNING]
|
||||||
> **端口冲突排障:** 如果桌面版启动后报端口被占用,可能是 `4310..4399` 范围内的端口全被其他应用占用了。
|
> **端口冲突排障:** 桌面版默认使用 `4000` 端口;如果该端口被其他应用占用:
|
||||||
> - 设置环境变量 `METAPI_DESKTOP_SERVER_PORT=<指定端口>` 固定到一个空闲端口
|
> - 设置环境变量 `METAPI_DESKTOP_SERVER_PORT=<指定端口>` 改到一个空闲端口
|
||||||
> - 或关闭占用这些端口的应用后重启 Metapi Desktop
|
> - 或关闭占用 `4000` 的应用后重启 Metapi Desktop
|
||||||
|
|
||||||
> [!NOTE]
|
> [!NOTE]
|
||||||
> 服务器部署统一推荐 Docker / Docker Compose,不再提供裸 Node.js 的 Release 压缩包。
|
> 服务器部署统一推荐 Docker / Docker Compose,不再提供裸 Node.js 的 Release 压缩包。
|
||||||
@@ -196,7 +200,7 @@ npm run dev
|
|||||||
|----------|----------|----------------|
|
|----------|----------|----------------|
|
||||||
| Docker / Docker Compose | `http://localhost:4000` | `http://localhost:4000` |
|
| Docker / Docker Compose | `http://localhost:4000` | `http://localhost:4000` |
|
||||||
| 本地开发 | `http://localhost:5173` | `http://localhost:4000` |
|
| 本地开发 | `http://localhost:5173` | `http://localhost:4000` |
|
||||||
| 桌面版 | 直接使用桌面窗口 | 通常是4312,先从日志里的 `Proxy API:` 行确认当前 `http://127.0.0.1:<port>` |
|
| 桌面版 | 直接使用桌面窗口 | 默认 `http://127.0.0.1:4000`;如果设置了 `METAPI_DESKTOP_SERVER_PORT`,则按日志里的实际端口访问,局域网其他设备改用当前机器 IP + 同一端口 |
|
||||||
|
|
||||||
### Docker / 本地开发:直接用 curl 验证
|
### Docker / 本地开发:直接用 curl 验证
|
||||||
|
|
||||||
@@ -212,23 +216,25 @@ curl -sS http://localhost:4000/v1/chat/completions \
|
|||||||
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}'
|
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}'
|
||||||
```
|
```
|
||||||
|
|
||||||
### 桌面版:先确认当前端口再验证
|
### 桌面版:默认直接用 4000 验证
|
||||||
|
|
||||||
打开托盘菜单的 `Open Logs Folder`,在最新日志里查找类似下面的启动信息:
|
打开托盘菜单的 `Open Logs Folder`,在最新日志里查找类似下面的启动信息:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
Dashboard: http://127.0.0.1:4312
|
Dashboard: http://127.0.0.1:4000
|
||||||
Proxy API: http://127.0.0.1:4312/v1/chat/completions
|
Proxy API: http://127.0.0.1:4000/v1/chat/completions
|
||||||
```
|
```
|
||||||
|
|
||||||
然后把日志里的实际端口替换进 curl:
|
如果你没有覆盖端口,可直接执行:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
curl -sS http://127.0.0.1:4312/v1/models \
|
curl -sS http://127.0.0.1:4000/v1/models \
|
||||||
-H "Authorization: Bearer your-proxy-sk-token"
|
-H "Authorization: Bearer your-proxy-sk-token"
|
||||||
```
|
```
|
||||||
|
|
||||||
返回正常响应,说明代理链路已经可用。
|
如果你显式设置了 `METAPI_DESKTOP_SERVER_PORT`,再把上面的 `4000` 替换成日志里的实际端口。返回正常响应,说明代理链路已经可用。
|
||||||
|
|
||||||
|
如果你要从同一局域网的其他设备访问桌面版,把上面的 `127.0.0.1` 替换成这台电脑的实际局域网 IP,并确认系统防火墙已放行对应端口。
|
||||||
|
|
||||||
## 下一步
|
## 下一步
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -154,7 +154,7 @@ curl -sS http://localhost:4000/v1/chat/completions \
|
|||||||
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"ping"}]}'
|
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"ping"}]}'
|
||||||
```
|
```
|
||||||
|
|
||||||
以上示例默认服务端部署监听 `localhost:4000`。Desktop 内置后端运行在 `127.0.0.1` 的动态端口,优先通过应用界面、托盘日志或健康检查日志排查,不要假定固定端口。
|
以上示例默认服务端部署监听 `localhost:4000`。Desktop 内置后端默认监听 `0.0.0.0:4000`;本机排查通常可直接使用 `127.0.0.1:4000`,如果显式设置了 `METAPI_DESKTOP_SERVER_PORT`,则按日志里的实际端口访问;局域网排查改用当前机器的实际 IP。
|
||||||
|
|
||||||
### 自动化监控建议
|
### 自动化监控建议
|
||||||
|
|
||||||
|
|||||||
+2
-8
@@ -9,7 +9,6 @@ import {
|
|||||||
} from 'electron';
|
} from 'electron';
|
||||||
import log from 'electron-log';
|
import log from 'electron-log';
|
||||||
import electronUpdater from 'electron-updater';
|
import electronUpdater from 'electron-updater';
|
||||||
import getPort, { portNumbers } from 'get-port';
|
|
||||||
import { spawn, type ChildProcess } from 'node:child_process';
|
import { spawn, type ChildProcess } from 'node:child_process';
|
||||||
import { existsSync, mkdirSync } from 'node:fs';
|
import { existsSync, mkdirSync } from 'node:fs';
|
||||||
import { join } from 'node:path';
|
import { join } from 'node:path';
|
||||||
@@ -18,6 +17,7 @@ import {
|
|||||||
createDesktopHealthUrl,
|
createDesktopHealthUrl,
|
||||||
createDesktopServerUrl,
|
createDesktopServerUrl,
|
||||||
isFatalServerExit,
|
isFatalServerExit,
|
||||||
|
resolveDesktopServerPort,
|
||||||
resolveDesktopServerWorkingDir,
|
resolveDesktopServerWorkingDir,
|
||||||
waitForServerReady,
|
waitForServerReady,
|
||||||
} from './runtime.js';
|
} from './runtime.js';
|
||||||
@@ -68,12 +68,6 @@ function resolveExternalServerUrl() {
|
|||||||
return (process.env.METAPI_DESKTOP_EXTERNAL_SERVER_URL || '').trim();
|
return (process.env.METAPI_DESKTOP_EXTERNAL_SERVER_URL || '').trim();
|
||||||
}
|
}
|
||||||
|
|
||||||
async function resolveServerPort() {
|
|
||||||
const forcedPort = Number.parseInt(process.env.METAPI_DESKTOP_SERVER_PORT || '', 10);
|
|
||||||
if (Number.isFinite(forcedPort) && forcedPort > 0) return forcedPort;
|
|
||||||
return getPort({ port: portNumbers(4310, 4399) });
|
|
||||||
}
|
|
||||||
|
|
||||||
function showMainWindow() {
|
function showMainWindow() {
|
||||||
if (!mainWindow) return;
|
if (!mainWindow) return;
|
||||||
if (mainWindow.isMinimized()) mainWindow.restore();
|
if (mainWindow.isMinimized()) mainWindow.restore();
|
||||||
@@ -185,7 +179,7 @@ async function waitForManagedServerReady(url: string) {
|
|||||||
async function startManagedBackend() {
|
async function startManagedBackend() {
|
||||||
ensureDesktopDirs();
|
ensureDesktopDirs();
|
||||||
const serverEntryPath = getServerEntryPath();
|
const serverEntryPath = getServerEntryPath();
|
||||||
const port = await resolveServerPort();
|
const port = resolveDesktopServerPort(process.env);
|
||||||
serverUrl = createDesktopServerUrl(port);
|
serverUrl = createDesktopServerUrl(port);
|
||||||
|
|
||||||
const env = buildDesktopServerEnv({
|
const env = buildDesktopServerEnv({
|
||||||
|
|||||||
@@ -3,12 +3,13 @@ import {
|
|||||||
buildDesktopServerEnv,
|
buildDesktopServerEnv,
|
||||||
createDesktopServerUrl,
|
createDesktopServerUrl,
|
||||||
isFatalServerExit,
|
isFatalServerExit,
|
||||||
|
resolveDesktopServerPort,
|
||||||
resolveDesktopServerWorkingDir,
|
resolveDesktopServerWorkingDir,
|
||||||
waitForServerReady,
|
waitForServerReady,
|
||||||
} from './runtime.js';
|
} from './runtime.js';
|
||||||
|
|
||||||
describe('desktop runtime helpers', () => {
|
describe('desktop runtime helpers', () => {
|
||||||
it('builds desktop server env with loopback host and app directories', () => {
|
it('builds desktop server env with external listen host and app directories', () => {
|
||||||
const env = buildDesktopServerEnv({
|
const env = buildDesktopServerEnv({
|
||||||
inheritedEnv: {
|
inheritedEnv: {
|
||||||
AUTH_TOKEN: 'admin-token',
|
AUTH_TOKEN: 'admin-token',
|
||||||
@@ -19,7 +20,7 @@ describe('desktop runtime helpers', () => {
|
|||||||
port: 4312,
|
port: 4312,
|
||||||
});
|
});
|
||||||
|
|
||||||
expect(env.HOST).toBe('127.0.0.1');
|
expect(env.HOST).toBe('0.0.0.0');
|
||||||
expect(env.PORT).toBe('4312');
|
expect(env.PORT).toBe('4312');
|
||||||
expect(env.DATA_DIR).toBe('/tmp/metapi-data');
|
expect(env.DATA_DIR).toBe('/tmp/metapi-data');
|
||||||
expect(env.METAPI_LOG_DIR).toBe('/tmp/metapi-logs');
|
expect(env.METAPI_LOG_DIR).toBe('/tmp/metapi-logs');
|
||||||
@@ -31,6 +32,16 @@ describe('desktop runtime helpers', () => {
|
|||||||
expect(createDesktopServerUrl(4312)).toBe('http://127.0.0.1:4312');
|
expect(createDesktopServerUrl(4312)).toBe('http://127.0.0.1:4312');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('defaults desktop backend port to 4000', () => {
|
||||||
|
expect(resolveDesktopServerPort({})).toBe(4000);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('honors explicit desktop backend port override', () => {
|
||||||
|
expect(resolveDesktopServerPort({
|
||||||
|
METAPI_DESKTOP_SERVER_PORT: '4312',
|
||||||
|
})).toBe(4312);
|
||||||
|
});
|
||||||
|
|
||||||
it('uses resources path as backend cwd for packaged desktop builds', () => {
|
it('uses resources path as backend cwd for packaged desktop builds', () => {
|
||||||
expect(resolveDesktopServerWorkingDir({
|
expect(resolveDesktopServerWorkingDir({
|
||||||
appPath: 'C:/Users/test/AppData/Local/Programs/Metapi/resources/app.asar',
|
appPath: 'C:/Users/test/AppData/Local/Programs/Metapi/resources/app.asar',
|
||||||
|
|||||||
+10
-1
@@ -23,6 +23,7 @@ type DesktopServerWorkingDirInput = {
|
|||||||
isPackaged: boolean;
|
isPackaged: boolean;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const DEFAULT_DESKTOP_SERVER_PORT = 4000;
|
||||||
const DEFAULT_READY_TIMEOUT_MS = 30_000;
|
const DEFAULT_READY_TIMEOUT_MS = 30_000;
|
||||||
const DEFAULT_READY_INTERVAL_MS = 250;
|
const DEFAULT_READY_INTERVAL_MS = 250;
|
||||||
|
|
||||||
@@ -31,9 +32,11 @@ function delay(ms: number) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
export function buildDesktopServerEnv(input: DesktopServerEnvInput): NodeJS.ProcessEnv {
|
export function buildDesktopServerEnv(input: DesktopServerEnvInput): NodeJS.ProcessEnv {
|
||||||
|
const host = (input.inheritedEnv?.HOST || '0.0.0.0').trim() || '0.0.0.0';
|
||||||
|
|
||||||
return {
|
return {
|
||||||
...(input.inheritedEnv || {}),
|
...(input.inheritedEnv || {}),
|
||||||
HOST: '127.0.0.1',
|
HOST: host,
|
||||||
PORT: String(input.port),
|
PORT: String(input.port),
|
||||||
DATA_DIR: input.userDataDir,
|
DATA_DIR: input.userDataDir,
|
||||||
METAPI_DESKTOP: '1',
|
METAPI_DESKTOP: '1',
|
||||||
@@ -49,6 +52,12 @@ export function createDesktopHealthUrl(port: number): string {
|
|||||||
return `${createDesktopServerUrl(port)}/api/desktop/health`;
|
return `${createDesktopServerUrl(port)}/api/desktop/health`;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export function resolveDesktopServerPort(env?: NodeJS.ProcessEnv): number {
|
||||||
|
const forcedPort = Number.parseInt(env?.METAPI_DESKTOP_SERVER_PORT || '', 10);
|
||||||
|
if (Number.isFinite(forcedPort) && forcedPort > 0) return forcedPort;
|
||||||
|
return DEFAULT_DESKTOP_SERVER_PORT;
|
||||||
|
}
|
||||||
|
|
||||||
export function resolveDesktopServerWorkingDir(input: DesktopServerWorkingDirInput): string {
|
export function resolveDesktopServerWorkingDir(input: DesktopServerWorkingDirInput): string {
|
||||||
return input.isPackaged ? input.resourcesPath : input.appPath;
|
return input.isPackaged ? input.resourcesPath : input.appPath;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ describe('buildConfig', () => {
|
|||||||
expect(config.dataDir).toBe('./data');
|
expect(config.dataDir).toBe('./data');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('keeps desktop deployments bound to loopback', () => {
|
it('aligns desktop deployments with server deployments for listen host', () => {
|
||||||
const config = buildConfig({
|
const config = buildConfig({
|
||||||
HOST: '0.0.0.0',
|
HOST: '0.0.0.0',
|
||||||
METAPI_DESKTOP: '1',
|
METAPI_DESKTOP: '1',
|
||||||
@@ -19,7 +19,7 @@ describe('buildConfig', () => {
|
|||||||
DATA_DIR: '/tmp/metapi-data',
|
DATA_DIR: '/tmp/metapi-data',
|
||||||
});
|
});
|
||||||
|
|
||||||
expect(config.listenHost).toBe('127.0.0.1');
|
expect(config.listenHost).toBe('0.0.0.0');
|
||||||
expect(config.port).toBe(4312);
|
expect(config.port).toBe(4312);
|
||||||
expect(config.dataDir).toBe('/tmp/metapi-data');
|
expect(config.dataDir).toBe('/tmp/metapi-data');
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -32,11 +32,7 @@ function parseDbType(value: string | undefined): 'sqlite' | 'mysql' | 'postgres'
|
|||||||
}
|
}
|
||||||
|
|
||||||
function parseListenHost(env: NodeJS.ProcessEnv): string {
|
function parseListenHost(env: NodeJS.ProcessEnv): string {
|
||||||
const requestedHost = (env.HOST || '0.0.0.0').trim() || '0.0.0.0';
|
return (env.HOST || '0.0.0.0').trim() || '0.0.0.0';
|
||||||
if (parseBoolean(env.METAPI_DESKTOP, false)) {
|
|
||||||
return '127.0.0.1';
|
|
||||||
}
|
|
||||||
return requestedHost;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
export function buildConfig(env: NodeJS.ProcessEnv) {
|
export function buildConfig(env: NodeJS.ProcessEnv) {
|
||||||
|
|||||||
@@ -1,2 +1,4 @@
|
|||||||
ALTER TABLE `downstream_api_keys` ADD COLUMN `group_name` TEXT;
|
ALTER TABLE `downstream_api_keys` ADD COLUMN `group_name` TEXT;
|
||||||
ALTER TABLE `downstream_api_keys` ADD COLUMN `tags` TEXT;
|
ALTER TABLE `downstream_api_keys` ADD COLUMN `tags` TEXT;
|
||||||
|
ALTER TABLE `proxy_logs` ADD COLUMN `downstream_api_key_id` INT;
|
||||||
|
CREATE INDEX `proxy_logs_downstream_api_key_created_at_idx` ON `proxy_logs` (`downstream_api_key_id`, `created_at`);
|
||||||
|
|||||||
@@ -1,2 +1,4 @@
|
|||||||
ALTER TABLE "downstream_api_keys" ADD COLUMN "group_name" TEXT;
|
ALTER TABLE "downstream_api_keys" ADD COLUMN "group_name" TEXT;
|
||||||
ALTER TABLE "downstream_api_keys" ADD COLUMN "tags" TEXT;
|
ALTER TABLE "downstream_api_keys" ADD COLUMN "tags" TEXT;
|
||||||
|
ALTER TABLE "proxy_logs" ADD COLUMN "downstream_api_key_id" INTEGER;
|
||||||
|
CREATE INDEX "proxy_logs_downstream_api_key_created_at_idx" ON "proxy_logs" ("downstream_api_key_id", "created_at");
|
||||||
|
|||||||
+12
-19
@@ -8,7 +8,6 @@ import * as schema from './schema.js';
|
|||||||
import { ensureSiteSchemaCompatibility, type SiteSchemaInspector } from './siteSchemaCompatibility.js';
|
import { ensureSiteSchemaCompatibility, type SiteSchemaInspector } from './siteSchemaCompatibility.js';
|
||||||
import { ensureRouteGroupingSchemaCompatibility } from './routeGroupingSchemaCompatibility.js';
|
import { ensureRouteGroupingSchemaCompatibility } from './routeGroupingSchemaCompatibility.js';
|
||||||
import { ensureProxyFileSchemaCompatibility } from './proxyFileSchemaCompatibility.js';
|
import { ensureProxyFileSchemaCompatibility } from './proxyFileSchemaCompatibility.js';
|
||||||
import { ensureSharedIndexSchemaCompatibility } from './sharedIndexSchemaCompatibility.js';
|
|
||||||
import { executeLegacyCompat, executeLegacyCompatSync } from './legacySchemaCompat.js';
|
import { executeLegacyCompat, executeLegacyCompatSync } from './legacySchemaCompat.js';
|
||||||
import { config } from '../config.js';
|
import { config } from '../config.js';
|
||||||
import { ensureRuntimeDatabaseReady } from '../runtimeDatabaseBootstrap.js';
|
import { ensureRuntimeDatabaseReady } from '../runtimeDatabaseBootstrap.js';
|
||||||
@@ -447,12 +446,6 @@ export async function ensureProxyFileCompatibilityColumns(): Promise<void> {
|
|||||||
await ensureProxyFileSchemaCompatibility(inspector);
|
await ensureProxyFileSchemaCompatibility(inspector);
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function ensureSharedIndexCompatibility(): Promise<void> {
|
|
||||||
const inspector = createRuntimeSchemaInspector();
|
|
||||||
if (!inspector) return;
|
|
||||||
await ensureSharedIndexSchemaCompatibility(inspector);
|
|
||||||
}
|
|
||||||
|
|
||||||
function ensureRouteGroupingSchema() {
|
function ensureRouteGroupingSchema() {
|
||||||
if (!tableExists('token_routes') || !tableExists('route_channels')) {
|
if (!tableExists('token_routes') || !tableExists('route_channels')) {
|
||||||
return;
|
return;
|
||||||
@@ -557,6 +550,18 @@ function ensureProxyLogBillingDetailsSchema() {
|
|||||||
proxyLogBillingDetailsColumnAvailable = true;
|
proxyLogBillingDetailsColumnAvailable = true;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function ensureProxyLogDownstreamApiKeyIdSchema() {
|
||||||
|
if (!tableExists('proxy_logs')) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!tableColumnExists('proxy_logs', 'downstream_api_key_id')) {
|
||||||
|
execSqliteLegacyCompat('ALTER TABLE proxy_logs ADD COLUMN downstream_api_key_id integer;');
|
||||||
|
}
|
||||||
|
|
||||||
|
proxyLogDownstreamApiKeyIdColumnAvailable = true;
|
||||||
|
}
|
||||||
|
|
||||||
function normalizeSchemaErrorMessage(error: unknown): string {
|
function normalizeSchemaErrorMessage(error: unknown): string {
|
||||||
if (typeof error === 'object' && error && 'message' in error) {
|
if (typeof error === 'object' && error && 'message' in error) {
|
||||||
return String((error as { message?: unknown }).message || '');
|
return String((error as { message?: unknown }).message || '');
|
||||||
@@ -635,18 +640,6 @@ export async function ensureProxyLogBillingDetailsColumn(): Promise<boolean> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
function ensureProxyLogDownstreamApiKeyIdSchema() {
|
|
||||||
if (!tableExists('proxy_logs')) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (!tableColumnExists('proxy_logs', 'downstream_api_key_id')) {
|
|
||||||
execSqliteLegacyCompat('ALTER TABLE proxy_logs ADD COLUMN downstream_api_key_id integer;');
|
|
||||||
}
|
|
||||||
|
|
||||||
proxyLogDownstreamApiKeyIdColumnAvailable = true;
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function hasProxyLogDownstreamApiKeyIdColumn(): Promise<boolean> {
|
export async function hasProxyLogDownstreamApiKeyIdColumn(): Promise<boolean> {
|
||||||
if (proxyLogDownstreamApiKeyIdColumnAvailable !== null) {
|
if (proxyLogDownstreamApiKeyIdColumnAvailable !== null) {
|
||||||
return proxyLogDownstreamApiKeyIdColumnAvailable;
|
return proxyLogDownstreamApiKeyIdColumnAvailable;
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ import baselineContract from './generated/fixtures/2026-03-14-baseline.schemaCon
|
|||||||
import currentContract from './generated/schemaContract.json' with { type: 'json' };
|
import currentContract from './generated/schemaContract.json' with { type: 'json' };
|
||||||
import { classifyLegacyCompatMutation } from './legacySchemaCompat.js';
|
import { classifyLegacyCompatMutation } from './legacySchemaCompat.js';
|
||||||
import { generateUpgradeSql } from './schemaArtifactGenerator.js';
|
import { generateUpgradeSql } from './schemaArtifactGenerator.js';
|
||||||
|
import type { SchemaContract, SchemaContractColumn } from './schemaContract.js';
|
||||||
import { describe, expect, it } from 'vitest';
|
import { describe, expect, it } from 'vitest';
|
||||||
import {
|
import {
|
||||||
__runtimeSchemaBootstrapTestUtils,
|
__runtimeSchemaBootstrapTestUtils,
|
||||||
@@ -17,6 +18,9 @@ function createStubClient(dialect: RuntimeSchemaDialect, executedSql: string[]):
|
|||||||
commit: async () => {},
|
commit: async () => {},
|
||||||
rollback: async () => {},
|
rollback: async () => {},
|
||||||
execute: async (sqlText: string) => {
|
execute: async (sqlText: string) => {
|
||||||
|
if (sqlText.trim().toLowerCase().startsWith('select')) {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
executedSql.push(sqlText);
|
executedSql.push(sqlText);
|
||||||
return [];
|
return [];
|
||||||
},
|
},
|
||||||
@@ -33,6 +37,16 @@ function createStubClient(dialect: RuntimeSchemaDialect, executedSql: string[]):
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function makeColumn(overrides: Partial<SchemaContractColumn> = {}): SchemaContractColumn {
|
||||||
|
return {
|
||||||
|
logicalType: 'text',
|
||||||
|
notNull: false,
|
||||||
|
defaultValue: null,
|
||||||
|
primaryKey: false,
|
||||||
|
...overrides,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
describe('runtime schema bootstrap', () => {
|
describe('runtime schema bootstrap', () => {
|
||||||
it.each(['mysql', 'postgres'] as const)('executes live-schema upgrade statements for %s', async (dialect) => {
|
it.each(['mysql', 'postgres'] as const)('executes live-schema upgrade statements for %s', async (dialect) => {
|
||||||
const executedSql: string[] = [];
|
const executedSql: string[] = [];
|
||||||
@@ -149,4 +163,72 @@ describe('runtime schema bootstrap', () => {
|
|||||||
|
|
||||||
expect(executedSql).toContain(targetSql);
|
expect(executedSql).toContain(targetSql);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('adds mysql text prefixes for new indexes when live datetime-like columns are still stored as text', async () => {
|
||||||
|
const executedSql: string[] = [];
|
||||||
|
const minimalContract: SchemaContract = {
|
||||||
|
tables: {
|
||||||
|
proxy_logs: {
|
||||||
|
columns: {
|
||||||
|
downstream_api_key_id: makeColumn({ logicalType: 'integer' }),
|
||||||
|
created_at: makeColumn({ logicalType: 'datetime', defaultValue: "datetime('now')" }),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
indexes: [
|
||||||
|
{
|
||||||
|
name: 'proxy_logs_downstream_api_key_created_at_idx',
|
||||||
|
table: 'proxy_logs',
|
||||||
|
columns: ['downstream_api_key_id', 'created_at'],
|
||||||
|
unique: false,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
uniques: [],
|
||||||
|
foreignKeys: [],
|
||||||
|
};
|
||||||
|
|
||||||
|
await ensureRuntimeDatabaseSchema({
|
||||||
|
...createStubClient('mysql', executedSql),
|
||||||
|
execute: async (sqlText: string) => {
|
||||||
|
if (sqlText.includes('FROM information_schema.columns')) {
|
||||||
|
return [[
|
||||||
|
{
|
||||||
|
table_name: 'proxy_logs',
|
||||||
|
column_name: 'downstream_api_key_id',
|
||||||
|
data_type: 'int',
|
||||||
|
column_type: 'int',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
table_name: 'proxy_logs',
|
||||||
|
column_name: 'created_at',
|
||||||
|
data_type: 'text',
|
||||||
|
column_type: 'text',
|
||||||
|
},
|
||||||
|
]];
|
||||||
|
}
|
||||||
|
|
||||||
|
executedSql.push(sqlText);
|
||||||
|
return [];
|
||||||
|
},
|
||||||
|
queryScalar: async (sqlText: string, params: unknown[] = []) => {
|
||||||
|
if (sqlText.includes('information_schema.tables')) {
|
||||||
|
return params[0] === 'proxy_logs' ? 1 : 0;
|
||||||
|
}
|
||||||
|
if (sqlText.includes('information_schema.columns')) {
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
},
|
||||||
|
}, {
|
||||||
|
currentContract: minimalContract,
|
||||||
|
liveContract: {
|
||||||
|
...minimalContract,
|
||||||
|
indexes: [],
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(executedSql).toContain(
|
||||||
|
'CREATE INDEX `proxy_logs_downstream_api_key_created_at_idx` ON `proxy_logs` (`downstream_api_key_id`, `created_at`(191))',
|
||||||
|
);
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -7,7 +7,11 @@ import {
|
|||||||
ensureLegacySchemaCompatibility,
|
ensureLegacySchemaCompatibility,
|
||||||
type LegacySchemaCompatInspector,
|
type LegacySchemaCompatInspector,
|
||||||
} from './legacySchemaCompat.js';
|
} from './legacySchemaCompat.js';
|
||||||
import { generateBootstrapSql, generateUpgradeSql } from './schemaArtifactGenerator.js';
|
import {
|
||||||
|
generateBootstrapSql,
|
||||||
|
generateUpgradeSql,
|
||||||
|
type MysqlIndexPrefixRequirementMap,
|
||||||
|
} from './schemaArtifactGenerator.js';
|
||||||
import { introspectLiveSchema } from './schemaIntrospection.js';
|
import { introspectLiveSchema } from './schemaIntrospection.js';
|
||||||
import { resolveGeneratedSchemaContractPath, type SchemaContract } from './schemaContract.js';
|
import { resolveGeneratedSchemaContractPath, type SchemaContract } from './schemaContract.js';
|
||||||
|
|
||||||
@@ -233,6 +237,92 @@ function buildCompatibleRuntimeBaseline(
|
|||||||
return baseline;
|
return baseline;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function collectIndexedColumns(contract: SchemaContract): Map<string, Set<string>> {
|
||||||
|
const indexedColumns = new Map<string, Set<string>>();
|
||||||
|
|
||||||
|
for (const index of [...contract.indexes, ...contract.uniques]) {
|
||||||
|
let columns = indexedColumns.get(index.table);
|
||||||
|
if (!columns) {
|
||||||
|
columns = new Set<string>();
|
||||||
|
indexedColumns.set(index.table, columns);
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const columnName of index.columns) {
|
||||||
|
columns.add(columnName);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return indexedColumns;
|
||||||
|
}
|
||||||
|
|
||||||
|
function requiresMysqlIndexPrefixForColumnType(columnType: string): boolean {
|
||||||
|
const normalizedType = columnType.trim().toLowerCase();
|
||||||
|
return normalizedType.includes('text') || normalizedType.includes('blob');
|
||||||
|
}
|
||||||
|
|
||||||
|
async function queryRuntimeRows(
|
||||||
|
client: RuntimeSchemaClient,
|
||||||
|
sqlText: string,
|
||||||
|
params: unknown[] = [],
|
||||||
|
): Promise<Array<Record<string, unknown>>> {
|
||||||
|
const result = await client.execute(sqlText, params);
|
||||||
|
|
||||||
|
if (!Array.isArray(result)) {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
|
||||||
|
const [first] = result;
|
||||||
|
if (Array.isArray(first)) {
|
||||||
|
return first as Array<Record<string, unknown>>;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (result.every((item) => typeof item === 'object' && item !== null && !Array.isArray(item))) {
|
||||||
|
return result as Array<Record<string, unknown>>;
|
||||||
|
}
|
||||||
|
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
|
||||||
|
async function resolveMySqlIndexPrefixRequirements(
|
||||||
|
client: RuntimeSchemaClient,
|
||||||
|
currentContract: SchemaContract,
|
||||||
|
): Promise<MysqlIndexPrefixRequirementMap> {
|
||||||
|
const indexedColumns = collectIndexedColumns(currentContract);
|
||||||
|
if (indexedColumns.size === 0) {
|
||||||
|
return {};
|
||||||
|
}
|
||||||
|
|
||||||
|
const rows = await queryRuntimeRows(client, `
|
||||||
|
SELECT
|
||||||
|
table_name AS table_name,
|
||||||
|
column_name AS column_name,
|
||||||
|
data_type AS data_type,
|
||||||
|
column_type AS column_type
|
||||||
|
FROM information_schema.columns
|
||||||
|
WHERE table_schema = DATABASE()
|
||||||
|
`);
|
||||||
|
|
||||||
|
const requirements: MysqlIndexPrefixRequirementMap = {};
|
||||||
|
for (const row of rows) {
|
||||||
|
const tableName = String(row.table_name || '');
|
||||||
|
const columnName = String(row.column_name || '');
|
||||||
|
const trackedColumns = indexedColumns.get(tableName);
|
||||||
|
if (!trackedColumns || !trackedColumns.has(columnName)) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
const declaredType = String(row.column_type || row.data_type || '');
|
||||||
|
if (!requiresMysqlIndexPrefixForColumnType(declaredType)) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
requirements[tableName] ??= {};
|
||||||
|
requirements[tableName][columnName] = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
return requirements;
|
||||||
|
}
|
||||||
|
|
||||||
async function createPostgresClient(connectionString: string, ssl: boolean): Promise<RuntimeSchemaClient> {
|
async function createPostgresClient(connectionString: string, ssl: boolean): Promise<RuntimeSchemaClient> {
|
||||||
const clientOptions: pg.ClientConfig = { connectionString };
|
const clientOptions: pg.ClientConfig = { connectionString };
|
||||||
if (ssl) {
|
if (ssl) {
|
||||||
@@ -346,9 +436,12 @@ function buildExternalUpgradeStatements(
|
|||||||
dialect: Exclude<RuntimeSchemaDialect, 'sqlite'>,
|
dialect: Exclude<RuntimeSchemaDialect, 'sqlite'>,
|
||||||
currentContract: SchemaContract,
|
currentContract: SchemaContract,
|
||||||
liveContract: SchemaContract,
|
liveContract: SchemaContract,
|
||||||
|
mysqlIndexPrefixRequirements?: MysqlIndexPrefixRequirementMap,
|
||||||
): string[] {
|
): string[] {
|
||||||
const compatibleBaseline = buildCompatibleRuntimeBaseline(currentContract, liveContract);
|
const compatibleBaseline = buildCompatibleRuntimeBaseline(currentContract, liveContract);
|
||||||
return splitSqlStatements(generateUpgradeSql(dialect, currentContract, compatibleBaseline));
|
return splitSqlStatements(generateUpgradeSql(dialect, currentContract, compatibleBaseline, {
|
||||||
|
mysqlIndexPrefixRequirements,
|
||||||
|
}));
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function ensureRuntimeDatabaseSchema(
|
export async function ensureRuntimeDatabaseSchema(
|
||||||
@@ -356,13 +449,23 @@ export async function ensureRuntimeDatabaseSchema(
|
|||||||
options: EnsureRuntimeDatabaseSchemaOptions = {},
|
options: EnsureRuntimeDatabaseSchemaOptions = {},
|
||||||
): Promise<void> {
|
): Promise<void> {
|
||||||
const currentContract = options.currentContract ?? readSchemaContract();
|
const currentContract = options.currentContract ?? readSchemaContract();
|
||||||
const statements = client.dialect === 'sqlite'
|
let statements: string[];
|
||||||
? splitSqlStatements(generateBootstrapSql('sqlite', currentContract))
|
|
||||||
: buildExternalUpgradeStatements(
|
if (client.dialect === 'sqlite') {
|
||||||
|
statements = splitSqlStatements(generateBootstrapSql('sqlite', currentContract));
|
||||||
|
} else {
|
||||||
|
const liveContract = await resolveLiveContract(client, options.liveContract);
|
||||||
|
const mysqlIndexPrefixRequirements = client.dialect === 'mysql'
|
||||||
|
? await resolveMySqlIndexPrefixRequirements(client, currentContract)
|
||||||
|
: undefined;
|
||||||
|
|
||||||
|
statements = buildExternalUpgradeStatements(
|
||||||
client.dialect,
|
client.dialect,
|
||||||
currentContract,
|
currentContract,
|
||||||
await resolveLiveContract(client, options.liveContract),
|
liveContract,
|
||||||
|
mysqlIndexPrefixRequirements,
|
||||||
);
|
);
|
||||||
|
}
|
||||||
|
|
||||||
for (const sqlText of statements) {
|
for (const sqlText of statements) {
|
||||||
await executeBootstrapStatement(client, sqlText);
|
await executeBootstrapStatement(client, sqlText);
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ import { readFileSync } from 'node:fs';
|
|||||||
import { dirname, resolve } from 'node:path';
|
import { dirname, resolve } from 'node:path';
|
||||||
import { fileURLToPath } from 'node:url';
|
import { fileURLToPath } from 'node:url';
|
||||||
import { describe, expect, it } from 'vitest';
|
import { describe, expect, it } from 'vitest';
|
||||||
import { generateDialectArtifacts, type GeneratedDialectArtifacts } from './schemaArtifactGenerator.js';
|
import { generateDialectArtifacts, generateUpgradeSql, type GeneratedDialectArtifacts } from './schemaArtifactGenerator.js';
|
||||||
import type { SchemaContract, SchemaContractColumn } from './schemaContract.js';
|
import type { SchemaContract, SchemaContractColumn } from './schemaContract.js';
|
||||||
|
|
||||||
const dbDir = dirname(fileURLToPath(import.meta.url));
|
const dbDir = dirname(fileURLToPath(import.meta.url));
|
||||||
@@ -123,6 +123,45 @@ describe('schema artifact generator', () => {
|
|||||||
expect(artifacts.mysqlBootstrap).not.toContain('`read`(191)');
|
expect(artifacts.mysqlBootstrap).not.toContain('`read`(191)');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('allows mysql upgrade generation to force prefix lengths for live text-backed columns', () => {
|
||||||
|
const previousContract: SchemaContract = {
|
||||||
|
tables: {
|
||||||
|
proxy_logs: {
|
||||||
|
columns: {
|
||||||
|
downstream_api_key_id: makeColumn({ logicalType: 'integer' }),
|
||||||
|
created_at: makeColumn({ logicalType: 'datetime', defaultValue: "datetime('now')" }),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
indexes: [],
|
||||||
|
uniques: [],
|
||||||
|
foreignKeys: [],
|
||||||
|
};
|
||||||
|
const currentContract: SchemaContract = {
|
||||||
|
...previousContract,
|
||||||
|
indexes: [
|
||||||
|
{
|
||||||
|
name: 'proxy_logs_downstream_api_key_created_at_idx',
|
||||||
|
table: 'proxy_logs',
|
||||||
|
columns: ['downstream_api_key_id', 'created_at'],
|
||||||
|
unique: false,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
const mysqlUpgrade = generateUpgradeSql('mysql', currentContract, previousContract, {
|
||||||
|
mysqlIndexPrefixRequirements: {
|
||||||
|
proxy_logs: {
|
||||||
|
created_at: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(mysqlUpgrade).toContain(
|
||||||
|
'CREATE INDEX `proxy_logs_downstream_api_key_created_at_idx` ON `proxy_logs` (`downstream_api_key_id`, `created_at`(191))',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
it('rejects destructive diffs when generating additive upgrades', () => {
|
it('rejects destructive diffs when generating additive upgrades', () => {
|
||||||
const current = readSchemaContract();
|
const current = readSchemaContract();
|
||||||
const previous = structuredClone(current);
|
const previous = structuredClone(current);
|
||||||
|
|||||||
@@ -17,8 +17,13 @@ export interface GeneratedDialectArtifacts {
|
|||||||
postgresUpgrade: string;
|
postgresUpgrade: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export type MysqlIndexPrefixRequirementMap = Record<string, Record<string, boolean>>;
|
||||||
|
|
||||||
type Dialect = 'mysql' | 'postgres';
|
type Dialect = 'mysql' | 'postgres';
|
||||||
type SqlDialect = 'sqlite' | Dialect;
|
type SqlDialect = 'sqlite' | Dialect;
|
||||||
|
type SqlGenerationOptions = {
|
||||||
|
mysqlIndexPrefixRequirements?: MysqlIndexPrefixRequirementMap;
|
||||||
|
};
|
||||||
|
|
||||||
function resolveDbDir(): string {
|
function resolveDbDir(): string {
|
||||||
return dirname(fileURLToPath(import.meta.url));
|
return dirname(fileURLToPath(import.meta.url));
|
||||||
@@ -36,6 +41,21 @@ function escapeMysqlTextPrefix(columnType: LogicalColumnType): string {
|
|||||||
return columnType === 'text' ? '(191)' : '';
|
return columnType === 'text' ? '(191)' : '';
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function resolveMysqlIndexPrefix(
|
||||||
|
tableName: string,
|
||||||
|
columnName: string,
|
||||||
|
contract: SchemaContract,
|
||||||
|
options?: SqlGenerationOptions,
|
||||||
|
): string {
|
||||||
|
const override = options?.mysqlIndexPrefixRequirements?.[tableName]?.[columnName];
|
||||||
|
if (override !== undefined) {
|
||||||
|
return override ? '(191)' : '';
|
||||||
|
}
|
||||||
|
|
||||||
|
const column = contract.tables[tableName]?.columns[columnName];
|
||||||
|
return column ? escapeMysqlTextPrefix(column.logicalType) : '';
|
||||||
|
}
|
||||||
|
|
||||||
function mapColumnType(dialect: SqlDialect, columnName: string, column: SchemaContractColumn): string {
|
function mapColumnType(dialect: SqlDialect, columnName: string, column: SchemaContractColumn): string {
|
||||||
if (dialect === 'sqlite') {
|
if (dialect === 'sqlite') {
|
||||||
switch (column.logicalType) {
|
switch (column.logicalType) {
|
||||||
@@ -177,22 +197,34 @@ function buildCreateTableStatement(
|
|||||||
return `CREATE TABLE IF NOT EXISTS ${quoteIdentifier(dialect, tableName)} (${parts.join(', ')})`;
|
return `CREATE TABLE IF NOT EXISTS ${quoteIdentifier(dialect, tableName)} (${parts.join(', ')})`;
|
||||||
}
|
}
|
||||||
|
|
||||||
function buildUniqueIndexStatement(dialect: SqlDialect, uniqueIndex: SchemaContractUnique, contract: SchemaContract): string {
|
function buildUniqueIndexStatement(
|
||||||
|
dialect: SqlDialect,
|
||||||
|
uniqueIndex: SchemaContractUnique,
|
||||||
|
contract: SchemaContract,
|
||||||
|
options?: SqlGenerationOptions,
|
||||||
|
): string {
|
||||||
const columns = uniqueIndex.columns
|
const columns = uniqueIndex.columns
|
||||||
.map((columnName) => {
|
.map((columnName) => {
|
||||||
const column = contract.tables[uniqueIndex.table]?.columns[columnName];
|
const suffix = dialect === 'mysql'
|
||||||
const suffix = dialect === 'mysql' && column ? escapeMysqlTextPrefix(column.logicalType) : '';
|
? resolveMysqlIndexPrefix(uniqueIndex.table, columnName, contract, options)
|
||||||
|
: '';
|
||||||
return `${quoteIdentifier(dialect, columnName)}${suffix}`;
|
return `${quoteIdentifier(dialect, columnName)}${suffix}`;
|
||||||
})
|
})
|
||||||
.join(', ');
|
.join(', ');
|
||||||
return `CREATE UNIQUE INDEX ${quoteIdentifier(dialect, uniqueIndex.name)} ON ${quoteIdentifier(dialect, uniqueIndex.table)} (${columns})`;
|
return `CREATE UNIQUE INDEX ${quoteIdentifier(dialect, uniqueIndex.name)} ON ${quoteIdentifier(dialect, uniqueIndex.table)} (${columns})`;
|
||||||
}
|
}
|
||||||
|
|
||||||
function buildIndexStatement(dialect: SqlDialect, index: SchemaContractIndex, contract: SchemaContract): string {
|
function buildIndexStatement(
|
||||||
|
dialect: SqlDialect,
|
||||||
|
index: SchemaContractIndex,
|
||||||
|
contract: SchemaContract,
|
||||||
|
options?: SqlGenerationOptions,
|
||||||
|
): string {
|
||||||
const columns = index.columns
|
const columns = index.columns
|
||||||
.map((columnName) => {
|
.map((columnName) => {
|
||||||
const column = contract.tables[index.table]?.columns[columnName];
|
const suffix = dialect === 'mysql'
|
||||||
const suffix = dialect === 'mysql' && column ? escapeMysqlTextPrefix(column.logicalType) : '';
|
? resolveMysqlIndexPrefix(index.table, columnName, contract, options)
|
||||||
|
: '';
|
||||||
return `${quoteIdentifier(dialect, columnName)}${suffix}`;
|
return `${quoteIdentifier(dialect, columnName)}${suffix}`;
|
||||||
})
|
})
|
||||||
.join(', ');
|
.join(', ');
|
||||||
@@ -317,6 +349,7 @@ export function generateUpgradeSql(
|
|||||||
dialect: SqlDialect,
|
dialect: SqlDialect,
|
||||||
currentContract: SchemaContract,
|
currentContract: SchemaContract,
|
||||||
previousContract?: SchemaContract | null,
|
previousContract?: SchemaContract | null,
|
||||||
|
options?: SqlGenerationOptions,
|
||||||
): string {
|
): string {
|
||||||
if (!previousContract) {
|
if (!previousContract) {
|
||||||
return `-- no previous schema contract available for ${dialect} additive upgrade generation\n`;
|
return `-- no previous schema contract available for ${dialect} additive upgrade generation\n`;
|
||||||
@@ -357,7 +390,7 @@ export function generateUpgradeSql(
|
|||||||
.filter((unique) => !previousUniqueNames.has(unique.name))
|
.filter((unique) => !previousUniqueNames.has(unique.name))
|
||||||
.slice()
|
.slice()
|
||||||
.sort((left, right) => left.name.localeCompare(right.name, 'en'))
|
.sort((left, right) => left.name.localeCompare(right.name, 'en'))
|
||||||
.map((unique) => buildUniqueIndexStatement(dialect, unique, currentContract));
|
.map((unique) => buildUniqueIndexStatement(dialect, unique, currentContract, options));
|
||||||
|
|
||||||
const currentUniqueNames = new Set(currentContract.uniques.map((unique) => unique.name));
|
const currentUniqueNames = new Set(currentContract.uniques.map((unique) => unique.name));
|
||||||
const previousIndexNames = new Set(previousContract.indexes.map((index) => index.name));
|
const previousIndexNames = new Set(previousContract.indexes.map((index) => index.name));
|
||||||
@@ -366,7 +399,7 @@ export function generateUpgradeSql(
|
|||||||
.filter((index) => !previousIndexNames.has(index.name))
|
.filter((index) => !previousIndexNames.has(index.name))
|
||||||
.slice()
|
.slice()
|
||||||
.sort((left, right) => left.name.localeCompare(right.name, 'en'))
|
.sort((left, right) => left.name.localeCompare(right.name, 'en'))
|
||||||
.map((index) => buildIndexStatement(dialect, index, currentContract));
|
.map((index) => buildIndexStatement(dialect, index, currentContract, options));
|
||||||
|
|
||||||
const statements = [...addedTableStatements, ...addColumnStatements, ...uniqueStatements, ...indexStatements];
|
const statements = [...addedTableStatements, ...addColumnStatements, ...uniqueStatements, ...indexStatements];
|
||||||
if (statements.length === 0) {
|
if (statements.length === 0) {
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ import { dirname, resolve } from 'node:path';
|
|||||||
import { fileURLToPath } from 'node:url';
|
import { fileURLToPath } from 'node:url';
|
||||||
import { describe, expect, it } from 'vitest';
|
import { describe, expect, it } from 'vitest';
|
||||||
import type { SchemaContract } from './schemaContract.js';
|
import type { SchemaContract } from './schemaContract.js';
|
||||||
|
import { SHARED_INDEX_COMPATIBILITY_SPECS } from './sharedIndexSchemaCompatibility.js';
|
||||||
|
|
||||||
const dbDir = dirname(fileURLToPath(import.meta.url));
|
const dbDir = dirname(fileURLToPath(import.meta.url));
|
||||||
const generatedDir = resolve(dbDir, 'generated');
|
const generatedDir = resolve(dbDir, 'generated');
|
||||||
@@ -63,4 +64,31 @@ describe('database schema parity', () => {
|
|||||||
expect(unknownTables).toEqual([]);
|
expect(unknownTables).toEqual([]);
|
||||||
expect(unknownIndexes).toEqual([]);
|
expect(unknownIndexes).toEqual([]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('does not duplicate contract-defined indexes inside shared index compatibility specs', () => {
|
||||||
|
const contract = JSON.parse(readFileSync(schemaContractPath, 'utf8')) as SchemaContract;
|
||||||
|
const contractIndexNames = new Set([
|
||||||
|
...contract.indexes.map((index) => index.name),
|
||||||
|
...contract.uniques.map((unique) => unique.name),
|
||||||
|
]);
|
||||||
|
|
||||||
|
const duplicatedSpecs = SHARED_INDEX_COMPATIBILITY_SPECS
|
||||||
|
.map((spec) => spec.indexName)
|
||||||
|
.filter((indexName) => contractIndexNames.has(indexName));
|
||||||
|
|
||||||
|
expect(duplicatedSpecs).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps proxy_logs downstream api key schema in the generated contract artifacts', () => {
|
||||||
|
const contract = JSON.parse(readFileSync(schemaContractPath, 'utf8')) as SchemaContract;
|
||||||
|
const mysqlBootstrap = readFileSync(resolve(generatedDir, 'mysql.bootstrap.sql'), 'utf8');
|
||||||
|
const postgresBootstrap = readFileSync(resolve(generatedDir, 'postgres.bootstrap.sql'), 'utf8');
|
||||||
|
|
||||||
|
expect(contract.tables.proxy_logs?.columns.downstream_api_key_id?.logicalType).toBe('integer');
|
||||||
|
expect(contract.indexes.some((index) => index.name === 'proxy_logs_downstream_api_key_created_at_idx')).toBe(true);
|
||||||
|
expect(mysqlBootstrap).toContain('`downstream_api_key_id`');
|
||||||
|
expect(mysqlBootstrap).toContain('`proxy_logs_downstream_api_key_created_at_idx`');
|
||||||
|
expect(postgresBootstrap).toContain('"downstream_api_key_id"');
|
||||||
|
expect(postgresBootstrap).toContain('"proxy_logs_downstream_api_key_created_at_idx"');
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -25,33 +25,10 @@ function createInspector(
|
|||||||
|
|
||||||
describe('ensureSharedIndexSchemaCompatibility', () => {
|
describe('ensureSharedIndexSchemaCompatibility', () => {
|
||||||
it.each([
|
it.each([
|
||||||
{
|
'sqlite',
|
||||||
dialect: 'sqlite' as const,
|
'mysql',
|
||||||
expectedSql: [
|
'postgres',
|
||||||
'CREATE UNIQUE INDEX IF NOT EXISTS model_availability_account_model_unique ON model_availability (account_id, model_name);',
|
] as const)('is a no-op for %s now that contract-defined indexes come from runtime bootstrap', async (dialect) => {
|
||||||
'CREATE INDEX IF NOT EXISTS proxy_logs_status_created_at_idx ON proxy_logs (status, created_at);',
|
|
||||||
'CREATE INDEX IF NOT EXISTS events_created_at_idx ON events (created_at);',
|
|
||||||
],
|
|
||||||
},
|
|
||||||
{
|
|
||||||
dialect: 'mysql' as const,
|
|
||||||
expectedSql: [
|
|
||||||
'CREATE UNIQUE INDEX `model_availability_account_model_unique` ON `model_availability` (`account_id`, `model_name`(191))',
|
|
||||||
'CREATE INDEX `proxy_logs_status_created_at_idx` ON `proxy_logs` (`status`, `created_at`(191))',
|
|
||||||
'CREATE INDEX `proxy_logs_downstream_api_key_created_at_idx` ON `proxy_logs` (`downstream_api_key_id`, `created_at`(191))',
|
|
||||||
'CREATE INDEX `events_created_at_idx` ON `events` (`created_at`(191))',
|
|
||||||
],
|
|
||||||
},
|
|
||||||
{
|
|
||||||
dialect: 'postgres' as const,
|
|
||||||
expectedSql: [
|
|
||||||
'CREATE UNIQUE INDEX IF NOT EXISTS "model_availability_account_model_unique" ON "model_availability" ("account_id", "model_name")',
|
|
||||||
'CREATE INDEX IF NOT EXISTS "proxy_logs_status_created_at_idx" ON "proxy_logs" ("status", "created_at")',
|
|
||||||
'CREATE INDEX IF NOT EXISTS "proxy_logs_downstream_api_key_created_at_idx" ON "proxy_logs" ("downstream_api_key_id", "created_at")',
|
|
||||||
'CREATE INDEX IF NOT EXISTS "events_created_at_idx" ON "events" ("created_at")',
|
|
||||||
],
|
|
||||||
},
|
|
||||||
])('creates shared indexes for $dialect', async ({ dialect, expectedSql }) => {
|
|
||||||
const { inspector, executedSql } = createInspector(dialect, {
|
const { inspector, executedSql } = createInspector(dialect, {
|
||||||
existingTables: [
|
existingTables: [
|
||||||
'sites',
|
'sites',
|
||||||
@@ -70,10 +47,7 @@ describe('ensureSharedIndexSchemaCompatibility', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
await ensureSharedIndexSchemaCompatibility(inspector);
|
await ensureSharedIndexSchemaCompatibility(inspector);
|
||||||
|
expect(executedSql).toEqual([]);
|
||||||
for (const sqlText of expectedSql) {
|
|
||||||
expect(executedSql).toContain(sqlText);
|
|
||||||
}
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it('skips indexes for tables that do not exist', async () => {
|
it('skips indexes for tables that do not exist', async () => {
|
||||||
@@ -83,8 +57,6 @@ describe('ensureSharedIndexSchemaCompatibility', () => {
|
|||||||
|
|
||||||
await ensureSharedIndexSchemaCompatibility(inspector);
|
await ensureSharedIndexSchemaCompatibility(inspector);
|
||||||
|
|
||||||
expect(executedSql).toEqual([
|
expect(executedSql).toEqual([]);
|
||||||
'CREATE INDEX IF NOT EXISTS "sites_status_idx" ON "sites" ("status")',
|
|
||||||
]);
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -12,341 +12,10 @@ type SharedIndexCompatibilitySpec = {
|
|||||||
createSql: Record<SharedIndexSchemaDialect, string>;
|
createSql: Record<SharedIndexSchemaDialect, string>;
|
||||||
};
|
};
|
||||||
|
|
||||||
export const SHARED_INDEX_COMPATIBILITY_SPECS: SharedIndexCompatibilitySpec[] = [
|
// Contract-defined indexes are owned by generated schema artifacts and
|
||||||
{
|
// runtimeSchemaBootstrap. This legacy hook is intentionally empty so startup
|
||||||
table: 'sites',
|
// does not maintain a second static source of index SQL.
|
||||||
indexName: 'sites_status_idx',
|
export const SHARED_INDEX_COMPATIBILITY_SPECS: SharedIndexCompatibilitySpec[] = [];
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS sites_status_idx ON sites (status);',
|
|
||||||
mysql: 'CREATE INDEX `sites_status_idx` ON `sites` (`status`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "sites_status_idx" ON "sites" ("status")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'accounts',
|
|
||||||
indexName: 'accounts_site_id_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS accounts_site_id_idx ON accounts (site_id);',
|
|
||||||
mysql: 'CREATE INDEX `accounts_site_id_idx` ON `accounts` (`site_id`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "accounts_site_id_idx" ON "accounts" ("site_id")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'accounts',
|
|
||||||
indexName: 'accounts_status_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS accounts_status_idx ON accounts (status);',
|
|
||||||
mysql: 'CREATE INDEX `accounts_status_idx` ON `accounts` (`status`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "accounts_status_idx" ON "accounts" ("status")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'accounts',
|
|
||||||
indexName: 'accounts_site_status_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS accounts_site_status_idx ON accounts (site_id, status);',
|
|
||||||
mysql: 'CREATE INDEX `accounts_site_status_idx` ON `accounts` (`site_id`, `status`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "accounts_site_status_idx" ON "accounts" ("site_id", "status")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'account_tokens',
|
|
||||||
indexName: 'account_tokens_account_id_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS account_tokens_account_id_idx ON account_tokens (account_id);',
|
|
||||||
mysql: 'CREATE INDEX `account_tokens_account_id_idx` ON `account_tokens` (`account_id`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "account_tokens_account_id_idx" ON "account_tokens" ("account_id")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'account_tokens',
|
|
||||||
indexName: 'account_tokens_account_enabled_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS account_tokens_account_enabled_idx ON account_tokens (account_id, enabled);',
|
|
||||||
mysql: 'CREATE INDEX `account_tokens_account_enabled_idx` ON `account_tokens` (`account_id`, `enabled`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "account_tokens_account_enabled_idx" ON "account_tokens" ("account_id", "enabled")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'account_tokens',
|
|
||||||
indexName: 'account_tokens_enabled_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS account_tokens_enabled_idx ON account_tokens (enabled);',
|
|
||||||
mysql: 'CREATE INDEX `account_tokens_enabled_idx` ON `account_tokens` (`enabled`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "account_tokens_enabled_idx" ON "account_tokens" ("enabled")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'checkin_logs',
|
|
||||||
indexName: 'checkin_logs_account_created_at_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS checkin_logs_account_created_at_idx ON checkin_logs (account_id, created_at);',
|
|
||||||
mysql: 'CREATE INDEX `checkin_logs_account_created_at_idx` ON `checkin_logs` (`account_id`, `created_at`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "checkin_logs_account_created_at_idx" ON "checkin_logs" ("account_id", "created_at")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'checkin_logs',
|
|
||||||
indexName: 'checkin_logs_created_at_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS checkin_logs_created_at_idx ON checkin_logs (created_at);',
|
|
||||||
mysql: 'CREATE INDEX `checkin_logs_created_at_idx` ON `checkin_logs` (`created_at`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "checkin_logs_created_at_idx" ON "checkin_logs" ("created_at")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'checkin_logs',
|
|
||||||
indexName: 'checkin_logs_status_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS checkin_logs_status_idx ON checkin_logs (status);',
|
|
||||||
mysql: 'CREATE INDEX `checkin_logs_status_idx` ON `checkin_logs` (`status`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "checkin_logs_status_idx" ON "checkin_logs" ("status")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'model_availability',
|
|
||||||
indexName: 'model_availability_account_model_unique',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE UNIQUE INDEX IF NOT EXISTS model_availability_account_model_unique ON model_availability (account_id, model_name);',
|
|
||||||
mysql: 'CREATE UNIQUE INDEX `model_availability_account_model_unique` ON `model_availability` (`account_id`, `model_name`(191))',
|
|
||||||
postgres: 'CREATE UNIQUE INDEX IF NOT EXISTS "model_availability_account_model_unique" ON "model_availability" ("account_id", "model_name")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'model_availability',
|
|
||||||
indexName: 'model_availability_account_available_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS model_availability_account_available_idx ON model_availability (account_id, available);',
|
|
||||||
mysql: 'CREATE INDEX `model_availability_account_available_idx` ON `model_availability` (`account_id`, `available`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "model_availability_account_available_idx" ON "model_availability" ("account_id", "available")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'model_availability',
|
|
||||||
indexName: 'model_availability_model_name_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS model_availability_model_name_idx ON model_availability (model_name);',
|
|
||||||
mysql: 'CREATE INDEX `model_availability_model_name_idx` ON `model_availability` (`model_name`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "model_availability_model_name_idx" ON "model_availability" ("model_name")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'token_model_availability',
|
|
||||||
indexName: 'token_model_availability_token_model_unique',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE UNIQUE INDEX IF NOT EXISTS token_model_availability_token_model_unique ON token_model_availability (token_id, model_name);',
|
|
||||||
mysql: 'CREATE UNIQUE INDEX `token_model_availability_token_model_unique` ON `token_model_availability` (`token_id`, `model_name`(191))',
|
|
||||||
postgres: 'CREATE UNIQUE INDEX IF NOT EXISTS "token_model_availability_token_model_unique" ON "token_model_availability" ("token_id", "model_name")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'token_model_availability',
|
|
||||||
indexName: 'token_model_availability_token_available_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS token_model_availability_token_available_idx ON token_model_availability (token_id, available);',
|
|
||||||
mysql: 'CREATE INDEX `token_model_availability_token_available_idx` ON `token_model_availability` (`token_id`, `available`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "token_model_availability_token_available_idx" ON "token_model_availability" ("token_id", "available")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'token_model_availability',
|
|
||||||
indexName: 'token_model_availability_model_name_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS token_model_availability_model_name_idx ON token_model_availability (model_name);',
|
|
||||||
mysql: 'CREATE INDEX `token_model_availability_model_name_idx` ON `token_model_availability` (`model_name`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "token_model_availability_model_name_idx" ON "token_model_availability" ("model_name")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'token_model_availability',
|
|
||||||
indexName: 'token_model_availability_available_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS token_model_availability_available_idx ON token_model_availability (available);',
|
|
||||||
mysql: 'CREATE INDEX `token_model_availability_available_idx` ON `token_model_availability` (`available`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "token_model_availability_available_idx" ON "token_model_availability" ("available")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'token_routes',
|
|
||||||
indexName: 'token_routes_model_pattern_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS token_routes_model_pattern_idx ON token_routes (model_pattern);',
|
|
||||||
mysql: 'CREATE INDEX `token_routes_model_pattern_idx` ON `token_routes` (`model_pattern`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "token_routes_model_pattern_idx" ON "token_routes" ("model_pattern")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'token_routes',
|
|
||||||
indexName: 'token_routes_enabled_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS token_routes_enabled_idx ON token_routes (enabled);',
|
|
||||||
mysql: 'CREATE INDEX `token_routes_enabled_idx` ON `token_routes` (`enabled`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "token_routes_enabled_idx" ON "token_routes" ("enabled")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'route_channels',
|
|
||||||
indexName: 'route_channels_route_id_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS route_channels_route_id_idx ON route_channels (route_id);',
|
|
||||||
mysql: 'CREATE INDEX `route_channels_route_id_idx` ON `route_channels` (`route_id`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "route_channels_route_id_idx" ON "route_channels" ("route_id")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'route_channels',
|
|
||||||
indexName: 'route_channels_account_id_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS route_channels_account_id_idx ON route_channels (account_id);',
|
|
||||||
mysql: 'CREATE INDEX `route_channels_account_id_idx` ON `route_channels` (`account_id`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "route_channels_account_id_idx" ON "route_channels" ("account_id")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'route_channels',
|
|
||||||
indexName: 'route_channels_token_id_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS route_channels_token_id_idx ON route_channels (token_id);',
|
|
||||||
mysql: 'CREATE INDEX `route_channels_token_id_idx` ON `route_channels` (`token_id`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "route_channels_token_id_idx" ON "route_channels" ("token_id")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'route_channels',
|
|
||||||
indexName: 'route_channels_route_enabled_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS route_channels_route_enabled_idx ON route_channels (route_id, enabled);',
|
|
||||||
mysql: 'CREATE INDEX `route_channels_route_enabled_idx` ON `route_channels` (`route_id`, `enabled`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "route_channels_route_enabled_idx" ON "route_channels" ("route_id", "enabled")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'route_channels',
|
|
||||||
indexName: 'route_channels_route_token_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS route_channels_route_token_idx ON route_channels (route_id, token_id);',
|
|
||||||
mysql: 'CREATE INDEX `route_channels_route_token_idx` ON `route_channels` (`route_id`, `token_id`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "route_channels_route_token_idx" ON "route_channels" ("route_id", "token_id")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'proxy_logs',
|
|
||||||
indexName: 'proxy_logs_created_at_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS proxy_logs_created_at_idx ON proxy_logs (created_at);',
|
|
||||||
mysql: 'CREATE INDEX `proxy_logs_created_at_idx` ON `proxy_logs` (`created_at`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "proxy_logs_created_at_idx" ON "proxy_logs" ("created_at")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'proxy_logs',
|
|
||||||
indexName: 'proxy_logs_account_created_at_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS proxy_logs_account_created_at_idx ON proxy_logs (account_id, created_at);',
|
|
||||||
mysql: 'CREATE INDEX `proxy_logs_account_created_at_idx` ON `proxy_logs` (`account_id`, `created_at`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "proxy_logs_account_created_at_idx" ON "proxy_logs" ("account_id", "created_at")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'proxy_logs',
|
|
||||||
indexName: 'proxy_logs_status_created_at_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS proxy_logs_status_created_at_idx ON proxy_logs (status, created_at);',
|
|
||||||
mysql: 'CREATE INDEX `proxy_logs_status_created_at_idx` ON `proxy_logs` (`status`, `created_at`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "proxy_logs_status_created_at_idx" ON "proxy_logs" ("status", "created_at")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'proxy_logs',
|
|
||||||
indexName: 'proxy_logs_model_actual_created_at_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS proxy_logs_model_actual_created_at_idx ON proxy_logs (model_actual, created_at);',
|
|
||||||
mysql: 'CREATE INDEX `proxy_logs_model_actual_created_at_idx` ON `proxy_logs` (`model_actual`(191), `created_at`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "proxy_logs_model_actual_created_at_idx" ON "proxy_logs" ("model_actual", "created_at")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'proxy_logs',
|
|
||||||
indexName: 'proxy_logs_downstream_api_key_created_at_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS proxy_logs_downstream_api_key_created_at_idx ON proxy_logs (downstream_api_key_id, created_at);',
|
|
||||||
mysql: 'CREATE INDEX `proxy_logs_downstream_api_key_created_at_idx` ON `proxy_logs` (`downstream_api_key_id`, `created_at`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "proxy_logs_downstream_api_key_created_at_idx" ON "proxy_logs" ("downstream_api_key_id", "created_at")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'proxy_video_tasks',
|
|
||||||
indexName: 'proxy_video_tasks_upstream_video_id_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS proxy_video_tasks_upstream_video_id_idx ON proxy_video_tasks (upstream_video_id);',
|
|
||||||
mysql: 'CREATE INDEX `proxy_video_tasks_upstream_video_id_idx` ON `proxy_video_tasks` (`upstream_video_id`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "proxy_video_tasks_upstream_video_id_idx" ON "proxy_video_tasks" ("upstream_video_id")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'proxy_video_tasks',
|
|
||||||
indexName: 'proxy_video_tasks_created_at_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS proxy_video_tasks_created_at_idx ON proxy_video_tasks (created_at);',
|
|
||||||
mysql: 'CREATE INDEX `proxy_video_tasks_created_at_idx` ON `proxy_video_tasks` (`created_at`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "proxy_video_tasks_created_at_idx" ON "proxy_video_tasks" ("created_at")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'downstream_api_keys',
|
|
||||||
indexName: 'downstream_api_keys_name_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS downstream_api_keys_name_idx ON downstream_api_keys (name);',
|
|
||||||
mysql: 'CREATE INDEX `downstream_api_keys_name_idx` ON `downstream_api_keys` (`name`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "downstream_api_keys_name_idx" ON "downstream_api_keys" ("name")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'downstream_api_keys',
|
|
||||||
indexName: 'downstream_api_keys_enabled_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS downstream_api_keys_enabled_idx ON downstream_api_keys (enabled);',
|
|
||||||
mysql: 'CREATE INDEX `downstream_api_keys_enabled_idx` ON `downstream_api_keys` (`enabled`)',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "downstream_api_keys_enabled_idx" ON "downstream_api_keys" ("enabled")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'downstream_api_keys',
|
|
||||||
indexName: 'downstream_api_keys_expires_at_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS downstream_api_keys_expires_at_idx ON downstream_api_keys (expires_at);',
|
|
||||||
mysql: 'CREATE INDEX `downstream_api_keys_expires_at_idx` ON `downstream_api_keys` (`expires_at`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "downstream_api_keys_expires_at_idx" ON "downstream_api_keys" ("expires_at")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'events',
|
|
||||||
indexName: 'events_read_created_at_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS events_read_created_at_idx ON events (read, created_at);',
|
|
||||||
mysql: 'CREATE INDEX `events_read_created_at_idx` ON `events` (`read`, `created_at`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "events_read_created_at_idx" ON "events" ("read", "created_at")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'events',
|
|
||||||
indexName: 'events_type_created_at_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS events_type_created_at_idx ON events (type, created_at);',
|
|
||||||
mysql: 'CREATE INDEX `events_type_created_at_idx` ON `events` (`type`, `created_at`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "events_type_created_at_idx" ON "events" ("type", "created_at")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
table: 'events',
|
|
||||||
indexName: 'events_created_at_idx',
|
|
||||||
createSql: {
|
|
||||||
sqlite: 'CREATE INDEX IF NOT EXISTS events_created_at_idx ON events (created_at);',
|
|
||||||
mysql: 'CREATE INDEX `events_created_at_idx` ON `events` (`created_at`(191))',
|
|
||||||
postgres: 'CREATE INDEX IF NOT EXISTS "events_created_at_idx" ON "events" ("created_at")',
|
|
||||||
},
|
|
||||||
},
|
|
||||||
];
|
|
||||||
|
|
||||||
function normalizeSchemaErrorMessage(error: unknown): string {
|
function normalizeSchemaErrorMessage(error: unknown): string {
|
||||||
if (typeof error === 'object' && error && 'message' in error) {
|
if (typeof error === 'object' && error && 'message' in error) {
|
||||||
|
|||||||
@@ -36,7 +36,6 @@ import {
|
|||||||
ensureProxyLogDownstreamApiKeyIdColumn,
|
ensureProxyLogDownstreamApiKeyIdColumn,
|
||||||
ensureProxyLogBillingDetailsColumn,
|
ensureProxyLogBillingDetailsColumn,
|
||||||
ensureRouteGroupingCompatibilityColumns,
|
ensureRouteGroupingCompatibilityColumns,
|
||||||
ensureSharedIndexCompatibility,
|
|
||||||
ensureSiteCompatibilityColumns,
|
ensureSiteCompatibilityColumns,
|
||||||
runtimeDbDialect,
|
runtimeDbDialect,
|
||||||
schema,
|
schema,
|
||||||
@@ -267,7 +266,6 @@ try {
|
|||||||
await ensureRouteGroupingCompatibilityColumns();
|
await ensureRouteGroupingCompatibilityColumns();
|
||||||
await ensureProxyFileCompatibilityColumns();
|
await ensureProxyFileCompatibilityColumns();
|
||||||
await ensureProxyLogDownstreamApiKeyIdColumn();
|
await ensureProxyLogDownstreamApiKeyIdColumn();
|
||||||
await ensureSharedIndexCompatibility();
|
|
||||||
const finalRows = await db.select().from(schema.settings).all();
|
const finalRows = await db.select().from(schema.settings).all();
|
||||||
const finalMap = toSettingsMap(finalRows);
|
const finalMap = toSettingsMap(finalRows);
|
||||||
applyRuntimeSettings(finalMap);
|
applyRuntimeSettings(finalMap);
|
||||||
|
|||||||
@@ -0,0 +1,115 @@
|
|||||||
|
import Fastify, { type FastifyInstance } from 'fastify';
|
||||||
|
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||||
|
import { mkdtempSync } from 'node:fs';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
import { eq } from 'drizzle-orm';
|
||||||
|
|
||||||
|
const refreshModelsForAccountMock = vi.fn();
|
||||||
|
const rebuildTokenRoutesFromAvailabilityMock = vi.fn();
|
||||||
|
|
||||||
|
vi.mock('../../services/modelService.js', () => ({
|
||||||
|
refreshModelsForAccount: (...args: unknown[]) => refreshModelsForAccountMock(...args),
|
||||||
|
rebuildTokenRoutesFromAvailability: (...args: unknown[]) => rebuildTokenRoutesFromAvailabilityMock(...args),
|
||||||
|
}));
|
||||||
|
|
||||||
|
type DbModule = typeof import('../../db/index.js');
|
||||||
|
|
||||||
|
describe('account token coverage refresh failure handling', () => {
|
||||||
|
let app: FastifyInstance;
|
||||||
|
let db: DbModule['db'];
|
||||||
|
let schema: DbModule['schema'];
|
||||||
|
let dataDir = '';
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
dataDir = mkdtempSync(join(tmpdir(), 'metapi-account-token-coverage-failure-'));
|
||||||
|
process.env.DATA_DIR = dataDir;
|
||||||
|
|
||||||
|
await import('../../db/migrate.js');
|
||||||
|
const dbModule = await import('../../db/index.js');
|
||||||
|
const accountTokenRoutesModule = await import('./accountTokens.js');
|
||||||
|
db = dbModule.db;
|
||||||
|
schema = dbModule.schema;
|
||||||
|
|
||||||
|
app = Fastify();
|
||||||
|
await app.register(accountTokenRoutesModule.accountTokensRoutes);
|
||||||
|
});
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
refreshModelsForAccountMock.mockReset();
|
||||||
|
rebuildTokenRoutesFromAvailabilityMock.mockReset();
|
||||||
|
|
||||||
|
await db.delete(schema.proxyLogs).run();
|
||||||
|
await db.delete(schema.checkinLogs).run();
|
||||||
|
await db.delete(schema.routeChannels).run();
|
||||||
|
await db.delete(schema.tokenRoutes).run();
|
||||||
|
await db.delete(schema.tokenModelAvailability).run();
|
||||||
|
await db.delete(schema.modelAvailability).run();
|
||||||
|
await db.delete(schema.accountTokens).run();
|
||||||
|
await db.delete(schema.accounts).run();
|
||||||
|
await db.delete(schema.sites).run();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
await app.close();
|
||||||
|
delete process.env.DATA_DIR;
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns success after token persistence even when coverage rebuild fails', async () => {
|
||||||
|
const site = await db.insert(schema.sites).values({
|
||||||
|
name: 'site-1',
|
||||||
|
url: 'https://site-1.example.com',
|
||||||
|
platform: 'new-api',
|
||||||
|
status: 'active',
|
||||||
|
}).returning().get();
|
||||||
|
|
||||||
|
const account = await db.insert(schema.accounts).values({
|
||||||
|
siteId: site.id,
|
||||||
|
username: 'user-1',
|
||||||
|
accessToken: 'acc-token-1',
|
||||||
|
status: 'active',
|
||||||
|
}).returning().get();
|
||||||
|
|
||||||
|
refreshModelsForAccountMock.mockResolvedValue({
|
||||||
|
accountId: account.id,
|
||||||
|
refreshed: true,
|
||||||
|
status: 'success',
|
||||||
|
errorCode: null,
|
||||||
|
errorMessage: '',
|
||||||
|
modelCount: 1,
|
||||||
|
modelsPreview: ['gpt-4o-mini'],
|
||||||
|
tokenScanned: 1,
|
||||||
|
discoveredByCredential: false,
|
||||||
|
discoveredApiToken: false,
|
||||||
|
});
|
||||||
|
rebuildTokenRoutesFromAvailabilityMock.mockRejectedValue(new Error('rebuild failed'));
|
||||||
|
|
||||||
|
const response = await app.inject({
|
||||||
|
method: 'POST',
|
||||||
|
url: '/api/account-tokens',
|
||||||
|
payload: {
|
||||||
|
accountId: account.id,
|
||||||
|
name: 'manual-token',
|
||||||
|
token: 'sk-manual-token',
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(response.statusCode).toBe(200);
|
||||||
|
expect(response.json()).toMatchObject({
|
||||||
|
success: true,
|
||||||
|
coverageRefresh: {
|
||||||
|
rebuild: {
|
||||||
|
success: false,
|
||||||
|
error: 'rebuild failed',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const tokens = await db.select()
|
||||||
|
.from(schema.accountTokens)
|
||||||
|
.where(eq(schema.accountTokens.accountId, account.id))
|
||||||
|
.all();
|
||||||
|
expect(tokens).toHaveLength(1);
|
||||||
|
expect(tokens[0]?.token).toBe('sk-manual-token');
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,203 @@
|
|||||||
|
import Fastify, { type FastifyInstance } from 'fastify';
|
||||||
|
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||||
|
import { mkdtempSync } from 'node:fs';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
import { eq } from 'drizzle-orm';
|
||||||
|
|
||||||
|
const getApiTokensMock = vi.fn();
|
||||||
|
const getApiTokenMock = vi.fn();
|
||||||
|
const createApiTokenMock = vi.fn();
|
||||||
|
const getUserGroupsMock = vi.fn();
|
||||||
|
const deleteApiTokenMock = vi.fn();
|
||||||
|
const getModelsMock = vi.fn();
|
||||||
|
|
||||||
|
vi.mock('../../services/platforms/index.js', () => ({
|
||||||
|
getAdapter: () => ({
|
||||||
|
getApiTokens: (...args: unknown[]) => getApiTokensMock(...args),
|
||||||
|
getApiToken: (...args: unknown[]) => getApiTokenMock(...args),
|
||||||
|
createApiToken: (...args: unknown[]) => createApiTokenMock(...args),
|
||||||
|
getUserGroups: (...args: unknown[]) => getUserGroupsMock(...args),
|
||||||
|
deleteApiToken: (...args: unknown[]) => deleteApiTokenMock(...args),
|
||||||
|
getModels: (...args: unknown[]) => getModelsMock(...args),
|
||||||
|
}),
|
||||||
|
}));
|
||||||
|
|
||||||
|
type DbModule = typeof import('../../db/index.js');
|
||||||
|
|
||||||
|
describe('account token coverage refresh', () => {
|
||||||
|
let app: FastifyInstance;
|
||||||
|
let db: DbModule['db'];
|
||||||
|
let schema: DbModule['schema'];
|
||||||
|
let dataDir = '';
|
||||||
|
let seedId = 0;
|
||||||
|
|
||||||
|
const nextSeed = () => {
|
||||||
|
seedId += 1;
|
||||||
|
return seedId;
|
||||||
|
};
|
||||||
|
|
||||||
|
const seedAccount = async (modelName: string) => {
|
||||||
|
const id = nextSeed();
|
||||||
|
const site = await db.insert(schema.sites).values({
|
||||||
|
name: `site-${id}`,
|
||||||
|
url: `https://site-${id}.example.com`,
|
||||||
|
platform: 'new-api',
|
||||||
|
status: 'active',
|
||||||
|
}).returning().get();
|
||||||
|
|
||||||
|
const account = await db.insert(schema.accounts).values({
|
||||||
|
siteId: site.id,
|
||||||
|
username: `user-${id}`,
|
||||||
|
accessToken: `acc-token-${id}`,
|
||||||
|
status: 'active',
|
||||||
|
}).returning().get();
|
||||||
|
|
||||||
|
await db.insert(schema.modelAvailability).values({
|
||||||
|
accountId: account.id,
|
||||||
|
modelName,
|
||||||
|
available: true,
|
||||||
|
}).run();
|
||||||
|
|
||||||
|
return { site, account };
|
||||||
|
};
|
||||||
|
|
||||||
|
const readTokenCandidates = async () => {
|
||||||
|
const response = await app.inject({
|
||||||
|
method: 'GET',
|
||||||
|
url: '/api/models/token-candidates',
|
||||||
|
});
|
||||||
|
expect(response.statusCode).toBe(200);
|
||||||
|
return response.json() as {
|
||||||
|
models: Record<string, Array<{ tokenId: number; accountId: number }>>;
|
||||||
|
modelsWithoutToken: Record<string, Array<{ accountId: number }>>;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
dataDir = mkdtempSync(join(tmpdir(), 'metapi-account-token-coverage-'));
|
||||||
|
process.env.DATA_DIR = dataDir;
|
||||||
|
|
||||||
|
await import('../../db/migrate.js');
|
||||||
|
const dbModule = await import('../../db/index.js');
|
||||||
|
const accountTokenRoutesModule = await import('./accountTokens.js');
|
||||||
|
const statsRoutesModule = await import('./stats.js');
|
||||||
|
db = dbModule.db;
|
||||||
|
schema = dbModule.schema;
|
||||||
|
|
||||||
|
app = Fastify();
|
||||||
|
await app.register(accountTokenRoutesModule.accountTokensRoutes);
|
||||||
|
await app.register(statsRoutesModule.statsRoutes);
|
||||||
|
});
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
getApiTokensMock.mockReset();
|
||||||
|
getApiTokenMock.mockReset();
|
||||||
|
createApiTokenMock.mockReset();
|
||||||
|
getUserGroupsMock.mockReset();
|
||||||
|
deleteApiTokenMock.mockReset();
|
||||||
|
getModelsMock.mockReset();
|
||||||
|
seedId = 0;
|
||||||
|
|
||||||
|
await db.delete(schema.proxyLogs).run();
|
||||||
|
await db.delete(schema.checkinLogs).run();
|
||||||
|
await db.delete(schema.routeChannels).run();
|
||||||
|
await db.delete(schema.tokenRoutes).run();
|
||||||
|
await db.delete(schema.tokenModelAvailability).run();
|
||||||
|
await db.delete(schema.modelAvailability).run();
|
||||||
|
await db.delete(schema.accountTokens).run();
|
||||||
|
await db.delete(schema.accounts).run();
|
||||||
|
await db.delete(schema.sites).run();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
await app.close();
|
||||||
|
delete process.env.DATA_DIR;
|
||||||
|
});
|
||||||
|
|
||||||
|
it('refreshes token coverage after manually adding an account token', async () => {
|
||||||
|
const modelName = 'gpt-4o-mini';
|
||||||
|
const { account } = await seedAccount(modelName);
|
||||||
|
getModelsMock.mockImplementation(async (_url: string, credential: string) => {
|
||||||
|
if (credential === account.accessToken || credential === 'sk-manual-token') {
|
||||||
|
return [modelName];
|
||||||
|
}
|
||||||
|
return [];
|
||||||
|
});
|
||||||
|
|
||||||
|
const response = await app.inject({
|
||||||
|
method: 'POST',
|
||||||
|
url: '/api/account-tokens',
|
||||||
|
payload: {
|
||||||
|
accountId: account.id,
|
||||||
|
name: 'manual-token',
|
||||||
|
token: 'sk-manual-token',
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(response.statusCode).toBe(200);
|
||||||
|
|
||||||
|
const token = await db.select()
|
||||||
|
.from(schema.accountTokens)
|
||||||
|
.where(eq(schema.accountTokens.accountId, account.id))
|
||||||
|
.get();
|
||||||
|
expect(token).toBeDefined();
|
||||||
|
|
||||||
|
const tokenAvailability = await db.select()
|
||||||
|
.from(schema.tokenModelAvailability)
|
||||||
|
.where(eq(schema.tokenModelAvailability.tokenId, token!.id))
|
||||||
|
.all();
|
||||||
|
expect(tokenAvailability.map((row) => row.modelName)).toContain(modelName);
|
||||||
|
|
||||||
|
const candidates = await readTokenCandidates();
|
||||||
|
expect(candidates.modelsWithoutToken[modelName]).toBeUndefined();
|
||||||
|
expect(candidates.models[modelName]).toEqual([
|
||||||
|
expect.objectContaining({
|
||||||
|
accountId: account.id,
|
||||||
|
tokenId: token!.id,
|
||||||
|
}),
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('refreshes token coverage after syncing account tokens from upstream', async () => {
|
||||||
|
const modelName = 'claude-sonnet-4-5-20250929';
|
||||||
|
const { account } = await seedAccount(modelName);
|
||||||
|
getApiTokensMock.mockResolvedValue([
|
||||||
|
{ name: 'default', key: 'sk-synced-token', enabled: true },
|
||||||
|
]);
|
||||||
|
getModelsMock.mockImplementation(async (_url: string, credential: string) => {
|
||||||
|
if (credential === account.accessToken || credential === 'sk-synced-token') {
|
||||||
|
return [modelName];
|
||||||
|
}
|
||||||
|
return [];
|
||||||
|
});
|
||||||
|
|
||||||
|
const response = await app.inject({
|
||||||
|
method: 'POST',
|
||||||
|
url: `/api/account-tokens/sync/${account.id}`,
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(response.statusCode).toBe(200);
|
||||||
|
|
||||||
|
const token = await db.select()
|
||||||
|
.from(schema.accountTokens)
|
||||||
|
.where(eq(schema.accountTokens.accountId, account.id))
|
||||||
|
.get();
|
||||||
|
expect(token).toBeDefined();
|
||||||
|
|
||||||
|
const tokenAvailability = await db.select()
|
||||||
|
.from(schema.tokenModelAvailability)
|
||||||
|
.where(eq(schema.tokenModelAvailability.tokenId, token!.id))
|
||||||
|
.all();
|
||||||
|
expect(tokenAvailability.map((row) => row.modelName)).toContain(modelName);
|
||||||
|
|
||||||
|
const candidates = await readTokenCandidates();
|
||||||
|
expect(candidates.modelsWithoutToken[modelName]).toBeUndefined();
|
||||||
|
expect(candidates.models[modelName]).toEqual([
|
||||||
|
expect.objectContaining({
|
||||||
|
accountId: account.id,
|
||||||
|
tokenId: token!.id,
|
||||||
|
}),
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -14,6 +14,11 @@ import {
|
|||||||
import { getAdapter } from '../../services/platforms/index.js';
|
import { getAdapter } from '../../services/platforms/index.js';
|
||||||
import { getCredentialModeFromExtraConfig, resolvePlatformUserId } from '../../services/accountExtraConfig.js';
|
import { getCredentialModeFromExtraConfig, resolvePlatformUserId } from '../../services/accountExtraConfig.js';
|
||||||
import { startBackgroundTask } from '../../services/backgroundTaskService.js';
|
import { startBackgroundTask } from '../../services/backgroundTaskService.js';
|
||||||
|
import {
|
||||||
|
rebuildTokenRoutesFromAvailability,
|
||||||
|
refreshModelsForAccount,
|
||||||
|
type ModelRefreshResult,
|
||||||
|
} from '../../services/modelService.js';
|
||||||
|
|
||||||
type AccountWithSiteRow = {
|
type AccountWithSiteRow = {
|
||||||
accounts: typeof schema.accounts.$inferSelect;
|
accounts: typeof schema.accounts.$inferSelect;
|
||||||
@@ -38,6 +43,25 @@ type SyncExecutionResult = {
|
|||||||
defaultTokenId?: number | null;
|
defaultTokenId?: number | null;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
type CoverageRefreshFailureItem = {
|
||||||
|
accountId: number;
|
||||||
|
refreshed: false;
|
||||||
|
status: 'failed';
|
||||||
|
errorCode: 'coverage_refresh_failed';
|
||||||
|
errorMessage: string;
|
||||||
|
modelCount: 0;
|
||||||
|
modelsPreview: string[];
|
||||||
|
reason: 'coverage_refresh_failed';
|
||||||
|
tokenScanned: 0;
|
||||||
|
discoveredByCredential: false;
|
||||||
|
discoveredApiToken: false;
|
||||||
|
};
|
||||||
|
|
||||||
|
type CoverageRefreshItem = ModelRefreshResult | CoverageRefreshFailureItem;
|
||||||
|
type CoverageRefreshRebuildResult =
|
||||||
|
| { success: true; result: Awaited<ReturnType<typeof rebuildTokenRoutesFromAvailability>> }
|
||||||
|
| { success: false; error: string };
|
||||||
|
|
||||||
const TOKEN_SYNC_TIMEOUT_MS = 15_000;
|
const TOKEN_SYNC_TIMEOUT_MS = 15_000;
|
||||||
const SYNC_ALL_BATCH_SIZE = 3;
|
const SYNC_ALL_BATCH_SIZE = 3;
|
||||||
|
|
||||||
@@ -337,6 +361,12 @@ async function executeSyncAllAccountTokens() {
|
|||||||
results.push(...batchResults);
|
results.push(...batchResults);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const coverageRefresh = await refreshCoverageForAccounts(
|
||||||
|
results
|
||||||
|
.filter((item) => item.status === 'synced')
|
||||||
|
.map((item) => item.accountId),
|
||||||
|
);
|
||||||
|
|
||||||
const summary = {
|
const summary = {
|
||||||
total: results.length,
|
total: results.length,
|
||||||
synced: results.filter((item) => item.status === 'synced').length,
|
synced: results.filter((item) => item.status === 'synced').length,
|
||||||
@@ -346,7 +376,72 @@ async function executeSyncAllAccountTokens() {
|
|||||||
updated: results.reduce((acc, item) => acc + item.updated, 0),
|
updated: results.reduce((acc, item) => acc + item.updated, 0),
|
||||||
};
|
};
|
||||||
|
|
||||||
return { summary, results };
|
return { summary, results, coverageRefresh };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function refreshCoverageForAccounts(accountIds: number[]) {
|
||||||
|
const uniqueAccountIds = Array.from(new Set(
|
||||||
|
accountIds.filter((id) => Number.isFinite(id) && id > 0),
|
||||||
|
));
|
||||||
|
|
||||||
|
if (uniqueAccountIds.length === 0) {
|
||||||
|
return { refresh: [], rebuild: null };
|
||||||
|
}
|
||||||
|
|
||||||
|
const refresh: CoverageRefreshItem[] = [];
|
||||||
|
for (let offset = 0; offset < uniqueAccountIds.length; offset += SYNC_ALL_BATCH_SIZE) {
|
||||||
|
const batch = uniqueAccountIds.slice(offset, offset + SYNC_ALL_BATCH_SIZE);
|
||||||
|
const settled = await Promise.allSettled(
|
||||||
|
batch.map(async (accountId) => refreshModelsForAccount(accountId)),
|
||||||
|
);
|
||||||
|
settled.forEach((result, index) => {
|
||||||
|
if (result.status === 'fulfilled') {
|
||||||
|
refresh.push(result.value);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const accountId = batch[index] || 0;
|
||||||
|
const errorMessage = result.reason instanceof Error ? result.reason.message : String(result.reason || 'coverage refresh failed');
|
||||||
|
refresh.push(buildCoverageRefreshFailureItem(accountId, errorMessage));
|
||||||
|
console.warn(`[account-tokens] coverage refresh failed for account ${accountId}: ${errorMessage}`);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let rebuild: CoverageRefreshRebuildResult | null = null;
|
||||||
|
try {
|
||||||
|
rebuild = {
|
||||||
|
success: true,
|
||||||
|
result: await rebuildTokenRoutesFromAvailability(),
|
||||||
|
};
|
||||||
|
} catch (error) {
|
||||||
|
const errorMessage = error instanceof Error ? error.message : String(error || 'route rebuild failed');
|
||||||
|
rebuild = {
|
||||||
|
success: false,
|
||||||
|
error: errorMessage,
|
||||||
|
};
|
||||||
|
console.warn(`[account-tokens] token route rebuild failed after coverage refresh: ${errorMessage}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
return { refresh, rebuild };
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildCoverageRefreshFailureItem(
|
||||||
|
accountId: number,
|
||||||
|
errorMessage: string,
|
||||||
|
): CoverageRefreshFailureItem {
|
||||||
|
return {
|
||||||
|
accountId,
|
||||||
|
refreshed: false,
|
||||||
|
status: 'failed',
|
||||||
|
errorCode: 'coverage_refresh_failed',
|
||||||
|
errorMessage,
|
||||||
|
modelCount: 0,
|
||||||
|
modelsPreview: [],
|
||||||
|
reason: 'coverage_refresh_failed',
|
||||||
|
tokenScanned: 0,
|
||||||
|
discoveredByCredential: false,
|
||||||
|
discoveredApiToken: false,
|
||||||
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function accountTokensRoutes(app: FastifyInstance) {
|
export async function accountTokensRoutes(app: FastifyInstance) {
|
||||||
@@ -416,8 +511,8 @@ export async function accountTokensRoutes(app: FastifyInstance) {
|
|||||||
} else if (existing.every((token) => !token.isDefault) && (body.enabled ?? true)) {
|
} else if (existing.every((token) => !token.isDefault) && (body.enabled ?? true)) {
|
||||||
await setDefaultToken(created.id);
|
await setDefaultToken(created.id);
|
||||||
}
|
}
|
||||||
|
const coverageRefresh = await refreshCoverageForAccounts([body.accountId]);
|
||||||
return { success: true, token: created };
|
return { success: true, token: created, coverageRefresh };
|
||||||
}
|
}
|
||||||
|
|
||||||
const account = row.accounts;
|
const account = row.accounts;
|
||||||
@@ -496,6 +591,7 @@ export async function accountTokensRoutes(app: FastifyInstance) {
|
|||||||
if (syncResult.status === 'skipped') {
|
if (syncResult.status === 'skipped') {
|
||||||
return reply.code(502).send({ success: false, message: syncResult.message || '站点未返回可用令牌' });
|
return reply.code(502).send({ success: false, message: syncResult.message || '站点未返回可用令牌' });
|
||||||
}
|
}
|
||||||
|
const coverageRefresh = await refreshCoverageForAccounts([account.id]);
|
||||||
|
|
||||||
const preferred = await db.select().from(schema.accountTokens)
|
const preferred = await db.select().from(schema.accountTokens)
|
||||||
.where(and(eq(schema.accountTokens.accountId, account.id), eq(schema.accountTokens.isDefault, true)))
|
.where(and(eq(schema.accountTokens.accountId, account.id), eq(schema.accountTokens.isDefault, true)))
|
||||||
@@ -509,6 +605,7 @@ export async function accountTokensRoutes(app: FastifyInstance) {
|
|||||||
success: true,
|
success: true,
|
||||||
createdViaUpstream: true,
|
createdViaUpstream: true,
|
||||||
...syncResult,
|
...syncResult,
|
||||||
|
coverageRefresh,
|
||||||
token,
|
token,
|
||||||
};
|
};
|
||||||
});
|
});
|
||||||
@@ -823,7 +920,8 @@ export async function accountTokensRoutes(app: FastifyInstance) {
|
|||||||
if (result.status === 'failed') {
|
if (result.status === 'failed') {
|
||||||
return reply.code(502).send({ success: false, message: result.message || '同步失败' });
|
return reply.code(502).send({ success: false, message: result.message || '同步失败' });
|
||||||
}
|
}
|
||||||
return { success: true, ...result };
|
const coverageRefresh = await refreshCoverageForAccounts([accountId]);
|
||||||
|
return { success: true, ...result, coverageRefresh };
|
||||||
});
|
});
|
||||||
|
|
||||||
app.post<{ Body?: { wait?: boolean } }>('/api/account-tokens/sync-all', async (request, reply) => {
|
app.post<{ Body?: { wait?: boolean } }>('/api/account-tokens/sync-all', async (request, reply) => {
|
||||||
|
|||||||
@@ -6,10 +6,19 @@ import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vites
|
|||||||
const fetchMock = vi.fn();
|
const fetchMock = vi.fn();
|
||||||
const selectChannelMock = vi.fn();
|
const selectChannelMock = vi.fn();
|
||||||
const selectNextChannelMock = vi.fn();
|
const selectNextChannelMock = vi.fn();
|
||||||
|
const recordSuccessMock = vi.fn();
|
||||||
const recordFailureMock = vi.fn();
|
const recordFailureMock = vi.fn();
|
||||||
|
const explainSelectionMock = vi.fn();
|
||||||
const invalidateTokenRouterCacheMock = vi.fn();
|
const invalidateTokenRouterCacheMock = vi.fn();
|
||||||
const authorizeDownstreamTokenMock = vi.fn();
|
const authorizeDownstreamTokenMock = vi.fn();
|
||||||
const consumeManagedKeyRequestMock = vi.fn();
|
const consumeManagedKeyRequestMock = vi.fn();
|
||||||
|
const isModelAllowedByPolicyOrAllowedRoutesMock = vi.fn();
|
||||||
|
const dbInsertValuesMock = vi.fn((_values?: unknown) => ({
|
||||||
|
run: () => undefined,
|
||||||
|
}));
|
||||||
|
const dbInsertMock = vi.fn((_table?: unknown) => ({
|
||||||
|
values: (values: unknown) => dbInsertValuesMock(values),
|
||||||
|
}));
|
||||||
|
|
||||||
vi.mock('undici', () => ({
|
vi.mock('undici', () => ({
|
||||||
fetch: (...args: unknown[]) => fetchMock(...args),
|
fetch: (...args: unknown[]) => fetchMock(...args),
|
||||||
@@ -19,8 +28,9 @@ vi.mock('../../services/tokenRouter.js', () => ({
|
|||||||
tokenRouter: {
|
tokenRouter: {
|
||||||
selectChannel: (...args: unknown[]) => selectChannelMock(...args),
|
selectChannel: (...args: unknown[]) => selectChannelMock(...args),
|
||||||
selectNextChannel: (...args: unknown[]) => selectNextChannelMock(...args),
|
selectNextChannel: (...args: unknown[]) => selectNextChannelMock(...args),
|
||||||
|
recordSuccess: (...args: unknown[]) => recordSuccessMock(...args),
|
||||||
recordFailure: (...args: unknown[]) => recordFailureMock(...args),
|
recordFailure: (...args: unknown[]) => recordFailureMock(...args),
|
||||||
explainSelection: vi.fn(async () => ({ selectedChannelId: 11 })),
|
explainSelection: (...args: unknown[]) => explainSelectionMock(...args),
|
||||||
},
|
},
|
||||||
invalidateTokenRouterCache: (...args: unknown[]) => invalidateTokenRouterCacheMock(...args),
|
invalidateTokenRouterCache: (...args: unknown[]) => invalidateTokenRouterCacheMock(...args),
|
||||||
}));
|
}));
|
||||||
@@ -28,6 +38,16 @@ vi.mock('../../services/tokenRouter.js', () => ({
|
|||||||
vi.mock('../../services/downstreamApiKeyService.js', () => ({
|
vi.mock('../../services/downstreamApiKeyService.js', () => ({
|
||||||
authorizeDownstreamToken: (...args: unknown[]) => authorizeDownstreamTokenMock(...args),
|
authorizeDownstreamToken: (...args: unknown[]) => authorizeDownstreamTokenMock(...args),
|
||||||
consumeManagedKeyRequest: (...args: unknown[]) => consumeManagedKeyRequestMock(...args),
|
consumeManagedKeyRequest: (...args: unknown[]) => consumeManagedKeyRequestMock(...args),
|
||||||
|
isModelAllowedByPolicyOrAllowedRoutes: (...args: unknown[]) => isModelAllowedByPolicyOrAllowedRoutesMock(...args),
|
||||||
|
}));
|
||||||
|
|
||||||
|
vi.mock('../../db/index.js', () => ({
|
||||||
|
db: {
|
||||||
|
insert: (arg: unknown) => dbInsertMock(arg),
|
||||||
|
},
|
||||||
|
schema: {
|
||||||
|
proxyLogs: {},
|
||||||
|
},
|
||||||
}));
|
}));
|
||||||
|
|
||||||
function parseSsePayloads(body: string): Array<Record<string, unknown>> {
|
function parseSsePayloads(body: string): Array<Record<string, unknown>> {
|
||||||
@@ -82,9 +102,14 @@ describe('gemini native proxy routes', () => {
|
|||||||
fetchMock.mockReset();
|
fetchMock.mockReset();
|
||||||
selectChannelMock.mockReset();
|
selectChannelMock.mockReset();
|
||||||
selectNextChannelMock.mockReset();
|
selectNextChannelMock.mockReset();
|
||||||
|
recordSuccessMock.mockReset();
|
||||||
recordFailureMock.mockReset();
|
recordFailureMock.mockReset();
|
||||||
|
explainSelectionMock.mockReset();
|
||||||
authorizeDownstreamTokenMock.mockReset();
|
authorizeDownstreamTokenMock.mockReset();
|
||||||
consumeManagedKeyRequestMock.mockReset();
|
consumeManagedKeyRequestMock.mockReset();
|
||||||
|
isModelAllowedByPolicyOrAllowedRoutesMock.mockReset();
|
||||||
|
dbInsertMock.mockClear();
|
||||||
|
dbInsertValuesMock.mockClear();
|
||||||
|
|
||||||
authorizeDownstreamTokenMock.mockResolvedValue({
|
authorizeDownstreamTokenMock.mockResolvedValue({
|
||||||
ok: true,
|
ok: true,
|
||||||
@@ -102,7 +127,10 @@ describe('gemini native proxy routes', () => {
|
|||||||
actualModel: 'gemini-2.5-flash',
|
actualModel: 'gemini-2.5-flash',
|
||||||
});
|
});
|
||||||
selectNextChannelMock.mockReturnValue(null);
|
selectNextChannelMock.mockReturnValue(null);
|
||||||
|
recordSuccessMock.mockResolvedValue(undefined);
|
||||||
recordFailureMock.mockResolvedValue(undefined);
|
recordFailureMock.mockResolvedValue(undefined);
|
||||||
|
explainSelectionMock.mockResolvedValue({ selectedChannelId: 11 });
|
||||||
|
isModelAllowedByPolicyOrAllowedRoutesMock.mockResolvedValue(true);
|
||||||
});
|
});
|
||||||
|
|
||||||
afterAll(async () => {
|
afterAll(async () => {
|
||||||
@@ -241,6 +269,108 @@ describe('gemini native proxy routes', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('filters Gemini native model list by downstream managed-key policy', async () => {
|
||||||
|
authorizeDownstreamTokenMock.mockResolvedValue({
|
||||||
|
ok: true,
|
||||||
|
source: 'managed',
|
||||||
|
token: 'sk-managed-gemini',
|
||||||
|
key: { id: 91 },
|
||||||
|
policy: { supportedModels: ['gemini-2.5-flash'], allowedRouteIds: [], siteWeightMultipliers: {} },
|
||||||
|
});
|
||||||
|
isModelAllowedByPolicyOrAllowedRoutesMock.mockImplementation(async (modelName: string) => modelName === 'gemini-2.5-flash');
|
||||||
|
explainSelectionMock.mockImplementation(async (modelName: string) => (
|
||||||
|
modelName === 'gemini-2.5-flash'
|
||||||
|
? { selectedChannelId: 11 }
|
||||||
|
: { selectedChannelId: undefined }
|
||||||
|
));
|
||||||
|
fetchMock.mockResolvedValue(new Response(JSON.stringify({
|
||||||
|
models: [
|
||||||
|
{ name: 'models/gemini-2.5-flash', displayName: 'Gemini 2.5 Flash' },
|
||||||
|
{ name: 'models/gemini-2.0-flash', displayName: 'Gemini 2.0 Flash' },
|
||||||
|
],
|
||||||
|
}), {
|
||||||
|
status: 200,
|
||||||
|
headers: { 'content-type': 'application/json' },
|
||||||
|
}));
|
||||||
|
|
||||||
|
const response = await app.inject({
|
||||||
|
method: 'GET',
|
||||||
|
url: '/v1beta/models',
|
||||||
|
headers: {
|
||||||
|
authorization: 'Bearer sk-managed-gemini',
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(response.statusCode).toBe(200);
|
||||||
|
expect(response.json()).toEqual({
|
||||||
|
models: [
|
||||||
|
{
|
||||||
|
name: 'models/gemini-2.5-flash',
|
||||||
|
displayName: 'Gemini 2.5 Flash',
|
||||||
|
},
|
||||||
|
],
|
||||||
|
});
|
||||||
|
expect(isModelAllowedByPolicyOrAllowedRoutesMock).toHaveBeenCalledWith('gemini-2.5-flash', { supportedModels: ['gemini-2.5-flash'], allowedRouteIds: [], siteWeightMultipliers: {} });
|
||||||
|
expect(isModelAllowedByPolicyOrAllowedRoutesMock).toHaveBeenCalledWith('gemini-2.0-flash', { supportedModels: ['gemini-2.5-flash'], allowedRouteIds: [], siteWeightMultipliers: {} });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('writes a proxy log row for successful native generateContent requests', async () => {
|
||||||
|
fetchMock.mockResolvedValue(new Response(JSON.stringify({
|
||||||
|
candidates: [
|
||||||
|
{
|
||||||
|
content: {
|
||||||
|
parts: [{ text: 'hello from gemini' }],
|
||||||
|
role: 'model',
|
||||||
|
},
|
||||||
|
finishReason: 'STOP',
|
||||||
|
},
|
||||||
|
],
|
||||||
|
usageMetadata: {
|
||||||
|
promptTokenCount: 10,
|
||||||
|
candidatesTokenCount: 5,
|
||||||
|
totalTokenCount: 15,
|
||||||
|
},
|
||||||
|
}), {
|
||||||
|
status: 200,
|
||||||
|
headers: { 'content-type': 'application/json' },
|
||||||
|
}));
|
||||||
|
|
||||||
|
const response = await app.inject({
|
||||||
|
method: 'POST',
|
||||||
|
url: '/v1beta/models/gemini-2.5-flash:generateContent',
|
||||||
|
headers: {
|
||||||
|
'x-goog-api-key': 'sk-managed-gemini',
|
||||||
|
},
|
||||||
|
payload: {
|
||||||
|
contents: [
|
||||||
|
{
|
||||||
|
role: 'user',
|
||||||
|
parts: [{ text: 'hello' }],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(response.statusCode).toBe(200);
|
||||||
|
expect(recordSuccessMock).toHaveBeenCalledWith(11, expect.any(Number), 0);
|
||||||
|
expect(dbInsertMock).toHaveBeenCalledTimes(1);
|
||||||
|
expect(dbInsertValuesMock).toHaveBeenCalledWith(expect.objectContaining({
|
||||||
|
routeId: 22,
|
||||||
|
channelId: 11,
|
||||||
|
accountId: 33,
|
||||||
|
modelRequested: 'gemini-2.5-flash',
|
||||||
|
modelActual: 'gemini-2.5-flash',
|
||||||
|
status: 'success',
|
||||||
|
httpStatus: 200,
|
||||||
|
retryCount: 0,
|
||||||
|
promptTokens: 10,
|
||||||
|
completionTokens: 5,
|
||||||
|
totalTokens: 15,
|
||||||
|
errorMessage: '[downstream:/v1beta/models/gemini-2.5-flash:generateContent] [upstream:/v1beta/models/gemini-2.5-flash:generateContent]',
|
||||||
|
createdAt: expect.any(String),
|
||||||
|
}));
|
||||||
|
});
|
||||||
|
|
||||||
it('forwards explicit gemini version paths through transformer-owned parsing helpers', async () => {
|
it('forwards explicit gemini version paths through transformer-owned parsing helpers', async () => {
|
||||||
fetchMock.mockResolvedValue(new Response(JSON.stringify({
|
fetchMock.mockResolvedValue(new Response(JSON.stringify({
|
||||||
candidates: [
|
candidates: [
|
||||||
@@ -842,4 +972,67 @@ describe('gemini native proxy routes', () => {
|
|||||||
expect(recordFailureMock).toHaveBeenCalledWith(11);
|
expect(recordFailureMock).toHaveBeenCalledWith(11);
|
||||||
expect(response.body).toContain('hello from fallback sse');
|
expect(response.body).toContain('hello from fallback sse');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('writes failed and successful proxy log rows for Gemini-native stream retries', async () => {
|
||||||
|
selectNextChannelMock.mockReturnValue({
|
||||||
|
channel: { id: 12, routeId: 22 },
|
||||||
|
site: { id: 45, name: 'gemini-site-2', url: 'https://generativelanguage.googleapis.com', platform: 'gemini' },
|
||||||
|
account: { id: 34, username: 'demo-user-2' },
|
||||||
|
tokenName: 'fallback',
|
||||||
|
tokenValue: 'gemini-key-2',
|
||||||
|
actualModel: 'gemini-2.5-flash',
|
||||||
|
});
|
||||||
|
|
||||||
|
const encoder = new TextEncoder();
|
||||||
|
const upstreamBody = new ReadableStream<Uint8Array>({
|
||||||
|
start(controller) {
|
||||||
|
controller.enqueue(encoder.encode('data: {"responseId":"resp-fallback","candidates":[{"content":{"role":"model","parts":[{"text":"hello from fallback sse"}]},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":11,"candidatesTokenCount":6,"totalTokenCount":17}}\r\n\r\n'));
|
||||||
|
controller.close();
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
fetchMock
|
||||||
|
.mockResolvedValueOnce(new Response(JSON.stringify({
|
||||||
|
error: { message: 'upstream unavailable' },
|
||||||
|
}), {
|
||||||
|
status: 500,
|
||||||
|
headers: { 'content-type': 'application/json' },
|
||||||
|
}))
|
||||||
|
.mockResolvedValueOnce(new Response(upstreamBody, {
|
||||||
|
status: 200,
|
||||||
|
headers: { 'content-type': 'text/event-stream; charset=utf-8' },
|
||||||
|
}));
|
||||||
|
|
||||||
|
const response = await app.inject({
|
||||||
|
method: 'POST',
|
||||||
|
url: '/v1beta/models/gemini-2.5-flash:streamGenerateContent?alt=sse',
|
||||||
|
headers: {
|
||||||
|
'x-goog-api-key': 'sk-managed-gemini',
|
||||||
|
},
|
||||||
|
payload: {
|
||||||
|
contents: [{ role: 'user', parts: [{ text: 'hello' }] }],
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(response.statusCode).toBe(200);
|
||||||
|
expect(dbInsertMock).toHaveBeenCalledTimes(2);
|
||||||
|
expect(dbInsertValuesMock).toHaveBeenNthCalledWith(1, expect.objectContaining({
|
||||||
|
channelId: 11,
|
||||||
|
status: 'failed',
|
||||||
|
httpStatus: 500,
|
||||||
|
retryCount: 0,
|
||||||
|
errorMessage: '[downstream:/v1beta/models/gemini-2.5-flash:streamGenerateContent] [upstream:/v1beta/models/gemini-2.5-flash:streamGenerateContent] {\"error\":{\"message\":\"upstream unavailable\"}}',
|
||||||
|
}));
|
||||||
|
expect(dbInsertValuesMock).toHaveBeenNthCalledWith(2, expect.objectContaining({
|
||||||
|
channelId: 12,
|
||||||
|
status: 'success',
|
||||||
|
httpStatus: 200,
|
||||||
|
retryCount: 1,
|
||||||
|
promptTokens: 11,
|
||||||
|
completionTokens: 6,
|
||||||
|
totalTokens: 17,
|
||||||
|
errorMessage: '[downstream:/v1beta/models/gemini-2.5-flash:streamGenerateContent] [upstream:/v1beta/models/gemini-2.5-flash:streamGenerateContent]',
|
||||||
|
}));
|
||||||
|
expect(recordSuccessMock).toHaveBeenCalledWith(12, expect.any(Number), 0);
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,8 +1,13 @@
|
|||||||
import { FastifyInstance, FastifyReply, FastifyRequest } from 'fastify';
|
import { FastifyInstance, FastifyReply, FastifyRequest } from 'fastify';
|
||||||
import { TextDecoder } from 'node:util';
|
import { TextDecoder } from 'node:util';
|
||||||
import { fetch } from 'undici';
|
import { fetch } from 'undici';
|
||||||
|
import { db, schema } from '../../db/index.js';
|
||||||
|
import { formatUtcSqlDateTime } from '../../services/localTimeService.js';
|
||||||
|
import { parseProxyUsage } from '../../services/proxyUsageParser.js';
|
||||||
|
import { isModelAllowedByPolicyOrAllowedRoutes } from '../../services/downstreamApiKeyService.js';
|
||||||
import { tokenRouter } from '../../services/tokenRouter.js';
|
import { tokenRouter } from '../../services/tokenRouter.js';
|
||||||
import { getDownstreamRoutingPolicy } from './downstreamPolicy.js';
|
import { getDownstreamRoutingPolicy } from './downstreamPolicy.js';
|
||||||
|
import { composeProxyLogMessage } from './logPathMeta.js';
|
||||||
import {
|
import {
|
||||||
geminiGenerateContentTransformer,
|
geminiGenerateContentTransformer,
|
||||||
} from '../../transformers/gemini/generate-content/index.js';
|
} from '../../transformers/gemini/generate-content/index.js';
|
||||||
@@ -14,6 +19,11 @@ const GEMINI_MODEL_PROBES = [
|
|||||||
'gemini-1.5-flash',
|
'gemini-1.5-flash',
|
||||||
'gemini-pro',
|
'gemini-pro',
|
||||||
];
|
];
|
||||||
|
const EMPTY_PROXY_USAGE = {
|
||||||
|
promptTokens: 0,
|
||||||
|
completionTokens: 0,
|
||||||
|
totalTokens: 0,
|
||||||
|
};
|
||||||
|
|
||||||
async function selectGeminiChannel(request: FastifyRequest) {
|
async function selectGeminiChannel(request: FastifyRequest) {
|
||||||
const policy = getDownstreamRoutingPolicy(request);
|
const policy = getDownstreamRoutingPolicy(request);
|
||||||
@@ -33,6 +43,106 @@ async function selectNextGeminiProbeChannel(request: FastifyRequest, excludeChan
|
|||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function resolveDownstreamPath(request: FastifyRequest): string {
|
||||||
|
const rawUrl = request.raw.url || request.url || '';
|
||||||
|
const withoutQuery = rawUrl.split('?')[0] || rawUrl;
|
||||||
|
return withoutQuery || '/v1beta/models';
|
||||||
|
}
|
||||||
|
|
||||||
|
function resolveUpstreamPath(apiVersion: string, modelActionPath: string): string {
|
||||||
|
const normalizedVersion = apiVersion.replace(/^\/+/, '');
|
||||||
|
const normalizedAction = modelActionPath.replace(/^\/+/, '');
|
||||||
|
return `/${normalizedVersion}/${normalizedAction}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function hasDownstreamModelRestrictions(policy: { supportedModels?: unknown; allowedRouteIds?: unknown }): boolean {
|
||||||
|
const supportedModels = Array.isArray(policy.supportedModels) ? policy.supportedModels : [];
|
||||||
|
const allowedRouteIds = Array.isArray(policy.allowedRouteIds) ? policy.allowedRouteIds : [];
|
||||||
|
return supportedModels.length > 0 || allowedRouteIds.length > 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
function extractGeminiListedModelName(item: unknown): string {
|
||||||
|
if (!item || typeof item !== 'object') return '';
|
||||||
|
const rawName = typeof (item as { name?: unknown }).name === 'string'
|
||||||
|
? (item as { name: string }).name.trim()
|
||||||
|
: '';
|
||||||
|
if (!rawName) return '';
|
||||||
|
return rawName.startsWith('models/') ? rawName.slice('models/'.length) : rawName;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function filterGeminiListedModelsForPolicy(
|
||||||
|
payload: unknown,
|
||||||
|
request: FastifyRequest,
|
||||||
|
): Promise<unknown> {
|
||||||
|
if (!payload || typeof payload !== 'object' || !Array.isArray((payload as { models?: unknown[] }).models)) {
|
||||||
|
return payload;
|
||||||
|
}
|
||||||
|
|
||||||
|
const policy = getDownstreamRoutingPolicy(request);
|
||||||
|
if (!hasDownstreamModelRestrictions(policy)) {
|
||||||
|
return payload;
|
||||||
|
}
|
||||||
|
|
||||||
|
const filteredModels: unknown[] = [];
|
||||||
|
for (const item of (payload as { models: unknown[] }).models) {
|
||||||
|
const modelName = extractGeminiListedModelName(item);
|
||||||
|
if (!modelName) continue;
|
||||||
|
if (!await isModelAllowedByPolicyOrAllowedRoutes(modelName, policy)) continue;
|
||||||
|
const decision = await tokenRouter.explainSelection?.(modelName, [], policy);
|
||||||
|
if (decision && typeof decision.selectedChannelId !== 'number') continue;
|
||||||
|
filteredModels.push(item);
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
...(payload as Record<string, unknown>),
|
||||||
|
models: filteredModels,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function logProxy(
|
||||||
|
selected: any,
|
||||||
|
modelRequested: string,
|
||||||
|
status: string,
|
||||||
|
httpStatus: number,
|
||||||
|
latencyMs: number,
|
||||||
|
errorMessage: string | null,
|
||||||
|
retryCount: number,
|
||||||
|
downstreamPath: string,
|
||||||
|
upstreamPath: string | null,
|
||||||
|
promptTokens = 0,
|
||||||
|
completionTokens = 0,
|
||||||
|
totalTokens = 0,
|
||||||
|
) {
|
||||||
|
try {
|
||||||
|
const createdAt = formatUtcSqlDateTime(new Date());
|
||||||
|
const normalizedErrorMessage = composeProxyLogMessage({
|
||||||
|
downstreamPath,
|
||||||
|
upstreamPath,
|
||||||
|
errorMessage,
|
||||||
|
});
|
||||||
|
await db.insert(schema.proxyLogs).values({
|
||||||
|
routeId: selected.channel.routeId,
|
||||||
|
channelId: selected.channel.id,
|
||||||
|
accountId: selected.account.id,
|
||||||
|
modelRequested,
|
||||||
|
modelActual: selected.actualModel || modelRequested,
|
||||||
|
status,
|
||||||
|
httpStatus,
|
||||||
|
latencyMs,
|
||||||
|
promptTokens,
|
||||||
|
completionTokens,
|
||||||
|
totalTokens,
|
||||||
|
estimatedCost: 0,
|
||||||
|
billingDetails: null,
|
||||||
|
errorMessage: normalizedErrorMessage,
|
||||||
|
retryCount,
|
||||||
|
createdAt,
|
||||||
|
}).run();
|
||||||
|
} catch (error) {
|
||||||
|
console.warn('[proxy/gemini] failed to write proxy log', error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
export async function geminiProxyRoute(app: FastifyInstance) {
|
export async function geminiProxyRoute(app: FastifyInstance) {
|
||||||
const listModels = async (request: FastifyRequest, reply: FastifyReply) => {
|
const listModels = async (request: FastifyRequest, reply: FastifyReply) => {
|
||||||
const apiVersion = geminiGenerateContentTransformer.resolveProxyApiVersion(
|
const apiVersion = geminiGenerateContentTransformer.resolveProxyApiVersion(
|
||||||
@@ -72,7 +182,9 @@ export async function geminiProxyRoute(app: FastifyInstance) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
return reply.code(upstream.status).send(JSON.parse(text));
|
const parsed = JSON.parse(text);
|
||||||
|
const filtered = await filterGeminiListedModelsForPolicy(parsed, request);
|
||||||
|
return reply.code(upstream.status).send(filtered);
|
||||||
} catch {
|
} catch {
|
||||||
return reply.code(upstream.status).type(upstream.headers.get('content-type') || 'application/json').send(text);
|
return reply.code(upstream.status).type(upstream.headers.get('content-type') || 'application/json').send(text);
|
||||||
}
|
}
|
||||||
@@ -107,6 +219,7 @@ export async function geminiProxyRoute(app: FastifyInstance) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const policy = getDownstreamRoutingPolicy(request);
|
const policy = getDownstreamRoutingPolicy(request);
|
||||||
|
const downstreamPath = resolveDownstreamPath(request);
|
||||||
const excludeChannelIds: number[] = [];
|
const excludeChannelIds: number[] = [];
|
||||||
let retryCount = 0;
|
let retryCount = 0;
|
||||||
let lastStatus = 503;
|
let lastStatus = 503;
|
||||||
@@ -132,7 +245,9 @@ export async function geminiProxyRoute(app: FastifyInstance) {
|
|||||||
/^models\/[^:]+/,
|
/^models\/[^:]+/,
|
||||||
`models/${selected.actualModel || requestedModel}`,
|
`models/${selected.actualModel || requestedModel}`,
|
||||||
);
|
);
|
||||||
|
const upstreamPath = resolveUpstreamPath(apiVersion, actualModelAction);
|
||||||
const query = new URLSearchParams(request.query as Record<string, string>).toString();
|
const query = new URLSearchParams(request.query as Record<string, string>).toString();
|
||||||
|
const startTime = Date.now();
|
||||||
try {
|
try {
|
||||||
const upstream = await fetch(
|
const upstream = await fetch(
|
||||||
geminiGenerateContentTransformer.resolveActionUrl(
|
geminiGenerateContentTransformer.resolveActionUrl(
|
||||||
@@ -156,6 +271,17 @@ export async function geminiProxyRoute(app: FastifyInstance) {
|
|||||||
lastContentType = contentType;
|
lastContentType = contentType;
|
||||||
lastText = await upstream.text();
|
lastText = await upstream.text();
|
||||||
await tokenRouter.recordFailure?.(selected.channel.id);
|
await tokenRouter.recordFailure?.(selected.channel.id);
|
||||||
|
await logProxy(
|
||||||
|
selected,
|
||||||
|
requestedModel,
|
||||||
|
'failed',
|
||||||
|
lastStatus,
|
||||||
|
Date.now() - startTime,
|
||||||
|
lastText,
|
||||||
|
retryCount,
|
||||||
|
downstreamPath,
|
||||||
|
upstreamPath,
|
||||||
|
);
|
||||||
if (retryCount < MAX_RETRIES) {
|
if (retryCount < MAX_RETRIES) {
|
||||||
retryCount += 1;
|
retryCount += 1;
|
||||||
continue;
|
continue;
|
||||||
@@ -174,6 +300,19 @@ export async function geminiProxyRoute(app: FastifyInstance) {
|
|||||||
reply.raw.setHeader('Content-Type', contentType || 'text/event-stream');
|
reply.raw.setHeader('Content-Type', contentType || 'text/event-stream');
|
||||||
const reader = upstream.body?.getReader();
|
const reader = upstream.body?.getReader();
|
||||||
if (!reader) {
|
if (!reader) {
|
||||||
|
const latency = Date.now() - startTime;
|
||||||
|
await tokenRouter.recordSuccess?.(selected.channel.id, latency, 0);
|
||||||
|
await logProxy(
|
||||||
|
selected,
|
||||||
|
requestedModel,
|
||||||
|
'success',
|
||||||
|
upstream.status,
|
||||||
|
latency,
|
||||||
|
null,
|
||||||
|
retryCount,
|
||||||
|
downstreamPath,
|
||||||
|
upstreamPath,
|
||||||
|
);
|
||||||
reply.raw.end();
|
reply.raw.end();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -209,21 +348,70 @@ export async function geminiProxyRoute(app: FastifyInstance) {
|
|||||||
reader.releaseLock();
|
reader.releaseLock();
|
||||||
reply.raw.end();
|
reply.raw.end();
|
||||||
}
|
}
|
||||||
|
const parsedUsage = parseProxyUsage(aggregateState);
|
||||||
|
const latency = Date.now() - startTime;
|
||||||
|
await tokenRouter.recordSuccess?.(selected.channel.id, latency, 0);
|
||||||
|
await logProxy(
|
||||||
|
selected,
|
||||||
|
requestedModel,
|
||||||
|
'success',
|
||||||
|
upstream.status,
|
||||||
|
latency,
|
||||||
|
null,
|
||||||
|
retryCount,
|
||||||
|
downstreamPath,
|
||||||
|
upstreamPath,
|
||||||
|
parsedUsage.promptTokens,
|
||||||
|
parsedUsage.completionTokens,
|
||||||
|
parsedUsage.totalTokens,
|
||||||
|
);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
const text = await upstream.text();
|
const text = await upstream.text();
|
||||||
|
const aggregateState = geminiGenerateContentTransformer.stream.createAggregateState();
|
||||||
|
let parsedUsage = EMPTY_PROXY_USAGE;
|
||||||
try {
|
try {
|
||||||
const parsed = JSON.parse(text);
|
const parsed = JSON.parse(text);
|
||||||
const aggregateState = geminiGenerateContentTransformer.stream.createAggregateState();
|
const responsePayload = geminiGenerateContentTransformer.stream.serializeUpstreamJsonPayload(
|
||||||
|
aggregateState,
|
||||||
|
parsed,
|
||||||
|
isStreamAction,
|
||||||
|
);
|
||||||
|
parsedUsage = parseProxyUsage(aggregateState);
|
||||||
|
const latency = Date.now() - startTime;
|
||||||
|
await tokenRouter.recordSuccess?.(selected.channel.id, latency, 0);
|
||||||
|
await logProxy(
|
||||||
|
selected,
|
||||||
|
requestedModel,
|
||||||
|
'success',
|
||||||
|
upstream.status,
|
||||||
|
latency,
|
||||||
|
null,
|
||||||
|
retryCount,
|
||||||
|
downstreamPath,
|
||||||
|
upstreamPath,
|
||||||
|
parsedUsage.promptTokens,
|
||||||
|
parsedUsage.completionTokens,
|
||||||
|
parsedUsage.totalTokens,
|
||||||
|
);
|
||||||
return reply.code(upstream.status).send(
|
return reply.code(upstream.status).send(
|
||||||
geminiGenerateContentTransformer.stream.serializeUpstreamJsonPayload(
|
responsePayload,
|
||||||
aggregateState,
|
|
||||||
parsed,
|
|
||||||
isStreamAction,
|
|
||||||
),
|
|
||||||
);
|
);
|
||||||
} catch {
|
} catch {
|
||||||
|
const latency = Date.now() - startTime;
|
||||||
|
await tokenRouter.recordSuccess?.(selected.channel.id, latency, 0);
|
||||||
|
await logProxy(
|
||||||
|
selected,
|
||||||
|
requestedModel,
|
||||||
|
'success',
|
||||||
|
upstream.status,
|
||||||
|
latency,
|
||||||
|
null,
|
||||||
|
retryCount,
|
||||||
|
downstreamPath,
|
||||||
|
upstreamPath,
|
||||||
|
);
|
||||||
return reply.code(upstream.status).type(contentType || 'application/json').send(text);
|
return reply.code(upstream.status).type(contentType || 'application/json').send(text);
|
||||||
}
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
@@ -236,6 +424,17 @@ export async function geminiProxyRoute(app: FastifyInstance) {
|
|||||||
},
|
},
|
||||||
});
|
});
|
||||||
await tokenRouter.recordFailure?.(selected.channel.id);
|
await tokenRouter.recordFailure?.(selected.channel.id);
|
||||||
|
await logProxy(
|
||||||
|
selected,
|
||||||
|
requestedModel,
|
||||||
|
'failed',
|
||||||
|
0,
|
||||||
|
Date.now() - startTime,
|
||||||
|
error instanceof Error ? error.message : 'Gemini upstream request failed',
|
||||||
|
retryCount,
|
||||||
|
downstreamPath,
|
||||||
|
upstreamPath,
|
||||||
|
);
|
||||||
if (retryCount < MAX_RETRIES) {
|
if (retryCount < MAX_RETRIES) {
|
||||||
retryCount += 1;
|
retryCount += 1;
|
||||||
continue;
|
continue;
|
||||||
|
|||||||
@@ -12,6 +12,61 @@ const MODEL_DISCOVERY_TIMEOUT_MS = 12_000;
|
|||||||
const MODEL_REFRESH_BATCH_SIZE = 3;
|
const MODEL_REFRESH_BATCH_SIZE = 3;
|
||||||
|
|
||||||
type ModelRefreshErrorCode = 'timeout' | 'unauthorized' | 'empty_models' | 'unknown';
|
type ModelRefreshErrorCode = 'timeout' | 'unauthorized' | 'empty_models' | 'unknown';
|
||||||
|
type ModelRefreshSkipCode = 'site_disabled' | 'adapter_or_status';
|
||||||
|
|
||||||
|
export type ModelRefreshAccountNotFoundResult = {
|
||||||
|
accountId: number;
|
||||||
|
refreshed: false;
|
||||||
|
status: 'failed';
|
||||||
|
errorCode: 'account_not_found';
|
||||||
|
errorMessage: '账号不存在';
|
||||||
|
modelCount: 0;
|
||||||
|
modelsPreview: string[];
|
||||||
|
reason: 'account_not_found';
|
||||||
|
};
|
||||||
|
|
||||||
|
export type ModelRefreshSkippedResult = {
|
||||||
|
accountId: number;
|
||||||
|
refreshed: false;
|
||||||
|
status: 'skipped';
|
||||||
|
errorCode: ModelRefreshSkipCode;
|
||||||
|
errorMessage: string;
|
||||||
|
modelCount: 0;
|
||||||
|
modelsPreview: string[];
|
||||||
|
reason: ModelRefreshSkipCode;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type ModelRefreshFailureResult = {
|
||||||
|
accountId: number;
|
||||||
|
refreshed: true;
|
||||||
|
status: 'failed';
|
||||||
|
errorCode: ModelRefreshErrorCode;
|
||||||
|
errorMessage: string;
|
||||||
|
modelCount: 0;
|
||||||
|
modelsPreview: string[];
|
||||||
|
tokenScanned: number;
|
||||||
|
discoveredByCredential: boolean;
|
||||||
|
discoveredApiToken: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type ModelRefreshSuccessResult = {
|
||||||
|
accountId: number;
|
||||||
|
refreshed: true;
|
||||||
|
status: 'success';
|
||||||
|
errorCode: null;
|
||||||
|
errorMessage: '';
|
||||||
|
modelCount: number;
|
||||||
|
modelsPreview: string[];
|
||||||
|
tokenScanned: number;
|
||||||
|
discoveredByCredential: boolean;
|
||||||
|
discoveredApiToken: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type ModelRefreshResult =
|
||||||
|
| ModelRefreshAccountNotFoundResult
|
||||||
|
| ModelRefreshSkippedResult
|
||||||
|
| ModelRefreshFailureResult
|
||||||
|
| ModelRefreshSuccessResult;
|
||||||
|
|
||||||
function classifyModelDiscoveryError(message: string): ModelRefreshErrorCode {
|
function classifyModelDiscoveryError(message: string): ModelRefreshErrorCode {
|
||||||
const lowered = message.toLowerCase();
|
const lowered = message.toLowerCase();
|
||||||
@@ -64,23 +119,88 @@ async function withTimeout<T>(fn: () => Promise<T>, timeoutMs: number, timeoutMe
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function refreshModelsForAccount(accountId: number) {
|
function buildAccountNotFoundRefreshResult(accountId: number): ModelRefreshAccountNotFoundResult {
|
||||||
|
return {
|
||||||
|
accountId,
|
||||||
|
refreshed: false,
|
||||||
|
status: 'failed',
|
||||||
|
errorCode: 'account_not_found',
|
||||||
|
errorMessage: '账号不存在',
|
||||||
|
modelCount: 0,
|
||||||
|
modelsPreview: [],
|
||||||
|
reason: 'account_not_found',
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildSkippedRefreshResult(
|
||||||
|
accountId: number,
|
||||||
|
code: ModelRefreshSkipCode,
|
||||||
|
errorMessage: string,
|
||||||
|
): ModelRefreshSkippedResult {
|
||||||
|
return {
|
||||||
|
accountId,
|
||||||
|
refreshed: false,
|
||||||
|
status: 'skipped',
|
||||||
|
errorCode: code,
|
||||||
|
errorMessage,
|
||||||
|
modelCount: 0,
|
||||||
|
modelsPreview: [],
|
||||||
|
reason: code,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildFailedRefreshResult(input: {
|
||||||
|
accountId: number;
|
||||||
|
errorCode: ModelRefreshErrorCode;
|
||||||
|
errorMessage: string;
|
||||||
|
tokenScanned: number;
|
||||||
|
discoveredByCredential: boolean;
|
||||||
|
discoveredApiToken: boolean;
|
||||||
|
}): ModelRefreshFailureResult {
|
||||||
|
return {
|
||||||
|
accountId: input.accountId,
|
||||||
|
refreshed: true,
|
||||||
|
status: 'failed',
|
||||||
|
errorCode: input.errorCode,
|
||||||
|
errorMessage: input.errorMessage,
|
||||||
|
modelCount: 0,
|
||||||
|
modelsPreview: [],
|
||||||
|
tokenScanned: input.tokenScanned,
|
||||||
|
discoveredByCredential: input.discoveredByCredential,
|
||||||
|
discoveredApiToken: input.discoveredApiToken,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildSuccessfulRefreshResult(input: {
|
||||||
|
accountId: number;
|
||||||
|
modelCount: number;
|
||||||
|
modelsPreview: string[];
|
||||||
|
tokenScanned: number;
|
||||||
|
discoveredByCredential: boolean;
|
||||||
|
discoveredApiToken: boolean;
|
||||||
|
}): ModelRefreshSuccessResult {
|
||||||
|
return {
|
||||||
|
accountId: input.accountId,
|
||||||
|
refreshed: true,
|
||||||
|
status: 'success',
|
||||||
|
errorCode: null,
|
||||||
|
errorMessage: '',
|
||||||
|
modelCount: input.modelCount,
|
||||||
|
modelsPreview: input.modelsPreview,
|
||||||
|
tokenScanned: input.tokenScanned,
|
||||||
|
discoveredByCredential: input.discoveredByCredential,
|
||||||
|
discoveredApiToken: input.discoveredApiToken,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function refreshModelsForAccount(accountId: number): Promise<ModelRefreshResult> {
|
||||||
const row = await db.select().from(schema.accounts)
|
const row = await db.select().from(schema.accounts)
|
||||||
.innerJoin(schema.sites, eq(schema.accounts.siteId, schema.sites.id))
|
.innerJoin(schema.sites, eq(schema.accounts.siteId, schema.sites.id))
|
||||||
.where(eq(schema.accounts.id, accountId))
|
.where(eq(schema.accounts.id, accountId))
|
||||||
.get();
|
.get();
|
||||||
|
|
||||||
if (!row) {
|
if (!row) {
|
||||||
return {
|
return buildAccountNotFoundRefreshResult(accountId);
|
||||||
accountId,
|
|
||||||
refreshed: false,
|
|
||||||
status: 'failed',
|
|
||||||
errorCode: 'account_not_found',
|
|
||||||
errorMessage: '账号不存在',
|
|
||||||
modelCount: 0,
|
|
||||||
modelsPreview: [],
|
|
||||||
reason: 'account_not_found',
|
|
||||||
};
|
|
||||||
}
|
}
|
||||||
|
|
||||||
const account = row.accounts;
|
const account = row.accounts;
|
||||||
@@ -103,29 +223,11 @@ export async function refreshModelsForAccount(accountId: number) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if (isSiteDisabled(site.status)) {
|
if (isSiteDisabled(site.status)) {
|
||||||
return {
|
return buildSkippedRefreshResult(accountId, 'site_disabled', '站点已禁用');
|
||||||
accountId,
|
|
||||||
refreshed: false,
|
|
||||||
status: 'skipped',
|
|
||||||
errorCode: 'site_disabled',
|
|
||||||
errorMessage: '站点已禁用',
|
|
||||||
modelCount: 0,
|
|
||||||
modelsPreview: [],
|
|
||||||
reason: 'site_disabled',
|
|
||||||
};
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!adapter || account.status !== 'active') {
|
if (!adapter || account.status !== 'active') {
|
||||||
return {
|
return buildSkippedRefreshResult(accountId, 'adapter_or_status', '平台不可用或账号未激活');
|
||||||
accountId,
|
|
||||||
refreshed: false,
|
|
||||||
status: 'skipped',
|
|
||||||
errorCode: 'adapter_or_status',
|
|
||||||
errorMessage: '平台不可用或账号未激活',
|
|
||||||
modelCount: 0,
|
|
||||||
modelsPreview: [],
|
|
||||||
reason: 'adapter_or_status',
|
|
||||||
};
|
|
||||||
}
|
}
|
||||||
|
|
||||||
const platformUserId = resolvePlatformUserId(account.extraConfig, account.username);
|
const platformUserId = resolvePlatformUserId(account.extraConfig, account.username);
|
||||||
@@ -266,18 +368,14 @@ export async function refreshModelsForAccount(accountId: number) {
|
|||||||
source: 'model-discovery',
|
source: 'model-discovery',
|
||||||
checkedAt: new Date().toISOString(),
|
checkedAt: new Date().toISOString(),
|
||||||
});
|
});
|
||||||
return {
|
return buildFailedRefreshResult({
|
||||||
accountId,
|
accountId,
|
||||||
refreshed: true,
|
|
||||||
status: 'failed',
|
|
||||||
errorCode,
|
errorCode,
|
||||||
errorMessage,
|
errorMessage,
|
||||||
modelCount: 0,
|
|
||||||
modelsPreview: [],
|
|
||||||
tokenScanned: scannedTokenCount,
|
tokenScanned: scannedTokenCount,
|
||||||
discoveredByCredential,
|
discoveredByCredential,
|
||||||
discoveredApiToken: !!discoveredApiToken,
|
discoveredApiToken: !!discoveredApiToken,
|
||||||
};
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
const checkedAt = new Date().toISOString();
|
const checkedAt = new Date().toISOString();
|
||||||
@@ -299,26 +397,22 @@ export async function refreshModelsForAccount(accountId: number) {
|
|||||||
});
|
});
|
||||||
|
|
||||||
const modelsPreview = Array.from(accountModels).slice(0, 10);
|
const modelsPreview = Array.from(accountModels).slice(0, 10);
|
||||||
return {
|
return buildSuccessfulRefreshResult({
|
||||||
accountId,
|
accountId,
|
||||||
refreshed: true,
|
|
||||||
status: 'success',
|
|
||||||
errorCode: null,
|
|
||||||
errorMessage: '',
|
|
||||||
modelCount: accountModels.size,
|
modelCount: accountModels.size,
|
||||||
modelsPreview,
|
modelsPreview,
|
||||||
tokenScanned: scannedTokenCount,
|
tokenScanned: scannedTokenCount,
|
||||||
discoveredByCredential,
|
discoveredByCredential,
|
||||||
discoveredApiToken: !!discoveredApiToken,
|
discoveredApiToken: !!discoveredApiToken,
|
||||||
};
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
async function refreshModelsForAllActiveAccounts() {
|
async function refreshModelsForAllActiveAccounts(): Promise<ModelRefreshResult[]> {
|
||||||
const accounts = await db.select({ id: schema.accounts.id }).from(schema.accounts)
|
const accounts = await db.select({ id: schema.accounts.id }).from(schema.accounts)
|
||||||
.where(eq(schema.accounts.status, 'active'))
|
.where(eq(schema.accounts.status, 'active'))
|
||||||
.all();
|
.all();
|
||||||
|
|
||||||
const results: any[] = [];
|
const results: ModelRefreshResult[] = [];
|
||||||
for (let offset = 0; offset < accounts.length; offset += MODEL_REFRESH_BATCH_SIZE) {
|
for (let offset = 0; offset < accounts.length; offset += MODEL_REFRESH_BATCH_SIZE) {
|
||||||
const batch = accounts.slice(offset, offset + MODEL_REFRESH_BATCH_SIZE);
|
const batch = accounts.slice(offset, offset + MODEL_REFRESH_BATCH_SIZE);
|
||||||
const batchResults = await Promise.all(batch.map(async (account) => refreshModelsForAccount(account.id)));
|
const batchResults = await Promise.all(batch.map(async (account) => refreshModelsForAccount(account.id)));
|
||||||
|
|||||||
@@ -154,4 +154,31 @@ describe('TokenRouter patterns and model mapping', () => {
|
|||||||
expect(decision.actualModel).toBe('claude-opus-4-5');
|
expect(decision.actualModel).toBe('claude-opus-4-5');
|
||||||
expect(exposedModels).toContain('claude-opus-4-6');
|
expect(exposedModels).toContain('claude-opus-4-6');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('prefers an exact route over a colliding group display-name alias', async () => {
|
||||||
|
await createRouteWithSingleChannel(
|
||||||
|
're:^claude-(opus|sonnet)-4-5$',
|
||||||
|
undefined,
|
||||||
|
{
|
||||||
|
displayName: 'claude-opus-4-6',
|
||||||
|
sourceModel: 'claude-opus-4-5',
|
||||||
|
},
|
||||||
|
);
|
||||||
|
const exact = await createRouteWithSingleChannel(
|
||||||
|
'claude-opus-4-6',
|
||||||
|
undefined,
|
||||||
|
{
|
||||||
|
sourceModel: 'claude-opus-4-6',
|
||||||
|
},
|
||||||
|
);
|
||||||
|
const router = new TokenRouter();
|
||||||
|
|
||||||
|
const selected = await router.selectChannel('claude-opus-4-6');
|
||||||
|
const decision = await router.explainSelection('claude-opus-4-6');
|
||||||
|
|
||||||
|
expect(selected).toBeTruthy();
|
||||||
|
expect(selected?.channel.id).toBe(exact.channel.id);
|
||||||
|
expect(selected?.actualModel).toBe('claude-opus-4-6');
|
||||||
|
expect(decision.actualModel).toBe('claude-opus-4-6');
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -271,6 +271,13 @@ export function matchesModelPattern(model: string, pattern: string): boolean {
|
|||||||
return minimatch(model, normalizedPattern);
|
return minimatch(model, normalizedPattern);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function isExactRouteModelPattern(pattern: string): boolean {
|
||||||
|
const normalizedPattern = (pattern || '').trim();
|
||||||
|
if (!normalizedPattern) return false;
|
||||||
|
if (isRegexModelPattern(normalizedPattern)) return false;
|
||||||
|
return !/[\*\?\[]/.test(normalizedPattern);
|
||||||
|
}
|
||||||
|
|
||||||
function normalizeRouteDisplayName(displayName: string | null | undefined): string {
|
function normalizeRouteDisplayName(displayName: string | null | undefined): string {
|
||||||
return (displayName || '').trim();
|
return (displayName || '').trim();
|
||||||
}
|
}
|
||||||
@@ -993,10 +1000,12 @@ export class TokenRouter {
|
|||||||
routes = routes.filter((route) => allowSet.has(route.id));
|
routes = routes.filter((route) => allowSet.has(route.id));
|
||||||
}
|
}
|
||||||
|
|
||||||
// Find matching route by model pattern or display alias.
|
const matchedRoute = routes.find((route) => (
|
||||||
const matchedRoute = routes.find((r) => {
|
isExactRouteModelPattern(route.modelPattern)
|
||||||
return matchesRouteRequestModel(model, r);
|
&& (route.modelPattern || '').trim() === model
|
||||||
});
|
))
|
||||||
|
|| routes.find((route) => isRouteDisplayNameMatch(model, route.displayName))
|
||||||
|
|| routes.find((route) => matchesModelPattern(model, route.modelPattern));
|
||||||
|
|
||||||
if (!matchedRoute) return null;
|
if (!matchedRoute) return null;
|
||||||
|
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ import {
|
|||||||
buildApiPayload,
|
buildApiPayload,
|
||||||
buildEmbeddingsRequestEnvelope,
|
buildEmbeddingsRequestEnvelope,
|
||||||
buildFileUploadRequestEnvelope,
|
buildFileUploadRequestEnvelope,
|
||||||
|
buildGeminiNativeConversationProxyEnvelope,
|
||||||
buildImagesEditRequestEnvelope,
|
buildImagesEditRequestEnvelope,
|
||||||
buildImagesGenerationsRequestEnvelope,
|
buildImagesGenerationsRequestEnvelope,
|
||||||
buildRawProxyRequestEnvelope,
|
buildRawProxyRequestEnvelope,
|
||||||
@@ -1027,30 +1028,6 @@ export default function ModelTester() {
|
|||||||
};
|
};
|
||||||
}, [buildConversationMessagesWithSystem, inputs.max_tokens, inputs.model, inputs.stream, inputs.temperature, inputs.top_p, parameterEnabled.max_tokens, parameterEnabled.temperature, parameterEnabled.top_p]);
|
}, [buildConversationMessagesWithSystem, inputs.max_tokens, inputs.model, inputs.stream, inputs.temperature, inputs.top_p, parameterEnabled.max_tokens, parameterEnabled.temperature, parameterEnabled.top_p]);
|
||||||
|
|
||||||
const buildGeminiBodyFromMessages = useCallback((baseMessages: ChatMessage[]) => {
|
|
||||||
const effectiveMessages = buildConversationMessagesWithSystem(baseMessages);
|
|
||||||
const systemContents = effectiveMessages
|
|
||||||
.filter((message) => message.role === 'system' || message.role === 'developer')
|
|
||||||
.map((message) => message.content.trim())
|
|
||||||
.filter(Boolean);
|
|
||||||
const contents = effectiveMessages
|
|
||||||
.filter((message) => message.role !== 'system' && message.role !== 'developer')
|
|
||||||
.map((message) => ({
|
|
||||||
role: message.role === 'assistant' ? 'model' : 'user',
|
|
||||||
parts: [{ text: message.content }],
|
|
||||||
}));
|
|
||||||
|
|
||||||
return {
|
|
||||||
...(systemContents.length > 0 ? { systemInstruction: { parts: [{ text: systemContents.join('\n\n') }] } } : {}),
|
|
||||||
contents,
|
|
||||||
generationConfig: {
|
|
||||||
...(parameterEnabled.temperature ? { temperature: inputs.temperature } : {}),
|
|
||||||
...(parameterEnabled.top_p ? { topP: inputs.top_p } : {}),
|
|
||||||
...(parameterEnabled.max_tokens ? { maxOutputTokens: inputs.max_tokens } : {}),
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}, [buildConversationMessagesWithSystem, inputs.max_tokens, inputs.temperature, inputs.top_p, parameterEnabled.max_tokens, parameterEnabled.temperature, parameterEnabled.top_p]);
|
|
||||||
|
|
||||||
const buildConversationProxyEnvelope = useCallback((baseMessages: ChatMessage[]): ProxyTestEnvelope => {
|
const buildConversationProxyEnvelope = useCallback((baseMessages: ChatMessage[]): ProxyTestEnvelope => {
|
||||||
const normalizedMessages = buildConversationMessagesWithSystem(baseMessages);
|
const normalizedMessages = buildConversationMessagesWithSystem(baseMessages);
|
||||||
|
|
||||||
@@ -1099,15 +1076,7 @@ export default function ModelTester() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if (inputs.protocol === 'gemini') {
|
if (inputs.protocol === 'gemini') {
|
||||||
return {
|
return buildGeminiNativeConversationProxyEnvelope(baseMessages, inputs, parameterEnabled);
|
||||||
method: 'POST',
|
|
||||||
path: `/gemini/v1beta/models/${encodeURIComponent(inputs.model)}:generateContent${inputs.stream ? '?alt=sse' : ''}`,
|
|
||||||
requestKind: 'json',
|
|
||||||
stream: inputs.stream,
|
|
||||||
jobMode: false,
|
|
||||||
rawMode: false,
|
|
||||||
jsonBody: buildGeminiBodyFromMessages(baseMessages),
|
|
||||||
};
|
|
||||||
}
|
}
|
||||||
|
|
||||||
const openAiEnvelope = buildApiPayload(normalizedMessages, { ...inputs, protocol: 'openai' }, parameterEnabled);
|
const openAiEnvelope = buildApiPayload(normalizedMessages, { ...inputs, protocol: 'openai' }, parameterEnabled);
|
||||||
@@ -1124,7 +1093,7 @@ export default function ModelTester() {
|
|||||||
rawMode: false,
|
rawMode: false,
|
||||||
jsonBody: openAiPayload,
|
jsonBody: openAiPayload,
|
||||||
};
|
};
|
||||||
}, [buildApiPayload, buildClaudeBodyFromMessages, buildConversationMessagesWithSystem, buildGeminiBodyFromMessages, buildResponsesBodyFromMessages, customRequestBody, customRequestMode, inputs, parameterEnabled]);
|
}, [buildApiPayload, buildClaudeBodyFromMessages, buildConversationMessagesWithSystem, buildResponsesBodyFromMessages, customRequestBody, customRequestMode, inputs, parameterEnabled]);
|
||||||
|
|
||||||
const buildModeProxyEnvelope = useCallback((): ProxyTestEnvelope | null => {
|
const buildModeProxyEnvelope = useCallback((): ProxyTestEnvelope | null => {
|
||||||
if (inputs.mode === 'embeddings') {
|
if (inputs.mode === 'embeddings') {
|
||||||
|
|||||||
@@ -32,6 +32,7 @@ import type {
|
|||||||
RouteDecision,
|
RouteDecision,
|
||||||
RouteIconOption,
|
RouteIconOption,
|
||||||
MissingTokenRouteSiteActionItem,
|
MissingTokenRouteSiteActionItem,
|
||||||
|
MissingTokenGroupRouteSiteActionItem,
|
||||||
GroupRouteItem,
|
GroupRouteItem,
|
||||||
} from './token-routes/types.js';
|
} from './token-routes/types.js';
|
||||||
import {
|
import {
|
||||||
@@ -59,6 +60,7 @@ const EMPTY_ROUTE_CANDIDATE_VIEW: RouteCandidateView = {
|
|||||||
tokenOptionsByAccountId: {},
|
tokenOptionsByAccountId: {},
|
||||||
};
|
};
|
||||||
const EMPTY_MISSING_ITEMS: MissingTokenRouteSiteActionItem[] = [];
|
const EMPTY_MISSING_ITEMS: MissingTokenRouteSiteActionItem[] = [];
|
||||||
|
const EMPTY_MISSING_GROUP_ITEMS: MissingTokenGroupRouteSiteActionItem[] = [];
|
||||||
const ROUTE_ICON_OPTIONS: RouteIconOption[] = [
|
const ROUTE_ICON_OPTIONS: RouteIconOption[] = [
|
||||||
{ value: '', label: '自动品牌图标', description: '按模型匹配规则自动识别品牌', iconText: '✦' },
|
{ value: '', label: '自动品牌图标', description: '按模型匹配规则自动识别品牌', iconText: '✦' },
|
||||||
];
|
];
|
||||||
@@ -68,6 +70,7 @@ export default function TokenRoutes() {
|
|||||||
const [routeSummaries, setRouteSummaries] = useState<RouteSummaryRow[]>([]);
|
const [routeSummaries, setRouteSummaries] = useState<RouteSummaryRow[]>([]);
|
||||||
const [modelCandidates, setModelCandidates] = useState<RouteModelCandidatesByModelName>({});
|
const [modelCandidates, setModelCandidates] = useState<RouteModelCandidatesByModelName>({});
|
||||||
const [missingTokenModelsByName, setMissingTokenModelsByName] = useState<MissingTokenModelsByName>({});
|
const [missingTokenModelsByName, setMissingTokenModelsByName] = useState<MissingTokenModelsByName>({});
|
||||||
|
const [missingTokenGroupModelsByName, setMissingTokenGroupModelsByName] = useState<MissingTokenModelsByName>({});
|
||||||
const [endpointTypesByModel, setEndpointTypesByModel] = useState<Record<string, string[]>>({});
|
const [endpointTypesByModel, setEndpointTypesByModel] = useState<Record<string, string[]>>({});
|
||||||
|
|
||||||
const [search, setSearch] = useState('');
|
const [search, setSearch] = useState('');
|
||||||
@@ -189,6 +192,9 @@ export default function TokenRoutes() {
|
|||||||
setMissingTokenModelsByName(
|
setMissingTokenModelsByName(
|
||||||
normalizeMissingTokenModels((candidateRows?.modelsWithoutToken || {}) as MissingTokenModelsByName),
|
normalizeMissingTokenModels((candidateRows?.modelsWithoutToken || {}) as MissingTokenModelsByName),
|
||||||
);
|
);
|
||||||
|
setMissingTokenGroupModelsByName(
|
||||||
|
normalizeMissingTokenModels((candidateRows?.modelsMissingTokenGroups || {}) as MissingTokenModelsByName),
|
||||||
|
);
|
||||||
setEndpointTypesByModel(candidateRows?.endpointTypesByModel || {});
|
setEndpointTypesByModel(candidateRows?.endpointTypesByModel || {});
|
||||||
const decisionPlaceholder: Record<number, RouteDecision | null> = {};
|
const decisionPlaceholder: Record<number, RouteDecision | null> = {};
|
||||||
for (const route of summaries) {
|
for (const route of summaries) {
|
||||||
@@ -586,7 +592,12 @@ export default function TokenRoutes() {
|
|||||||
|
|
||||||
if (search.trim()) {
|
if (search.trim()) {
|
||||||
const q = search.trim().toLowerCase();
|
const q = search.trim().toLowerCase();
|
||||||
list = list.filter((route) => route.modelPattern.toLowerCase().includes(q));
|
list = list.filter((route) => {
|
||||||
|
const modelPattern = route.modelPattern.toLowerCase();
|
||||||
|
const displayName = (route.displayName || '').toLowerCase();
|
||||||
|
const title = resolveRouteTitle(route).toLowerCase();
|
||||||
|
return modelPattern.includes(q) || displayName.includes(q) || title.includes(q);
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
return list;
|
return list;
|
||||||
@@ -636,6 +647,10 @@ export default function TokenRoutes() {
|
|||||||
() => buildRouteMissingTokenIndex(routePatterns, missingTokenModelsByName, matchesModelPattern),
|
() => buildRouteMissingTokenIndex(routePatterns, missingTokenModelsByName, matchesModelPattern),
|
||||||
[routePatterns, missingTokenModelsByName],
|
[routePatterns, missingTokenModelsByName],
|
||||||
);
|
);
|
||||||
|
const routeMissingTokenGroupIndex = useMemo(
|
||||||
|
() => buildRouteMissingTokenIndex(routePatterns, missingTokenGroupModelsByName, matchesModelPattern),
|
||||||
|
[routePatterns, missingTokenGroupModelsByName],
|
||||||
|
);
|
||||||
|
|
||||||
const getRouteCandidateView = (routeId: number): RouteCandidateView => {
|
const getRouteCandidateView = (routeId: number): RouteCandidateView => {
|
||||||
return routeModelCandidateIndex[routeId] || EMPTY_ROUTE_CANDIDATE_VIEW;
|
return routeModelCandidateIndex[routeId] || EMPTY_ROUTE_CANDIDATE_VIEW;
|
||||||
@@ -782,6 +797,56 @@ export default function TokenRoutes() {
|
|||||||
return result;
|
return result;
|
||||||
}, [routeMissingTokenIndex]);
|
}, [routeMissingTokenIndex]);
|
||||||
|
|
||||||
|
const missingTokenGroupItemsByRouteId = useMemo(() => {
|
||||||
|
const result: Record<number, MissingTokenGroupRouteSiteActionItem[]> = {};
|
||||||
|
for (const routeId of Object.keys(routeMissingTokenGroupIndex).map(Number)) {
|
||||||
|
const missingGroupHints = routeMissingTokenGroupIndex[routeId] || [];
|
||||||
|
const siteMap = new Map<string, MissingTokenGroupRouteSiteActionItem>();
|
||||||
|
for (const hint of missingGroupHints) {
|
||||||
|
for (const account of hint.accounts) {
|
||||||
|
if (!Number.isFinite(account.accountId) || account.accountId <= 0) continue;
|
||||||
|
const siteName = (account.siteName || '').trim() || `site-${account.siteId || 'unknown'}`;
|
||||||
|
const key = `${account.siteId || 0}::${siteName.toLowerCase()}`;
|
||||||
|
const accountLabel = account.username || `account-${account.accountId}`;
|
||||||
|
const missingGroups = Array.isArray(account.missingGroups) ? account.missingGroups : [];
|
||||||
|
const requiredGroups = Array.isArray(account.requiredGroups) ? account.requiredGroups : [];
|
||||||
|
const availableGroups = Array.isArray(account.availableGroups) ? account.availableGroups : [];
|
||||||
|
const existing = siteMap.get(key);
|
||||||
|
if (!existing) {
|
||||||
|
siteMap.set(key, {
|
||||||
|
key,
|
||||||
|
siteName,
|
||||||
|
accountId: account.accountId,
|
||||||
|
accountLabel,
|
||||||
|
missingGroups: [...missingGroups],
|
||||||
|
requiredGroups: [...requiredGroups],
|
||||||
|
availableGroups: [...availableGroups],
|
||||||
|
...(account.groupCoverageUncertain === true ? { groupCoverageUncertain: true } : {}),
|
||||||
|
});
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (account.accountId < existing.accountId) {
|
||||||
|
existing.accountId = account.accountId;
|
||||||
|
existing.accountLabel = accountLabel;
|
||||||
|
}
|
||||||
|
existing.missingGroups = Array.from(new Set([...existing.missingGroups, ...missingGroups]))
|
||||||
|
.sort((a, b) => a.localeCompare(b, undefined, { sensitivity: 'base' }));
|
||||||
|
existing.requiredGroups = Array.from(new Set([...existing.requiredGroups, ...requiredGroups]))
|
||||||
|
.sort((a, b) => a.localeCompare(b, undefined, { sensitivity: 'base' }));
|
||||||
|
existing.availableGroups = Array.from(new Set([...existing.availableGroups, ...availableGroups]))
|
||||||
|
.sort((a, b) => a.localeCompare(b, undefined, { sensitivity: 'base' }));
|
||||||
|
if (account.groupCoverageUncertain === true) {
|
||||||
|
existing.groupCoverageUncertain = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result[routeId] = Array.from(siteMap.values()).sort((a, b) => (
|
||||||
|
a.siteName.localeCompare(b.siteName, undefined, { sensitivity: 'base' })
|
||||||
|
));
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}, [routeMissingTokenGroupIndex]);
|
||||||
|
|
||||||
// Stable callbacks for RouteCard memo (use refs to avoid dependency on closure variables)
|
// Stable callbacks for RouteCard memo (use refs to avoid dependency on closure variables)
|
||||||
const toggleExpandRef = useRef(toggleExpand);
|
const toggleExpandRef = useRef(toggleExpand);
|
||||||
toggleExpandRef.current = toggleExpand;
|
toggleExpandRef.current = toggleExpand;
|
||||||
@@ -1075,6 +1140,7 @@ export default function TokenRoutes() {
|
|||||||
onDeleteChannel={stableDeleteChannel}
|
onDeleteChannel={stableDeleteChannel}
|
||||||
onChannelDragEnd={stableChannelDragEnd}
|
onChannelDragEnd={stableChannelDragEnd}
|
||||||
missingTokenSiteItems={missingTokenSiteItemsByRouteId[route.id] || EMPTY_MISSING_ITEMS}
|
missingTokenSiteItems={missingTokenSiteItemsByRouteId[route.id] || EMPTY_MISSING_ITEMS}
|
||||||
|
missingTokenGroupItems={missingTokenGroupItemsByRouteId[route.id] || EMPTY_MISSING_GROUP_ITEMS}
|
||||||
onCreateTokenForMissing={stableCreateTokenForMissing}
|
onCreateTokenForMissing={stableCreateTokenForMissing}
|
||||||
onAddChannel={stableAddChannel}
|
onAddChannel={stableAddChannel}
|
||||||
expandedSourceGroupMap={expandedSourceGroupMap}
|
expandedSourceGroupMap={expandedSourceGroupMap}
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ import {
|
|||||||
buildApiPayload,
|
buildApiPayload,
|
||||||
buildEmbeddingsRequestEnvelope,
|
buildEmbeddingsRequestEnvelope,
|
||||||
buildFileUploadRequestEnvelope,
|
buildFileUploadRequestEnvelope,
|
||||||
|
buildGeminiNativeConversationProxyEnvelope,
|
||||||
buildRawProxyRequestEnvelope,
|
buildRawProxyRequestEnvelope,
|
||||||
buildSearchRequestEnvelope,
|
buildSearchRequestEnvelope,
|
||||||
collectModelTesterModelNames,
|
collectModelTesterModelNames,
|
||||||
@@ -321,6 +322,39 @@ describe('modelTesterSession', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('builds gemini native proxy envelope without loading assistant placeholders', () => {
|
||||||
|
const payload = buildGeminiNativeConversationProxyEnvelope(
|
||||||
|
[
|
||||||
|
{ id: 'u1', role: 'user', content: 'hello', createAt: 1 },
|
||||||
|
{ id: 'a1', role: 'assistant', content: '', createAt: 2, status: MESSAGE_STATUS.LOADING },
|
||||||
|
{ id: 'a2', role: 'assistant', content: 'partial', createAt: 3, status: MESSAGE_STATUS.INCOMPLETE },
|
||||||
|
{ id: 'a3', role: 'assistant', content: 'done', createAt: 4, status: MESSAGE_STATUS.COMPLETE },
|
||||||
|
],
|
||||||
|
{
|
||||||
|
...DEFAULT_INPUTS,
|
||||||
|
model: 'gemini-2.5-pro',
|
||||||
|
protocol: 'gemini',
|
||||||
|
systemPrompt: 'system text',
|
||||||
|
stream: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
...DEFAULT_PARAMETER_ENABLED,
|
||||||
|
max_tokens: true,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(payload.path).toBe('/gemini/v1beta/models/gemini-2.5-pro:generateContent?alt=sse');
|
||||||
|
expect(payload.jobMode).toBe(false);
|
||||||
|
expect(payload.jsonBody).toEqual({
|
||||||
|
systemInstruction: { parts: [{ text: 'system text' }] },
|
||||||
|
contents: [
|
||||||
|
{ role: 'user', parts: [{ text: 'hello' }] },
|
||||||
|
{ role: 'model', parts: [{ text: 'done' }] },
|
||||||
|
],
|
||||||
|
generationConfig: { temperature: 0.7, maxOutputTokens: 4096 },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
it('serializes conversation file parts into openai and responses payloads', () => {
|
it('serializes conversation file parts into openai and responses payloads', () => {
|
||||||
const message = {
|
const message = {
|
||||||
id: 'u1',
|
id: 'u1',
|
||||||
|
|||||||
@@ -949,6 +949,19 @@ export const buildConversationRequestEnvelope = (
|
|||||||
jsonBody: buildConversationJsonBody(messages, inputs, parameterEnabled),
|
jsonBody: buildConversationJsonBody(messages, inputs, parameterEnabled),
|
||||||
});
|
});
|
||||||
|
|
||||||
|
export const buildGeminiNativeConversationProxyEnvelope = (
|
||||||
|
messages: ChatMessage[],
|
||||||
|
inputs: ModelTesterInputs,
|
||||||
|
parameterEnabled: ParameterEnabled,
|
||||||
|
): TesterProxyEnvelope => {
|
||||||
|
const envelope = buildConversationRequestEnvelope(messages, { ...inputs, protocol: 'gemini' }, parameterEnabled);
|
||||||
|
return {
|
||||||
|
...envelope,
|
||||||
|
path: `/gemini${envelope.path}${inputs.stream ? '?alt=sse' : ''}`,
|
||||||
|
jobMode: false,
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
export const buildEmbeddingsRequestEnvelope = (
|
export const buildEmbeddingsRequestEnvelope = (
|
||||||
inputText: string,
|
inputText: string,
|
||||||
inputs: ModelTesterInputs,
|
inputs: ModelTesterInputs,
|
||||||
|
|||||||
@@ -16,6 +16,12 @@ export function buildVisibleRouteList<T extends RouteListVisibilityItem>(
|
|||||||
isExactModelPattern: (pattern: string) => boolean,
|
isExactModelPattern: (pattern: string) => boolean,
|
||||||
matchesModelPattern: (model: string, pattern: string) => boolean,
|
matchesModelPattern: (model: string, pattern: string) => boolean,
|
||||||
): T[] {
|
): T[] {
|
||||||
|
const exactModelNames = new Set(
|
||||||
|
routes
|
||||||
|
.filter((route) => isExactModelPattern(route.modelPattern))
|
||||||
|
.map((route) => (route.modelPattern || '').trim())
|
||||||
|
.filter(Boolean),
|
||||||
|
);
|
||||||
const coveringGroups = routes.filter((route) => (
|
const coveringGroups = routes.filter((route) => (
|
||||||
route.enabled
|
route.enabled
|
||||||
&& !isExactModelPattern(route.modelPattern)
|
&& !isExactModelPattern(route.modelPattern)
|
||||||
@@ -33,6 +39,7 @@ export function buildVisibleRouteList<T extends RouteListVisibilityItem>(
|
|||||||
|
|
||||||
return !coveringGroups.some((groupRoute) => (
|
return !coveringGroups.some((groupRoute) => (
|
||||||
groupRoute.id !== route.id
|
groupRoute.id !== route.id
|
||||||
|
&& !exactModelNames.has((groupRoute.displayName || '').trim())
|
||||||
&& matchesModelPattern(exactModel, groupRoute.modelPattern)
|
&& matchesModelPattern(exactModel, groupRoute.modelPattern)
|
||||||
));
|
));
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -3,6 +3,10 @@ export type MissingTokenModelAccount = {
|
|||||||
username: string | null;
|
username: string | null;
|
||||||
siteId: number;
|
siteId: number;
|
||||||
siteName: string;
|
siteName: string;
|
||||||
|
missingGroups?: string[];
|
||||||
|
requiredGroups?: string[];
|
||||||
|
availableGroups?: string[];
|
||||||
|
groupCoverageUncertain?: boolean;
|
||||||
};
|
};
|
||||||
|
|
||||||
export type MissingTokenModelsByName = Record<string, MissingTokenModelAccount[]>;
|
export type MissingTokenModelsByName = Record<string, MissingTokenModelAccount[]>;
|
||||||
@@ -29,11 +33,21 @@ export function normalizeMissingTokenModels(
|
|||||||
if (!account || !Number.isFinite(account.accountId)) continue;
|
if (!account || !Number.isFinite(account.accountId)) continue;
|
||||||
const accountName = (account.username || '').trim();
|
const accountName = (account.username || '').trim();
|
||||||
const siteName = String(account.siteName || '').trim();
|
const siteName = String(account.siteName || '').trim();
|
||||||
|
const normalizeLabels = (labels: unknown): string[] => Array.isArray(labels)
|
||||||
|
? Array.from(new Set(labels
|
||||||
|
.map((label) => String(label || '').trim())
|
||||||
|
.filter(Boolean)))
|
||||||
|
.sort((a, b) => a.localeCompare(b, undefined, { sensitivity: 'base' }))
|
||||||
|
: [];
|
||||||
accountMap.set(account.accountId, {
|
accountMap.set(account.accountId, {
|
||||||
accountId: account.accountId,
|
accountId: account.accountId,
|
||||||
username: accountName || null,
|
username: accountName || null,
|
||||||
siteId: account.siteId,
|
siteId: account.siteId,
|
||||||
siteName,
|
siteName,
|
||||||
|
...(normalizeLabels(account.missingGroups).length > 0 ? { missingGroups: normalizeLabels(account.missingGroups) } : {}),
|
||||||
|
...(normalizeLabels(account.requiredGroups).length > 0 ? { requiredGroups: normalizeLabels(account.requiredGroups) } : {}),
|
||||||
|
...(normalizeLabels(account.availableGroups).length > 0 ? { availableGroups: normalizeLabels(account.availableGroups) } : {}),
|
||||||
|
...(account.groupCoverageUncertain === true ? { groupCoverageUncertain: true } : {}),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
if (accountMap.size > 0) {
|
if (accountMap.size > 0) {
|
||||||
|
|||||||
@@ -23,6 +23,7 @@ import type {
|
|||||||
RouteDecision,
|
RouteDecision,
|
||||||
RouteDecisionCandidate,
|
RouteDecisionCandidate,
|
||||||
MissingTokenRouteSiteActionItem,
|
MissingTokenRouteSiteActionItem,
|
||||||
|
MissingTokenGroupRouteSiteActionItem,
|
||||||
RouteRoutingStrategy,
|
RouteRoutingStrategy,
|
||||||
} from './types.js';
|
} from './types.js';
|
||||||
import type { RouteCandidateView, RouteTokenOption } from '../helpers/routeModelCandidatesIndex.js';
|
import type { RouteCandidateView, RouteTokenOption } from '../helpers/routeModelCandidatesIndex.js';
|
||||||
@@ -61,6 +62,7 @@ type RouteCardProps = {
|
|||||||
onChannelDragEnd: (routeId: number, event: DragEndEvent) => void;
|
onChannelDragEnd: (routeId: number, event: DragEndEvent) => void;
|
||||||
// Missing token hints
|
// Missing token hints
|
||||||
missingTokenSiteItems: MissingTokenRouteSiteActionItem[];
|
missingTokenSiteItems: MissingTokenRouteSiteActionItem[];
|
||||||
|
missingTokenGroupItems: MissingTokenGroupRouteSiteActionItem[];
|
||||||
onCreateTokenForMissing: (accountId: number, modelName: string) => void;
|
onCreateTokenForMissing: (accountId: number, modelName: string) => void;
|
||||||
// Add channel
|
// Add channel
|
||||||
onAddChannel: (routeId: number) => void;
|
onAddChannel: (routeId: number) => void;
|
||||||
@@ -104,6 +106,7 @@ function RouteCardInner({
|
|||||||
onDeleteChannel,
|
onDeleteChannel,
|
||||||
onChannelDragEnd,
|
onChannelDragEnd,
|
||||||
missingTokenSiteItems,
|
missingTokenSiteItems,
|
||||||
|
missingTokenGroupItems,
|
||||||
onCreateTokenForMissing,
|
onCreateTokenForMissing,
|
||||||
onAddChannel,
|
onAddChannel,
|
||||||
expandedSourceGroupMap,
|
expandedSourceGroupMap,
|
||||||
@@ -298,22 +301,43 @@ function RouteCardInner({
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
{/* Missing token hints + Add channel button */}
|
{/* Missing token hints + Add channel button */}
|
||||||
<div style={{ display: 'flex', alignItems: 'center', justifyContent: 'space-between', gap: 8, marginBottom: 10, flexWrap: 'wrap' }}>
|
<div style={{ display: 'flex', alignItems: 'flex-start', justifyContent: 'space-between', gap: 8, marginBottom: 10, flexWrap: 'wrap' }}>
|
||||||
{missingTokenSiteItems.length > 0 ? (
|
{(missingTokenSiteItems.length > 0 || missingTokenGroupItems.length > 0) ? (
|
||||||
<div style={{ display: 'flex', alignItems: 'center', gap: 6, flexWrap: 'wrap', flex: 1 }}>
|
<div style={{ display: 'flex', flexDirection: 'column', gap: 6, flex: 1 }}>
|
||||||
<span style={{ fontSize: 12, color: 'var(--color-text-secondary)' }}>{tr('待注册站点')}:</span>
|
{missingTokenSiteItems.length > 0 && (
|
||||||
{missingTokenSiteItems.map((item) => (
|
<div style={{ display: 'flex', alignItems: 'center', gap: 6, flexWrap: 'wrap' }}>
|
||||||
<button
|
<span style={{ fontSize: 12, color: 'var(--color-text-secondary)' }}>{tr('待注册站点')}:</span>
|
||||||
key={`missing-${route.id}-${item.key}`}
|
{missingTokenSiteItems.map((item) => (
|
||||||
type="button"
|
<button
|
||||||
onClick={() => onCreateTokenForMissing(item.accountId, route.modelPattern)}
|
key={`missing-${route.id}-${item.key}`}
|
||||||
className="badge badge-info missing-token-site-tag"
|
type="button"
|
||||||
data-tooltip={`点击跳转到令牌创建(预选 ${item.siteName}/${item.accountLabel})`}
|
onClick={() => onCreateTokenForMissing(item.accountId, route.modelPattern)}
|
||||||
style={{ fontSize: 11, cursor: 'pointer' }}
|
className="badge badge-info missing-token-site-tag"
|
||||||
>
|
data-tooltip={`点击跳转到令牌创建(预选 ${item.siteName}/${item.accountLabel})`}
|
||||||
{item.siteName}
|
style={{ fontSize: 11, cursor: 'pointer' }}
|
||||||
</button>
|
>
|
||||||
))}
|
{item.siteName}
|
||||||
|
</button>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{missingTokenGroupItems.length > 0 && (
|
||||||
|
<div style={{ display: 'flex', alignItems: 'center', gap: 6, flexWrap: 'wrap' }}>
|
||||||
|
<span style={{ fontSize: 12, color: 'var(--color-text-secondary)' }}>{tr('缺少分组')}:</span>
|
||||||
|
{missingTokenGroupItems.map((item) => (
|
||||||
|
<button
|
||||||
|
key={`missing-group-${route.id}-${item.key}`}
|
||||||
|
type="button"
|
||||||
|
onClick={() => onCreateTokenForMissing(item.accountId, route.modelPattern)}
|
||||||
|
className="badge badge-warning missing-token-group-tag"
|
||||||
|
data-tooltip={`缺少分组:${item.missingGroups.join('、') || '未知'}${item.availableGroups.length > 0 ? `;已覆盖:${item.availableGroups.join('、')}` : ''}${item.groupCoverageUncertain ? ';当前分组覆盖存在不确定性' : ''}`}
|
||||||
|
style={{ fontSize: 11, cursor: 'pointer' }}
|
||||||
|
>
|
||||||
|
{item.siteName}
|
||||||
|
</button>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
</div>
|
</div>
|
||||||
) : <div />}
|
) : <div />}
|
||||||
<button
|
<button
|
||||||
|
|||||||
@@ -124,6 +124,17 @@ export type MissingTokenRouteSiteActionItem = {
|
|||||||
accountLabel: string;
|
accountLabel: string;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
export type MissingTokenGroupRouteSiteActionItem = {
|
||||||
|
key: string;
|
||||||
|
siteName: string;
|
||||||
|
accountId: number;
|
||||||
|
accountLabel: string;
|
||||||
|
missingGroups: string[];
|
||||||
|
requiredGroups: string[];
|
||||||
|
availableGroups: string[];
|
||||||
|
groupCoverageUncertain?: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
export type SortableChannelRowProps = {
|
export type SortableChannelRowProps = {
|
||||||
channel: RouteChannel;
|
channel: RouteChannel;
|
||||||
decisionCandidate?: RouteDecisionCandidate;
|
decisionCandidate?: RouteDecisionCandidate;
|
||||||
|
|||||||
@@ -289,6 +289,63 @@ describe('TokenRoutes grouped source models', () => {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('renders missing-token-group hints separately from missing-token site tags', async () => {
|
||||||
|
apiMock.getRoutesSummary.mockResolvedValue([
|
||||||
|
{
|
||||||
|
id: 1, modelPattern: 'claude-opus-4-6', displayName: 'claude-opus-4-6',
|
||||||
|
displayIcon: null, modelMapping: null, enabled: true,
|
||||||
|
channelCount: 0, enabledChannelCount: 0, siteNames: [],
|
||||||
|
decisionSnapshot: null, decisionRefreshedAt: null,
|
||||||
|
},
|
||||||
|
]);
|
||||||
|
apiMock.getModelTokenCandidates.mockResolvedValue({
|
||||||
|
models: {},
|
||||||
|
modelsMissingTokenGroups: {
|
||||||
|
'claude-opus-4-6': [
|
||||||
|
{
|
||||||
|
accountId: 101,
|
||||||
|
username: 'linuxdo_4677',
|
||||||
|
siteId: 11,
|
||||||
|
siteName: '香草api',
|
||||||
|
missingGroups: ['opus'],
|
||||||
|
requiredGroups: ['default', 'opus'],
|
||||||
|
availableGroups: ['default'],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
});
|
||||||
|
apiMock.getRouteChannels.mockResolvedValue([]);
|
||||||
|
|
||||||
|
let root: ReturnType<typeof create> | null = null;
|
||||||
|
try {
|
||||||
|
await act(async () => {
|
||||||
|
root = create(
|
||||||
|
<MemoryRouter initialEntries={['/routes']}>
|
||||||
|
<ToastProvider>
|
||||||
|
<TokenRoutes />
|
||||||
|
</ToastProvider>
|
||||||
|
</MemoryRouter>,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
await flushMicrotasks();
|
||||||
|
|
||||||
|
const expandBtn = root.root.find((node) =>
|
||||||
|
node.type === 'div' && String(node.props.className || '').includes('route-card-collapsed'),
|
||||||
|
);
|
||||||
|
await act(async () => {
|
||||||
|
expandBtn.props.onClick();
|
||||||
|
});
|
||||||
|
await flushMicrotasks();
|
||||||
|
|
||||||
|
const text = collectText(root.root);
|
||||||
|
expect(text).toContain('缺少分组');
|
||||||
|
expect(text).toContain('香草api');
|
||||||
|
expect(text).not.toContain('待注册站点');
|
||||||
|
} finally {
|
||||||
|
root?.unmount();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
it('maps endpoint types to expected brand icons in filter panel', async () => {
|
it('maps endpoint types to expected brand icons in filter panel', async () => {
|
||||||
apiMock.getRoutesSummary.mockResolvedValue([
|
apiMock.getRoutesSummary.mockResolvedValue([
|
||||||
{
|
{
|
||||||
@@ -518,6 +575,86 @@ describe('TokenRoutes grouped source models', () => {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('keeps exact routes visible when a group display name collides with a real exact model', async () => {
|
||||||
|
apiMock.getRoutesSummary.mockResolvedValue([
|
||||||
|
{
|
||||||
|
id: 1, modelPattern: 'gpt-4o-mini', displayName: 'gpt-4o-mini',
|
||||||
|
displayIcon: null, modelMapping: null, enabled: true,
|
||||||
|
channelCount: 0, enabledChannelCount: 0, siteNames: [],
|
||||||
|
decisionSnapshot: null, decisionRefreshedAt: null,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: 2, modelPattern: 'official/gpt-4o-mini', displayName: 'official/gpt-4o-mini',
|
||||||
|
displayIcon: null, modelMapping: null, enabled: true,
|
||||||
|
channelCount: 0, enabledChannelCount: 0, siteNames: [],
|
||||||
|
decisionSnapshot: null, decisionRefreshedAt: null,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: 3, modelPattern: 're:^(gpt-4o-mini|official/gpt-4o-mini)$', displayName: 'gpt-4o-mini',
|
||||||
|
displayIcon: null, modelMapping: null, enabled: true,
|
||||||
|
channelCount: 0, enabledChannelCount: 0, siteNames: [],
|
||||||
|
decisionSnapshot: null, decisionRefreshedAt: null,
|
||||||
|
},
|
||||||
|
]);
|
||||||
|
|
||||||
|
let root: ReturnType<typeof create> | null = null;
|
||||||
|
try {
|
||||||
|
await act(async () => {
|
||||||
|
root = create(
|
||||||
|
<MemoryRouter initialEntries={['/routes']}>
|
||||||
|
<ToastProvider>
|
||||||
|
<TokenRoutes />
|
||||||
|
</ToastProvider>
|
||||||
|
</MemoryRouter>,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
await flushMicrotasks();
|
||||||
|
|
||||||
|
const normalizedText = collectText(root.root).replace(/\s+/g, '');
|
||||||
|
expect(normalizedText).toContain('共3条路由');
|
||||||
|
expect(normalizedText).not.toContain('共1条路由');
|
||||||
|
} finally {
|
||||||
|
root?.unmount();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('searches routes by display name as well as model pattern', async () => {
|
||||||
|
apiMock.getRoutesSummary.mockResolvedValue([
|
||||||
|
{
|
||||||
|
id: 31, modelPattern: 're:^claude-(opus|sonnet)-4-6$', displayName: 'claude-4-6-group',
|
||||||
|
displayIcon: null, modelMapping: null, enabled: true,
|
||||||
|
channelCount: 0, enabledChannelCount: 0, siteNames: [],
|
||||||
|
decisionSnapshot: null, decisionRefreshedAt: null,
|
||||||
|
},
|
||||||
|
]);
|
||||||
|
|
||||||
|
let root: ReturnType<typeof create> | null = null;
|
||||||
|
try {
|
||||||
|
await act(async () => {
|
||||||
|
root = create(
|
||||||
|
<MemoryRouter initialEntries={['/routes']}>
|
||||||
|
<ToastProvider>
|
||||||
|
<TokenRoutes />
|
||||||
|
</ToastProvider>
|
||||||
|
</MemoryRouter>,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
await flushMicrotasks();
|
||||||
|
|
||||||
|
const searchInput = findInputByPlaceholder(root.root, '搜索模型路由');
|
||||||
|
await act(async () => {
|
||||||
|
searchInput.props.onChange({ target: { value: 'claude-4-6-group' } });
|
||||||
|
});
|
||||||
|
await flushMicrotasks();
|
||||||
|
|
||||||
|
const normalizedText = collectText(root.root).replace(/\s+/g, '');
|
||||||
|
expect(normalizedText).toContain('共1条路由');
|
||||||
|
expect(normalizedText).not.toContain('没有匹配的路由');
|
||||||
|
} finally {
|
||||||
|
root?.unmount();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
it('enters edit mode and seeds the group form with the current route values', async () => {
|
it('enters edit mode and seeds the group form with the current route values', async () => {
|
||||||
apiMock.getRoutesSummary.mockResolvedValue([
|
apiMock.getRoutesSummary.mockResolvedValue([
|
||||||
{
|
{
|
||||||
|
|||||||
Reference in New Issue
Block a user