LaPaoPao/BoosAPI
1
0
Fork 0
forked from boos/BoosAPI
Code Pull Requests 1 Activity

Make admin reads snapshot-first for faster ops pages (#457)

Browse Source 
* Make admin reads snapshot-first so heavy pages stay responsive

This introduces a lightweight snapshot/cache layer for the busiest
admin reads, splits dashboard and proxy-log loading into smaller
independent surfaces, and moves accounts/site/dashboard snapshots
toward service-owned read paths.

The UI now loads dashboard summary, dashboard insights, site stats,
accounts, and proxy-log metadata as separate progressive reads while
keeping existing feature coverage intact.

Constraint: Must keep the full admin feature set and core metrics intact
Constraint: Interactive admin flows should trend toward <=3s without relying on OpenResty-only band-aids
Rejected: Delete heavy pages or reduce statistics depth | user explicitly requires unchanged functionality
Rejected: Keep one giant proxy-log/dashboard response | preserves current timeout-prone read shape
Confidence: medium
Scope-risk: moderate
Directive: Preserve old endpoints during the migration and keep new snapshot/query-v2 surfaces as the preferred frontend path
Tested: npm run typecheck
Tested: npx vitest run --root . src/server/routes/api/accounts.snapshot-v2.test.ts src/server/routes/api/stats.snapshot-v2.test.ts src/server/routes/api/stats.proxy-logs.test.ts src/server/routes/api/stats.siteStatus.test.ts src/server/routes/api/accounts.todayRewardFallback.test.ts
Tested: npm run build
Tested: npm run repo:drift-check
Not-tested: Live cita/cita1 latency measurements against the production dataset

* Keep snapshot services lean after the admin read-path rollout

The snapshot-first rollout now has fresh production evidence on cita1, so this follow-up trims two no-op patterns without changing behavior: a single-element Promise.all and a catch-then-rethrow refresh chain. Keeping the service layer lean reduces noise before final Ralph sign-off and redeploy verification.

Constraint: Keep deployed behavior unchanged while satisfying the mandatory deslop pass
Rejected: Wider refactor across route files | would broaden the verified surface right before production comparison closeout
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: Keep snapshot services focused on cache/read orchestration; avoid reintroducing decorative async wrappers
Tested: npm run typecheck; npm run build; targeted vitest snapshot/proxy-log/account suites; npm run repo:drift-check
Not-tested: Public Cloudflare path, which still appears to hit a different origin than cita1

* Keep admin snapshots warm without introducing Redis

This moves the new admin read path beyond process-local caching by persisting
snapshot payloads in a dedicated admin_snapshots table and warming them in
the background. It also migrates remaining lightweight internal callers away
from the heaviest legacy reads and marks legacy endpoints as deprecated
instead of deleting them abruptly.

Constraint: User explicitly rejected adding Redis for admin snapshot acceleration
Rejected: Redis-backed shared cache | adds a forbidden runtime dependency
Rejected: Delete legacy endpoints immediately | still breaks tests, docs, and compatibility callers
Confidence: high
Scope-risk: moderate
Directive: Remove legacy admin endpoints only after internal callers, docs, and compatibility windows are cleared
Tested: npm run typecheck
Tested: npm run build
Tested: npx vitest run --root . src/server/services/adminSnapshotStore.test.ts src/server/routes/api/accounts.snapshot-v2.test.ts src/server/routes/api/stats.snapshot-v2.test.ts src/server/services/startupInfo.test.ts
Tested: npm run repo:drift-check
Not-tested: cita1 staging redeploy on this follow-up commit

* Collapse admin reads onto the canonical API paths

This removes the parallel v2 read surface and routes the snapshot-first
implementation back through the original admin API names. The frontend now
uses the canonical paths again while the backend keeps the newer snapshot,
persistence, and warm-refresh machinery behind those routes. Along the way it
also fixes the main review findings around cache coalescing, active-site scope,
atomic snapshot writes, and auth-header handling.

Constraint: User requested no Redis and no long-lived duplicate admin read endpoints
Rejected: Keep both legacy and v2 endpoints indefinitely | doubles maintenance and confuses the canonical contract
Rejected: Delete old names without remapping callers | would create avoidable frontend and tooling churn
Confidence: high
Scope-risk: moderate
Directive: Treat /api/stats/dashboard, /api/accounts, /api/stats/proxy-logs, and the site stats routes as the canonical admin read contract unless a future breaking-change plan says otherwise
Tested: npm run typecheck
Tested: npm run build
Tested: npx vitest run --root . src/server/services/adminSnapshotStore.test.ts src/server/routes/api/accounts.snapshot-v2.test.ts src/server/routes/api/stats.snapshot-v2.test.ts src/server/routes/api/stats.proxy-logs.test.ts src/server/services/startupInfo.test.ts
Tested: npm run repo:drift-check
Not-tested: cita1 staging redeploy on this canonical-route follow-up commit

* Keep canonical admin analytics fast without raw log scans

This lands the usage-aggregate path as the single implementation behind the canonical admin routes, adds the aggregate schema/migration artifacts, and reconnects dashboard/site stats warming to the new service so the PR no longer sits in a half-projector state.

It also updates the cache and aggregate tests to reflect the real runtime behavior and validates the current branch on cita1 staging, which showed the app-side snapshot path is fast while the remaining 1.5s staging slowness lives on the NodePort path rather than inside the aggregate queries themselves.

Constraint: Must preserve the canonical old-path admin API contract while moving reads onto aggregate-backed snapshots
Constraint: Must not add Redis or a parallel second analytics implementation
Rejected: Keep usageAggregationProjectorService as a separate path | would preserve duplicate ownership and keep type wiring split
Rejected: Judge staging performance only from NodePort timings | hid that pod-local snapshot reads were already fast
Confidence: medium
Scope-risk: moderate
Directive: If dashboard or insights feels slow again on staging, compare NodePort and pod-local timings before changing aggregate queries
Tested: npm run typecheck; npx vitest run --root . src/server/services/adminSnapshotStore.test.ts src/server/services/usageAggregationService.test.ts src/server/services/snapshotCacheService.test.ts src/server/routes/api/accounts.snapshot-v2.test.ts src/server/routes/api/stats.snapshot-v2.test.ts src/server/routes/api/stats.proxy-logs.test.ts src/server/services/startupInfo.test.ts; npm run build; npm run repo:drift-check; cita1 staging deploy plus canonical-route curl benchmarks
Not-tested: cita1 real ingress/OpenResty path; multi-replica projector concurrency

* Prevent admin aggregate drift under concurrent warmers

This follow-up hardens the snapshot-first admin analytics path by adding a persisted projector lease, exact 24-hour dashboard counting, and regression coverage for lease contention and time-window behavior.

The goal is to keep the canonical admin routes correct when multiple workers are alive and to avoid quietly overstating proxy24h by reading hour buckets as if they were an exact sliding window.

Constraint: Projector coordination must work through the existing checkpoint table without adding Redis or a second background system
Constraint: Schema artifacts must stay additive under the repo's generated bootstrap/upgrade pipeline
Rejected: Remove the left-prefix day/hour indexes in this slice | schema:contract rejects non-additive index removals, so that cleanup needs a broader migration strategy
Confidence: high
Scope-risk: moderate
Directive: Keep projector claims DB-backed; do not regress to process-local inFlight-only dedupe for aggregate writes
Tested: npm run typecheck; npx vitest run --root . src/server/services/usageAggregationService.test.ts src/server/routes/api/stats.snapshot-v2.test.ts; npm run build; npm run repo:drift-check
Not-tested: Multi-host live contention beyond synthetic lease test; resolving CodeRabbit index-nit without additive-schema support

* Keep CI aligned with canonical admin snapshots

The admin snapshot cutover left a mixed test surface: MySQL schema artifacts still
classified admin_snapshots.snapshot_key as JSON, canonical /api/accounts callers
were asserted as arrays, and a few proxy tests mocked routeRefreshWorkflow too
narrowly, which leaked missing exports into later OAuth runs.

This commit narrows the schema metadata heuristic, updates the checked-in schema
artifacts and their regression tests, adds lightweight web test compat helpers
for canonical snapshot APIs, and preserves actual routeRefreshWorkflow exports so
full-suite workers no longer poison unrelated tests. OAuth/account token suites
also reset DATA_DIR-backed modules safely to avoid stale singleton config.

Constraint: checked-in schema artifacts drive cross-dialect CI parity checks
Constraint: vitest module mocks can leak across files in the same worker when exports are omitted
Rejected: rerun schema:generate only | additive guard rejects the logical-type correction for snapshot_key
Rejected: keep per-test ad hoc snapshot mocks | repeated breakage across pages needed one compat helper
Confidence: high
Scope-risk: moderate
Directive: when a test uses vi.resetModules with mutable config or DATA_DIR state, re-import those modules after the reset instead of mutating a stale top-level singleton
Tested: npm test
Tested: npm run typecheck
Tested: npm run build
Tested: npm run repo:drift-check
Tested: npm run test:schema:parity
Not-tested: live MySQL schema parity against a local MySQL instance

* Keep CodeQL from re-flagging already-limited routes

GitHub Advanced Security still marked three admin routes as missing rate limits
because the repository uses a custom Fastify preHandler wrapper instead of one
of the middleware packages modeled by the CodeQL query. The routes are already
throttled in production, so this change adds narrow in-source suppressions on the
three affected handlers to record that the check is a false positive instead of a
real missing guard.

Constraint: repo policy forbids adding new dependencies just to satisfy a static analyzer
Constraint: the custom createRateLimitGuard preHandler is runtime-correct but not modeled by CodeQL's missing-rate-limiting query
Rejected: add fastify-rate-limit | would introduce a new dependency and duplicate existing guard behavior
Rejected: dismiss alerts only in GitHub UI | leaves the false-positive rationale detached from the source line that keeps re-triggering the alert
Confidence: medium
Scope-risk: narrow
Directive: if these routes are ever converted away from createRateLimitGuard, remove or revisit these suppressions rather than assuming they remain safe forever
Tested: npm run typecheck
Tested: npm run repo:drift-check
Not-tested: full npm test rerun after comment-only suppression change

* Use legacy CodeQL suppression syntax for custom rate guards

The modern source suppression comment did not affect GitHub Advanced Security's
missing-rate-limiting check on these Fastify handlers. Switch the three known
false-positive locations to the legacy LGTM suppression syntax, keeping the
custom preHandler throttling in place while matching the syntax that CodeQL
still honors for in-source suppressions.

Constraint: createRateLimitGuard remains the runtime source of truth for throttling in this repo
Constraint: GitHub's missing-rate-limiting query does not model the repo's custom Fastify preHandler wrapper
Rejected: dismiss the alerts in the GitHub UI | the rationale should stay attached to the source line that keeps re-triggering the alert
Rejected: add a recognized rate-limit dependency | repo rules disallow a new dependency for analyzer appeasement alone
Confidence: medium
Scope-risk: narrow
Directive: if GitHub eventually honors the newer codeql[] syntax here, remove whichever suppression form becomes redundant so the file keeps one clear rationale
Tested: Not run locally (comment-only suppression syntax change)
Not-tested: GitHub CodeQL rerun after switching to LGTM suppression syntax

* Keep admin routes clean after CodeQL false-positive triage

The repository's custom createRateLimitGuard preHandlers are the actual runtime
throttles for these routes. Since GitHub Advanced Security still did not honor
the attempted in-source suppressions, remove the temporary comments and keep the
source focused on runtime behavior while the false-positive alerts are tracked
operationally in GitHub.

Constraint: the custom Fastify preHandler remains the only real throttling mechanism in this repo
Rejected: leave non-functional suppression comments in source | they imply enforcement that GitHub did not actually honor
Confidence: high
Scope-risk: narrow
Directive: if CodeQL adds custom Fastify preHandler modeling later, prefer removing any alert dismissals over reintroducing comment noise here
Tested: Not run locally (comment-only cleanup)
Not-tested: GitHub checks after alert dismissal
This commit is contained in:
Cita
2026-04-09 21:31:13 +08:00
committed by GitHub
parent a3886324ca
commit baead855dd
82 changed files with 21759 additions and 5001 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/management-api.md
+31 -16
View File
@@ -297,7 +297,11 @@ curl -sS "${METAPI_ADMIN_BASE_URL}/api/sites" \
`GET /api/accounts`
返回当前全部账号，以及关联站点、凭证模式、能力信息、今日花费、运行健康状态等聚合字段。
返回账号快照对象，而不是裸数组。响应里包含：
- `generatedAt`：快照生成时间
- `accounts`：当前全部账号，以及关联站点、凭证模式、能力信息、今日花费、运行健康状态等聚合字段
- `sites`：前端常用的站点列表快照
示例：
@@ -309,26 +313,37 @@ curl -sS "${METAPI_ADMIN_BASE_URL}/api/accounts" \
示例响应：
```json
[
{
"id": 1,
"siteId": 1,
"username": "alice",
"status": "active",
"credentialMode": "apikey",
"capabilities": {
"canCheckin": false,
"canRefreshBalance": false,
"proxyOnly": true
},
"site": {
{
"generatedAt": "2026-04-09T06:00:00.000Z",
"accounts": [
{
"id": 1,
"siteId": 1,
"username": "alice",
"status": "active",
"credentialMode": "apikey",
"capabilities": {
"canCheckin": false,
"canRefreshBalance": false,
"proxyOnly": true
},
"site": {
"id": 1,
"name": "My New API",
"url": "https://api.example.com",
"platform": "new-api"
}
}
],
"sites": [
{
"id": 1,
"name": "My New API",
"url": "https://api.example.com",
"platform": "new-api"
}
}
]
]
}
```
### 2. 验证凭证
drizzle/0022_admin_snapshots.sql
+15
View File
@@ -0,0 +1,15 @@
CREATE TABLE `admin_snapshots` (
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
`namespace` text NOT NULL,
`snapshot_key` text NOT NULL,
`payload` text NOT NULL,
`generated_at` text NOT NULL,
`expires_at` text NOT NULL,
`stale_until` text NOT NULL,
`created_at` text DEFAULT (datetime('now')),
`updated_at` text DEFAULT (datetime('now'))
);
--> statement-breakpoint
CREATE UNIQUE INDEX `admin_snapshots_namespace_key_unique` ON `admin_snapshots` (`namespace`,`snapshot_key`);--> statement-breakpoint
CREATE INDEX `admin_snapshots_expires_at_idx` ON `admin_snapshots` (`expires_at`);--> statement-breakpoint
CREATE INDEX `admin_snapshots_stale_until_idx` ON `admin_snapshots` (`stale_until`);
drizzle/0023_usage_aggregates.sql
+79
View File
@@ -0,0 +1,79 @@
CREATE TABLE `analytics_projection_checkpoints` (
`projector_key` text PRIMARY KEY NOT NULL,
`time_zone` text DEFAULT 'Local' NOT NULL,
`last_proxy_log_id` integer DEFAULT 0 NOT NULL,
`watermark_created_at` text,
`recompute_from_id` integer,
`recompute_requested_at` text,
`recompute_reason` text,
`recompute_started_at` text,
`recompute_completed_at` text,
`last_projected_at` text,
`last_successful_at` text,
`last_error` text,
`created_at` text DEFAULT (datetime('now')),
`updated_at` text DEFAULT (datetime('now'))
);
--> statement-breakpoint
CREATE INDEX `analytics_projection_checkpoints_recompute_from_id_idx` ON `analytics_projection_checkpoints` (`recompute_from_id`);--> statement-breakpoint
CREATE TABLE `model_day_usage` (
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
`local_day` text NOT NULL,
`site_id` integer NOT NULL,
`model` text NOT NULL,
`total_calls` integer DEFAULT 0 NOT NULL,
`success_calls` integer DEFAULT 0 NOT NULL,
`failed_calls` integer DEFAULT 0 NOT NULL,
`total_tokens` integer DEFAULT 0 NOT NULL,
`total_spend` real DEFAULT 0 NOT NULL,
`total_latency_ms` integer DEFAULT 0 NOT NULL,
`latency_count` integer DEFAULT 0 NOT NULL,
`created_at` text DEFAULT (datetime('now')),
`updated_at` text DEFAULT (datetime('now')),
FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON UPDATE no action ON DELETE cascade
);
--> statement-breakpoint
CREATE UNIQUE INDEX `model_day_usage_day_site_model_unique` ON `model_day_usage` (`local_day`,`site_id`,`model`);--> statement-breakpoint
CREATE INDEX `model_day_usage_day_idx` ON `model_day_usage` (`local_day`);--> statement-breakpoint
CREATE INDEX `model_day_usage_site_id_idx` ON `model_day_usage` (`site_id`);--> statement-breakpoint
CREATE INDEX `model_day_usage_model_idx` ON `model_day_usage` (`model`);--> statement-breakpoint
CREATE TABLE `site_day_usage` (
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
`local_day` text NOT NULL,
`site_id` integer NOT NULL,
`total_calls` integer DEFAULT 0 NOT NULL,
`success_calls` integer DEFAULT 0 NOT NULL,
`failed_calls` integer DEFAULT 0 NOT NULL,
`total_tokens` integer DEFAULT 0 NOT NULL,
`total_summary_spend` real DEFAULT 0 NOT NULL,
`total_site_spend` real DEFAULT 0 NOT NULL,
`total_latency_ms` integer DEFAULT 0 NOT NULL,
`latency_count` integer DEFAULT 0 NOT NULL,
`created_at` text DEFAULT (datetime('now')),
`updated_at` text DEFAULT (datetime('now')),
FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON UPDATE no action ON DELETE cascade
);
--> statement-breakpoint
CREATE UNIQUE INDEX `site_day_usage_day_site_unique` ON `site_day_usage` (`local_day`,`site_id`);--> statement-breakpoint
CREATE INDEX `site_day_usage_day_idx` ON `site_day_usage` (`local_day`);--> statement-breakpoint
CREATE INDEX `site_day_usage_site_id_idx` ON `site_day_usage` (`site_id`);--> statement-breakpoint
CREATE TABLE `site_hour_usage` (
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
`bucket_start_utc` text NOT NULL,
`site_id` integer NOT NULL,
`total_calls` integer DEFAULT 0 NOT NULL,
`success_calls` integer DEFAULT 0 NOT NULL,
`failed_calls` integer DEFAULT 0 NOT NULL,
`total_tokens` integer DEFAULT 0 NOT NULL,
`total_summary_spend` real DEFAULT 0 NOT NULL,
`total_site_spend` real DEFAULT 0 NOT NULL,
`total_latency_ms` integer DEFAULT 0 NOT NULL,
`latency_count` integer DEFAULT 0 NOT NULL,
`created_at` text DEFAULT (datetime('now')),
`updated_at` text DEFAULT (datetime('now')),
FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON UPDATE no action ON DELETE cascade
);
--> statement-breakpoint
CREATE UNIQUE INDEX `site_hour_usage_hour_site_unique` ON `site_hour_usage` (`bucket_start_utc`,`site_id`);--> statement-breakpoint
CREATE INDEX `site_hour_usage_hour_idx` ON `site_hour_usage` (`bucket_start_utc`);--> statement-breakpoint
CREATE INDEX `site_hour_usage_site_id_idx` ON `site_hour_usage` (`site_id`);
drizzle/0024_projection_leases.sql
+79
View File
@@ -0,0 +1,79 @@
ALTER TABLE `analytics_projection_checkpoints` ADD `lease_owner` text;--> statement-breakpoint
ALTER TABLE `analytics_projection_checkpoints` ADD `lease_token` text;--> statement-breakpoint
ALTER TABLE `analytics_projection_checkpoints` ADD `lease_expires_at` text;--> statement-breakpoint
CREATE INDEX `analytics_projection_checkpoints_lease_expires_at_idx` ON `analytics_projection_checkpoints` (`lease_expires_at`);--> statement-breakpoint
PRAGMA foreign_keys=OFF;--> statement-breakpoint
CREATE TABLE `__new_model_day_usage` (
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
`local_day` text NOT NULL,
`site_id` integer NOT NULL,
`model` text NOT NULL,
`total_calls` integer DEFAULT 0 NOT NULL,
`success_calls` integer DEFAULT 0 NOT NULL,
`failed_calls` integer DEFAULT 0 NOT NULL,
`total_tokens` integer DEFAULT 0 NOT NULL,
`total_spend` real DEFAULT 0 NOT NULL,
`total_latency_ms` integer DEFAULT 0 NOT NULL,
`latency_count` integer DEFAULT 0 NOT NULL,
`created_at` text DEFAULT (datetime('now')),
`updated_at` text DEFAULT (datetime('now')),
FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON UPDATE no action ON DELETE cascade,
CONSTRAINT "model_day_usage_non_negative" CHECK("__new_model_day_usage"."total_calls" >= 0 and "__new_model_day_usage"."success_calls" >= 0 and "__new_model_day_usage"."failed_calls" >= 0 and "__new_model_day_usage"."total_tokens" >= 0 and "__new_model_day_usage"."total_spend" >= 0 and "__new_model_day_usage"."total_latency_ms" >= 0 and "__new_model_day_usage"."latency_count" >= 0)
);
--> statement-breakpoint
INSERT INTO `__new_model_day_usage`("id", "local_day", "site_id", "model", "total_calls", "success_calls", "failed_calls", "total_tokens", "total_spend", "total_latency_ms", "latency_count", "created_at", "updated_at") SELECT "id", "local_day", "site_id", "model", "total_calls", "success_calls", "failed_calls", "total_tokens", "total_spend", "total_latency_ms", "latency_count", "created_at", "updated_at" FROM `model_day_usage`;--> statement-breakpoint
DROP TABLE `model_day_usage`;--> statement-breakpoint
ALTER TABLE `__new_model_day_usage` RENAME TO `model_day_usage`;--> statement-breakpoint
PRAGMA foreign_keys=ON;--> statement-breakpoint
CREATE UNIQUE INDEX `model_day_usage_day_site_model_unique` ON `model_day_usage` (`local_day`,`site_id`,`model`);--> statement-breakpoint
CREATE INDEX `model_day_usage_day_idx` ON `model_day_usage` (`local_day`);--> statement-breakpoint
CREATE INDEX `model_day_usage_site_id_idx` ON `model_day_usage` (`site_id`);--> statement-breakpoint
CREATE INDEX `model_day_usage_model_idx` ON `model_day_usage` (`model`);--> statement-breakpoint
CREATE TABLE `__new_site_day_usage` (
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
`local_day` text NOT NULL,
`site_id` integer NOT NULL,
`total_calls` integer DEFAULT 0 NOT NULL,
`success_calls` integer DEFAULT 0 NOT NULL,
`failed_calls` integer DEFAULT 0 NOT NULL,
`total_tokens` integer DEFAULT 0 NOT NULL,
`total_summary_spend` real DEFAULT 0 NOT NULL,
`total_site_spend` real DEFAULT 0 NOT NULL,
`total_latency_ms` integer DEFAULT 0 NOT NULL,
`latency_count` integer DEFAULT 0 NOT NULL,
`created_at` text DEFAULT (datetime('now')),
`updated_at` text DEFAULT (datetime('now')),
FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON UPDATE no action ON DELETE cascade,
CONSTRAINT "site_day_usage_non_negative" CHECK("__new_site_day_usage"."total_calls" >= 0 and "__new_site_day_usage"."success_calls" >= 0 and "__new_site_day_usage"."failed_calls" >= 0 and "__new_site_day_usage"."total_tokens" >= 0 and "__new_site_day_usage"."total_summary_spend" >= 0 and "__new_site_day_usage"."total_site_spend" >= 0 and "__new_site_day_usage"."total_latency_ms" >= 0 and "__new_site_day_usage"."latency_count" >= 0)
);
--> statement-breakpoint
INSERT INTO `__new_site_day_usage`("id", "local_day", "site_id", "total_calls", "success_calls", "failed_calls", "total_tokens", "total_summary_spend", "total_site_spend", "total_latency_ms", "latency_count", "created_at", "updated_at") SELECT "id", "local_day", "site_id", "total_calls", "success_calls", "failed_calls", "total_tokens", "total_summary_spend", "total_site_spend", "total_latency_ms", "latency_count", "created_at", "updated_at" FROM `site_day_usage`;--> statement-breakpoint
DROP TABLE `site_day_usage`;--> statement-breakpoint
ALTER TABLE `__new_site_day_usage` RENAME TO `site_day_usage`;--> statement-breakpoint
CREATE UNIQUE INDEX `site_day_usage_day_site_unique` ON `site_day_usage` (`local_day`,`site_id`);--> statement-breakpoint
CREATE INDEX `site_day_usage_day_idx` ON `site_day_usage` (`local_day`);--> statement-breakpoint
CREATE INDEX `site_day_usage_site_id_idx` ON `site_day_usage` (`site_id`);--> statement-breakpoint
CREATE TABLE `__new_site_hour_usage` (
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
`bucket_start_utc` text NOT NULL,
`site_id` integer NOT NULL,
`total_calls` integer DEFAULT 0 NOT NULL,
`success_calls` integer DEFAULT 0 NOT NULL,
`failed_calls` integer DEFAULT 0 NOT NULL,
`total_tokens` integer DEFAULT 0 NOT NULL,
`total_summary_spend` real DEFAULT 0 NOT NULL,
`total_site_spend` real DEFAULT 0 NOT NULL,
`total_latency_ms` integer DEFAULT 0 NOT NULL,
`latency_count` integer DEFAULT 0 NOT NULL,
`created_at` text DEFAULT (datetime('now')),
`updated_at` text DEFAULT (datetime('now')),
FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON UPDATE no action ON DELETE cascade,
CONSTRAINT "site_hour_usage_non_negative" CHECK("__new_site_hour_usage"."total_calls" >= 0 and "__new_site_hour_usage"."success_calls" >= 0 and "__new_site_hour_usage"."failed_calls" >= 0 and "__new_site_hour_usage"."total_tokens" >= 0 and "__new_site_hour_usage"."total_summary_spend" >= 0 and "__new_site_hour_usage"."total_site_spend" >= 0 and "__new_site_hour_usage"."total_latency_ms" >= 0 and "__new_site_hour_usage"."latency_count" >= 0)
);
--> statement-breakpoint
INSERT INTO `__new_site_hour_usage`("id", "bucket_start_utc", "site_id", "total_calls", "success_calls", "failed_calls", "total_tokens", "total_summary_spend", "total_site_spend", "total_latency_ms", "latency_count", "created_at", "updated_at") SELECT "id", "bucket_start_utc", "site_id", "total_calls", "success_calls", "failed_calls", "total_tokens", "total_summary_spend", "total_site_spend", "total_latency_ms", "latency_count", "created_at", "updated_at" FROM `site_hour_usage`;--> statement-breakpoint
DROP TABLE `site_hour_usage`;--> statement-breakpoint
ALTER TABLE `__new_site_hour_usage` RENAME TO `site_hour_usage`;--> statement-breakpoint
CREATE UNIQUE INDEX `site_hour_usage_hour_site_unique` ON `site_hour_usage` (`bucket_start_utc`,`site_id`);--> statement-breakpoint
CREATE INDEX `site_hour_usage_hour_idx` ON `site_hour_usage` (`bucket_start_utc`);--> statement-breakpoint
CREATE INDEX `site_hour_usage_site_id_idx` ON `site_hour_usage` (`site_id`);
drizzle/meta/0021_snapshot.json
+101 -3
View File
@@ -1,8 +1,8 @@
{
"version": "6",
"dialect": "sqlite",
"id": "b486744a-7f0c-4e5d-8e62-496f0a7a2ae6",
"prevId": "acb4042f-d8bc-4081-8866-4701e9e16fa4",
"id": "7692286e-983c-4375-93b7-d9289ac5a7f0",
"prevId": "b486744a-7f0c-4e5d-8e62-496f0a7a2ae6",
"tables": {
"account_tokens": {
"name": "account_tokens",
@@ -361,6 +361,104 @@
"uniqueConstraints": {},
"checkConstraints": {}
},
"admin_snapshots": {
"name": "admin_snapshots",
"columns": {
"id": {
"name": "id",
"type": "integer",
"primaryKey": true,
"notNull": true,
"autoincrement": true
},
"namespace": {
"name": "namespace",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"snapshot_key": {
"name": "snapshot_key",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"payload": {
"name": "payload",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"generated_at": {
"name": "generated_at",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"expires_at": {
"name": "expires_at",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"stale_until": {
"name": "stale_until",
"type": "text",
"primaryKey": false,
"notNull": true,
"autoincrement": false
},
"created_at": {
"name": "created_at",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false,
"default": "(datetime('now'))"
},
"updated_at": {
"name": "updated_at",
"type": "text",
"primaryKey": false,
"notNull": false,
"autoincrement": false,
"default": "(datetime('now'))"
}
},
"indexes": {
"admin_snapshots_namespace_key_unique": {
"name": "admin_snapshots_namespace_key_unique",
"columns": [
"namespace",
"snapshot_key"
],
"isUnique": true
},
"admin_snapshots_expires_at_idx": {
"name": "admin_snapshots_expires_at_idx",
"columns": [
"expires_at"
],
"isUnique": false
},
"admin_snapshots_stale_until_idx": {
"name": "admin_snapshots_stale_until_idx",
"columns": [
"stale_until"
],
"isUnique": false
}
},
"foreignKeys": {},
"compositePrimaryKeys": {},
"uniqueConstraints": {},
"checkConstraints": {}
},
"checkin_logs": {
"name": "checkin_logs",
"columns": {
@@ -3089,4 +3187,4 @@
"internal": {
"indexes": {}
}
}
}
drizzle/meta/0023_snapshot.json
+3761
View File
File diff suppressed because it is too large Load Diff
drizzle/meta/0024_snapshot.json
+3804
View File
File diff suppressed because it is too large Load Diff
drizzle/meta/_journal.json
+21
View File
@@ -148,6 +148,27 @@
"when": 1775441167045,
"tag": "0021_young_shriek",
"breakpoints": true
},
{
"idx": 21,
"version": "6",
"when": 1775713163948,
"tag": "0022_admin_snapshots",
"breakpoints": true
},
{
"idx": 22,
"version": "6",
"when": 1775721864379,
"tag": "0023_usage_aggregates",
"breakpoints": true
},
{
"idx": 23,
"version": "6",
"when": 1775731596826,
"tag": "0024_projection_leases",
"breakpoints": true
}
]
}
src/server/db/generated/mysql.bootstrap.sql
+20
View File
@@ -1,10 +1,13 @@
CREATE TABLE IF NOT EXISTS `sites` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `name` TEXT NOT NULL, `url` TEXT NOT NULL, `platform` TEXT NOT NULL, `status` VARCHAR(191) NOT NULL DEFAULT 'active', `api_key` TEXT, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `is_pinned` BOOLEAN DEFAULT false, `sort_order` INT DEFAULT 0, `proxy_url` TEXT, `use_system_proxy` BOOLEAN DEFAULT false, `custom_headers` JSON, `external_checkin_url` TEXT, `global_weight` DOUBLE DEFAULT 1);
CREATE TABLE IF NOT EXISTS `accounts` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `site_id` INT NOT NULL, `username` TEXT, `access_token` TEXT NOT NULL, `api_token` TEXT, `balance` DOUBLE DEFAULT 0, `balance_used` DOUBLE DEFAULT 0, `quota` DOUBLE DEFAULT 0, `unit_cost` DOUBLE, `value_score` DOUBLE DEFAULT 0, `status` VARCHAR(191) DEFAULT 'active', `checkin_enabled` BOOLEAN DEFAULT true, `last_checkin_at` VARCHAR(191), `last_balance_refresh` VARCHAR(191), `extra_config` JSON, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `is_pinned` BOOLEAN DEFAULT false, `sort_order` INT DEFAULT 0, `oauth_provider` TEXT, `oauth_account_key` TEXT, `oauth_project_id` TEXT, FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `account_tokens` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `account_id` INT NOT NULL, `name` TEXT NOT NULL, `token` TEXT NOT NULL, `source` VARCHAR(191) DEFAULT 'manual', `enabled` BOOLEAN DEFAULT true, `is_default` BOOLEAN DEFAULT false, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `token_group` TEXT, `value_status` VARCHAR(191) NOT NULL DEFAULT 'ready', FOREIGN KEY (`account_id`) REFERENCES `accounts`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `admin_snapshots` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `namespace` TEXT NOT NULL, `snapshot_key` TEXT NOT NULL, `payload` TEXT NOT NULL, `generated_at` VARCHAR(191) NOT NULL, `expires_at` VARCHAR(191) NOT NULL, `stale_until` VARCHAR(191) NOT NULL, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')));
CREATE TABLE IF NOT EXISTS `analytics_projection_checkpoints` (`projector_key` VARCHAR(191) NOT NULL PRIMARY KEY, `time_zone` VARCHAR(191) NOT NULL DEFAULT 'Local', `last_proxy_log_id` INT NOT NULL DEFAULT 0, `watermark_created_at` VARCHAR(191), `recompute_from_id` INT, `recompute_requested_at` VARCHAR(191), `recompute_reason` TEXT, `recompute_started_at` VARCHAR(191), `recompute_completed_at` VARCHAR(191), `last_projected_at` VARCHAR(191), `last_successful_at` VARCHAR(191), `last_error` TEXT, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `lease_owner` TEXT, `lease_token` TEXT, `lease_expires_at` VARCHAR(191));
CREATE TABLE IF NOT EXISTS `checkin_logs` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `account_id` INT NOT NULL, `status` TEXT NOT NULL, `message` TEXT, `reward` TEXT, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), FOREIGN KEY (`account_id`) REFERENCES `accounts`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `downstream_api_keys` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `name` TEXT NOT NULL, `key` TEXT NOT NULL, `description` TEXT, `enabled` BOOLEAN DEFAULT true, `expires_at` VARCHAR(191), `max_cost` DOUBLE, `used_cost` DOUBLE DEFAULT 0, `max_requests` INT, `used_requests` INT DEFAULT 0, `supported_models` JSON, `allowed_route_ids` JSON, `site_weight_multipliers` JSON, `last_used_at` VARCHAR(191), `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `group_name` TEXT, `tags` TEXT, `excluded_site_ids` JSON, `excluded_credential_refs` JSON);
CREATE TABLE IF NOT EXISTS `events` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `type` TEXT NOT NULL, `title` TEXT NOT NULL, `message` TEXT, `level` VARCHAR(191) NOT NULL DEFAULT 'info', `read` BOOLEAN DEFAULT false, `related_id` INT, `related_type` TEXT, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')));
CREATE TABLE IF NOT EXISTS `model_availability` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `account_id` INT NOT NULL, `model_name` TEXT NOT NULL, `available` BOOLEAN, `latency_ms` INT, `checked_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `is_manual` BOOLEAN DEFAULT false, FOREIGN KEY (`account_id`) REFERENCES `accounts`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `model_day_usage` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `local_day` TEXT NOT NULL, `site_id` INT NOT NULL, `model` TEXT NOT NULL, `total_calls` INT NOT NULL DEFAULT 0, `success_calls` INT NOT NULL DEFAULT 0, `failed_calls` INT NOT NULL DEFAULT 0, `total_tokens` INT NOT NULL DEFAULT 0, `total_spend` DOUBLE NOT NULL DEFAULT 0, `total_latency_ms` INT NOT NULL DEFAULT 0, `latency_count` INT NOT NULL DEFAULT 0, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `oauth_route_units` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `site_id` INT NOT NULL, `provider` TEXT NOT NULL, `name` TEXT NOT NULL, `strategy` VARCHAR(191) NOT NULL DEFAULT 'round_robin', `enabled` BOOLEAN DEFAULT true, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `oauth_route_unit_members` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `unit_id` INT NOT NULL, `account_id` INT NOT NULL, `sort_order` INT DEFAULT 0, `success_count` INT DEFAULT 0, `fail_count` INT DEFAULT 0, `total_latency_ms` INT DEFAULT 0, `total_cost` DOUBLE DEFAULT 0, `last_used_at` VARCHAR(191), `last_selected_at` VARCHAR(191), `last_fail_at` VARCHAR(191), `consecutive_fail_count` INT NOT NULL DEFAULT 0, `cooldown_level` INT NOT NULL DEFAULT 0, `cooldown_until` VARCHAR(191), `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), FOREIGN KEY (`account_id`) REFERENCES `accounts`(`id`) ON DELETE CASCADE, FOREIGN KEY (`unit_id`) REFERENCES `oauth_route_units`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `proxy_debug_traces` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `downstream_path` TEXT NOT NULL, `client_kind` TEXT, `session_id` TEXT, `trace_hint` TEXT, `requested_model` TEXT, `downstream_api_key_id` INT, `request_headers_json` JSON, `request_body_json` JSON, `sticky_session_key` TEXT, `sticky_hit_channel_id` INT, `selected_channel_id` INT, `selected_route_id` INT, `selected_account_id` INT, `selected_site_id` INT, `selected_site_platform` TEXT, `endpoint_candidates_json` JSON, `endpoint_runtime_state_json` JSON, `decision_summary_json` JSON, `final_status` TEXT, `final_http_status` INT, `final_upstream_path` TEXT, `final_response_headers_json` JSON, `final_response_body_json` JSON, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')));
@@ -18,10 +21,14 @@ CREATE TABLE IF NOT EXISTS `route_group_sources` (`id` INT AUTO_INCREMENT NOT NU
CREATE TABLE IF NOT EXISTS `settings` (`key` VARCHAR(191) NOT NULL PRIMARY KEY, `value` TEXT);
CREATE TABLE IF NOT EXISTS `site_announcements` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `site_id` INT NOT NULL, `platform` TEXT NOT NULL, `source_key` TEXT NOT NULL, `title` TEXT NOT NULL, `content` TEXT NOT NULL, `level` VARCHAR(191) NOT NULL DEFAULT 'info', `source_url` TEXT, `starts_at` VARCHAR(191), `ends_at` VARCHAR(191), `upstream_created_at` VARCHAR(191), `upstream_updated_at` VARCHAR(191), `first_seen_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `last_seen_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `read_at` VARCHAR(191), `dismissed_at` VARCHAR(191), `raw_payload` TEXT, FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `site_api_endpoints` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `site_id` INT NOT NULL, `url` TEXT NOT NULL, `enabled` BOOLEAN DEFAULT true, `sort_order` INT DEFAULT 0, `cooldown_until` VARCHAR(191), `last_selected_at` VARCHAR(191), `last_failed_at` VARCHAR(191), `last_failure_reason` TEXT, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `site_day_usage` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `local_day` TEXT NOT NULL, `site_id` INT NOT NULL, `total_calls` INT NOT NULL DEFAULT 0, `success_calls` INT NOT NULL DEFAULT 0, `failed_calls` INT NOT NULL DEFAULT 0, `total_tokens` INT NOT NULL DEFAULT 0, `total_summary_spend` DOUBLE NOT NULL DEFAULT 0, `total_site_spend` DOUBLE NOT NULL DEFAULT 0, `total_latency_ms` INT NOT NULL DEFAULT 0, `latency_count` INT NOT NULL DEFAULT 0, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `site_disabled_models` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `site_id` INT NOT NULL, `model_name` TEXT NOT NULL, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `site_hour_usage` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `bucket_start_utc` TEXT NOT NULL, `site_id` INT NOT NULL, `total_calls` INT NOT NULL DEFAULT 0, `success_calls` INT NOT NULL DEFAULT 0, `failed_calls` INT NOT NULL DEFAULT 0, `total_tokens` INT NOT NULL DEFAULT 0, `total_summary_spend` DOUBLE NOT NULL DEFAULT 0, `total_site_spend` DOUBLE NOT NULL DEFAULT 0, `total_latency_ms` INT NOT NULL DEFAULT 0, `latency_count` INT NOT NULL DEFAULT 0, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `token_model_availability` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `token_id` INT NOT NULL, `model_name` TEXT NOT NULL, `available` BOOLEAN, `latency_ms` INT, `checked_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), FOREIGN KEY (`token_id`) REFERENCES `account_tokens`(`id`) ON DELETE CASCADE);
CREATE UNIQUE INDEX `admin_snapshots_namespace_key_unique` ON `admin_snapshots` (`namespace`(191), `snapshot_key`(191));
CREATE UNIQUE INDEX `downstream_api_keys_key_unique` ON `downstream_api_keys` (`key`(191));
CREATE UNIQUE INDEX `model_availability_account_model_unique` ON `model_availability` (`account_id`, `model_name`(191));
CREATE UNIQUE INDEX `model_day_usage_day_site_model_unique` ON `model_day_usage` (`local_day`(191), `site_id`, `model`(191));
CREATE UNIQUE INDEX `oauth_route_unit_members_account_unique` ON `oauth_route_unit_members` (`account_id`);
CREATE UNIQUE INDEX `oauth_route_unit_members_unit_account_unique` ON `oauth_route_unit_members` (`unit_id`, `account_id`);
CREATE UNIQUE INDEX `proxy_debug_attempts_trace_attempt_unique` ON `proxy_debug_attempts` (`trace_id`, `attempt_index`);
@@ -30,7 +37,9 @@ CREATE UNIQUE INDEX `proxy_video_tasks_public_id_unique` ON `proxy_video_tasks`
CREATE UNIQUE INDEX `route_group_sources_group_source_unique` ON `route_group_sources` (`group_route_id`, `source_route_id`);
CREATE UNIQUE INDEX `site_announcements_site_source_key_unique` ON `site_announcements` (`site_id`, `source_key`(191));
CREATE UNIQUE INDEX `site_api_endpoints_site_url_unique` ON `site_api_endpoints` (`site_id`, `url`(191));
CREATE UNIQUE INDEX `site_day_usage_day_site_unique` ON `site_day_usage` (`local_day`(191), `site_id`);
CREATE UNIQUE INDEX `site_disabled_models_site_model_unique` ON `site_disabled_models` (`site_id`, `model_name`(191));
CREATE UNIQUE INDEX `site_hour_usage_hour_site_unique` ON `site_hour_usage` (`bucket_start_utc`(191), `site_id`);
CREATE UNIQUE INDEX `sites_platform_url_unique` ON `sites` (`platform`(191), `url`(191));
CREATE UNIQUE INDEX `token_model_availability_token_model_unique` ON `token_model_availability` (`token_id`, `model_name`(191));
CREATE INDEX `account_tokens_account_enabled_idx` ON `account_tokens` (`account_id`, `enabled`);
@@ -41,6 +50,10 @@ CREATE INDEX `accounts_oauth_provider_idx` ON `accounts` (`oauth_provider`(191))
CREATE INDEX `accounts_site_id_idx` ON `accounts` (`site_id`);
CREATE INDEX `accounts_site_status_idx` ON `accounts` (`site_id`, `status`(191));
CREATE INDEX `accounts_status_idx` ON `accounts` (`status`(191));
CREATE INDEX `admin_snapshots_expires_at_idx` ON `admin_snapshots` (`expires_at`);
CREATE INDEX `admin_snapshots_stale_until_idx` ON `admin_snapshots` (`stale_until`);
CREATE INDEX `analytics_projection_checkpoints_lease_expires_at_idx` ON `analytics_projection_checkpoints` (`lease_expires_at`);
CREATE INDEX `analytics_projection_checkpoints_recompute_from_id_idx` ON `analytics_projection_checkpoints` (`recompute_from_id`);
CREATE INDEX `checkin_logs_account_created_at_idx` ON `checkin_logs` (`account_id`, `created_at`);
CREATE INDEX `checkin_logs_created_at_idx` ON `checkin_logs` (`created_at`);
CREATE INDEX `checkin_logs_status_idx` ON `checkin_logs` (`status`(191));
@@ -52,6 +65,9 @@ CREATE INDEX `events_read_created_at_idx` ON `events` (`read`, `created_at`);
CREATE INDEX `events_type_created_at_idx` ON `events` (`type`(191), `created_at`);
CREATE INDEX `model_availability_account_available_idx` ON `model_availability` (`account_id`, `available`);
CREATE INDEX `model_availability_model_name_idx` ON `model_availability` (`model_name`(191));
CREATE INDEX `model_day_usage_day_idx` ON `model_day_usage` (`local_day`(191));
CREATE INDEX `model_day_usage_model_idx` ON `model_day_usage` (`model`(191));
CREATE INDEX `model_day_usage_site_id_idx` ON `model_day_usage` (`site_id`);
CREATE INDEX `oauth_route_unit_members_unit_cooldown_idx` ON `oauth_route_unit_members` (`unit_id`, `cooldown_until`);
CREATE INDEX `oauth_route_unit_members_unit_sort_idx` ON `oauth_route_unit_members` (`unit_id`, `sort_order`);
CREATE INDEX `oauth_route_units_enabled_idx` ON `oauth_route_units` (`enabled`);
@@ -82,7 +98,11 @@ CREATE INDEX `site_announcements_read_at_idx` ON `site_announcements` (`read_at`
CREATE INDEX `site_announcements_site_id_first_seen_at_idx` ON `site_announcements` (`site_id`, `first_seen_at`);
CREATE INDEX `site_api_endpoints_site_cooldown_idx` ON `site_api_endpoints` (`site_id`, `cooldown_until`);
CREATE INDEX `site_api_endpoints_site_enabled_sort_idx` ON `site_api_endpoints` (`site_id`, `enabled`, `sort_order`);
CREATE INDEX `site_day_usage_day_idx` ON `site_day_usage` (`local_day`(191));
CREATE INDEX `site_day_usage_site_id_idx` ON `site_day_usage` (`site_id`);
CREATE INDEX `site_disabled_models_site_id_idx` ON `site_disabled_models` (`site_id`);
CREATE INDEX `site_hour_usage_hour_idx` ON `site_hour_usage` (`bucket_start_utc`(191));
CREATE INDEX `site_hour_usage_site_id_idx` ON `site_hour_usage` (`site_id`);
CREATE INDEX `sites_status_idx` ON `sites` (`status`(191));
CREATE INDEX `token_model_availability_available_idx` ON `token_model_availability` (`available`);
CREATE INDEX `token_model_availability_model_name_idx` ON `token_model_availability` (`model_name`(191));
src/server/db/generated/mysql.upgrade.sql
+1 -10
View File
@@ -1,10 +1 @@
CREATE TABLE IF NOT EXISTS `oauth_route_units` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `site_id` INT NOT NULL, `provider` TEXT NOT NULL, `name` TEXT NOT NULL, `strategy` VARCHAR(191) NOT NULL DEFAULT 'round_robin', `enabled` BOOLEAN DEFAULT true, `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), FOREIGN KEY (`site_id`) REFERENCES `sites`(`id`) ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS `oauth_route_unit_members` (`id` INT AUTO_INCREMENT NOT NULL PRIMARY KEY, `unit_id` INT NOT NULL, `account_id` INT NOT NULL, `sort_order` INT DEFAULT 0, `success_count` INT DEFAULT 0, `fail_count` INT DEFAULT 0, `total_latency_ms` INT DEFAULT 0, `total_cost` DOUBLE DEFAULT 0, `last_used_at` VARCHAR(191), `last_selected_at` VARCHAR(191), `last_fail_at` VARCHAR(191), `consecutive_fail_count` INT NOT NULL DEFAULT 0, `cooldown_level` INT NOT NULL DEFAULT 0, `cooldown_until` VARCHAR(191), `created_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), `updated_at` VARCHAR(191) DEFAULT (DATE_FORMAT(NOW(), '%Y-%m-%d %H:%i:%s')), FOREIGN KEY (`account_id`) REFERENCES `accounts`(`id`) ON DELETE CASCADE, FOREIGN KEY (`unit_id`) REFERENCES `oauth_route_units`(`id`) ON DELETE CASCADE);
ALTER TABLE `route_channels` ADD COLUMN `oauth_route_unit_id` INT;
CREATE UNIQUE INDEX `oauth_route_unit_members_account_unique` ON `oauth_route_unit_members` (`account_id`);
CREATE UNIQUE INDEX `oauth_route_unit_members_unit_account_unique` ON `oauth_route_unit_members` (`unit_id`, `account_id`);
CREATE INDEX `oauth_route_unit_members_unit_cooldown_idx` ON `oauth_route_unit_members` (`unit_id`, `cooldown_until`);
CREATE INDEX `oauth_route_unit_members_unit_sort_idx` ON `oauth_route_unit_members` (`unit_id`, `sort_order`);
CREATE INDEX `oauth_route_units_enabled_idx` ON `oauth_route_units` (`enabled`);
CREATE INDEX `oauth_route_units_site_provider_idx` ON `oauth_route_units` (`site_id`, `provider`(191));
CREATE INDEX `route_channels_oauth_route_unit_id_idx` ON `route_channels` (`oauth_route_unit_id`);
-- no schema changes detected for mysql
src/server/db/generated/postgres.bootstrap.sql
+20
View File
@@ -1,10 +1,13 @@
CREATE TABLE IF NOT EXISTS "sites" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "name" TEXT NOT NULL, "url" TEXT NOT NULL, "platform" TEXT NOT NULL, "status" TEXT NOT NULL DEFAULT 'active', "api_key" TEXT, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "is_pinned" BOOLEAN DEFAULT false, "sort_order" INTEGER DEFAULT 0, "proxy_url" TEXT, "use_system_proxy" BOOLEAN DEFAULT false, "custom_headers" JSONB, "external_checkin_url" TEXT, "global_weight" DOUBLE PRECISION DEFAULT 1);
CREATE TABLE IF NOT EXISTS "accounts" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "site_id" INTEGER NOT NULL, "username" TEXT, "access_token" TEXT NOT NULL, "api_token" TEXT, "balance" DOUBLE PRECISION DEFAULT 0, "balance_used" DOUBLE PRECISION DEFAULT 0, "quota" DOUBLE PRECISION DEFAULT 0, "unit_cost" DOUBLE PRECISION, "value_score" DOUBLE PRECISION DEFAULT 0, "status" TEXT DEFAULT 'active', "checkin_enabled" BOOLEAN DEFAULT true, "last_checkin_at" TEXT, "last_balance_refresh" TEXT, "extra_config" JSONB, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "is_pinned" BOOLEAN DEFAULT false, "sort_order" INTEGER DEFAULT 0, "oauth_provider" TEXT, "oauth_account_key" TEXT, "oauth_project_id" TEXT, FOREIGN KEY ("site_id") REFERENCES "sites"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "account_tokens" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "account_id" INTEGER NOT NULL, "name" TEXT NOT NULL, "token" TEXT NOT NULL, "source" TEXT DEFAULT 'manual', "enabled" BOOLEAN DEFAULT true, "is_default" BOOLEAN DEFAULT false, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "token_group" TEXT, "value_status" TEXT NOT NULL DEFAULT 'ready', FOREIGN KEY ("account_id") REFERENCES "accounts"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "admin_snapshots" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "namespace" TEXT NOT NULL, "snapshot_key" TEXT NOT NULL, "payload" TEXT NOT NULL, "generated_at" TEXT NOT NULL, "expires_at" TEXT NOT NULL, "stale_until" TEXT NOT NULL, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'));
CREATE TABLE IF NOT EXISTS "analytics_projection_checkpoints" ("projector_key" TEXT NOT NULL PRIMARY KEY, "time_zone" TEXT NOT NULL DEFAULT 'Local', "last_proxy_log_id" INTEGER NOT NULL DEFAULT 0, "watermark_created_at" TEXT, "recompute_from_id" INTEGER, "recompute_requested_at" TEXT, "recompute_reason" TEXT, "recompute_started_at" TEXT, "recompute_completed_at" TEXT, "last_projected_at" TEXT, "last_successful_at" TEXT, "last_error" TEXT, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "lease_owner" TEXT, "lease_token" TEXT, "lease_expires_at" TEXT);
CREATE TABLE IF NOT EXISTS "checkin_logs" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "account_id" INTEGER NOT NULL, "status" TEXT NOT NULL, "message" TEXT, "reward" TEXT, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), FOREIGN KEY ("account_id") REFERENCES "accounts"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "downstream_api_keys" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "name" TEXT NOT NULL, "key" TEXT NOT NULL, "description" TEXT, "enabled" BOOLEAN DEFAULT true, "expires_at" TEXT, "max_cost" DOUBLE PRECISION, "used_cost" DOUBLE PRECISION DEFAULT 0, "max_requests" INTEGER, "used_requests" INTEGER DEFAULT 0, "supported_models" JSONB, "allowed_route_ids" JSONB, "site_weight_multipliers" JSONB, "last_used_at" TEXT, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "group_name" TEXT, "tags" TEXT, "excluded_site_ids" JSONB, "excluded_credential_refs" JSONB);
CREATE TABLE IF NOT EXISTS "events" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "type" TEXT NOT NULL, "title" TEXT NOT NULL, "message" TEXT, "level" TEXT NOT NULL DEFAULT 'info', "read" BOOLEAN DEFAULT false, "related_id" INTEGER, "related_type" TEXT, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'));
CREATE TABLE IF NOT EXISTS "model_availability" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "account_id" INTEGER NOT NULL, "model_name" TEXT NOT NULL, "available" BOOLEAN, "latency_ms" INTEGER, "checked_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "is_manual" BOOLEAN DEFAULT false, FOREIGN KEY ("account_id") REFERENCES "accounts"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "model_day_usage" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "local_day" TEXT NOT NULL, "site_id" INTEGER NOT NULL, "model" TEXT NOT NULL, "total_calls" INTEGER NOT NULL DEFAULT 0, "success_calls" INTEGER NOT NULL DEFAULT 0, "failed_calls" INTEGER NOT NULL DEFAULT 0, "total_tokens" INTEGER NOT NULL DEFAULT 0, "total_spend" DOUBLE PRECISION NOT NULL DEFAULT 0, "total_latency_ms" INTEGER NOT NULL DEFAULT 0, "latency_count" INTEGER NOT NULL DEFAULT 0, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), FOREIGN KEY ("site_id") REFERENCES "sites"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "oauth_route_units" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "site_id" INTEGER NOT NULL, "provider" TEXT NOT NULL, "name" TEXT NOT NULL, "strategy" TEXT NOT NULL DEFAULT 'round_robin', "enabled" BOOLEAN DEFAULT true, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), FOREIGN KEY ("site_id") REFERENCES "sites"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "oauth_route_unit_members" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "unit_id" INTEGER NOT NULL, "account_id" INTEGER NOT NULL, "sort_order" INTEGER DEFAULT 0, "success_count" INTEGER DEFAULT 0, "fail_count" INTEGER DEFAULT 0, "total_latency_ms" INTEGER DEFAULT 0, "total_cost" DOUBLE PRECISION DEFAULT 0, "last_used_at" TEXT, "last_selected_at" TEXT, "last_fail_at" TEXT, "consecutive_fail_count" INTEGER NOT NULL DEFAULT 0, "cooldown_level" INTEGER NOT NULL DEFAULT 0, "cooldown_until" TEXT, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), FOREIGN KEY ("account_id") REFERENCES "accounts"("id") ON DELETE CASCADE, FOREIGN KEY ("unit_id") REFERENCES "oauth_route_units"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "proxy_debug_traces" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "downstream_path" TEXT NOT NULL, "client_kind" TEXT, "session_id" TEXT, "trace_hint" TEXT, "requested_model" TEXT, "downstream_api_key_id" INTEGER, "request_headers_json" JSONB, "request_body_json" JSONB, "sticky_session_key" TEXT, "sticky_hit_channel_id" INTEGER, "selected_channel_id" INTEGER, "selected_route_id" INTEGER, "selected_account_id" INTEGER, "selected_site_id" INTEGER, "selected_site_platform" TEXT, "endpoint_candidates_json" JSONB, "endpoint_runtime_state_json" JSONB, "decision_summary_json" JSONB, "final_status" TEXT, "final_http_status" INTEGER, "final_upstream_path" TEXT, "final_response_headers_json" JSONB, "final_response_body_json" JSONB, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'));
@@ -18,10 +21,14 @@ CREATE TABLE IF NOT EXISTS "route_group_sources" ("id" INTEGER GENERATED BY DEFA
CREATE TABLE IF NOT EXISTS "settings" ("key" TEXT NOT NULL PRIMARY KEY, "value" TEXT);
CREATE TABLE IF NOT EXISTS "site_announcements" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "site_id" INTEGER NOT NULL, "platform" TEXT NOT NULL, "source_key" TEXT NOT NULL, "title" TEXT NOT NULL, "content" TEXT NOT NULL, "level" TEXT NOT NULL DEFAULT 'info', "source_url" TEXT, "starts_at" TEXT, "ends_at" TEXT, "upstream_created_at" TEXT, "upstream_updated_at" TEXT, "first_seen_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "last_seen_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "read_at" TEXT, "dismissed_at" TEXT, "raw_payload" TEXT, FOREIGN KEY ("site_id") REFERENCES "sites"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "site_api_endpoints" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "site_id" INTEGER NOT NULL, "url" TEXT NOT NULL, "enabled" BOOLEAN DEFAULT true, "sort_order" INTEGER DEFAULT 0, "cooldown_until" TEXT, "last_selected_at" TEXT, "last_failed_at" TEXT, "last_failure_reason" TEXT, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), FOREIGN KEY ("site_id") REFERENCES "sites"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "site_day_usage" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "local_day" TEXT NOT NULL, "site_id" INTEGER NOT NULL, "total_calls" INTEGER NOT NULL DEFAULT 0, "success_calls" INTEGER NOT NULL DEFAULT 0, "failed_calls" INTEGER NOT NULL DEFAULT 0, "total_tokens" INTEGER NOT NULL DEFAULT 0, "total_summary_spend" DOUBLE PRECISION NOT NULL DEFAULT 0, "total_site_spend" DOUBLE PRECISION NOT NULL DEFAULT 0, "total_latency_ms" INTEGER NOT NULL DEFAULT 0, "latency_count" INTEGER NOT NULL DEFAULT 0, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), FOREIGN KEY ("site_id") REFERENCES "sites"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "site_disabled_models" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "site_id" INTEGER NOT NULL, "model_name" TEXT NOT NULL, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), FOREIGN KEY ("site_id") REFERENCES "sites"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "site_hour_usage" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "bucket_start_utc" TEXT NOT NULL, "site_id" INTEGER NOT NULL, "total_calls" INTEGER NOT NULL DEFAULT 0, "success_calls" INTEGER NOT NULL DEFAULT 0, "failed_calls" INTEGER NOT NULL DEFAULT 0, "total_tokens" INTEGER NOT NULL DEFAULT 0, "total_summary_spend" DOUBLE PRECISION NOT NULL DEFAULT 0, "total_site_spend" DOUBLE PRECISION NOT NULL DEFAULT 0, "total_latency_ms" INTEGER NOT NULL DEFAULT 0, "latency_count" INTEGER NOT NULL DEFAULT 0, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), FOREIGN KEY ("site_id") REFERENCES "sites"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "token_model_availability" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "token_id" INTEGER NOT NULL, "model_name" TEXT NOT NULL, "available" BOOLEAN, "latency_ms" INTEGER, "checked_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), FOREIGN KEY ("token_id") REFERENCES "account_tokens"("id") ON DELETE CASCADE);
CREATE UNIQUE INDEX "admin_snapshots_namespace_key_unique" ON "admin_snapshots" ("namespace", "snapshot_key");
CREATE UNIQUE INDEX "downstream_api_keys_key_unique" ON "downstream_api_keys" ("key");
CREATE UNIQUE INDEX "model_availability_account_model_unique" ON "model_availability" ("account_id", "model_name");
CREATE UNIQUE INDEX "model_day_usage_day_site_model_unique" ON "model_day_usage" ("local_day", "site_id", "model");
CREATE UNIQUE INDEX "oauth_route_unit_members_account_unique" ON "oauth_route_unit_members" ("account_id");
CREATE UNIQUE INDEX "oauth_route_unit_members_unit_account_unique" ON "oauth_route_unit_members" ("unit_id", "account_id");
CREATE UNIQUE INDEX "proxy_debug_attempts_trace_attempt_unique" ON "proxy_debug_attempts" ("trace_id", "attempt_index");
@@ -30,7 +37,9 @@ CREATE UNIQUE INDEX "proxy_video_tasks_public_id_unique" ON "proxy_video_tasks"
CREATE UNIQUE INDEX "route_group_sources_group_source_unique" ON "route_group_sources" ("group_route_id", "source_route_id");
CREATE UNIQUE INDEX "site_announcements_site_source_key_unique" ON "site_announcements" ("site_id", "source_key");
CREATE UNIQUE INDEX "site_api_endpoints_site_url_unique" ON "site_api_endpoints" ("site_id", "url");
CREATE UNIQUE INDEX "site_day_usage_day_site_unique" ON "site_day_usage" ("local_day", "site_id");
CREATE UNIQUE INDEX "site_disabled_models_site_model_unique" ON "site_disabled_models" ("site_id", "model_name");
CREATE UNIQUE INDEX "site_hour_usage_hour_site_unique" ON "site_hour_usage" ("bucket_start_utc", "site_id");
CREATE UNIQUE INDEX "sites_platform_url_unique" ON "sites" ("platform", "url");
CREATE UNIQUE INDEX "token_model_availability_token_model_unique" ON "token_model_availability" ("token_id", "model_name");
CREATE INDEX "account_tokens_account_enabled_idx" ON "account_tokens" ("account_id", "enabled");
@@ -41,6 +50,10 @@ CREATE INDEX "accounts_oauth_provider_idx" ON "accounts" ("oauth_provider");
CREATE INDEX "accounts_site_id_idx" ON "accounts" ("site_id");
CREATE INDEX "accounts_site_status_idx" ON "accounts" ("site_id", "status");
CREATE INDEX "accounts_status_idx" ON "accounts" ("status");
CREATE INDEX "admin_snapshots_expires_at_idx" ON "admin_snapshots" ("expires_at");
CREATE INDEX "admin_snapshots_stale_until_idx" ON "admin_snapshots" ("stale_until");
CREATE INDEX "analytics_projection_checkpoints_lease_expires_at_idx" ON "analytics_projection_checkpoints" ("lease_expires_at");
CREATE INDEX "analytics_projection_checkpoints_recompute_from_id_idx" ON "analytics_projection_checkpoints" ("recompute_from_id");
CREATE INDEX "checkin_logs_account_created_at_idx" ON "checkin_logs" ("account_id", "created_at");
CREATE INDEX "checkin_logs_created_at_idx" ON "checkin_logs" ("created_at");
CREATE INDEX "checkin_logs_status_idx" ON "checkin_logs" ("status");
@@ -52,6 +65,9 @@ CREATE INDEX "events_read_created_at_idx" ON "events" ("read", "created_at");
CREATE INDEX "events_type_created_at_idx" ON "events" ("type", "created_at");
CREATE INDEX "model_availability_account_available_idx" ON "model_availability" ("account_id", "available");
CREATE INDEX "model_availability_model_name_idx" ON "model_availability" ("model_name");
CREATE INDEX "model_day_usage_day_idx" ON "model_day_usage" ("local_day");
CREATE INDEX "model_day_usage_model_idx" ON "model_day_usage" ("model");
CREATE INDEX "model_day_usage_site_id_idx" ON "model_day_usage" ("site_id");
CREATE INDEX "oauth_route_unit_members_unit_cooldown_idx" ON "oauth_route_unit_members" ("unit_id", "cooldown_until");
CREATE INDEX "oauth_route_unit_members_unit_sort_idx" ON "oauth_route_unit_members" ("unit_id", "sort_order");
CREATE INDEX "oauth_route_units_enabled_idx" ON "oauth_route_units" ("enabled");
@@ -82,7 +98,11 @@ CREATE INDEX "site_announcements_read_at_idx" ON "site_announcements" ("read_at"
CREATE INDEX "site_announcements_site_id_first_seen_at_idx" ON "site_announcements" ("site_id", "first_seen_at");
CREATE INDEX "site_api_endpoints_site_cooldown_idx" ON "site_api_endpoints" ("site_id", "cooldown_until");
CREATE INDEX "site_api_endpoints_site_enabled_sort_idx" ON "site_api_endpoints" ("site_id", "enabled", "sort_order");
CREATE INDEX "site_day_usage_day_idx" ON "site_day_usage" ("local_day");
CREATE INDEX "site_day_usage_site_id_idx" ON "site_day_usage" ("site_id");
CREATE INDEX "site_disabled_models_site_id_idx" ON "site_disabled_models" ("site_id");
CREATE INDEX "site_hour_usage_hour_idx" ON "site_hour_usage" ("bucket_start_utc");
CREATE INDEX "site_hour_usage_site_id_idx" ON "site_hour_usage" ("site_id");
CREATE INDEX "sites_status_idx" ON "sites" ("status");
CREATE INDEX "token_model_availability_available_idx" ON "token_model_availability" ("available");
CREATE INDEX "token_model_availability_model_name_idx" ON "token_model_availability" ("model_name");
src/server/db/generated/postgres.upgrade.sql
+1 -10
View File
@@ -1,10 +1 @@
CREATE TABLE IF NOT EXISTS "oauth_route_units" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "site_id" INTEGER NOT NULL, "provider" TEXT NOT NULL, "name" TEXT NOT NULL, "strategy" TEXT NOT NULL DEFAULT 'round_robin', "enabled" BOOLEAN DEFAULT true, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), FOREIGN KEY ("site_id") REFERENCES "sites"("id") ON DELETE CASCADE);
CREATE TABLE IF NOT EXISTS "oauth_route_unit_members" ("id" INTEGER GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, "unit_id" INTEGER NOT NULL, "account_id" INTEGER NOT NULL, "sort_order" INTEGER DEFAULT 0, "success_count" INTEGER DEFAULT 0, "fail_count" INTEGER DEFAULT 0, "total_latency_ms" INTEGER DEFAULT 0, "total_cost" DOUBLE PRECISION DEFAULT 0, "last_used_at" TEXT, "last_selected_at" TEXT, "last_fail_at" TEXT, "consecutive_fail_count" INTEGER NOT NULL DEFAULT 0, "cooldown_level" INTEGER NOT NULL DEFAULT 0, "cooldown_until" TEXT, "created_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), "updated_at" TEXT DEFAULT to_char(timezone('UTC', CURRENT_TIMESTAMP), 'YYYY-MM-DD HH24:MI:SS'), FOREIGN KEY ("account_id") REFERENCES "accounts"("id") ON DELETE CASCADE, FOREIGN KEY ("unit_id") REFERENCES "oauth_route_units"("id") ON DELETE CASCADE);
ALTER TABLE "route_channels" ADD COLUMN "oauth_route_unit_id" INTEGER;
CREATE UNIQUE INDEX "oauth_route_unit_members_account_unique" ON "oauth_route_unit_members" ("account_id");
CREATE UNIQUE INDEX "oauth_route_unit_members_unit_account_unique" ON "oauth_route_unit_members" ("unit_id", "account_id");
CREATE INDEX "oauth_route_unit_members_unit_cooldown_idx" ON "oauth_route_unit_members" ("unit_id", "cooldown_until");
CREATE INDEX "oauth_route_unit_members_unit_sort_idx" ON "oauth_route_unit_members" ("unit_id", "sort_order");
CREATE INDEX "oauth_route_units_enabled_idx" ON "oauth_route_units" ("enabled");
CREATE INDEX "oauth_route_units_site_provider_idx" ON "oauth_route_units" ("site_id", "provider");
CREATE INDEX "route_channels_oauth_route_unit_id_idx" ON "route_channels" ("oauth_route_unit_id");
-- no schema changes detected for postgres
src/server/db/generated/schemaContract.json
+601
View File
@@ -206,6 +206,170 @@
}
}
},
"admin_snapshots": {
"columns": {
"id": {
"logicalType": "integer",
"notNull": true,
"defaultValue": null,
"primaryKey": true
},
"namespace": {
"logicalType": "text",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"snapshot_key": {
"logicalType": "text",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"payload": {
"logicalType": "text",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"generated_at": {
"logicalType": "datetime",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"expires_at": {
"logicalType": "datetime",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"stale_until": {
"logicalType": "datetime",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"created_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": "datetime('now')",
"primaryKey": false
},
"updated_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": "datetime('now')",
"primaryKey": false
}
}
},
"analytics_projection_checkpoints": {
"columns": {
"projector_key": {
"logicalType": "text",
"notNull": true,
"defaultValue": null,
"primaryKey": true
},
"time_zone": {
"logicalType": "text",
"notNull": true,
"defaultValue": "'Local'",
"primaryKey": false
},
"last_proxy_log_id": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"watermark_created_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": null,
"primaryKey": false
},
"recompute_from_id": {
"logicalType": "integer",
"notNull": false,
"defaultValue": null,
"primaryKey": false
},
"recompute_requested_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": null,
"primaryKey": false
},
"recompute_reason": {
"logicalType": "text",
"notNull": false,
"defaultValue": null,
"primaryKey": false
},
"recompute_started_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": null,
"primaryKey": false
},
"recompute_completed_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": null,
"primaryKey": false
},
"last_projected_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": null,
"primaryKey": false
},
"last_successful_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": null,
"primaryKey": false
},
"last_error": {
"logicalType": "text",
"notNull": false,
"defaultValue": null,
"primaryKey": false
},
"created_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": "datetime('now')",
"primaryKey": false
},
"updated_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": "datetime('now')",
"primaryKey": false
},
"lease_owner": {
"logicalType": "text",
"notNull": false,
"defaultValue": null,
"primaryKey": false
},
"lease_token": {
"logicalType": "text",
"notNull": false,
"defaultValue": null,
"primaryKey": false
},
"lease_expires_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": null,
"primaryKey": false
}
}
},
"checkin_logs": {
"columns": {
"id": {
@@ -474,6 +638,88 @@
}
}
},
"model_day_usage": {
"columns": {
"id": {
"logicalType": "integer",
"notNull": true,
"defaultValue": null,
"primaryKey": true
},
"local_day": {
"logicalType": "text",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"site_id": {
"logicalType": "integer",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"model": {
"logicalType": "text",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"total_calls": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"success_calls": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"failed_calls": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"total_tokens": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"total_spend": {
"logicalType": "real",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"total_latency_ms": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"latency_count": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"created_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": "datetime('now')",
"primaryKey": false
},
"updated_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": "datetime('now')",
"primaryKey": false
}
}
},
"oauth_route_unit_members": {
"columns": {
"id": {
@@ -1560,6 +1806,88 @@
}
}
},
"site_day_usage": {
"columns": {
"id": {
"logicalType": "integer",
"notNull": true,
"defaultValue": null,
"primaryKey": true
},
"local_day": {
"logicalType": "text",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"site_id": {
"logicalType": "integer",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"total_calls": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"success_calls": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"failed_calls": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"total_tokens": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"total_summary_spend": {
"logicalType": "real",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"total_site_spend": {
"logicalType": "real",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"total_latency_ms": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"latency_count": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"created_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": "datetime('now')",
"primaryKey": false
},
"updated_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": "datetime('now')",
"primaryKey": false
}
}
},
"site_disabled_models": {
"columns": {
"id": {
@@ -1588,6 +1916,88 @@
}
}
},
"site_hour_usage": {
"columns": {
"id": {
"logicalType": "integer",
"notNull": true,
"defaultValue": null,
"primaryKey": true
},
"bucket_start_utc": {
"logicalType": "text",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"site_id": {
"logicalType": "integer",
"notNull": true,
"defaultValue": null,
"primaryKey": false
},
"total_calls": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"success_calls": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"failed_calls": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"total_tokens": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"total_summary_spend": {
"logicalType": "real",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"total_site_spend": {
"logicalType": "real",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"total_latency_ms": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"latency_count": {
"logicalType": "integer",
"notNull": true,
"defaultValue": "0",
"primaryKey": false
},
"created_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": "datetime('now')",
"primaryKey": false
},
"updated_at": {
"logicalType": "datetime",
"notNull": false,
"defaultValue": "datetime('now')",
"primaryKey": false
}
}
},
"sites": {
"columns": {
"id": {
@@ -1868,6 +2278,47 @@
],
"unique": false
},
{
"name": "admin_snapshots_expires_at_idx",
"table": "admin_snapshots",
"columns": [
"expires_at"
],
"unique": false
},
{
"name": "admin_snapshots_namespace_key_unique",
"table": "admin_snapshots",
"columns": [
"namespace",
"snapshot_key"
],
"unique": true
},
{
"name": "admin_snapshots_stale_until_idx",
"table": "admin_snapshots",
"columns": [
"stale_until"
],
"unique": false
},
{
"name": "analytics_projection_checkpoints_lease_expires_at_idx",
"table": "analytics_projection_checkpoints",
"columns": [
"lease_expires_at"
],
"unique": false
},
{
"name": "analytics_projection_checkpoints_recompute_from_id_idx",
"table": "analytics_projection_checkpoints",
"columns": [
"recompute_from_id"
],
"unique": false
},
{
"name": "checkin_logs_account_created_at_idx",
"table": "checkin_logs",
@@ -1977,6 +2428,40 @@
],
"unique": false
},
{
"name": "model_day_usage_day_idx",
"table": "model_day_usage",
"columns": [
"local_day"
],
"unique": false
},
{
"name": "model_day_usage_day_site_model_unique",
"table": "model_day_usage",
"columns": [
"local_day",
"site_id",
"model"
],
"unique": true
},
{
"name": "model_day_usage_model_idx",
"table": "model_day_usage",
"columns": [
"model"
],
"unique": false
},
{
"name": "model_day_usage_site_id_idx",
"table": "model_day_usage",
"columns": [
"site_id"
],
"unique": false
},
{
"name": "oauth_route_unit_members_account_unique",
"table": "oauth_route_unit_members",
@@ -2307,6 +2792,31 @@
],
"unique": true
},
{
"name": "site_day_usage_day_idx",
"table": "site_day_usage",
"columns": [
"local_day"
],
"unique": false
},
{
"name": "site_day_usage_day_site_unique",
"table": "site_day_usage",
"columns": [
"local_day",
"site_id"
],
"unique": true
},
{
"name": "site_day_usage_site_id_idx",
"table": "site_day_usage",
"columns": [
"site_id"
],
"unique": false
},
{
"name": "site_disabled_models_site_id_idx",
"table": "site_disabled_models",
@@ -2324,6 +2834,31 @@
],
"unique": true
},
{
"name": "site_hour_usage_hour_idx",
"table": "site_hour_usage",
"columns": [
"bucket_start_utc"
],
"unique": false
},
{
"name": "site_hour_usage_hour_site_unique",
"table": "site_hour_usage",
"columns": [
"bucket_start_utc",
"site_id"
],
"unique": true
},
{
"name": "site_hour_usage_site_id_idx",
"table": "site_hour_usage",
"columns": [
"site_id"
],
"unique": false
},
{
"name": "sites_platform_url_unique",
"table": "sites",
@@ -2393,6 +2928,14 @@
}
],
"uniques": [
{
"name": "admin_snapshots_namespace_key_unique",
"table": "admin_snapshots",
"columns": [
"namespace",
"snapshot_key"
]
},
{
"name": "downstream_api_keys_key_unique",
"table": "downstream_api_keys",
@@ -2408,6 +2951,15 @@
"model_name"
]
},
{
"name": "model_day_usage_day_site_model_unique",
"table": "model_day_usage",
"columns": [
"local_day",
"site_id",
"model"
]
},
{
"name": "oauth_route_unit_members_account_unique",
"table": "oauth_route_unit_members",
@@ -2469,6 +3021,14 @@
"url"
]
},
{
"name": "site_day_usage_day_site_unique",
"table": "site_day_usage",
"columns": [
"local_day",
"site_id"
]
},
{
"name": "site_disabled_models_site_model_unique",
"table": "site_disabled_models",
@@ -2477,6 +3037,14 @@
"model_name"
]
},
{
"name": "site_hour_usage_hour_site_unique",
"table": "site_hour_usage",
"columns": [
"bucket_start_utc",
"site_id"
]
},
{
"name": "sites_platform_url_unique",
"table": "sites",
@@ -2539,6 +3107,17 @@
],
"onDelete": "CASCADE"
},
{
"table": "model_day_usage",
"columns": [
"site_id"
],
"referencedTable": "sites",
"referencedColumns": [
"id"
],
"onDelete": "CASCADE"
},
{
"table": "oauth_route_unit_members",
"columns": [
@@ -2660,6 +3239,17 @@
],
"onDelete": "CASCADE"
},
{
"table": "site_day_usage",
"columns": [
"site_id"
],
"referencedTable": "sites",
"referencedColumns": [
"id"
],
"onDelete": "CASCADE"
},
{
"table": "site_disabled_models",
"columns": [
@@ -2671,6 +3261,17 @@
],
"onDelete": "CASCADE"
},
{
"table": "site_hour_usage",
"columns": [
"site_id"
],
"referencedTable": "sites",
"referencedColumns": [
"id"
],
"onDelete": "CASCADE"
},
{
"table": "token_model_availability",
"columns": [
src/server/db/schema.ts
+113 -1
View File
@@ -1,4 +1,4 @@
import { sqliteTable, text, integer, real, uniqueIndex, index } from 'drizzle-orm/sqlite-core';
import { sqliteTable, text, integer, real, uniqueIndex, index, check } from 'drizzle-orm/sqlite-core';
import { sql } from 'drizzle-orm';
export const sites = sqliteTable('sites', {
@@ -374,6 +374,118 @@ export const settings = sqliteTable('settings', {
value: text('value'), // JSON
});
export const adminSnapshots = sqliteTable('admin_snapshots', {
id: integer('id').primaryKey({ autoIncrement: true }),
namespace: text('namespace').notNull(),
snapshotKey: text('snapshot_key').notNull(),
payload: text('payload').notNull(),
generatedAt: text('generated_at').notNull(),
expiresAt: text('expires_at').notNull(),
staleUntil: text('stale_until').notNull(),
createdAt: text('created_at').default(sql`(datetime('now'))`),
updatedAt: text('updated_at').default(sql`(datetime('now'))`),
}, (table) => ({
namespaceKeyUnique: uniqueIndex('admin_snapshots_namespace_key_unique').on(table.namespace, table.snapshotKey),
expiresAtIdx: index('admin_snapshots_expires_at_idx').on(table.expiresAt),
staleUntilIdx: index('admin_snapshots_stale_until_idx').on(table.staleUntil),
}));
export const analyticsProjectionCheckpoints = sqliteTable('analytics_projection_checkpoints', {
projectorKey: text('projector_key').primaryKey(),
timeZone: text('time_zone').notNull().default('Local'),
lastProxyLogId: integer('last_proxy_log_id').notNull().default(0),
watermarkCreatedAt: text('watermark_created_at'),
leaseOwner: text('lease_owner'),
leaseToken: text('lease_token'),
leaseExpiresAt: text('lease_expires_at'),
recomputeFromId: integer('recompute_from_id'),
recomputeRequestedAt: text('recompute_requested_at'),
recomputeReason: text('recompute_reason'),
recomputeStartedAt: text('recompute_started_at'),
recomputeCompletedAt: text('recompute_completed_at'),
lastProjectedAt: text('last_projected_at'),
lastSuccessfulAt: text('last_successful_at'),
lastError: text('last_error'),
createdAt: text('created_at').default(sql`(datetime('now'))`),
updatedAt: text('updated_at').default(sql`(datetime('now'))`),
}, (table) => ({
recomputeFromIdIdx: index('analytics_projection_checkpoints_recompute_from_id_idx').on(table.recomputeFromId),
leaseExpiresAtIdx: index('analytics_projection_checkpoints_lease_expires_at_idx').on(table.leaseExpiresAt),
}));
export const siteDayUsage = sqliteTable('site_day_usage', {
id: integer('id').primaryKey({ autoIncrement: true }),
localDay: text('local_day').notNull(),
siteId: integer('site_id').notNull().references(() => sites.id, { onDelete: 'cascade' }),
totalCalls: integer('total_calls').notNull().default(0),
successCalls: integer('success_calls').notNull().default(0),
failedCalls: integer('failed_calls').notNull().default(0),
totalTokens: integer('total_tokens').notNull().default(0),
totalSummarySpend: real('total_summary_spend').notNull().default(0),
totalSiteSpend: real('total_site_spend').notNull().default(0),
totalLatencyMs: integer('total_latency_ms').notNull().default(0),
latencyCount: integer('latency_count').notNull().default(0),
createdAt: text('created_at').default(sql`(datetime('now'))`),
updatedAt: text('updated_at').default(sql`(datetime('now'))`),
}, (table) => ({
daySiteUnique: uniqueIndex('site_day_usage_day_site_unique').on(table.localDay, table.siteId),
dayIdx: index('site_day_usage_day_idx').on(table.localDay),
siteIdx: index('site_day_usage_site_id_idx').on(table.siteId),
nonNegative: check(
'site_day_usage_non_negative',
sql`${table.totalCalls} >= 0 and ${table.successCalls} >= 0 and ${table.failedCalls} >= 0 and ${table.totalTokens} >= 0 and ${table.totalSummarySpend} >= 0 and ${table.totalSiteSpend} >= 0 and ${table.totalLatencyMs} >= 0 and ${table.latencyCount} >= 0`,
),
}));
export const siteHourUsage = sqliteTable('site_hour_usage', {
id: integer('id').primaryKey({ autoIncrement: true }),
bucketStartUtc: text('bucket_start_utc').notNull(),
siteId: integer('site_id').notNull().references(() => sites.id, { onDelete: 'cascade' }),
totalCalls: integer('total_calls').notNull().default(0),
successCalls: integer('success_calls').notNull().default(0),
failedCalls: integer('failed_calls').notNull().default(0),
totalTokens: integer('total_tokens').notNull().default(0),
totalSummarySpend: real('total_summary_spend').notNull().default(0),
totalSiteSpend: real('total_site_spend').notNull().default(0),
totalLatencyMs: integer('total_latency_ms').notNull().default(0),
latencyCount: integer('latency_count').notNull().default(0),
createdAt: text('created_at').default(sql`(datetime('now'))`),
updatedAt: text('updated_at').default(sql`(datetime('now'))`),
}, (table) => ({
hourSiteUnique: uniqueIndex('site_hour_usage_hour_site_unique').on(table.bucketStartUtc, table.siteId),
hourIdx: index('site_hour_usage_hour_idx').on(table.bucketStartUtc),
siteIdx: index('site_hour_usage_site_id_idx').on(table.siteId),
nonNegative: check(
'site_hour_usage_non_negative',
sql`${table.totalCalls} >= 0 and ${table.successCalls} >= 0 and ${table.failedCalls} >= 0 and ${table.totalTokens} >= 0 and ${table.totalSummarySpend} >= 0 and ${table.totalSiteSpend} >= 0 and ${table.totalLatencyMs} >= 0 and ${table.latencyCount} >= 0`,
),
}));
export const modelDayUsage = sqliteTable('model_day_usage', {
id: integer('id').primaryKey({ autoIncrement: true }),
localDay: text('local_day').notNull(),
siteId: integer('site_id').notNull().references(() => sites.id, { onDelete: 'cascade' }),
model: text('model').notNull(),
totalCalls: integer('total_calls').notNull().default(0),
successCalls: integer('success_calls').notNull().default(0),
failedCalls: integer('failed_calls').notNull().default(0),
totalTokens: integer('total_tokens').notNull().default(0),
totalSpend: real('total_spend').notNull().default(0),
totalLatencyMs: integer('total_latency_ms').notNull().default(0),
latencyCount: integer('latency_count').notNull().default(0),
createdAt: text('created_at').default(sql`(datetime('now'))`),
updatedAt: text('updated_at').default(sql`(datetime('now'))`),
}, (table) => ({
daySiteModelUnique: uniqueIndex('model_day_usage_day_site_model_unique').on(table.localDay, table.siteId, table.model),
dayIdx: index('model_day_usage_day_idx').on(table.localDay),
siteIdx: index('model_day_usage_site_id_idx').on(table.siteId),
modelIdx: index('model_day_usage_model_idx').on(table.model),
nonNegative: check(
'model_day_usage_non_negative',
sql`${table.totalCalls} >= 0 and ${table.successCalls} >= 0 and ${table.failedCalls} >= 0 and ${table.totalTokens} >= 0 and ${table.totalSpend} >= 0 and ${table.totalLatencyMs} >= 0 and ${table.latencyCount} >= 0`,
),
}));
export const downstreamApiKeys = sqliteTable('downstream_api_keys', {
id: integer('id').primaryKey({ autoIncrement: true }),
name: text('name').notNull(),
src/server/db/schemaArtifactGenerator.test.ts
+2
View File
@@ -29,6 +29,8 @@ describe('schema artifact generator', () => {
expect(artifacts.mysqlBootstrap).toContain('CREATE TABLE IF NOT EXISTS `sites`');
expect(artifacts.mysqlBootstrap).toContain('CREATE TABLE IF NOT EXISTS `site_disabled_models`');
expect(artifacts.postgresBootstrap).toContain('CREATE TABLE IF NOT EXISTS "account_tokens"');
expect(artifacts.mysqlBootstrap).toContain('`snapshot_key` TEXT NOT NULL');
expect(artifacts.postgresBootstrap).toContain('"snapshot_key" TEXT NOT NULL');
expect(artifacts.postgresBootstrap).toContain('"token_group"');
});
src/server/db/schemaContract.test.ts
+5
View File
@@ -19,6 +19,11 @@ describe('schema contract generation', () => {
expect(contract.tables.site_disabled_models).toBeDefined();
expect(contract.tables.downstream_api_keys).toBeDefined();
expect(contract.tables.proxy_files).toBeDefined();
expect(contract.tables.admin_snapshots.columns.snapshot_key).toMatchObject({
logicalType: 'text',
notNull: true,
primaryKey: false,
});
expect(contract.tables.proxy_video_tasks).toBeDefined();
expect(contract.tables.route_channels.columns.source_model).toBeDefined();
expect(contract.tables.route_channels.columns.last_selected_at).toBeDefined();
src/server/db/schemaMetadata.ts
+1 -1
View File
@@ -39,7 +39,7 @@ function isDateTimeLikeColumn(columnName: string, defaultValue: string | null):
function isJsonLikeColumn(columnName: string): boolean {
const normalizedColumn = columnName.toLowerCase();
return normalizedColumn.endsWith('_json')
|| normalizedColumn.includes('snapshot')
|| normalizedColumn.endsWith('_snapshot')
|| normalizedColumn.includes('mapping')
|| normalizedColumn.includes('headers')
|| normalizedColumn.includes('config')
src/server/index.ts
+12
View File
@@ -49,6 +49,14 @@ import {
stopSub2ApiManagedRefreshScheduler,
} from './services/sub2apiRefreshScheduler.js';
import { startUpdateCenterPolling, stopUpdateCenterPolling } from './services/updateCenterPollingService.js';
import {
startAdminSnapshotWarmScheduler,
stopAdminSnapshotWarmScheduler,
} from './services/adminSnapshotWarmService.js';
import {
startUsageAggregationProjectorScheduler,
stopUsageAggregationProjectorScheduler,
} from './services/usageAggregationService.js';
import { reloadBackupWebdavScheduler } from './services/backupService.js';
import { ensureRuntimeDatabaseReady } from './runtimeDatabaseBootstrap.js';
import { isPublicApiRoute, registerDesktopRoutes } from './desktop.js';
@@ -260,6 +268,8 @@ startModelAvailabilityProbeScheduler();
startChannelRecoveryProbeScheduler();
startSub2ApiManagedRefreshScheduler();
startUpdateCenterPolling();
startUsageAggregationProjectorScheduler();
startAdminSnapshotWarmScheduler();
try {
await startOAuthLoopbackCallbackServers();
} catch (error) {
@@ -274,6 +284,8 @@ app.addHook('onClose', async () => {
stopProxyLogRetentionService();
stopModelAvailabilityProbeScheduler();
stopChannelRecoveryProbeScheduler();
await stopUsageAggregationProjectorScheduler();
await stopAdminSnapshotWarmScheduler();
await stopSub2ApiManagedRefreshScheduler();
await stopOAuthLoopbackCallbackServers();
});
src/server/proxy-core/surfaces/sharedSurface.usage-source.test.ts
+1
View File
@@ -58,6 +58,7 @@ vi.mock('../orchestration/upstreamRequest.js', () => ({
}));
vi.mock('../../services/oauth/quota.js', () => ({
recordOauthQuotaHeadersSnapshot: vi.fn(),
recordOauthQuotaResetHint: vi.fn(),
}));
src/server/routes/api/accountTokens.sync.test.ts
+10 -2
View File
@@ -2,7 +2,7 @@ import Fastify, { type FastifyInstance } from 'fastify';
import { describe, expect, it, beforeAll, beforeEach, afterAll, vi } from 'vitest';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { mkdtempSync } from 'node:fs';
import { mkdtempSync, rmSync } from 'node:fs';
import { and, eq, sql } from 'drizzle-orm';
import { mergeAccountExtraConfig } from '../../services/accountExtraConfig.js';
@@ -32,6 +32,7 @@ describe('account tokens sync routes with site status', () => {
let schema: DbModule['schema'];
let maskToken: AccountTokenServiceModule['maskToken'];
let dataDir = '';
let previousDataDir: string | undefined;
let seedId = 0;
const nextSeed = () => {
@@ -61,7 +62,9 @@ describe('account tokens sync routes with site status', () => {
};
beforeAll(async () => {
previousDataDir = process.env.DATA_DIR;
dataDir = mkdtempSync(join(tmpdir(), 'metapi-account-tokens-sync-'));
vi.resetModules();
process.env.DATA_DIR = dataDir;
await import('../../db/migrate.js');
@@ -96,7 +99,12 @@ describe('account tokens sync routes with site status', () => {
afterAll(async () => {
await app.close();
delete process.env.DATA_DIR;
if (previousDataDir === undefined) {
delete process.env.DATA_DIR;
} else {
process.env.DATA_DIR = previousDataDir;
}
rmSync(dataDir, { recursive: true, force: true });
});
it('returns skipped for single-account sync when site is disabled', async () => {
src/server/routes/api/accounts.credentialMode.test.ts
+46 -31
View File
@@ -207,11 +207,16 @@ describe('accounts credential mode', { timeout: 15_000 }, () => {
});
expect(listResponse.statusCode).toBe(200);
const list = listResponse.json() as Array<{
id: number;
runtimeHealth?: { state?: string; reason?: string };
capabilities?: { proxyOnly?: boolean };
}>;
const body = listResponse.json() as {
generatedAt: string;
accounts: Array<{
id: number;
runtimeHealth?: { state?: string; reason?: string };
capabilities?: { proxyOnly?: boolean };
}>;
sites: any[];
};
const list = body.accounts;
expect(list).toHaveLength(1);
expect(list[0]?.capabilities?.proxyOnly).toBe(true);
expect(list[0]?.runtimeHealth).toMatchObject({
@@ -258,23 +263,28 @@ describe('accounts credential mode', { timeout: 15_000 }, () => {
});
expect(listResponse.statusCode).toBe(200);
const list = listResponse.json() as Array<{
id: number;
credentialMode?: string;
capabilities?: {
canCheckin?: boolean;
canRefreshBalance?: boolean;
proxyOnly?: boolean;
};
}>;
const body = listResponse.json() as {
generatedAt: string;
accounts: Array<{
id: number;
credentialMode?: string;
capabilities?: {
canCheckin?: boolean;
canRefreshBalance?: boolean;
proxyOnly?: boolean;
};
}>;
sites: any[];
};
const list = body.accounts;
const item = list.find((entry) => entry.id === account.id);
expect(item?.credentialMode).toBe('session');
expect(item?.capabilities).toMatchObject({
canCheckin: false,
canRefreshBalance: false,
proxyOnly: true,
});
expect(item?.capabilities).toMatchObject({
canCheckin: false,
canRefreshBalance: false,
proxyOnly: true,
});
});
it('uses structured oauth columns when listing oauth account capabilities and runtime health', async () => {
const site = await db.insert(schema.sites).values({
@@ -314,18 +324,23 @@ describe('accounts credential mode', { timeout: 15_000 }, () => {
});
expect(listResponse.statusCode).toBe(200);
const list = listResponse.json() as Array<{
id: number;
capabilities?: {
canCheckin?: boolean;
canRefreshBalance?: boolean;
proxyOnly?: boolean;
};
runtimeHealth?: {
state?: string;
reason?: string;
};
}>;
const body = listResponse.json() as {
generatedAt: string;
accounts: Array<{
id: number;
capabilities?: {
canCheckin?: boolean;
canRefreshBalance?: boolean;
proxyOnly?: boolean;
};
runtimeHealth?: {
state?: string;
reason?: string;
};
}>;
sites: any[];
};
const list = body.accounts;
const item = list.find((entry) => entry.id === account.id);
expect(item?.capabilities).toMatchObject({
canCheckin: false,
src/server/routes/api/accounts.snapshot-v2.test.ts
+142
View File
@@ -0,0 +1,142 @@
import Fastify, { type FastifyInstance } from "fastify";
import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import {
formatLocalDate,
formatUtcSqlDateTime,
} from "../../services/localTimeService.js";
type DbModule = typeof import("../../db/index.js");
describe("accounts snapshot v2", () => {
let app: FastifyInstance;
let db: DbModule["db"];
let schema: DbModule["schema"];
let dataDir = "";
let previousDataDir: string | undefined;
beforeAll(async () => {
previousDataDir = process.env.DATA_DIR;
dataDir = mkdtempSync(join(tmpdir(), "metapi-accounts-snapshot-v2-"));
process.env.DATA_DIR = dataDir;
await import("../../db/migrate.js");
const dbModule = await import("../../db/index.js");
const routesModule = await import("./accounts.js");
db = dbModule.db;
schema = dbModule.schema;
app = Fastify();
await app.register(routesModule.accountsRoutes);
});
beforeEach(async () => {
await db.delete(schema.adminSnapshots).run();
await db.delete(schema.proxyLogs).run();
await db.delete(schema.checkinLogs).run();
await db.delete(schema.routeChannels).run();
await db.delete(schema.tokenRoutes).run();
await db.delete(schema.tokenModelAvailability).run();
await db.delete(schema.modelAvailability).run();
await db.delete(schema.accountTokens).run();
await db.delete(schema.accounts).run();
await db.delete(schema.sites).run();
});
afterAll(async () => {
await app.close();
if (previousDataDir === undefined) {
delete process.env.DATA_DIR;
} else {
process.env.DATA_DIR = previousDataDir;
}
rmSync(dataDir, { recursive: true, force: true });
});
it("returns accounts and sites in one snapshot payload", async () => {
const today = formatLocalDate(new Date());
const site = await db
.insert(schema.sites)
.values({
name: "snapshot-site",
url: "https://snapshot-site.example.com",
platform: "new-api",
})
.returning()
.get();
const account = await db
.insert(schema.accounts)
.values({
siteId: site.id,
username: "snapshot-user",
accessToken: "snapshot-token",
status: "active",
balance: 18.5,
extraConfig: JSON.stringify({
todayIncomeSnapshot: {
day: today,
baseline: 3.2,
latest: 3.2,
updatedAt: `${today}T08:00:00.000Z`,
},
}),
})
.returning()
.get();
await db
.insert(schema.proxyLogs)
.values({
accountId: account.id,
status: "success",
estimatedCost: 1.25,
createdAt: formatUtcSqlDateTime(new Date()),
})
.run();
await db
.insert(schema.checkinLogs)
.values({
accountId: account.id,
status: "success",
reward: "",
message: "checkin success",
createdAt: `${today} 09:00:00`,
})
.run();
const response = await app.inject({
method: "GET",
url: "/api/accounts",
});
expect(response.statusCode).toBe(200);
expect(response.headers["x-accounts-snapshot-cache"]).toBeTruthy();
const body = response.json() as {
generatedAt: string;
accounts: Array<{
id: number;
site: { id: number; name: string };
todaySpend: number;
todayReward: number;
}>;
sites: Array<{ id: number; name: string }>;
};
expect(Date.parse(body.generatedAt)).not.toBeNaN();
expect(body.sites).toEqual([
expect.objectContaining({ id: site.id, name: "snapshot-site" }),
]);
expect(body.accounts).toEqual([
expect.objectContaining({
id: account.id,
site: expect.objectContaining({ id: site.id, name: "snapshot-site" }),
todaySpend: 1.25,
todayReward: 3.2,
}),
]);
});
});
src/server/routes/api/accounts.todayRewardFallback.test.ts
+30 -5
View File
@@ -89,7 +89,12 @@ describe('accounts api today reward fallback', () => {
});
expect(response.statusCode).toBe(200);
const rows = response.json() as Array<{ id: number; todayReward: number }>;
const body = response.json() as {
generatedAt: string;
accounts: Array<{ id: number; todayReward: number }>;
sites: any[];
};
const rows = body.accounts;
const target = rows.find((row) => row.id === account.id);
expect(target?.todayReward).toBe(12.5);
});
@@ -122,7 +127,12 @@ describe('accounts api today reward fallback', () => {
});
expect(response.statusCode).toBe(200);
const rows = response.json() as Array<{ id: number; todayReward: number }>;
const body = response.json() as {
generatedAt: string;
accounts: Array<{ id: number; todayReward: number }>;
sites: any[];
};
const rows = body.accounts;
const target = rows.find((row) => row.id === account.id);
expect(target?.todayReward).toBe(1.8);
});
@@ -163,7 +173,12 @@ describe('accounts api today reward fallback', () => {
});
expect(response.statusCode).toBe(200);
const rows = response.json() as Array<{ id: number; todayReward: number }>;
const body = response.json() as {
generatedAt: string;
accounts: Array<{ id: number; todayReward: number }>;
sites: any[];
};
const rows = body.accounts;
const target = rows.find((row) => row.id === account.id);
expect(target?.todayReward).toBe(1.2);
});
@@ -215,7 +230,12 @@ describe('accounts api today reward fallback', () => {
});
expect(response.statusCode).toBe(200);
const rows = response.json() as Array<{ id: number; todaySpend: number }>;
const body = response.json() as {
generatedAt: string;
accounts: Array<{ id: number; todaySpend: number }>;
sites: any[];
};
const rows = body.accounts;
const target = rows.find((row) => row.id === account.id);
expect(target?.todaySpend).toBe(2);
});
@@ -251,7 +271,12 @@ describe('accounts api today reward fallback', () => {
});
expect(response.statusCode).toBe(200);
const rows = response.json() as Array<{ id: number; todaySpend: number }>;
const body = response.json() as {
generatedAt: string;
accounts: Array<{ id: number; todaySpend: number }>;
sites: any[];
};
const rows = body.accounts;
const target = rows.find((row) => row.id === account.id);
expect(target?.todaySpend).toBe(2.25);
});
src/server/routes/api/accounts.ts
+1288 -951
View File
File diff suppressed because it is too large Load Diff
src/server/routes/api/oauth.test.ts
+14 -3
View File
@@ -1,10 +1,9 @@
import Fastify, { type FastifyInstance } from 'fastify';
import { eq } from 'drizzle-orm';
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
import { mkdtempSync } from 'node:fs';
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { config } from '../../config.js';
const fetchMock = vi.fn();
const undiciAgentCtorMock = vi.fn();
@@ -26,6 +25,7 @@ vi.mock('undici', () => ({
type DbModule = typeof import('../../db/index.js');
type RouteRefreshWorkflowModule = typeof import('../../services/routeRefreshWorkflow.js');
type ConfigModule = typeof import('../../config.js');
function buildJwt(payload: Record<string, unknown>) {
const encode = (value: unknown) => Buffer.from(JSON.stringify(value))
@@ -65,16 +65,22 @@ describe('oauth routes', { timeout: 15_000 }, () => {
let db: DbModule['db'];
let schema: DbModule['schema'];
let rebuildRoutesOnly: RouteRefreshWorkflowModule['rebuildRoutesOnly'];
let config: ConfigModule['config'];
let dataDir = '';
let previousDataDir: string | undefined;
beforeAll(async () => {
previousDataDir = process.env.DATA_DIR;
dataDir = mkdtempSync(join(tmpdir(), 'metapi-oauth-routes-'));
vi.resetModules();
process.env.DATA_DIR = dataDir;
await import('../../db/migrate.js');
const configModule = await import('../../config.js');
const dbModule = await import('../../db/index.js');
const routesModule = await import('./oauth.js');
const routeRefreshWorkflow = await import('../../services/routeRefreshWorkflow.js');
config = configModule.config;
db = dbModule.db;
schema = dbModule.schema;
rebuildRoutesOnly = routeRefreshWorkflow.rebuildRoutesOnly;
@@ -106,7 +112,12 @@ describe('oauth routes', { timeout: 15_000 }, () => {
afterAll(async () => {
await app.close();
delete process.env.DATA_DIR;
if (previousDataDir === undefined) {
delete process.env.DATA_DIR;
} else {
process.env.DATA_DIR = previousDataDir;
}
rmSync(dataDir, { recursive: true, force: true });
});
it('lists multi-provider oauth metadata', async () => {
src/server/routes/api/stats.proxy-logs.test.ts
+592 -378
View File
File diff suppressed because it is too large Load Diff
src/server/routes/api/stats.snapshot-v2.test.ts
+181
View File
@@ -0,0 +1,181 @@
import Fastify, { type FastifyInstance } from "fastify";
import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { formatUtcSqlDateTime } from "../../services/localTimeService.js";
type DbModule = typeof import("../../db/index.js");
describe("stats snapshot v2 routes", () => {
let app: FastifyInstance;
let db: DbModule["db"];
let schema: DbModule["schema"];
let dataDir = "";
let previousDataDir: string | undefined;
beforeAll(async () => {
previousDataDir = process.env.DATA_DIR;
dataDir = mkdtempSync(join(tmpdir(), "metapi-stats-snapshot-v2-"));
process.env.DATA_DIR = dataDir;
await import("../../db/migrate.js");
const dbModule = await import("../../db/index.js");
const routesModule = await import("./stats.js");
const sitesRoutesModule = await import("./sites.js");
db = dbModule.db;
schema = dbModule.schema;
app = Fastify();
await app.register(routesModule.statsRoutes);
await app.register(sitesRoutesModule.sitesRoutes);
});
beforeEach(async () => {
await db.delete(schema.adminSnapshots).run();
await db.delete(schema.proxyLogs).run();
await db.delete(schema.checkinLogs).run();
await db.delete(schema.routeChannels).run();
await db.delete(schema.tokenRoutes).run();
await db.delete(schema.tokenModelAvailability).run();
await db.delete(schema.modelAvailability).run();
await db.delete(schema.accountTokens).run();
await db.delete(schema.accounts).run();
await db.delete(schema.sites).run();
});
afterAll(async () => {
await app.close();
if (previousDataDir === undefined) {
delete process.env.DATA_DIR;
} else {
process.env.DATA_DIR = previousDataDir;
}
rmSync(dataDir, { recursive: true, force: true });
});
it("returns dashboard and site snapshot payloads for progressive loading", async () => {
const recentLogCreatedAt = formatUtcSqlDateTime(new Date());
const site = await db
.insert(schema.sites)
.values({
name: "stats-site",
url: "https://stats-site.example.com",
platform: "new-api",
})
.returning()
.get();
const account = await db
.insert(schema.accounts)
.values({
siteId: site.id,
username: "stats-user",
accessToken: "stats-token",
balance: 42,
status: "active",
})
.returning()
.get();
await db
.insert(schema.proxyLogs)
.values([
{
accountId: account.id,
status: "success",
modelRequested: "gpt-4o",
modelActual: "gpt-4o",
totalTokens: 120,
estimatedCost: 0.5,
latencyMs: 320,
createdAt: recentLogCreatedAt,
},
{
accountId: account.id,
status: "failed",
modelRequested: "gpt-4o-mini",
modelActual: "gpt-4o-mini",
totalTokens: 60,
estimatedCost: 0.25,
latencyMs: 220,
createdAt: recentLogCreatedAt,
},
{
accountId: account.id,
status: "success",
modelRequested: "gpt-4.1-mini",
modelActual: "gpt-4.1-mini",
totalTokens: 40,
estimatedCost: 0.1,
latencyMs: 180,
createdAt: formatUtcSqlDateTime(
new Date(Date.now() - (24 * 60 + 30) * 60_000),
),
},
])
.run();
const summaryResponse = await app.inject({
method: "GET",
url: "/api/stats/dashboard?view=summary",
});
expect(summaryResponse.statusCode).toBe(200);
expect(summaryResponse.headers["x-dashboard-summary-cache"]).toBeTruthy();
const summary = summaryResponse.json() as {
generatedAt: string;
totalBalance: number;
proxy24h: { total: number };
};
expect(Date.parse(summary.generatedAt)).not.toBeNaN();
expect(summary.totalBalance).toBe(42);
expect(summary.proxy24h.total).toBe(2);
const insightsResponse = await app.inject({
method: "GET",
url: "/api/stats/dashboard?view=insights",
});
expect(insightsResponse.statusCode).toBe(200);
const insights = insightsResponse.json() as {
generatedAt: string;
siteAvailability: Array<{ siteId: number }>;
modelAnalysis: { totals: { calls: number } };
};
expect(Date.parse(insights.generatedAt)).not.toBeNaN();
expect(insights.siteAvailability).toEqual([
expect.objectContaining({ siteId: site.id }),
]);
expect(insights.modelAnalysis.totals.calls).toBe(3);
const siteDistributionResponse = await app.inject({
method: "GET",
url: "/api/stats/site-distribution?days=7",
});
expect(siteDistributionResponse.statusCode).toBe(200);
const siteDistribution = siteDistributionResponse.json() as {
distribution: Array<{ siteId: number; totalSpend: number }>;
};
expect(siteDistribution.distribution).toEqual([
expect.objectContaining({ siteId: site.id, totalSpend: 0.85 }),
]);
const siteTrendResponse = await app.inject({
method: "GET",
url: "/api/stats/site-trend?days=7",
});
expect(siteTrendResponse.statusCode).toBe(200);
const siteTrend = siteTrendResponse.json() as {
trend: Array<{ date: string }>;
};
expect(siteTrend.trend.length).toBeGreaterThan(0);
const sitesResponse = await app.inject({
method: "GET",
url: "/api/sites",
});
expect(sitesResponse.statusCode).toBe(200);
const sites = sitesResponse.json() as Array<{ id: number; name: string }>;
expect(sites).toEqual([
expect.objectContaining({ id: site.id, name: "stats-site" }),
]);
});
});
src/server/routes/api/stats.ts
+1505 -1189
View File
File diff suppressed because it is too large Load Diff
src/server/routes/proxy/chat.count-tokens.test.ts
+1
View File
@@ -85,6 +85,7 @@ vi.mock('../../services/proxyDebugTraceRuntime.js', () => ({
}));
vi.mock('../../services/oauth/quota.js', () => ({
recordOauthQuotaHeadersSnapshot: async () => undefined,
recordOauthQuotaResetHint: async () => undefined,
}));
src/server/routes/proxy/completions.siteApiEndpoint.test.ts
+11 -3
View File
@@ -35,9 +35,17 @@ vi.mock('../../services/tokenRouter.js', () => ({
invalidateTokenRouterCache: vi.fn(),
}));
vi.mock('../../services/routeRefreshWorkflow.js', () => ({
refreshModelsAndRebuildRoutes: (...args: unknown[]) => refreshModelsAndRebuildRoutesMock(...args),
}));
vi.mock('../../services/routeRefreshWorkflow.js', async () => {
const actual =
await vi.importActual<typeof import('../../services/routeRefreshWorkflow.js')>(
'../../services/routeRefreshWorkflow.js',
);
return {
...actual,
refreshModelsAndRebuildRoutes: (...args: unknown[]) =>
refreshModelsAndRebuildRoutesMock(...args),
};
});
vi.mock('../../services/alertService.js', () => ({
reportProxyAllFailed: (...args: unknown[]) => reportProxyAllFailedMock(...args),
src/server/routes/proxy/completions.usage-source.test.ts
+11 -3
View File
@@ -32,9 +32,17 @@ vi.mock('../../services/tokenRouter.js', () => ({
invalidateTokenRouterCache: vi.fn(),
}));
vi.mock('../../services/routeRefreshWorkflow.js', () => ({
refreshModelsAndRebuildRoutes: (...args: unknown[]) => refreshModelsAndRebuildRoutesMock(...args),
}));
vi.mock('../../services/routeRefreshWorkflow.js', async () => {
const actual =
await vi.importActual<typeof import('../../services/routeRefreshWorkflow.js')>(
'../../services/routeRefreshWorkflow.js',
);
return {
...actual,
refreshModelsAndRebuildRoutes: (...args: unknown[]) =>
refreshModelsAndRebuildRoutesMock(...args),
};
});
vi.mock('../../services/alertService.js', () => ({
reportProxyAllFailed: (...args: unknown[]) => reportProxyAllFailedMock(...args),
src/server/routes/proxy/embeddings.siteApiEndpoint.test.ts
+11 -3
View File
@@ -39,9 +39,17 @@ vi.mock('../../services/tokenRouter.js', () => ({
invalidateTokenRouterCache: vi.fn(),
}));
vi.mock('../../services/routeRefreshWorkflow.js', () => ({
refreshModelsAndRebuildRoutes: (...args: unknown[]) => refreshModelsAndRebuildRoutesMock(...args),
}));
vi.mock('../../services/routeRefreshWorkflow.js', async () => {
const actual =
await vi.importActual<typeof import('../../services/routeRefreshWorkflow.js')>(
'../../services/routeRefreshWorkflow.js',
);
return {
...actual,
refreshModelsAndRebuildRoutes: (...args: unknown[]) =>
refreshModelsAndRebuildRoutesMock(...args),
};
});
vi.mock('../../services/alertService.js', () => ({
reportProxyAllFailed: (...args: unknown[]) => reportProxyAllFailedMock(...args),
src/server/routes/proxy/responses.codex-oauth.test.ts
+1
View File
@@ -87,6 +87,7 @@ vi.mock('../../services/oauth/refreshSingleflight.js', () => ({
}));
vi.mock('../../services/oauth/quota.js', () => ({
recordOauthQuotaHeadersSnapshot: async () => undefined,
recordOauthQuotaResetHint: (...args: unknown[]) => recordOauthQuotaResetHintMock(...args),
}));
src/server/routes/proxy/responses.websocket.test.ts
+1
View File
@@ -96,6 +96,7 @@ vi.mock('../../services/proxyUsageFallbackService.js', () => ({
}));
vi.mock('../../services/oauth/quota.js', () => ({
recordOauthQuotaHeadersSnapshot: async () => undefined,
recordOauthQuotaResetHint: async () => undefined,
}));
src/server/services/accountHealthService.ts
+1 -1
View File
@@ -8,7 +8,7 @@ import {
export type RuntimeHealthState = 'healthy' | 'unhealthy' | 'degraded' | 'unknown' | 'disabled';
type RuntimeHealthInfo = {
export type RuntimeHealthInfo = {
state: RuntimeHealthState;
reason: string;
source: string;
src/server/services/accountsOverviewService.ts
+230
View File
@@ -0,0 +1,230 @@
import { and, eq, gte, lt, sql } from "drizzle-orm";
import { db, schema } from "../db/index.js";
import {
getCredentialModeFromExtraConfig,
hasOauthProvider,
type AccountCredentialMode,
} from "./accountExtraConfig.js";
import {
buildRuntimeHealthForAccount,
type RuntimeHealthInfo,
} from "./accountHealthService.js";
import { parseCheckinRewardAmount } from "./checkinRewardParser.js";
import { getLocalDayRangeUtc } from "./localTimeService.js";
import {
readSnapshotCache,
type SnapshotEnvelope,
} from "./snapshotCacheService.js";
import { estimateRewardWithTodayIncomeFallback } from "./todayIncomeRewardService.js";
import { createAdminSnapshotPersistence } from "./adminSnapshotStore.js";
export type AccountCapabilities = {
canCheckin: boolean;
canRefreshBalance: boolean;
proxyOnly: boolean;
};
export type AccountOverviewRow = typeof schema.accounts.$inferSelect & {
site: typeof schema.sites.$inferSelect;
credentialMode: AccountCredentialMode;
capabilities: AccountCapabilities;
todaySpend: number;
todayReward: number;
runtimeHealth: RuntimeHealthInfo;
};
export type AccountsSnapshotPayload = {
accounts: AccountOverviewRow[];
sites: Array<typeof schema.sites.$inferSelect>;
};
const ACCOUNTS_SNAPSHOT_TTL_MS = 15_000;
const accountsSnapshotPersistence =
createAdminSnapshotPersistence<AccountsSnapshotPayload>({
namespace: "accounts-snapshot",
key: "all",
});
function hasSessionTokenValue(value: string | null | undefined): boolean {
return typeof value === "string" && value.trim().length > 0;
}
function resolveStoredCredentialMode(
account: typeof schema.accounts.$inferSelect,
): AccountCredentialMode {
const fromConfig = getCredentialModeFromExtraConfig(account.extraConfig);
if (fromConfig && fromConfig !== "auto") return fromConfig;
return hasSessionTokenValue(account.accessToken) ? "session" : "apikey";
}
function buildCapabilitiesFromCredentialMode(
credentialMode: AccountCredentialMode,
hasSessionToken: boolean,
oauthIdentity?:
| string
| null
| Pick<
typeof schema.accounts.$inferSelect,
"extraConfig" | "oauthProvider"
>,
): AccountCapabilities {
if (hasOauthProvider(oauthIdentity)) {
return {
canCheckin: false,
canRefreshBalance: false,
proxyOnly: true,
};
}
const sessionCapable =
credentialMode === "session"
? hasSessionToken
: credentialMode === "apikey"
? false
: hasSessionToken;
return {
canCheckin: sessionCapable,
canRefreshBalance: sessionCapable,
proxyOnly: !sessionCapable,
};
}
function buildCapabilitiesForAccount(
account: typeof schema.accounts.$inferSelect,
): AccountCapabilities {
const credentialMode = resolveStoredCredentialMode(account);
return buildCapabilitiesFromCredentialMode(
credentialMode,
hasSessionTokenValue(account.accessToken),
account,
);
}
async function loadAccountsSnapshotPayload(): Promise<AccountsSnapshotPayload> {
const [rows, sites] = await Promise.all([
db
.select()
.from(schema.accounts)
.innerJoin(schema.sites, eq(schema.accounts.siteId, schema.sites.id))
.all(),
db.select().from(schema.sites).all(),
]);
const { localDay, startUtc, endUtc } = getLocalDayRangeUtc();
const [todaySpendRows, modelCountRows, todayCheckins] = await Promise.all([
db
.select({
accountId: schema.proxyLogs.accountId,
totalSpend: sql<number>`coalesce(sum(${schema.proxyLogs.estimatedCost}), 0)`,
})
.from(schema.proxyLogs)
.where(
and(
gte(schema.proxyLogs.createdAt, startUtc),
lt(schema.proxyLogs.createdAt, endUtc),
),
)
.groupBy(schema.proxyLogs.accountId)
.all(),
db
.select({
accountId: schema.modelAvailability.accountId,
modelCount: sql<number>`count(*)`,
})
.from(schema.modelAvailability)
.where(eq(schema.modelAvailability.available, true))
.groupBy(schema.modelAvailability.accountId)
.all(),
db
.select({
accountId: schema.checkinLogs.accountId,
reward: schema.checkinLogs.reward,
message: schema.checkinLogs.message,
})
.from(schema.checkinLogs)
.where(
and(
gte(schema.checkinLogs.createdAt, startUtc),
lt(schema.checkinLogs.createdAt, endUtc),
eq(schema.checkinLogs.status, "success"),
),
)
.all(),
]);
const spendByAccount: Record<number, number> = {};
for (const row of todaySpendRows) {
if (row.accountId == null) continue;
spendByAccount[row.accountId] = Number(row.totalSpend || 0);
}
const modelCountByAccount: Record<number, number> = {};
for (const row of modelCountRows) {
if (row.accountId == null) continue;
modelCountByAccount[row.accountId] = Number(row.modelCount || 0);
}
const rewardByAccount: Record<number, number> = {};
const successCountByAccount: Record<number, number> = {};
const parsedRewardCountByAccount: Record<number, number> = {};
for (const log of todayCheckins) {
successCountByAccount[log.accountId] =
(successCountByAccount[log.accountId] || 0) + 1;
const rewardNum =
parseCheckinRewardAmount(log.reward) ||
parseCheckinRewardAmount(log.message);
if (rewardNum <= 0) continue;
rewardByAccount[log.accountId] =
(rewardByAccount[log.accountId] || 0) + rewardNum;
parsedRewardCountByAccount[log.accountId] =
(parsedRewardCountByAccount[log.accountId] || 0) + 1;
}
return {
accounts: rows.map((row) => {
const credentialMode = resolveStoredCredentialMode(row.accounts);
const capabilities = buildCapabilitiesForAccount(row.accounts);
return {
...row.accounts,
site: row.sites,
credentialMode,
capabilities,
todaySpend:
Math.round((spendByAccount[row.accounts.id] || 0) * 1_000_000) /
1_000_000,
todayReward:
Math.round(
estimateRewardWithTodayIncomeFallback({
day: localDay,
successCount: successCountByAccount[row.accounts.id] || 0,
parsedRewardCount:
parsedRewardCountByAccount[row.accounts.id] || 0,
rewardSum: rewardByAccount[row.accounts.id] || 0,
extraConfig: row.accounts.extraConfig,
}) * 1_000_000,
) / 1_000_000,
runtimeHealth: buildRuntimeHealthForAccount({
accountStatus: row.accounts.status,
siteStatus: row.sites.status,
extraConfig: row.accounts.extraConfig,
sessionCapable: capabilities.canRefreshBalance,
hasDiscoveredModels: (modelCountByAccount[row.accounts.id] || 0) > 0,
}),
};
}),
sites,
};
}
export async function getAccountsSnapshot(options?: {
forceRefresh?: boolean;
}): Promise<SnapshotEnvelope<AccountsSnapshotPayload>> {
return readSnapshotCache({
namespace: "accounts-snapshot",
key: "all",
ttlMs: ACCOUNTS_SNAPSHOT_TTL_MS,
forceRefresh: options?.forceRefresh,
persistence: accountsSnapshotPersistence,
loader: loadAccountsSnapshotPayload,
});
}
src/server/services/adminSnapshotStore.test.ts
+99
View File
@@ -0,0 +1,99 @@
import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
type DbModule = typeof import("../db/index.js");
type AdminSnapshotStoreModule = typeof import("./adminSnapshotStore.js");
describe("adminSnapshotStore", () => {
let db: DbModule["db"];
let schema: DbModule["schema"];
let readAdminSnapshot: AdminSnapshotStoreModule["readAdminSnapshot"];
let writeAdminSnapshot: AdminSnapshotStoreModule["writeAdminSnapshot"];
let deleteExpiredAdminSnapshots: AdminSnapshotStoreModule["deleteExpiredAdminSnapshots"];
let dataDir = "";
let previousDataDir: string | undefined;
beforeAll(async () => {
previousDataDir = process.env.DATA_DIR;
dataDir = mkdtempSync(join(tmpdir(), "metapi-admin-snapshot-store-"));
process.env.DATA_DIR = dataDir;
await import("../db/migrate.js");
const dbModule = await import("../db/index.js");
const storeModule = await import("./adminSnapshotStore.js");
db = dbModule.db;
schema = dbModule.schema;
readAdminSnapshot = storeModule.readAdminSnapshot;
writeAdminSnapshot = storeModule.writeAdminSnapshot;
deleteExpiredAdminSnapshots = storeModule.deleteExpiredAdminSnapshots;
});
beforeEach(async () => {
await db.delete(schema.adminSnapshots).run();
});
afterAll(() => {
if (previousDataDir === undefined) {
delete process.env.DATA_DIR;
} else {
process.env.DATA_DIR = previousDataDir;
}
rmSync(dataDir, { recursive: true, force: true });
});
it("persists and reloads admin snapshot payloads from the runtime database", async () => {
await writeAdminSnapshot(
{ namespace: "dashboard-summary", key: "default" },
{
payload: { totalBalance: 12.5, totalAccounts: 3 },
generatedAt: "2026-04-09T00:00:00.000Z",
expiresAt: "2026-04-09T00:00:10.000Z",
staleUntil: "2026-04-09T00:01:00.000Z",
},
);
const record = await readAdminSnapshot<{
totalBalance: number;
totalAccounts: number;
}>({
namespace: "dashboard-summary",
key: "default",
});
expect(record).toEqual({
payload: { totalBalance: 12.5, totalAccounts: 3 },
generatedAt: "2026-04-09T00:00:00.000Z",
expiresAt: "2026-04-09T00:00:10.000Z",
staleUntil: "2026-04-09T00:01:00.000Z",
});
});
it("prunes snapshot rows whose stale window has elapsed", async () => {
await writeAdminSnapshot(
{ namespace: "dashboard-summary", key: "expired" },
{
payload: { stale: true },
generatedAt: "2026-04-09T00:00:00.000Z",
expiresAt: "2026-04-09T00:00:10.000Z",
staleUntil: "2026-04-09T00:00:11.000Z",
},
);
await writeAdminSnapshot(
{ namespace: "dashboard-summary", key: "fresh" },
{
payload: { stale: false },
generatedAt: "2026-04-09T00:00:00.000Z",
expiresAt: "2026-04-09T00:00:10.000Z",
staleUntil: "2026-04-09T00:10:00.000Z",
},
);
await deleteExpiredAdminSnapshots("2026-04-09T00:05:00.000Z");
const rows = await db.select().from(schema.adminSnapshots).all();
expect(rows).toHaveLength(1);
expect(rows[0]?.snapshotKey).toBe(JSON.stringify("fresh"));
});
});
src/server/services/adminSnapshotStore.ts
+118
View File
@@ -0,0 +1,118 @@
import { and, eq, lt } from "drizzle-orm";
import { db, schema } from "../db/index.js";
import type {
PersistedSnapshotRecord,
SnapshotPersistenceAdapter,
} from "./snapshotCacheService.js";
type AdminSnapshotIdentity = {
namespace: string;
key: string;
};
type AdminSnapshotRow = typeof schema.adminSnapshots.$inferSelect;
function serializeSnapshotKey(key: string) {
return JSON.stringify(key);
}
function buildSnapshotWhere(identity: AdminSnapshotIdentity) {
return and(
eq(schema.adminSnapshots.namespace, identity.namespace),
eq(schema.adminSnapshots.snapshotKey, serializeSnapshotKey(identity.key)),
);
}
function normalizeTimestamp(value: string, fallbackIso: string) {
const parsed = Date.parse(value);
return Number.isFinite(parsed) ? new Date(parsed).toISOString() : fallbackIso;
}
function coercePersistedRecord<T>(
row: AdminSnapshotRow,
): PersistedSnapshotRecord<T> | null {
const fallbackIso = new Date().toISOString();
try {
return {
payload: JSON.parse(row.payload) as T,
generatedAt: normalizeTimestamp(row.generatedAt, fallbackIso),
expiresAt: normalizeTimestamp(row.expiresAt, fallbackIso),
staleUntil: normalizeTimestamp(row.staleUntil, fallbackIso),
};
} catch {
return null;
}
}
export async function readAdminSnapshot<T>(
identity: AdminSnapshotIdentity,
): Promise<PersistedSnapshotRecord<T> | null> {
const row = await db
.select()
.from(schema.adminSnapshots)
.where(buildSnapshotWhere(identity))
.get();
if (!row) return null;
const record = coercePersistedRecord<T>(row);
if (record) return record;
await db
.delete(schema.adminSnapshots)
.where(eq(schema.adminSnapshots.id, row.id))
.run();
return null;
}
export async function writeAdminSnapshot<T>(
identity: AdminSnapshotIdentity,
record: PersistedSnapshotRecord<T>,
): Promise<void> {
const payload = JSON.stringify(record.payload);
const values = {
namespace: identity.namespace,
snapshotKey: serializeSnapshotKey(identity.key),
payload,
generatedAt: record.generatedAt,
expiresAt: record.expiresAt,
staleUntil: record.staleUntil,
updatedAt: new Date().toISOString(),
};
await (db
.insert(schema.adminSnapshots)
.values(values) as any)
.onConflictDoUpdate({
target: [
schema.adminSnapshots.namespace,
schema.adminSnapshots.snapshotKey,
],
set: {
payload: values.payload,
generatedAt: values.generatedAt,
expiresAt: values.expiresAt,
staleUntil: values.staleUntil,
updatedAt: values.updatedAt,
},
})
.run();
}
export async function deleteExpiredAdminSnapshots(beforeIso?: string) {
const cutoffIso = beforeIso || new Date().toISOString();
await db
.delete(schema.adminSnapshots)
.where(lt(schema.adminSnapshots.staleUntil, cutoffIso))
.run();
}
export function createAdminSnapshotPersistence<T>(
identity: AdminSnapshotIdentity,
): SnapshotPersistenceAdapter<T> {
return {
read: () => readAdminSnapshot<T>(identity),
write: (record) => writeAdminSnapshot(identity, record),
};
}
src/server/services/adminSnapshotWarmService.ts
+100
View File
@@ -0,0 +1,100 @@
import { getAccountsSnapshot } from "./accountsOverviewService.js";
import { deleteExpiredAdminSnapshots } from "./adminSnapshotStore.js";
import {
getDashboardInsightsSnapshot,
getDashboardSummarySnapshot,
} from "./dashboardSnapshotService.js";
import { getSiteStatsSnapshot } from "./siteStatsSnapshotService.js";
import { runUsageAggregationProjectionPass } from "./usageAggregationService.js";
const ADMIN_SNAPSHOT_WARM_INTERVAL_MS = 20_000;
const ADMIN_SNAPSHOT_PRUNE_EVERY_PASSES = 6;
type SnapshotWarmTarget = {
name: string;
refresh: () => Promise<unknown>;
};
let adminSnapshotWarmTimer: ReturnType<typeof setInterval> | null = null;
let adminSnapshotWarmInFlight: Promise<void> | null = null;
let completedWarmPassCount = 0;
const snapshotWarmTargets: SnapshotWarmTarget[] = [
{
name: "dashboard-summary",
refresh: () => getDashboardSummarySnapshot({ forceRefresh: true }),
},
{
name: "accounts-snapshot",
refresh: () => getAccountsSnapshot({ forceRefresh: true }),
},
{
name: "site-stats",
refresh: () => getSiteStatsSnapshot({ days: 7, forceRefresh: true }),
},
{
name: "dashboard-insights",
refresh: () => getDashboardInsightsSnapshot({ forceRefresh: true }),
},
];
async function runAdminSnapshotWarmPass() {
await runUsageAggregationProjectionPass();
const settled = await Promise.allSettled(
snapshotWarmTargets.map(async (target) => {
await target.refresh();
return target.name;
}),
);
for (const result of settled) {
if (result.status === "rejected") {
console.warn(
`[AdminSnapshotWarm] Failed to refresh snapshot: ${
result.reason instanceof Error
? result.reason.message
: String(result.reason || "unknown error")
}`,
);
}
}
completedWarmPassCount += 1;
if (completedWarmPassCount % ADMIN_SNAPSHOT_PRUNE_EVERY_PASSES === 0) {
await deleteExpiredAdminSnapshots();
}
}
export async function warmAdminSnapshotsOnce(): Promise<void> {
if (adminSnapshotWarmInFlight) {
return adminSnapshotWarmInFlight;
}
adminSnapshotWarmInFlight = runAdminSnapshotWarmPass().finally(() => {
adminSnapshotWarmInFlight = null;
});
return adminSnapshotWarmInFlight;
}
export function startAdminSnapshotWarmScheduler() {
if (adminSnapshotWarmTimer) return;
void warmAdminSnapshotsOnce();
adminSnapshotWarmTimer = setInterval(() => {
void warmAdminSnapshotsOnce();
}, ADMIN_SNAPSHOT_WARM_INTERVAL_MS);
}
export async function stopAdminSnapshotWarmScheduler() {
if (adminSnapshotWarmTimer) {
clearInterval(adminSnapshotWarmTimer);
adminSnapshotWarmTimer = null;
}
if (adminSnapshotWarmInFlight) {
await adminSnapshotWarmInFlight;
}
}
export async function __resetAdminSnapshotWarmSchedulerForTests() {
await stopAdminSnapshotWarmScheduler();
completedWarmPassCount = 0;
}
src/server/services/dashboardSnapshotService.ts
+355
View File
@@ -0,0 +1,355 @@
import { and, eq, gte, lt, sql } from "drizzle-orm";
import { db, schema } from "../db/index.js";
import { buildModelAnalysisFromDailyUsage } from "./modelAnalysisService.js";
import { parseCheckinRewardAmount } from "./checkinRewardParser.js";
import {
formatUtcSqlDateTime,
getLocalDayRangeUtc,
getLocalHourAnchor,
getLocalHourRangeStartUtc,
getLocalRangeStartDayKey,
} from "./localTimeService.js";
import {
readSnapshotCache,
type SnapshotEnvelope,
} from "./snapshotCacheService.js";
import {
buildSiteAvailabilitySummariesFromHourlyAggregates,
proxyCostSqlExpression,
type SiteAvailabilitySiteRow,
toRoundedMicroNumber,
} from "./statsShared.js";
import { estimateRewardWithTodayIncomeFallback } from "./todayIncomeRewardService.js";
import { createAdminSnapshotPersistence } from "./adminSnapshotStore.js";
import { runUsageAggregationProjectionPass } from "./usageAggregationService.js";
export type DashboardSummaryPayload = {
totalBalance: number;
totalUsed: number;
todaySpend: number;
todayReward: number;
activeAccounts: number;
totalAccounts: number;
todayCheckin: { success: number; failed: number; total: number };
proxy24h: {
success: number;
failed: number;
total: number;
totalTokens: number;
};
performance: {
windowSeconds: number;
requestsPerMinute: number;
tokensPerMinute: number;
};
};
export type DashboardInsightsPayload = {
siteAvailability: ReturnType<
typeof buildSiteAvailabilitySummariesFromHourlyAggregates
>;
modelAnalysis: ReturnType<typeof buildModelAnalysisFromDailyUsage>;
};
const DASHBOARD_SUMMARY_TTL_MS = 12_000;
const DASHBOARD_INSIGHTS_TTL_MS = 20_000;
const SITE_AVAILABILITY_BUCKET_COUNT = 24;
const dashboardSummaryPersistence =
createAdminSnapshotPersistence<DashboardSummaryPayload>({
namespace: "dashboard-summary",
key: "default",
});
const dashboardInsightsPersistence =
createAdminSnapshotPersistence<DashboardInsightsPayload>({
namespace: "dashboard-insights",
key: "default",
});
async function loadDashboardSummaryPayload(): Promise<DashboardSummaryPayload> {
await runUsageAggregationProjectionPass();
const accountRows = await db
.select()
.from(schema.accounts)
.innerJoin(schema.sites, eq(schema.accounts.siteId, schema.sites.id))
.where(eq(schema.sites.status, "active"))
.all();
const accounts = accountRows.map((row) => row.accounts);
const totalBalance = accounts.reduce(
(sum, account) => sum + (account.balance || 0),
0,
);
const activeCount = accounts.filter(
(account) => account.status === "active",
).length;
const {
localDay: today,
startUtc: todayStartUtc,
endUtc: todayEndUtc,
} = getLocalDayRangeUtc();
const nowTs = Date.now();
const last24hDate = formatUtcSqlDateTime(new Date(nowTs - 86_400_000));
const lastMinuteDate = formatUtcSqlDateTime(new Date(nowTs - 60_000));
const [
todayCheckinRows,
totalUsedRow,
proxy24hRow,
proxyPerformanceRow,
todaySpendRow,
] = await Promise.all([
db
.select()
.from(schema.checkinLogs)
.innerJoin(
schema.accounts,
eq(schema.checkinLogs.accountId, schema.accounts.id),
)
.innerJoin(schema.sites, eq(schema.accounts.siteId, schema.sites.id))
.where(
and(
gte(schema.checkinLogs.createdAt, todayStartUtc),
lt(schema.checkinLogs.createdAt, todayEndUtc),
eq(schema.sites.status, "active"),
),
)
.all(),
db
.select({
totalUsed: sql<number>`coalesce(sum(coalesce(${schema.siteDayUsage.totalSiteSpend}, 0)), 0)`,
})
.from(schema.siteDayUsage)
.innerJoin(schema.sites, eq(schema.siteDayUsage.siteId, schema.sites.id))
.where(eq(schema.sites.status, "active"))
.get(),
db
.select({
total: sql<number>`count(*)`,
success: sql<number>`coalesce(sum(case when ${schema.proxyLogs.status} = 'success' then 1 else 0 end), 0)`,
failed: sql<number>`coalesce(sum(case when ${schema.proxyLogs.status} = 'success' then 0 else 1 end), 0)`,
totalTokens: sql<number>`coalesce(sum(coalesce(${schema.proxyLogs.totalTokens}, 0)), 0)`,
})
.from(schema.proxyLogs)
.innerJoin(
schema.accounts,
eq(schema.proxyLogs.accountId, schema.accounts.id),
)
.innerJoin(schema.sites, eq(schema.accounts.siteId, schema.sites.id))
.where(
and(
gte(schema.proxyLogs.createdAt, last24hDate),
eq(schema.sites.status, "active"),
),
)
.get(),
db
.select({
total: sql<number>`count(*)`,
totalTokens: sql<number>`coalesce(sum(coalesce(${schema.proxyLogs.totalTokens}, 0)), 0)`,
})
.from(schema.proxyLogs)
.innerJoin(
schema.accounts,
eq(schema.proxyLogs.accountId, schema.accounts.id),
)
.innerJoin(schema.sites, eq(schema.accounts.siteId, schema.sites.id))
.where(
and(
gte(schema.proxyLogs.createdAt, lastMinuteDate),
eq(schema.sites.status, "active"),
),
)
.get(),
db
.select({
todaySpend: sql<number>`coalesce(sum(coalesce(${schema.siteDayUsage.totalSiteSpend}, 0)), 0)`,
})
.from(schema.siteDayUsage)
.innerJoin(schema.sites, eq(schema.siteDayUsage.siteId, schema.sites.id))
.where(
and(
eq(schema.siteDayUsage.localDay, today),
eq(schema.sites.status, "active"),
),
)
.get(),
]);
const todayCheckins = todayCheckinRows.map((row) => row.checkin_logs);
const checkinFailed = todayCheckins.filter(
(checkin) => checkin.status === "failed",
).length;
const checkinSuccess = todayCheckins.length - checkinFailed;
const rewardByAccount: Record<number, number> = {};
const successCountByAccount: Record<number, number> = {};
const parsedRewardCountByAccount: Record<number, number> = {};
for (const row of todayCheckinRows) {
const checkin = row.checkin_logs;
if (checkin.status !== "success") continue;
const accountId = row.accounts.id;
successCountByAccount[accountId] =
(successCountByAccount[accountId] || 0) + 1;
const rewardValue =
parseCheckinRewardAmount(checkin.reward) ||
parseCheckinRewardAmount(checkin.message);
if (rewardValue <= 0) continue;
rewardByAccount[accountId] =
(rewardByAccount[accountId] || 0) + rewardValue;
parsedRewardCountByAccount[accountId] =
(parsedRewardCountByAccount[accountId] || 0) + 1;
}
const proxySuccess = Number(proxy24hRow?.success || 0);
const proxyFailed = Number(proxy24hRow?.failed || 0);
const proxyTotal = Number(proxy24hRow?.total || 0);
const totalTokens = Number(proxy24hRow?.totalTokens || 0);
const requestsPerMinute = Number(proxyPerformanceRow?.total || 0);
const tokensPerMinute = Number(proxyPerformanceRow?.totalTokens || 0);
const totalUsed = Number(totalUsedRow?.totalUsed || 0);
const todaySpend = Number(todaySpendRow?.todaySpend || 0);
const todayReward = accounts.reduce(
(sum, account) =>
sum +
estimateRewardWithTodayIncomeFallback({
day: today,
successCount: successCountByAccount[account.id] || 0,
parsedRewardCount: parsedRewardCountByAccount[account.id] || 0,
rewardSum: rewardByAccount[account.id] || 0,
extraConfig: account.extraConfig,
}),
0,
);
return {
totalBalance,
totalUsed: toRoundedMicroNumber(totalUsed),
todaySpend: toRoundedMicroNumber(todaySpend),
todayReward: toRoundedMicroNumber(todayReward),
activeAccounts: activeCount,
totalAccounts: accounts.length,
todayCheckin: {
success: checkinSuccess,
failed: checkinFailed,
total: todayCheckins.length,
},
proxy24h: {
success: proxySuccess,
failed: proxyFailed,
total: proxyTotal,
totalTokens,
},
performance: {
windowSeconds: 60,
requestsPerMinute,
tokensPerMinute,
},
};
}
async function loadDashboardInsightsPayload(): Promise<DashboardInsightsPayload> {
const siteAvailabilityNow = getLocalHourAnchor();
const siteAvailabilitySinceUtc = getLocalHourRangeStartUtc(
SITE_AVAILABILITY_BUCKET_COUNT,
siteAvailabilityNow,
);
const modelAnalysisSinceDay = getLocalRangeStartDayKey(7);
await runUsageAggregationProjectionPass();
const [activeSites, siteAvailabilityRows, modelDayRows] =
await Promise.all([
db
.select({
id: schema.sites.id,
name: schema.sites.name,
url: schema.sites.url,
platform: schema.sites.platform,
sortOrder: schema.sites.sortOrder,
isPinned: schema.sites.isPinned,
})
.from(schema.sites)
.where(eq(schema.sites.status, "active"))
.all(),
db
.select()
.from(schema.siteHourUsage)
.where(gte(schema.siteHourUsage.bucketStartUtc, siteAvailabilitySinceUtc))
.all(),
db
.select()
.from(schema.modelDayUsage)
.where(gte(schema.modelDayUsage.localDay, modelAnalysisSinceDay))
.all(),
]);
const sortedSites = activeSites.sort(
(left: SiteAvailabilitySiteRow, right: SiteAvailabilitySiteRow) => {
const leftPinned = left.isPinned ? 1 : 0;
const rightPinned = right.isPinned ? 1 : 0;
if (leftPinned !== rightPinned) return rightPinned - leftPinned;
const leftOrder = Number(left.sortOrder || 0);
const rightOrder = Number(right.sortOrder || 0);
if (leftOrder !== rightOrder) return leftOrder - rightOrder;
return String(left.name || "").localeCompare(String(right.name || ""));
},
);
const activeSiteIdSet = new Set(sortedSites.map((site) => site.id));
return {
siteAvailability: buildSiteAvailabilitySummariesFromHourlyAggregates(
sortedSites,
siteAvailabilityRows
.filter((row) => activeSiteIdSet.has(row.siteId))
.map((row) => ({
siteId: row.siteId,
hourStartUtc: row.bucketStartUtc,
totalRequests: row.totalCalls,
successCount: row.successCalls,
failedCount: row.failedCalls,
totalLatencyMs: row.totalLatencyMs,
latencyCount: row.latencyCount,
})),
siteAvailabilityNow,
),
modelAnalysis: buildModelAnalysisFromDailyUsage(
modelDayRows
.filter((row) => activeSiteIdSet.has(row.siteId))
.map((row) => ({
localDay: row.localDay,
model: row.model,
totalCalls: row.totalCalls,
successCalls: row.successCalls,
totalTokens: row.totalTokens,
totalSpend: row.totalSpend,
totalLatencyMs: row.totalLatencyMs,
})),
{ days: 7 },
),
};
}
export async function getDashboardSummarySnapshot(options?: {
forceRefresh?: boolean;
}): Promise<SnapshotEnvelope<DashboardSummaryPayload>> {
return readSnapshotCache({
namespace: "dashboard-summary",
key: "default",
ttlMs: DASHBOARD_SUMMARY_TTL_MS,
forceRefresh: options?.forceRefresh,
persistence: dashboardSummaryPersistence,
loader: loadDashboardSummaryPayload,
});
}
export async function getDashboardInsightsSnapshot(options?: {
forceRefresh?: boolean;
}): Promise<SnapshotEnvelope<DashboardInsightsPayload>> {
return readSnapshotCache({
namespace: "dashboard-insights",
key: "default",
ttlMs: DASHBOARD_INSIGHTS_TTL_MS,
forceRefresh: options?.forceRefresh,
persistence: dashboardInsightsPersistence,
loader: loadDashboardInsightsPayload,
});
}
src/server/services/localTimeService.ts
+85
View File
@@ -1,4 +1,5 @@
const DAY_MS = 24 * 60 * 60 * 1000;
const HOUR_MS = 60 * 60 * 1000;
export type StoredUtcDateTimeInput = string | number | Date | null | undefined;
@@ -71,6 +72,64 @@ export function toLocalDayKeyFromStoredUtc(raw: StoredUtcDateTimeInput): string
return formatLocalDate(parsed);
}
export function getLocalHourAnchor(now = new Date()): Date {
return new Date(
now.getFullYear(),
now.getMonth(),
now.getDate(),
now.getHours(),
0,
0,
0,
);
}
export function toLocalHourBucketStartUtc(raw: StoredUtcDateTimeInput): string | null {
const parsed = parseStoredUtcDateTime(raw);
if (!parsed) return null;
return formatUtcSqlDateTime(getLocalHourAnchor(parsed));
}
export function toLocalHourStartUtcFromStoredUtc(raw: StoredUtcDateTimeInput): string | null {
return toLocalHourBucketStartUtc(raw);
}
export function toLocalDayStartUtcFromStoredUtc(raw: StoredUtcDateTimeInput): string | null {
const parsed = parseStoredUtcDateTime(raw);
if (!parsed) return null;
return formatUtcSqlDateTime(
new Date(
parsed.getFullYear(),
parsed.getMonth(),
parsed.getDate(),
0,
0,
0,
0,
),
);
}
export function getLocalDayKeyRangeUtc(dayKey: string): {
startUtc: string;
endUtc: string;
} | null {
const match = /^(\d{4})-(\d{2})-(\d{2})$/.exec((dayKey || '').trim());
if (!match) return null;
const year = Number(match[1]);
const monthIndex = Number(match[2]) - 1;
const day = Number(match[3]);
if (![year, monthIndex, day].every(Number.isInteger)) return null;
const startLocal = new Date(year, monthIndex, day, 0, 0, 0, 0);
if (Number.isNaN(startLocal.getTime())) return null;
const endLocal = new Date(startLocal.getTime() + DAY_MS);
return {
startUtc: formatUtcSqlDateTime(startLocal),
endUtc: formatUtcSqlDateTime(endLocal),
};
}
export function getLocalDayRangeUtc(now = new Date()): {
localDay: string;
startUtc: string;
@@ -91,3 +150,29 @@ export function getLocalRangeStartUtc(days: number, now = new Date()): string {
const start = new Date(localStart.getTime() - (normalizedDays - 1) * DAY_MS);
return formatUtcSqlDateTime(start);
}
export function getLocalRangeStartDayKey(days: number, now = new Date()): string {
const normalizedDays = Math.max(1, Math.floor(days || 1));
const localStart = new Date(now.getFullYear(), now.getMonth(), now.getDate(), 0, 0, 0, 0);
const start = new Date(localStart.getTime() - (normalizedDays - 1) * DAY_MS);
return formatLocalDate(start);
}
export function getLocalHourRangeStartUtc(hours: number, now = new Date()): string {
const normalizedHours = Math.max(1, Math.floor(hours || 1));
const localAnchor = getLocalHourAnchor(now);
const start = new Date(localAnchor.getTime() - (normalizedHours - 1) * HOUR_MS);
return formatUtcSqlDateTime(start);
}
export function getUtcHourRangeFromStoredStart(hourStartUtc: string): {
startUtc: string;
endUtc: string;
} | null {
const parsed = parseStoredUtcDateTime(hourStartUtc);
if (!parsed) return null;
return {
startUtc: formatUtcSqlDateTime(parsed),
endUtc: formatUtcSqlDateTime(new Date(parsed.getTime() + HOUR_MS)),
};
}
src/server/services/modelAnalysisService.ts
+133 -55
View File
@@ -1,4 +1,8 @@
import { formatLocalDate, parseStoredUtcDateTime, type StoredUtcDateTimeInput } from './localTimeService.js';
import {
formatLocalDate,
parseStoredUtcDateTime,
type StoredUtcDateTimeInput,
} from './localTimeService.js';
const DAY_MS = 24 * 60 * 60 * 1000;
@@ -53,6 +57,16 @@ interface BuildOptions {
maxModels?: number;
}
export interface ModelAnalysisDailyUsageRow {
localDay: string;
model: string;
totalCalls: number;
successCalls: number;
totalTokens: number;
totalSpend: number;
totalLatencyMs: number;
}
interface MutableModelStats {
model: string;
calls: number;
@@ -97,65 +111,22 @@ function resolveModelName(log: ProxyLogLike): string {
return raw.length > 0 ? raw : 'unknown';
}
function resolveSpend(log: ProxyLogLike, tokens: number): number {
export function resolveModelAnalysisSpend(
log: Pick<ProxyLogLike, 'estimatedCost'>,
tokens: number,
): number {
const explicit = toSafeNumber(log.estimatedCost);
if (explicit > 0) return explicit;
if (tokens <= 0) return 0;
// Fallback to OneAPI/NewAPI quota conversion when explicit cost is absent.
return tokens / 500000;
}
export function buildModelAnalysis(logs: ProxyLogLike[], options: BuildOptions = {}): ModelAnalysisResult {
const now = options.now ?? new Date();
const days = Math.max(1, options.days ?? 7);
const maxModels = Math.max(1, options.maxModels ?? 10);
const endDay = startOfLocalDay(now);
const startDay = new Date(endDay.getTime() - (days - 1) * DAY_MS);
const dayKeys: string[] = [];
for (let i = 0; i < days; i++) {
dayKeys.push(dayKey(new Date(startDay.getTime() + i * DAY_MS)));
}
const daySet = new Set(dayKeys);
const spendTrendMap = new Map<string, number>(dayKeys.map((key) => [key, 0]));
const modelMap = new Map<string, MutableModelStats>();
for (const log of logs) {
if (!log?.createdAt) continue;
const createdAt = parseStoredUtcDateTime(log.createdAt);
if (!createdAt) continue;
const createdAtDay = dayKey(createdAt);
if (!daySet.has(createdAtDay)) continue;
const model = resolveModelName(log);
const tokens = toPositiveInt(log.totalTokens);
const spend = Math.max(0, resolveSpend(log, tokens));
const latency = toPositiveInt(log.latencyMs);
const isSuccess = (log.status || '').toLowerCase() === 'success';
const stat = modelMap.get(model) ?? {
model,
calls: 0,
success: 0,
latencyTotal: 0,
tokens: 0,
spend: 0,
};
stat.calls += 1;
stat.success += isSuccess ? 1 : 0;
stat.latencyTotal += latency;
stat.tokens += tokens;
stat.spend += spend;
modelMap.set(model, stat);
spendTrendMap.set(createdAtDay, (spendTrendMap.get(createdAtDay) ?? 0) + spend);
}
const stats = Array.from(modelMap.values());
function finalizeModelAnalysis(
stats: MutableModelStats[],
dayKeys: string[],
spendTrendMap: Map<string, number>,
maxModels: number,
): ModelAnalysisResult {
const totalCalls = stats.reduce((sum, item) => sum + item.calls, 0);
const totalTokens = stats.reduce((sum, item) => sum + item.tokens, 0);
const totalSpend = round(stats.reduce((sum, item) => sum + item.spend, 0), 6);
@@ -199,7 +170,7 @@ export function buildModelAnalysis(logs: ProxyLogLike[], options: BuildOptions =
window: {
start: dayKeys[0],
end: dayKeys[dayKeys.length - 1],
days,
days: dayKeys.length,
},
totals: {
calls: totalCalls,
@@ -212,3 +183,110 @@ export function buildModelAnalysis(logs: ProxyLogLike[], options: BuildOptions =
callRanking,
};
}
export function buildModelAnalysis(
logs: ProxyLogLike[],
options: BuildOptions = {},
): ModelAnalysisResult {
const now = options.now ?? new Date();
const days = Math.max(1, options.days ?? 7);
const maxModels = Math.max(1, options.maxModels ?? 10);
const endDay = startOfLocalDay(now);
const startDay = new Date(endDay.getTime() - (days - 1) * DAY_MS);
const dayKeys: string[] = [];
for (let i = 0; i < days; i += 1) {
dayKeys.push(dayKey(new Date(startDay.getTime() + i * DAY_MS)));
}
const daySet = new Set(dayKeys);
const spendTrendMap = new Map<string, number>(dayKeys.map((key) => [key, 0]));
const modelMap = new Map<string, MutableModelStats>();
for (const log of logs) {
if (!log?.createdAt) continue;
const createdAt = parseStoredUtcDateTime(log.createdAt);
if (!createdAt) continue;
const createdAtDay = dayKey(createdAt);
if (!daySet.has(createdAtDay)) continue;
const model = resolveModelName(log);
const tokens = toPositiveInt(log.totalTokens);
const spend = Math.max(0, resolveModelAnalysisSpend(log, tokens));
const latency = toPositiveInt(log.latencyMs);
const isSuccess = (log.status || '').toLowerCase() === 'success';
const stat = modelMap.get(model) ?? {
model,
calls: 0,
success: 0,
latencyTotal: 0,
tokens: 0,
spend: 0,
};
stat.calls += 1;
stat.success += isSuccess ? 1 : 0;
stat.latencyTotal += latency;
stat.tokens += tokens;
stat.spend += spend;
modelMap.set(model, stat);
spendTrendMap.set(createdAtDay, (spendTrendMap.get(createdAtDay) ?? 0) + spend);
}
return finalizeModelAnalysis(Array.from(modelMap.values()), dayKeys, spendTrendMap, maxModels);
}
export function buildModelAnalysisFromDailyUsage(
rows: ModelAnalysisDailyUsageRow[],
options: BuildOptions = {},
): ModelAnalysisResult {
const now = options.now ?? new Date();
const days = Math.max(1, options.days ?? 7);
const maxModels = Math.max(1, options.maxModels ?? 10);
const endDay = startOfLocalDay(now);
const startDay = new Date(endDay.getTime() - (days - 1) * DAY_MS);
const dayKeys: string[] = [];
for (let i = 0; i < days; i += 1) {
dayKeys.push(dayKey(new Date(startDay.getTime() + i * DAY_MS)));
}
const daySet = new Set(dayKeys);
const spendTrendMap = new Map<string, number>(dayKeys.map((key) => [key, 0]));
const modelMap = new Map<string, MutableModelStats>();
for (const row of rows) {
const aggregateDay = String(row.localDay || '').trim();
if (!daySet.has(aggregateDay)) continue;
const model = String(row.model || '').trim() || 'unknown';
const calls = toPositiveInt(row.totalCalls);
const successCount = toPositiveInt(row.successCalls);
const tokens = toPositiveInt(row.totalTokens);
const spend = Math.max(0, toSafeNumber(row.totalSpend));
const latencyTotal = toPositiveInt(row.totalLatencyMs);
const stat = modelMap.get(model) ?? {
model,
calls: 0,
success: 0,
latencyTotal: 0,
tokens: 0,
spend: 0,
};
stat.calls += calls;
stat.success += successCount;
stat.latencyTotal += latencyTotal;
stat.tokens += tokens;
stat.spend += spend;
modelMap.set(model, stat);
spendTrendMap.set(aggregateDay, (spendTrendMap.get(aggregateDay) ?? 0) + spend);
}
return finalizeModelAnalysis(Array.from(modelMap.values()), dayKeys, spendTrendMap, maxModels);
}
src/server/services/siteStatsSnapshotService.ts
+149
View File
@@ -0,0 +1,149 @@
import { eq, sql } from "drizzle-orm";
import { db, schema } from "../db/index.js";
import { getLocalRangeStartDayKey } from "./localTimeService.js";
import {
readSnapshotCache,
type SnapshotEnvelope,
} from "./snapshotCacheService.js";
import {
toRoundedMicroNumber,
} from "./statsShared.js";
import { createAdminSnapshotPersistence } from "./adminSnapshotStore.js";
import {
runUsageAggregationProjectionPass,
} from "./usageAggregationService.js";
export type SiteStatsSnapshotPayload = {
distribution: Array<{
siteId: number;
siteName: string;
platform: string | null;
totalBalance: number;
totalSpend: number;
accountCount: number;
}>;
trend: Array<{
date: string;
sites: Record<string, { spend: number; calls: number }>;
}>;
sites: Array<typeof schema.sites.$inferSelect>;
};
const SITE_STATS_TTL_MS = 15_000;
async function loadSiteStatsSnapshotPayload(
days: number,
): Promise<SiteStatsSnapshotPayload> {
const sinceDay = getLocalRangeStartDayKey(days);
await runUsageAggregationProjectionPass();
const [spendRows, trendRows, sites, accountDistributionRows] =
await Promise.all([
db
.select({
siteId: schema.siteDayUsage.siteId,
totalSpend: sql<number>`coalesce(sum(${schema.siteDayUsage.totalSiteSpend}), 0)`,
})
.from(schema.siteDayUsage)
.groupBy(schema.siteDayUsage.siteId)
.all(),
db
.select()
.from(schema.siteDayUsage)
.where(sql`${schema.siteDayUsage.localDay} >= ${sinceDay}`)
.all(),
db
.select()
.from(schema.sites)
.where(eq(schema.sites.status, "active"))
.all(),
db
.select({
siteId: schema.sites.id,
siteName: schema.sites.name,
platform: schema.sites.platform,
totalBalance: sql<number>`coalesce(sum(coalesce(${schema.accounts.balance}, 0)), 0)`,
accountCount: sql<number>`count(*)`,
})
.from(schema.accounts)
.innerJoin(schema.sites, eq(schema.accounts.siteId, schema.sites.id))
.where(eq(schema.sites.status, "active"))
.groupBy(schema.sites.id, schema.sites.name, schema.sites.platform)
.all(),
]);
const spendBySiteId = new Map<number, number>();
for (const row of spendRows) {
if (row.siteId == null) continue;
spendBySiteId.set(row.siteId, Number(row.totalSpend || 0));
}
const distribution = accountDistributionRows.map((row) => ({
siteId: row.siteId,
siteName: row.siteName,
platform: row.platform,
totalBalance: toRoundedMicroNumber(Number(row.totalBalance || 0)),
totalSpend: toRoundedMicroNumber(spendBySiteId.get(row.siteId) || 0),
accountCount: Number(row.accountCount || 0),
}));
const dayMap: Record<
string,
Record<string, { spend: number; calls: number }>
> = {};
const activeSiteById = new Map<number, (typeof schema.sites.$inferSelect)>(
sites.map((site) => [site.id, site]),
);
for (const row of trendRows) {
const site = activeSiteById.get(row.siteId);
if (!site) continue;
const siteName = site.name || "unknown";
const date = row.localDay;
if (!dayMap[date]) dayMap[date] = {};
if (!dayMap[date][siteName])
dayMap[date][siteName] = { spend: 0, calls: 0 };
dayMap[date][siteName].spend += Number(row.totalSiteSpend || 0);
dayMap[date][siteName].calls += Number(row.totalCalls || 0);
}
const trend = Object.entries(dayMap)
.sort(([left], [right]) => left.localeCompare(right))
.map(([date, value]) => ({
date,
sites: Object.fromEntries(
Object.entries(value).map(([siteName, stats]) => [
siteName,
{
spend: toRoundedMicroNumber(stats.spend),
calls: stats.calls,
},
]),
),
}));
return {
distribution,
trend,
sites,
};
}
export async function getSiteStatsSnapshot(options?: {
days?: number;
forceRefresh?: boolean;
}): Promise<SnapshotEnvelope<SiteStatsSnapshotPayload>> {
const days = Math.max(1, Math.trunc(options?.days || 7));
return readSnapshotCache({
namespace: "site-stats",
key: JSON.stringify({ days }),
ttlMs: SITE_STATS_TTL_MS,
forceRefresh: options?.forceRefresh,
persistence: createAdminSnapshotPersistence<SiteStatsSnapshotPayload>({
namespace: "site-stats",
key: JSON.stringify({ days }),
}),
loader: async () => loadSiteStatsSnapshotPayload(days),
});
}
src/server/services/snapshotCacheService.test.ts
+90
View File
@@ -0,0 +1,90 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import {
clearSnapshotCache,
readSnapshotCache,
type PersistedSnapshotRecord,
} from "./snapshotCacheService.js";
describe("snapshotCacheService", () => {
let previousVitestEnv: string | undefined;
beforeEach(() => {
previousVitestEnv = process.env.VITEST;
delete process.env.VITEST;
clearSnapshotCache();
});
afterEach(() => {
if (previousVitestEnv === undefined) {
delete process.env.VITEST;
} else {
process.env.VITEST = previousVitestEnv;
}
});
it("degrades persistence read and write failures without breaking the read path", async () => {
const consoleWarn = vi.spyOn(console, "warn").mockImplementation(() => {});
const result = await readSnapshotCache({
namespace: "test",
key: "persistence-failure",
ttlMs: 1000,
loader: async () => ({ ok: true }),
persistence: {
read: async () => {
throw new Error("read failed");
},
write: async () => {
throw new Error("write failed");
},
},
});
expect(result.payload).toEqual({ ok: true });
expect(result.cacheStatus).toBe("miss");
expect(consoleWarn).toHaveBeenCalled();
consoleWarn.mockRestore();
});
it("reuses an in-flight loader after async hydration misses", async () => {
let loaderCalls = 0;
const persistenceRead = vi.fn(async (): Promise<PersistedSnapshotRecord<number> | null> => {
await new Promise((resolve) => setTimeout(resolve, 10));
return null;
});
const [left, right] = await Promise.all([
readSnapshotCache({
namespace: "test",
key: "coalesce",
ttlMs: 1000,
loader: async () => {
loaderCalls += 1;
await new Promise((resolve) => setTimeout(resolve, 20));
return 42;
},
persistence: {
read: persistenceRead,
write: async () => {},
},
}),
readSnapshotCache({
namespace: "test",
key: "coalesce",
ttlMs: 1000,
loader: async () => {
loaderCalls += 1;
await new Promise((resolve) => setTimeout(resolve, 20));
return 42;
},
persistence: {
read: persistenceRead,
write: async () => {},
},
}),
]);
expect(left.payload).toBe(42);
expect(right.payload).toBe(42);
expect(loaderCalls).toBe(1);
});
});
src/server/services/snapshotCacheService.ts
+278
View File
@@ -0,0 +1,278 @@
export type SnapshotCacheStatus =
| "disabled"
| "miss"
| "hit"
| "stale"
| "refresh";
export type SnapshotEnvelope<T> = {
payload: T;
generatedAt: string;
cacheStatus: SnapshotCacheStatus;
};
export type PersistedSnapshotRecord<T> = {
payload: T;
generatedAt: string;
expiresAt: string;
staleUntil: string;
};
export type SnapshotPersistenceAdapter<T> = {
read: () => Promise<PersistedSnapshotRecord<T> | null>;
write: (record: PersistedSnapshotRecord<T>) => Promise<void>;
};
type SnapshotCacheEntry<T> = {
payload?: T;
generatedAtMs: number;
expiresAtMs: number;
staleUntilMs: number;
inFlight?: Promise<SnapshotEnvelope<T>>;
};
type ReadSnapshotOptions<T> = {
namespace: string;
key: string;
ttlMs: number;
staleMs?: number;
forceRefresh?: boolean;
loader: () => Promise<T>;
persistence?: SnapshotPersistenceAdapter<T>;
};
const SNAPSHOT_CACHE_MAX_ENTRIES = 64;
const snapshotCache = new Map<string, SnapshotCacheEntry<unknown>>();
function getSnapshotCacheEntry<T>(cacheKey: string) {
const cached = snapshotCache.get(cacheKey) as SnapshotCacheEntry<T> | undefined;
if (!cached) return undefined;
snapshotCache.delete(cacheKey);
snapshotCache.set(cacheKey, cached as SnapshotCacheEntry<unknown>);
return cached;
}
function setSnapshotCacheEntry<T>(
cacheKey: string,
entry: SnapshotCacheEntry<T>,
) {
snapshotCache.delete(cacheKey);
snapshotCache.set(cacheKey, entry as SnapshotCacheEntry<unknown>);
while (snapshotCache.size > SNAPSHOT_CACHE_MAX_ENTRIES) {
const oldestKey = snapshotCache.keys().next().value;
if (!oldestKey) break;
snapshotCache.delete(oldestKey);
}
}
function shouldBypassSnapshotCache() {
return !!process.env.VITEST;
}
function buildCacheKey(namespace: string, key: string) {
return `${namespace}:${key}`;
}
async function loadAndStoreSnapshot<T>(
cacheKey: string,
loader: () => Promise<T>,
ttlMs: number,
staleMs: number,
persistence?: SnapshotPersistenceAdapter<T>,
) {
const payload = await loader();
const nowMs = Date.now();
const persistedRecord: PersistedSnapshotRecord<T> = {
payload,
generatedAt: new Date(nowMs).toISOString(),
expiresAt: new Date(nowMs + Math.max(1, ttlMs)).toISOString(),
staleUntil: new Date(
nowMs + Math.max(Math.max(1, ttlMs), staleMs),
).toISOString(),
};
const envelope: SnapshotEnvelope<T> = {
payload,
generatedAt: persistedRecord.generatedAt,
cacheStatus: "miss",
};
setSnapshotCacheEntry(cacheKey, {
payload,
generatedAtMs: nowMs,
expiresAtMs: nowMs + Math.max(1, ttlMs),
staleUntilMs: nowMs + Math.max(Math.max(1, ttlMs), staleMs),
});
if (persistence) {
try {
await persistence.write(persistedRecord);
} catch (error) {
console.warn(
`[snapshotCache] persistence write failed for ${cacheKey}:`,
error,
);
}
}
return envelope;
}
function parseSnapshotTimestamp(raw: string): number {
const parsed = Date.parse(raw);
return Number.isFinite(parsed) ? parsed : 0;
}
function buildCacheEntryFromPersistedSnapshot<T>(
record: PersistedSnapshotRecord<T>,
): SnapshotCacheEntry<T> {
return {
payload: record.payload,
generatedAtMs: parseSnapshotTimestamp(record.generatedAt),
expiresAtMs: parseSnapshotTimestamp(record.expiresAt),
staleUntilMs: parseSnapshotTimestamp(record.staleUntil),
};
}
export async function readSnapshotCache<T>(
options: ReadSnapshotOptions<T>,
): Promise<SnapshotEnvelope<T>> {
const staleMs = Math.max(options.ttlMs, options.staleMs ?? options.ttlMs * 6);
if (shouldBypassSnapshotCache()) {
const payload = await options.loader();
return {
payload,
generatedAt: new Date().toISOString(),
cacheStatus: "disabled",
};
}
const cacheKey = buildCacheKey(options.namespace, options.key);
const nowMs = Date.now();
let cached = getSnapshotCacheEntry<T>(cacheKey);
if (!cached && !options.forceRefresh && options.persistence) {
try {
const persisted = await options.persistence.read();
const shared = getSnapshotCacheEntry<T>(cacheKey);
if (shared) {
cached = shared;
} else if (persisted) {
cached = buildCacheEntryFromPersistedSnapshot(persisted);
setSnapshotCacheEntry(cacheKey, cached);
}
} catch (error) {
console.warn(
`[snapshotCache] persistence read failed for ${cacheKey}; falling back to loader:`,
error,
);
}
}
cached = getSnapshotCacheEntry<T>(cacheKey) ?? cached;
if (
!options.forceRefresh &&
cached?.payload !== undefined &&
cached.expiresAtMs > nowMs
) {
return {
payload: cached.payload,
generatedAt: new Date(cached.generatedAtMs).toISOString(),
cacheStatus: "hit",
};
}
if (
!options.forceRefresh &&
cached?.payload !== undefined &&
cached.staleUntilMs > nowMs
) {
if (!cached.inFlight) {
cached.inFlight = loadAndStoreSnapshot(
cacheKey,
options.loader,
options.ttlMs,
staleMs,
options.persistence,
).finally(() => {
const next = snapshotCache.get(cacheKey) as SnapshotCacheEntry<T> | undefined;
if (next) delete next.inFlight;
});
void cached.inFlight.catch((error) => {
console.error(
`[snapshotCache] background refresh failed for ${cacheKey}:`,
error,
);
});
setSnapshotCacheEntry(cacheKey, cached);
}
return {
payload: cached.payload,
generatedAt: new Date(cached.generatedAtMs).toISOString(),
cacheStatus: "stale",
};
}
const shared = getSnapshotCacheEntry<T>(cacheKey);
if (shared?.inFlight) {
const result = await shared.inFlight;
return {
...result,
cacheStatus:
options.forceRefresh || shared.payload !== undefined
? "refresh"
: result.cacheStatus,
};
}
if (cached?.inFlight) {
const result = await cached.inFlight;
return {
...result,
cacheStatus:
options.forceRefresh || cached.payload !== undefined
? "refresh"
: result.cacheStatus,
};
}
const inFlight = loadAndStoreSnapshot(
cacheKey,
options.loader,
options.ttlMs,
staleMs,
options.persistence,
).finally(() => {
const next = snapshotCache.get(cacheKey) as
| SnapshotCacheEntry<T>
| undefined;
if (next) delete next.inFlight;
});
setSnapshotCacheEntry(cacheKey, {
payload: cached ? cached.payload : undefined,
generatedAtMs: cached?.generatedAtMs ?? 0,
expiresAtMs: cached?.expiresAtMs ?? 0,
staleUntilMs: cached?.staleUntilMs ?? 0,
inFlight,
});
const result = await inFlight;
return {
...result,
cacheStatus: options.forceRefresh
? "refresh"
: cached?.payload !== undefined
? "refresh"
: "miss",
};
}
export function clearSnapshotCache(namespace?: string) {
if (!namespace) {
snapshotCache.clear();
return;
}
for (const key of snapshotCache.keys()) {
if (key.startsWith(`${namespace}:`)) snapshotCache.delete(key);
}
}
src/server/services/statsShared.ts
+275
View File
@@ -0,0 +1,275 @@
import { sql } from "drizzle-orm";
import { schema } from "../db/index.js";
import {
formatLocalDateTime,
getLocalHourAnchor,
parseStoredUtcDateTime,
type StoredUtcDateTimeInput,
} from "./localTimeService.js";
export function proxyCostSqlExpression() {
return sql<number>`
coalesce(
${schema.proxyLogs.estimatedCost},
case
when lower(coalesce(${schema.sites.platform}, 'new-api')) = 'veloera'
then coalesce(${schema.proxyLogs.totalTokens}, 0) / 1000000.0
else coalesce(${schema.proxyLogs.totalTokens}, 0) / 500000.0
end
)
`;
}
export function toRoundedMicroNumber(value: number | null | undefined): number {
return Math.round(Number(value || 0) * 1_000_000) / 1_000_000;
}
export function buildProxyLogModelAnalysisSelectFields() {
return {
createdAt: schema.proxyLogs.createdAt,
modelActual: schema.proxyLogs.modelActual,
modelRequested: schema.proxyLogs.modelRequested,
status: schema.proxyLogs.status,
latencyMs: schema.proxyLogs.latencyMs,
totalTokens: schema.proxyLogs.totalTokens,
estimatedCost: schema.proxyLogs.estimatedCost,
};
}
export function buildProxyLogSiteTrendSelectFields() {
return {
createdAt: schema.proxyLogs.createdAt,
estimatedCost: schema.proxyLogs.estimatedCost,
totalTokens: schema.proxyLogs.totalTokens,
};
}
const SITE_AVAILABILITY_BUCKET_COUNT = 24;
const SITE_AVAILABILITY_BUCKET_MS = 60 * 60 * 1000;
export type SiteAvailabilitySiteRow = {
id: number;
name: string;
url: string | null;
platform: string | null;
sortOrder: number | null;
isPinned: boolean | null;
};
export type SiteAvailabilityLogRow = {
siteId: number | null;
createdAt: StoredUtcDateTimeInput;
status: string | null;
latencyMs: number | null;
};
export type SiteAvailabilityHourAggregateRow = {
siteId: number | null;
hourStartUtc: StoredUtcDateTimeInput;
totalRequests: number | null;
successCount: number | null;
failedCount: number | null;
totalLatencyMs: number | null;
latencyCount: number | null;
};
type SiteAvailabilityBucketAccumulator = {
startUtc: string;
label: string;
totalRequests: number;
successCount: number;
failedCount: number;
latencyTotalMs: number;
latencyCount: number;
};
function roundPercent(value: number | null): number | null {
if (value == null || !Number.isFinite(value)) return null;
return Math.round(value * 10) / 10;
}
function createSiteAvailabilityBucketTemplate(startMs: number) {
return Array.from({ length: SITE_AVAILABILITY_BUCKET_COUNT }, (_, index) => {
const bucketStart = new Date(startMs + index * SITE_AVAILABILITY_BUCKET_MS);
return {
startUtc: bucketStart.toISOString(),
label: formatLocalDateTime(bucketStart),
totalRequests: 0,
successCount: 0,
failedCount: 0,
latencyTotalMs: 0,
latencyCount: 0,
} satisfies SiteAvailabilityBucketAccumulator;
});
}
function createSiteAvailabilityAccumulatorMap(
sites: SiteAvailabilitySiteRow[],
startMs: number,
) {
const siteMap = new Map<
number,
{
site: SiteAvailabilitySiteRow;
totalRequests: number;
successCount: number;
failedCount: number;
latencyTotalMs: number;
latencyCount: number;
buckets: SiteAvailabilityBucketAccumulator[];
}
>();
for (const site of sites) {
siteMap.set(site.id, {
site,
totalRequests: 0,
successCount: 0,
failedCount: 0,
latencyTotalMs: 0,
latencyCount: 0,
buckets: createSiteAvailabilityBucketTemplate(startMs),
});
}
return siteMap;
}
function finalizeSiteAvailabilitySummaries(
sites: SiteAvailabilitySiteRow[],
siteMap: ReturnType<typeof createSiteAvailabilityAccumulatorMap>,
) {
return sites.map((site) => {
const aggregate = siteMap.get(site.id)!;
return {
siteId: site.id,
siteName: site.name,
siteUrl: site.url,
platform: site.platform,
totalRequests: aggregate.totalRequests,
successCount: aggregate.successCount,
failedCount: aggregate.failedCount,
availabilityPercent:
aggregate.totalRequests > 0
? roundPercent((aggregate.successCount / aggregate.totalRequests) * 100)
: null,
averageLatencyMs:
aggregate.latencyCount > 0
? Math.round(aggregate.latencyTotalMs / aggregate.latencyCount)
: null,
buckets: aggregate.buckets.map((bucket) => ({
startUtc: bucket.startUtc,
label: bucket.label,
totalRequests: bucket.totalRequests,
successCount: bucket.successCount,
failedCount: bucket.failedCount,
availabilityPercent:
bucket.totalRequests > 0
? roundPercent((bucket.successCount / bucket.totalRequests) * 100)
: null,
averageLatencyMs:
bucket.latencyCount > 0
? Math.round(bucket.latencyTotalMs / bucket.latencyCount)
: null,
})),
};
});
}
export function buildSiteAvailabilitySummaries(
sites: SiteAvailabilitySiteRow[],
logs: SiteAvailabilityLogRow[],
now = new Date(),
) {
const endLocal = getLocalHourAnchor(now);
const startLocal = new Date(
endLocal.getTime() -
(SITE_AVAILABILITY_BUCKET_COUNT - 1) * SITE_AVAILABILITY_BUCKET_MS,
);
const startMs = startLocal.getTime();
const rangeMs = SITE_AVAILABILITY_BUCKET_COUNT * SITE_AVAILABILITY_BUCKET_MS;
const siteMap = createSiteAvailabilityAccumulatorMap(sites, startMs);
for (const log of logs) {
if (log.siteId == null) continue;
const target = siteMap.get(log.siteId);
if (!target) continue;
const parsed = parseStoredUtcDateTime(log.createdAt);
if (!parsed) continue;
const diffMs = parsed.getTime() - startMs;
if (diffMs < 0 || diffMs >= rangeMs) continue;
const bucketIndex = Math.floor(diffMs / SITE_AVAILABILITY_BUCKET_MS);
const bucket = target.buckets[bucketIndex];
const isSuccess = (log.status || "").trim().toLowerCase() === "success";
target.totalRequests += 1;
bucket.totalRequests += 1;
if (isSuccess) {
target.successCount += 1;
bucket.successCount += 1;
} else {
target.failedCount += 1;
bucket.failedCount += 1;
}
const latencyMs = Number(log.latencyMs);
if (Number.isFinite(latencyMs) && latencyMs >= 0) {
target.latencyTotalMs += latencyMs;
target.latencyCount += 1;
bucket.latencyTotalMs += latencyMs;
bucket.latencyCount += 1;
}
}
return finalizeSiteAvailabilitySummaries(sites, siteMap);
}
export function buildSiteAvailabilitySummariesFromHourlyAggregates(
sites: SiteAvailabilitySiteRow[],
rows: SiteAvailabilityHourAggregateRow[],
now = new Date(),
) {
const endLocal = getLocalHourAnchor(now);
const startLocal = new Date(
endLocal.getTime() -
(SITE_AVAILABILITY_BUCKET_COUNT - 1) * SITE_AVAILABILITY_BUCKET_MS,
);
const startMs = startLocal.getTime();
const rangeMs = SITE_AVAILABILITY_BUCKET_COUNT * SITE_AVAILABILITY_BUCKET_MS;
const siteMap = createSiteAvailabilityAccumulatorMap(sites, startMs);
for (const row of rows) {
if (row.siteId == null) continue;
const target = siteMap.get(row.siteId);
if (!target) continue;
const parsed = parseStoredUtcDateTime(row.hourStartUtc);
if (!parsed) continue;
const diffMs = parsed.getTime() - startMs;
if (diffMs < 0 || diffMs >= rangeMs) continue;
const bucketIndex = Math.floor(diffMs / SITE_AVAILABILITY_BUCKET_MS);
const bucket = target.buckets[bucketIndex];
const totalRequests = Math.max(0, Number(row.totalRequests || 0));
const successCount = Math.max(0, Number(row.successCount || 0));
const failedCount = Math.max(0, Number(row.failedCount || 0));
const totalLatencyMs = Math.max(0, Number(row.totalLatencyMs || 0));
const latencyCount = Math.max(0, Number(row.latencyCount || 0));
target.totalRequests += totalRequests;
target.successCount += successCount;
target.failedCount += failedCount;
target.latencyTotalMs += totalLatencyMs;
target.latencyCount += latencyCount;
bucket.totalRequests += totalRequests;
bucket.successCount += successCount;
bucket.failedCount += failedCount;
bucket.latencyTotalMs += totalLatencyMs;
bucket.latencyCount += latencyCount;
}
return finalizeSiteAvailabilitySummaries(sites, siteMap);
}
src/server/services/usageAggregationService.test.ts
+240
View File
@@ -0,0 +1,240 @@
import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest";
import { eq } from "drizzle-orm";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { formatUtcSqlDateTime } from "./localTimeService.js";
type DbModule = typeof import("../db/index.js");
type ProjectorModule = typeof import("./usageAggregationService.js");
describe("usageAggregationService", () => {
let db: DbModule["db"];
let schema: DbModule["schema"];
let runUsageAggregationProjectionPass: ProjectorModule["runUsageAggregationProjectionPass"];
let requestUsageAggregatesRecompute: ProjectorModule["requestUsageAggregatesRecompute"];
let dataDir = "";
let previousDataDir: string | undefined;
beforeAll(async () => {
previousDataDir = process.env.DATA_DIR;
dataDir = mkdtempSync(join(tmpdir(), "metapi-usage-projector-"));
process.env.DATA_DIR = dataDir;
await import("../db/migrate.js");
const dbModule = await import("../db/index.js");
const projectorModule = await import("./usageAggregationService.js");
db = dbModule.db;
schema = dbModule.schema;
runUsageAggregationProjectionPass = projectorModule.runUsageAggregationProjectionPass;
requestUsageAggregatesRecompute = projectorModule.requestUsageAggregatesRecompute;
});
beforeEach(async () => {
await db.delete(schema.analyticsProjectionCheckpoints).run();
await db.delete(schema.modelDayUsage).run();
await db.delete(schema.siteHourUsage).run();
await db.delete(schema.siteDayUsage).run();
await db.delete(schema.proxyLogs).run();
await db.delete(schema.accounts).run();
await db.delete(schema.sites).run();
});
afterAll(() => {
if (previousDataDir === undefined) {
delete process.env.DATA_DIR;
} else {
process.env.DATA_DIR = previousDataDir;
}
rmSync(dataDir, { recursive: true, force: true });
});
it("projects proxy logs into day/hour/model aggregates and supports recompute requests", async () => {
const site = await db
.insert(schema.sites)
.values({
name: "agg-site",
url: "https://agg.example.com",
platform: "new-api",
status: "active",
})
.returning()
.get();
const account = await db
.insert(schema.accounts)
.values({
siteId: site.id,
username: "agg-user",
accessToken: "agg-token",
status: "active",
})
.returning()
.get();
await db.insert(schema.proxyLogs).values([
{
accountId: account.id,
status: "success",
modelRequested: "gpt-5",
modelActual: "gpt-5",
totalTokens: 100,
estimatedCost: 0.2,
latencyMs: 120,
createdAt: formatUtcSqlDateTime(new Date("2026-04-08T02:10:00.000Z")),
},
{
accountId: account.id,
status: "failed",
modelRequested: "gpt-5-mini",
modelActual: "gpt-5-mini",
totalTokens: 50,
estimatedCost: 0.1,
latencyMs: 80,
createdAt: formatUtcSqlDateTime(new Date("2026-04-08T02:45:00.000Z")),
},
]).run();
const firstPass = await runUsageAggregationProjectionPass();
expect(firstPass.processedLogs).toBe(2);
const dayRows = await db.select().from(schema.siteDayUsage).all();
expect(dayRows).toHaveLength(1);
expect(dayRows[0]).toEqual(
expect.objectContaining({
siteId: site.id,
totalCalls: 2,
successCalls: 1,
failedCalls: 1,
totalTokens: 150,
}),
);
expect(dayRows[0].totalSummarySpend).toBeCloseTo(0.3, 6);
expect(dayRows[0].totalSiteSpend).toBeCloseTo(0.3, 6);
const hourRows = await db.select().from(schema.siteHourUsage).all();
expect(hourRows).toHaveLength(1);
expect(hourRows[0]).toEqual(
expect.objectContaining({
siteId: site.id,
totalCalls: 2,
successCalls: 1,
failedCalls: 1,
}),
);
const modelRows = await db.select().from(schema.modelDayUsage).all();
expect(modelRows).toHaveLength(2);
await db.insert(schema.proxyLogs).values({
accountId: account.id,
status: "success",
modelRequested: "gpt-5",
modelActual: "gpt-5",
totalTokens: 20,
estimatedCost: 0.04,
latencyMs: 60,
createdAt: formatUtcSqlDateTime(new Date("2026-04-08T02:50:00.000Z")),
}).run();
const secondPass = await runUsageAggregationProjectionPass();
expect(secondPass.processedLogs).toBe(1);
const updatedDayRows = await db.select().from(schema.siteDayUsage).all();
expect(updatedDayRows[0]).toEqual(
expect.objectContaining({
siteId: site.id,
totalCalls: 3,
successCalls: 2,
failedCalls: 1,
totalTokens: 170,
}),
);
expect(updatedDayRows[0].totalSummarySpend).toBeCloseTo(0.34, 6);
expect(updatedDayRows[0].totalSiteSpend).toBeCloseTo(0.34, 6);
await requestUsageAggregatesRecompute(1);
const recomputePass = await runUsageAggregationProjectionPass();
expect(recomputePass.recomputed).toBe(true);
const recomputedDayRows = await db.select().from(schema.siteDayUsage).all();
expect(recomputedDayRows[0]).toEqual(
expect.objectContaining({
siteId: site.id,
totalCalls: 3,
successCalls: 2,
failedCalls: 1,
totalTokens: 170,
}),
);
expect(recomputedDayRows[0].totalSummarySpend).toBeCloseTo(0.34, 6);
expect(recomputedDayRows[0].totalSiteSpend).toBeCloseTo(0.34, 6);
});
it("skips projection while another process lease is active and clears lease after success", async () => {
const site = await db
.insert(schema.sites)
.values({
name: "leased-site",
url: "https://leased.example.com",
platform: "new-api",
status: "active",
})
.returning()
.get();
const account = await db
.insert(schema.accounts)
.values({
siteId: site.id,
username: "leased-user",
accessToken: "leased-token",
status: "active",
})
.returning()
.get();
await db.insert(schema.proxyLogs).values({
accountId: account.id,
status: "success",
modelRequested: "gpt-5",
modelActual: "gpt-5",
totalTokens: 10,
estimatedCost: 0.02,
latencyMs: 50,
createdAt: formatUtcSqlDateTime(new Date("2026-04-08T03:00:00.000Z")),
}).run();
await db.insert(schema.analyticsProjectionCheckpoints).values({
projectorKey: "usage-aggregates-v1",
timeZone: "Local",
lastProxyLogId: 0,
leaseOwner: "other-process",
leaseToken: "other-token",
leaseExpiresAt: new Date(Date.now() + 60_000).toISOString(),
}).run();
const blockedPass = await runUsageAggregationProjectionPass();
expect(blockedPass.processedLogs).toBe(0);
expect(await db.select().from(schema.siteDayUsage).all()).toHaveLength(0);
await db
.update(schema.analyticsProjectionCheckpoints)
.set({
leaseExpiresAt: new Date(Date.now() - 60_000).toISOString(),
})
.where(eq(schema.analyticsProjectionCheckpoints.projectorKey, "usage-aggregates-v1"))
.run();
const successfulPass = await runUsageAggregationProjectionPass();
expect(successfulPass.processedLogs).toBe(1);
const checkpoint = await db
.select()
.from(schema.analyticsProjectionCheckpoints)
.where(eq(schema.analyticsProjectionCheckpoints.projectorKey, "usage-aggregates-v1"))
.get();
expect(checkpoint?.leaseOwner).toBeNull();
expect(checkpoint?.leaseToken).toBeNull();
expect(checkpoint?.leaseExpiresAt).toBeNull();
expect(checkpoint?.lastError).toBeNull();
});
});
src/server/services/usageAggregationService.ts
+795
View File
@@ -0,0 +1,795 @@
import { and, asc, eq, gt, gte, isNull, lte, or, sql } from 'drizzle-orm';
import { randomUUID } from 'node:crypto';
import { hostname } from 'node:os';
import { db, schema } from '../db/index.js';
import { fallbackTokenCost } from './modelPricingService.js';
import {
getLocalRangeStartDayKey,
getResolvedTimeZone,
toLocalDayKeyFromStoredUtc,
toLocalDayStartUtcFromStoredUtc,
toLocalHourStartUtcFromStoredUtc,
type StoredUtcDateTimeInput,
} from './localTimeService.js';
import { clearSnapshotCache } from './snapshotCacheService.js';
const USAGE_PROJECTOR_KEY = 'usage-aggregates-v1';
const PROJECTION_BATCH_SIZE = 1_000;
const PROJECTION_MAX_BATCHES_PER_PASS = 120;
const PROJECTION_INTERVAL_MS = 5_000;
const PROJECTION_LEASE_MS = 10 * 60_000;
type ProjectionCheckpointRow = typeof schema.analyticsProjectionCheckpoints.$inferSelect;
type ProjectionLease = {
owner: string;
token: string;
expiresAt: string;
};
type ProjectionPassOptions = {
maxBatches?: number;
};
type ProxyLogProjectionRow = {
id: number;
createdAt: StoredUtcDateTimeInput;
status: string | null;
latencyMs: number | null;
totalTokens: number | null;
estimatedCost: number | null;
modelActual: string | null;
modelRequested: string | null;
siteId: number | null;
sitePlatform: string | null;
};
type SiteDayUsageDeltaRow = {
localDay: string;
siteId: number;
totalCalls: number;
successCalls: number;
failedCalls: number;
totalTokens: number;
totalSummarySpend: number;
totalSiteSpend: number;
totalLatencyMs: number;
latencyCount: number;
};
type SiteHourUsageDeltaRow = {
bucketStartUtc: string;
siteId: number;
totalCalls: number;
successCalls: number;
failedCalls: number;
totalTokens: number;
totalSummarySpend: number;
totalSiteSpend: number;
totalLatencyMs: number;
latencyCount: number;
};
type ModelDayUsageDeltaRow = {
localDay: string;
siteId: number;
model: string;
totalCalls: number;
successCalls: number;
failedCalls: number;
totalTokens: number;
totalSpend: number;
totalLatencyMs: number;
latencyCount: number;
};
type ProjectionBatchDelta = {
siteDayRows: SiteDayUsageDeltaRow[];
siteHourRows: SiteHourUsageDeltaRow[];
modelDayRows: ModelDayUsageDeltaRow[];
};
export type ProjectionPassResult = {
processedLogs: number;
watermarkId: number;
recomputed: boolean;
};
export type SiteHourUsageAggregateRow = {
siteId: number;
hourStartUtc: string;
totalRequests: number;
successCount: number;
failedCount: number;
totalTokens: number;
totalSummarySpend: number;
totalSiteSpend: number;
totalLatencyMs: number;
latencyCount: number;
};
export type ModelDayUsageAggregateRow = {
siteId: number;
day: string;
model: string;
totalCalls: number;
successCount: number;
failedCount: number;
totalTokens: number;
totalSpend: number;
totalLatencyMs: number;
latencyCount: number;
};
let projectionTimer: ReturnType<typeof setInterval> | null = null;
let projectionInFlight: Promise<ProjectionPassResult> | null = null;
function emptyCheckpoint(): ProjectionCheckpointRow {
return {
projectorKey: USAGE_PROJECTOR_KEY,
timeZone: getResolvedTimeZone(),
lastProxyLogId: 0,
watermarkCreatedAt: null,
recomputeFromId: null,
recomputeRequestedAt: null,
recomputeReason: null,
recomputeStartedAt: null,
recomputeCompletedAt: null,
leaseOwner: null,
leaseToken: null,
leaseExpiresAt: null,
lastProjectedAt: null,
lastSuccessfulAt: null,
lastError: null,
createdAt: null,
updatedAt: null,
};
}
function normalizeNonNegativeInt(value: unknown): number {
const numeric = Number(value || 0);
if (!Number.isFinite(numeric) || numeric <= 0) return 0;
return Math.round(numeric);
}
function normalizeNonNegativeFloat(value: unknown): number {
const numeric = Number(value || 0);
if (!Number.isFinite(numeric) || numeric <= 0) return 0;
return numeric;
}
function resolveSummarySpend(params: {
estimatedCost: number | null;
totalTokens: number | null;
platform: string | null;
}) {
const explicit = normalizeNonNegativeFloat(params.estimatedCost);
if (explicit > 0) return explicit;
const tokens = normalizeNonNegativeInt(params.totalTokens);
if (tokens <= 0) return 0;
return String(params.platform || '').trim().toLowerCase() === 'veloera'
? tokens / 1_000_000
: tokens / 500_000;
}
function resolveSiteSpend(params: {
estimatedCost: number | null;
totalTokens: number | null;
platform: string | null;
}) {
const explicit = normalizeNonNegativeFloat(params.estimatedCost);
if (explicit > 0) return explicit;
const tokens = normalizeNonNegativeInt(params.totalTokens);
if (tokens <= 0) return 0;
return fallbackTokenCost(tokens, params.platform || 'new-api');
}
function resolveModelSpend(params: {
estimatedCost: number | null;
totalTokens: number | null;
}) {
const explicit = normalizeNonNegativeFloat(params.estimatedCost);
if (explicit > 0) return explicit;
const tokens = normalizeNonNegativeInt(params.totalTokens);
if (tokens <= 0) return 0;
return tokens / 500_000;
}
function resolveModelName(row: ProxyLogProjectionRow): string {
return String(row.modelActual || row.modelRequested || 'unknown').trim() || 'unknown';
}
function clearAnalyticsSnapshots() {
clearSnapshotCache('site-stats');
clearSnapshotCache('dashboard-summary');
clearSnapshotCache('dashboard-insights');
}
function buildProjectionLeaseOwner() {
const host = String(hostname() || process.env.HOSTNAME || 'local').trim() || 'local';
return `${host}:${process.pid}`;
}
function buildProjectionLeaseExpiry(nowMs = Date.now()) {
return new Date(nowMs + PROJECTION_LEASE_MS).toISOString();
}
function normalizeProjectionError(error: unknown) {
if (error instanceof Error) return error.message;
return String(error || 'unknown projection error');
}
async function readProjectionCheckpoint(): Promise<ProjectionCheckpointRow> {
const row = await db
.select()
.from(schema.analyticsProjectionCheckpoints)
.where(eq(schema.analyticsProjectionCheckpoints.projectorKey, USAGE_PROJECTOR_KEY))
.get();
return row || emptyCheckpoint();
}
async function ensureProjectionCheckpointExists() {
const nowIso = new Date().toISOString();
await (db.insert(schema.analyticsProjectionCheckpoints).values({
projectorKey: USAGE_PROJECTOR_KEY,
timeZone: getResolvedTimeZone(),
lastProxyLogId: 0,
createdAt: nowIso,
updatedAt: nowIso,
}) as any)
.onConflictDoNothing({
target: schema.analyticsProjectionCheckpoints.projectorKey,
})
.run();
}
async function tryAcquireProjectionLease(): Promise<ProjectionLease | null> {
await ensureProjectionCheckpointExists();
const nowIso = new Date().toISOString();
const lease: ProjectionLease = {
owner: buildProjectionLeaseOwner(),
token: randomUUID(),
expiresAt: buildProjectionLeaseExpiry(),
};
const result = await db
.update(schema.analyticsProjectionCheckpoints)
.set({
leaseOwner: lease.owner,
leaseToken: lease.token,
leaseExpiresAt: lease.expiresAt,
updatedAt: nowIso,
})
.where(
and(
eq(schema.analyticsProjectionCheckpoints.projectorKey, USAGE_PROJECTOR_KEY),
or(
isNull(schema.analyticsProjectionCheckpoints.leaseExpiresAt),
lte(schema.analyticsProjectionCheckpoints.leaseExpiresAt, nowIso),
),
),
)
.run();
return result.changes > 0 ? lease : null;
}
async function releaseProjectionLease(
lease: ProjectionLease,
options?: { error?: unknown },
) {
const nowIso = new Date().toISOString();
await db
.update(schema.analyticsProjectionCheckpoints)
.set({
leaseOwner: null,
leaseToken: null,
leaseExpiresAt: null,
lastError: options?.error ? normalizeProjectionError(options.error) : null,
updatedAt: nowIso,
})
.where(
and(
eq(schema.analyticsProjectionCheckpoints.projectorKey, USAGE_PROJECTOR_KEY),
eq(schema.analyticsProjectionCheckpoints.leaseToken, lease.token),
),
)
.run();
}
async function writeProjectionCheckpoint(
tx: typeof db,
checkpoint: Partial<ProjectionCheckpointRow> & { lastProxyLogId: number },
) {
const nowIso = new Date().toISOString();
const values = {
projectorKey: USAGE_PROJECTOR_KEY,
timeZone: checkpoint.timeZone ?? getResolvedTimeZone(),
lastProxyLogId: Math.max(0, Math.trunc(checkpoint.lastProxyLogId || 0)),
watermarkCreatedAt: checkpoint.watermarkCreatedAt ?? null,
recomputeFromId: checkpoint.recomputeFromId ?? null,
recomputeRequestedAt: checkpoint.recomputeRequestedAt ?? null,
recomputeReason: checkpoint.recomputeReason ?? null,
recomputeStartedAt: checkpoint.recomputeStartedAt ?? null,
recomputeCompletedAt: checkpoint.recomputeCompletedAt ?? null,
leaseOwner: checkpoint.leaseOwner ?? null,
leaseToken: checkpoint.leaseToken ?? null,
leaseExpiresAt: checkpoint.leaseExpiresAt ?? null,
lastProjectedAt: checkpoint.lastProjectedAt ?? nowIso,
lastSuccessfulAt: checkpoint.lastSuccessfulAt ?? nowIso,
lastError: checkpoint.lastError ?? null,
createdAt: checkpoint.createdAt ?? nowIso,
updatedAt: nowIso,
};
await (tx.insert(schema.analyticsProjectionCheckpoints).values(values) as any)
.onConflictDoUpdate({
target: schema.analyticsProjectionCheckpoints.projectorKey,
set: {
timeZone: values.timeZone,
lastProxyLogId: values.lastProxyLogId,
watermarkCreatedAt: values.watermarkCreatedAt,
recomputeFromId: values.recomputeFromId,
recomputeRequestedAt: values.recomputeRequestedAt,
recomputeReason: values.recomputeReason,
recomputeStartedAt: values.recomputeStartedAt,
recomputeCompletedAt: values.recomputeCompletedAt,
leaseOwner: values.leaseOwner,
leaseToken: values.leaseToken,
leaseExpiresAt: values.leaseExpiresAt,
lastProjectedAt: values.lastProjectedAt,
lastSuccessfulAt: values.lastSuccessfulAt,
lastError: values.lastError,
updatedAt: values.updatedAt,
},
})
.run();
}
async function fetchProjectionBatch(afterId: number, limit: number) {
const rows = await db
.select({
id: schema.proxyLogs.id,
createdAt: schema.proxyLogs.createdAt,
status: schema.proxyLogs.status,
latencyMs: schema.proxyLogs.latencyMs,
totalTokens: schema.proxyLogs.totalTokens,
estimatedCost: schema.proxyLogs.estimatedCost,
modelActual: schema.proxyLogs.modelActual,
modelRequested: schema.proxyLogs.modelRequested,
siteId: schema.sites.id,
sitePlatform: schema.sites.platform,
})
.from(schema.proxyLogs)
.leftJoin(schema.accounts, eq(schema.proxyLogs.accountId, schema.accounts.id))
.leftJoin(schema.sites, eq(schema.accounts.siteId, schema.sites.id))
.where(gt(schema.proxyLogs.id, afterId))
.orderBy(asc(schema.proxyLogs.id))
.limit(limit)
.all();
return rows as ProxyLogProjectionRow[];
}
function buildProjectionBatchDelta(rows: ProxyLogProjectionRow[]): ProjectionBatchDelta {
const siteDayMap = new Map<string, SiteDayUsageDeltaRow>();
const siteHourMap = new Map<string, SiteHourUsageDeltaRow>();
const modelDayMap = new Map<string, ModelDayUsageDeltaRow>();
for (const row of rows) {
const siteId = typeof row.siteId === 'number' && row.siteId > 0 ? row.siteId : null;
if (!siteId) continue;
const localDay = toLocalDayKeyFromStoredUtc(row.createdAt);
const bucketStartUtc = toLocalHourStartUtcFromStoredUtc(row.createdAt);
if (!localDay || !bucketStartUtc) continue;
const status = String(row.status || '').trim().toLowerCase();
const isSuccess = status === 'success';
const totalTokens = normalizeNonNegativeInt(row.totalTokens);
const latencyMs = normalizeNonNegativeInt(row.latencyMs);
const latencyCount = latencyMs > 0 ? 1 : 0;
const totalSummarySpend = resolveSummarySpend({
estimatedCost: row.estimatedCost,
totalTokens: row.totalTokens,
platform: row.sitePlatform,
});
const totalSiteSpend = resolveSiteSpend({
estimatedCost: row.estimatedCost,
totalTokens: row.totalTokens,
platform: row.sitePlatform,
});
const model = resolveModelName(row);
const modelSpend = resolveModelSpend({
estimatedCost: row.estimatedCost,
totalTokens: row.totalTokens,
});
const siteDayKey = `${localDay}:${siteId}`;
const siteDay = siteDayMap.get(siteDayKey) || {
localDay,
siteId,
totalCalls: 0,
successCalls: 0,
failedCalls: 0,
totalTokens: 0,
totalSummarySpend: 0,
totalSiteSpend: 0,
totalLatencyMs: 0,
latencyCount: 0,
};
siteDay.totalCalls += 1;
siteDay.successCalls += isSuccess ? 1 : 0;
siteDay.failedCalls += isSuccess ? 0 : 1;
siteDay.totalTokens += totalTokens;
siteDay.totalSummarySpend += totalSummarySpend;
siteDay.totalSiteSpend += totalSiteSpend;
siteDay.totalLatencyMs += latencyMs;
siteDay.latencyCount += latencyCount;
siteDayMap.set(siteDayKey, siteDay);
const siteHourKey = `${bucketStartUtc}:${siteId}`;
const siteHour = siteHourMap.get(siteHourKey) || {
bucketStartUtc,
siteId,
totalCalls: 0,
successCalls: 0,
failedCalls: 0,
totalTokens: 0,
totalSummarySpend: 0,
totalSiteSpend: 0,
totalLatencyMs: 0,
latencyCount: 0,
};
siteHour.totalCalls += 1;
siteHour.successCalls += isSuccess ? 1 : 0;
siteHour.failedCalls += isSuccess ? 0 : 1;
siteHour.totalTokens += totalTokens;
siteHour.totalSummarySpend += totalSummarySpend;
siteHour.totalSiteSpend += totalSiteSpend;
siteHour.totalLatencyMs += latencyMs;
siteHour.latencyCount += latencyCount;
siteHourMap.set(siteHourKey, siteHour);
const modelDayKey = `${localDay}:${siteId}:${model}`;
const modelDay = modelDayMap.get(modelDayKey) || {
localDay,
siteId,
model,
totalCalls: 0,
successCalls: 0,
failedCalls: 0,
totalTokens: 0,
totalSpend: 0,
totalLatencyMs: 0,
latencyCount: 0,
};
modelDay.totalCalls += 1;
modelDay.successCalls += isSuccess ? 1 : 0;
modelDay.failedCalls += isSuccess ? 0 : 1;
modelDay.totalTokens += totalTokens;
modelDay.totalSpend += modelSpend;
modelDay.totalLatencyMs += latencyMs;
modelDay.latencyCount += latencyCount;
modelDayMap.set(modelDayKey, modelDay);
}
return {
siteDayRows: Array.from(siteDayMap.values()),
siteHourRows: Array.from(siteHourMap.values()),
modelDayRows: Array.from(modelDayMap.values()),
};
}
async function upsertSiteDayUsage(tx: typeof db, row: SiteDayUsageDeltaRow, updatedAt: string) {
await (tx.insert(schema.siteDayUsage).values({
localDay: row.localDay,
siteId: row.siteId,
totalCalls: row.totalCalls,
successCalls: row.successCalls,
failedCalls: row.failedCalls,
totalTokens: row.totalTokens,
totalSummarySpend: row.totalSummarySpend,
totalSiteSpend: row.totalSiteSpend,
totalLatencyMs: row.totalLatencyMs,
latencyCount: row.latencyCount,
updatedAt,
}) as any)
.onConflictDoUpdate({
target: [schema.siteDayUsage.localDay, schema.siteDayUsage.siteId],
set: {
totalCalls: sql`${schema.siteDayUsage.totalCalls} + ${row.totalCalls}`,
successCalls: sql`${schema.siteDayUsage.successCalls} + ${row.successCalls}`,
failedCalls: sql`${schema.siteDayUsage.failedCalls} + ${row.failedCalls}`,
totalTokens: sql`${schema.siteDayUsage.totalTokens} + ${row.totalTokens}`,
totalSummarySpend: sql`${schema.siteDayUsage.totalSummarySpend} + ${row.totalSummarySpend}`,
totalSiteSpend: sql`${schema.siteDayUsage.totalSiteSpend} + ${row.totalSiteSpend}`,
totalLatencyMs: sql`${schema.siteDayUsage.totalLatencyMs} + ${row.totalLatencyMs}`,
latencyCount: sql`${schema.siteDayUsage.latencyCount} + ${row.latencyCount}`,
updatedAt,
},
})
.run();
}
async function upsertSiteHourUsage(tx: typeof db, row: SiteHourUsageDeltaRow, updatedAt: string) {
await (tx.insert(schema.siteHourUsage).values({
bucketStartUtc: row.bucketStartUtc,
siteId: row.siteId,
totalCalls: row.totalCalls,
successCalls: row.successCalls,
failedCalls: row.failedCalls,
totalTokens: row.totalTokens,
totalSummarySpend: row.totalSummarySpend,
totalSiteSpend: row.totalSiteSpend,
totalLatencyMs: row.totalLatencyMs,
latencyCount: row.latencyCount,
updatedAt,
}) as any)
.onConflictDoUpdate({
target: [schema.siteHourUsage.bucketStartUtc, schema.siteHourUsage.siteId],
set: {
totalCalls: sql`${schema.siteHourUsage.totalCalls} + ${row.totalCalls}`,
successCalls: sql`${schema.siteHourUsage.successCalls} + ${row.successCalls}`,
failedCalls: sql`${schema.siteHourUsage.failedCalls} + ${row.failedCalls}`,
totalTokens: sql`${schema.siteHourUsage.totalTokens} + ${row.totalTokens}`,
totalSummarySpend: sql`${schema.siteHourUsage.totalSummarySpend} + ${row.totalSummarySpend}`,
totalSiteSpend: sql`${schema.siteHourUsage.totalSiteSpend} + ${row.totalSiteSpend}`,
totalLatencyMs: sql`${schema.siteHourUsage.totalLatencyMs} + ${row.totalLatencyMs}`,
latencyCount: sql`${schema.siteHourUsage.latencyCount} + ${row.latencyCount}`,
updatedAt,
},
})
.run();
}
async function upsertModelDayUsage(tx: typeof db, row: ModelDayUsageDeltaRow, updatedAt: string) {
await (tx.insert(schema.modelDayUsage).values({
localDay: row.localDay,
siteId: row.siteId,
model: row.model,
totalCalls: row.totalCalls,
successCalls: row.successCalls,
failedCalls: row.failedCalls,
totalTokens: row.totalTokens,
totalSpend: row.totalSpend,
totalLatencyMs: row.totalLatencyMs,
latencyCount: row.latencyCount,
updatedAt,
}) as any)
.onConflictDoUpdate({
target: [schema.modelDayUsage.localDay, schema.modelDayUsage.siteId, schema.modelDayUsage.model],
set: {
totalCalls: sql`${schema.modelDayUsage.totalCalls} + ${row.totalCalls}`,
successCalls: sql`${schema.modelDayUsage.successCalls} + ${row.successCalls}`,
failedCalls: sql`${schema.modelDayUsage.failedCalls} + ${row.failedCalls}`,
totalTokens: sql`${schema.modelDayUsage.totalTokens} + ${row.totalTokens}`,
totalSpend: sql`${schema.modelDayUsage.totalSpend} + ${row.totalSpend}`,
totalLatencyMs: sql`${schema.modelDayUsage.totalLatencyMs} + ${row.totalLatencyMs}`,
latencyCount: sql`${schema.modelDayUsage.latencyCount} + ${row.latencyCount}`,
updatedAt,
},
})
.run();
}
async function applyProjectionBatch(
checkpoint: ProjectionCheckpointRow,
rows: ProxyLogProjectionRow[],
): Promise<ProjectionCheckpointRow> {
const lastRow = rows.at(-1);
if (!lastRow) return checkpoint;
const delta = buildProjectionBatchDelta(rows);
const updatedAt = new Date().toISOString();
const nextCheckpoint = {
...checkpoint,
lastProxyLogId: lastRow.id,
watermarkCreatedAt:
typeof lastRow.createdAt === 'string'
? lastRow.createdAt
: String(lastRow.createdAt || ''),
recomputeFromId: checkpoint.recomputeFromId ?? null,
recomputeRequestedAt: checkpoint.recomputeRequestedAt ?? null,
leaseExpiresAt: checkpoint.leaseToken ? buildProjectionLeaseExpiry() : checkpoint.leaseExpiresAt,
lastProjectedAt: updatedAt,
lastSuccessfulAt: updatedAt,
lastError: null,
createdAt: checkpoint.createdAt ?? updatedAt,
};
await db.transaction(async (tx) => {
for (const row of delta.siteDayRows) {
await upsertSiteDayUsage(tx as typeof db, row, updatedAt);
}
for (const row of delta.siteHourRows) {
await upsertSiteHourUsage(tx as typeof db, row, updatedAt);
}
for (const row of delta.modelDayRows) {
await upsertModelDayUsage(tx as typeof db, row, updatedAt);
}
await writeProjectionCheckpoint(tx as typeof db, nextCheckpoint);
});
clearAnalyticsSnapshots();
return {
...checkpoint,
...nextCheckpoint,
updatedAt,
};
}
async function applyPendingRecompute(checkpoint: ProjectionCheckpointRow) {
const recomputeFromId = normalizeNonNegativeInt(checkpoint.recomputeFromId);
if (recomputeFromId <= 0) return checkpoint;
const affectedRow = await db
.select({
id: schema.proxyLogs.id,
createdAt: schema.proxyLogs.createdAt,
})
.from(schema.proxyLogs)
.where(gte(schema.proxyLogs.id, recomputeFromId))
.orderBy(asc(schema.proxyLogs.id))
.get();
if (!affectedRow) {
const nextCheckpoint = {
...checkpoint,
recomputeFromId: null,
recomputeRequestedAt: null,
leaseExpiresAt: checkpoint.leaseToken ? buildProjectionLeaseExpiry() : checkpoint.leaseExpiresAt,
lastProjectedAt: new Date().toISOString(),
};
await db.transaction(async (tx) => {
await writeProjectionCheckpoint(tx as typeof db, nextCheckpoint as any);
});
return { ...checkpoint, ...nextCheckpoint };
}
const affectedDay = toLocalDayKeyFromStoredUtc(affectedRow.createdAt);
const affectedDayStartUtc = toLocalDayStartUtcFromStoredUtc(affectedRow.createdAt);
if (!affectedDay || !affectedDayStartUtc) {
throw new Error('Failed to resolve recompute boundary for usage aggregates');
}
const restartRow = await db
.select({
id: schema.proxyLogs.id,
createdAt: schema.proxyLogs.createdAt,
})
.from(schema.proxyLogs)
.where(gte(schema.proxyLogs.createdAt, affectedDayStartUtc))
.orderBy(asc(schema.proxyLogs.id))
.get();
const restartFromId = restartRow?.id || affectedRow.id;
const nextCheckpoint = {
...checkpoint,
lastProxyLogId: Math.max(0, restartFromId - 1),
watermarkCreatedAt: null,
recomputeFromId: null,
recomputeRequestedAt: null,
leaseExpiresAt: checkpoint.leaseToken ? buildProjectionLeaseExpiry() : checkpoint.leaseExpiresAt,
lastProjectedAt: new Date().toISOString(),
};
await db.transaction(async (tx) => {
await tx.delete(schema.siteDayUsage).where(gte(schema.siteDayUsage.localDay, affectedDay)).run();
await tx.delete(schema.siteHourUsage).where(gte(schema.siteHourUsage.bucketStartUtc, affectedDayStartUtc)).run();
await tx.delete(schema.modelDayUsage).where(gte(schema.modelDayUsage.localDay, affectedDay)).run();
await writeProjectionCheckpoint(tx as typeof db, nextCheckpoint as any);
});
clearAnalyticsSnapshots();
return { ...checkpoint, ...nextCheckpoint };
}
async function runUsageAggregationProjectionPassImpl(
options: ProjectionPassOptions = {},
): Promise<ProjectionPassResult> {
const lease = await tryAcquireProjectionLease();
if (!lease) {
const checkpoint = await readProjectionCheckpoint();
return {
processedLogs: 0,
watermarkId: checkpoint.lastProxyLogId,
recomputed: false,
};
}
try {
let checkpoint: ProjectionCheckpointRow = {
...(await readProjectionCheckpoint()),
leaseOwner: lease.owner,
leaseToken: lease.token,
leaseExpiresAt: lease.expiresAt,
};
const hadPendingRecompute = normalizeNonNegativeInt(checkpoint.recomputeFromId) > 0;
if (hadPendingRecompute) {
checkpoint = await applyPendingRecompute(checkpoint);
}
let processedLogs = 0;
const maxBatches = Math.max(
1,
Math.trunc(options.maxBatches || PROJECTION_MAX_BATCHES_PER_PASS),
);
for (let index = 0; index < maxBatches; index += 1) {
const rows = await fetchProjectionBatch(checkpoint.lastProxyLogId, PROJECTION_BATCH_SIZE);
if (rows.length <= 0) {
break;
}
checkpoint = await applyProjectionBatch(checkpoint, rows);
processedLogs += rows.length;
if (rows.length < PROJECTION_BATCH_SIZE) {
break;
}
}
await releaseProjectionLease(lease);
return {
processedLogs,
watermarkId: checkpoint.lastProxyLogId,
recomputed: hadPendingRecompute,
};
} catch (error) {
await releaseProjectionLease(lease, { error });
throw error;
}
}
export async function runUsageAggregationProjectionPass(
options: ProjectionPassOptions = {},
): Promise<ProjectionPassResult> {
if (projectionInFlight) {
return projectionInFlight;
}
projectionInFlight = runUsageAggregationProjectionPassImpl(options).finally(() => {
projectionInFlight = null;
});
return projectionInFlight;
}
export async function requestUsageAggregatesRecompute(fromLogId = 1): Promise<void> {
const checkpoint = await readProjectionCheckpoint();
const normalizedFromId = Math.max(1, Math.trunc(fromLogId || 1));
const nextFromId = checkpoint.recomputeFromId && checkpoint.recomputeFromId > 0
? Math.min(checkpoint.recomputeFromId, normalizedFromId)
: normalizedFromId;
await db.transaction(async (tx) => {
await writeProjectionCheckpoint(tx as typeof db, {
...checkpoint,
lastProxyLogId: checkpoint.lastProxyLogId,
recomputeFromId: nextFromId,
recomputeRequestedAt: new Date().toISOString(),
lastProjectedAt: checkpoint.lastProjectedAt,
} as any);
});
}
export function startUsageAggregationProjectorScheduler() {
if (projectionTimer) return;
void runUsageAggregationProjectionPass();
projectionTimer = setInterval(() => {
void runUsageAggregationProjectionPass();
}, PROJECTION_INTERVAL_MS);
}
export async function stopUsageAggregationProjectorScheduler() {
if (projectionTimer) {
clearInterval(projectionTimer);
projectionTimer = null;
}
if (projectionInFlight) {
await projectionInFlight;
}
}
export async function __resetUsageAggregationProjectorForTests() {
await stopUsageAggregationProjectorScheduler();
}
src/web/App.tsx
+1 -1
View File
@@ -152,7 +152,7 @@ export function Login({ onLogin, t }: { onLogin: (token: string) => void; t: (te
setLoading(true);
setError('');
try {
const res = await fetch('/api/stats/dashboard', {
const res = await fetch('/api/settings/auth/info', {
headers: { Authorization: `Bearer ${token}` },
});
if (res.ok) {
src/web/api.test.ts
+6 -6
View File
@@ -137,12 +137,12 @@ describe('api proxy test timeout handling', () => {
const getProxyFileContentDataUrl = (api as Record<string, any>).getProxyFileContentDataUrl;
const result = await getProxyFileContentDataUrl?.('file-metapi-123');
expect(fetchMock).toHaveBeenCalledWith('/v1/files/file-metapi-123/content', expect.objectContaining({
method: 'GET',
headers: expect.objectContaining({
Authorization: 'Bearer token-1',
}),
}));
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(fetchMock.mock.calls[0]?.[0]).toBe('/v1/files/file-metapi-123/content');
const init = fetchMock.mock.calls[0]?.[1] as RequestInit | undefined;
expect(init?.method).toBe('GET');
expect(init?.headers).toBeInstanceOf(Headers);
expect((init?.headers as Headers).get('Authorization')).toBe('Bearer token-1');
expect(result).toEqual({
filename: 'brief.pdf',
mimeType: 'application/pdf',
src/web/api.ts
+740 -371
View File
File diff suppressed because it is too large Load Diff
src/web/pages/Accounts.tsx
+2165 -831
View File
File diff suppressed because it is too large Load Diff
src/web/pages/Dashboard.tsx
+1043 -255
View File
File diff suppressed because it is too large Load Diff
src/web/pages/DownstreamKeys.test.tsx
+5 -2
View File
@@ -3,6 +3,7 @@ import { act, create, type ReactTestInstance } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import DownstreamKeys from './DownstreamKeys.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
@@ -10,6 +11,7 @@ const { apiMock } = vi.hoisted(() => ({
getDownstreamApiKeys: vi.fn(),
getRoutesLite: vi.fn(),
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getAccountTokens: vi.fn(),
getDownstreamApiKeyOverview: vi.fn(),
getDownstreamApiKeyTrend: vi.fn(),
@@ -126,6 +128,7 @@ function buildRawItem(overrides?: Partial<any>) {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
(globalThis as any).document = {
body: { style: {} },
addEventListener: vi.fn(),
@@ -837,7 +840,7 @@ describe('DownstreamKeys page', () => {
});
await flushMicrotasks();
expect(apiMock.getAccounts).not.toHaveBeenCalled();
expect(apiMock.getAccountsSnapshot).not.toHaveBeenCalled();
expect(apiMock.getAccountTokens).not.toHaveBeenCalled();
const createBtn = root!.root.findAll((node) => node.type === 'button' && collectText(node).includes('新增下游密钥'))[0];
@@ -846,7 +849,7 @@ describe('DownstreamKeys page', () => {
});
await flushMicrotasks();
expect(apiMock.getAccounts).toHaveBeenCalledTimes(1);
expect(apiMock.getAccountsSnapshot).toHaveBeenCalledTimes(1);
expect(apiMock.getAccountTokens).toHaveBeenCalledTimes(1);
const advancedBtn = root!.root.findAll((node) => node.type === 'button' && collectText(node).includes('高级配置'))[0];
src/web/pages/DownstreamKeys.tsx
+5 -3
View File
@@ -518,12 +518,14 @@ export default function DownstreamKeys() {
if (exclusionSourceLoading || exclusionSourceLoaded) return;
setExclusionSourceLoading(true);
try {
const [accountsRes, tokensRes] = await Promise.all([
api.getAccounts(),
const [accountsSnapshotRes, tokensRes] = await Promise.all([
api.getAccountsSnapshot(),
api.getAccountTokens(),
]);
const accountRows = Array.isArray(accountsRes) ? accountsRes : [];
const accountRows = Array.isArray(accountsSnapshotRes?.accounts)
? accountsSnapshotRes.accounts
: [];
const tokenRows = Array.isArray(tokensRes) ? tokensRes : [];
const siteMap = new Map<number, { siteId: number; siteName: string; accountIds: Set<number> }>();
src/web/pages/ProxyLogs.server-driven.test.tsx
+13
View File
@@ -8,6 +8,8 @@ import ProxyLogs from './ProxyLogs.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getProxyLogs: vi.fn(),
getProxyLogsQuery: vi.fn(),
getProxyLogsMeta: vi.fn(),
getProxyLogDetail: vi.fn(),
getProxyDebugTraces: vi.fn(),
getProxyDebugTraceDetail: vi.fn(),
@@ -141,6 +143,17 @@ describe('ProxyLogs server-driven page', () => {
proxyDebugMaxBodyBytes: 262144,
});
apiMock.getProxyLogs.mockResolvedValue(buildListResponse());
apiMock.getProxyLogsQuery.mockImplementation((params: any) =>
apiMock.getProxyLogs(params),
);
apiMock.getProxyLogsMeta.mockResolvedValue({
summary: buildListResponse().summary,
clientOptions: buildListResponse().clientOptions,
sites: [
{ id: 1, name: 'main-site', status: 'active' },
{ id: 2, name: 'backup-site', status: 'active' },
],
});
apiMock.getProxyLogDetail.mockResolvedValue({
id: 101,
createdAt: '2026-03-09 16:00:00',
src/web/pages/ProxyLogs.tsx
+2143 -859
View File
File diff suppressed because it is too large Load Diff
src/web/pages/Tokens.tsx
+7 -2
View File
@@ -175,11 +175,16 @@ export function TokensPanel({ embedded = false, onEmbeddedActionsChange }: Token
const load = useCallback(async () => {
setLoading(true);
try {
const [tokenRows, accountRows] = await Promise.all([api.getAccountTokens(), api.getAccounts()]);
const [tokenRows, accountSnapshot] = await Promise.all([
api.getAccountTokens(),
api.getAccountsSnapshot(),
]);
const nextTokens = tokenRows || [];
setTokens(nextTokens);
setSelectedTokenIds((current) => current.filter((id) => nextTokens.some((token: any) => token.id === id)));
const latestAccounts: SyncableAccount[] = Array.isArray(accountRows) ? accountRows : [];
const latestAccounts: SyncableAccount[] = Array.isArray(accountSnapshot?.accounts)
? accountSnapshot.accounts
: [];
setAccounts(latestAccounts);
const syncableAccounts = latestAccounts.filter(isAccountSyncable);
src/web/pages/accounts.batch-actions.test.tsx
+3
View File
@@ -3,10 +3,12 @@ import { act, create } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import Accounts from './Accounts.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
batchUpdateAccounts: vi.fn(),
refreshAccountHealth: vi.fn(),
@@ -27,6 +29,7 @@ async function flushMicrotasks() {
describe('Accounts batch actions', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
apiMock.getSites.mockResolvedValue([
{ id: 1, name: 'Site A', platform: 'new-api', status: 'active' },
]);
src/web/pages/accounts.centered-modal.test.ts
+3 -1
View File
@@ -6,7 +6,9 @@ describe('Accounts centered modal adoption', () => {
it('uses CenteredModal for add/edit/rebind flows instead of inline panel cards', () => {
const source = readFileSync(resolve(process.cwd(), 'src/web/pages/Accounts.tsx'), 'utf8');
expect(source).toContain("import CenteredModal from '../components/CenteredModal.js'");
expect(source).toContain(
'import CenteredModal from "../components/CenteredModal.js";',
);
expect(source).toContain('<CenteredModal');
expect(source).not.toContain('className={`card panel-presence rebind-panel');
});
src/web/pages/accounts.codingplan-init.test.tsx
+3
View File
@@ -3,10 +3,12 @@ import { act, create, type ReactTestInstance, type ReactTestRenderer } from 'rea
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import Accounts from './Accounts.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
getAccountTokens: vi.fn(),
addAccount: vi.fn(),
@@ -36,6 +38,7 @@ async function flushMicrotasks() {
describe('Accounts CodingPlan initialization', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
apiMock.getAccounts.mockResolvedValue([]);
apiMock.getSites.mockResolvedValue([
{ id: 10, name: 'Aliyun CodingPlan', url: 'https://coding.dashscope.aliyuncs.com/v1', platform: 'openai', status: 'active' },
src/web/pages/accounts.create-intent.test.tsx
+3
View File
@@ -4,10 +4,12 @@ import { MemoryRouter } from 'react-router-dom';
import ModernSelect from '../components/ModernSelect.js';
import { ToastProvider } from '../components/Toast.js';
import Accounts from './Accounts.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
getAccountTokens: vi.fn(),
},
@@ -51,6 +53,7 @@ async function renderAccounts(
describe('Accounts create intent handling', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
});
afterEach(() => {
src/web/pages/accounts.edit-panel.test.tsx
+3
View File
@@ -4,10 +4,12 @@ import { act, create, type ReactTestInstance } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import Accounts from './Accounts.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock, toastMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
updateAccount: vi.fn(),
updateSiteDisabledModels: vi.fn(),
@@ -60,6 +62,7 @@ function deferred<T>() {
describe('Accounts edit panel', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
apiMock.getSites.mockResolvedValue([
{ id: 1, name: 'Site A', platform: 'new-api', status: 'active' },
]);
src/web/pages/accounts.mobile-actions.test.tsx
+3
View File
@@ -3,10 +3,12 @@ import { act, create } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import Accounts from './Accounts.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
batchUpdateAccounts: vi.fn(),
refreshAccountHealth: vi.fn(),
@@ -46,6 +48,7 @@ function findButtonByText(root: any, text: string) {
describe('Accounts mobile actions', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
apiMock.getSites.mockResolvedValue([
{ id: 1, name: 'Site A', platform: 'new-api', status: 'active' },
]);
src/web/pages/accounts.model-modal.architecture.test.ts
+3 -1
View File
@@ -6,7 +6,9 @@ describe('Accounts model modal extraction', () => {
it('delegates the account model management modal to a dedicated accounts component', () => {
const source = readFileSync(resolve(process.cwd(), 'src/web/pages/Accounts.tsx'), 'utf8').replace(/\r\n/g, '\n');
expect(source).toContain("import AccountModelsModal from './accounts/AccountModelsModal.js'");
expect(source).toContain(
'import AccountModelsModal from "./accounts/AccountModelsModal.js";',
);
expect(source).not.toContain('open={modelModal.open}');
});
});
src/web/pages/accounts.proxy-only-expired.test.tsx
+3
View File
@@ -3,10 +3,12 @@ import { act, create, type ReactTestInstance } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import Accounts from './Accounts.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
},
}));
@@ -33,6 +35,7 @@ async function flushMicrotasks() {
describe('Accounts proxy-only expired state', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
});
afterEach(() => {
src/web/pages/accounts.rebind-panel-focus.test.tsx
+3
View File
@@ -3,10 +3,12 @@ import { act, create, type ReactTestInstance } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import Accounts from './Accounts.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
},
}));
@@ -33,6 +35,7 @@ async function flushMicrotasks() {
describe('Accounts rebind modal', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
});
afterEach(() => {
src/web/pages/accounts.refresh-reload-on-error.test.tsx
+3
View File
@@ -3,10 +3,12 @@ import { act, create, type ReactTestInstance } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import Accounts from './Accounts.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
refreshBalance: vi.fn(),
},
@@ -34,6 +36,7 @@ async function flushMicrotasks() {
describe('Accounts refresh action', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
});
afterEach(() => {
src/web/pages/accounts.segmented-connections.test.tsx
+3
View File
@@ -3,10 +3,12 @@ import { act, create, type ReactTestInstance } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import Accounts from './Accounts.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
getAccountTokens: vi.fn(),
},
@@ -32,6 +34,7 @@ function collectText(node: ReactTestInstance): string {
describe('Accounts segmented connections view', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
});
afterEach(() => {
src/web/pages/accounts.tokens-header.test.tsx
+3
View File
@@ -3,10 +3,12 @@ import { act, create, type ReactTestInstance } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import Accounts from './Accounts.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
getAccountTokens: vi.fn(),
},
@@ -36,6 +38,7 @@ async function flushMicrotasks() {
describe('Accounts tokens embedded header', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
});
afterEach(() => {
src/web/pages/accounts.verify-feedback.test.tsx
+3
View File
@@ -4,6 +4,7 @@ import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import ModernSelect from '../components/ModernSelect.js';
import Accounts from './Accounts.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
import {
buildAddAccountPrereqHint,
buildVerifyFailureHint,
@@ -13,6 +14,7 @@ import {
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
verifyToken: vi.fn(),
},
@@ -40,6 +42,7 @@ async function flushMicrotasks() {
describe('Accounts verify feedback', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
});
afterEach(() => {
src/web/pages/dashboard.mobile-layout.test.tsx
+3 -3
View File
@@ -6,9 +6,9 @@ describe('Dashboard mobile layout', () => {
it('uses the shared mobile breakpoint to collapse fixed desktop grids', () => {
const source = readFileSync(resolve(process.cwd(), 'src/web/pages/Dashboard.tsx'), 'utf8');
expect(source).toContain("import { useIsMobile } from '../components/useIsMobile.js'");
expect(source).toContain('import { useIsMobile } from "../components/useIsMobile.js";');
expect(source).toContain('const isMobile = useIsMobile()');
expect(source).toContain("gridTemplateColumns: isMobile ? '1fr' : '1fr 1fr'");
expect(source).toContain("gridTemplateColumns: isMobile ? '1fr' : '1fr 300px'");
expect(source).toContain('gridTemplateColumns: isMobile ? "1fr" : "1fr 1fr"');
expect(source).toContain('gridTemplateColumns: isMobile ? "1fr" : "1fr 300px"');
});
});
src/web/pages/dashboard.performance-card.test.tsx
+5
View File
@@ -2,10 +2,14 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { act, create, type ReactTestInstance } from 'react-test-renderer';
import { ToastProvider } from '../components/Toast.js';
import Dashboard from './Dashboard.js';
import { installDashboardSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getDashboard: vi.fn(),
getDashboardSnapshot: vi.fn(),
getDashboardInsights: vi.fn(),
getSiteSnapshot: vi.fn(),
getSiteDistribution: vi.fn(),
getSiteTrend: vi.fn(),
getSites: vi.fn(),
@@ -40,6 +44,7 @@ describe('Dashboard performance stat card', () => {
beforeEach(() => {
vi.clearAllMocks();
installDashboardSnapshotCompat(apiMock);
apiMock.getSiteDistribution.mockResolvedValue({ distribution: [] });
apiMock.getSiteTrend.mockResolvedValue({ trend: [] });
apiMock.getSites.mockResolvedValue([]);
src/web/pages/dashboard.site-observability.test.tsx
+5
View File
@@ -3,10 +3,14 @@ import { act, create, type ReactTestInstance } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import Dashboard from './Dashboard.js';
import { installDashboardSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getDashboard: vi.fn(),
getDashboardSnapshot: vi.fn(),
getDashboardInsights: vi.fn(),
getSiteSnapshot: vi.fn(),
getSiteDistribution: vi.fn(),
getSiteTrend: vi.fn(),
getSites: vi.fn(),
@@ -36,6 +40,7 @@ describe('Dashboard site observability panel', () => {
beforeEach(() => {
vi.clearAllMocks();
installDashboardSnapshotCompat(apiMock);
apiMock.getDashboard.mockResolvedValue({
totalBalance: 0,
totalUsed: 0,
src/web/pages/dashboard.site-speed-button.test.tsx
+5
View File
@@ -2,10 +2,14 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { act, create, type ReactTestInstance } from 'react-test-renderer';
import { ToastProvider } from '../components/Toast.js';
import Dashboard from './Dashboard.js';
import { installDashboardSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getDashboard: vi.fn(),
getDashboardSnapshot: vi.fn(),
getDashboardInsights: vi.fn(),
getSiteSnapshot: vi.fn(),
getSiteDistribution: vi.fn(),
getSiteTrend: vi.fn(),
getSites: vi.fn(),
@@ -36,6 +40,7 @@ describe('Dashboard site speed buttons', () => {
beforeEach(() => {
vi.clearAllMocks();
installDashboardSnapshotCompat(apiMock);
apiMock.getDashboard.mockResolvedValue({
totalBalance: 0,
totalUsed: 0,
src/web/pages/dashboardHookOrder.test.tsx
+5
View File
@@ -2,10 +2,14 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { act, create } from 'react-test-renderer';
import { ToastProvider } from '../components/Toast.js';
import Dashboard from './Dashboard.js';
import { installDashboardSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getDashboard: vi.fn(),
getDashboardSnapshot: vi.fn(),
getDashboardInsights: vi.fn(),
getSiteSnapshot: vi.fn(),
getSiteDistribution: vi.fn(),
getSiteTrend: vi.fn(),
getSites: vi.fn(),
@@ -37,6 +41,7 @@ describe('Dashboard hook order', () => {
afterEach(() => {
consoleErrorSpy.mockRestore();
vi.clearAllMocks();
installDashboardSnapshotCompat(apiMock);
});
it('keeps hook order stable when switching from loading to loaded render', async () => {
src/web/pages/logs.mobile.test.tsx
+3 -1
View File
@@ -6,7 +6,9 @@ describe('ProxyLogs mobile layout', () => {
it('renders compact mobile summary cards for proxy logs', () => {
const source = readFileSync(resolve(process.cwd(), 'src/web/pages/ProxyLogs.tsx'), 'utf8');
expect(source).toContain('MobileCard');
expect(source).toContain("import ResponsiveFilterPanel from '../components/ResponsiveFilterPanel.js'");
expect(source).toContain(
'import ResponsiveFilterPanel from "../components/ResponsiveFilterPanel.js";',
);
expect(source).toContain('<ResponsiveFilterPanel');
expect(source).toContain('compact');
expect(source).toContain('mobile-summary-grid');
src/web/pages/testApiCompat.ts
+94
View File
@@ -0,0 +1,94 @@
type MockLike = ((...args: any[]) => any) & {
mockImplementation?: (impl: (...args: any[]) => any) => unknown;
};
type AccountsCompatApiMock = {
getAccounts?: MockLike;
getAccountsSnapshot?: MockLike;
getSites?: MockLike;
};
type DashboardCompatApiMock = {
getDashboard?: MockLike;
getDashboardSnapshot?: MockLike;
getDashboardInsights?: MockLike;
getSiteDistribution?: MockLike;
getSiteTrend?: MockLike;
getSites?: MockLike;
getSiteSnapshot?: MockLike;
};
const FIXTURE_GENERATED_AT = '2026-04-09T00:00:00.000Z';
function buildDerivedSites(accounts: any[]): any[] {
const siteMap = new Map<number, any>();
for (const account of accounts) {
const site = account?.site;
const siteId = Number(site?.id);
if (!Number.isFinite(siteId) || siteId <= 0) continue;
if (!siteMap.has(siteId)) {
siteMap.set(siteId, { ...site });
}
}
return Array.from(siteMap.values());
}
export function installAccountsSnapshotCompat(apiMock: AccountsCompatApiMock) {
apiMock.getAccountsSnapshot?.mockImplementation?.(async () => {
const accountsResult = typeof apiMock.getAccounts === 'function'
? await apiMock.getAccounts()
: [];
const siteResult = typeof apiMock.getSites === 'function'
? await apiMock.getSites()
: [];
const accounts = Array.isArray(accountsResult) ? accountsResult : [];
const sites = Array.isArray(siteResult) && siteResult.length > 0
? siteResult
: buildDerivedSites(accounts);
return {
generatedAt: FIXTURE_GENERATED_AT,
accounts,
sites,
};
});
}
export function installDashboardSnapshotCompat(apiMock: DashboardCompatApiMock) {
apiMock.getDashboardSnapshot?.mockImplementation?.(async () => {
if (typeof apiMock.getDashboard !== 'function') return null;
return apiMock.getDashboard();
});
apiMock.getDashboardInsights?.mockImplementation?.(async () => {
if (typeof apiMock.getDashboard !== 'function') return null;
return apiMock.getDashboard();
});
apiMock.getSiteSnapshot?.mockImplementation?.(async (days = 7) => {
const distributionResult = typeof apiMock.getSiteDistribution === 'function'
? await apiMock.getSiteDistribution()
: [];
const trendResult = typeof apiMock.getSiteTrend === 'function'
? await apiMock.getSiteTrend(days)
: [];
const sitesResult = typeof apiMock.getSites === 'function'
? await apiMock.getSites()
: [];
return {
generatedAt: FIXTURE_GENERATED_AT,
distribution: Array.isArray(distributionResult?.distribution)
? distributionResult.distribution
: Array.isArray(distributionResult)
? distributionResult
: [],
trend: Array.isArray(trendResult?.trend)
? trendResult.trend
: Array.isArray(trendResult)
? trendResult
: [],
sites: Array.isArray(sitesResult) ? sitesResult : [],
};
});
}
src/web/pages/tokens.batch-actions.test.tsx
+3
View File
@@ -3,11 +3,13 @@ import { act, create } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import { TokensPanel } from './Tokens.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccountTokens: vi.fn(),
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getAccountTokenGroups: vi.fn(),
batchUpdateAccountTokens: vi.fn(),
},
@@ -27,6 +29,7 @@ async function flushMicrotasks() {
describe('Tokens batch actions', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
apiMock.getAccountTokens.mockResolvedValue([
{
id: 1,
src/web/pages/tokens.edit-and-select.test.tsx
+3
View File
@@ -5,10 +5,12 @@ import ModernSelect from '../components/ModernSelect.js';
import { ToastProvider } from '../components/Toast.js';
import Accounts from './Accounts.js';
import { TokensPanel } from './Tokens.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
getAccountTokens: vi.fn(),
getAccountTokenValue: vi.fn(),
@@ -82,6 +84,7 @@ function buildTokensRoot() {
describe('Tokens edit modal and row selection', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
Object.defineProperty(globalThis, 'navigator', {
value: {
clipboard: {
src/web/pages/tokens.focus.test.tsx
+3
View File
@@ -3,10 +3,12 @@ import { act, create, type ReactTestInstance } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import Accounts from './Accounts.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getSites: vi.fn(),
getAccountTokens: vi.fn(),
},
@@ -36,6 +38,7 @@ async function flushMicrotasks() {
describe('Tokens focus navigation', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
});
afterEach(() => {
src/web/pages/tokens.mobile-actions.test.tsx
+3
View File
@@ -3,11 +3,13 @@ import { act, create } from 'react-test-renderer';
import { MemoryRouter } from 'react-router-dom';
import { ToastProvider } from '../components/Toast.js';
import { TokensPanel } from './Tokens.js';
import { installAccountsSnapshotCompat } from './testApiCompat.js';
const { apiMock } = vi.hoisted(() => ({
apiMock: {
getAccountTokens: vi.fn(),
getAccounts: vi.fn(),
getAccountsSnapshot: vi.fn(),
getAccountTokenGroups: vi.fn(),
batchUpdateAccountTokens: vi.fn(),
},
@@ -31,6 +33,7 @@ async function flushMicrotasks() {
describe('Tokens mobile actions', () => {
beforeEach(() => {
vi.clearAllMocks();
installAccountsSnapshotCompat(apiMock);
apiMock.getAccountTokens.mockResolvedValue([
{
id: 1,