feat: merge opus comfyui and login updates
This commit is contained in:
boos
@@ -0,0 +1,78 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Per-user container lifecycle
|
||||
The system SHALL maintain at most one ComfyUI container per authenticated user. The container SHALL be started lazily on the user's first request to `/comfy/*` or `/api/video-agent/*` and SHALL be reaped after 60 minutes of inactivity.
|
||||
|
||||
#### Scenario: First request starts the container
|
||||
- **WHEN** an authenticated user with no running container makes a request to `/comfy/*`
|
||||
- **THEN** the system starts a container named `comfyui-user-<user_id>`, allocates a port from 8200–8299, records `last_used_at = now`, and forwards the request once the container's HTTP port is reachable
|
||||
|
||||
#### Scenario: Subsequent request reuses the container
|
||||
- **WHEN** the same user makes another request within 60 minutes
|
||||
- **THEN** the system reuses the existing container, updates `last_used_at = now`, and forwards the request without restarting
|
||||
|
||||
#### Scenario: Idle container is reaped
|
||||
- **WHEN** a container's `last_used_at` is older than 60 minutes AND it has no active WebSocket connections
|
||||
- **THEN** the system stops the container via the runtime's `stop` method and releases its port back to the pool
|
||||
|
||||
### Requirement: Concurrency cap
|
||||
The system SHALL refuse to start a new container when 30 are already running.
|
||||
|
||||
#### Scenario: 31st user is rejected
|
||||
- **WHEN** 30 containers are running and a 31st distinct user requests `/comfy/*`
|
||||
- **THEN** the system responds with HTTP 503 and a `Retry-After: 60` header, and does NOT evict any existing container
|
||||
|
||||
#### Scenario: Cap frees on idle reap
|
||||
- **WHEN** any of the 30 containers is reaped
|
||||
- **THEN** the next pending user request that arrives starts a fresh container in the freed slot
|
||||
|
||||
### Requirement: Resource limits
|
||||
Each container SHALL run with `--cpus=2`, `--memory=2g`, `--memory-swap=2g`, `--pids-limit=512`, `--read-only` rootfs, `--cap-drop=ALL`, and `--security-opt=no-new-privileges`.
|
||||
|
||||
#### Scenario: Container respects CPU and memory caps
|
||||
- **WHEN** a new container is started
|
||||
- **THEN** `docker inspect` shows `HostConfig.NanoCpus = 2000000000`, `HostConfig.Memory = 2147483648`, `HostConfig.MemorySwap = 2147483648`
|
||||
|
||||
#### Scenario: Container has no extra capabilities
|
||||
- **WHEN** a new container is started
|
||||
- **THEN** `docker inspect` shows `HostConfig.CapDrop` contains `ALL` and `HostConfig.SecurityOpt` contains `no-new-privileges`
|
||||
|
||||
### Requirement: Persistent per-user volumes
|
||||
Each container SHALL mount the host directory `/data/comfyui-users/<user_id>/` so that custom workflows, inputs, and outputs survive container restarts.
|
||||
|
||||
#### Scenario: Workflow saved in one session is visible after restart
|
||||
- **WHEN** a user saves a workflow file to ComfyUI's `user/` directory, then triggers a container restart (idle reap + new request)
|
||||
- **THEN** the workflow file is still present in the new container's `user/` directory
|
||||
|
||||
#### Scenario: Volume is per-user
|
||||
- **WHEN** user A saves a file and user B starts their own container
|
||||
- **THEN** user B's container does NOT see user A's file
|
||||
|
||||
### Requirement: Built-in API key injection
|
||||
Each container SHALL receive `BOOSAPI_BUILTIN_KEY`, `BOOSAPI_BASE`, and `BOOSAPI_KEY` as environment variables so ComfyUI's built-in OpenAI nodes (via the patched `_helpers.py`) and the `bosapi_comfy_nodes.py` custom nodes both authenticate successfully.
|
||||
|
||||
#### Scenario: Internal OpenAI node authenticates
|
||||
- **WHEN** a workflow uses a built-in ComfyUI OpenAI node and the container has `BOOSAPI_BUILTIN_KEY` set
|
||||
- **THEN** the node's outbound request to the BoosAPI shim includes `Authorization: Bearer <BOOSAPI_BUILTIN_KEY>`
|
||||
|
||||
#### Scenario: Custom BoosAPI node authenticates
|
||||
- **WHEN** a workflow uses `BoosAPITextGen` from `bosapi_comfy_nodes.py` and the container has `BOOSAPI_KEY` set
|
||||
- **THEN** the node's request to `BOOSAPI_BASE` includes `Authorization: Bearer <BOOSAPI_KEY>`
|
||||
|
||||
### Requirement: Container state observable to callers
|
||||
The pool SHALL expose each container's state as `starting`, `ready`, or `stopped` so the frontend can render a loading UI during cold start.
|
||||
|
||||
#### Scenario: Status reflects starting then ready
|
||||
- **WHEN** a user's first request triggers a container start
|
||||
- **THEN** `pool.state(user_id)` returns `starting` until the HTTP port is reachable, then `ready`
|
||||
|
||||
#### Scenario: Status reflects stopped after reap
|
||||
- **WHEN** a container has been idle-reaped
|
||||
- **THEN** `pool.state(user_id)` returns `stopped`
|
||||
|
||||
### Requirement: Reconcile on startup
|
||||
On API restart, the pool SHALL list all `comfyui-user-*` containers from the runtime and reconstruct its in-memory map (user_id, port, last_used_at).
|
||||
|
||||
#### Scenario: Existing containers are adopted
|
||||
- **WHEN** the API process restarts while a `comfyui-user-42` container is still running
|
||||
- **THEN** the new pool's map contains an entry for user 42 with the original container's port, and the next request from user 42 reuses it
|
||||
@@ -0,0 +1,45 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Cookie-authenticated reverse proxy at /comfy/*
|
||||
The system SHALL mount a reverse proxy at `/comfy/*` that resolves the caller's `user_id` from the existing JWT cookie and forwards both HTTP and WebSocket traffic to that user's container.
|
||||
|
||||
#### Scenario: Anonymous request is rejected
|
||||
- **WHEN** a request to `/comfy/` arrives without a valid JWT cookie
|
||||
- **THEN** the proxy responds with HTTP 401 and does NOT touch the container pool
|
||||
|
||||
#### Scenario: Authenticated HTTP request reaches the right container
|
||||
- **WHEN** user 7 requests `GET /comfy/api/object_info`
|
||||
- **THEN** the proxy forwards the request body, query, and headers to `http://127.0.0.1:<port_for_user_7>/api/object_info` and streams the response back unchanged
|
||||
|
||||
#### Scenario: Authenticated WebSocket upgrade is proxied
|
||||
- **WHEN** user 7's browser opens `wss://host/comfy/ws?clientId=abc`
|
||||
- **THEN** the proxy upgrades the connection, opens a WS to `ws://127.0.0.1:<port_for_user_7>/ws?clientId=abc`, and pipes frames in both directions until either side closes
|
||||
|
||||
### Requirement: No URL rewriting of asset paths
|
||||
The proxy SHALL forward request paths to the container with the `/comfy` prefix stripped and SHALL NOT rewrite response bodies.
|
||||
|
||||
#### Scenario: Asset path is preserved
|
||||
- **WHEN** the browser requests `/comfy/extensions/core/foo.js`
|
||||
- **THEN** the proxy forwards `/extensions/core/foo.js` to the container, and the response body is returned byte-for-byte
|
||||
|
||||
#### Scenario: ComfyUI's root-relative URLs work
|
||||
- **WHEN** the container returns HTML referencing `/scripts/app.js`
|
||||
- **THEN** the browser fetches `/comfy/scripts/app.js` (because the page was served under `/comfy/`) and the proxy resolves it correctly
|
||||
|
||||
### Requirement: Cold-start handling
|
||||
If a request arrives while the user's container is `starting`, the proxy SHALL hold the request (up to 30 s) until the container reports `ready`, instead of returning 502.
|
||||
|
||||
#### Scenario: Request waits for cold start
|
||||
- **WHEN** user 7's first request to `/comfy/` arrives and the container needs 12 s to start
|
||||
- **THEN** the proxy waits for the readiness signal and forwards the request once the container is ready, returning a normal 200
|
||||
|
||||
#### Scenario: Cold-start timeout
|
||||
- **WHEN** a container fails to become ready within 30 s
|
||||
- **THEN** the proxy responds with HTTP 504 and the pool marks the container `failed` for the next idle scan
|
||||
|
||||
### Requirement: Last-used timestamp updated on every frame
|
||||
Every HTTP request and WebSocket frame proxied to the container SHALL update `pool.containers[user_id].last_used_at = now`.
|
||||
|
||||
#### Scenario: Long-running WS keeps container alive
|
||||
- **WHEN** a user holds a WS connection open for 90 minutes while exchanging frames every minute
|
||||
- **THEN** the container is not reaped at the 60-minute mark
|
||||
@@ -0,0 +1,34 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Backend-agnostic ContainerRuntime trait
|
||||
The system SHALL expose container lifecycle behind a single `ContainerRuntime` trait so the pool, proxy, and route handlers do NOT depend on Docker directly.
|
||||
|
||||
#### Scenario: Pool uses only the trait
|
||||
- **WHEN** the source for `services/comfyui_pool.rs` is inspected
|
||||
- **THEN** it imports `ContainerRuntime` and `ContainerSpec` only — never `bollard`, `docker`, or `kube` types
|
||||
|
||||
#### Scenario: Trait surface is sufficient for v1
|
||||
- **WHEN** a `DockerRuntime` is provided
|
||||
- **THEN** the pool can start containers, stop them, check liveness, and list existing `comfyui-user-*` containers using only the trait's methods
|
||||
|
||||
### Requirement: DockerRuntime implementation
|
||||
The system SHALL ship a `DockerRuntime` that implements `ContainerRuntime` against the local Docker daemon, either via the host socket `/var/run/docker.sock` or the `docker` CLI.
|
||||
|
||||
#### Scenario: Start spawns a container
|
||||
- **WHEN** `DockerRuntime::start(spec)` is called with `image=bosapi-comfyui:multi-tenant`, `name=comfyui-user-7`, port mapping, env, and volume mounts
|
||||
- **THEN** a container with that name exists, is running, and exposes the chosen host port
|
||||
|
||||
#### Scenario: Stop removes the container
|
||||
- **WHEN** `DockerRuntime::stop("comfyui-user-7")` is called
|
||||
- **THEN** the container is stopped and removed (image used `--rm`-equivalent semantics) and its name becomes reusable
|
||||
|
||||
#### Scenario: List finds existing user containers
|
||||
- **WHEN** `DockerRuntime::list_user_containers()` is called
|
||||
- **THEN** it returns one entry per running container whose name starts with `comfyui-user-`, including its port mapping
|
||||
|
||||
### Requirement: K3S extension point reserved
|
||||
The trait SHALL be designed such that a future `K3sRuntime` implementing the same surface can replace `DockerRuntime` without changes to the pool, proxy, or route handlers.
|
||||
|
||||
#### Scenario: Swapping runtimes does not break callers
|
||||
- **WHEN** `AppState::comfyui_pool` is constructed with a `Box<dyn ContainerRuntime>` instead of a concrete `DockerRuntime`
|
||||
- **THEN** all callers compile and behavior is unchanged for the `DockerRuntime` instance
|
||||
@@ -0,0 +1,41 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: video-agent handlers route through the pool
|
||||
Every handler under `routes/video_agent.rs` SHALL resolve the caller's `user_id` from the JWT cookie and operate on `state.comfyui_pool.get_or_start(user_id)` instead of the global `state.comfyui` singleton. The wire format of existing endpoints SHALL NOT change.
|
||||
|
||||
#### Scenario: status reflects per-user container
|
||||
- **WHEN** user 7 calls `GET /api/video-agent/status`
|
||||
- **THEN** the response describes user 7's container only, including a new `containerState: "starting" | "ready" | "stopped"` field, and the pre-existing JSON keys are preserved
|
||||
|
||||
#### Scenario: start is idempotent per user
|
||||
- **WHEN** user 7 calls `POST /api/video-agent/start` twice in a row
|
||||
- **THEN** the first call starts (or reuses) the container and returns 200, the second call returns 200 immediately without spawning a duplicate container
|
||||
|
||||
#### Scenario: stop only stops the caller's container
|
||||
- **WHEN** user 7 calls `POST /api/video-agent/stop`
|
||||
- **THEN** user 7's container is stopped and removed, and any other user's running container is unaffected
|
||||
|
||||
#### Scenario: chat / analyze / generate-* call the caller's container
|
||||
- **WHEN** user 7 calls any of `/api/video-agent/{chat,analyze,character-prompts,generate-character,scene-prompt,generate-scene,generate-tts,full-workflow}`
|
||||
- **THEN** the handler forwards the request to user 7's container's HTTP endpoint and returns the response
|
||||
|
||||
### Requirement: Backwards-compatible response shape
|
||||
Existing JSON keys returned by `/api/video-agent/*` SHALL remain present and identical to the singleton-era response. Only additive fields are allowed.
|
||||
|
||||
#### Scenario: status payload is a superset
|
||||
- **WHEN** the frontend (unchanged) parses the response of `/api/video-agent/status`
|
||||
- **THEN** every field it currently reads is still present with the same type and meaning; only `containerState` (and any new optional fields) are added
|
||||
|
||||
### Requirement: Authentication required
|
||||
All `/api/video-agent/*` routes SHALL require a valid JWT cookie. Anonymous requests SHALL return HTTP 401 without touching the pool.
|
||||
|
||||
#### Scenario: Anonymous request blocked
|
||||
- **WHEN** an unauthenticated client calls `POST /api/video-agent/start`
|
||||
- **THEN** the response is HTTP 401 and no container is started
|
||||
|
||||
### Requirement: Removal of global state.comfyui
|
||||
The `comfyui: Arc<ComfyUIProcess>` field on `AppState` SHALL be removed once `comfyui_pool` is the sole code path. No call site SHALL retain a reference to the singleton.
|
||||
|
||||
#### Scenario: Codebase contains no singleton refs
|
||||
- **WHEN** the source tree is grepped for `state.comfyui` (without the `_pool` suffix)
|
||||
- **THEN** there are zero matches outside of historical comments
|
||||
Reference in New Issue
Block a user