feat: add desktop agent workbench
This commit is contained in:
boos
@@ -0,0 +1 @@
|
||||
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
|
||||
@@ -0,0 +1,44 @@
|
||||
# Desktop Agent Workbench Integration Design
|
||||
|
||||
## Scope
|
||||
|
||||
This deliverable adds the desktop UI shell and local interactions for Agent Workbench, and connects Agent mode to the existing desktop Agent command path. It intentionally avoids new Rust commands, database tables, billing, and remote media generation until the page contract is stable.
|
||||
|
||||
## Frontend Boundaries
|
||||
|
||||
- `types/index.ts` owns the `PageKey` addition.
|
||||
- `components/layout/Sidebar.tsx` owns navigation entry placement.
|
||||
- `components/layout/ContentArea.tsx` owns routing from `PageKey` to page component.
|
||||
- `pages/AgentWorkbenchPage.tsx` owns the creator composer, local attachment state, mode state, model selection, Agent handoff, and placeholder result/status for media modes.
|
||||
- `stores/providerStore.ts` remains the source of provider/model data. The page reads `providers` and calls `fetchProviders`; it does not duplicate provider fetching logic.
|
||||
- `stores/conversationStore.ts` remains the source for conversation creation and Agent streaming. The Workbench reuses `createConversation`, `updateConversation`, and `sendAgentMessage`.
|
||||
- `stores/skillStore.ts` remains the source for skill installation/listing. The Workbench reuses `loadSkills`, `installSkill`, and `toggleSkill` for third-party skills.
|
||||
- `lib/browserMock.ts` mirrors skill install/list/toggle enough for browser preview.
|
||||
- `i18n/locales/*.json` owns navigation labels. The first page UI can use Chinese literal copy to match current product direction, but nav should use locale keys.
|
||||
|
||||
## Data Flow
|
||||
|
||||
Provider store -> enabled providers -> enabled chat models -> AgentWorkbenchPage model options -> selected provider/model or custom model fallback.
|
||||
|
||||
File input -> browser `File` objects -> base64 `AttachmentInput` payload -> local attachment chips -> existing Agent attachment persistence during `agent_query`.
|
||||
|
||||
Mode selector -> local `creationMode` -> mode label/model controls/result copy.
|
||||
|
||||
Agent submit -> create conversation -> set `mode: "agent"` -> upsert Agent session via `agent_update_session` -> `sendAgentMessage` -> Chat page streaming display.
|
||||
|
||||
Non-Agent submit -> local result/status card. No external side effects yet.
|
||||
|
||||
Skills menu -> detect `jianying-editor` -> if missing install `luoluoluo22/jianying-editor-skill` via existing `install_skill` command -> if disabled enable it -> insert `请使用 jianying-editor Skill 自动创建剪映草稿：` into the prompt -> Agent submit loads enabled skills in the backend and can invoke the skill tool.
|
||||
|
||||
## Compatibility
|
||||
|
||||
Existing pages and settings should be unaffected. Adding a `PageKey` requires exhaustive switch and sidebar updates in the same change.
|
||||
|
||||
## Follow-up Hooks
|
||||
|
||||
Future work can replace the non-Agent local submit handler with:
|
||||
|
||||
- BoosAPI `/api/agent/models` for remote model catalog.
|
||||
- BoosAPI `/api/jimeng/*` for image/video generation.
|
||||
- Tauri file staging command for persistent uploads.
|
||||
- A dedicated JianYing setup/status panel for Python dependency checks, JianYing draft path detection, and platform/version warnings.
|
||||
@@ -0,0 +1 @@
|
||||
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
|
||||
@@ -0,0 +1,23 @@
|
||||
# Implementation Plan
|
||||
|
||||
1. Add `agent-workbench` to desktop page types, sidebar nav, content routing, and nav locale files.
|
||||
2. Create `AgentWorkbenchPage.tsx` with:
|
||||
- mode definitions
|
||||
- provider-derived model options
|
||||
- custom model fallback
|
||||
- local attachment selection/removal with `AttachmentInput` conversion
|
||||
- mode dropdown, auto toggle, skills and subject popovers
|
||||
- Agent mode submit that creates an Agent conversation/session and calls existing streaming flow
|
||||
- skills menu action for `luoluoluo22/jianying-editor-skill` install/enable and prompt priming
|
||||
- placeholder submit status/result for media modes
|
||||
3. Extend browser preview mock skill commands so install/list/toggle reflects the JianYing skill integration.
|
||||
4. Add focused CSS to `desktop/src/index.css`, using restrained desktop styling and avoiding global side effects.
|
||||
5. Run validation:
|
||||
- `bun run typecheck`
|
||||
- `bun run build`
|
||||
6. Review diff for unrelated churn before final report.
|
||||
|
||||
## Validation
|
||||
|
||||
- `bun run typecheck` passed in `/root/metapi/desktop` after JianYing skill integration.
|
||||
- `bun run build` passed in `/root/metapi/desktop` after JianYing skill integration.
|
||||
@@ -0,0 +1,34 @@
|
||||
# Desktop Agent Workbench integration
|
||||
|
||||
## Goal
|
||||
|
||||
Add a first usable Agent Workbench entry to the desktop client so users can start from the same creator-style surface used in the BoosAPI web UI: choose a creation mode, select a model, attach references, and type a prompt.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Add an Agent Workbench page to the desktop navigation without disrupting existing Chat, Drawing, Skills, Knowledge, Memory, Gateway, Files, or Settings flows.
|
||||
- The first screen should match the creator-composer direction: centered prompt, large input surface, upload affordance, creation mode selector, auto toggle, model selector, skills button, subject button, and submit button.
|
||||
- Creation modes must include at least Agent, image generation, video generation, music generation, voice generation, digital human, and action imitation. Unsupported modes may show “coming soon” copy but must remain selectable.
|
||||
- Agent mode should expose a conversation model selector populated from existing enabled desktop provider models where possible, with a custom model fallback.
|
||||
- Upload affordance should accept local reference files, display removable attachment chips, and pass attachments into existing Agent chat flow when Agent mode submits.
|
||||
- The page must follow the desktop app’s existing React/Ant Design/Zustand conventions and be reachable from the sidebar.
|
||||
- The implementation must avoid new backend schema. Existing Tauri commands may be reused.
|
||||
- The Workbench skills menu should expose a JianYing/剪映 automation entry that installs/enables `luoluoluo22/jianying-editor-skill` through the existing Desktop skills system and primes the Agent prompt.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] Sidebar contains an Agent Workbench entry and selecting it renders the new page.
|
||||
- [x] Agent Workbench page has a centered creator composer with mode dropdown, model control, upload button, `@` subject popover, and submit button.
|
||||
- [x] Available chat models are derived from existing enabled desktop providers/models, with a custom model option.
|
||||
- [x] Selecting a mode updates the composer label and mode-specific model controls where applicable.
|
||||
- [x] Local file selection displays attachment chips and supports removing them.
|
||||
- [x] Agent mode creates an Agent conversation/session and sends the prompt plus attachments through the existing Agent stream flow.
|
||||
- [x] Skills menu can install/enable the JianYing editor skill and insert a matching Agent prompt prefix.
|
||||
- [x] Unsupported submit paths do not crash; they show a clear in-page result/status.
|
||||
- [x] `bun run typecheck` and `bun run build` pass in `/root/metapi/desktop`.
|
||||
|
||||
## Notes
|
||||
|
||||
- Keep `prd.md` focused on requirements, constraints, and acceptance criteria.
|
||||
- Lightweight tasks can remain PRD-only.
|
||||
- For complex tasks, add `design.md` for technical design and `implement.md` for execution planning before `task.py start`.
|
||||
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"id": "desktop-agent-workbench",
|
||||
"name": "desktop-agent-workbench",
|
||||
"title": "Desktop Agent Workbench integration",
|
||||
"description": "",
|
||||
"status": "in_progress",
|
||||
"dev_type": null,
|
||||
"scope": null,
|
||||
"package": "vendor/ComfyUI_frontend",
|
||||
"priority": "P2",
|
||||
"creator": "boos4721",
|
||||
"assignee": "boos4721",
|
||||
"createdAt": "2026-06-25",
|
||||
"completedAt": null,
|
||||
"branch": null,
|
||||
"base_branch": "feat/embed-desktop",
|
||||
"worktree_path": null,
|
||||
"commit": null,
|
||||
"pr_url": null,
|
||||
"subtasks": [],
|
||||
"children": [],
|
||||
"parent": null,
|
||||
"relatedFiles": [],
|
||||
"notes": "",
|
||||
"meta": {}
|
||||
}
|
||||
Reference in New Issue
Block a user