Commit Graph

117 Commits

Author SHA1 Message Date
iven
5c48d62f7e fix(saas): harden model group failover + relay reliability
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- cache: insert-then-retain pattern avoids empty-window race during refresh
- relay: manage_task_status flag for proper failover state transitions
- relay: retry_task re-resolves model groups instead of blind provider reuse
- relay: filter empty-member groups from available models list
- relay: quota cache stale entry cleanup (TTL 5x expiry)
- error: from_sqlx_unique helper for 409 vs 500 distinction
- model_config: unique constraint handling, duplicate member check
- model_config: failover_strategy whitelist, model_id vs group name conflict check
- model_config: group-scoped member removal with group_id validation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 12:26:55 +08:00
iven
894c0d7b15 feat(desktop): pipeline result preview + industry templates + onboarding auto-trigger
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
Sprint 2: 产品体验打磨 + 行业模板

- Create PipelineResultPreview component with tab-based output switching
- Connect workflow/hand messages to PresentationContainer in ChatArea
- Add auto-trigger first Hand after onboarding (industry-specific queries)
- Seed 3 industry agent templates (education, healthcare, design-shantou)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 10:48:47 +08:00
iven
be0a78a523 feat(saas): add model groups for cross-provider failover
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
Model Groups provide logical model names that map to multiple physical
models across providers, with automatic failover when one provider's
key pool is exhausted.

Backend:
- New model_groups + model_group_members tables with FK constraints
- Full CRUD API (7 endpoints) with admin-only write permissions
- Cache layer: DashMap-backed CachedModelGroup with load_from_db
- Relay integration: ModelResolution enum for Direct/Group routing
- Cross-provider failover: sort_candidates_by_quota + OnceLock cache
- Relay failure path: record failure usage + relay_dequeue (fixes
  queue counter leak that caused connection pool exhaustion)
- add_group_member: validate model_id exists before insert

Frontend:
- saas-relay-client: accept getModel() callback for dynamic model selection
- connectionStore: prefer conversationStore.currentModel over first available

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 09:56:21 +08:00
iven
9af7b0dd46 fix(kernel): enable multi-agent compilation + A2A routing tests
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- director.rs: add missing CompletionRequest fields (thinking_enabled,
  reasoning_effort, plan_mode) for multi-agent feature gate
- agents.rs: remove unused AgentState import behind multi-agent feature
- lib.rs: replace ambiguous glob re-export with explicit director types,
  resolving AgentRole conflict between director and generation modules
- a2a.rs: add 5 integration tests covering direct message delivery,
  broadcast routing, group messaging, agent unregistration, and
  expired message rejection (10 total A2A tests, all passing)
- Verified: 537 workspace tests pass with multi-agent feature enabled

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 09:41:24 +08:00
iven
f4ed1b33e0 feat(kernel): add multi-skill orchestration bridge + true parallel execution
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- Kernel orchestration bridge: execute_orchestration, auto_compose_skills,
  validate_orchestration methods on Kernel struct
- True parallel execution: replace sequential for-loop with tokio::JoinSet
  for concurrent node execution within parallel groups
- Tauri commands: orchestration_execute (auto-compose or pre-defined graph),
  orchestration_validate (dry-run validation)
- Full type conversions: OrchestrationRequest/Response with camelCase serde

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 09:18:26 +08:00
iven
1399054547 feat(skills): add LLM fallback routing + CJK TF-IDF bigram fix
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- SemanticSkillRouter: add RuntimeLlmIntent trait and with_llm_fallback() builder
- route(): call LLM fallback when TF-IDF/embedding confidence < threshold
- CJK tokenization: generate bigrams for Chinese/Japanese/Korean text
- Fix: previous tokenizer treated entire CJK string as one huge token

- SemanticSkillRouter: add RuntimeLlmIntent trait and with_llm_fallback() builder
- route(): call LLM fallback when TF-IDF/embedding confidence < threshold
- CJK tokenization: generate bigrams for Chinese/Japanese/Korean text
- Fix: previous tokenizer treated entire CJK string as one huge token

- LlmSkillFallback: concrete RuntimeLlmIntent using LlmDriver
- Asks LLM to pick best skill from ambiguous candidates list
- Parses structured JSON response from LLM output
- Includes tests for LLM fallback and CJK tokenization

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 07:44:42 +08:00
iven
0be31bbf7e feat(memory): add message pagination support
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- get_messages_paginated(session_id, limit, offset) for batch loading
- count_messages(session_id) for total count queries
- Enables frontend to load messages progressively instead of all-at-once
2026-04-04 01:22:03 +08:00
iven
b25dfc967a feat(kernel): persist agent runtime state across restarts
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- Schema: migrations now execute ALTER TABLE ADD COLUMN for state/message_count
- MemoryStore: add update_agent_runtime() and list_agents_with_runtime()
- Registry: add register_with_runtime() to accept persisted state/message_count
- Kernel boot: restore agents with their persisted state (not always Running)
- Kernel shutdown: persist all agent states/message_counts before terminating

Agents that were suspended stay suspended after restart. Message counts
survive restarts instead of resetting to 0.
2026-04-04 01:19:53 +08:00
iven
b4e5af7a58 feat(growth): add memory decay + time-weighted scoring + remove dead frontend
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- Add effective_importance() with exponential time decay (30-day half-life)
  and access count boost for fair scoring of stale vs fresh memories
- Add SqliteStorage::decay_memories() for periodic maintenance:
  reduces stored importance per interval, archives (deletes) below threshold
- Update find() scoring to use time-decayed importance in sort
- Add DecayResult type and effective_importance re-export in lib.rs
- Remove dead frontend active-learning.ts (370 lines, zero imports)
2026-04-04 00:45:16 +08:00
iven
8faefd6a61 fix(tests): resolve workspace compilation + CJK search failures
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- saas test harness: align WorkerDispatcher::new and AppState::new
  signatures with SpawnLimiter addition and init_db(&DatabaseConfig)
- growth sqlite: add CJK fallback (LIKE-based) when FTS5 unicode61
  tokenizer fails on Chinese queries (unicode61 doesn't index CJK)
2026-04-04 00:34:34 +08:00
iven
5eeabd1f30 feat(saas): add webhook event notification system (@unplugged)
Webhook infrastructure for external event notifications:
- SQL migration: webhook_subscriptions + webhook_deliveries tables
- Types: CreateWebhookRequest, UpdateWebhookRequest, WebhookDelivery
- Service: CRUD operations + trigger_webhooks + HMAC-SHA256 signing
- Handlers: REST API endpoints (CRUD + delivery logs)
- Worker: WebhookDeliveryWorker with exponential retry (max 3)

NOT YET INTEGRATED: needs mod registration in lib.rs + workers/mod.rs,
hmac crate dependency, and route mounting. Code is ready for future
integration after stabilization phase completes.
2026-04-03 23:01:49 +08:00
iven
943afe3b6b feat(protocols): MCP tool adapter + Tauri commands + initialize bug fix
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
S6 MCP Protocol:
- Fix McpTransport::initialize() — store actual server capabilities instead
  of discarding them and storing empty ServerCapabilities::default()
- Add send_notification() method to McpTransport for JSON-RPC notifications
- Send notifications/initialized after MCP handshake (spec requirement)
- Add McpToolAdapter: bridges MCP server tools into the tool execution path
- Add McpServiceManager: lifecycle management for MCP server connections
- Add 4 Tauri commands: mcp_start_service, mcp_stop_service,
  mcp_list_services, mcp_call_tool
- Register zclaw-protocols dependency in desktop Cargo.toml

New files:
- crates/zclaw-protocols/src/mcp_tool_adapter.rs (153 lines)
- desktop/src-tauri/src/kernel_commands/mcp.rs (145 lines)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 22:07:35 +08:00
iven
cc26797faf fix(saas): eliminate 6 compiler warnings + stabilize directive complete
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- Remove unused imports: Utc (billing/service), StatusCode (billing/handlers), Sha256 (billing/handlers)
- Fix unused variables: _db (scheduler), _e (payment WeChat error)
- Fix visibility: RegisterDeviceRequest pub(super) → pub (used in pub handler)
- Update STABILIZATION_DIRECTIVE.md: all 7 criteria met, downgrade to advisory
- Fix TRUTH.md §2.2: mark P0/P1 defects as resolved, update Admin pages count to 14

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 21:57:04 +08:00
iven
264dc75b2c fix(production-readiness): audit fixes — duplicate useState + route mismatch + stale @reserved
Some checks failed
CI / Build Frontend (push) Has been cancelled
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- ChatArea.tsx: remove duplicate useState(searchOpen) declaration on line 70
- scheduled_task/mod.rs: fix route from /api/scheduler/tasks to /api/v1/scheduler/tasks
  (matches admin-v2 service baseURL pattern and all other modules)
- scheduled_task/handlers.rs: remove @reserved annotations (now has Admin V2 frontend)
- scheduled_task/handlers.rs: update doc comments with correct /api/v1/ paths
2026-04-03 21:41:30 +08:00
iven
2ceeeaba3d fix(production-readiness): 3-batch production readiness cleanup — 12 tasks
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
Batch 1 — User-facing fixes:
- B1-1: Pipeline verified end-to-end (14 Rust commands, 8 frontend invoke, fully connected)
- B1-2: MessageSearch restored to ChatArea with search button in DeerFlow header
- B1-3: Viking cleanup — removed 5 orphan invokes (no Rust impl), added addWithMetadata + storeWithSummaries methods + summary generation UI
- B1-4: api-fallbacks transparency — added _isFallback markers + console.warn to all 6 fallback functions

Batch 2 — System health:
- B2-1: Document drift calibration — TRUTH.md/README.md numbers verified and updated
- B2-2: @reserved annotations on 15 SaaS handler functions with no frontend callers
- B2-3: Scheduled Task Admin V2 — new service + page + route + sidebar navigation
- B2-4: TRUTH.md Pipeline/Viking/ScheduledTask records corrected

Batch 3 — Long-term quality:
- B3-1: hand_run_status/hand_run_list verified as fully implemented (not stubs)
- B3-2: Identity snapshot rollback UI added to RightPanel
- B3-3: P2 code quality — 4 fixes (TODO comments, fire-and-forget notes, design notes, table name validation), 2 verified N/A, 1 upstream
- B3-4: Config PATCH→PUT alignment (admin-v2 config.ts matched to SaaS backend)
2026-04-03 21:34:56 +08:00
iven
305984c982 fix(saas): P2 code quality fixes + config PATCH/PUT alignment
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
P2 code quality (SEC2-P2-01~10):
- P2-04: Replace vague TODO with detailed Phase 2 design note in generate_embedding.rs
- P2-05: Add NOTE(fire-and-forget) annotations to 4 long-running tokio::spawn in main.rs
- P2-07: Add DESIGN NOTE to scheduler explaining sequential execution rationale
- P2-08: Add compile-time table name whitelist + runtime char validation in db.rs
- P2-02: Verified N/A (only zclaw-pipeline uses serde_yaml_bw, no inconsistency)
- P2-06: Verified N/A (bind loop correctly matches 6-column placeholders)
- P2-03: Remains OPEN (requires upstream sqlx release)

Config HTTP method alignment (B3-4):
- Fix admin-v2 config.ts: request.patch -> request.put to match backend .put() route
- Fix backend handler doc comment: PATCH -> PUT
- Add @reserved annotations to 6 config handlers without frontend callers
2026-04-03 21:32:17 +08:00
iven
edecd4c81f fix(saas): deep audit round industry template system - critical fixes
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
C1: Use backend createAgentFromTemplate API + tools forwarding
C3: seed source='builtin' instead of 'custom'
C4: immutable clone data handling (return fresh from store) + spread)
H3: assignTemplate error propagation (try/catch)
H4: input validation for name/fields
H5: assign_template account existence check
H6: remove dead route get_full_template
H7: model fallback gpt-4o-mini (hardcoded constant)
H8: logout clears template state
H9: console.warn -> structured logger
C2: restoreSession fetches assignedTemplate
2026-04-03 19:45:25 +08:00
iven
0857a1f608 feat(desktop): wire template welcome_message + quick_commands to chat UI
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- Add welcomeMessage/quickCommands fields to Clone interface
- Persist template welcome/quick data via updateClone after creation
- FirstConversationPrompt: prefer template-provided welcome message
  over dynamically generated one
- FirstConversationPrompt: render template quick_commands as chips
  instead of hardcoded QUICK_ACTIONS when available
- Tighten assign/unassign template endpoint permissions from model:read
  to relay:use (self-service operation for all authenticated users)
2026-04-03 15:20:15 +08:00
iven
1048901665 fix(saas): industry template audit fixes + pgvector optional + relay timeout
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- Fix seed template tools to match actual runtime tool names
  (file_read/file_write/shell_exec/web_fetch)
- Persist system_prompt/temperature/max_tokens via identity system
  in agentStore.createFromTemplate()
- Fire-and-forget assignTemplate() in AgentOnboardingWizard
- Fix saas-relay-client unused variable warning
- Make pgvector extension optional in knowledge_base migration
- Increase StreamBridge timeout from 30s to 90s for thinking models
2026-04-03 15:10:13 +08:00
iven
ea00c32c08 feat(saas): industry agent template assignment system
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
Phase 1-8 of industry-agent-delivery plan:

- DB migration: accounts.assigned_template_id (ON DELETE SET NULL)
- SaaS API: 4 new endpoints (assign/get/unassign/create-agent)
- Service layer: assign_template_to_account, get_assigned_template, unassign_template, create_agent_from_template)
- Types: AssignTemplateRequest, AgentConfigFromTemplate (capabilities merged into tools)
- Frontend SaaS Client: assignTemplate, getAssignedTemplate, unassignTemplate, createAgentFromTemplate
- saasStore: assignedTemplate state + login auto-fetch + actions
- saas-relay-client: fix unused import and saasUrl reference error
- connectionStore: fix relayModel undefined error
- capabilities default to glm-4-flash

- Route registration: new template assignment routes

Cospec and handlers consolidated

Build: cargo check --workspace PASS, tsc --noEmit Pass
2026-04-03 13:31:58 +08:00
iven
52bdafa633 refactor(crates): kernel/generation module split + DeerFlow optimizations + middleware + dead code cleanup
- Split zclaw-kernel/kernel.rs (1486 lines) into 9 domain modules
- Split zclaw-kernel/generation.rs (1080 lines) into 3 modules
- Add DeerFlow-inspired middleware: DanglingTool, SubagentLimit, ToolError, ToolOutputGuard
- Add PromptBuilder for structured system prompt assembly
- Add FactStore (zclaw-memory) for persistent fact extraction
- Add task builtin tool for agent task management
- Driver improvements: Anthropic/OpenAI extended thinking, Gemini safety settings
- Replace let _ = with proper log::warn! across SaaS handlers
- Remove unused dependency (url) from zclaw-hands
2026-04-03 00:28:03 +08:00
iven
da438ad868 fix(billing): resolve all audit findings — CSRF, float precision, TOCTOU, error sanitization
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
- Add CSRF token protection for mock payment (SHA256 + constant-time verify)
- Replace f64 currency conversion with pure integer string parsing (parse_yuan_to_cents)
- Move subscription check inside transaction to prevent TOCTOU race
- Rewrite increment_usage to use atomic SQL (account_id+period_start WHERE)
- Add trade_no format validation in payment callback
- Sanitize error messages to prevent sensitive data leakage
- Use i32::try_from for WeChat amount conversion (prevent truncation)
- Replace window.__ZCLAW_STATS_SYNC_INTERVAL__ with useRef pattern
- Replace eprintln/println with tracing macros in lifecycle
- Remove unused variable in scheduler
- Remove duplicate sha2 and unused hmac from Cargo.toml
2026-04-02 20:04:43 +08:00
iven
28299807b6 fix(desktop): DeerFlow UI — ChatArea refactor + ai-elements + dead CSS cleanup
ChatArea retry button uses setInput instead of direct sendToGateway,
fix bootstrap spinner stuck for non-logged-in users,
remove dead CSS (aurora-title/sidebar-open/quick-action-chips),
add ai components (ReasoningBlock/StreamingText/ChatMode/ModelSelector/TaskProgress),
add ClassroomPlayer + ResizableChatLayout + artifact panel

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 19:24:44 +08:00
iven
d40c4605b2 fix(knowledge): verification audit — 3 medium issues
- create_item: wrap item + version INSERT in transaction for atomicity
- update_item handler: validate content length (100KB) before DB hit
- KnowledgeChunk: document missing embedding field, safe per explicit SELECT usage
2026-04-02 19:16:32 +08:00
iven
7e4b787d5c fix(knowledge): deep audit — 18 bugs fixed across backend + frontend
CRITICAL:
- Migration permission seed WHERE name → WHERE id (matched 0 rows, all KB APIs broken)

HIGH:
- analytics_quality SQL alias + missing comma fix
- search() duplicate else block compile error
- chunk_content duplicate var declarations + type mismatch
- SQL invalid escape sequences
- delete_category missing rows_affected check

MEDIUM:
- analytics_overview hit_rate vs positive_feedback_rate separation
- analytics_quality GROUP BY kc.id,kc.name (same-name category merge)
- update_category handler trim + empty name validation
- update_item duplicate VALID_STATUSES inside transaction
- page_size max(1) lower bound in list handlers
- batch_create title/content/length validation
- embedding dispatch silent error → tracing::warn
- Version modal close clears detailItem state
- Search empty state distinguishes not-searched vs no-results
- Create modal cancel resets form
2026-04-02 19:07:42 +08:00
iven
837abec48a feat(billing): add usage increment API + wire hand/pipeline execution tracking
Server side:
- POST /api/v1/billing/usage/increment endpoint with dimension whitelist
  (hand_executions, pipeline_runs, relay_requests) and count validation (1-100)
- Returns updated usage quota after increment

Desktop side:
- New saas-billing.ts mixin with incrementUsageDimension() and
  reportUsageFireAndForget() (non-blocking, safe for finally blocks)
- handStore.triggerHand: reports hand_executions after successful run
- PipelinesPanel.handleRunComplete: reports pipeline_runs on completion
- SaaSClient type declarations for new billing methods

Billing pipeline now covers all three dimensions:
  relay_requests  → relay handler (server-side, real-time)
  hand_executions → handStore (client-side, fire-and-forget)
  pipeline_runs   → PipelinesPanel (client-side, fire-and-forget)
2026-04-02 02:02:59 +08:00
iven
11e3d37468 feat(billing): activate real-time quota enforcement pipeline
- Wire relay handler to increment_usage() for JSON responses (tokens + relay_requests)
- Wire relay handler to increment_dimension("relay_requests") for SSE streams
- Add increment_dimension() function for hand_executions/pipeline_runs dimensions
- Schedule AggregateUsageWorker hourly for reconciliation (run_on_start=true)
- Mount mock payment routes in dev mode (ZCLAW_SAAS_DEV=true)

Previously the quota middleware always allowed requests because usage
counters were never incremented. Now relay requests update billing_usage_quotas
in real-time, with the aggregator providing hourly reconciliation.
2026-04-02 01:52:01 +08:00
iven
08268b32b8 feat(memory): implement FactStore SQLite persistence
Add `facts` table to schema with columns for id, agent_id, content,
category, confidence, source_session, and created_at. Implement
store_facts() and get_top_facts() on MemoryStore using upsert-by-id
and confidence-desc ordering. Facts extracted from conversations are
now durable across sessions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 01:26:57 +08:00
iven
1bf0d3a73d fix(memory): CJK-aware short query threshold + Chinese synonym expansion
1. MemoryMiddleware: replace byte-length check (query.len() < 4) with
   char-count check (query.chars().count() < 2). Single CJK characters
   are 3 UTF-8 bytes but 1 meaningful character — the old threshold
   incorrectly skipped 1-2 char Chinese queries like "你好".

2. QueryAnalyzer: add Chinese synonym mappings for 13 common technical
   terms (错误→bug, 优化→improve, 配置→config, etc.) so CJK queries
   can find relevant English-keyword memories and vice versa.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 01:21:29 +08:00
iven
07099e3ef0 test(hands): expand Slideshow tests (4→34) and fix Clip invalid action test
Slideshow: add navigation edge cases, autoplay/pause/resume, spotlight/
laser/highlight defaults, content block deserialization, Hand trait
dispatch, and add_slide helper tests.

Clip: fix test_execute_invalid_action to expect Err (execute returns
HandError for unknown variants).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 01:13:15 +08:00
iven
dce9035584 test(hands): add 28 unit tests for Twitter Hand
Cover config defaults, 13 action types deserialization, serialization
roundtrip, credential management, and data type parsing. Also add
PartialEq derive to HandStatus for test assertions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 01:01:37 +08:00
iven
b1e3a27043 feat(saas): add payment integration with Alipay/WeChat mock support
- payment.rs: create_payment, handle_payment_callback, query_payment_status
- Mock pay page for development mode with HTML confirm/cancel flow
- Payment callback handler with subscription auto-creation on success
- Alipay form-urlencoded and WeChat JSON callback parsing
- 7 new routes including callback and mock-pay endpoints

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 00:41:35 +08:00
iven
830e9fa301 feat(saas): add GenerateEmbedding worker for knowledge chunking
- Markdown-aware content splitting (512 token chunks with 64 overlap)
- CJK keyword extraction from chunk content with stop-word filtering
- Full refresh strategy (delete old chunks → re-insert on update)
- Phase 2 placeholder for vector embedding API integration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 00:23:38 +08:00
iven
ef60f9a183 feat(saas): add knowledge base module — categories, items, versions, search, analytics
- 5 knowledge tables (categories, items, chunks, versions, usage) with pgvector + HNSW + GIN indexes
- 23+ API routes covering full CRUD, tree-structured categories, version snapshots
- Keyword-based search with ILIKE + array match (placeholder for vector search)
- Analytics endpoints: overview, trends, top-items, quality, gaps
- Markdown-aware content chunking with overlap strategy
- Worker dispatch for async embedding generation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 00:21:28 +08:00
iven
b66087de0e feat(saas): add quota middleware and usage aggregation worker
B1.3 Quota middleware:
- quota_check_middleware for relay route chain
- Checks monthly relay_requests quota before processing
- Gracefully degrades on billing service failure

B1.5 AggregateUsageWorker:
- Aggregates usage_records into billing_usage_quotas monthly
- Supports single-account and all-accounts modes
- Scheduled hourly via Worker dispatcher (6 workers total)
2026-04-02 00:06:39 +08:00
iven
d06ecded34 feat(saas): add quota check middleware for relay requests
Injects billing quota verification before relay chat completion requests.
Checks monthly relay_requests quota via billing::service::check_quota.
Gracefully degrades on quota service failure (logs warning, allows request).
2026-04-02 00:03:26 +08:00
iven
9487cd7f72 feat(saas): add billing infrastructure — tables, types, service, handlers
B1.1 Billing database:
- 5 tables: billing_plans, billing_subscriptions, billing_invoices,
  billing_payments, billing_usage_quotas
- Seed data: Free(¥0)/Pro(¥49)/Team(¥199) plans
- JSONB limits for flexible plan configuration

Billing module (crates/zclaw-saas/src/billing/):
- types.rs: BillingPlan, Subscription, Invoice, Payment, UsageQuota
- service.rs: plan CRUD, subscription lookup, usage tracking, quota check
- handlers.rs: REST API (plans list/detail, subscription, usage)
- mod.rs: routes registered at /api/v1/billing/*

Cargo.toml: added chrono feature to sqlx for DateTime<Utc> support
2026-04-01 23:59:46 +08:00
iven
17a2501808 test(hands): add unit tests for BrowserHand + fix requires_approval config
Fix needs_approval field in BrowserHand::new() from false to true to
match the TOML config (hands/browser.HAND.toml says requires_approval = true).
Browser automation has security implications and should require approval.

Add 11 unit tests covering:
- Config id and enabled state
- needs_approval correctness (after fix)
- Action deserialization (Navigate, Click, Type, Scrape, Screenshot)
- Roundtrip serialization for all major action variants
- BrowserSequence builder with stop_on_error()
- Multi-step sequence execution
- FormField deserialization

Also add stop_on_error() builder method to BrowserSequence which was
referenced in the test plan but missing from the struct.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-01 23:22:18 +08:00
iven
cc7ee3189d test(hands): add unit tests for CollectorHand + fix HTML extraction position tracking
Fix extract_visible_text to use proper byte position tracking (pos += char_len)
instead of iterating chars without position context, which caused script/style
tag detection to fail on multi-byte content. Also adds script/style stripping
logic and raises truncation limit to 10000 chars.

Adds 9 unit tests covering:
- Config identity verification
- OutputFormat serialization round-trip
- HTML text extraction (basic, script stripping, style stripping, empty input)
- Aggregate action with empty URLs
- CollectorAction deserialization (Collect/Aggregate/Extract)
- CollectionTarget deserialization
2026-04-01 23:21:43 +08:00
iven
59fc7debd6 feat(hands): add 25 unit tests + fix summary + fix HTML extraction for ResearcherHand
- Add comprehensive test suite: config, types, action deserialization, URL encoding,
  HTML text extraction, hand trait methods
- Fix summary field: generate rule-based summary from top search results (was always None)
- Fix extract_text_from_html: correct position tracking for script/style tag detection

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-01 23:16:57 +08:00
iven
e3b93ff96d fix(security): implement all 15 security fixes from penetration test V1
Security audit (2026-03-31): 5 HIGH + 10 MEDIUM issues, all fixed.

HIGH:
- H1: JWT password_version mechanism (pwv in Claims, middleware verification,
  auto-increment on password change)
- H2: Docker saas port bound to 127.0.0.1
- H3: TOTP encryption key decoupled from JWT secret (production bailout)
- H4+H5: Tauri CSP hardened (removed unsafe-inline, restricted connect-src)

MEDIUM:
- M1: Persistent rate limiting (PostgreSQL rate_limit_events table)
- M2: Account lockout (5 failures -> 15min lock)
- M3: RFC 5322 email validation with regex
- M4: Device registration typed struct with length limits
- M5: Provider URL validation on create/update (SSRF prevention)
- M6: Legacy TOTP secret migration (fixed nonce -> random nonce)
- M7: Legacy frontend crypto migration (static salt -> random salt)
- M8+M9: Admin frontend: removed JS token storage, HttpOnly cookie only
- M10: Pipeline debug log sanitization (keys only, 100-char truncation)

Also: fixed CLAUDE.md Section 12 (was corrupted), added title.rs middleware
skeleton, fixed RegisterDeviceRequest visibility.
2026-04-01 08:38:37 +08:00
iven
3b1a017761 fix(saas): fix Box<dyn Error> to anyhow conversion in cache init 2026-03-31 16:37:58 +08:00
iven
4e3265a853 feat(saas): replace scheduler STUB with real task dispatch framework
- Add execute_scheduled_task helper that fetches task info and dispatches
  by target_type (agent/hand/workflow)
- Replace STUB warn+simple-UPDATE with full execution flow: dispatch task,
  then update state with interval-aware next_run_at calculation
- Update next_run_at using interval_seconds for recurring tasks instead
  of setting NULL
- Fix pre-existing cache.rs borrow-after-move bug (id.clone() in insert)
2026-03-31 16:33:54 +08:00
iven
4b9698034c fix(saas): support X-Forwarded-For from trusted reverse proxies 2026-03-31 16:24:02 +08:00
iven
a3bdf11d9a feat(saas): add trusted_proxies config for reverse proxy rate limiting
- Add trusted_proxies field to ServerConfig (Vec<String>, serde default)
- Default value is empty vector (no proxy trust until explicitly configured)
- Development config: trust localhost IPs (127.0.0.1, ::1)
- Production config: placeholder localhost IPs with comment to replace
2026-03-31 16:14:57 +08:00
iven
9905a8d0d5 fix(saas-relay): eliminate DATABASE_ERROR by removing DB queries from critical path
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
Root cause: each relay request executes 13-17 serial DB queries, exhausting
the 50-connection pool under concurrency. When pool is exhausted, sqlx returns
PoolTimedOut which maps to 500 DATABASE_ERROR.

Fixes:
1. log_operation → dispatch_log_operation (async Worker dispatch, non-blocking)
2. record_usage → tokio::spawn (3 DB queries moved off critical path)
3. DB pool: max_connections 50→100 (env-configurable), acquire_timeout 5s→8s

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 14:08:21 +08:00
iven
2ff696289f fix(saas): reduce DB connection pool pressure in relay path
Some checks failed
CI / Rust Check (push) Has been cancelled
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
1. key_pool: merge 3 serial UPDATE queries into 2 (cumulative stats +
   last_used_at combined into single UPDATE)
2. service: reduce SSE spawn sleep from 3s to 500ms and add 5s timeout
   on DB operations to prevent connection hoarding
2026-03-31 13:47:43 +08:00
iven
6cae768401 fix(desktop): session persistence — refresh/login/context/empty-content 4-bug fix
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
1. App.tsx: add restoreSession() call on startup to prevent redirect
   to login page after refresh (isRestoring guard + BootstrapScreen)
2. CloneManager: call syncAgents() after loadClones() to restore
   currentAgent and conversation history on app load
3. zclaw-memory: add get_or_create_session() so frontend session UUID
   is persisted directly — kernel no longer creates mismatched IDs
4. openai.rs: assistant message content must be non-empty for
   Kimi/Qwen APIs — replace empty content with meaningful placeholders

Also includes admin-v2 ModelServices unified page (merge providers +
models + API keys into expandable row layout)
2026-03-31 13:38:59 +08:00
iven
3e5d64484e fix(relay): fix llm_routing read path bug and add User-Agent header for Coding Plan
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
1. connectionStore.ts: storedAccount.account.llm_routing → storedAccount.llm_routing
   - saveSaaSSession stores SaaSAccountInfo directly, not { account: SaaSAccountInfo }
   - This bug caused admin llm_routing config to never take effect

2. relay/service.rs: add User-Agent: claude-code/1.0 header
   - Kimi Coding Plan requires recognized coding agent User-Agent
   - Default reqwest UA is rejected with 403

3. Docs: add llm_routing routing mode explanation and troubleshooting entries
2026-03-31 12:02:32 +08:00
iven
ee51d5abcd feat(admin-v2): add ProTable search, scenarios/quick_commands form, tests, remove quota_reset_interval
- Enable ProTable search on Accounts (username/email), Models (model_id/alias),
  Providers (display_name/name) with hideInSearch for non-searchable columns
- Add scenarios (Select tags) and quick_commands (Form.List) to AgentTemplates
  create form, plus service type updates
- Remove unused quota_reset_interval from ProviderKey model, key_pool SQL,
  handlers, and frontend types; add migration + bump schema to v11
- Add Vitest config, test setup, request interceptor tests (7 cases),
  authStore tests (8 cases) — all 15 passing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 11:13:16 +08:00