iven/zclaw_openfang
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
353 Commits 2 Branches 0 Tags
7a3334384a02c3a6909d94e66e4e784db01c480c
Commit Graph

27 Commits

Author SHA1 Message Date
iven
305984c982 fix(saas): P2 code quality fixes + config PATCH/PUT alignment
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
Details
CI / Unit Tests (push) Has been cancelled
Details
CI / Build Frontend (push) Has been cancelled
Details
CI / Rust Check (push) Has been cancelled
Details
CI / Security Scan (push) Has been cancelled
Details
CI / E2E Tests (push) Has been cancelled
Details 
P2 code quality (SEC2-P2-01~10):
- P2-04: Replace vague TODO with detailed Phase 2 design note in generate_embedding.rs
- P2-05: Add NOTE(fire-and-forget) annotations to 4 long-running tokio::spawn in main.rs
- P2-07: Add DESIGN NOTE to scheduler explaining sequential execution rationale
- P2-08: Add compile-time table name whitelist + runtime char validation in db.rs
- P2-02: Verified N/A (only zclaw-pipeline uses serde_yaml_bw, no inconsistency)
- P2-06: Verified N/A (bind loop correctly matches 6-column placeholders)
- P2-03: Remains OPEN (requires upstream sqlx release)

Config HTTP method alignment (B3-4):
- Fix admin-v2 config.ts: request.patch -> request.put to match backend .put() route
- Fix backend handler doc comment: PATCH -> PUT
- Add @reserved annotations to 6 config handlers without frontend callers
 2026-04-03 21:32:17 +08:00
iven
28299807b6 fix(desktop): DeerFlow UI — ChatArea refactor + ai-elements + dead CSS cleanup 
ChatArea retry button uses setInput instead of direct sendToGateway,
fix bootstrap spinner stuck for non-logged-in users,
remove dead CSS (aurora-title/sidebar-open/quick-action-chips),
add ai components (ReasoningBlock/StreamingText/ChatMode/ModelSelector/TaskProgress),
add ClassroomPlayer + ResizableChatLayout + artifact panel

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-02 19:24:44 +08:00
iven
11e3d37468 feat(billing): activate real-time quota enforcement pipeline 
- Wire relay handler to increment_usage() for JSON responses (tokens + relay_requests)
- Wire relay handler to increment_dimension("relay_requests") for SSE streams
- Add increment_dimension() function for hand_executions/pipeline_runs dimensions
- Schedule AggregateUsageWorker hourly for reconciliation (run_on_start=true)
- Mount mock payment routes in dev mode (ZCLAW_SAAS_DEV=true)

Previously the quota middleware always allowed requests because usage
counters were never incremented. Now relay requests update billing_usage_quotas
in real-time, with the aggregator providing hourly reconciliation.
 2026-04-02 01:52:01 +08:00
iven
830e9fa301 feat(saas): add GenerateEmbedding worker for knowledge chunking 
- Markdown-aware content splitting (512 token chunks with 64 overlap)
- CJK keyword extraction from chunk content with stop-word filtering
- Full refresh strategy (delete old chunks → re-insert on update)
- Phase 2 placeholder for vector embedding API integration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-02 00:23:38 +08:00
iven
ef60f9a183 feat(saas): add knowledge base module — categories, items, versions, search, analytics 
- 5 knowledge tables (categories, items, chunks, versions, usage) with pgvector + HNSW + GIN indexes
- 23+ API routes covering full CRUD, tree-structured categories, version snapshots
- Keyword-based search with ILIKE + array match (placeholder for vector search)
- Analytics endpoints: overview, trends, top-items, quality, gaps
- Markdown-aware content chunking with overlap strategy
- Worker dispatch for async embedding generation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-02 00:21:28 +08:00
iven
b66087de0e feat(saas): add quota middleware and usage aggregation worker 
B1.3 Quota middleware:
- quota_check_middleware for relay route chain
- Checks monthly relay_requests quota before processing
- Gracefully degrades on billing service failure

B1.5 AggregateUsageWorker:
- Aggregates usage_records into billing_usage_quotas monthly
- Supports single-account and all-accounts modes
- Scheduled hourly via Worker dispatcher (6 workers total)
 2026-04-02 00:06:39 +08:00
iven
d06ecded34 feat(saas): add quota check middleware for relay requests 
Injects billing quota verification before relay chat completion requests.
Checks monthly relay_requests quota via billing::service::check_quota.
Gracefully degrades on quota service failure (logs warning, allows request).
 2026-04-02 00:03:26 +08:00
iven
9487cd7f72 feat(saas): add billing infrastructure — tables, types, service, handlers 
B1.1 Billing database:
- 5 tables: billing_plans, billing_subscriptions, billing_invoices,
  billing_payments, billing_usage_quotas
- Seed data: Free(¥0)/Pro(¥49)/Team(¥199) plans
- JSONB limits for flexible plan configuration

Billing module (crates/zclaw-saas/src/billing/):
- types.rs: BillingPlan, Subscription, Invoice, Payment, UsageQuota
- service.rs: plan CRUD, subscription lookup, usage tracking, quota check
- handlers.rs: REST API (plans list/detail, subscription, usage)
- mod.rs: routes registered at /api/v1/billing/*

Cargo.toml: added chrono feature to sqlx for DateTime<Utc> support
 2026-04-01 23:59:46 +08:00
iven
e3b93ff96d fix(security): implement all 15 security fixes from penetration test V1 
Security audit (2026-03-31): 5 HIGH + 10 MEDIUM issues, all fixed.

HIGH:
- H1: JWT password_version mechanism (pwv in Claims, middleware verification,
  auto-increment on password change)
- H2: Docker saas port bound to 127.0.0.1
- H3: TOTP encryption key decoupled from JWT secret (production bailout)
- H4+H5: Tauri CSP hardened (removed unsafe-inline, restricted connect-src)

MEDIUM:
- M1: Persistent rate limiting (PostgreSQL rate_limit_events table)
- M2: Account lockout (5 failures -> 15min lock)
- M3: RFC 5322 email validation with regex
- M4: Device registration typed struct with length limits
- M5: Provider URL validation on create/update (SSRF prevention)
- M6: Legacy TOTP secret migration (fixed nonce -> random nonce)
- M7: Legacy frontend crypto migration (static salt -> random salt)
- M8+M9: Admin frontend: removed JS token storage, HttpOnly cookie only
- M10: Pipeline debug log sanitization (keys only, 100-char truncation)

Also: fixed CLAUDE.md Section 12 (was corrupted), added title.rs middleware
skeleton, fixed RegisterDeviceRequest visibility.
 2026-04-01 08:38:37 +08:00
iven
3b1a017761 fix(saas): fix Box<dyn Error> to anyhow conversion in cache init 2026-03-31 16:37:58 +08:00
iven
eb956d0dce feat: 新增管理后台前端项目及安全加固
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
Details
CI / Unit Tests (push) Has been cancelled
Details
CI / Build Frontend (push) Has been cancelled
Details
CI / Rust Check (push) Has been cancelled
Details
CI / Security Scan (push) Has been cancelled
Details
CI / E2E Tests (push) Has been cancelled
Details 
refactor(saas): 重构认证中间件与限流策略
- 登录限流调整为5次/分钟/IP
- 注册限流调整为3次/小时/IP
- GET请求不计入限流

fix(saas): 修复调度器时间戳处理
- 使用NOW()替代文本时间戳
- 兼容TEXT和TIMESTAMPTZ列类型

feat(saas): 实现环境变量插值
- 支持${ENV_VAR}语法解析
- 数据库密码支持环境变量注入

chore: 新增前端管理界面
- 基于React+Ant Design Pro
- 包含路由守卫/错误边界
- 对接58个API端点

docs: 更新安全加固文档
- 新增密钥管理规范
- 记录P0安全项审计结果
- 补充TLS终止说明

test: 完善配置解析单元测试
- 新增环境变量插值测试用例
 2026-03-31 00:11:33 +08:00
iven
a0bbd4ba82 feat(scheduler): 定时任务后端持久化 + Pipeline trigger 编译修复 
S4/S8 定时任务后端:
- 新增 scheduled_tasks 表 (migration v7)
- 新增 scheduled_task CRUD 模块 (handlers/service/types)
- 注册 /api/scheduler/tasks 路由 (GET/POST/PATCH/DELETE)
- 新增 start_user_task_scheduler() 30秒轮询循环
- 支持 cron/interval/once 三种调度类型
- once 类型执行后自动禁用

修复:
- pipeline_commands.rs: 修复 pipeline.trigger 字段不存在的编译错误
  (Pipeline 结构体无 trigger 字段，改用 metadata.tags/description)
 2026-03-30 19:46:45 +08:00
iven
c2aff09811 feat(security): Auth Token HttpOnly Cookie — XSS 安全加固 
后端:
- axum-extra 启用 cookie feature
- login/register/refresh 设置 HttpOnly + Secure + SameSite=Strict cookies
- 新增 POST /api/v1/auth/logout 清除 cookies
- auth_middleware 支持 cookie 提取路径（fallback from header）
- CORS: 添加 allow_credentials(true) + COOKIE header

前端 (admin-v2):
- authStore: token 仅存内存，不再写 localStorage（account 保留）
- request.ts: 添加 withCredentials: true 发送 cookies
- 修复 refresh token rotation bug（之前不更新 stored refreshToken）
- logout 调用后端清除 cookie 端点

向后兼容: API 客户端仍可用 Authorization: Bearer header
Desktop (Ed25519 设备认证) 完全不受影响
 2026-03-30 19:30:42 +08:00
iven
ba2c6a6105 fix(saas): P1 审计修复 — 连接池断路器 + Worker重试 + XSS防护 + 状态机SQL解析器 
P1 修复内容:
- F7: health handler 连接池容量检查 (80%阈值返回503 degraded)
- F9: SSE spawned task 并发限制 (Semaphore 16 permits)
- F10: Key Pool 单次 JOIN 查询优化 (消除 N+1)
- F12: CORS panic → 配置错误
- F14: 连接池使用率计算修正 (ratio = used*100/total)
- F15: SQL 迁移解析器替换为状态机 (支持 $$, DO $body$, 存储过程)
- Worker 重试机制: 失败任务通过 mpsc channel 重新入队
- DOMPurify XSS 防护 (PipelineResultPreview)
- Admin V2: ErrorBoundary + SWR全局配置 + 请求优化
 2026-03-30 14:21:39 +08:00
iven
bc8c77e7fe fix(security): P0 审计修复 — 6项关键安全/编译问题 
F1: kernel.rs multi-agent 编译错误 — 重排 spawn_agent 中 A2A 注册顺序，
    在 config 被 registry.register() 消费前使用
F2: saas-config.toml 从 git 追踪中移除 — 包含数据库密码已进入版本历史
F3: config.rs 硬编码开发密钥改用 #[cfg(debug_assertions)] 编译时门控 —
    dev fallback 密钥不再进入 release 构建
F4: 公共认证端点添加 IP 速率限制 (20 RPM) — 防止暴力破解
F5: SSE relay 路由分离出全局 15s TimeoutLayer — 避免长流式响应被截断
F6: Provider API 密钥入库前 AES-256-GCM 加密 — 明文存储修复

附带：完整审计报告 docs/superpowers/specs/2026-03-30-comprehensive-audit-report.md
 2026-03-30 13:32:22 +08:00
iven
834aa12076 fix: P0 panic风险修复 + P1编译warnings清零 + P2代码/文档清理
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
Details
CI / Unit Tests (push) Has been cancelled
Details
CI / Build Frontend (push) Has been cancelled
Details
CI / Rust Check (push) Has been cancelled
Details
CI / Security Scan (push) Has been cancelled
Details
CI / E2E Tests (push) Has been cancelled
Details 
P0 安全性:
- account/handlers.rs: .unwrap() → .expect() 语义化错误信息
- relay/handlers.rs: SSE Response .unwrap() → .expect()

P1 编译质量 (6 warnings → 0):
- kernel.rs: 移除未使用的 Capability import 和 config_clone 变量
- pipeline_commands.rs: 未使用变量 id → _id
- db.rs: 移除多余括号
- relay/service.rs: 移除未使用的 StreamExt import
- telemetry/service.rs: 抑制 param_idx 未读赋值警告
- main.rs: TcpKeepalive::with_retries() Linux-only 条件编译

P2 代码清理:
- 移除 handStore/HandsPanel/HandTaskPanel/gateway-api/SchedulerPanel 调试 console.log
- SchedulerPanel: 修复 updateWorkflow 未解构导致 TS 编译错误
- 文档清理 zclaw-channels 已移除 crate 的引用
 2026-03-30 11:33:47 +08:00
iven
13c0b18bbc feat: Batch 5-9 — GrowthIntegration桥接、验证补全、死代码清理、Pipeline模板、Speech/Twitter真实实现
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
Details
CI / Unit Tests (push) Has been cancelled
Details
CI / Build Frontend (push) Has been cancelled
Details
CI / Rust Check (push) Has been cancelled
Details
CI / Security Scan (push) Has been cancelled
Details
CI / E2E Tests (push) Has been cancelled
Details 
Batch 5 (P0): GrowthIntegration 接入 Tauri
- Kernel 新增 set_viking()/set_extraction_driver() 桥接 SqliteStorage
- 中间件链共享存储，MemoryExtractor 接入 LLM 驱动

Batch 6 (P1): 输入验证 + Heartbeat
- Relay 验证补全（stream 兼容检查、API key 格式校验）
- UUID 类型校验、SessionId 错误返回
- Heartbeat 默认开启 + 首次聊天自动初始化

Batch 7 (P2): 死代码清理
- zclaw-channels 整体移除（317 行）
- multi-agent 特性门控、admin 方法标注

Batch 8 (P2): Pipeline 模板
- PipelineMetadata 新增 annotations 字段
- pipeline_templates 命令 + 2 个示例模板
- fallback driver base_url 修复（doubao/qwen/deepseek 端点）

Batch 9 (P1): SpeechHand/TwitterHand 真实实现
- SpeechHand: tts_method 字段 + Browser TTS 前端集成 (Web Speech API)
- TwitterHand: 12 个 action 全部替换为 Twitter API v2 真实 HTTP 调用
- chatStore/useAutomationEvents 双路径 TTS 触发
 2026-03-30 09:24:50 +08:00
iven
8b9d506893 refactor(saas): 架构重构 + 性能优化 — 借鉴 loco-rs 模式 
Phase 0: 知识库
- docs/knowledge-base/loco-rs-patterns.md — loco-rs 10 个可借鉴模式研究

Phase 1: 数据层重构
- crates/zclaw-saas/src/models/ — 15 个 FromRow 类型化模型
- Login 3 次查询合并为 1 次 AccountLoginRow 查询
- 所有 service 文件从元组解构迁移到 FromRow 结构体

Phase 2: Worker + Scheduler 系统
- crates/zclaw-saas/src/workers/ — Worker trait + 5 个具体实现
- crates/zclaw-saas/src/scheduler.rs — TOML 声明式调度器
- crates/zclaw-saas/src/tasks/ — CLI 任务系统

Phase 3: 性能修复
- Relay N+1 查询 → 精准 SQL (relay/handlers.rs)
- Config RwLock → AtomicU32 无锁 rate limit (state.rs, middleware.rs)
- SSE std::sync::Mutex → tokio::sync::Mutex (relay/service.rs)
- /auth/refresh 阻塞清理 → Scheduler 定期执行

Phase 4: 多环境配置
- config/saas-{development,production,test}.toml
- ZCLAW_ENV 环境选择 + ZCLAW_SAAS_CONFIG 精确覆盖
- scheduler 配置集成到 TOML
 2026-03-29 19:21:48 +08:00
iven
5fdf96c3f5 chore: 提交所有工作进度 — SaaS 后端增强、Admin UI、桌面端集成 
包含大量 SaaS 平台改进、Admin 管理后台更新、桌面端集成完善、
文档同步、测试文件重构等内容。为 QA 测试准备干净工作树。
 2026-03-29 10:46:41 +08:00
iven
8cce2283f7 fix(saas): P0 安全修复 + P1 功能补全 — 角色提升、Admin 引导、IP 记录、密码修改 
P0 安全修复:
- 修复 account update 自角色提升漏洞: 非 admin 用户更新自己时剥离 role 字段
- 添加 Admin 引导机制: accounts 表为空时自动从环境变量创建 super_admin

P1 功能补全:
- 所有 17 个 log_operation 调用点传入真实客户端 IP (ConnectInfo + X-Forwarded-For)
- AuthContext 新增 client_ip 字段, middleware 层自动提取
- main.rs 使用 into_make_service_with_connect_info 启用 SocketAddr 注入
- 新增 PUT /api/v1/auth/password 密码修改端点 (验证旧密码 + argon2 哈希)
- 桌面端 SaaS 设置页添加密码修改 UI (折叠式表单)
- SaaSClient 添加 changePassword() 方法
- 集成测试修复: 注入模拟 ConnectInfo 适配 onshot 测试模式
 2026-03-27 14:45:47 +08:00
iven
d760b9ca10 feat(saas): Phase 1 后端能力补强 — API Token 认证、真实 SSE 流式、速率限制 
Phase 1.1: API Token 认证中间件
- auth_middleware 新增 zclaw_ 前缀 token 分支 (SHA-256 验证)
- 合并 token 自身权限与角色权限，异步更新 last_used_at
- 添加 GET /api/v1/auth/me 端点返回当前用户信息
- get_role_permissions 改为 pub(crate) 供中间件调用

Phase 1.2: 真实 SSE 流式中转
- RelayResponse::Sse 改为 axum::body::Body (bytes_stream)
- 流式请求超时提升至 300s，转发 SSE headers (Cache-Control, Connection)
- 添加 futures 依赖用于 StreamExt

Phase 1.3: 滑动窗口速率限制中间件
- 按 account_id 做 per-minute 限流 (默认 60 rpm + 10 burst)
- 超限返回 429 + Retry-After header
- RateLimitConfig 支持配置化，DashMap 存储时间戳

21 tests passed, zero warnings.
 2026-03-27 13:49:45 +08:00
iven
900430d93e fix(saas): 修复安全审查发现的 Critical/High/Medium 问题 
- Critical: 移除注册接口的 role 字段，固定为 "user" 防止权限提升
- High: 生产环境未配置 cors_origins 时拒绝启动而非默认全开放
- Medium: 增强 SSRF 防护 — 阻止 IPv6 映射地址、私有 IP 网段、十进制 IP 格式
 2026-03-27 13:09:59 +08:00
iven
94bf387aee fix(saas): 安全修复 — IDOR防护、SSRF防护、JWT密钥强制、错误信息脱敏、CORS配置化 
- account: admin 权限守卫 (list_accounts/get_account/update_status/list_logs)
- relay: SSRF 防护 (禁止内网地址、限制 http scheme、30s 超时)
- config: 生产环境强制 ZCLAW_SAAS_JWT_SECRET 环境变量
- error: 500 错误不再泄露内部细节给客户端
- main: CORS 支持配置白名单 origins
- 全部 21 个测试通过 (7 unit + 14 integration)
 2026-03-27 13:07:20 +08:00
iven
00a08c9f9b feat(saas): Phase 4 — 配置迁移模块 
- 配置项 CRUD (列表/详情/创建/更新/删除)
- 配置分析端点 (按类别汇总, SaaS 托管统计)
- 13 个默认配置项种子数据 (server/agent/memory/llm)
- 配置同步协议 (客户端→SaaS, SaaS 优先策略)
- 同步日志记录和查询
- 3 个新集成测试覆盖配置迁移端点
 2026-03-27 12:58:02 +08:00
iven
a99a3df9dd feat(saas): Phase 3 — 模型请求中转服务 
- OpenAI 兼容 API 代理 (/api/v1/relay/chat/completions)
- 中转任务管理 (创建/查询/状态跟踪)
- 可用模型列表端点 (仅 enabled providers+models)
- 任务生命周期 (queued → processing → completed/failed)
- 用量自动记录 (token 统计 + 错误追踪)
- 3 个新集成测试覆盖中转端点
 2026-03-27 12:58:02 +08:00
iven
fec64af565 feat(saas): Phase 2 — 模型配置模块 
- Provider CRUD (列表/详情/创建/更新/删除)
- Model CRUD (列表/详情/创建/更新/删除)
- Account API Key 管理 (创建/轮换/撤销/掩码显示)
- Usage 统计 (总量/按模型/按天, 支持时间/供应商/模型过滤)
- 权限控制 (provider:manage, model:manage)
- 3 个新集成测试覆盖 providers/models/keys
 2026-03-27 12:58:02 +08:00
iven
a2f8112d69 feat(saas): Phase 1 — 基础框架与账号管理模块 
- 新增 zclaw-saas crate 作为 workspace 成员
- 配置系统 (TOML + 环境变量覆盖)
- 错误类型体系 (SaasError 16 变体, IntoResponse)
- SQLite 数据库 (12 表 schema, 内存/文件双模式, 3 系统角色种子数据)
- JWT 认证 (签发/验证/刷新)
- Argon2id 密码哈希
- 认证中间件 (公开/受保护路由分层)
- 账号管理 CRUD + API Token 管理 + 操作日志
- 7 单元测试 + 5 集成测试全部通过
 2026-03-27 12:58:01 +08:00