test: complete exploratory test results for all 4 storylines + sign-off
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
Storyline 3 (极客张): - 3.4 PASS: SaaS Relay SSE chain verified - 3.6 FAIL: BUG-009 confirmed - middleware chain bypassed - Others NOT TESTED: require Tauri kernel local mode Storyline 4 (妈妈): - 4.1 PASS: simple mode UI + message flow - Others SKIP/NOT TESTED: voice input, cold start Sign-off report updated with full test matrix and release recommendation. Blocking: BUG-009 (DataMasking bypass in SaaS Relay mode)
This commit is contained in:
94
docs/test-results/2026-04-09-exploratory/storyline-3-geek.md
Normal file
94
docs/test-results/2026-04-09-exploratory/storyline-3-geek.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# 故事线 3 测试记录 — 技术用户"极客张"
|
||||
|
||||
> 日期: 2026-04-09
|
||||
> 测试人: Claude (自动化) + iven (指导)
|
||||
> 连接模式: SaaS Relay
|
||||
|
||||
## 场景 3.1 — 多模型配置
|
||||
|
||||
**状态: NOT TESTED (SaaS Relay 限制)**
|
||||
|
||||
- SaaS Relay 模式下模型由 SaaS Token Pool 决定,不支持自定义 API Key/BaseUrl
|
||||
- 模型选择器显示"闪速"(GLM-4-Flash) 和 "GLM-4.7" 两个选项
|
||||
- 需本地 Tauri 模式测试完整自定义模型功能
|
||||
|
||||
## 场景 3.2 — Agent 创建与模板
|
||||
|
||||
**状态: PARTIAL PASS**
|
||||
|
||||
| 验证点 | 结果 | 备注 |
|
||||
|--------|------|------|
|
||||
| 智能体 Tab 显示 | ✅ | 侧边栏有"智能体"选项卡 |
|
||||
| Agent 列表 | ✅ | 显示"默认助手" |
|
||||
| 模板加载 | ❌ | SaaS fetchAvailableTemplates() 返回列表有限 |
|
||||
| Agent 创建 | NOT TESTED | "创建新 Agent"按钮在 SaaS Relay 模式下不可用 |
|
||||
|
||||
## 场景 3.3 — Pipeline 编辑与执行
|
||||
|
||||
**状态: NOT TESTED (SaaS Relay 限制)**
|
||||
|
||||
- Pipeline 编辑器需要 Tauri kernel 本地能力
|
||||
- SaaS Relay 模式无 Pipeline 执行路径
|
||||
|
||||
## 场景 3.4 — SaaS Relay 完整链路 (核心)
|
||||
|
||||
**状态: PASS**
|
||||
|
||||
| 验证点 | 结果 | 备注 |
|
||||
|--------|------|------|
|
||||
| SSE 流式响应 | ✅ | 字符逐显,首 token ~2s |
|
||||
| SSE 解析正确性 | ✅ | delta/onDelta/onComplete 正常 |
|
||||
| [DONE] 事件 | ✅ | 流结束触发 onComplete |
|
||||
| 思考过程显示 | ✅ | GLM-4.7 reasoning 可见 |
|
||||
| 对话历史传递 | ✅ | BUG-008 修复后验证通过 |
|
||||
| 连接模式切换 | ✅ | 简洁/专业模式平滑切换 |
|
||||
|
||||
**未测**: 流式取消(BUG-010)、SaaS unreachable 降级、stream_timeout
|
||||
|
||||
## 场景 3.5 — Browser Hand
|
||||
|
||||
**状态: NOT TESTED**
|
||||
|
||||
- Browser Hand 需要 Tauri kernel,SaaS Relay 模式无法触发
|
||||
|
||||
## 场景 3.6 — 中间件链验证
|
||||
|
||||
**状态: FAIL (BUG-009)**
|
||||
|
||||
| 验证点 | 结果 | 备注 |
|
||||
|--------|------|------|
|
||||
| DataMasking | ❌ | 身份证号原样发给 LLM,未脱敏 |
|
||||
| ButlerRouter | ❌ | 关键词分类未生效(无 system prompt 增强) |
|
||||
| TrajectoryRecorder | ❌ | 轨迹未记录到 SQLite |
|
||||
|
||||
**测试内容**: 发送"患者张三,身份证号110101199001011234"。
|
||||
**结果**: AI 收到完整身份证号,未经过 DataMasking 中间件处理。AI 自身在回复中提到"隐私保护:患者身份证号已脱敏处理",但这只是 AI 的主动行为,不是系统级脱敏。
|
||||
**根因**: SaaS Relay 模式直接走 HTTP POST → SaaS → LLM,完全绕过本地 14 层中间件链。
|
||||
|
||||
## 场景 3.7 — 记忆管道深度测试
|
||||
|
||||
**状态: NOT TESTED**
|
||||
|
||||
- 需要 5 轮深度对话 + 30s 等待 + FTS5 搜索验证
|
||||
- 记忆提取依赖本地 SQLite + embedding pipeline
|
||||
|
||||
## 场景 3.8 — 多 Agent 身份切换
|
||||
|
||||
**状态: NOT TESTED**
|
||||
|
||||
- 只有默认助手 Agent,无法测试多 Agent 切换
|
||||
|
||||
## 总结
|
||||
|
||||
| 场景 | 状态 | 关键发现 |
|
||||
|------|------|----------|
|
||||
| 3.1 多模型 | NOT TESTED | SaaS Relay 限制 |
|
||||
| 3.2 Agent 创建 | PARTIAL | 列表可见,创建受限 |
|
||||
| 3.3 Pipeline | NOT TESTED | 需要 Tauri kernel |
|
||||
| 3.4 Relay 链路 | **PASS** | SSE/历史/思考过程均正常 |
|
||||
| 3.5 Browser | NOT TESTED | 需要 Tauri kernel |
|
||||
| 3.6 中间件链 | **FAIL** | BUG-009: 全部中间件被绕过 |
|
||||
| 3.7 记忆管道 | NOT TESTED | 需本地 SQLite |
|
||||
| 3.8 多 Agent | NOT TESTED | 仅一个 Agent |
|
||||
|
||||
**关键发现**: SaaS Relay 模式在纯聊天场景下工作良好(BUG-008 修复后),但完全绕过了本地中间件链(DataMasking/ButlerRouter/TrajectoryRecorder),这是 P1 安全问题 (BUG-009)。
|
||||
Reference in New Issue
Block a user