test: complete exploratory test results for all 4 storylines + sign-off
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
Storyline 3 (极客张): - 3.4 PASS: SaaS Relay SSE chain verified - 3.6 FAIL: BUG-009 confirmed - middleware chain bypassed - Others NOT TESTED: require Tauri kernel local mode Storyline 4 (妈妈): - 4.1 PASS: simple mode UI + message flow - Others SKIP/NOT TESTED: voice input, cold start Sign-off report updated with full test matrix and release recommendation. Blocking: BUG-009 (DataMasking bypass in SaaS Relay mode)
This commit is contained in:
@@ -2,59 +2,105 @@
|
||||
|
||||
> 日期: 2026-04-09
|
||||
> 测试人: Claude + iven
|
||||
> 连接模式: SaaS Relay (主要)
|
||||
|
||||
## 探索性测试执行概况
|
||||
|
||||
| 故事线 | 场景数 | PASS | PARTIAL | FAIL | NOT TESTED |
|
||||
|--------|--------|------|---------|------|------------|
|
||||
| 1. 医院行政小李 | 8 | 5 | 1 | 0 | 2 |
|
||||
| 2. 教师王老师 | 4 | 3 | 1 | 0 | 0 |
|
||||
| 3. 极客张 | 8 | 1 | 1 | 1 | 5 |
|
||||
| 4. 妈妈 | 4 | 2 | 0 | 0 | 1 |
|
||||
| **合计** | **24** | **11** | **3** | **1** | **8** |
|
||||
|
||||
## 测试中发现并修复的 Bug
|
||||
|
||||
| Bug | 严重度 | 描述 | 状态 |
|
||||
|-----|--------|------|------|
|
||||
| BUG-001 | P0 | PostgreSQL SUM() NUMERIC 类型转换 | FIXED |
|
||||
| BUG-003 | P1 | saasStore require() 在 Vite ESM 不可用 | FIXED |
|
||||
| BUG-004 | P1 | Health check 连接池使用率公式错误 | FIXED |
|
||||
| BUG-008 | P1 | SaaS Relay 不发送对话历史 | FIXED |
|
||||
|
||||
## 未修复的开放 Bug
|
||||
|
||||
| Bug | 严重度 | 描述 | 影响 |
|
||||
|-----|--------|------|------|
|
||||
| BUG-009 | P1 | SaaS Relay 绕过全部14层中间件链 | PHI未脱敏 |
|
||||
| BUG-010 | P3 | 流式响应无明确取消按钮 | UX问题 |
|
||||
| BUG-011 | P2 | Admin V2 仪表盘SQL类型错误 | 数据异常 |
|
||||
| BUG-007 | P2 | Admin V2 authStore 测试19个失败 | 测试债务 |
|
||||
|
||||
---
|
||||
|
||||
## 构建与测试
|
||||
- [ ] cargo check --workspace 通过, warnings ≤ 8
|
||||
- [ ] npx tsc --noEmit 通过, 零错误
|
||||
- [x] npx tsc --noEmit 通过, 零错误
|
||||
- [ ] 684 Rust workspace tests 全通过
|
||||
- [ ] 138 SaaS integration tests 全通过
|
||||
- [ ] 330 Frontend tests 全通过
|
||||
- [ ] 61 Admin V2 tests 全通过
|
||||
- [ ] 61 Admin V2 tests 全通过 (BUG-007: 19个失败)
|
||||
- [ ] E2E Playwright tests 全通过 (18 spec 文件)
|
||||
|
||||
## P0 Bug
|
||||
- [ ] 零未解决 P0 bug
|
||||
- [ ] 近期修复已回归验证
|
||||
- [x] 零未解决 P0 bug (BUG-001 已修复)
|
||||
- [x] 近期修复已回归验证 (BUG-001/003/004/008 均重测通过)
|
||||
|
||||
## P1 Bug
|
||||
- [ ] 所有 P1 已解决或有文档记录(含替代方案)
|
||||
- [ ] 空 AI 响应处理已验证
|
||||
- [ ] SaaS relay SSE 错误处理已验证
|
||||
- [ ] **BUG-009 未解决**: SaaS Relay 绕过中间件链,PHI 数据未脱敏
|
||||
- [x] SaaS relay SSE 错误处理已验证
|
||||
- [x] 对话历史传递已验证 (BUG-008 修复后)
|
||||
|
||||
## 关键缺口区域(手动 E2E)
|
||||
- [ ] SaaSRelay(SSE) 通过场景 3.4 验证
|
||||
- [ ] streamStore 通过流式生命周期场景验证
|
||||
- [ ] Quiz Hand 通过场景 2.2 验证
|
||||
- [ ] Slideshow Hand 通过场景 2.3 验证
|
||||
- [ ] Speech Hand 通过场景 2.4 验证
|
||||
- [ ] DataMasking 中间件通过场景 3.6 验证
|
||||
- [ ] ButlerRouter 4域分类通过场景 1.2-1.5 验证
|
||||
- [ ] 冷启动流程通过场景 1.1 和 4.1 验证
|
||||
- [ ] 双 UI 模式切换通过场景 1.3/2.6/4.7 验证
|
||||
- [ ] 记忆管道闭环通过场景 1.6 和 3.7 验证
|
||||
- [ ] NL Schedule Parser 通过场景 1.7 验证
|
||||
- [x] SaaSRelay(SSE) 通过场景 3.4 验证 — PASS
|
||||
- [x] streamStore 通过流式生命周期场景验证 — PASS
|
||||
- [ ] Quiz Hand 通过场景 2.2 验证 — 文本输出 PASS,Hand 触发 NOT TESTED
|
||||
- [ ] Slideshow Hand 通过场景 2.3 验证 — 文本输出 PASS,Hand 触发 NOT TESTED
|
||||
- [ ] Speech Hand 通过场景 2.4 验证 — 朗读指导 PASS,TTS NOT TESTED
|
||||
- [ ] **DataMasking 中间件通过场景 3.6 验证 — FAIL (BUG-009)**
|
||||
- [x] ButlerRouter 4域分类 — 间接验证 (AI 以正确角色回应),直接验证需本地模式
|
||||
- [ ] 冷启动流程通过场景 1.1 和 4.3 验证 — SKIP (非首次安装)
|
||||
- [x] 双 UI 模式切换通过场景 1.3/4.1 验证 — PASS
|
||||
- [ ] 记忆管道闭环通过场景 1.6 和 3.7 验证 — 1.6 PASS (上下文), 3.7 NOT TESTED
|
||||
- [ ] NL Schedule Parser 通过场景 1.7 验证 — PARTIAL (SaaS 模式下不触发)
|
||||
|
||||
## 用户体验
|
||||
- [ ] "妈妈"画像可在10分钟内无帮助完成一项任务
|
||||
- [ ] "小李"画像可端到端完成会议纪要工作流
|
||||
- [ ] "王老师"画像可生成并交付测验
|
||||
- [ ] "极客张"画像可配置自定义模型并执行 Pipeline
|
||||
- [ ] 所有用户可见文本均为中文(无原始英文错误)
|
||||
- [ ] 深色模式完全正常
|
||||
- [ ] 模式切换动画流畅无卡顿
|
||||
- [ ] "妈妈"画像可在10分钟内无帮助完成一项任务 — 简洁模式 UI PASS,需真人验证
|
||||
- [x] "小李"画像可端到端完成会议纪要工作流 — PASS (1.3)
|
||||
- [x] "王老师"画像可生成并交付测验 — PASS (2.2)
|
||||
- [ ] "极客张"画像可配置自定义模型并执行 Pipeline — NOT TESTED (SaaS 限制)
|
||||
- [x] 所有用户可见文本均为中文(无原始英文错误)
|
||||
- [ ] 深色模式完全正常 — 未测试
|
||||
- [x] 模式切换动画流畅无卡顿
|
||||
|
||||
## 安全与合规
|
||||
- [ ] API key 不存储在 localStorage(仅 OS keyring)
|
||||
- [ ] DataMasking 从提示中移除 PHI
|
||||
- [ ] SaaS CORS 白名单正确配置
|
||||
- [ ] Cookie 设置正确: HttpOnly + Secure + SameSite=Strict
|
||||
- [ ] API key 不存储在 localStorage(仅 OS keyring) — SaaS 模式下不适用
|
||||
- [ ] **DataMasking 从提示中移除 PHI — FAIL (BUG-009)**
|
||||
- [x] SaaS CORS 白名单正确配置
|
||||
- [x] Cookie 设置正确: HttpOnly + Secure + SameSite=Strict
|
||||
|
||||
## Admin 后台
|
||||
- [ ] 15个页面全部加载无错误
|
||||
- [ ] 关键页面 CRUD 正常(Accounts/AgentTemplates/ModelServices/Knowledge)
|
||||
- [ ] Admin 认证使用 HttpOnly cookie
|
||||
- [ ] 关键页面 CRUD 正常
|
||||
- [x] Admin 认证使用 HttpOnly cookie
|
||||
|
||||
## 性能与资源
|
||||
- [ ] SSE 流式后无内存泄漏
|
||||
- [ ] 100+消息对话无 UI 卡顿
|
||||
- [ ] 100+记忆条目后搜索仍 < 2s
|
||||
- [x] SSE 流式后无内存泄漏 (观察30分钟未发现)
|
||||
- [ ] 100+消息对话无 UI 卡顿 — 未达到100条
|
||||
- [ ] 100+记忆条目后搜索仍 < 2s — 未测试
|
||||
|
||||
---
|
||||
|
||||
## 发布建议
|
||||
|
||||
**阻塞项**: BUG-009 (SaaS Relay 绕过中间件链) — 如果 SaaS Relay 是主连接模式,必须在发布前修复 DataMasking。
|
||||
|
||||
**建议路径**:
|
||||
1. 修复 BUG-009 (在 SaaS relay handler 中实现 DataMasking)
|
||||
2. 切换到 Tauri 本地模式补充测试 (场景 3.1/3.3/3.5/3.7/3.8)
|
||||
3. 冷启动清除数据重测 (场景 1.1/4.3)
|
||||
4. 离线物理断网测试 (场景 1.8)
|
||||
|
||||
**签核**: 待 iven 确认
|
||||
**日期**: 2026-04-09
|
||||
|
||||
Reference in New Issue
Block a user