zclaw_openfang/docs/wiki-restructure/design-spec.md

# Wiki Restructure Design

> Date: 2026-04-22
> Status: Approved
> Author: Claude + User brainstorming session

## 1. Problem Statement

Current wiki (16 files, ~155KB) has three structural problems:

1. **No task-oriented navigation** — Cannot go from symptom to module quickly
2. **Duplicate content** — Middleware/security/evolution described in 3+ pages
3. **Missing integration contracts** — Cross-module boundaries undocumented
4. **Growing append-only sections** — log.md already 31KB, known-issues.md 13KB

The wiki's primary reader is a Claude AI session that reads it at conversation start to orient itself. Secondary reader is the human developer.

## 2. Design Principles

1. **Wiki documents what code cannot tell you** — WHY decisions, navigation shortcuts, traps, invariants
2. **Code logic sections focus on flows + invariants + algorithms** — NOT field lists or function signatures
3. **Page size budget** — index ≤ 120 lines, module pages 100-200 lines (3-6KB)
4. **Single source of truth per topic** — No content duplicated across pages; use references
5. **Append-only sections are capped** — log.md capped at 50 entries, old entries archived

## 3. Structure

### 3.1 Level 1: `index.md` — Navigation + Symptom Index

```
wiki/index.md
├── Project one-liner
├── Key numbers table (cross-validated with TRUTH.md)
├── System data flow diagram (existing ASCII art)
├── Module navigation tree (one-line description per module)
├── Symptom navigation table (NEW)
└── Module dependency map (who calls who)
```

**Symptom Navigation Table** (NEW):

| Symptom | First check | Then check | Common root cause |
|---------|-------------|------------|-------------------|
| Stream stuck | routing | chat → middleware | Connection lost / SaaS relay timeout |
| Memory not injected | memory | middleware | FTS5 index empty / middleware skipped |
| Hand trigger failed | hands-skills | middleware | Tool call blocked by Guardrail |
| SaaS relay 502 | saas | routing | Token Pool exhausted / Key expired |
| Model switch not working | routing | chat | SaaS whitelist vs local config mismatch |
| Agent creation failed | chat | saas | Permission or persistence issue |
| Pipeline execution stuck | pipeline | middleware | DAG cycle / missing dependency |
| Admin page 403 | saas | security | JWT expired / admin_guard blocked |

**Target**: ≤ 120 lines. A new AI session reads index and immediately knows which modules to open.

### 3.2 Level 2: Module Pages (~15)

Each module page has 5 sections in reading priority order:

#### Section 1: Design Decisions (WHY)

- Why this module was designed this way
- Historical context and background
- Trade-offs made and alternatives rejected
- Key architectural decisions

Format: prose paragraphs + Q&A pairs for important decisions.

#### Section 2: Key Files + Data Flow (WHERE)

- Core files table (3-7 files, one-line responsibility each)
- Module-internal data flow diagram (ASCII or mermaid)
- **Integration contracts** (NEW):
  - What this module calls upstream
  - What this module exposes downstream
  - Interface shapes at boundaries

#### Section 3: Code Logic (LOGIC)

Focus on three types of information that code alone cannot efficiently convey:

- **Key data flows**: Cross-function/cross-file complete paths
- **Invariants**: Constraints that must always hold (marked with ⚡)
- **Non-obvious algorithms**: Logic that is hard to understand from reading code alone

Explicitly EXCLUDED:
- Field lists (read from code)
- Function signatures (read from code)
- CRUD operations (obvious from code)
- Anything that can be answered by `grep`

#### Section 4: Active Issues + Gotchas (GOTCHAS)

- Active issues (0-5 items, removed when fixed)
- Historical pitfall records (≤ 10 items, distilled to one lesson each)
- ⚠️ Warnings (error-prone areas)

#### Section 5: Change Log (CHANGES)

- Last 5 changes (format: date + one-liner)
- Older changes → global `log.md`

### 3.3 Module Page Template

```markdown
---
title: {Module Name}
updated: {YYYY-MM-DD}
status: active | stable | developing
tags: [{module-specific tags}]
---

# {Module Name}

> From [[index]]. Related: [[related-module-1]] [[related-module-2]]

## Design Decisions

{Why this module exists, key design choices, tradeoffs}

## Key Files + Data Flow

### Core Files

| File | Responsibility |
|------|---------------|
| `path/to/file` | One-line description |

### Data Flow

```
{ASCII flow diagram}
```

### Integration Contracts

> Format: Direction | Module | Interface (Rust trait / Tauri invoke / TS function) | Trigger

| Direction | Module | Interface | Trigger |
|-----------|--------|-----------|---------|
| Calls → | {module} | `{rust_fn / tauri_invoke / ts_fn}` | {when/why} |
| Called by ← | {module} | `{rust_fn / tauri_invoke / ts_fn}` | {when/why} |

<!-- Example (middleware.md):
| Calls → | runtime | `MiddlewareChain::run_before_completion()` | Every chat request before LLM call |
| Called by ← | kernel | `kernel/mod.rs:create_middleware_chain()` | Kernel boot, once per session |
| Called by ← | saas | HTTP relay handler | SaaS relay routes (10 HTTP middleware) |
| Provides → | all modules | `AgentMiddleware` trait | 14 implementations registered |
-->

## Code Logic

### Key Data Flows

{Cross-function paths with intent}

### Invariants

⚡ {Invariant 1}: {description of what must always be true}

⚡ {Invariant 2}: {description}

### Non-obvious Algorithms

{Algorithms that are hard to understand from reading code}

## Active Issues + Gotchas

### Active Issues

| Issue | Severity | Status | Notes |
|-------|----------|--------|-------|
| {description} | P{0-3} | Open | {context} |

### Historical Pitfalls

- {Lesson learned}: {one-line description of what went wrong and the fix}

### Warnings

⚠️ {Warning}: {what to watch out for}

## Change Log

| Date | Change |
|------|--------|
| {YYYY-MM-DD} | {one-line description} |
```

## 4. Migration Plan

### 4.0 Execution Order

Migration must follow this sequence due to cross-page dependencies:

1. **Phase A — Archive/cap** (no dependencies): Cap `log.md` at 50 entries, archive old to `wiki/archive/`. Convert `known-issues.md` to pointer. Archive `hermes-analysis.md`.
2. **Phase B — Single source of truth**: Restructure `middleware.md` first (other pages reference it).
3. **Phase C — Dependent pages**: `saas.md`, `security.md`, `memory.md` (remove middleware/evolution duplicates, add contracts).
4. **Phase D — Remaining modules**: `routing.md`, `chat.md`, `butler.md`, `hands-skills.md`, `pipeline.md`, `data-model.md`.
5. **Phase E — Index last**: `index.md` restructure (depends on all modules being complete).
6. **Phase F — feature-map.md**: Distribute chain traces to module "Code Logic" sections, convert to index page.

**Rollback strategy**: Migrate one module per commit. Any partial state is internally consistent per-page. `git revert` on a single commit restores that module's old version.

### 4.1 Per-Page Source-to-Target Mapping

#### `index.md` (8KB → target ≤ 120 lines)

| Current Content | Action | Destination |
|----------------|--------|-------------|
| Key numbers table | Keep | Stay (cross-validated with TRUTH.md) |
| System data flow diagram | Keep | Stay |
| Module navigation tree | Keep | Stay |
| Architecture Q&A "Why 14 middleware" | Move | → `middleware.md` Design Decisions |
| Architecture Q&A "Why 管家 default" | Move | → `butler.md` Design Decisions |
| Architecture Q&A "Why 3 ChatStream" | Move | → `chat.md` Design Decisions |
| Architecture Q&A "Why SaaS relay" | Move | → `routing.md` Design Decisions |
| Architecture Q&A "Evolution engine" | Move | → `memory.md` Design Decisions |
| Symptom navigation table | NEW | Add after navigation tree |

#### `middleware.md` (7KB → target 150-200 lines)

| Current Content | Action | Destination |
|----------------|--------|-------------|
| Design thought | Keep as "Design Decisions" | Expand with WHY from index Q&A |
| 14-layer table | Keep | Core Files + Data Flow |
| Execution flow diagram | Keep | Code Logic |
| SaaS HTTP middleware (10 layers) | Keep | Integration Contracts |
| "11/14 no tests" warning | Keep | Active Issues |
| API interface (trait) | Trim to key data flow | Code Logic (flows only) |

#### `routing.md` (13KB → target 150-200 lines)

| Current Content | Action | Destination |
|----------------|--------|-------------|
| 5-branch decision tree | Keep | Code Logic → Key Data Flows |
| Store layer listing (25 stores) | Remove | Split: chat stores → `chat.md`, saas stores → `saas.md`, connection stores stay |
| lib/ file listing (75 files) | Remove | → `development.md` reference appendix |
| Model routing full chain | Keep (simplified) | Code Logic → Key Data Flows |
| Tauri commands table | Keep | Integration Contracts |

#### `chat.md` (6KB → target 150-200 lines)

| Current Content | Action | Destination |
|----------------|--------|-------------|
| 3 ChatStream implementations | Keep as Design Decision | Add WHY from index Q&A |
| Store 拆分 (5 Store) | Move | → Key Files table |
| Send message flow | Keep | Code Logic → Key Data Flows |
| Add invariants | NEW | e.g., ⚡ sessionKey must be consistent within a conversation |
| Add integration contracts | NEW | Calls → routing (getClient), middleware (chain), saas (relay) |

#### `memory.md` (19KB → target 200 lines, largest compression needed)

| Current Content | Action | Destination |
|----------------|--------|-------------|
| Memory pipeline design | Keep as Design Decisions | + Absorb WHY from index Q&A |
| FTS5/TF-IDF/embedding details | Keep invariants and flows | Code Logic |
| Hermes insights (from hermes-analysis.md) | Distill 3-5 key lessons | Design Decisions (one paragraph) + Gotchas |
| Detailed extraction logic | Trim to flows + invariants | Archive detailed prose to `wiki/archive/` |
| Cross-session injection fix | Keep as historical pitfall | Gotchas |
| Profile store connection fix | Keep as historical pitfall | Gotchas |

#### `saas.md` (10KB → target 150-200 lines)

| Current Content | Action | Destination |
|----------------|--------|-------------|
| Auth flow (JWT/Cookie/TOTP) | Remove details | → `security.md` owns design, saas.md keeps reference |
| Billing/subscription | Keep | Code Logic → Key Data Flows |
| Admin V2 | Keep summary | Key Files |
| Token Pool RPM/TPM | Keep | Code Logic → Non-obvious Algorithms |
| Add integration contracts | NEW | Calls → relay, Called by ← desktop client |

#### `security.md` (6KB → target 150-200 lines)

| Current Content | Action | Destination |
|----------------|--------|-------------|
| Auth flow details | OWN this content | Absorb from saas.md, become single source |
| JWT/Cookie/TOTP details | Keep | Code Logic |
| Rate limiting | Keep | Code Logic |
| Add integration contracts | NEW | Provides auth middleware to SaaS, crypto utils to client |

#### Other modules (`butler`, `hands-skills`, `pipeline`, `data-model`)

All follow same pattern: keep existing design/code sections, add integration contracts, add invariants, trim to size budget.

#### `feature-map.md` (15KB → Convert to index)

| Current Content | Action | Destination |
|----------------|--------|-------------|
| F-01~F-05 chat chains | Distribute | → `chat.md` Code Logic as chain trace reference |
| F-06~F-10 memory chains | Distribute | → `memory.md` Code Logic |
| F-11~F-15 hand chains | Distribute | → `hands-skills.md` Code Logic |
| Remaining chains | Distribute | → Corresponding module pages |
| File itself | Keep as index | Module → feature chain mapping only |

### 4.2 Pages to Merge/Archive

| Page | Action | Destination |
|------|--------|-------------|
| `known-issues.md` | Convert to pointer | Active issues → per-module, global file = links only |
| `log.md` | Cap at 50 entries | Archive old entries to `wiki/archive/log-{YYYY-MM}.md` |
| `hermes-analysis.md` | Archive | Key insights → `memory.md` Gotchas, file → `wiki/archive/` |
| `development.md` | Keep as-is | Global dev standards, not per-module |

### 4.3 Duplicate Content Resolution

| Content | Current Location | New Owner | Others |
|---------|-----------------|-----------|--------|
| Middleware descriptions | middleware + saas + security | `middleware.md` | Reference only |
| Security mechanisms | security + saas | `security.md` | saas.md references |
| Evolution engine | memory + middleware + index | `memory.md` | Others reference |
| Store listing (25) | routing.md | Split: chat→chat, saas→saas, etc. | routing.md keeps connection stores |
| lib/ file listing (75) | routing.md | `development.md` or dedicated reference | routing.md removes |

## 5. Validation Criteria

- [ ] New AI session can locate any module's core files from index in ≤ 2 hops
- [ ] Each module page has integration contracts section
- [ ] No content duplicated across ≥ 3 pages
- [ ] index.md ≤ 120 lines
- [ ] Each module page 100-200 lines
- [ ] log.md ≤ 50 active entries
- [ ] Symptom navigation table covers top 8 common debugging scenarios

## 6. Risks and Mitigations

| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| Module pages exceed size budget | Medium | AI context waste | Trim during migration, move details to archive |
| Invariants drift from code | Medium | Misleading docs | Add "last verified" date, check during code changes |
| Integration contracts incomplete | High | Gap remains | Start with existing cross-references, fill during next debug session |
| Migration breaks existing workflow | Low | Confusion | Migrate one module at a time, verify after each |

## 7. Open Questions

- Should we add a `wiki/templates/module-template.md` for consistency?