feat(knowledge): Phase A 知识库可见性隔离 + 结构化数据源 + 蒸馏Worker
Some checks failed
CI / Lint & TypeCheck (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
CI / Rust Check (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled

- knowledge_items 增加 visibility(public/private) + account_id 字段
- 新建 structured_sources + structured_rows 表 (Excel JSONB 行级存储)
- 结构化数据源 CRUD API (5 路由: list/get/rows/delete/query)
- 安全查询: JSONB GIN 索引 + 可见性过滤 + 行数限制
- 蒸馏 Worker: 复用 Provider Key Pool 调 DeepSeek/Qwen API
- L0 质量过滤: 长度/隐私检测
- create_item 增加 is_admin 参数控制可见性默认值
- generate_embedding: extract_keywords_from_text 改为 pub 复用

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
iven
2026-04-12 18:36:05 +08:00
parent b8fb76375c
commit c3593d3438
10 changed files with 846 additions and 20 deletions

View File

@@ -13,6 +13,7 @@ use zclaw_saas::workers::record_usage::RecordUsageWorker;
use zclaw_saas::workers::update_last_used::UpdateLastUsedWorker;
use zclaw_saas::workers::aggregate_usage::AggregateUsageWorker;
use zclaw_saas::workers::generate_embedding::GenerateEmbeddingWorker;
use zclaw_saas::workers::DistillationWorker;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
@@ -48,8 +49,18 @@ async fn main() -> anyhow::Result<()> {
dispatcher.register(UpdateLastUsedWorker);
dispatcher.register(AggregateUsageWorker);
dispatcher.register(GenerateEmbeddingWorker);
// 蒸馏 Worker需要加密密钥来解密 provider API key
match config.api_key_encryption_key() {
Ok(enc_key) => {
dispatcher.register(DistillationWorker::new(enc_key));
info!("DistillationWorker registered");
}
Err(e) => tracing::warn!("DistillationWorker skipped (no enc key): {}", e),
}
dispatcher.start(); // 必须在所有 register() 之后调用
info!("Worker dispatcher initialized (7 workers registered)");
info!("Worker dispatcher initialized (8 workers registered)");
// 优雅停机令牌 — 取消后所有 SSE 流和长连接立即终止
let shutdown_token = CancellationToken::new();