iven/zclaw_openfang
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

release(v0.2.0): streaming, MCP protocol, Browser Hand, security enhancements

Browse Source 
## Major Features

### Streaming Response System
- Implement LlmDriver trait with `stream()` method returning async Stream
- Add SSE parsing for Anthropic and OpenAI API streaming
- Integrate Tauri event system for frontend streaming (`stream:chunk` events)
- Add StreamChunk types: Delta, ToolStart, ToolEnd, Complete, Error

### MCP Protocol Implementation
- Add MCP JSON-RPC 2.0 types (mcp_types.rs)
- Implement stdio-based MCP transport (mcp_transport.rs)
- Support tool discovery, execution, and resource operations

### Browser Hand Implementation
- Complete browser automation with Playwright-style actions
- Support Navigate, Click, Type, Scrape, Screenshot, Wait actions
- Add educational Hands: Whiteboard, Slideshow, Speech, Quiz

### Security Enhancements
- Implement command whitelist/blacklist for shell_exec tool
- Add SSRF protection with private IP blocking
- Create security.toml configuration file

## Test Improvements
- Fix test import paths (security-utils, setup)
- Fix vi.mock hoisting issues with vi.hoisted()
- Update test expectations for validateUrl and sanitizeFilename
- Add getUnsupportedLocalGatewayStatus mock

## Documentation Updates
- Update architecture documentation
- Improve configuration reference
- Add quick-start guide updates

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
iven
2026-03-24 03:24:24 +08:00
parent e49ba4460b
commit 3ff08faa56
78 changed files with 29575 additions and 1682 deletions
Download Patch File Download Diff File Expand all files Collapse all files

425
crates/zclaw-hands/src/hands/slideshow.rs Normal file
View File

@@ -0,0 +1,425 @@
//! Slideshow Hand - Presentation control capabilities
//!
//! Provides slideshow control for teaching:
//! - next_slide/prev_slide: Navigation
//! - goto_slide: Jump to specific slide
//! - spotlight: Highlight elements
//! - laser: Show laser pointer
//! - highlight: Highlight areas
//! - play_animation: Trigger animations
use async_trait::async_trait;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use std::sync::Arc;
use tokio::sync::RwLock;
use zclaw_types::Result;
use crate::{Hand, HandConfig, HandContext, HandResult, HandStatus};
/// Slideshow action types
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "action", rename_all = "snake_case")]
pub enum SlideshowAction {
/// Go to next slide
NextSlide,
/// Go to previous slide
PrevSlide,
/// Go to specific slide
GotoSlide {
slide_number: usize,
},
/// Spotlight/highlight an element
Spotlight {
element_id: String,
#[serde(default = "default_spotlight_duration")]
duration_ms: u64,
},
/// Show laser pointer at position
Laser {
x: f64,
y: f64,
#[serde(default = "default_laser_duration")]
duration_ms: u64,
},
/// Highlight a rectangular area
Highlight {
x: f64,
y: f64,
width: f64,
height: f64,
#[serde(default)]
color: Option<String>,
#[serde(default = "default_highlight_duration")]
duration_ms: u64,
},
/// Play animation
PlayAnimation {
animation_id: String,
},
/// Pause auto-play
Pause,
/// Resume auto-play
Resume,
/// Start auto-play
AutoPlay {
#[serde(default = "default_interval")]
interval_ms: u64,
},
/// Stop auto-play
StopAutoPlay,
/// Get current state
GetState,
/// Set slide content (for dynamic slides)
SetContent {
slide_number: usize,
content: SlideContent,
},
}
fn default_spotlight_duration() -> u64 { 2000 }
fn default_laser_duration() -> u64 { 3000 }
fn default_highlight_duration() -> u64 { 2000 }
fn default_interval() -> u64 { 5000 }
/// Slide content structure
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SlideContent {
pub title: String,
#[serde(default)]
pub subtitle: Option<String>,
#[serde(default)]
pub content: Vec<ContentBlock>,
#[serde(default)]
pub notes: Option<String>,
#[serde(default)]
pub background: Option<String>,
}
/// Content block types
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ContentBlock {
Text { text: String, style: Option<TextStyle> },
Image { url: String, alt: Option<String> },
List { items: Vec<String>, ordered: bool },
Code { code: String, language: Option<String> },
Math { latex: String },
Table { headers: Vec<String>, rows: Vec<Vec<String>> },
Chart { chart_type: String, data: serde_json::Value },
}
/// Text style options
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct TextStyle {
#[serde(default)]
pub bold: bool,
#[serde(default)]
pub italic: bool,
#[serde(default)]
pub size: Option<u32>,
#[serde(default)]
pub color: Option<String>,
}
/// Slideshow state
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SlideshowState {
pub current_slide: usize,
pub total_slides: usize,
pub is_playing: bool,
pub auto_play_interval_ms: u64,
pub slides: Vec<SlideContent>,
}
impl Default for SlideshowState {
fn default() -> Self {
Self {
current_slide: 0,
total_slides: 0,
is_playing: false,
auto_play_interval_ms: 5000,
slides: Vec::new(),
}
}
}
/// Slideshow Hand implementation
pub struct SlideshowHand {
config: HandConfig,
state: Arc<RwLock<SlideshowState>>,
}
impl SlideshowHand {
/// Create a new slideshow hand
pub fn new() -> Self {
Self {
config: HandConfig {
id: "slideshow".to_string(),
name: "Slideshow".to_string(),
description: "Control presentation slides and highlights".to_string(),
needs_approval: false,
dependencies: vec![],
input_schema: Some(serde_json::json!({
"type": "object",
"properties": {
"action": { "type": "string" },
"slide_number": { "type": "integer" },
"element_id": { "type": "string" },
}
})),
tags: vec!["presentation".to_string(), "education".to_string()],
enabled: true,
},
state: Arc::new(RwLock::new(SlideshowState::default())),
}
}
/// Create with slides (async version)
pub async fn with_slides_async(slides: Vec<SlideContent>) -> Self {
let hand = Self::new();
let mut state = hand.state.write().await;
state.total_slides = slides.len();
state.slides = slides;
drop(state);
hand
}
/// Execute a slideshow action
pub async fn execute_action(&self, action: SlideshowAction) -> Result<HandResult> {
let mut state = self.state.write().await;
match action {
SlideshowAction::NextSlide => {
if state.current_slide < state.total_slides.saturating_sub(1) {
state.current_slide += 1;
}
Ok(HandResult::success(serde_json::json!({
"status": "next",
"current_slide": state.current_slide,
"total_slides": state.total_slides,
})))
}
SlideshowAction::PrevSlide => {
if state.current_slide > 0 {
state.current_slide -= 1;
}
Ok(HandResult::success(serde_json::json!({
"status": "prev",
"current_slide": state.current_slide,
"total_slides": state.total_slides,
})))
}
SlideshowAction::GotoSlide { slide_number } => {
if slide_number < state.total_slides {
state.current_slide = slide_number;
Ok(HandResult::success(serde_json::json!({
"status": "goto",
"current_slide": state.current_slide,
"slide_content": state.slides.get(slide_number),
})))
} else {
Ok(HandResult::error(format!("Slide {} out of range", slide_number)))
}
}
SlideshowAction::Spotlight { element_id, duration_ms } => {
Ok(HandResult::success(serde_json::json!({
"status": "spotlight",
"element_id": element_id,
"duration_ms": duration_ms,
})))
}
SlideshowAction::Laser { x, y, duration_ms } => {
Ok(HandResult::success(serde_json::json!({
"status": "laser",
"x": x,
"y": y,
"duration_ms": duration_ms,
})))
}
SlideshowAction::Highlight { x, y, width, height, color, duration_ms } => {
Ok(HandResult::success(serde_json::json!({
"status": "highlight",
"x": x, "y": y,
"width": width, "height": height,
"color": color.unwrap_or_else(|| "#ffcc00".to_string()),
"duration_ms": duration_ms,
})))
}
SlideshowAction::PlayAnimation { animation_id } => {
Ok(HandResult::success(serde_json::json!({
"status": "animation",
"animation_id": animation_id,
})))
}
SlideshowAction::Pause => {
state.is_playing = false;
Ok(HandResult::success(serde_json::json!({
"status": "paused",
})))
}
SlideshowAction::Resume => {
state.is_playing = true;
Ok(HandResult::success(serde_json::json!({
"status": "resumed",
})))
}
SlideshowAction::AutoPlay { interval_ms } => {
state.is_playing = true;
state.auto_play_interval_ms = interval_ms;
Ok(HandResult::success(serde_json::json!({
"status": "autoplay",
"interval_ms": interval_ms,
})))
}
SlideshowAction::StopAutoPlay => {
state.is_playing = false;
Ok(HandResult::success(serde_json::json!({
"status": "stopped",
})))
}
SlideshowAction::GetState => {
Ok(HandResult::success(serde_json::to_value(&*state).unwrap_or(Value::Null)))
}
SlideshowAction::SetContent { slide_number, content } => {
if slide_number < state.slides.len() {
state.slides[slide_number] = content.clone();
Ok(HandResult::success(serde_json::json!({
"status": "content_set",
"slide_number": slide_number,
})))
} else if slide_number == state.slides.len() {
state.slides.push(content);
state.total_slides = state.slides.len();
Ok(HandResult::success(serde_json::json!({
"status": "slide_added",
"slide_number": slide_number,
})))
} else {
Ok(HandResult::error(format!("Invalid slide number: {}", slide_number)))
}
}
}
}
/// Get current state
pub async fn get_state(&self) -> SlideshowState {
self.state.read().await.clone()
}
/// Add a slide
pub async fn add_slide(&self, content: SlideContent) {
let mut state = self.state.write().await;
state.slides.push(content);
state.total_slides = state.slides.len();
}
}
impl Default for SlideshowHand {
fn default() -> Self {
Self::new()
}
}
#[async_trait]
impl Hand for SlideshowHand {
fn config(&self) -> &HandConfig {
&self.config
}
async fn execute(&self, _context: &HandContext, input: Value) -> Result<HandResult> {
let action: SlideshowAction = match serde_json::from_value(input) {
Ok(a) => a,
Err(e) => {
return Ok(HandResult::error(format!("Invalid slideshow action: {}", e)));
}
};
self.execute_action(action).await
}
fn status(&self) -> HandStatus {
HandStatus::Idle
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_slideshow_creation() {
let hand = SlideshowHand::new();
assert_eq!(hand.config().id, "slideshow");
}
#[tokio::test]
async fn test_navigation() {
let hand = SlideshowHand::with_slides_async(vec![
SlideContent { title: "Slide 1".to_string(), subtitle: None, content: vec![], notes: None, background: None },
SlideContent { title: "Slide 2".to_string(), subtitle: None, content: vec![], notes: None, background: None },
SlideContent { title: "Slide 3".to_string(), subtitle: None, content: vec![], notes: None, background: None },
]).await;
// Next
hand.execute_action(SlideshowAction::NextSlide).await.unwrap();
assert_eq!(hand.get_state().await.current_slide, 1);
// Goto
hand.execute_action(SlideshowAction::GotoSlide { slide_number: 2 }).await.unwrap();
assert_eq!(hand.get_state().await.current_slide, 2);
// Prev
hand.execute_action(SlideshowAction::PrevSlide).await.unwrap();
assert_eq!(hand.get_state().await.current_slide, 1);
}
#[tokio::test]
async fn test_spotlight() {
let hand = SlideshowHand::new();
let action = SlideshowAction::Spotlight {
element_id: "title".to_string(),
duration_ms: 2000,
};
let result = hand.execute_action(action).await.unwrap();
assert!(result.success);
}
#[tokio::test]
async fn test_laser() {
let hand = SlideshowHand::new();
let action = SlideshowAction::Laser {
x: 100.0,
y: 200.0,
duration_ms: 3000,
};
let result = hand.execute_action(action).await.unwrap();
assert!(result.success);
}
#[tokio::test]
async fn test_set_content() {
let hand = SlideshowHand::new();
let content = SlideContent {
title: "Test Slide".to_string(),
subtitle: Some("Subtitle".to_string()),
content: vec![ContentBlock::Text {
text: "Hello".to_string(),
style: None,
}],
notes: Some("Speaker notes".to_string()),
background: None,
};
let result = hand.execute_action(SlideshowAction::SetContent {
slide_number: 0,
content,
}).await.unwrap();
assert!(result.success);
assert_eq!(hand.get_state().await.total_slides, 1);
}
}