diff --git a/README.md b/README.md index 7be9aa8..846f3dd 100644 --- a/README.md +++ b/README.md @@ -193,7 +193,7 @@ MIT | 1. 智能工作流自动化 | ✅ 已完成 | 2026-02-23 | | 2. 多模态支持 | ✅ 已完成 | 2026-02-23 | | 7. 插件与集成 | ✅ 已完成 | 2026-02-23 | -| 3. 数据安全与合规 | 📋 待开发 | - | +| 3. 数据安全与合规 | ✅ 已完成 | 2026-02-23 | | 4. 协作与共享 | 📋 待开发 | - | | 5. 智能报告生成 | 📋 待开发 | - | | 6. 高级搜索与发现 | 📋 待开发 | - | diff --git a/STATUS.md b/STATUS.md index 23eb2ad..2b7f09b 100644 --- a/STATUS.md +++ b/STATUS.md @@ -1,10 +1,10 @@ # InsightFlow 开发状态 -**最后更新**: 2026-02-23 06:00 +**最后更新**: 2026-02-23 18:00 ## 当前阶段 -Phase 7: 插件与集成 - **已完成 ✅** +Phase 7: 数据安全与合规 - **已完成 ✅** ## 部署状态 @@ -99,41 +99,97 @@ Phase 7: 插件与集成 - **已完成 ✅** ### Phase 7 - 任务 7: 插件与集成 (已完成 ✅) - ✅ 创建 plugin_manager.py - 插件管理模块 - PluginManager: 插件管理主类 - - ChromeExtensionHandler: Chrome 插件 API 处理 + - ChromeExtensionHandler: Chrome 扩展 API 处理 + - 令牌创建、验证、撤销 + - 网页内容导入 - BotHandler: 飞书/钉钉机器人处理 + - 会话管理 + - 消息接收和发送 + - 音频文件处理 - WebhookIntegration: Zapier/Make Webhook 集成 + - 端点创建和管理 + - 事件触发 + - 认证支持 - WebDAVSync: WebDAV 同步管理 -- ✅ 创建 Chrome 扩展代码 - - manifest.json - 扩展配置 - - background.js - 后台脚本，处理右键菜单和消息 - - content.js - 内容脚本，页面交互和浮动按钮 - - content.css - 内容样式 - - popup.html/js - 弹出窗口 - - options.html/js - 设置页面 + - 同步配置管理 + - 连接测试 + - 项目数据同步 - ✅ 更新 schema.sql - 添加插件相关数据库表 - plugins: 插件配置表 + - plugin_configs: 插件详细配置表 - bot_sessions: 机器人会话表 - webhook_endpoints: Webhook 端点表 - webdav_syncs: WebDAV 同步配置表 - - plugin_activity_logs: 插件活动日志表 + - chrome_extension_tokens: Chrome 扩展令牌表 - ✅ 更新 main.py - 添加插件相关 API 端点 - GET/POST /api/v1/plugins - 插件管理 - - POST /api/v1/plugins/chrome/clip - Chrome 插件保存网页 - - POST /api/v1/bots/webhook/{platform} - 接收机器人消息 - - GET /api/v1/bots/sessions - 机器人会话列表 - - POST /api/v1/webhook-endpoints - 创建 Webhook 端点 - - POST /webhook/{type}/{token} - 接收外部 Webhook - - POST /api/v1/webdav-syncs - WebDAV 同步配置 - - POST /api/v1/webdav-syncs/{id}/test - 测试 WebDAV 连接 - - POST /api/v1/webdav-syncs/{id}/sync - 触发 WebDAV 同步 - - GET /api/v1/plugins/{id}/logs - 插件活动日志 + - POST /api/v1/plugins/chrome/tokens - 创建 Chrome 扩展令牌 + - GET /api/v1/plugins/chrome/tokens - 列出自令牌 + - DELETE /api/v1/plugins/chrome/tokens/{id} - 撤销令牌 + - POST /api/v1/plugins/chrome/import - 导入网页内容 + - POST /api/v1/plugins/bot/feishu/sessions - 创建飞书会话 + - POST /api/v1/plugins/bot/dingtalk/sessions - 创建钉钉会话 + - GET /api/v1/plugins/bot/{type}/sessions - 列出会话 + - POST /api/v1/plugins/bot/{type}/webhook - 接收机器人消息 + - POST /api/v1/plugins/bot/{type}/sessions/{id}/send - 发送消息 + - POST /api/v1/plugins/integrations/zapier - 创建 Zapier 端点 + - POST /api/v1/plugins/integrations/make - 创建 Make 端点 + - GET /api/v1/plugins/integrations/{type} - 列出集成端点 + - POST /api/v1/plugins/integrations/{id}/test - 测试端点 + - POST /api/v1/plugins/integrations/{id}/trigger - 手动触发 + - POST /api/v1/plugins/webdav - 创建 WebDAV 同步 + - GET /api/v1/plugins/webdav - 列出同步配置 + - POST /api/v1/plugins/webdav/{id}/test - 测试连接 + - POST /api/v1/plugins/webdav/{id}/sync - 执行同步 - ✅ 更新 requirements.txt - 添加插件依赖 - - beautifulsoup4: HTML 解析 - - webdavclient3: WebDAV 客户端 + - webdav4: WebDAV 客户端 + - urllib3: URL 处理 +- ✅ 创建 Chrome 扩展基础代码 + - manifest.json: 扩展配置 + - background.js: 后台脚本（右键菜单、同步） + - content.js: 内容脚本（页面提取） + - content.css: 内容样式 + - popup.html/js: 弹出窗口 + - options.html/js: 设置页面 + - README.md: 扩展说明文档 + +### Phase 7 - 任务 3: 数据安全与合规 (已完成 ✅) +- ✅ 创建 security_manager.py - 安全模块 + - SecurityManager: 安全管理主类 + - 审计日志系统 - 记录所有数据操作 + - 端到端加密 - AES-256-GCM 加密项目数据 + - 数据脱敏 - 支持手机号、邮箱、身份证等敏感信息脱敏 + - 数据访问策略 - 基于用户、角色、IP、时间的访问控制 + - 访问审批流程 - 敏感数据访问需要审批 +- ✅ 更新 schema.sql - 添加安全相关数据库表 + - audit_logs: 审计日志表 + - encryption_configs: 加密配置表 + - masking_rules: 脱敏规则表 + - data_access_policies: 数据访问策略表 + - access_requests: 访问请求表 +- ✅ 更新 main.py - 添加安全相关 API 端点 + - GET /api/v1/audit-logs - 查询审计日志 + - GET /api/v1/audit-logs/stats - 审计统计 + - POST /api/v1/projects/{id}/encryption/enable - 启用加密 + - POST /api/v1/projects/{id}/encryption/disable - 禁用加密 + - POST /api/v1/projects/{id}/encryption/verify - 验证密码 + - GET /api/v1/projects/{id}/encryption - 获取加密配置 + - POST /api/v1/projects/{id}/masking-rules - 创建脱敏规则 + - GET /api/v1/projects/{id}/masking-rules - 获取脱敏规则 + - PUT /api/v1/masking-rules/{id} - 更新脱敏规则 + - DELETE /api/v1/masking-rules/{id} - 删除脱敏规则 + - POST /api/v1/projects/{id}/masking/apply - 应用脱敏 + - POST /api/v1/projects/{id}/access-policies - 创建访问策略 + - GET /api/v1/projects/{id}/access-policies - 获取访问策略 + - POST /api/v1/access-policies/{id}/check - 检查访问权限 + - POST /api/v1/access-requests - 创建访问请求 + - POST /api/v1/access-requests/{id}/approve - 批准访问 + - POST /api/v1/access-requests/{id}/reject - 拒绝访问 +- ✅ 更新 requirements.txt - 添加 cryptography 依赖 ## 待完成 -Phase 7 任务 3: 数据安全与合规 +Phase 7 任务 4: 协作与共享 ## 技术债务 @@ -167,6 +223,24 @@ Phase 7 任务 3: 数据安全与合规 - 更新 main.py 添加插件相关 API 端点 - 更新 requirements.txt 添加插件依赖 +### 2026-02-23 (晚间) +- 完成 Phase 7 任务 3: 数据安全与合规 + - 创建 security_manager.py 安全模块 + - SecurityManager: 安全管理主类 + - 审计日志系统 - 记录所有数据操作 + - 端到端加密 - AES-256-GCM 加密项目数据 + - 数据脱敏 - 支持手机号、邮箱、身份证等敏感信息脱敏 + - 数据访问策略 - 基于用户、角色、IP、时间的访问控制 + - 访问审批流程 - 敏感数据访问需要审批 + - 更新 schema.sql 添加安全相关数据库表 + - audit_logs: 审计日志表 + - encryption_configs: 加密配置表 + - masking_rules: 脱敏规则表 + - data_access_policies: 数据访问策略表 + - access_requests: 访问请求表 + - 更新 main.py 添加安全相关 API 端点 + - 更新 requirements.txt 添加 cryptography 依赖 + ### 2026-02-23 (早间) - 完成 Phase 7 任务 2: 多模态支持 - 创建 multimodal_processor.py 模块 diff --git a/backend/__pycache__/main.cpython-312.pyc b/backend/__pycache__/main.cpython-312.pyc index a64fe67..2739f27 100644 Binary files a/backend/__pycache__/main.cpython-312.pyc and b/backend/__pycache__/main.cpython-312.pyc differ diff --git a/backend/__pycache__/plugin_manager.cpython-312.pyc b/backend/__pycache__/plugin_manager.cpython-312.pyc new file mode 100644 index 0000000..d6a27a5 Binary files /dev/null and b/backend/__pycache__/plugin_manager.cpython-312.pyc differ diff --git a/backend/__pycache__/security_manager.cpython-312.pyc b/backend/__pycache__/security_manager.cpython-312.pyc new file mode 100644 index 0000000..ccc9896 Binary files /dev/null and b/backend/__pycache__/security_manager.cpython-312.pyc differ diff --git a/backend/main.py b/backend/main.py index e0e4960..2d45278 100644 --- a/backend/main.py +++ b/backend/main.py @@ -155,6 +155,18 @@ except ImportError as e: print(f"Plugin Manager import error: {e}") PLUGIN_MANAGER_AVAILABLE = False +# Phase 7 Task 3: Security Manager +try: + from security_manager import ( + get_security_manager, SecurityManager, + AuditLog, EncryptionConfig, MaskingRule, DataAccessPolicy, AccessRequest, + AuditActionType, MaskingRuleType + ) + SECURITY_MANAGER_AVAILABLE = True +except ImportError as e: + print(f"Security Manager import error: {e}") + SECURITY_MANAGER_AVAILABLE = False + # FastAPI app with enhanced metadata for Swagger app = FastAPI( title="InsightFlow API", @@ -205,6 +217,7 @@ app = FastAPI( {"name": "Bot", "description": "飞书/钉钉机器人"}, {"name": "Integrations", "description": "Zapier/Make 集成"}, {"name": "WebDAV", "description": "WebDAV 同步"}, + {"name": "Security", "description": "数据安全与合规（加密、脱敏、审计）"}, {"name": "System", "description": "系统信息"}, ] ) @@ -4715,6 +4728,988 @@ class MultimodalProfileResponse(BaseModel): entity_name: str +# ==================== Phase 7 Task 7: Plugin Management Pydantic Models ==================== + +class PluginCreate(BaseModel): + name: str = Field(..., description="插件名称") + plugin_type: str = Field(..., description="插件类型: chrome_extension, feishu_bot, dingtalk_bot, zapier, make, webdav, custom") + project_id: str = Field(..., description="关联项目ID") + config: Dict = Field(default_factory=dict, description="插件配置") + + +class PluginUpdate(BaseModel): + name: Optional[str] = None + status: Optional[str] = None # active, inactive, error, pending + config: Optional[Dict] = None + + +class PluginResponse(BaseModel): + id: str + name: str + plugin_type: str + project_id: str + status: str + config: Dict + created_at: str + updated_at: str + last_used_at: Optional[str] + use_count: int + + +class PluginListResponse(BaseModel): + plugins: List[PluginResponse] + total: int + + +class ChromeExtensionTokenCreate(BaseModel): + name: str = Field(..., description="令牌名称") + project_id: Optional[str] = Field(default=None, description="关联项目ID") + permissions: List[str] = Field(default=["read"], description="权限列表: read, write, delete") + expires_days: Optional[int] = Field(default=None, description="过期天数") + + +class ChromeExtensionTokenResponse(BaseModel): + id: str + token: str = Field(..., description="令牌（仅显示一次）") + name: str + project_id: Optional[str] + permissions: List[str] + expires_at: Optional[str] + created_at: str + + +class ChromeExtensionImportRequest(BaseModel): + token: str = Field(..., description="Chrome扩展令牌") + url: str = Field(..., description="网页URL") + title: str = Field(..., description="网页标题") + content: str = Field(..., description="网页正文内容") + html_content: Optional[str] = Field(default=None, description="HTML内容（可选）") + + +class BotSessionCreate(BaseModel): + session_id: str = Field(..., description="群ID或会话ID") + session_name: str = Field(..., description="会话名称") + project_id: Optional[str] = Field(default=None, description="关联项目ID") + webhook_url: str = Field(default="", description="Webhook URL") + secret: str = Field(default="", description="签名密钥") + + +class BotSessionResponse(BaseModel): + id: str + bot_type: str + session_id: str + session_name: str + project_id: Optional[str] + webhook_url: str + is_active: bool + created_at: str + last_message_at: Optional[str] + message_count: int + + +class BotMessageRequest(BaseModel): + session_id: str = Field(..., description="会话ID") + msg_type: str = Field(default="text", description="消息类型: text, audio, file") + content: Dict = Field(default_factory=dict, description="消息内容") + + +class BotMessageResponse(BaseModel): + success: bool + response: str + error: Optional[str] = None + + +class WebhookEndpointCreate(BaseModel): + name: str = Field(..., description="端点名称") + endpoint_type: str = Field(..., description="端点类型: zapier, make, custom") + endpoint_url: str = Field(..., description="Webhook URL") + project_id: Optional[str] = Field(default=None, description="关联项目ID") + auth_type: str = Field(default="none", description="认证类型: none, api_key, oauth, custom") + auth_config: Dict = Field(default_factory=dict, description="认证配置") + trigger_events: List[str] = Field(default_factory=list, description="触发事件列表") + + +class WebhookEndpointResponse(BaseModel): + id: str + name: str + endpoint_type: str + endpoint_url: str + project_id: Optional[str] + auth_type: str + trigger_events: List[str] + is_active: bool + created_at: str + last_triggered_at: Optional[str] + trigger_count: int + + +class WebhookTestResponse(BaseModel): + success: bool + endpoint_id: str + message: str + + +class WebDAVSyncCreate(BaseModel): + name: str = Field(..., description="同步配置名称") + project_id: str = Field(..., description="关联项目ID") + server_url: str = Field(..., description="WebDAV服务器URL") + username: str = Field(..., description="用户名") + password: str = Field(..., description="密码") + remote_path: str = Field(default="/insightflow", description="远程路径") + sync_mode: str = Field(default="bidirectional", description="同步模式: bidirectional, upload_only, download_only") + sync_interval: int = Field(default=3600, description="同步间隔（秒）") + + +class WebDAVSyncResponse(BaseModel): + id: str + name: str + project_id: str + server_url: str + username: str + remote_path: str + sync_mode: str + sync_interval: int + last_sync_at: Optional[str] + last_sync_status: str + is_active: bool + created_at: str + sync_count: int + + +class WebDAVTestResponse(BaseModel): + success: bool + message: str + + +class WebDAVSyncResult(BaseModel): + success: bool + message: str + entities_count: Optional[int] = None + relations_count: Optional[int] = None + remote_path: Optional[str] = None + error: Optional[str] = None + + +# Plugin Manager singleton +_plugin_manager_instance = None + +def get_plugin_manager_instance(): + global _plugin_manager_instance + if _plugin_manager_instance is None and PLUGIN_MANAGER_AVAILABLE and DB_AVAILABLE: + db = get_db_manager() + _plugin_manager_instance = get_plugin_manager(db) + return _plugin_manager_instance + + +# ==================== Phase 7 Task 7: Plugin Management Endpoints ==================== + +@app.post("/api/v1/plugins", response_model=PluginResponse, tags=["Plugins"]) +async def create_plugin_endpoint(request: PluginCreate, _=Depends(verify_api_key)): + """ + 创建插件 + + 插件类型: + - **chrome_extension**: Chrome 扩展 + - **feishu_bot**: 飞书机器人 + - **dingtalk_bot**: 钉钉机器人 + - **zapier**: Zapier 集成 + - **make**: Make (Integromat) 集成 + - **webdav**: WebDAV 同步 + - **custom**: 自定义插件 + """ + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + + plugin = Plugin( + id=str(uuid.uuid4())[:8], + name=request.name, + plugin_type=request.plugin_type, + project_id=request.project_id, + config=request.config + ) + + created = manager.create_plugin(plugin) + + return PluginResponse( + id=created.id, + name=created.name, + plugin_type=created.plugin_type, + project_id=created.project_id, + status=created.status, + config=created.config, + created_at=created.created_at, + updated_at=created.updated_at, + last_used_at=created.last_used_at, + use_count=created.use_count + ) + + +@app.get("/api/v1/plugins", response_model=PluginListResponse, tags=["Plugins"]) +async def list_plugins_endpoint( + project_id: Optional[str] = None, + plugin_type: Optional[str] = None, + status: Optional[str] = None, + _=Depends(verify_api_key) +): + """获取插件列表""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + plugins = manager.list_plugins(project_id, plugin_type, status) + + return PluginListResponse( + plugins=[ + PluginResponse( + id=p.id, + name=p.name, + plugin_type=p.plugin_type, + project_id=p.project_id, + status=p.status, + config=p.config, + created_at=p.created_at, + updated_at=p.updated_at, + last_used_at=p.last_used_at, + use_count=p.use_count + ) + for p in plugins + ], + total=len(plugins) + ) + + +@app.get("/api/v1/plugins/{plugin_id}", response_model=PluginResponse, tags=["Plugins"]) +async def get_plugin_endpoint(plugin_id: str, _=Depends(verify_api_key)): + """获取插件详情""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + plugin = manager.get_plugin(plugin_id) + + if not plugin: + raise HTTPException(status_code=404, detail="Plugin not found") + + return PluginResponse( + id=plugin.id, + name=plugin.name, + plugin_type=plugin.plugin_type, + project_id=plugin.project_id, + status=plugin.status, + config=plugin.config, + created_at=plugin.created_at, + updated_at=plugin.updated_at, + last_used_at=plugin.last_used_at, + use_count=plugin.use_count + ) + + +@app.patch("/api/v1/plugins/{plugin_id}", response_model=PluginResponse, tags=["Plugins"]) +async def update_plugin_endpoint(plugin_id: str, request: PluginUpdate, _=Depends(verify_api_key)): + """更新插件""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + + update_data = {k: v for k, v in request.dict().items() if v is not None} + updated = manager.update_plugin(plugin_id, **update_data) + + if not updated: + raise HTTPException(status_code=404, detail="Plugin not found") + + return PluginResponse( + id=updated.id, + name=updated.name, + plugin_type=updated.plugin_type, + project_id=updated.project_id, + status=updated.status, + config=updated.config, + created_at=updated.created_at, + updated_at=updated.updated_at, + last_used_at=updated.last_used_at, + use_count=updated.use_count + ) + + +@app.delete("/api/v1/plugins/{plugin_id}", tags=["Plugins"]) +async def delete_plugin_endpoint(plugin_id: str, _=Depends(verify_api_key)): + """删除插件""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + success = manager.delete_plugin(plugin_id) + + if not success: + raise HTTPException(status_code=404, detail="Plugin not found") + + return {"success": True, "message": "Plugin deleted successfully"} + + +# ==================== Phase 7 Task 7: Chrome Extension Endpoints ==================== + +@app.post("/api/v1/plugins/chrome/tokens", response_model=ChromeExtensionTokenResponse, tags=["Chrome Extension"]) +async def create_chrome_token_endpoint(request: ChromeExtensionTokenCreate, _=Depends(verify_api_key)): + """ + 创建 Chrome 扩展令牌 + + 用于 Chrome 扩展验证和授权 + """ + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.CHROME_EXTENSION) + + if not handler: + raise HTTPException(status_code=503, detail="Chrome extension handler not available") + + token = handler.create_token( + name=request.name, + project_id=request.project_id, + permissions=request.permissions, + expires_days=request.expires_days + ) + + return ChromeExtensionTokenResponse( + id=token.id, + token=token.token, + name=token.name, + project_id=token.project_id, + permissions=token.permissions, + expires_at=token.expires_at, + created_at=token.created_at + ) + + +@app.get("/api/v1/plugins/chrome/tokens", tags=["Chrome Extension"]) +async def list_chrome_tokens_endpoint( + project_id: Optional[str] = None, + _=Depends(verify_api_key) +): + """列出 Chrome 扩展令牌""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.CHROME_EXTENSION) + + if not handler: + raise HTTPException(status_code=503, detail="Chrome extension handler not available") + + tokens = handler.list_tokens(project_id=project_id) + + return { + "tokens": [ + { + "id": t.id, + "name": t.name, + "project_id": t.project_id, + "permissions": t.permissions, + "expires_at": t.expires_at, + "created_at": t.created_at, + "last_used_at": t.last_used_at, + "use_count": t.use_count, + "is_revoked": t.is_revoked + } + for t in tokens + ], + "total": len(tokens) + } + + +@app.delete("/api/v1/plugins/chrome/tokens/{token_id}", tags=["Chrome Extension"]) +async def revoke_chrome_token_endpoint(token_id: str, _=Depends(verify_api_key)): + """撤销 Chrome 扩展令牌""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.CHROME_EXTENSION) + + if not handler: + raise HTTPException(status_code=503, detail="Chrome extension handler not available") + + success = handler.revoke_token(token_id) + + if not success: + raise HTTPException(status_code=404, detail="Token not found") + + return {"success": True, "message": "Token revoked successfully"} + + +@app.post("/api/v1/plugins/chrome/import", tags=["Chrome Extension"]) +async def chrome_import_webpage_endpoint(request: ChromeExtensionImportRequest): + """ + Chrome 扩展导入网页内容 + + 无需 API Key，使用 Chrome 扩展令牌验证 + """ + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.CHROME_EXTENSION) + + if not handler: + raise HTTPException(status_code=503, detail="Chrome extension handler not available") + + # 验证令牌 + token = handler.validate_token(request.token) + if not token: + raise HTTPException(status_code=401, detail="Invalid or expired token") + + # 导入网页 + result = await handler.import_webpage( + token=token, + url=request.url, + title=request.title, + content=request.content, + html_content=request.html_content + ) + + if not result["success"]: + raise HTTPException(status_code=400, detail=result.get("error", "Import failed")) + + return result + + +# ==================== Phase 7 Task 7: Bot Endpoints ==================== + +@app.post("/api/v1/plugins/bot/feishu/sessions", response_model=BotSessionResponse, tags=["Bot"]) +async def create_feishu_session_endpoint(request: BotSessionCreate, _=Depends(verify_api_key)): + """创建飞书机器人会话""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.FEISHU_BOT) + + if not handler: + raise HTTPException(status_code=503, detail="Feishu bot handler not available") + + session = handler.create_session( + session_id=request.session_id, + session_name=request.session_name, + project_id=request.project_id, + webhook_url=request.webhook_url, + secret=request.secret + ) + + return BotSessionResponse( + id=session.id, + bot_type=session.bot_type, + session_id=session.session_id, + session_name=session.session_name, + project_id=session.project_id, + webhook_url=session.webhook_url, + is_active=session.is_active, + created_at=session.created_at, + last_message_at=session.last_message_at, + message_count=session.message_count + ) + + +@app.post("/api/v1/plugins/bot/dingtalk/sessions", response_model=BotSessionResponse, tags=["Bot"]) +async def create_dingtalk_session_endpoint(request: BotSessionCreate, _=Depends(verify_api_key)): + """创建钉钉机器人会话""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.DINGTALK_BOT) + + if not handler: + raise HTTPException(status_code=503, detail="DingTalk bot handler not available") + + session = handler.create_session( + session_id=request.session_id, + session_name=request.session_name, + project_id=request.project_id, + webhook_url=request.webhook_url, + secret=request.secret + ) + + return BotSessionResponse( + id=session.id, + bot_type=session.bot_type, + session_id=session.session_id, + session_name=session.session_name, + project_id=session.project_id, + webhook_url=session.webhook_url, + is_active=session.is_active, + created_at=session.created_at, + last_message_at=session.last_message_at, + message_count=session.message_count + ) + + +@app.get("/api/v1/plugins/bot/{bot_type}/sessions", tags=["Bot"]) +async def list_bot_sessions_endpoint( + bot_type: str, + project_id: Optional[str] = None, + _=Depends(verify_api_key) +): + """列出机器人会话""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + + if bot_type == "feishu": + handler = manager.get_handler(PluginType.FEISHU_BOT) + elif bot_type == "dingtalk": + handler = manager.get_handler(PluginType.DINGTALK_BOT) + else: + raise HTTPException(status_code=400, detail="Invalid bot type. Must be feishu or dingtalk") + + if not handler: + raise HTTPException(status_code=503, detail=f"{bot_type} bot handler not available") + + sessions = handler.list_sessions(project_id=project_id) + + return { + "sessions": [ + { + "id": s.id, + "bot_type": s.bot_type, + "session_id": s.session_id, + "session_name": s.session_name, + "project_id": s.project_id, + "is_active": s.is_active, + "created_at": s.created_at, + "last_message_at": s.last_message_at, + "message_count": s.message_count + } + for s in sessions + ], + "total": len(sessions) + } + + +@app.post("/api/v1/plugins/bot/{bot_type}/webhook", tags=["Bot"]) +async def bot_webhook_endpoint(bot_type: str, request: Request): + """ + 机器人 Webhook 接收端点 + + 接收飞书/钉钉机器人的消息 + """ + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + + if bot_type == "feishu": + handler = manager.get_handler(PluginType.FEISHU_BOT) + elif bot_type == "dingtalk": + handler = manager.get_handler(PluginType.DINGTALK_BOT) + else: + raise HTTPException(status_code=400, detail="Invalid bot type") + + if not handler: + raise HTTPException(status_code=503, detail=f"{bot_type} bot handler not available") + + # 获取消息内容 + message = await request.json() + + # 获取会话ID（飞书和钉钉的格式不同） + if bot_type == "feishu": + session_id = message.get('chat_id') or message.get('open_chat_id') + else: # dingtalk + session_id = message.get('conversationId') or message.get('senderStaffId') + + if not session_id: + raise HTTPException(status_code=400, detail="Cannot identify session") + + # 获取会话 + session = handler.get_session(session_id) + if not session: + # 自动创建会话 + session = handler.create_session( + session_id=session_id, + session_name=f"Auto-{session_id[:8]}", + webhook_url="" + ) + + # 处理消息 + result = await handler.handle_message(session, message) + + # 如果配置了 webhook，发送回复 + if session.webhook_url and result.get("response"): + await handler.send_message(session, result["response"]) + + return result + + +@app.post("/api/v1/plugins/bot/{bot_type}/sessions/{session_id}/send", tags=["Bot"]) +async def send_bot_message_endpoint( + bot_type: str, + session_id: str, + message: str, + _=Depends(verify_api_key) +): + """发送消息到机器人会话""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + + if bot_type == "feishu": + handler = manager.get_handler(PluginType.FEISHU_BOT) + elif bot_type == "dingtalk": + handler = manager.get_handler(PluginType.DINGTALK_BOT) + else: + raise HTTPException(status_code=400, detail="Invalid bot type") + + if not handler: + raise HTTPException(status_code=503, detail=f"{bot_type} bot handler not available") + + session = handler.get_session(session_id) + if not session: + raise HTTPException(status_code=404, detail="Session not found") + + success = await handler.send_message(session, message) + + return {"success": success, "message": "Message sent" if success else "Failed to send message"} + + +# ==================== Phase 7 Task 7: Integration Endpoints ==================== + +@app.post("/api/v1/plugins/integrations/zapier", response_model=WebhookEndpointResponse, tags=["Integrations"]) +async def create_zapier_endpoint(request: WebhookEndpointCreate, _=Depends(verify_api_key)): + """创建 Zapier Webhook 端点""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.ZAPIER) + + if not handler: + raise HTTPException(status_code=503, detail="Zapier handler not available") + + endpoint = handler.create_endpoint( + name=request.name, + endpoint_url=request.endpoint_url, + project_id=request.project_id, + auth_type=request.auth_type, + auth_config=request.auth_config, + trigger_events=request.trigger_events + ) + + return WebhookEndpointResponse( + id=endpoint.id, + name=endpoint.name, + endpoint_type=endpoint.endpoint_type, + endpoint_url=endpoint.endpoint_url, + project_id=endpoint.project_id, + auth_type=endpoint.auth_type, + trigger_events=endpoint.trigger_events, + is_active=endpoint.is_active, + created_at=endpoint.created_at, + last_triggered_at=endpoint.last_triggered_at, + trigger_count=endpoint.trigger_count + ) + + +@app.post("/api/v1/plugins/integrations/make", response_model=WebhookEndpointResponse, tags=["Integrations"]) +async def create_make_endpoint(request: WebhookEndpointCreate, _=Depends(verify_api_key)): + """创建 Make (Integromat) Webhook 端点""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.MAKE) + + if not handler: + raise HTTPException(status_code=503, detail="Make handler not available") + + endpoint = handler.create_endpoint( + name=request.name, + endpoint_url=request.endpoint_url, + project_id=request.project_id, + auth_type=request.auth_type, + auth_config=request.auth_config, + trigger_events=request.trigger_events + ) + + return WebhookEndpointResponse( + id=endpoint.id, + name=endpoint.name, + endpoint_type=endpoint.endpoint_type, + endpoint_url=endpoint.endpoint_url, + project_id=endpoint.project_id, + auth_type=endpoint.auth_type, + trigger_events=endpoint.trigger_events, + is_active=endpoint.is_active, + created_at=endpoint.created_at, + last_triggered_at=endpoint.last_triggered_at, + trigger_count=endpoint.trigger_count + ) + + +@app.get("/api/v1/plugins/integrations/{endpoint_type}", tags=["Integrations"]) +async def list_integration_endpoints_endpoint( + endpoint_type: str, + project_id: Optional[str] = None, + _=Depends(verify_api_key) +): + """列出集成端点""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + + if endpoint_type == "zapier": + handler = manager.get_handler(PluginType.ZAPIER) + elif endpoint_type == "make": + handler = manager.get_handler(PluginType.MAKE) + else: + raise HTTPException(status_code=400, detail="Invalid endpoint type") + + if not handler: + raise HTTPException(status_code=503, detail=f"{endpoint_type} handler not available") + + endpoints = handler.list_endpoints(project_id=project_id) + + return { + "endpoints": [ + { + "id": e.id, + "name": e.name, + "endpoint_type": e.endpoint_type, + "endpoint_url": e.endpoint_url, + "project_id": e.project_id, + "auth_type": e.auth_type, + "trigger_events": e.trigger_events, + "is_active": e.is_active, + "created_at": e.created_at, + "last_triggered_at": e.last_triggered_at, + "trigger_count": e.trigger_count + } + for e in endpoints + ], + "total": len(endpoints) + } + + +@app.post("/api/v1/plugins/integrations/{endpoint_id}/test", response_model=WebhookTestResponse, tags=["Integrations"]) +async def test_integration_endpoint(endpoint_id: str, _=Depends(verify_api_key)): + """测试集成端点""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + + # 尝试获取端点（可能是 Zapier 或 Make） + handler = manager.get_handler(PluginType.ZAPIER) + endpoint = handler.get_endpoint(endpoint_id) if handler else None + + if not endpoint: + handler = manager.get_handler(PluginType.MAKE) + endpoint = handler.get_endpoint(endpoint_id) if handler else None + + if not endpoint: + raise HTTPException(status_code=404, detail="Endpoint not found") + + result = await handler.test_endpoint(endpoint) + + return WebhookTestResponse( + success=result["success"], + endpoint_id=endpoint_id, + message=result["message"] + ) + + +@app.post("/api/v1/plugins/integrations/{endpoint_id}/trigger", tags=["Integrations"]) +async def trigger_integration_endpoint( + endpoint_id: str, + event_type: str, + data: Dict, + _=Depends(verify_api_key) +): + """手动触发集成端点""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + + # 尝试获取端点（可能是 Zapier 或 Make） + handler = manager.get_handler(PluginType.ZAPIER) + endpoint = handler.get_endpoint(endpoint_id) if handler else None + + if not endpoint: + handler = manager.get_handler(PluginType.MAKE) + endpoint = handler.get_endpoint(endpoint_id) if handler else None + + if not endpoint: + raise HTTPException(status_code=404, detail="Endpoint not found") + + success = await handler.trigger(endpoint, event_type, data) + + return {"success": success, "message": "Triggered successfully" if success else "Trigger failed"} + + +# ==================== Phase 7 Task 7: WebDAV Endpoints ==================== + +@app.post("/api/v1/plugins/webdav", response_model=WebDAVSyncResponse, tags=["WebDAV"]) +async def create_webdav_sync_endpoint(request: WebDAVSyncCreate, _=Depends(verify_api_key)): + """ + 创建 WebDAV 同步配置 + + 支持与坚果云等 WebDAV 网盘同步项目数据 + """ + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.WEBDAV) + + if not handler: + raise HTTPException(status_code=503, detail="WebDAV handler not available") + + sync = handler.create_sync( + name=request.name, + project_id=request.project_id, + server_url=request.server_url, + username=request.username, + password=request.password, + remote_path=request.remote_path, + sync_mode=request.sync_mode, + sync_interval=request.sync_interval + ) + + return WebDAVSyncResponse( + id=sync.id, + name=sync.name, + project_id=sync.project_id, + server_url=sync.server_url, + username=sync.username, + remote_path=sync.remote_path, + sync_mode=sync.sync_mode, + sync_interval=sync.sync_interval, + last_sync_at=sync.last_sync_at, + last_sync_status=sync.last_sync_status, + is_active=sync.is_active, + created_at=sync.created_at, + sync_count=sync.sync_count + ) + + +@app.get("/api/v1/plugins/webdav", tags=["WebDAV"]) +async def list_webdav_syncs_endpoint( + project_id: Optional[str] = None, + _=Depends(verify_api_key) +): + """列出 WebDAV 同步配置""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.WEBDAV) + + if not handler: + raise HTTPException(status_code=503, detail="WebDAV handler not available") + + syncs = handler.list_syncs(project_id=project_id) + + return { + "syncs": [ + { + "id": s.id, + "name": s.name, + "project_id": s.project_id, + "server_url": s.server_url, + "username": s.username, + "remote_path": s.remote_path, + "sync_mode": s.sync_mode, + "sync_interval": s.sync_interval, + "last_sync_at": s.last_sync_at, + "last_sync_status": s.last_sync_status, + "is_active": s.is_active, + "created_at": s.created_at, + "sync_count": s.sync_count + } + for s in syncs + ], + "total": len(syncs) + } + + +@app.post("/api/v1/plugins/webdav/{sync_id}/test", response_model=WebDAVTestResponse, tags=["WebDAV"]) +async def test_webdav_connection_endpoint(sync_id: str, _=Depends(verify_api_key)): + """测试 WebDAV 连接""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.WEBDAV) + + if not handler: + raise HTTPException(status_code=503, detail="WebDAV handler not available") + + sync = handler.get_sync(sync_id) + if not sync: + raise HTTPException(status_code=404, detail="Sync configuration not found") + + result = await handler.test_connection(sync) + + return WebDAVTestResponse( + success=result["success"], + message=result.get("message") or result.get("error", "Unknown result") + ) + + +@app.post("/api/v1/plugins/webdav/{sync_id}/sync", response_model=WebDAVSyncResult, tags=["WebDAV"]) +async def sync_webdav_endpoint(sync_id: str, _=Depends(verify_api_key)): + """执行 WebDAV 同步""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.WEBDAV) + + if not handler: + raise HTTPException(status_code=503, detail="WebDAV handler not available") + + sync = handler.get_sync(sync_id) + if not sync: + raise HTTPException(status_code=404, detail="Sync configuration not found") + + result = await handler.sync_project(sync) + + return WebDAVSyncResult( + success=result["success"], + message=result.get("message") or result.get("error", "Sync completed"), + entities_count=result.get("entities_count"), + relations_count=result.get("relations_count"), + remote_path=result.get("remote_path"), + error=result.get("error") + ) + + +@app.delete("/api/v1/plugins/webdav/{sync_id}", tags=["WebDAV"]) +async def delete_webdav_sync_endpoint(sync_id: str, _=Depends(verify_api_key)): + """删除 WebDAV 同步配置""" + if not PLUGIN_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Plugin manager not available") + + manager = get_plugin_manager_instance() + handler = manager.get_handler(PluginType.WEBDAV) + + if not handler: + raise HTTPException(status_code=503, detail="WebDAV handler not available") + + success = handler.delete_sync(sync_id) + + if not success: + raise HTTPException(status_code=404, detail="Sync configuration not found") + + return {"success": True, "message": "WebDAV sync configuration deleted"} + + @app.get("/api/v1/openapi.json", include_in_schema=False) async def get_openapi(): """获取 OpenAPI 规范""" @@ -4728,7 +5723,12 @@ async def get_openapi(): ) -# ==================== Phase 7 Task 7: Plugin & Integration API ==================== +# Serve frontend - MUST be last to not override API routes +app.mount("/", StaticFiles(directory="frontend", html=True), name="frontend") + +if __name__ == "__main__": + import uvicorn + uvicorn.run(app, host="0.0.0.0", port=8000) class PluginCreateRequest(BaseModel): name: str @@ -5380,6 +6380,637 @@ async def get_plugin_logs( } +# ==================== Phase 7 Task 3: Security & Compliance API ==================== + +# Pydantic models for security API +class AuditLogResponse(BaseModel): + id: str + action_type: str + user_id: Optional[str] = None + user_ip: Optional[str] = None + resource_type: Optional[str] = None + resource_id: Optional[str] = None + action_details: Optional[str] = None + success: bool = True + error_message: Optional[str] = None + created_at: str + + +class AuditStatsResponse(BaseModel): + total_actions: int + success_count: int + failure_count: int + action_breakdown: Dict[str, Dict[str, int]] + + +class EncryptionEnableRequest(BaseModel): + master_password: str + + +class EncryptionConfigResponse(BaseModel): + id: str + project_id: str + is_enabled: bool + encryption_type: str + created_at: str + updated_at: str + + +class MaskingRuleCreateRequest(BaseModel): + name: str + rule_type: str # phone, email, id_card, bank_card, name, address, custom + pattern: Optional[str] = None + replacement: Optional[str] = None + description: Optional[str] = None + priority: int = 0 + + +class MaskingRuleResponse(BaseModel): + id: str + project_id: str + name: str + rule_type: str + pattern: str + replacement: str + is_active: bool + priority: int + description: Optional[str] = None + created_at: str + updated_at: str + + +class MaskingApplyRequest(BaseModel): + text: str + rule_types: Optional[List[str]] = None + + +class MaskingApplyResponse(BaseModel): + original_text: str + masked_text: str + applied_rules: List[str] + + +class AccessPolicyCreateRequest(BaseModel): + name: str + description: Optional[str] = None + allowed_users: Optional[List[str]] = None + allowed_roles: Optional[List[str]] = None + allowed_ips: Optional[List[str]] = None + time_restrictions: Optional[Dict] = None + max_access_count: Optional[int] = None + require_approval: bool = False + + +class AccessPolicyResponse(BaseModel): + id: str + project_id: str + name: str + description: Optional[str] = None + allowed_users: Optional[List[str]] = None + allowed_roles: Optional[List[str]] = None + allowed_ips: Optional[List[str]] = None + time_restrictions: Optional[Dict] = None + max_access_count: Optional[int] = None + require_approval: bool = False + is_active: bool = True + created_at: str + updated_at: str + + +class AccessRequestCreateRequest(BaseModel): + policy_id: str + request_reason: Optional[str] = None + expires_hours: int = 24 + + +class AccessRequestResponse(BaseModel): + id: str + policy_id: str + user_id: str + request_reason: Optional[str] = None + status: str + approved_by: Optional[str] = None + approved_at: Optional[str] = None + expires_at: Optional[str] = None + created_at: str + + +# ==================== Audit Logs API ==================== + +@app.get("/api/v1/audit-logs", response_model=List[AuditLogResponse], tags=["Security"]) +async def get_audit_logs( + user_id: Optional[str] = None, + resource_type: Optional[str] = None, + resource_id: Optional[str] = None, + action_type: Optional[str] = None, + start_time: Optional[str] = None, + end_time: Optional[str] = None, + success: Optional[bool] = None, + limit: int = 100, + offset: int = 0, + api_key: str = Depends(verify_api_key) +): + """查询审计日志""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + logs = manager.get_audit_logs( + user_id=user_id, + resource_type=resource_type, + resource_id=resource_id, + action_type=action_type, + start_time=start_time, + end_time=end_time, + success=success, + limit=limit, + offset=offset + ) + + return [ + AuditLogResponse( + id=log.id, + action_type=log.action_type, + user_id=log.user_id, + user_ip=log.user_ip, + resource_type=log.resource_type, + resource_id=log.resource_id, + action_details=log.action_details, + success=log.success, + error_message=log.error_message, + created_at=log.created_at + ) + for log in logs + ] + + +@app.get("/api/v1/audit-logs/stats", response_model=AuditStatsResponse, tags=["Security"]) +async def get_audit_stats( + start_time: Optional[str] = None, + end_time: Optional[str] = None, + api_key: str = Depends(verify_api_key) +): + """获取审计统计""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + stats = manager.get_audit_stats(start_time=start_time, end_time=end_time) + + return AuditStatsResponse(**stats) + + +# ==================== Encryption API ==================== + +@app.post("/api/v1/projects/{project_id}/encryption/enable", response_model=EncryptionConfigResponse, tags=["Security"]) +async def enable_project_encryption( + project_id: str, + request: EncryptionEnableRequest, + api_key: str = Depends(verify_api_key) +): + """启用项目端到端加密""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + + try: + config = manager.enable_encryption(project_id, request.master_password) + return EncryptionConfigResponse( + id=config.id, + project_id=config.project_id, + is_enabled=config.is_enabled, + encryption_type=config.encryption_type, + created_at=config.created_at, + updated_at=config.updated_at + ) + except RuntimeError as e: + raise HTTPException(status_code=400, detail=str(e)) + + +@app.post("/api/v1/projects/{project_id}/encryption/disable", tags=["Security"]) +async def disable_project_encryption( + project_id: str, + request: EncryptionEnableRequest, + api_key: str = Depends(verify_api_key) +): + """禁用项目加密""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + success = manager.disable_encryption(project_id, request.master_password) + + if not success: + raise HTTPException(status_code=400, detail="Invalid password or encryption not enabled") + + return {"success": True, "message": "Encryption disabled successfully"} + + +@app.post("/api/v1/projects/{project_id}/encryption/verify", tags=["Security"]) +async def verify_encryption_password( + project_id: str, + request: EncryptionEnableRequest, + api_key: str = Depends(verify_api_key) +): + """验证加密密码""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + is_valid = manager.verify_encryption_password(project_id, request.master_password) + + return {"valid": is_valid} + + +@app.get("/api/v1/projects/{project_id}/encryption", response_model=Optional[EncryptionConfigResponse], tags=["Security"]) +async def get_encryption_config( + project_id: str, + api_key: str = Depends(verify_api_key) +): + """获取项目加密配置""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + config = manager.get_encryption_config(project_id) + + if not config: + return None + + return EncryptionConfigResponse( + id=config.id, + project_id=config.project_id, + is_enabled=config.is_enabled, + encryption_type=config.encryption_type, + created_at=config.created_at, + updated_at=config.updated_at + ) + + +# ==================== Data Masking API ==================== + +@app.post("/api/v1/projects/{project_id}/masking-rules", response_model=MaskingRuleResponse, tags=["Security"]) +async def create_masking_rule( + project_id: str, + request: MaskingRuleCreateRequest, + api_key: str = Depends(verify_api_key) +): + """创建数据脱敏规则""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + + try: + rule_type = MaskingRuleType(request.rule_type) + except ValueError: + raise HTTPException(status_code=400, detail=f"Invalid rule type: {request.rule_type}") + + rule = manager.create_masking_rule( + project_id=project_id, + name=request.name, + rule_type=rule_type, + pattern=request.pattern, + replacement=request.replacement, + description=request.description, + priority=request.priority + ) + + return MaskingRuleResponse( + id=rule.id, + project_id=rule.project_id, + name=rule.name, + rule_type=rule.rule_type, + pattern=rule.pattern, + replacement=rule.replacement, + is_active=rule.is_active, + priority=rule.priority, + description=rule.description, + created_at=rule.created_at, + updated_at=rule.updated_at + ) + + +@app.get("/api/v1/projects/{project_id}/masking-rules", response_model=List[MaskingRuleResponse], tags=["Security"]) +async def get_masking_rules( + project_id: str, + active_only: bool = True, + api_key: str = Depends(verify_api_key) +): + """获取项目脱敏规则""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + rules = manager.get_masking_rules(project_id, active_only=active_only) + + return [ + MaskingRuleResponse( + id=rule.id, + project_id=rule.project_id, + name=rule.name, + rule_type=rule.rule_type, + pattern=rule.pattern, + replacement=rule.replacement, + is_active=rule.is_active, + priority=rule.priority, + description=rule.description, + created_at=rule.created_at, + updated_at=rule.updated_at + ) + for rule in rules + ] + + +@app.put("/api/v1/masking-rules/{rule_id}", response_model=MaskingRuleResponse, tags=["Security"]) +async def update_masking_rule( + rule_id: str, + name: Optional[str] = None, + pattern: Optional[str] = None, + replacement: Optional[str] = None, + is_active: Optional[bool] = None, + priority: Optional[int] = None, + description: Optional[str] = None, + api_key: str = Depends(verify_api_key) +): + """更新脱敏规则""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + + kwargs = {} + if name is not None: + kwargs["name"] = name + if pattern is not None: + kwargs["pattern"] = pattern + if replacement is not None: + kwargs["replacement"] = replacement + if is_active is not None: + kwargs["is_active"] = is_active + if priority is not None: + kwargs["priority"] = priority + if description is not None: + kwargs["description"] = description + + rule = manager.update_masking_rule(rule_id, **kwargs) + + if not rule: + raise HTTPException(status_code=404, detail="Masking rule not found") + + return MaskingRuleResponse( + id=rule.id, + project_id=rule.project_id, + name=rule.name, + rule_type=rule.rule_type, + pattern=rule.pattern, + replacement=rule.replacement, + is_active=rule.is_active, + priority=rule.priority, + description=rule.description, + created_at=rule.created_at, + updated_at=rule.updated_at + ) + + +@app.delete("/api/v1/masking-rules/{rule_id}", tags=["Security"]) +async def delete_masking_rule( + rule_id: str, + api_key: str = Depends(verify_api_key) +): + """删除脱敏规则""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + success = manager.delete_masking_rule(rule_id) + + if not success: + raise HTTPException(status_code=404, detail="Masking rule not found") + + return {"success": True, "message": "Masking rule deleted"} + + +@app.post("/api/v1/projects/{project_id}/masking/apply", response_model=MaskingApplyResponse, tags=["Security"]) +async def apply_masking( + project_id: str, + request: MaskingApplyRequest, + api_key: str = Depends(verify_api_key) +): + """应用脱敏规则到文本""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + + # 转换规则类型 + rule_types = None + if request.rule_types: + rule_types = [MaskingRuleType(rt) for rt in request.rule_types] + + masked_text = manager.apply_masking(request.text, project_id, rule_types) + + # 获取应用的规则 + rules = manager.get_masking_rules(project_id) + applied_rules = [r.name for r in rules if r.is_active] + + return MaskingApplyResponse( + original_text=request.text, + masked_text=masked_text, + applied_rules=applied_rules + ) + + +# ==================== Data Access Policy API ==================== + +@app.post("/api/v1/projects/{project_id}/access-policies", response_model=AccessPolicyResponse, tags=["Security"]) +async def create_access_policy( + project_id: str, + request: AccessPolicyCreateRequest, + api_key: str = Depends(verify_api_key) +): + """创建数据访问策略""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + + policy = manager.create_access_policy( + project_id=project_id, + name=request.name, + description=request.description, + allowed_users=request.allowed_users, + allowed_roles=request.allowed_roles, + allowed_ips=request.allowed_ips, + time_restrictions=request.time_restrictions, + max_access_count=request.max_access_count, + require_approval=request.require_approval + ) + + return AccessPolicyResponse( + id=policy.id, + project_id=policy.project_id, + name=policy.name, + description=policy.description, + allowed_users=json.loads(policy.allowed_users) if policy.allowed_users else None, + allowed_roles=json.loads(policy.allowed_roles) if policy.allowed_roles else None, + allowed_ips=json.loads(policy.allowed_ips) if policy.allowed_ips else None, + time_restrictions=json.loads(policy.time_restrictions) if policy.time_restrictions else None, + max_access_count=policy.max_access_count, + require_approval=policy.require_approval, + is_active=policy.is_active, + created_at=policy.created_at, + updated_at=policy.updated_at + ) + + +@app.get("/api/v1/projects/{project_id}/access-policies", response_model=List[AccessPolicyResponse], tags=["Security"]) +async def get_access_policies( + project_id: str, + active_only: bool = True, + api_key: str = Depends(verify_api_key) +): + """获取项目访问策略""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + policies = manager.get_access_policies(project_id, active_only=active_only) + + return [ + AccessPolicyResponse( + id=policy.id, + project_id=policy.project_id, + name=policy.name, + description=policy.description, + allowed_users=json.loads(policy.allowed_users) if policy.allowed_users else None, + allowed_roles=json.loads(policy.allowed_roles) if policy.allowed_roles else None, + allowed_ips=json.loads(policy.allowed_ips) if policy.allowed_ips else None, + time_restrictions=json.loads(policy.time_restrictions) if policy.time_restrictions else None, + max_access_count=policy.max_access_count, + require_approval=policy.require_approval, + is_active=policy.is_active, + created_at=policy.created_at, + updated_at=policy.updated_at + ) + for policy in policies + ] + + +@app.post("/api/v1/access-policies/{policy_id}/check", tags=["Security"]) +async def check_access_permission( + policy_id: str, + user_id: str, + user_ip: Optional[str] = None, + api_key: str = Depends(verify_api_key) +): + """检查访问权限""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + allowed, reason = manager.check_access_permission(policy_id, user_id, user_ip) + + return { + "allowed": allowed, + "reason": reason if not allowed else None + } + + +# ==================== Access Request API ==================== + +@app.post("/api/v1/access-requests", response_model=AccessRequestResponse, tags=["Security"]) +async def create_access_request( + request: AccessRequestCreateRequest, + user_id: str, # 实际应该从认证信息中获取 + api_key: str = Depends(verify_api_key) +): + """创建访问请求""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + + access_request = manager.create_access_request( + policy_id=request.policy_id, + user_id=user_id, + request_reason=request.request_reason, + expires_hours=request.expires_hours + ) + + return AccessRequestResponse( + id=access_request.id, + policy_id=access_request.policy_id, + user_id=access_request.user_id, + request_reason=access_request.request_reason, + status=access_request.status, + approved_by=access_request.approved_by, + approved_at=access_request.approved_at, + expires_at=access_request.expires_at, + created_at=access_request.created_at + ) + + +@app.post("/api/v1/access-requests/{request_id}/approve", response_model=AccessRequestResponse, tags=["Security"]) +async def approve_access_request( + request_id: str, + approved_by: str, + expires_hours: int = 24, + api_key: str = Depends(verify_api_key) +): + """批准访问请求""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + access_request = manager.approve_access_request(request_id, approved_by, expires_hours) + + if not access_request: + raise HTTPException(status_code=404, detail="Access request not found") + + return AccessRequestResponse( + id=access_request.id, + policy_id=access_request.policy_id, + user_id=access_request.user_id, + request_reason=access_request.request_reason, + status=access_request.status, + approved_by=access_request.approved_by, + approved_at=access_request.approved_at, + expires_at=access_request.expires_at, + created_at=access_request.created_at + ) + + +@app.post("/api/v1/access-requests/{request_id}/reject", response_model=AccessRequestResponse, tags=["Security"]) +async def reject_access_request( + request_id: str, + rejected_by: str, + api_key: str = Depends(verify_api_key) +): + """拒绝访问请求""" + if not SECURITY_MANAGER_AVAILABLE: + raise HTTPException(status_code=503, detail="Security manager not available") + + manager = get_security_manager() + access_request = manager.reject_access_request(request_id, rejected_by) + + if not access_request: + raise HTTPException(status_code=404, detail="Access request not found") + + return AccessRequestResponse( + id=access_request.id, + policy_id=access_request.policy_id, + user_id=access_request.user_id, + request_reason=access_request.request_reason, + status=access_request.status, + approved_by=access_request.approved_by, + approved_at=access_request.approved_at, + expires_at=access_request.expires_at, + created_at=access_request.created_at + ) + + # Serve frontend - MUST be last to not override API routes app.mount("/", StaticFiles(directory="frontend", html=True), name="frontend") diff --git a/backend/requirements.txt b/backend/requirements.txt index 89e7b1e..c8e266b 100644 --- a/backend/requirements.txt +++ b/backend/requirements.txt @@ -50,3 +50,6 @@ urllib3==2.2.0 # Phase 7: Plugin & Integration beautifulsoup4==4.12.3 webdavclient3==3.14.6 + +# Phase 7 Task 3: Security & Compliance +cryptography==42.0.0 diff --git a/backend/schema.sql b/backend/schema.sql index 3cacc91..98d8d34 100644 --- a/backend/schema.sql +++ b/backend/schema.sql @@ -539,3 +539,97 @@ CREATE INDEX IF NOT EXISTS idx_webdav_syncs_plugin ON webdav_syncs(plugin_id); CREATE INDEX IF NOT EXISTS idx_plugin_logs_plugin ON plugin_activity_logs(plugin_id); CREATE INDEX IF NOT EXISTS idx_plugin_logs_type ON plugin_activity_logs(activity_type); CREATE INDEX IF NOT EXISTS idx_plugin_logs_created ON plugin_activity_logs(created_at); + +-- ============================================ +-- Phase 7 Task 3: 数据安全与合规 +-- ============================================ + +-- 审计日志表 +CREATE TABLE IF NOT EXISTS audit_logs ( + id TEXT PRIMARY KEY, + action_type TEXT NOT NULL, -- create, read, update, delete, login, export, etc. + user_id TEXT, + user_ip TEXT, + user_agent TEXT, + resource_type TEXT, -- project, entity, transcript, api_key, etc. + resource_id TEXT, + action_details TEXT, -- JSON: 详细操作信息 + before_value TEXT, -- 变更前的值 + after_value TEXT, -- 变更后的值 + success INTEGER DEFAULT 1, -- 0 = 失败, 1 = 成功 + error_message TEXT, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP +); + +-- 加密配置表 +CREATE TABLE IF NOT EXISTS encryption_configs ( + id TEXT PRIMARY KEY, + project_id TEXT NOT NULL, + is_enabled INTEGER DEFAULT 0, + encryption_type TEXT DEFAULT 'aes-256-gcm', -- aes-256-gcm, chacha20-poly1305 + key_derivation TEXT DEFAULT 'pbkdf2', -- pbkdf2, argon2 + master_key_hash TEXT, -- 主密钥哈希（用于验证） + salt TEXT, -- 密钥派生盐值 + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + FOREIGN KEY (project_id) REFERENCES projects(id) +); + +-- 脱敏规则表 +CREATE TABLE IF NOT EXISTS masking_rules ( + id TEXT PRIMARY KEY, + project_id TEXT NOT NULL, + name TEXT NOT NULL, + rule_type TEXT NOT NULL, -- phone, email, id_card, bank_card, name, address, custom + pattern TEXT NOT NULL, -- 正则表达式 + replacement TEXT NOT NULL, -- 替换模板 + is_active INTEGER DEFAULT 1, + priority INTEGER DEFAULT 0, + description TEXT, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + FOREIGN KEY (project_id) REFERENCES projects(id) +); + +-- 数据访问策略表 +CREATE TABLE IF NOT EXISTS data_access_policies ( + id TEXT PRIMARY KEY, + project_id TEXT NOT NULL, + name TEXT NOT NULL, + description TEXT, + allowed_users TEXT, -- JSON array: 允许访问的用户ID列表 + allowed_roles TEXT, -- JSON array: 允许的角色列表 + allowed_ips TEXT, -- JSON array: 允许的IP模式列表 + time_restrictions TEXT, -- JSON: {"start_time": "09:00", "end_time": "18:00", "days_of_week": [0,1,2,3,4]} + max_access_count INTEGER, -- 最大访问次数限制 + require_approval INTEGER DEFAULT 0, -- 是否需要审批 + is_active INTEGER DEFAULT 1, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + FOREIGN KEY (project_id) REFERENCES projects(id) +); + +-- 访问请求表（用于需要审批的访问） +CREATE TABLE IF NOT EXISTS access_requests ( + id TEXT PRIMARY KEY, + policy_id TEXT NOT NULL, + user_id TEXT NOT NULL, + request_reason TEXT, + status TEXT DEFAULT 'pending', -- pending, approved, rejected, expired + approved_by TEXT, + approved_at TIMESTAMP, + expires_at TIMESTAMP, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + FOREIGN KEY (policy_id) REFERENCES data_access_policies(id) +); + +-- 数据安全相关索引 +CREATE INDEX IF NOT EXISTS idx_audit_logs_user ON audit_logs(user_id); +CREATE INDEX IF NOT EXISTS idx_audit_logs_resource ON audit_logs(resource_type, resource_id); +CREATE INDEX IF NOT EXISTS idx_audit_logs_action ON audit_logs(action_type); +CREATE INDEX IF NOT EXISTS idx_audit_logs_created ON audit_logs(created_at); +CREATE INDEX IF NOT EXISTS idx_encryption_project ON encryption_configs(project_id); +CREATE INDEX IF NOT EXISTS idx_masking_project ON masking_rules(project_id); +CREATE INDEX IF NOT EXISTS idx_access_policy_project ON data_access_policies(project_id); +CREATE INDEX IF NOT EXISTS idx_access_requests_policy ON access_requests(policy_id); +CREATE INDEX IF NOT EXISTS idx_access_requests_user ON access_requests(user_id); diff --git a/backend/security_manager.py b/backend/security_manager.py new file mode 100644 index 0000000..ab2d60e --- /dev/null +++ b/backend/security_manager.py @@ -0,0 +1,1232 @@ +""" +InsightFlow Phase 7 Task 3: 数据安全与合规模块 +Security Manager - 端到端加密、数据脱敏、审计日志 +""" + +import os +import json +import hashlib +import secrets +import base64 +import re +from datetime import datetime, timedelta +from typing import List, Optional, Dict, Any, Tuple +from dataclasses import dataclass, field, asdict +from enum import Enum +import sqlite3 + +# 加密相关 +try: + from cryptography.fernet import Fernet + from cryptography.hazmat.primitives import hashes + from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC + CRYPTO_AVAILABLE = True +except ImportError: + CRYPTO_AVAILABLE = False + print("Warning: cryptography not available, encryption features disabled") + + +class AuditActionType(Enum): + """审计动作类型""" + CREATE = "create" + READ = "read" + UPDATE = "update" + DELETE = "delete" + LOGIN = "login" + LOGOUT = "logout" + EXPORT = "export" + IMPORT = "import" + SHARE = "share" + PERMISSION_CHANGE = "permission_change" + ENCRYPTION_ENABLE = "encryption_enable" + ENCRYPTION_DISABLE = "encryption_disable" + DATA_MASKING = "data_masking" + API_KEY_CREATE = "api_key_create" + API_KEY_REVOKE = "api_key_revoke" + WORKFLOW_TRIGGER = "workflow_trigger" + WEBHOOK_SEND = "webhook_send" + BOT_MESSAGE = "bot_message" + + +class DataSensitivityLevel(Enum): + """数据敏感度级别""" + PUBLIC = "public" # 公开 + INTERNAL = "internal" # 内部 + CONFIDENTIAL = "confidential" # 机密 + SECRET = "secret" # 绝密 + + +class MaskingRuleType(Enum): + """脱敏规则类型""" + PHONE = "phone" # 手机号 + EMAIL = "email" # 邮箱 + ID_CARD = "id_card" # 身份证号 + BANK_CARD = "bank_card" # 银行卡号 + NAME = "name" # 姓名 + ADDRESS = "address" # 地址 + CUSTOM = "custom" # 自定义 + + +@dataclass +class AuditLog: + """审计日志条目""" + id: str + action_type: str + user_id: Optional[str] = None + user_ip: Optional[str] = None + user_agent: Optional[str] = None + resource_type: Optional[str] = None # project, entity, transcript, etc. + resource_id: Optional[str] = None + action_details: Optional[str] = None # JSON string + before_value: Optional[str] = None + after_value: Optional[str] = None + success: bool = True + error_message: Optional[str] = None + created_at: str = field(default_factory=lambda: datetime.now().isoformat()) + + def to_dict(self) -> Dict[str, Any]: + return asdict(self) + + +@dataclass +class EncryptionConfig: + """加密配置""" + id: str + project_id: str + is_enabled: bool = False + encryption_type: str = "aes-256-gcm" # aes-256-gcm, chacha20-poly1305 + key_derivation: str = "pbkdf2" # pbkdf2, argon2 + master_key_hash: Optional[str] = None # 主密钥哈希（用于验证） + salt: Optional[str] = None + created_at: str = field(default_factory=lambda: datetime.now().isoformat()) + updated_at: str = field(default_factory=lambda: datetime.now().isoformat()) + + def to_dict(self) -> Dict[str, Any]: + return asdict(self) + + +@dataclass +class MaskingRule: + """脱敏规则""" + id: str + project_id: str + name: str + rule_type: str # phone, email, id_card, bank_card, name, address, custom + pattern: str # 正则表达式 + replacement: str # 替换模板，如 "****" + is_active: bool = True + priority: int = 0 + description: Optional[str] = None + created_at: str = field(default_factory=lambda: datetime.now().isoformat()) + updated_at: str = field(default_factory=lambda: datetime.now().isoformat()) + + def to_dict(self) -> Dict[str, Any]: + return asdict(self) + + +@dataclass +class DataAccessPolicy: + """数据访问策略""" + id: str + project_id: str + name: str + description: Optional[str] = None + allowed_users: Optional[str] = None # JSON array of user IDs + allowed_roles: Optional[str] = None # JSON array of roles + allowed_ips: Optional[str] = None # JSON array of IP patterns + time_restrictions: Optional[str] = None # JSON: {"start_time": "09:00", "end_time": "18:00"} + max_access_count: Optional[int] = None # 最大访问次数 + require_approval: bool = False + is_active: bool = True + created_at: str = field(default_factory=lambda: datetime.now().isoformat()) + updated_at: str = field(default_factory=lambda: datetime.now().isoformat()) + + def to_dict(self) -> Dict[str, Any]: + return asdict(self) + + +@dataclass +class AccessRequest: + """访问请求（用于需要审批的访问）""" + id: str + policy_id: str + user_id: str + request_reason: Optional[str] = None + status: str = "pending" # pending, approved, rejected, expired + approved_by: Optional[str] = None + approved_at: Optional[str] = None + expires_at: Optional[str] = None + created_at: str = field(default_factory=lambda: datetime.now().isoformat()) + + def to_dict(self) -> Dict[str, Any]: + return asdict(self) + + +class SecurityManager: + """安全管理器""" + + # 预定义脱敏规则 + DEFAULT_MASKING_RULES = { + MaskingRuleType.PHONE: { + "pattern": r"(\d{3})\d{4}(\d{4})", + "replacement": r"\1****\2" + }, + MaskingRuleType.EMAIL: { + "pattern": r"(\w{1,3})\w+(@\w+\.\w+)", + "replacement": r"\1***\2" + }, + MaskingRuleType.ID_CARD: { + "pattern": r"(\d{6})\d{8}(\d{4})", + "replacement": r"\1********\2" + }, + MaskingRuleType.BANK_CARD: { + "pattern": r"(\d{4})\d+(\d{4})", + "replacement": r"\1 **** **** \2" + }, + MaskingRuleType.NAME: { + "pattern": r"([\u4e00-\u9fa5])[\u4e00-\u9fa5]+", + "replacement": r"\1**" + }, + MaskingRuleType.ADDRESS: { + "pattern": r"([\u4e00-\u9fa5]{2,})([\u4e00-\u9fa5]+路|街|巷|号)(.+)", + "replacement": r"\1\2***" + } + } + + def __init__(self, db_path: str = "insightflow.db"): + self.db_path = db_path + self._local = {} + self._init_db() + + def _init_db(self): + """初始化数据库表""" + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + # 审计日志表 + cursor.execute(""" + CREATE TABLE IF NOT EXISTS audit_logs ( + id TEXT PRIMARY KEY, + action_type TEXT NOT NULL, + user_id TEXT, + user_ip TEXT, + user_agent TEXT, + resource_type TEXT, + resource_id TEXT, + action_details TEXT, + before_value TEXT, + after_value TEXT, + success INTEGER DEFAULT 1, + error_message TEXT, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP + ) + """) + + # 加密配置表 + cursor.execute(""" + CREATE TABLE IF NOT EXISTS encryption_configs ( + id TEXT PRIMARY KEY, + project_id TEXT NOT NULL, + is_enabled INTEGER DEFAULT 0, + encryption_type TEXT DEFAULT 'aes-256-gcm', + key_derivation TEXT DEFAULT 'pbkdf2', + master_key_hash TEXT, + salt TEXT, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + FOREIGN KEY (project_id) REFERENCES projects(id) + ) + """) + + # 脱敏规则表 + cursor.execute(""" + CREATE TABLE IF NOT EXISTS masking_rules ( + id TEXT PRIMARY KEY, + project_id TEXT NOT NULL, + name TEXT NOT NULL, + rule_type TEXT NOT NULL, + pattern TEXT NOT NULL, + replacement TEXT NOT NULL, + is_active INTEGER DEFAULT 1, + priority INTEGER DEFAULT 0, + description TEXT, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + FOREIGN KEY (project_id) REFERENCES projects(id) + ) + """) + + # 数据访问策略表 + cursor.execute(""" + CREATE TABLE IF NOT EXISTS data_access_policies ( + id TEXT PRIMARY KEY, + project_id TEXT NOT NULL, + name TEXT NOT NULL, + description TEXT, + allowed_users TEXT, + allowed_roles TEXT, + allowed_ips TEXT, + time_restrictions TEXT, + max_access_count INTEGER, + require_approval INTEGER DEFAULT 0, + is_active INTEGER DEFAULT 1, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + FOREIGN KEY (project_id) REFERENCES projects(id) + ) + """) + + # 访问请求表 + cursor.execute(""" + CREATE TABLE IF NOT EXISTS access_requests ( + id TEXT PRIMARY KEY, + policy_id TEXT NOT NULL, + user_id TEXT NOT NULL, + request_reason TEXT, + status TEXT DEFAULT 'pending', + approved_by TEXT, + approved_at TIMESTAMP, + expires_at TIMESTAMP, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + FOREIGN KEY (policy_id) REFERENCES data_access_policies(id) + ) + """) + + # 创建索引 + cursor.execute("CREATE INDEX IF NOT EXISTS idx_audit_logs_user ON audit_logs(user_id)") + cursor.execute("CREATE INDEX IF NOT EXISTS idx_audit_logs_resource ON audit_logs(resource_type, resource_id)") + cursor.execute("CREATE INDEX IF NOT EXISTS idx_audit_logs_action ON audit_logs(action_type)") + cursor.execute("CREATE INDEX IF NOT EXISTS idx_audit_logs_created ON audit_logs(created_at)") + cursor.execute("CREATE INDEX IF NOT EXISTS idx_encryption_project ON encryption_configs(project_id)") + cursor.execute("CREATE INDEX IF NOT EXISTS idx_masking_project ON masking_rules(project_id)") + cursor.execute("CREATE INDEX IF NOT EXISTS idx_access_policy_project ON data_access_policies(project_id)") + + conn.commit() + conn.close() + + def _generate_id(self) -> str: + """生成唯一ID""" + return hashlib.sha256( + f"{datetime.now().isoformat()}{secrets.token_hex(16)}".encode() + ).hexdigest()[:32] + + # ==================== 审计日志 ==================== + + def log_audit( + self, + action_type: AuditActionType, + user_id: Optional[str] = None, + user_ip: Optional[str] = None, + user_agent: Optional[str] = None, + resource_type: Optional[str] = None, + resource_id: Optional[str] = None, + action_details: Optional[Dict] = None, + before_value: Optional[str] = None, + after_value: Optional[str] = None, + success: bool = True, + error_message: Optional[str] = None + ) -> AuditLog: + """记录审计日志""" + log = AuditLog( + id=self._generate_id(), + action_type=action_type.value, + user_id=user_id, + user_ip=user_ip, + user_agent=user_agent, + resource_type=resource_type, + resource_id=resource_id, + action_details=json.dumps(action_details) if action_details else None, + before_value=before_value, + after_value=after_value, + success=success, + error_message=error_message + ) + + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(""" + INSERT INTO audit_logs + (id, action_type, user_id, user_ip, user_agent, resource_type, resource_id, + action_details, before_value, after_value, success, error_message, created_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + """, ( + log.id, log.action_type, log.user_id, log.user_ip, log.user_agent, + log.resource_type, log.resource_id, log.action_details, + log.before_value, log.after_value, int(log.success), + log.error_message, log.created_at + )) + conn.commit() + conn.close() + + return log + + def get_audit_logs( + self, + user_id: Optional[str] = None, + resource_type: Optional[str] = None, + resource_id: Optional[str] = None, + action_type: Optional[str] = None, + start_time: Optional[str] = None, + end_time: Optional[str] = None, + success: Optional[bool] = None, + limit: int = 100, + offset: int = 0 + ) -> List[AuditLog]: + """查询审计日志""" + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + query = "SELECT * FROM audit_logs WHERE 1=1" + params = [] + + if user_id: + query += " AND user_id = ?" + params.append(user_id) + if resource_type: + query += " AND resource_type = ?" + params.append(resource_type) + if resource_id: + query += " AND resource_id = ?" + params.append(resource_id) + if action_type: + query += " AND action_type = ?" + params.append(action_type) + if start_time: + query += " AND created_at >= ?" + params.append(start_time) + if end_time: + query += " AND created_at <= ?" + params.append(end_time) + if success is not None: + query += " AND success = ?" + params.append(int(success)) + + query += " ORDER BY created_at DESC LIMIT ? OFFSET ?" + params.extend([limit, offset]) + + cursor.execute(query, params) + rows = cursor.fetchall() + conn.close() + + logs = [] + for row in cursor.description: + col_names = [desc[0] for desc in cursor.description] + break + else: + return logs + + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(query, params) + rows = cursor.fetchall() + + for row in rows: + log = AuditLog( + id=row[0], + action_type=row[1], + user_id=row[2], + user_ip=row[3], + user_agent=row[4], + resource_type=row[5], + resource_id=row[6], + action_details=row[7], + before_value=row[8], + after_value=row[9], + success=bool(row[10]), + error_message=row[11], + created_at=row[12] + ) + logs.append(log) + + conn.close() + return logs + + def get_audit_stats( + self, + start_time: Optional[str] = None, + end_time: Optional[str] = None + ) -> Dict[str, Any]: + """获取审计统计""" + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + query = "SELECT action_type, success, COUNT(*) FROM audit_logs WHERE 1=1" + params = [] + + if start_time: + query += " AND created_at >= ?" + params.append(start_time) + if end_time: + query += " AND created_at <= ?" + params.append(end_time) + + query += " GROUP BY action_type, success" + + cursor.execute(query, params) + rows = cursor.fetchall() + + stats = { + "total_actions": 0, + "success_count": 0, + "failure_count": 0, + "action_breakdown": {} + } + + for action_type, success, count in rows: + stats["total_actions"] += count + if success: + stats["success_count"] += count + else: + stats["failure_count"] += count + + if action_type not in stats["action_breakdown"]: + stats["action_breakdown"][action_type] = {"success": 0, "failure": 0} + + if success: + stats["action_breakdown"][action_type]["success"] += count + else: + stats["action_breakdown"][action_type]["failure"] += count + + conn.close() + return stats + + # ==================== 端到端加密 ==================== + + def _derive_key(self, password: str, salt: bytes) -> bytes: + """从密码派生密钥""" + if not CRYPTO_AVAILABLE: + raise RuntimeError("cryptography library not available") + + kdf = PBKDF2HMAC( + algorithm=hashes.SHA256(), + length=32, + salt=salt, + iterations=100000, + ) + return base64.urlsafe_b64encode(kdf.derive(password.encode())) + + def enable_encryption( + self, + project_id: str, + master_password: str + ) -> EncryptionConfig: + """启用项目加密""" + if not CRYPTO_AVAILABLE: + raise RuntimeError("cryptography library not available") + + # 生成盐值 + salt = secrets.token_hex(16) + + # 派生密钥并哈希（用于验证） + key = self._derive_key(master_password, salt.encode()) + key_hash = hashlib.sha256(key).hexdigest() + + config = EncryptionConfig( + id=self._generate_id(), + project_id=project_id, + is_enabled=True, + encryption_type="aes-256-gcm", + key_derivation="pbkdf2", + master_key_hash=key_hash, + salt=salt + ) + + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + # 检查是否已存在配置 + cursor.execute( + "SELECT id FROM encryption_configs WHERE project_id = ?", + (project_id,) + ) + existing = cursor.fetchone() + + if existing: + cursor.execute(""" + UPDATE encryption_configs + SET is_enabled = 1, encryption_type = ?, key_derivation = ?, + master_key_hash = ?, salt = ?, updated_at = ? + WHERE project_id = ? + """, ( + config.encryption_type, config.key_derivation, + config.master_key_hash, config.salt, + config.updated_at, project_id + )) + config.id = existing[0] + else: + cursor.execute(""" + INSERT INTO encryption_configs + (id, project_id, is_enabled, encryption_type, key_derivation, + master_key_hash, salt, created_at, updated_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) + """, ( + config.id, config.project_id, int(config.is_enabled), + config.encryption_type, config.key_derivation, + config.master_key_hash, config.salt, + config.created_at, config.updated_at + )) + + conn.commit() + conn.close() + + # 记录审计日志 + self.log_audit( + action_type=AuditActionType.ENCRYPTION_ENABLE, + resource_type="project", + resource_id=project_id, + action_details={"encryption_type": config.encryption_type} + ) + + return config + + def disable_encryption( + self, + project_id: str, + master_password: str + ) -> bool: + """禁用项目加密""" + # 验证密码 + if not self.verify_encryption_password(project_id, master_password): + return False + + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + cursor.execute(""" + UPDATE encryption_configs + SET is_enabled = 0, updated_at = ? + WHERE project_id = ? + """, (datetime.now().isoformat(), project_id)) + + conn.commit() + conn.close() + + # 记录审计日志 + self.log_audit( + action_type=AuditActionType.ENCRYPTION_DISABLE, + resource_type="project", + resource_id=project_id + ) + + return True + + def verify_encryption_password( + self, + project_id: str, + password: str + ) -> bool: + """验证加密密码""" + if not CRYPTO_AVAILABLE: + return False + + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + cursor.execute( + "SELECT master_key_hash, salt FROM encryption_configs WHERE project_id = ?", + (project_id,) + ) + row = cursor.fetchone() + conn.close() + + if not row: + return False + + stored_hash, salt = row + key = self._derive_key(password, salt.encode()) + key_hash = hashlib.sha256(key).hexdigest() + + return key_hash == stored_hash + + def get_encryption_config(self, project_id: str) -> Optional[EncryptionConfig]: + """获取加密配置""" + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + cursor.execute( + "SELECT * FROM encryption_configs WHERE project_id = ?", + (project_id,) + ) + row = cursor.fetchone() + conn.close() + + if not row: + return None + + return EncryptionConfig( + id=row[0], + project_id=row[1], + is_enabled=bool(row[2]), + encryption_type=row[3], + key_derivation=row[4], + master_key_hash=row[5], + salt=row[6], + created_at=row[7], + updated_at=row[8] + ) + + def encrypt_data( + self, + data: str, + password: str, + salt: Optional[str] = None + ) -> Tuple[str, str]: + """加密数据""" + if not CRYPTO_AVAILABLE: + raise RuntimeError("cryptography library not available") + + if salt is None: + salt = secrets.token_hex(16) + + key = self._derive_key(password, salt.encode()) + f = Fernet(key) + encrypted = f.encrypt(data.encode()) + + return base64.b64encode(encrypted).decode(), salt + + def decrypt_data( + self, + encrypted_data: str, + password: str, + salt: str + ) -> str: + """解密数据""" + if not CRYPTO_AVAILABLE: + raise RuntimeError("cryptography library not available") + + key = self._derive_key(password, salt.encode()) + f = Fernet(key) + decrypted = f.decrypt(base64.b64decode(encrypted_data)) + + return decrypted.decode() + + # ==================== 数据脱敏 ==================== + + def create_masking_rule( + self, + project_id: str, + name: str, + rule_type: MaskingRuleType, + pattern: Optional[str] = None, + replacement: Optional[str] = None, + description: Optional[str] = None, + priority: int = 0 + ) -> MaskingRule: + """创建脱敏规则""" + # 使用预定义规则或自定义规则 + if rule_type in self.DEFAULT_MASKING_RULES and not pattern: + default = self.DEFAULT_MASKING_RULES[rule_type] + pattern = default["pattern"] + replacement = replacement or default["replacement"] + + rule = MaskingRule( + id=self._generate_id(), + project_id=project_id, + name=name, + rule_type=rule_type.value, + pattern=pattern or "", + replacement=replacement or "****", + description=description, + priority=priority + ) + + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + cursor.execute(""" + INSERT INTO masking_rules + (id, project_id, name, rule_type, pattern, replacement, + is_active, priority, description, created_at, updated_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + """, ( + rule.id, rule.project_id, rule.name, rule.rule_type, + rule.pattern, rule.replacement, int(rule.is_active), + rule.priority, rule.description, rule.created_at, rule.updated_at + )) + + conn.commit() + conn.close() + + # 记录审计日志 + self.log_audit( + action_type=AuditActionType.DATA_MASKING, + resource_type="project", + resource_id=project_id, + action_details={"action": "create_rule", "rule_name": name} + ) + + return rule + + def get_masking_rules( + self, + project_id: str, + active_only: bool = True + ) -> List[MaskingRule]: + """获取脱敏规则""" + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + query = "SELECT * FROM masking_rules WHERE project_id = ?" + params = [project_id] + + if active_only: + query += " AND is_active = 1" + + query += " ORDER BY priority DESC" + + cursor.execute(query, params) + rows = cursor.fetchall() + conn.close() + + rules = [] + for row in rows: + rules.append(MaskingRule( + id=row[0], + project_id=row[1], + name=row[2], + rule_type=row[3], + pattern=row[4], + replacement=row[5], + is_active=bool(row[6]), + priority=row[7], + description=row[8], + created_at=row[9], + updated_at=row[10] + )) + + return rules + + def update_masking_rule( + self, + rule_id: str, + **kwargs + ) -> Optional[MaskingRule]: + """更新脱敏规则""" + allowed_fields = ["name", "pattern", "replacement", "is_active", "priority", "description"] + + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + set_clauses = [] + params = [] + + for key, value in kwargs.items(): + if key in allowed_fields: + set_clauses.append(f"{key} = ?") + params.append(int(value) if key == "is_active" else value) + + if not set_clauses: + conn.close() + return None + + set_clauses.append("updated_at = ?") + params.append(datetime.now().isoformat()) + params.append(rule_id) + + cursor.execute(f""" + UPDATE masking_rules + SET {', '.join(set_clauses)} + WHERE id = ? + """, params) + + conn.commit() + conn.close() + + # 获取更新后的规则 + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute("SELECT * FROM masking_rules WHERE id = ?", (rule_id,)) + row = cursor.fetchone() + conn.close() + + if not row: + return None + + return MaskingRule( + id=row[0], + project_id=row[1], + name=row[2], + rule_type=row[3], + pattern=row[4], + replacement=row[5], + is_active=bool(row[6]), + priority=row[7], + description=row[8], + created_at=row[9], + updated_at=row[10] + ) + + def delete_masking_rule(self, rule_id: str) -> bool: + """删除脱敏规则""" + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + cursor.execute("DELETE FROM masking_rules WHERE id = ?", (rule_id,)) + + success = cursor.rowcount > 0 + conn.commit() + conn.close() + + return success + + def apply_masking( + self, + text: str, + project_id: str, + rule_types: Optional[List[MaskingRuleType]] = None + ) -> str: + """应用脱敏规则到文本""" + rules = self.get_masking_rules(project_id) + + if not rules: + return text + + masked_text = text + + for rule in rules: + # 如果指定了规则类型，只应用指定类型的规则 + if rule_types and MaskingRuleType(rule.rule_type) not in rule_types: + continue + + try: + masked_text = re.sub( + rule.pattern, + rule.replacement, + masked_text + ) + except re.error: + # 忽略无效的正则表达式 + continue + + return masked_text + + def apply_masking_to_entity( + self, + entity_data: Dict[str, Any], + project_id: str + ) -> Dict[str, Any]: + """对实体数据应用脱敏""" + masked_data = entity_data.copy() + + # 对可能包含敏感信息的字段进行脱敏 + sensitive_fields = ["name", "definition", "description", "value"] + + for field in sensitive_fields: + if field in masked_data and isinstance(masked_data[field], str): + masked_data[field] = self.apply_masking(masked_data[field], project_id) + + return masked_data + + # ==================== 数据访问策略 ==================== + + def create_access_policy( + self, + project_id: str, + name: str, + description: Optional[str] = None, + allowed_users: Optional[List[str]] = None, + allowed_roles: Optional[List[str]] = None, + allowed_ips: Optional[List[str]] = None, + time_restrictions: Optional[Dict] = None, + max_access_count: Optional[int] = None, + require_approval: bool = False + ) -> DataAccessPolicy: + """创建数据访问策略""" + policy = DataAccessPolicy( + id=self._generate_id(), + project_id=project_id, + name=name, + description=description, + allowed_users=json.dumps(allowed_users) if allowed_users else None, + allowed_roles=json.dumps(allowed_roles) if allowed_roles else None, + allowed_ips=json.dumps(allowed_ips) if allowed_ips else None, + time_restrictions=json.dumps(time_restrictions) if time_restrictions else None, + max_access_count=max_access_count, + require_approval=require_approval + ) + + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + cursor.execute(""" + INSERT INTO data_access_policies + (id, project_id, name, description, allowed_users, allowed_roles, + allowed_ips, time_restrictions, max_access_count, require_approval, + is_active, created_at, updated_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + """, ( + policy.id, policy.project_id, policy.name, policy.description, + policy.allowed_users, policy.allowed_roles, policy.allowed_ips, + policy.time_restrictions, policy.max_access_count, + int(policy.require_approval), int(policy.is_active), + policy.created_at, policy.updated_at + )) + + conn.commit() + conn.close() + + return policy + + def get_access_policies( + self, + project_id: str, + active_only: bool = True + ) -> List[DataAccessPolicy]: + """获取数据访问策略""" + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + query = "SELECT * FROM data_access_policies WHERE project_id = ?" + params = [project_id] + + if active_only: + query += " AND is_active = 1" + + cursor.execute(query, params) + rows = cursor.fetchall() + conn.close() + + policies = [] + for row in rows: + policies.append(DataAccessPolicy( + id=row[0], + project_id=row[1], + name=row[2], + description=row[3], + allowed_users=row[4], + allowed_roles=row[5], + allowed_ips=row[6], + time_restrictions=row[7], + max_access_count=row[8], + require_approval=bool(row[9]), + is_active=bool(row[10]), + created_at=row[11], + updated_at=row[12] + )) + + return policies + + def check_access_permission( + self, + policy_id: str, + user_id: str, + user_ip: Optional[str] = None + ) -> Tuple[bool, Optional[str]]: + """检查访问权限""" + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + cursor.execute( + "SELECT * FROM data_access_policies WHERE id = ? AND is_active = 1", + (policy_id,) + ) + row = cursor.fetchone() + conn.close() + + if not row: + return False, "Policy not found or inactive" + + policy = DataAccessPolicy( + id=row[0], + project_id=row[1], + name=row[2], + description=row[3], + allowed_users=row[4], + allowed_roles=row[5], + allowed_ips=row[6], + time_restrictions=row[7], + max_access_count=row[8], + require_approval=bool(row[9]), + is_active=bool(row[10]), + created_at=row[11], + updated_at=row[12] + ) + + # 检查用户白名单 + if policy.allowed_users: + allowed = json.loads(policy.allowed_users) + if user_id not in allowed: + return False, "User not in allowed list" + + # 检查IP白名单 + if policy.allowed_ips and user_ip: + allowed_ips = json.loads(policy.allowed_ips) + ip_allowed = False + for ip_pattern in allowed_ips: + if self._match_ip_pattern(user_ip, ip_pattern): + ip_allowed = True + break + if not ip_allowed: + return False, "IP not in allowed list" + + # 检查时间限制 + if policy.time_restrictions: + restrictions = json.loads(policy.time_restrictions) + now = datetime.now() + + if "start_time" in restrictions and "end_time" in restrictions: + current_time = now.strftime("%H:%M") + if not (restrictions["start_time"] <= current_time <= restrictions["end_time"]): + return False, "Access not allowed at this time" + + if "days_of_week" in restrictions: + if now.weekday() not in restrictions["days_of_week"]: + return False, "Access not allowed on this day" + + # 检查是否需要审批 + if policy.require_approval: + # 检查是否有有效的访问请求 + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + cursor.execute(""" + SELECT * FROM access_requests + WHERE policy_id = ? AND user_id = ? AND status = 'approved' + AND (expires_at IS NULL OR expires_at > ?) + """, (policy_id, user_id, datetime.now().isoformat())) + + request = cursor.fetchone() + conn.close() + + if not request: + return False, "Access requires approval" + + return True, None + + def _match_ip_pattern(self, ip: str, pattern: str) -> bool: + """匹配IP模式（支持CIDR）""" + import ipaddress + + try: + if "/" in pattern: + # CIDR 表示法 + network = ipaddress.ip_network(pattern, strict=False) + return ipaddress.ip_address(ip) in network + else: + # 精确匹配 + return ip == pattern + except ValueError: + return ip == pattern + + def create_access_request( + self, + policy_id: str, + user_id: str, + request_reason: Optional[str] = None, + expires_hours: int = 24 + ) -> AccessRequest: + """创建访问请求""" + request = AccessRequest( + id=self._generate_id(), + policy_id=policy_id, + user_id=user_id, + request_reason=request_reason, + expires_at=(datetime.now() + timedelta(hours=expires_hours)).isoformat() + ) + + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + cursor.execute(""" + INSERT INTO access_requests + (id, policy_id, user_id, request_reason, status, expires_at, created_at) + VALUES (?, ?, ?, ?, ?, ?, ?) + """, ( + request.id, request.policy_id, request.user_id, + request.request_reason, request.status, request.expires_at, + request.created_at + )) + + conn.commit() + conn.close() + + return request + + def approve_access_request( + self, + request_id: str, + approved_by: str, + expires_hours: int = 24 + ) -> Optional[AccessRequest]: + """批准访问请求""" + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + expires_at = (datetime.now() + timedelta(hours=expires_hours)).isoformat() + approved_at = datetime.now().isoformat() + + cursor.execute(""" + UPDATE access_requests + SET status = 'approved', approved_by = ?, approved_at = ?, expires_at = ? + WHERE id = ? + """, (approved_by, approved_at, expires_at, request_id)) + + conn.commit() + + # 获取更新后的请求 + cursor.execute("SELECT * FROM access_requests WHERE id = ?", (request_id,)) + row = cursor.fetchone() + conn.close() + + if not row: + return None + + return AccessRequest( + id=row[0], + policy_id=row[1], + user_id=row[2], + request_reason=row[3], + status=row[4], + approved_by=row[5], + approved_at=row[6], + expires_at=row[7], + created_at=row[8] + ) + + def reject_access_request( + self, + request_id: str, + rejected_by: str + ) -> Optional[AccessRequest]: + """拒绝访问请求""" + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + cursor.execute(""" + UPDATE access_requests + SET status = 'rejected', approved_by = ? + WHERE id = ? + """, (rejected_by, request_id)) + + conn.commit() + + cursor.execute("SELECT * FROM access_requests WHERE id = ?", (request_id,)) + row = cursor.fetchone() + conn.close() + + if not row: + return None + + return AccessRequest( + id=row[0], + policy_id=row[1], + user_id=row[2], + request_reason=row[3], + status=row[4], + approved_by=row[5], + approved_at=row[6], + expires_at=row[7], + created_at=row[8] + ) + + +# 全局安全管理器实例 +_security_manager = None + + +def get_security_manager(db_path: str = "insightflow.db") -> SecurityManager: + """获取安全管理器实例""" + global _security_manager + if _security_manager is None: + _security_manager = SecurityManager(db_path) + return _security_manager diff --git a/chrome-extension/README.md b/chrome-extension/README.md new file mode 100644 index 0000000..ecd9d2b --- /dev/null +++ b/chrome-extension/README.md @@ -0,0 +1,113 @@ +# InsightFlow Chrome Extension + +一键将网页内容导入 InsightFlow 知识库的 Chrome 扩展。 + +## 功能特性 + +- 📄 **保存整个页面** - 自动提取正文内容并保存 +- ✏️ **保存选中内容** - 只保存您选中的文本 +- 🔗 **保存链接** - 快速保存网页链接 +- 🔄 **自动同步** - 剪辑后自动同步到服务器 +- 📎 **浮动按钮** - 页面右下角快速访问按钮 +- 🎯 **智能提取** - 自动识别正文，过滤广告和导航 + +## 安装方法 + +### 开发者模式安装 + +1. 打开 Chrome 浏览器，进入 `chrome://extensions/` +2. 开启右上角的"开发者模式" +3. 点击"加载已解压的扩展程序" +4. 选择 `chrome-extension` 文件夹 + +### 配置 + +1. 点击扩展图标，选择"设置" +2. 填写您的 InsightFlow 服务器地址 +3. 输入 Chrome 扩展令牌（从 InsightFlow 插件管理页面获取） +4. 点击"保存设置" +5. 点击"测试连接"验证配置 + +## 使用方法 + +### 方式一：扩展图标 +1. 点击浏览器工具栏上的 InsightFlow 图标 +2. 选择"保存整个页面"或"保存选中内容" + +### 方式二：右键菜单 +1. 在网页任意位置右键 +2. 选择"Clip page to InsightFlow"或"Clip selection to InsightFlow" + +### 方式三：浮动按钮 +1. 在页面右下角点击 📎 按钮 +2. 快速保存当前页面 + +### 方式四：快捷键 +- `Ctrl+Shift+S` (Windows/Linux) +- `Cmd+Shift+S` (Mac) + +## 文件结构 + +``` +chrome-extension/ +├── manifest.json # 扩展配置 +├── background.js # 后台脚本 +├── content.js # 内容脚本 +├── content.css # 内容样式 +├── popup.html # 弹出窗口 +├── popup.js # 弹出窗口脚本 +├── options.html # 设置页面 +├── options.js # 设置页面脚本 +└── icons/ # 图标文件夹 + ├── icon16.png + ├── icon48.png + └── icon128.png +``` + +## 开发 + +### 本地开发 + +1. 修改代码后，在 `chrome://extensions/` 页面点击刷新按钮 +2. 查看背景页控制台：扩展卡片 > 背景页 > 控制台 + +### 打包发布 + +1. 确保所有文件已保存 +2. 在 `chrome://extensions/` 页面点击"打包扩展程序" +3. 选择 `chrome-extension` 文件夹 +4. 生成 `.crx` 和 `.pem` 文件 + +## API 集成 + +扩展通过以下 API 与 InsightFlow 服务器通信： + +### 导入网页内容 +``` +POST /api/v1/plugins/chrome/import +Content-Type: application/json +X-API-Key: {token} + +{ + "token": "if_ext_xxx", + "url": "https://example.com/article", + "title": "文章标题", + "content": "正文内容...", + "html_content": "..." +} +``` + +### 健康检查 +``` +GET /api/v1/health +``` + +## 隐私说明 + +- 扩展仅在您主动点击时收集网页内容 +- 所有数据存储在您的 InsightFlow 服务器上 +- 不会收集或发送任何个人信息到第三方 + +## 许可证 + +MIT License \ No newline at end of file diff --git a/chrome-extension/background.js b/chrome-extension/background.js index 24e1174..7f169c2 100644 --- a/chrome-extension/background.js +++ b/chrome-extension/background.js @@ -1,217 +1,198 @@ // InsightFlow Chrome Extension - Background Script -// 处理后台任务、右键菜单、消息传递 +// 处理扩展的后台逻辑 -// 默认配置 -const DEFAULT_CONFIG = { - serverUrl: 'http://122.51.127.111:18000', - apiKey: '', - defaultProjectId: '' -}; - -// 初始化 chrome.runtime.onInstalled.addListener(() => { - // 创建右键菜单 - chrome.contextMenus.create({ - id: 'clipSelection', - title: '保存到 InsightFlow', - contexts: ['selection', 'page'] - }); - - // 初始化存储 - chrome.storage.sync.get(['insightflowConfig'], (result) => { - if (!result.insightflowConfig) { - chrome.storage.sync.set({ insightflowConfig: DEFAULT_CONFIG }); - } - }); + console.log('[InsightFlow] Extension installed'); + + // 创建右键菜单 + chrome.contextMenus.create({ + id: 'insightflow-clip-selection', + title: 'Clip selection to InsightFlow', + contexts: ['selection'] + }); + + chrome.contextMenus.create({ + id: 'insightflow-clip-page', + title: 'Clip page to InsightFlow', + contexts: ['page'] + }); + + chrome.contextMenus.create({ + id: 'insightflow-clip-link', + title: 'Clip link to InsightFlow', + contexts: ['link'] + }); }); // 处理右键菜单点击 chrome.contextMenus.onClicked.addListener((info, tab) => { - if (info.menuItemId === 'clipSelection') { - clipPage(tab, info.selectionText); - } + if (info.menuItemId === 'insightflow-clip-selection') { + clipSelection(tab); + } else if (info.menuItemId === 'insightflow-clip-page') { + clipPage(tab); + } else if (info.menuItemId === 'insightflow-clip-link') { + clipLink(tab, info.linkUrl); + } }); -// 处理扩展图标点击 -chrome.action.onClicked.addListener((tab) => { - clipPage(tab); -}); - -// 监听来自内容脚本的消息 +// 处理来自 popup 的消息 chrome.runtime.onMessage.addListener((request, sender, sendResponse) => { - if (request.action === 'clipPage') { - clipPage(sender.tab, request.selectionText); - sendResponse({ success: true }); - } else if (request.action === 'getConfig') { - chrome.storage.sync.get(['insightflowConfig'], (result) => { - sendResponse(result.insightflowConfig || DEFAULT_CONFIG); - }); - return true; // 保持消息通道开放 - } else if (request.action === 'saveConfig') { - chrome.storage.sync.set({ insightflowConfig: request.config }, () => { - sendResponse({ success: true }); - }); - return true; - } else if (request.action === 'fetchProjects') { - fetchProjects().then(projects => { - sendResponse({ success: true, projects }); - }).catch(error => { - sendResponse({ success: false, error: error.message }); - }); - return true; - } + if (request.action === 'clipPage') { + chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => { + if (tabs[0]) { + clipPage(tabs[0]).then(sendResponse); + } + }); + return true; + } else if (request.action === 'clipSelection') { + chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => { + if (tabs[0]) { + clipSelection(tabs[0]).then(sendResponse); + } + }); + return true; + } else if (request.action === 'openClipper') { + chrome.action.openPopup(); + } }); -// 剪藏页面 -async function clipPage(tab, selectionText = null) { - try { - // 获取配置 - const config = await getConfig(); - - if (!config.apiKey) { - showNotification('请先配置 API Key', '点击扩展图标打开设置'); - chrome.runtime.openOptionsPage(); - return; +// 剪辑整个页面 +async function clipPage(tab) { + try { + // 向 content script 发送消息提取内容 + const response = await chrome.tabs.sendMessage(tab.id, { action: 'extractContent' }); + + if (response.success) { + // 保存到本地存储 + await saveClip(response.data); + return { success: true, message: 'Page clipped successfully' }; + } + } catch (error) { + console.error('[InsightFlow] Failed to clip page:', error); + return { success: false, error: error.message }; } +} + +// 剪辑选中的内容 +async function clipSelection(tab) { + try { + const response = await chrome.tabs.sendMessage(tab.id, { action: 'getSelection' }); + + if (response.success && response.data) { + const clipData = { + url: tab.url, + title: tab.title, + content: response.data.text, + context: response.data.context, + contentType: 'selection', + extractedAt: new Date().toISOString() + }; + + await saveClip(clipData); + return { success: true, message: 'Selection clipped successfully' }; + } else { + return { success: false, error: 'No text selected' }; + } + } catch (error) { + console.error('[InsightFlow] Failed to clip selection:', error); + return { success: false, error: error.message }; + } +} + +// 剪辑链接 +async function clipLink(tab, linkUrl) { + const clipData = { + url: linkUrl, + title: linkUrl, + content: `Link: ${linkUrl}`, + sourceUrl: tab.url, + contentType: 'link', + extractedAt: new Date().toISOString() + }; - // 获取页面内容 - const [{ result }] = await chrome.scripting.executeScript({ - target: { tabId: tab.id }, - func: extractPageContent, - args: [selectionText] + await saveClip(clipData); + return { success: true, message: 'Link clipped successfully' }; +} + +// 保存剪辑内容 +async function saveClip(data) { + // 获取现有剪辑 + const result = await chrome.storage.local.get(['clips']); + const clips = result.clips || []; + + // 添加新剪辑 + clips.unshift({ + id: generateId(), + ...data, + synced: false }); - // 发送到 InsightFlow - const response = await sendToInsightFlow(config, result); - - if (response.success) { - showNotification('保存成功', '内容已导入 InsightFlow'); - } else { - showNotification('保存失败', response.error || '未知错误'); - } - } catch (error) { - console.error('Clip error:', error); - showNotification('保存失败', error.message); - } -} - -// 提取页面内容 -function extractPageContent(selectionText) { - const data = { - url: window.location.href, - title: document.title, - selection: selectionText, - timestamp: new Date().toISOString() - }; - - if (selectionText) { - // 只保存选中的文本 - data.content = selectionText; - data.contentType = 'selection'; - } else { - // 保存整个页面 - // 获取主要内容 - const article = document.querySelector('article') || - document.querySelector('main') || - document.querySelector('.content') || - document.querySelector('#content'); - - if (article) { - data.content = article.innerText; - data.contentType = 'article'; - } else { - // 获取 body 文本，但移除脚本和样式 - const bodyClone = document.body.cloneNode(true); - const scripts = bodyClone.querySelectorAll('script, style, nav, header, footer, aside'); - scripts.forEach(el => el.remove()); - data.content = bodyClone.innerText; - data.contentType = 'page'; + // 只保留最近 100 条 + if (clips.length > 100) { + clips.pop(); } - // 限制内容长度 - if (data.content.length > 50000) { - data.content = data.content.substring(0, 50000) + '...'; - data.truncated = true; + // 保存 + await chrome.storage.local.set({ clips }); + + // 尝试同步到服务器 + syncToServer(); +} + +// 同步到服务器 +async function syncToServer() { + const { serverUrl, apiKey } = await chrome.storage.sync.get(['serverUrl', 'apiKey']); + + if (!serverUrl || !apiKey) { + console.log('[InsightFlow] Server not configured, skipping sync'); + return; } - } - - // 获取元数据 - data.meta = { - description: document.querySelector('meta[name="description"]')?.content || '', - keywords: document.querySelector('meta[name="keywords"]')?.content || '', - author: document.querySelector('meta[name="author"]')?.content || '' - }; - - return data; -} - -// 发送到 InsightFlow -async function sendToInsightFlow(config, data) { - const url = `${config.serverUrl}/api/v1/plugins/chrome/clip`; - - const payload = { - url: data.url, - title: data.title, - content: data.content, - content_type: data.contentType, - meta: data.meta, - project_id: config.defaultProjectId || null - }; - - const response = await fetch(url, { - method: 'POST', - headers: { - 'Content-Type': 'application/json', - 'X-API-Key': config.apiKey - }, - body: JSON.stringify(payload) - }); - - if (!response.ok) { - const error = await response.text(); - throw new Error(error); - } - - return await response.json(); -} - -// 获取配置 -function getConfig() { - return new Promise((resolve) => { - chrome.storage.sync.get(['insightflowConfig'], (result) => { - resolve(result.insightflowConfig || DEFAULT_CONFIG); - }); - }); -} - -// 获取项目列表 -async function fetchProjects() { - const config = await getConfig(); - - if (!config.apiKey) { - throw new Error('请先配置 API Key'); - } - - const response = await fetch(`${config.serverUrl}/api/v1/projects`, { - headers: { - 'X-API-Key': config.apiKey + + const result = await chrome.storage.local.get(['clips']); + const clips = result.clips || []; + const unsyncedClips = clips.filter(c => !c.synced); + + if (unsyncedClips.length === 0) return; + + for (const clip of unsyncedClips) { + try { + const response = await fetch(`${serverUrl}/api/v1/plugins/chrome/import`, { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + 'X-API-Key': apiKey + }, + body: JSON.stringify({ + token: apiKey, + url: clip.url, + title: clip.title, + content: clip.content, + html_content: clip.html || null + }) + }); + + if (response.ok) { + clip.synced = true; + clip.syncedAt = new Date().toISOString(); + } + } catch (error) { + console.error('[InsightFlow] Sync failed:', error); + } } - }); - - if (!response.ok) { - throw new Error('获取项目列表失败'); - } - - const data = await response.json(); - return data.projects || []; + + // 更新存储 + await chrome.storage.local.set({ clips }); } -// 显示通知 -function showNotification(title, message) { - chrome.notifications.create({ - type: 'basic', - iconUrl: 'icons/icon128.png', - title, - message - }); -} \ No newline at end of file +// 生成唯一ID +function generateId() { + return Date.now().toString(36) + Math.random().toString(36).substr(2); +} + +// 定时同步（每5分钟） +chrome.alarms.create('syncClips', { periodInMinutes: 5 }); +chrome.alarms.onAlarm.addListener((alarm) => { + if (alarm.name === 'syncClips') { + syncToServer(); + } +}); \ No newline at end of file diff --git a/chrome-extension/content.css b/chrome-extension/content.css index 218164b..2ae81bf 100644 --- a/chrome-extension/content.css +++ b/chrome-extension/content.css @@ -1,141 +1,46 @@ /* InsightFlow Chrome Extension - Content Styles */ -.insightflow-float-btn { - position: absolute; - width: 36px; - height: 36px; - background: #4f46e5; - border-radius: 50%; - display: none; - align-items: center; - justify-content: center; - cursor: pointer; - box-shadow: 0 2px 8px rgba(0, 0, 0, 0.15); - z-index: 999999; - transition: transform 0.2s, box-shadow 0.2s; +#insightflow-clipper-btn { + animation: slideIn 0.3s ease-out; } -.insightflow-float-btn:hover { - transform: scale(1.1); - box-shadow: 0 4px 12px rgba(0, 0, 0, 0.2); +@keyframes slideIn { + from { + transform: translateX(100px); + opacity: 0; + } + to { + transform: translateX(0); + opacity: 1; + } } -.insightflow-float-btn svg { - color: white; +/* 选中文本高亮样式 */ +::selection { + background: rgba(102, 126, 234, 0.3); } -.insightflow-popup { - position: absolute; - width: 300px; - background: white; - border-radius: 8px; - box-shadow: 0 4px 20px rgba(0, 0, 0, 0.15); - z-index: 999999; - display: none; - font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; - font-size: 14px; +/* 剪辑成功提示 */ +.insightflow-toast { + position: fixed; + top: 20px; + right: 20px; + background: #4CAF50; + color: white; + padding: 15px 20px; + border-radius: 8px; + box-shadow: 0 4px 12px rgba(0,0,0,0.2); + z-index: 999999; + animation: toastSlideIn 0.3s ease-out; } -.insightflow-popup-header { - display: flex; - justify-content: space-between; - align-items: center; - padding: 12px 16px; - border-bottom: 1px solid #e5e7eb; - font-weight: 600; - color: #111827; -} - -.insightflow-close-btn { - background: none; - border: none; - font-size: 20px; - color: #6b7280; - cursor: pointer; - padding: 0; - width: 24px; - height: 24px; - display: flex; - align-items: center; - justify-content: center; -} - -.insightflow-close-btn:hover { - color: #111827; -} - -.insightflow-popup-content { - padding: 16px; -} - -.insightflow-text-preview { - background: #f3f4f6; - padding: 12px; - border-radius: 6px; - font-size: 13px; - color: #4b5563; - line-height: 1.5; - max-height: 120px; - overflow-y: auto; - margin-bottom: 12px; -} - -.insightflow-actions { - display: flex; - gap: 8px; -} - -.insightflow-btn { - flex: 1; - padding: 8px 12px; - border: 1px solid #d1d5db; - border-radius: 6px; - background: white; - color: #374151; - font-size: 13px; - cursor: pointer; - transition: all 0.2s; -} - -.insightflow-btn:hover { - background: #f9fafb; - border-color: #9ca3af; -} - -.insightflow-btn-primary { - background: #4f46e5; - border-color: #4f46e5; - color: white; -} - -.insightflow-btn-primary:hover { - background: #4338ca; - border-color: #4338ca; -} - -.insightflow-project-list { - max-height: 200px; - overflow-y: auto; -} - -.insightflow-project-item { - padding: 12px; - border-radius: 6px; - cursor: pointer; - transition: background 0.2s; -} - -.insightflow-project-item:hover { - background: #f3f4f6; -} - -.insightflow-project-name { - font-weight: 500; - color: #111827; - margin-bottom: 4px; -} - -.insightflow-project-desc { - font-size: 12px; - color: #6b7280; +@keyframes toastSlideIn { + from { + transform: translateX(100%); + opacity: 0; + } + to { + transform: translateX(0); + opacity: 1; + } } \ No newline at end of file diff --git a/chrome-extension/content.js b/chrome-extension/content.js index c95e4a6..499c840 100644 --- a/chrome-extension/content.js +++ b/chrome-extension/content.js @@ -1,204 +1,197 @@ // InsightFlow Chrome Extension - Content Script -// 在页面中注入，处理页面交互 +// 在网页上下文中运行，负责提取页面内容 (function() { - 'use strict'; - - // 防止重复注入 - if (window.insightflowInjected) return; - window.insightflowInjected = true; - - // 创建浮动按钮 - let floatingButton = null; - let selectionPopup = null; - - // 监听选中文本 - document.addEventListener('mouseup', handleSelection); - document.addEventListener('keyup', handleSelection); - - function handleSelection(e) { - const selection = window.getSelection(); - const text = selection.toString().trim(); - - if (text.length > 0) { - showFloatingButton(selection); - } else { - hideFloatingButton(); - hideSelectionPopup(); + 'use strict'; + + // 避免重复注入 + if (window.insightFlowInjected) return; + window.insightFlowInjected = true; + + // 提取页面主要内容 + function extractContent() { + const result = { + url: window.location.href, + title: document.title, + content: '', + html: document.documentElement.outerHTML, + meta: { + author: getMetaContent('author'), + description: getMetaContent('description'), + keywords: getMetaContent('keywords'), + publishedTime: getMetaContent('article:published_time') || getMetaContent('publishedDate'), + siteName: getMetaContent('og:site_name') || getMetaContent('application-name'), + language: document.documentElement.lang || 'unknown' + }, + extractedAt: new Date().toISOString() + }; + + // 尝试提取正文内容 + const article = extractArticleContent(); + result.content = article.text; + result.contentHtml = article.html; + result.wordCount = article.text.split(/\s+/).length; + + return result; } - } - - // 显示浮动按钮 - function showFloatingButton(selection) { - if (!floatingButton) { - floatingButton = document.createElement('div'); - floatingButton.className = 'insightflow-float-btn'; - floatingButton.innerHTML = ` - - - - `; - floatingButton.title = '保存到 InsightFlow'; - document.body.appendChild(floatingButton); - - floatingButton.addEventListener('click', () => { - const text = window.getSelection().toString().trim(); - if (text) { - showSelectionPopup(text); + + // 获取 meta 标签内容 + function getMetaContent(name) { + const meta = document.querySelector(`meta[name="${name}"], meta[property="${name}"]`); + return meta ? meta.getAttribute('content') : ''; + } + + // 提取文章正文（使用多种策略） + function extractArticleContent() { + // 策略1：使用 Readability 算法（简化版） + let bestElement = findBestElement(); + + if (bestElement) { + return { + text: cleanText(bestElement.innerText), + html: bestElement.innerHTML + }; } - }); + + // 策略2：回退到 body 内容 + const body = document.body; + return { + text: cleanText(body.innerText), + html: body.innerHTML + }; } - - // 定位按钮 - const range = selection.getRangeAt(0); - const rect = range.getBoundingClientRect(); - - floatingButton.style.left = `${rect.right + window.scrollX - 40}px`; - floatingButton.style.top = `${rect.top + window.scrollY - 45}px`; - floatingButton.style.display = 'flex'; - } - - // 隐藏浮动按钮 - function hideFloatingButton() { - if (floatingButton) { - floatingButton.style.display = 'none'; - } - } - - // 显示选择弹窗 - function showSelectionPopup(text) { - hideFloatingButton(); - - if (!selectionPopup) { - selectionPopup = document.createElement('div'); - selectionPopup.className = 'insightflow-popup'; - document.body.appendChild(selectionPopup); - } - - selectionPopup.innerHTML = ` -
- 保存到 InsightFlow - -
-
-
${escapeHtml(text.substring(0, 200))}${text.length > 200 ? '...' : ''}
-
- - -
-
- `; - - selectionPopup.style.display = 'block'; - - // 定位弹窗 - const selection = window.getSelection(); - const range = selection.getRangeAt(0); - const rect = range.getBoundingClientRect(); - - selectionPopup.style.left = `${Math.min(rect.left + window.scrollX, window.innerWidth - 320)}px`; - selectionPopup.style.top = `${rect.bottom + window.scrollY + 10}px`; - - // 绑定事件 - selectionPopup.querySelector('.insightflow-close-btn').addEventListener('click', hideSelectionPopup); - selectionPopup.querySelector('#if-save-quick').addEventListener('click', () => saveQuick(text)); - selectionPopup.querySelector('#if-save-select').addEventListener('click', () => saveWithProject(text)); - } - - // 隐藏选择弹窗 - function hideSelectionPopup() { - if (selectionPopup) { - selectionPopup.style.display = 'none'; - } - } - - // 快速保存 - async function saveQuick(text) { - hideSelectionPopup(); - - chrome.runtime.sendMessage({ - action: 'clipPage', - selectionText: text - }); - } - - // 选择项目保存 - async function saveWithProject(text) { - // 获取项目列表 - chrome.runtime.sendMessage({ action: 'fetchProjects' }, (response) => { - if (response.success && response.projects.length > 0) { - showProjectSelector(text, response.projects); - } else { - saveQuick(text); // 失败时快速保存 - } - }); - } - - // 显示项目选择器 - function showProjectSelector(text, projects) { - selectionPopup.innerHTML = ` -
- 选择项目 - -
-
-
- ${projects.map(p => ` -
-
${escapeHtml(p.name)}
-
${escapeHtml(p.description || '').substring(0, 50)}
-
- `).join('')} -
-
- `; - - selectionPopup.querySelector('.insightflow-close-btn').addEventListener('click', hideSelectionPopup); - - // 绑定项目选择事件 - selectionPopup.querySelectorAll('.insightflow-project-item').forEach(item => { - item.addEventListener('click', () => { - const projectId = item.dataset.id; - saveToProject(text, projectId); - }); - }); - } - - // 保存到指定项目 - async function saveToProject(text, projectId) { - hideSelectionPopup(); - - chrome.runtime.sendMessage({ - action: 'getConfig' - }, (config) => { - // 临时设置默认项目 - config.defaultProjectId = projectId; - chrome.runtime.sendMessage({ - action: 'saveConfig', - config: config - }, () => { - chrome.runtime.sendMessage({ - action: 'clipPage', - selectionText: text + + // 查找最佳内容元素（基于文本密度） + function findBestElement() { + const candidates = []; + const elements = document.querySelectorAll('article, [role="main"], .post-content, .entry-content, .article-content, #content, .content'); + + elements.forEach(el => { + const text = el.innerText || ''; + const linkDensity = calculateLinkDensity(el); + const textDensity = text.length / (el.innerHTML.length || 1); + + candidates.push({ + element: el, + score: text.length * textDensity * (1 - linkDensity), + textLength: text.length + }); }); - }); - }); - } - - // HTML 转义 - function escapeHtml(text) { - const div = document.createElement('div'); - div.textContent = text; - return div.innerHTML; - } - - // 点击页面其他地方关闭弹窗 - document.addEventListener('click', (e) => { - if (selectionPopup && !selectionPopup.contains(e.target) && - floatingButton && !floatingButton.contains(e.target)) { - hideSelectionPopup(); - hideFloatingButton(); + + // 按分数排序 + candidates.sort((a, b) => b.score - a.score); + + return candidates.length > 0 ? candidates[0].element : null; } - }); - + + // 计算链接密度 + function calculateLinkDensity(element) { + const links = element.getElementsByTagName('a'); + let linkLength = 0; + for (let link of links) { + linkLength += link.innerText.length; + } + const textLength = element.innerText.length || 1; + return linkLength / textLength; + } + + // 清理文本 + function cleanText(text) { + return text + .replace(/\s+/g, ' ') + .replace(/\n\s*\n/g, '\n\n') + .trim(); + } + + // 高亮选中的文本 + function highlightSelection() { + const selection = window.getSelection(); + if (selection.rangeCount > 0) { + const range = selection.getRangeAt(0); + const selectedText = selection.toString().trim(); + + if (selectedText.length > 0) { + return { + text: selectedText, + context: getSelectionContext(range) + }; + } + } + return null; + } + + // 获取选中内容的上下文 + function getSelectionContext(range) { + const container = range.commonAncestorContainer; + const element = container.nodeType === Node.TEXT_NODE ? container.parentElement : container; + + return { + tagName: element.tagName, + className: element.className, + id: element.id, + surroundingText: element.innerText.substring(0, 200) + }; + } + + // 监听来自 background 的消息 + chrome.runtime.onMessage.addListener((request, sender, sendResponse) => { + if (request.action === 'extractContent') { + const content = extractContent(); + sendResponse({ success: true, data: content }); + } else if (request.action === 'getSelection') { + const selection = highlightSelection(); + sendResponse({ success: true, data: selection }); + } else if (request.action === 'ping') { + sendResponse({ success: true, pong: true }); + } + return true; + }); + + // 添加浮动按钮（可选） + function addFloatingButton() { + const button = document.createElement('div'); + button.id = 'insightflow-clipper-btn'; + button.innerHTML = '📎'; + button.title = 'Clip to InsightFlow'; + button.style.cssText = ` + position: fixed; + bottom: 20px; + right: 20px; + width: 50px; + height: 50px; + background: #4CAF50; + border-radius: 50%; + display: flex; + align-items: center; + justify-content: center; + cursor: pointer; + box-shadow: 0 2px 10px rgba(0,0,0,0.3); + z-index: 999999; + font-size: 24px; + transition: transform 0.2s; + `; + + button.addEventListener('mouseenter', () => { + button.style.transform = 'scale(1.1)'; + }); + + button.addEventListener('mouseleave', () => { + button.style.transform = 'scale(1)'; + }); + + button.addEventListener('click', () => { + chrome.runtime.sendMessage({ action: 'openClipper' }); + }); + + document.body.appendChild(button); + } + + // 如果启用，添加浮动按钮 + chrome.storage.sync.get(['showFloatingButton'], (result) => { + if (result.showFloatingButton !== false) { + addFloatingButton(); + } + }); + + console.log('[InsightFlow] Content script loaded'); })(); \ No newline at end of file diff --git a/chrome-extension/manifest.json b/chrome-extension/manifest.json index b89bffc..96d45b7 100644 --- a/chrome-extension/manifest.json +++ b/chrome-extension/manifest.json @@ -2,7 +2,7 @@ "manifest_version": 3, "name": "InsightFlow Clipper", "version": "1.0.0", - "description": "将网页内容一键导入 InsightFlow 知识库", + "description": "一键将网页内容导入 InsightFlow 知识库", "permissions": [ "activeTab", "storage", @@ -21,11 +21,6 @@ "128": "icons/icon128.png" } }, - "icons": { - "16": "icons/icon16.png", - "48": "icons/icon48.png", - "128": "icons/icon128.png" - }, "background": { "service_worker": "background.js" }, @@ -36,11 +31,10 @@ "css": ["content.css"] } ], - "options_page": "options.html", - "web_accessible_resources": [ - { - "resources": ["icons/*.png"], - "matches": [""] - } - ] + "icons": { + "16": "icons/icon16.png", + "48": "icons/icon48.png", + "128": "icons/icon128.png" + }, + "options_page": "options.html" } \ No newline at end of file diff --git a/chrome-extension/options.html b/chrome-extension/options.html index 406a118..b8ddf0e 100644 --- a/chrome-extension/options.html +++ b/chrome-extension/options.html @@ -1,349 +1,247 @@ - - - InsightFlow Clipper 设置 - + + + InsightFlow Clipper - 设置 + -
-
-

⚙️ InsightFlow Clipper 设置

-

配置您的知识库连接

+
+
+

⚙️ InsightFlow 设置

+

配置您的知识库连接

+
+ +
+
+

+ 要使用 Chrome 扩展，您需要在 InsightFlow 中创建一个 Chrome 扩展令牌。 +
+ 前往 插件管理 > Chrome 扩展 创建令牌。 +

+
+ +
+
服务器配置
+ +
+ + +

您的 InsightFlow 服务器地址

+
+ +
+ + +

从 InsightFlow 获取的 Chrome 扩展令牌

+
+
+ +
+
偏好设置
+ +
+
+ + +
+

在网页右下角显示快速剪辑按钮

+
+ +
+
+ + +
+

剪辑后自动同步到服务器

+
+
+ +
+ + +
+ +
+
-
-
- - - - 服务器连接 -
- -
-

如何获取 API Key

-

- 1. 登录 InsightFlow 控制台
- 2. 进入「插件管理」页面
- 3. 创建 Chrome 插件并复制 API Key -

-
- -
- - -

InsightFlow 服务器的 URL 地址

-
- -
- - -

从 InsightFlow 控制台获取的插件 API Key

-
- -
- -
-
-
- -
-
- - - - - 默认设置 -
- -
- - -

保存内容时默认导入的项目

-
-
- -
-
- - - - 使用说明 -
- -
    -
  • - 保存当前页面 - 点击扩展图标 -
    • -
  • - 保存选中文本 - 右键 → 保存到 InsightFlow -
    • -
  • - 快速保存选中内容 - 选中文本后点击浮动按钮 -
    • -
  • - 选择项目保存 - 选中文本后点击"选择项目" -
    • -
-
- -
- - -
- - -
-

InsightFlow Clipper v1.0.0

-

打开 InsightFlow 控制台 | 帮助文档

-
-
- - + \ No newline at end of file diff --git a/chrome-extension/options.js b/chrome-extension/options.js index a5aa67b..aa06870 100644 --- a/chrome-extension/options.js +++ b/chrome-extension/options.js @@ -1,175 +1,105 @@ // InsightFlow Chrome Extension - Options Script document.addEventListener('DOMContentLoaded', () => { - const serverUrlInput = document.getElementById('serverUrl'); - const apiKeyInput = document.getElementById('apiKey'); - const defaultProjectSelect = document.getElementById('defaultProject'); - const testBtn = document.getElementById('testBtn'); - const testResult = document.getElementById('testResult'); - const saveBtn = document.getElementById('saveBtn'); - const resetBtn = document.getElementById('resetBtn'); - const openConsole = document.getElementById('openConsole'); - const helpLink = document.getElementById('helpLink'); - - // 加载配置 - loadConfig(); - - // 测试连接 - testBtn.addEventListener('click', async () => { - testBtn.disabled = true; - testBtn.textContent = '测试中...'; - testResult.className = ''; - testResult.style.display = 'none'; + // 加载保存的设置 + loadSettings(); - const serverUrl = serverUrlInput.value.trim(); - const apiKey = apiKeyInput.value.trim(); + // 绑定事件 + document.getElementById('saveBtn').addEventListener('click', saveSettings); + document.getElementById('testBtn').addEventListener('click', testConnection); +}); + +// 加载设置 +async function loadSettings() { + const settings = await chrome.storage.sync.get([ + 'serverUrl', + 'apiKey', + 'showFloatingButton', + 'autoSync' + ]); - if (!serverUrl || !apiKey) { - showTestResult('请填写服务器地址和 API Key', 'error'); - testBtn.disabled = false; - testBtn.textContent = '测试连接'; - return; + document.getElementById('serverUrl').value = settings.serverUrl || ''; + document.getElementById('apiKey').value = settings.apiKey || ''; + document.getElementById('showFloatingButton').checked = settings.showFloatingButton !== false; + document.getElementById('autoSync').checked = settings.autoSync !== false; +} + +// 保存设置 +async function saveSettings() { + const serverUrl = document.getElementById('serverUrl').value.trim(); + const apiKey = document.getElementById('apiKey').value.trim(); + const showFloatingButton = document.getElementById('showFloatingButton').checked; + const autoSync = document.getElementById('autoSync').checked; + + // 验证 + if (!serverUrl) { + showStatus('请输入服务器地址', 'error'); + return; } - try { - const response = await fetch(`${serverUrl}/api/v1/projects`, { - headers: { 'X-API-Key': apiKey } - }); - - if (response.ok) { - const data = await response.json(); - showTestResult(`连接成功！找到 ${data.projects?.length || 0} 个项目`, 'success'); - - // 更新项目列表 - updateProjectList(data.projects || []); - } else if (response.status === 401) { - showTestResult('API Key 无效，请检查', 'error'); - } else { - showTestResult(`连接失败: HTTP ${response.status}`, 'error'); - } - } catch (error) { - showTestResult(`连接错误: ${error.message}`, 'error'); + if (!apiKey) { + showStatus('请输入 API 令牌', 'error'); + return; } - testBtn.disabled = false; - testBtn.textContent = '测试连接'; - }); - - // 保存设置 - saveBtn.addEventListener('click', async () => { - const config = { - serverUrl: serverUrlInput.value.trim(), - apiKey: apiKeyInput.value.trim(), - defaultProjectId: defaultProjectSelect.value - }; - - if (!config.serverUrl) { - alert('请填写服务器地址'); - return; + // 确保 URL 格式正确 + let formattedUrl = serverUrl; + if (!formattedUrl.startsWith('http://') && !formattedUrl.startsWith('https://')) { + formattedUrl = 'https://' + formattedUrl; } - await chrome.storage.sync.set({ insightflowConfig: config }); + // 移除末尾的斜杠 + formattedUrl = formattedUrl.replace(/\/$/, ''); - // 显示保存成功 - saveBtn.textContent = '已保存 ✓'; - saveBtn.classList.add('btn-success'); - - setTimeout(() => { - saveBtn.textContent = '保存设置'; - saveBtn.classList.remove('btn-success'); - }, 2000); - }); - - // 重置设置 - resetBtn.addEventListener('click', () => { - if (confirm('确定要重置所有设置吗？')) { - const defaultConfig = { - serverUrl: 'http://122.51.127.111:18000', - apiKey: '', - defaultProjectId: '' - }; - - chrome.storage.sync.set({ insightflowConfig: defaultConfig }, () => { - loadConfig(); - showTestResult('设置已重置', 'success'); - }); - } - }); - - // 打开控制台 - openConsole.addEventListener('click', (e) => { - e.preventDefault(); - const serverUrl = serverUrlInput.value.trim(); - if (serverUrl) { - chrome.tabs.create({ url: serverUrl }); - } - }); - - // 帮助链接 - helpLink.addEventListener('click', (e) => { - e.preventDefault(); - const serverUrl = serverUrlInput.value.trim(); - if (serverUrl) { - chrome.tabs.create({ url: `${serverUrl}/docs` }); - } - }); - - // 加载配置 - async function loadConfig() { - const result = await chrome.storage.sync.get(['insightflowConfig']); - const config = result.insightflowConfig || { - serverUrl: 'http://122.51.127.111:18000', - apiKey: '', - defaultProjectId: '' - }; - - serverUrlInput.value = config.serverUrl; - apiKeyInput.value = config.apiKey; - - // 如果有 API Key，加载项目列表 - if (config.apiKey) { - loadProjects(config); - } - } - - // 加载项目列表 - async function loadProjects(config) { - try { - const response = await fetch(`${config.serverUrl}/api/v1/projects`, { - headers: { 'X-API-Key': config.apiKey } - }); - - if (response.ok) { - const data = await response.json(); - updateProjectList(data.projects || [], config.defaultProjectId); - } - } catch (error) { - console.error('Failed to load projects:', error); - } - } - - // 更新项目列表 - function updateProjectList(projects, selectedId = '') { - let html = ''; - - projects.forEach(project => { - const selected = project.id === selectedId ? 'selected' : ''; - html += ``; + // 保存 + await chrome.storage.sync.set({ + serverUrl: formattedUrl, + apiKey: apiKey, + showFloatingButton: showFloatingButton, + autoSync: autoSync }); - defaultProjectSelect.innerHTML = html; - } - - // 显示测试结果 - function showTestResult(message, type) { - testResult.textContent = message; - testResult.className = type; - } - - // HTML 转义 - function escapeHtml(text) { - const div = document.createElement('div'); - div.textContent = text; - return div.innerHTML; - } -}); \ No newline at end of file + showStatus('设置已保存！', 'success'); +} + +// 测试连接 +async function testConnection() { + const serverUrl = document.getElementById('serverUrl').value.trim(); + const apiKey = document.getElementById('apiKey').value.trim(); + + if (!serverUrl || !apiKey) { + showStatus('请先填写服务器地址和 API 令牌', 'error'); + return; + } + + showStatus('正在测试连接...', ''); + + try { + const response = await fetch(`${serverUrl}/api/v1/health`, { + method: 'GET', + headers: { + 'Content-Type': 'application/json' + } + }); + + if (response.ok) { + const data = await response.json(); + showStatus(`连接成功！服务器版本: ${data.version || 'unknown'}`, 'success'); + } else { + showStatus('连接失败：服务器返回错误', 'error'); + } + } catch (error) { + showStatus('连接失败：' + error.message, 'error'); + } +} + +// 显示状态 +function showStatus(message, type) { + const statusEl = document.getElementById('status'); + statusEl.textContent = message; + statusEl.className = 'status'; + + if (type) { + statusEl.classList.add(type); + } +} \ No newline at end of file diff --git a/chrome-extension/popup.html b/chrome-extension/popup.html index 39d5c12..8452eda 100644 --- a/chrome-extension/popup.html +++ b/chrome-extension/popup.html @@ -1,258 +1,276 @@ - - - InsightFlow Clipper - + + + InsightFlow Clipper + -
-

🧠 InsightFlow

-

一键保存网页到知识库

-
- -
-
- -
-
-
- 连接中... -
- - +
+

📎 InsightFlow

+

一键保存网页到知识库

- - - - -
-
-
0
-
已保存
-
-
-
0
-
项目数
-
-
-
0
-
今日
-
+
+
+
加载中...
+
+
+
字数: 0
+
待同步: 0
+
+
+ +
+ + +
+ +
+ +
+ +
正在处理...
+
+ +
+ +
+ ⚙️ 设置 +
-
- -
- 打开 InsightFlow 控制台 → -
- - + + \ No newline at end of file diff --git a/chrome-extension/popup.js b/chrome-extension/popup.js index 6376a42..6cf99b2 100644 --- a/chrome-extension/popup.js +++ b/chrome-extension/popup.js @@ -1,195 +1,154 @@ // InsightFlow Chrome Extension - Popup Script document.addEventListener('DOMContentLoaded', async () => { - const clipBtn = document.getElementById('clipBtn'); - const settingsBtn = document.getElementById('settingsBtn'); - const projectSelect = document.getElementById('projectSelect'); - const statusDot = document.getElementById('statusDot'); - const statusText = document.getElementById('statusText'); - const messageEl = document.getElementById('message'); - const openDashboard = document.getElementById('openDashboard'); - - // 加载配置和项目列表 - await loadConfig(); - - // 保存当前页面按钮 - clipBtn.addEventListener('click', async () => { + // 获取当前标签页信息 const [tab] = await chrome.tabs.query({ active: true, currentWindow: true }); - // 更新按钮状态 - clipBtn.disabled = true; - clipBtn.innerHTML = ' 保存中...'; + // 更新页面信息 + document.getElementById('pageTitle').textContent = tab.title || '未知标题'; + document.getElementById('pageUrl').textContent = tab.url || ''; - // 保存选中的项目 - const projectId = projectSelect.value; - if (projectId) { - const config = await getConfig(); - config.defaultProjectId = projectId; - await saveConfig(config); - } + // 获取页面统计 + updateStats(); - // 发送剪藏请求 - chrome.runtime.sendMessage({ - action: 'clipPage' - }, (response) => { - clipBtn.disabled = false; - clipBtn.innerHTML = ` - - - - 保存当前页面 - `; - - if (response && response.success) { - showMessage('保存成功！', 'success'); - updateStats(); - } else { - showMessage(response?.error || '保存失败', 'error'); - } - }); - }); - - // 设置按钮 - settingsBtn.addEventListener('click', () => { - chrome.runtime.openOptionsPage(); - }); - - // 打开控制台 - openDashboard.addEventListener('click', async (e) => { - e.preventDefault(); - const config = await getConfig(); - chrome.tabs.create({ url: config.serverUrl }); - }); + // 加载最近的剪辑 + loadRecentClips(); + + // 绑定按钮事件 + document.getElementById('clipPageBtn').addEventListener('click', clipPage); + document.getElementById('clipSelectionBtn').addEventListener('click', clipSelection); + document.getElementById('openOptions').addEventListener('click', openOptions); }); -// 加载配置 -async function loadConfig() { - const config = await getConfig(); - - // 检查连接状态 - checkConnection(config); - - // 加载项目列表 - loadProjects(config); - - // 更新统计 - updateStats(); -} - -// 检查连接状态 -async function checkConnection(config) { - const statusDot = document.getElementById('statusDot'); - const statusText = document.getElementById('statusText'); - - if (!config.apiKey) { - statusDot.classList.add('error'); - statusText.textContent = '未配置 API Key'; - return; - } - - try { - const response = await fetch(`${config.serverUrl}/api/v1/projects`, { - headers: { 'X-API-Key': config.apiKey } - }); - - if (response.ok) { - statusText.textContent = '已连接'; - } else { - statusDot.classList.add('error'); - statusText.textContent = '连接失败'; - } - } catch (error) { - statusDot.classList.add('error'); - statusText.textContent = '连接错误'; - } -} - -// 加载项目列表 -async function loadProjects(config) { - const projectSelect = document.getElementById('projectSelect'); - - if (!config.apiKey) { - projectSelect.innerHTML = ''; - return; - } - - try { - const response = await fetch(`${config.serverUrl}/api/v1/projects`, { - headers: { 'X-API-Key': config.apiKey } - }); - - if (response.ok) { - const data = await response.json(); - const projects = data.projects || []; - - // 更新项目数统计 - document.getElementById('projectCount').textContent = projects.length; - - // 填充下拉框 - let html = ''; - projects.forEach(project => { - const selected = project.id === config.defaultProjectId ? 'selected' : ''; - html += ``; - }); - projectSelect.innerHTML = html; - } - } catch (error) { - console.error('Failed to load projects:', error); - } -} - -// 更新统计 +// 更新统计信息 async function updateStats() { - // 从存储中获取统计数据 - const result = await chrome.storage.local.get(['clipStats']); - const stats = result.clipStats || { total: 0, today: 0, lastDate: null }; - - // 检查是否需要重置今日计数 - const today = new Date().toDateString(); - if (stats.lastDate !== today) { - stats.today = 0; - stats.lastDate = today; - await chrome.storage.local.set({ clipStats: stats }); - } - - document.getElementById('clipCount').textContent = stats.total; - document.getElementById('todayCount').textContent = stats.today; + const [tab] = await chrome.tabs.query({ active: true, currentWindow: true }); + + // 获取字数统计 + try { + const response = await chrome.tabs.sendMessage(tab.id, { action: 'extractContent' }); + if (response.success) { + document.getElementById('wordCount').textContent = response.data.wordCount || 0; + } + } catch (error) { + console.log('Content script not available'); + } + + // 获取待同步数量 + const result = await chrome.storage.local.get(['clips']); + const clips = result.clips || []; + const pendingCount = clips.filter(c => !c.synced).length; + document.getElementById('pendingCount').textContent = pendingCount; } -// 显示消息 -function showMessage(text, type) { - const messageEl = document.getElementById('message'); - messageEl.textContent = text; - messageEl.className = `message ${type}`; - - setTimeout(() => { - messageEl.className = 'message'; - }, 3000); +// 保存整个页面 +async function clipPage() { + setLoading(true); + + try { + const [tab] = await chrome.tabs.query({ active: true, currentWindow: true }); + + // 发送消息给 background script + const response = await chrome.runtime.sendMessage({ action: 'clipPage' }); + + if (response.success) { + showStatus('页面已保存！', 'success'); + loadRecentClips(); + updateStats(); + } else { + showStatus(response.error || '保存失败', 'error'); + } + } catch (error) { + showStatus('保存失败: ' + error.message, 'error'); + } finally { + setLoading(false); + } } -// 获取配置 -function getConfig() { - return new Promise((resolve) => { - chrome.storage.sync.get(['insightflowConfig'], (result) => { - resolve(result.insightflowConfig || { - serverUrl: 'http://122.51.127.111:18000', - apiKey: '', - defaultProjectId: '' - }); - }); - }); +// 保存选中内容 +async function clipSelection() { + setLoading(true); + + try { + const [tab] = await chrome.tabs.query({ active: true, currentWindow: true }); + + const response = await chrome.runtime.sendMessage({ action: 'clipSelection' }); + + if (response.success) { + showStatus('选中内容已保存！', 'success'); + loadRecentClips(); + updateStats(); + } else { + showStatus(response.error || '保存失败', 'error'); + } + } catch (error) { + showStatus('保存失败: ' + error.message, 'error'); + } finally { + setLoading(false); + } } -// 保存配置 -function saveConfig(config) { - return new Promise((resolve) => { - chrome.storage.sync.set({ insightflowConfig: config }, resolve); - }); +// 加载最近的剪辑 +async function loadRecentClips() { + const result = await chrome.storage.local.get(['clips']); + const clips = result.clips || []; + + const clipsList = document.getElementById('clipsList'); + clipsList.innerHTML = ''; + + // 只显示最近5条 + const recentClips = clips.slice(0, 5); + + for (const clip of recentClips) { + const clipEl = document.createElement('div'); + clipEl.className = 'clip-item'; + + const title = clip.title || '未命名'; + const time = new Date(clip.extractedAt).toLocaleString('zh-CN'); + const statusClass = clip.synced ? 'synced' : 'pending'; + const statusText = clip.synced ? '已同步' : '待同步'; + + clipEl.innerHTML = ` +
${escapeHtml(title)}
+
${time}
+ ${statusText} + `; + + clipsList.appendChild(clipEl); + } +} + +// 打开设置页面 +function openOptions(e) { + e.preventDefault(); + chrome.runtime.openOptionsPage(); +} + +// 显示状态消息 +function showStatus(message, type) { + const statusEl = document.getElementById('status'); + statusEl.textContent = message; + statusEl.className = 'status ' + type; + + setTimeout(() => { + statusEl.textContent = ''; + statusEl.className = 'status'; + }, 3000); +} + +// 设置加载状态 +function setLoading(loading) { + const loadingEl = document.getElementById('loading'); + if (loading) { + loadingEl.classList.add('active'); + } else { + loadingEl.classList.remove('active'); + } } // HTML 转义 function escapeHtml(text) { - const div = document.createElement('div'); - div.textContent = text; - return div.innerHTML; + const div = document.createElement('div'); + div.textContent = text; + return div.innerHTML; } \ No newline at end of file diff --git a/docs/PHASE7_TASK7_SUMMARY.md b/docs/PHASE7_TASK7_SUMMARY.md index f5276cc..dead55c 100644 --- a/docs/PHASE7_TASK7_SUMMARY.md +++ b/docs/PHASE7_TASK7_SUMMARY.md @@ -1,207 +1,143 @@ -# InsightFlow Phase 7 Task 7 开发总结 +# Phase 7 任务 7 开发完成总结 -## 开发内容 +## 已完成的工作 -### 1. 插件管理模块 (plugin_manager.py) +### 1. 创建 plugin_manager.py 模块 -创建了完整的插件与集成系统，包含以下核心组件： +实现了完整的插件与集成系统，包含以下核心类： -#### PluginManager - 插件管理主类 -- 插件 CRUD 操作 -- API Key 生成和管理 -- 插件活动日志记录 -- 支持多种插件类型：Chrome 扩展、机器人、Webhook、WebDAV +#### PluginManager +- 插件的 CRUD 操作 +- 插件配置的加密存储 +- 插件使用统计 -#### ChromeExtensionHandler - Chrome 插件处理器 -- 验证 Chrome 插件请求 -- 提取网页内容（使用 BeautifulSoup） -- 创建网页剪藏 +#### ChromeExtensionHandler +- Chrome 扩展令牌管理（创建、验证、撤销） +- 网页内容导入（自动提取正文、保存为文档） +- 权限控制（read/write/delete） -#### BotHandler - 机器人处理器 -- 支持飞书、钉钉、Slack 消息解析 -- 发送消息到各平台 -- 会话管理 +#### BotHandler +- 飞书/钉钉机器人会话管理 +- 消息接收和发送 +- 音频文件处理（支持群内直接分析） +- Webhook 签名验证 -#### WebhookIntegration - Webhook 集成处理器 -- Zapier/Make 集成 -- 签名验证 -- 数据处理和转发 +#### WebhookIntegration +- Zapier/Make Webhook 端点管理 +- 事件触发机制 +- 多种认证方式（API Key、Bearer、OAuth） +- 支持 5000+ 应用连接 -#### WebDAVSync - WebDAV 同步处理器 +#### WebDAVSyncManager +- WebDAV 同步配置管理 - 连接测试 -- 文件列表获取 -- 文件上传/下载 +- 项目数据导出和同步 +- 支持坚果云等 WebDAV 网盘 -### 2. Chrome 扩展代码 +### 2. 更新 schema.sql -创建了完整的 Chrome 扩展，包含： +添加了以下数据库表： +- `plugins`: 插件配置表 +- `plugin_configs`: 插件详细配置表 +- `bot_sessions`: 机器人会话表 +- `webhook_endpoints`: Webhook 端点表 +- `webdav_syncs`: WebDAV 同步配置表 +- `chrome_extension_tokens`: Chrome 扩展令牌表 -#### manifest.json -- Manifest V3 配置 -- 权限声明 -- 图标配置 +### 3. 更新 main.py -#### background.js -- 右键菜单创建 -- 页面剪藏逻辑 -- 消息处理 - -#### content.js -- 选中文本检测 -- 浮动按钮显示 -- 弹窗交互 - -#### content.css -- 浮动按钮样式 -- 弹窗样式 -- 项目列表样式 - -#### popup.html/js -- 扩展弹出窗口 -- 项目选择 -- 快速保存 - -#### options.html/js -- 设置页面 -- API Key 配置 -- 连接测试 - -### 3. 数据库更新 (schema.sql) - -新增以下表： - -- **plugins**: 插件配置表 -- **bot_sessions**: 机器人会话表 -- **webhook_endpoints**: Webhook 端点表 -- **webdav_syncs**: WebDAV 同步配置表 -- **plugin_activity_logs**: 插件活动日志表 - -### 4. API 端点 (main.py) - -新增以下 API： +添加了完整的插件相关 API 端点： #### 插件管理 - `POST /api/v1/plugins` - 创建插件 - `GET /api/v1/plugins` - 列出插件 - `GET /api/v1/plugins/{id}` - 获取插件详情 +- `PATCH /api/v1/plugins/{id}` - 更新插件 - `DELETE /api/v1/plugins/{id}` - 删除插件 -- `POST /api/v1/plugins/{id}/regenerate-key` - 重新生成 API Key #### Chrome 扩展 -- `POST /api/v1/plugins/chrome/clip` - 保存网页内容 +- `POST /api/v1/plugins/chrome/tokens` - 创建令牌 +- `GET /api/v1/plugins/chrome/tokens` - 列出自令牌 +- `DELETE /api/v1/plugins/chrome/tokens/{id}` - 撤销令牌 +- `POST /api/v1/plugins/chrome/import` - 导入网页内容 #### 机器人 -- `POST /api/v1/bots/webhook/{platform}` - 接收机器人消息 -- `GET /api/v1/bots/sessions` - 列出机器人会话 +- `POST /api/v1/plugins/bot/feishu/sessions` - 创建飞书会话 +- `POST /api/v1/plugins/bot/dingtalk/sessions` - 创建钉钉会话 +- `GET /api/v1/plugins/bot/{type}/sessions` - 列出会话 +- `POST /api/v1/plugins/bot/{type}/webhook` - 接收消息 +- `POST /api/v1/plugins/bot/{type}/sessions/{id}/send` - 发送消息 -#### Webhook 集成 -- `POST /api/v1/webhook-endpoints` - 创建 Webhook 端点 -- `GET /api/v1/webhook-endpoints` - 列出 Webhook 端点 -- `POST /webhook/{type}/{token}` - 接收外部 Webhook +#### 集成 +- `POST /api/v1/plugins/integrations/zapier` - 创建 Zapier 端点 +- `POST /api/v1/plugins/integrations/make` - 创建 Make 端点 +- `GET /api/v1/plugins/integrations/{type}` - 列出端点 +- `POST /api/v1/plugins/integrations/{id}/test` - 测试端点 +- `POST /api/v1/plugins/integrations/{id}/trigger` - 手动触发 #### WebDAV -- `POST /api/v1/webdav-syncs` - 创建 WebDAV 同步配置 -- `GET /api/v1/webdav-syncs` - 列出 WebDAV 同步配置 -- `POST /api/v1/webdav-syncs/{id}/test` - 测试连接 -- `POST /api/v1/webdav-syncs/{id}/sync` - 触发同步 +- `POST /api/v1/plugins/webdav` - 创建同步配置 +- `GET /api/v1/plugins/webdav` - 列出配置 +- `POST /api/v1/plugins/webdav/{id}/test` - 测试连接 +- `POST /api/v1/plugins/webdav/{id}/sync` - 执行同步 +- `DELETE /api/v1/plugins/webdav/{id}` - 删除配置 -#### 日志 -- `GET /api/v1/plugins/{id}/logs` - 获取插件活动日志 +### 4. 更新 requirements.txt -### 5. 依赖更新 (requirements.txt) +添加了必要的依赖： +- `webdav4==0.9.8` - WebDAV 客户端 +- `urllib3==2.2.0` - URL 处理 -新增依赖： -- `beautifulsoup4==4.12.3` - HTML 解析 -- `webdavclient3==3.14.6` - WebDAV 客户端 +### 5. 创建 Chrome 扩展基础代码 -## 使用说明 +完整的 Chrome 扩展实现： +- `manifest.json` - 扩展配置（Manifest V3） +- `background.js` - 后台脚本（右键菜单、消息处理、自动同步） +- `content.js` - 内容脚本（页面内容提取、浮动按钮） +- `content.css` - 内容样式 +- `popup.html/js` - 弹出窗口（保存页面、查看剪辑历史） +- `options.html/js` - 设置页面（服务器配置、令牌设置） +- `README.md` - 扩展使用说明 -### Chrome 扩展安装 +## 功能特性 -1. 打开 Chrome 扩展管理页面 (chrome://extensions/) -2. 开启"开发者模式" -3. 点击"加载已解压的扩展程序" -4. 选择 `chrome-extension` 文件夹 +### Chrome 插件 +- ✅ 一键保存整个网页（智能提取正文） +- ✅ 保存选中的文本内容 +- ✅ 保存链接 +- ✅ 浮动按钮快速访问 +- ✅ 右键菜单支持 +- ✅ 自动同步到服务器 +- ✅ 离线缓存，稍后同步 -### Chrome 扩展配置 +### 飞书/钉钉机器人 +- ✅ 群内直接分析音频文件 +- ✅ 命令交互（/help, /status, /analyze） +- ✅ 消息自动回复 +- ✅ Webhook 签名验证 -1. 点击扩展图标打开设置 -2. 输入 InsightFlow 服务器地址 -3. 从 InsightFlow 控制台获取 API Key -4. 测试连接 -5. 选择默认项目（可选） +### Zapier/Make 集成 +- ✅ 创建 Webhook 端点 +- ✅ 事件触发机制 +- ✅ 支持 5000+ 应用连接 +- ✅ 多种认证方式 -### 使用 Chrome 扩展 +### WebDAV 同步 +- ✅ 与坚果云等网盘联动 +- ✅ 项目数据自动同步 +- ✅ 连接测试 +- ✅ 增量同步支持 -- **保存当前页面**: 点击扩展图标 → 点击"保存当前页面" -- **保存选中文本**: 选中页面文本 → 点击浮动按钮 → 选择保存方式 -- **右键保存**: 右键点击页面 → "保存到 InsightFlow" +## API 文档 -### 创建机器人插件 +所有 API 都已在 Swagger/OpenAPI 文档中注册，访问： +- Swagger UI: `/docs` +- ReDoc: `/redoc` -```bash -curl -X POST http://localhost:18000/api/v1/plugins \ - -H "X-API-Key: your_api_key" \ - -H "Content-Type: application/json" \ - -d '{ - "name": "飞书机器人", - "plugin_type": "feishu_bot", - "project_id": "your_project_id" - }' -``` +## 下一步工作 -### 创建 Webhook 端点 - -```bash -curl -X POST http://localhost:18000/api/v1/webhook-endpoints \ - -H "X-API-Key: your_api_key" \ - -H "Content-Type: application/json" \ - -d '{ - "plugin_id": "your_plugin_id", - "name": "Zapier Integration", - "endpoint_type": "zapier", - "target_project_id": "your_project_id" - }' -``` - -### 配置 WebDAV 同步 - -```bash -curl -X POST http://localhost:18000/api/v1/webdav-syncs \ - -H "X-API-Key: your_api_key" \ - -H "Content-Type: application/json" \ - -d '{ - "plugin_id": "your_plugin_id", - "name": "坚果云同步", - "server_url": "https://dav.jianguoyun.com/dav/", - "username": "your_username", - "password": "your_password", - "remote_path": "/InsightFlow", - "sync_direction": "bidirectional" - }' -``` - -## 开发进度 - -Phase 7 开发进度更新： - -| 任务 | 状态 | 完成时间 | -|------|------|----------| -| 1. 智能工作流自动化 | ✅ 已完成 | 2026-02-23 | -| 2. 多模态支持 | ✅ 已完成 | 2026-02-23 | -| 7. 插件与集成 | ✅ 已完成 | 2026-02-23 | -| 3. 数据安全与合规 | 📋 待开发 | - | -| 4. 协作与共享 | 📋 待开发 | - | -| 5. 智能报告生成 | 📋 待开发 | - | -| 6. 高级搜索与发现 | 📋 待开发 | - | -| 8. 性能优化与扩展 | 📋 待开发 | - | - -## 下一步 - -按照建议的开发顺序，接下来应该开发： - -**Phase 7 任务 3: 数据安全与合规** +Phase 7 任务 3: 数据安全与合规 - 端到端加密 - 数据脱敏 - 审计日志 -- GDPR/数据合规支持 +- GDPR 合规支持 \ No newline at end of file