Commit Graph

16 Commits

Author SHA1 Message Date
Lv, Qi
0c975bb8f1 Refactor: Remove legacy analysis results and implement workflow history
- **Common Contracts**: Updated DTOs and models to support workflow history; removed legacy analysis result DTOs.
- **Data Persistence Service**:
    - Removed `analysis_results` table logic and API endpoints.
    - Implemented `workflow_history` API and DB access (`history.rs`).
    - Fixed compilation errors and updated tests.
    - Exposed Postgres port in `docker-compose.yml` for easier debugging/offline checks.
- **API Gateway**:
    - Implemented `history` endpoints (get history list, get by ID).
    - Removed legacy `analysis-results` endpoints.
    - Fixed routing and handler logic in `api.rs`.
- **Report Generator Service**:
    - Removed dependency on legacy `analysis-results` persistence calls.
    - Fixed compilation errors.
- **Workflow Orchestrator**: Fixed warnings and minor logic issues.
- **Providers**: Updated provider services (alphavantage, tushare, finnhub, yfinance, mock) to align with contract changes.
- **Frontend**:
    - Updated `ReportPage` and stores to use new workflow history.
    - Added `RecentReportsDropdown` component.
    - Cleaned up `RealtimeLogs` component.
- **Documentation**: Moved completed design tasks to `completed/` and added refactoring context docs.

Confirmed all services pass `cargo check`.
2025-11-29 14:46:44 +08:00
Lv, Qi
b8eab4dfd5 feat: fix error propagation in report-generator and workflow-orchestrator 2025-11-27 23:07:42 +08:00
Lv, Qi
a59b994a92 WIP: Commit all pending changes across services, frontend, and docs
- Sync updates for provider services (AlphaVantage, Finnhub, YFinance, Tushare)
- Update Frontend components and pages for recent config changes
- Update API Gateway and Registry
- Include design docs and tasks status
2025-11-27 02:45:56 +08:00
Lv, Qi
70b1a27978 feat: implement API DTO export and verify E2E tests
- contracts: Add #[api_dto] macros to core structs for OpenAPI support
- api-gateway: Integrate utoipa for Swagger UI and OpenAPI spec generation
- config: Implement dynamic configuration APIs for DataSources and LLM Providers
- tests: Refactor E2E tests to support dynamic provider configuration and fix timeouts
- docs: Update backend status in backend_todos.md
2025-11-22 22:14:01 +08:00
Lv, Qi
b41eaf8b99 Refactor frontend to Vite+React SPA and update docs
Major architectural shift from Next.js to a lightweight Vite + React SPA model ("Puppet Architecture") to better support real-time workflow visualization and strict type safety.

Key Changes:
1. **Architecture & Build**:
   - Initialized Vite + React + TypeScript project.
   - Configured Tailwind CSS v4 and Shadcn UI.
   - Archived legacy Next.js frontend to 'frontend/archive/v2_nextjs'.

2. **Core Features**:
   - **Dashboard**: Implemented startup page with Symbol, Market, and Template selection.
   - **Report Page**:
     - **Workflow Visualization**: Integrated ReactFlow to show dynamic DAG of analysis tasks.
     - **Real-time Status**: Implemented Mock SSE logic to simulate task progress, logs, and status changes.
     - **Multi-Tab Interface**: Dynamic tabs for 'Overview', 'Fundamental Data', and analysis modules.
     - **Streaming Markdown**: Enabled typewriter-style streaming rendering for analysis reports using 'react-markdown'.
   - **Config Page**: Implemented settings for AI Providers, Data Sources, and Templates using TanStack Query.

3. **Documentation**:
   - Created v2.0 User Guide ('docs/1_requirements/20251122_[Active]_user-guide_v2.md').
   - Implemented 'DocsPage' in frontend to render the user guide directly within the app.

4. **Backend Alignment**:
   - Created 'docs/frontend/backend_todos.md' outlining necessary backend adaptations (OpenAPI, Progress tracking).

This commit establishes the full frontend 'shell' ready for backend integration.
2025-11-22 19:37:36 +08:00
Lv, Qi
4881ac8603 feat: finalize backend readiness (config, limits, docs)
- Expose API Gateway port 4000 in docker-compose for local dev
- Enable dynamic API_GATEWAY_URL in Next.js config
- Add 64K context hard limit in report-generator to avoid LLM errors
- Add backend API readiness report
2025-11-22 00:57:23 +08:00
Lv, Qi
0cb31e363e Refactor E2E tests and improve error handling in Orchestrator
- Fix `simple_test_analysis` template in E2E test setup to align with Orchestrator's data fetch logic.
- Implement and verify additional E2E scenarios:
    - Scenario C: Partial Provider Failure (verified error propagation fix in Orchestrator).
    - Scenario D: Invalid Symbol input.
    - Scenario E: Analysis Module failure.
- Update `WorkflowStateMachine::handle_report_failed` to correctly scope error broadcasting to the specific task instead of failing effectively silently or broadly.
- Update testing strategy documentation to reflect completed Phase 4 testing.
- Skip Scenario B (Orchestrator Restart) as persistence is not yet implemented (decision made to defer persistence).
2025-11-21 20:44:32 +08:00
Lv, Qi
d28f3c5266 feat: update analysis workflow and fix LLM client connection issues
- Enhance LlmClient to handle malformed URLs and HTML error responses
- Improve logging in report-generator-service worker
- Update frontend API routes and hooks for analysis
- Update various service configurations and persistence logic
2025-11-19 17:30:52 +08:00
Lv, Qi
c8be576526 fix(yfinance): enable crumb auth and fix config loading
- Enable dynamic config loading from DB to respect enabled status
- Implement Yahoo Finance Crumb authentication flow
- Add browser-like User-Agent and cookie store
- Map .SH symbols to .SS for Yahoo compatibility
- Add Google DNS to docker-compose to fix container resolution issues
2025-11-19 06:53:14 +08:00
Lv, Qi
60e6c8f61b feat(config): 将服务配置全面迁移至数据库
本次提交完成了一项重要的架构重构,将所有外部服务的API凭证管理从环境变量迁移到了中心化的数据库配置中。

主要变更:

1.  **统一配置源**:
    -   `data-persistence-service` 现已提供 `/api/v1/configs/data_sources` 端点,用于统一管理数据源配置。
    -   所有配置(LLM 和数据源)现在都通过数据库的 `system_config` 表进行管理,实现了“单一事实源”。

2.  **增强服务韧性**:
    -   重构了 `finnhub-`, `tushare-`, `alphavantage-provider-service`。
    -   这些服务在启动时不再强制要求 API Key。
    -   引入了动态配置轮询器 (`config_poller`),服务现在可以定期从数据库获取最新配置。
    -   实现了“降级模式”:当配置缺失时,服务会进入 `Degraded` 状态并暂停处理消息,而不是直接崩溃。配置恢复后,服务会自动回到 `Active` 状态。
    -   `/health` 端点现在能准确反映服务的真实运行状态。

3.  **前端易用性提升**:
    -   您在 `/config` 页面上增加了“数据源配置”面板,允许用户通过 UI 动态更新所有 API Token。

4.  **部署简化**:
    -   从 `docker-compose.yml` 中移除了所有已废弃的 `_API_KEY` 环境变量,消除了启动时的警告。

这项重构显著提升了系统的可维护性、健壮性和用户体验,为未来的功能扩展奠定了坚实的基础。
2025-11-18 05:58:18 +08:00
Lv, Qi
53d69a00e5 fix(services): make providers observable and robust
- Fix Dockerfile stub builds; compile full sources (no empty binaries)
- Add ca-certificates and curl in runtime images for TLS/healthchecks
- Enable RUST_LOG and RUST_BACKTRACE for all providers
- Add HTTP /health healthchecks in docker-compose for ports 8000-8004
- Standardize Tushare /health to structured HealthStatus JSON
- Enforce strict config validation (FINNHUB_API_KEY, TUSHARE_API_TOKEN)
- Map provider API keys via .env in docker-compose
- Log provider_services at API Gateway startup for diagnostics

Outcome: provider containers no longer exit silently; missing keys fail fast with explicit errors; health and logs are consistent across modules.
2025-11-17 04:40:51 +08:00
Lv, Qi
9d62a53b73 refactor(architecture): Align frontend & docs with DB gateway pattern
本次提交旨在完成一次架构一致性重构,核心目标是使前端代码和相关文档完全符合“`data-persistence-service`是唯一数据库守门人”的设计原则。

主要变更包括:
1.  **移除前端数据库直连**:
    *   从`docker-compose.yml`中删除了`frontend`服务的`DATABASE_URL`。
    *   彻底移除了`frontend`项目中的`Prisma`依赖、配置文件和客户端实例。
2.  **清理前端UI**:
    *   从配置页面中删除了所有与数据库设置相关的UI组件和业务逻辑。
3.  **同步更新文档**:
    *   更新了《用户使用文档》和《需求文档》,移除了所有提及或要求前端进行数据库配置的过时内容。

此次重构后,系统前端的数据交互已完全收敛至`api-gateway`,确保了架构的统一性、健壮性和高内聚。
2025-11-17 01:29:56 +08:00
Lv, Qi
ceffa0d95c chore(build): unify Dockerfiles and root .dockerignore; fix services after deps upgrade
- secrecy: migrate Secret<String> -> SecretString across services; adjust usages
- warnings: clean unused imports/structs (no behavior change)
- docker-compose: remove unnecessary host port mappings; rely on app-network; keep frontend only
- build: set build.context to repo root; reference per-service Dockerfiles
- add: root .dockerignore to exclude target/node_modules/ref/archive/docs and logs
- data-persistence-service: refine cargo-chef with path deps; slim context; copy correct targets
- Dockerfiles: normalize multi-stage builds; fix WORKDIR and release binary paths
- nats: resolve 4222 conflict by not binding to host
- verified: all rust services build successfully with new flow
2025-11-16 23:34:28 +08:00
Lv, Qi
5327e76aaa chore: 提交本轮 Rust 架构迁移相关改动
- docker-compose: 下线 Python backend/config-service,切换至 config-service-rs
- archive: 归档 legacy Python 目录至 archive/python/*
- services: 新增/更新 common-contracts、api-gateway、各 provider、report-generator-service、config-service-rs
- data-persistence-service: API/system 模块与模型/DTO 调整
- frontend: 更新 useApi 与 API 路由
- docs: 更新路线图并勾选光荣退役
- cleanup: 移除 data-distance-service 占位测试
2025-11-16 20:55:46 +08:00
Lv, Qi
21155bc4f8 feat(realtime): 接入前端实时报价并完善后端缓存
前端: 新增 RealTimeQuoteResponse 类型;新增 useRealtimeQuote Hook 并在报告页图表旁展示价格与时间戳(严格 TTL,无兜底)

FastAPI: 新增 GET /financials/{market}/{symbol}/realtime?max_age_seconds=.. 只读端点;通过 DataPersistenceClient 读取 Rust 缓存

Rust: 新增 realtime_quotes hypertable 迁移;新增 POST /api/v1/market-data/quotes 与 GET /api/v1/market-data/quotes/{symbol}?market=..;新增 DTO/Model/DB 函数;修正 #[api] 宏与路径参数;生成 SQLx 离线缓存 (.sqlx) 以支持离线构建

Python: DataPersistenceClient 新增 upsert/get 实时报价,并调整 GET 路径与参数

说明: TradingView 图表是第三方 websocket,不受我们缓存控制;页面数值展示走自有缓存通路,统一且可控。
2025-11-09 05:12:14 +08:00
Lv, Qi
3d0fd6f704 refactor(phase0-1): 容器化与配置服务拆分,并清理根目录
- 新增 docker-compose 与 Tiltfile,容器化 backend/frontend/postgres(宿主口+10000)
- 新增 services/config-service(GET /api/v1/system, /analysis-modules),并加入 compose
- backend ConfigManager 移除本地文件回退,强制依赖 config-service
- 新增 backend/frontend Dockerfile
- 清理根目录:移动 pm2.config.js -> deployment/;dev.py -> scripts/;删除根 package.json 与 lock
- 新增 .gitignore,忽略二进制与临时文件
2025-11-08 21:07:38 +08:00