Commit Graph

12 Commits

Author SHA1 Message Date
Lv, Qi
e9e4d0c1b3 chore: massive update covering recent refactoring and bug fixes
- fix: infinite message loop in workflow orchestrator
- feat: restore realtime LLM streaming from report generator to frontend
- refactor: major update to provider services (generic workers, workflow adapters)
- refactor: common contracts and message definitions updated
- feat: enhanced logging and observability in orchestrator
- docs: update project management tasks and status
- chore: dependency updates and config adjustments
2025-11-30 19:17:02 +08:00
Lv, Qi
5dc13fa735 fix: resolve infinite feedback loop in orchestrator and restore realtime LLM streaming 2025-11-30 19:16:02 +08:00
Lv, Qi
70b30b39d8 Refactor: Complete transition from analysis_results to workflow_history
- Removed dependencies on analysis_results table
- Implemented workflow_history storage in Data Persistence Service
- Updated Workflow Orchestrator to save workflow snapshots to history
- Refactored Frontend to consume workflow_history and fetch reports via VGCS
- Fixed Data Providers (Tushare, YFinance) to report output paths in metadata
- Updated documentation and task status
2025-11-29 17:55:54 +08:00
Lv, Qi
0c975bb8f1 Refactor: Remove legacy analysis results and implement workflow history
- **Common Contracts**: Updated DTOs and models to support workflow history; removed legacy analysis result DTOs.
- **Data Persistence Service**:
    - Removed `analysis_results` table logic and API endpoints.
    - Implemented `workflow_history` API and DB access (`history.rs`).
    - Fixed compilation errors and updated tests.
    - Exposed Postgres port in `docker-compose.yml` for easier debugging/offline checks.
- **API Gateway**:
    - Implemented `history` endpoints (get history list, get by ID).
    - Removed legacy `analysis-results` endpoints.
    - Fixed routing and handler logic in `api.rs`.
- **Report Generator Service**:
    - Removed dependency on legacy `analysis-results` persistence calls.
    - Fixed compilation errors.
- **Workflow Orchestrator**: Fixed warnings and minor logic issues.
- **Providers**: Updated provider services (alphavantage, tushare, finnhub, yfinance, mock) to align with contract changes.
- **Frontend**:
    - Updated `ReportPage` and stores to use new workflow history.
    - Added `RecentReportsDropdown` component.
    - Cleaned up `RealtimeLogs` component.
- **Documentation**: Moved completed design tasks to `completed/` and added refactoring context docs.

Confirmed all services pass `cargo check`.
2025-11-29 14:46:44 +08:00
Lv, Qi
03b53aed71 feat: Refactor Analysis Context Mechanism and Generic Worker
- Implemented Unified Context Mechanism (Task 20251127):
  - Decoupled intent (Module) from resolution (Orchestrator).
  - Added ContextResolver for resolving input bindings (Manual Glob/Auto LLM).
  - Added IOBinder for managing physical paths.
  - Updated GenerateReportCommand to support explicit input bindings and output paths.

- Refactored Report Worker to Generic Execution (Task 20251128):
  - Removed hardcoded financial DTOs and specific formatting logic.
  - Implemented Generic YAML-based context assembly for better LLM readability.
  - Added detailed execution tracing (Sidecar logs).
  - Fixed input data collision bug by using full paths as context keys.

- Updated Tushare Provider to support dynamic output paths.
- Updated Common Contracts with new configuration models.
2025-11-28 20:11:17 +08:00
Lv, Qi
91a6dfc4c1 feat: Implement Context Inspector and fix workflow merge base
- **Observability**: Added `ContextExplorer` component for viewing task input/output file trees and diffs.
- **Workflow Engine**:
  - Implemented Initial Commit logic in `StartWorkflow` to ensure all tasks share a common merge base.
  - Added `input_commit` tracking to `DagScheduler` and events.
  - Exposed `vgcs` (Git) operations via Orchestrator API (tree, blob, diff).
- **API Gateway**: Added proxy routes for Context Inspector (`/api/context/...`).
- **UI**:
  - Refactored `ReportPage` to remove legacy "Fundamental Data" tab.
  - Added "Context Inspector" button in Task Detail view with Dialog support.
  - Visualized Added/Modified/Deleted files with color coding in Context Explorer.
- **Cleanup**: Removed unused `FinancialTable` component.
2025-11-28 02:53:25 +08:00
Lv, Qi
b90388b76e feat: add task display names and improve report page UX
- Backend: Add display_name to TaskNode and DagNode for human-readable names
- Frontend: Update visualizer and tabs to use display_name
- UX: Implement global scrolling layout for ReportPage
- UX: Add sticky behavior with visual feedback for Workflow Sidebar
- UX: Fix RealtimeLogs scrolling and layout issues
2025-11-28 01:01:17 +08:00
Lv, Qi
a68a95338b feat: fix error propagation in report-generator and workflow-orchestrator 2025-11-27 23:07:05 +08:00
Lv, Qi
ca1eddd244 Refactor Orchestrator to DAG-based scheduler with Git Context merging
- Implemented `DagScheduler` in `workflow-orchestrator` to manage task dependencies and commit history.
- Added `vgcs.merge_commits` in `workflow-context` for smart 3-way merge of parallel task branches.
- Introduced generic `WorkflowTaskCommand` and `WorkflowTaskEvent` in `common-contracts`.
- Adapted `tushare-provider-service` with `generic_worker` to support Git-based context read/write.
- Updated NATS subjects to support wildcard routing for generic workflow commands.
2025-11-27 02:42:25 +08:00
Lv, Qi
70b1a27978 feat: implement API DTO export and verify E2E tests
- contracts: Add #[api_dto] macros to core structs for OpenAPI support
- api-gateway: Integrate utoipa for Swagger UI and OpenAPI spec generation
- config: Implement dynamic configuration APIs for DataSources and LLM Providers
- tests: Refactor E2E tests to support dynamic provider configuration and fix timeouts
- docs: Update backend status in backend_todos.md
2025-11-22 22:14:01 +08:00
Lv, Qi
b41eaf8b99 Refactor frontend to Vite+React SPA and update docs
Major architectural shift from Next.js to a lightweight Vite + React SPA model ("Puppet Architecture") to better support real-time workflow visualization and strict type safety.

Key Changes:
1. **Architecture & Build**:
   - Initialized Vite + React + TypeScript project.
   - Configured Tailwind CSS v4 and Shadcn UI.
   - Archived legacy Next.js frontend to 'frontend/archive/v2_nextjs'.

2. **Core Features**:
   - **Dashboard**: Implemented startup page with Symbol, Market, and Template selection.
   - **Report Page**:
     - **Workflow Visualization**: Integrated ReactFlow to show dynamic DAG of analysis tasks.
     - **Real-time Status**: Implemented Mock SSE logic to simulate task progress, logs, and status changes.
     - **Multi-Tab Interface**: Dynamic tabs for 'Overview', 'Fundamental Data', and analysis modules.
     - **Streaming Markdown**: Enabled typewriter-style streaming rendering for analysis reports using 'react-markdown'.
   - **Config Page**: Implemented settings for AI Providers, Data Sources, and Templates using TanStack Query.

3. **Documentation**:
   - Created v2.0 User Guide ('docs/1_requirements/20251122_[Active]_user-guide_v2.md').
   - Implemented 'DocsPage' in frontend to render the user guide directly within the app.

4. **Backend Alignment**:
   - Created 'docs/frontend/backend_todos.md' outlining necessary backend adaptations (OpenAPI, Progress tracking).

This commit establishes the full frontend 'shell' ready for backend integration.
2025-11-22 19:37:36 +08:00
Lv, Qi
0cb31e363e Refactor E2E tests and improve error handling in Orchestrator
- Fix `simple_test_analysis` template in E2E test setup to align with Orchestrator's data fetch logic.
- Implement and verify additional E2E scenarios:
    - Scenario C: Partial Provider Failure (verified error propagation fix in Orchestrator).
    - Scenario D: Invalid Symbol input.
    - Scenario E: Analysis Module failure.
- Update `WorkflowStateMachine::handle_report_failed` to correctly scope error broadcasting to the specific task instead of failing effectively silently or broadly.
- Update testing strategy documentation to reflect completed Phase 4 testing.
- Skip Scenario B (Orchestrator Restart) as persistence is not yet implemented (decision made to defer persistence).
2025-11-21 20:44:32 +08:00