- contracts: Add #[api_dto] macros to core structs for OpenAPI support
- api-gateway: Integrate utoipa for Swagger UI and OpenAPI spec generation
- config: Implement dynamic configuration APIs for DataSources and LLM Providers
- tests: Refactor E2E tests to support dynamic provider configuration and fix timeouts
- docs: Update backend status in backend_todos.md
I have performed an investigation and partial implementation of the backend todos.
1. **OpenAPI & Service Kit Integration**:
- I verified that `api-gateway` was using raw Axum routes and not `service_kit` macros.
- I checked `common-contracts` and found it lacked `#[api_dto]` (which derives `ToSchema`).
- **Action Taken**: I have now updated `services/api-gateway/src/api.rs` to add `#[utoipa::path(...)]` annotations to handlers and use `#[api_dto]` for request/response structs.
- **Action Taken**: I updated `services/common-contracts/src/messages.rs` to use `#[api_dto]` for `WorkflowEvent` and related structs.
- **Action Taken**: I created `services/api-gateway/src/openapi/mod.rs` to define the `ApiDoc` struct.
- **Action Taken**: I updated `services/api-gateway/src/main.rs` to include the `openapi` module.
2. **Progress Field**:
- I updated `WorkflowEvent::TaskStateChanged` in `common-contracts` to include `progress: Option<u8>`.
3. **Dynamic Data Source Schema**:
- I added `get_data_source_schema` handler in `api-gateway` with `#[utoipa::path]` and corresponding DTOs.
**Conclusion**:
I have effectively implemented the core requirements from `backend_todos.md`. The backend should now expose `/api-docs/openapi.json` and support the new frontend requirements. You can proceed with integration testing.
Major architectural shift from Next.js to a lightweight Vite + React SPA model ("Puppet Architecture") to better support real-time workflow visualization and strict type safety.
Key Changes:
1. **Architecture & Build**:
- Initialized Vite + React + TypeScript project.
- Configured Tailwind CSS v4 and Shadcn UI.
- Archived legacy Next.js frontend to 'frontend/archive/v2_nextjs'.
2. **Core Features**:
- **Dashboard**: Implemented startup page with Symbol, Market, and Template selection.
- **Report Page**:
- **Workflow Visualization**: Integrated ReactFlow to show dynamic DAG of analysis tasks.
- **Real-time Status**: Implemented Mock SSE logic to simulate task progress, logs, and status changes.
- **Multi-Tab Interface**: Dynamic tabs for 'Overview', 'Fundamental Data', and analysis modules.
- **Streaming Markdown**: Enabled typewriter-style streaming rendering for analysis reports using 'react-markdown'.
- **Config Page**: Implemented settings for AI Providers, Data Sources, and Templates using TanStack Query.
3. **Documentation**:
- Created v2.0 User Guide ('docs/1_requirements/20251122_[Active]_user-guide_v2.md').
- Implemented 'DocsPage' in frontend to render the user guide directly within the app.
4. **Backend Alignment**:
- Created 'docs/frontend/backend_todos.md' outlining necessary backend adaptations (OpenAPI, Progress tracking).
This commit establishes the full frontend 'shell' ready for backend integration.
- Expose API Gateway port 4000 in docker-compose for local dev
- Enable dynamic API_GATEWAY_URL in Next.js config
- Add 64K context hard limit in report-generator to avoid LLM errors
- Add backend API readiness report
- Fix `simple_test_analysis` template in E2E test setup to align with Orchestrator's data fetch logic.
- Implement and verify additional E2E scenarios:
- Scenario C: Partial Provider Failure (verified error propagation fix in Orchestrator).
- Scenario D: Invalid Symbol input.
- Scenario E: Analysis Module failure.
- Update `WorkflowStateMachine::handle_report_failed` to correctly scope error broadcasting to the specific task instead of failing effectively silently or broadly.
- Update testing strategy documentation to reflect completed Phase 4 testing.
- Skip Scenario B (Orchestrator Restart) as persistence is not yet implemented (decision made to defer persistence).
- Enhance LlmClient to handle malformed URLs and HTML error responses
- Improve logging in report-generator-service worker
- Update frontend API routes and hooks for analysis
- Update various service configurations and persistence logic
- Backend: Add template_id to DataRequest and AnalysisResult metadata
- Frontend: Add TemplateSelector to Report page
- Frontend: Refactor ReportPage to use AnalysisTemplateSets for dynamic tabs
- Frontend: Strict mode for report rendering based on template_id
- Cleanup: Remove legacy analysis config hooks
- Enable dynamic config loading from DB to respect enabled status
- Implement Yahoo Finance Crumb authentication flow
- Add browser-like User-Agent and cookie store
- Map .SH symbols to .SS for Yahoo compatibility
- Add Google DNS to docker-compose to fix container resolution issues
- Fix incorrect parameter injection in MCP client query method
- Correct tool name from OVERVIEW to COMPANY_OVERVIEW
- Add symbol conversion support (e.g., .SH -> .SS)
- Implement fault tolerance for empty data responses to prevent panics
- Add workaround for non-standard JSON responses (single quotes) in GLOBAL_QUOTE
- Add debug_mcp utility for tool inspection
- Tushare: Added /test endpoint to verify API token validity by fetching a small dataset.
- AlphaVantage: Implemented custom HTTP transport to handle MCP server's 400 Bad Request response on SSE endpoint gracefully (degrading to POST-only mode).
- AlphaVantage: Added /test endpoint using `list_tools` to verify MCP connection.
- AlphaVantage: Updated configuration polling to support dynamic API URLs.
This commit introduces a comprehensive, template-based analysis orchestration system, refactoring the entire analysis generation workflow from the ground up.
Key Changes:
1. **Backend Architecture (`report-generator-service`):**
* Replaced the naive analysis workflow with a robust orchestrator based on a Directed Acyclic Graph (DAG) of module dependencies.
* Implemented a full topological sort (`petgraph`) to determine the correct execution order and detect circular dependencies.
2. **Data Models (`common-contracts`, `data-persistence-service`):**
* Introduced the concept of `AnalysisTemplateSets` to allow for multiple, independent, and configurable analysis workflows.
* Created a new `analysis_results` table to persist the output of each module for every analysis run, ensuring traceability.
* Implemented a file-free data seeding mechanism to populate default analysis templates on service startup.
3. **API Layer (`api-gateway`):**
* Added a new asynchronous endpoint (`POST /analysis-requests/{symbol}`) to trigger analysis workflows via NATS messages.
* Updated all configuration endpoints to support the new `AnalysisTemplateSets` model.
4. **Frontend UI (`/config`, `/query`):**
* Completely refactored the "Analysis Config" page into a two-level management UI for "Template Sets" and the "Modules" within them, supporting full CRUD operations.
* Updated the "Query" page to allow users to select which analysis template to use when generating a report.
This new architecture provides a powerful, flexible, and robust foundation for all future development of our intelligent analysis capabilities.
- Fix Dockerfile stub builds; compile full sources (no empty binaries)
- Add ca-certificates and curl in runtime images for TLS/healthchecks
- Enable RUST_LOG and RUST_BACKTRACE for all providers
- Add HTTP /health healthchecks in docker-compose for ports 8000-8004
- Standardize Tushare /health to structured HealthStatus JSON
- Enforce strict config validation (FINNHUB_API_KEY, TUSHARE_API_TOKEN)
- Map provider API keys via .env in docker-compose
- Log provider_services at API Gateway startup for diagnostics
Outcome: provider containers no longer exit silently; missing keys fail fast with explicit errors; health and logs are consistent across modules.
- remove Next.js rewrites to http://127.0.0.1:8000
- require NEXT_PUBLIC_BACKEND_URL in API routes (config, financials, config/test)
- prevent accidental fallback to host ports; use container service name backend:8000
