feat: update analysis workflow and fix LLM client connection issues
- Enhance LlmClient to handle malformed URLs and HTML error responses - Improve logging in report-generator-service worker - Update frontend API routes and hooks for analysis - Update various service configurations and persistence logic
This commit is contained in:
parent
e855a16c69
commit
d28f3c5266
@ -83,6 +83,7 @@ services:
|
|||||||
SERVER_PORT: 4000
|
SERVER_PORT: 4000
|
||||||
NATS_ADDR: nats://nats:4222
|
NATS_ADDR: nats://nats:4222
|
||||||
DATA_PERSISTENCE_SERVICE_URL: http://data-persistence-service:3000/api/v1
|
DATA_PERSISTENCE_SERVICE_URL: http://data-persistence-service:3000/api/v1
|
||||||
|
REPORT_GENERATOR_SERVICE_URL: http://report-generator-service:8004
|
||||||
# provider_services via explicit JSON for deterministic parsing
|
# provider_services via explicit JSON for deterministic parsing
|
||||||
PROVIDER_SERVICES: '["http://alphavantage-provider-service:8000", "http://tushare-provider-service:8001", "http://finnhub-provider-service:8002", "http://yfinance-provider-service:8003"]'
|
PROVIDER_SERVICES: '["http://alphavantage-provider-service:8000", "http://tushare-provider-service:8001", "http://finnhub-provider-service:8002", "http://yfinance-provider-service:8003"]'
|
||||||
RUST_LOG: info,axum=info
|
RUST_LOG: info,axum=info
|
||||||
@ -94,6 +95,7 @@ services:
|
|||||||
- tushare-provider-service
|
- tushare-provider-service
|
||||||
- finnhub-provider-service
|
- finnhub-provider-service
|
||||||
- yfinance-provider-service
|
- yfinance-provider-service
|
||||||
|
- report-generator-service
|
||||||
networks:
|
networks:
|
||||||
- app-network
|
- app-network
|
||||||
healthcheck:
|
healthcheck:
|
||||||
|
|||||||
@ -0,0 +1,90 @@
|
|||||||
|
# 分析流程优化与数据缓存机制修复
|
||||||
|
|
||||||
|
## 1. 问题背景 (Problem Statement)
|
||||||
|
|
||||||
|
根据系统日志分析与代码排查,当前系统存在以下关键问题:
|
||||||
|
|
||||||
|
1. **数据源重复请求 (Missing Cache Logic)**:
|
||||||
|
* `yfinance-provider-service` (及其他数据服务) 在接收到任务指令时,未检查本地数据库是否存在有效数据,而是直接向外部 API 发起请求。
|
||||||
|
* 这导致每次用户点击,都会触发耗时约 1.5s 的外部抓取,既慢又浪费资源。
|
||||||
|
|
||||||
|
2. **任务依赖与执行时序错乱 (Race Condition)**:
|
||||||
|
* `api-gateway` 在接收到请求时,**同时**触发了数据抓取 (`DATA_FETCH_QUEUE`) 和分析报告生成 (`ANALYSIS_COMMANDS_QUEUE`)。
|
||||||
|
* 导致 `report-generator-service` 在数据还没抓回来时就启动了,读到空数据(或旧数据)后瞬间完成,导致 Token 消耗为 0,报告内容为空。
|
||||||
|
|
||||||
|
3. **前端无法展示数据 (Frontend Data Visibility)**:
|
||||||
|
* **根本原因**: API Gateway 路由缺失与路径映射错误。
|
||||||
|
* 前端 BFF (`frontend/src/app/api/financials/...`) 试图请求 `${BACKEND_BASE}/market-data/financial-statements/...`。
|
||||||
|
* 然而,`api-gateway` **并未暴露** 此路由(仅暴露了 `/v1/companies/{symbol}/profile`)。
|
||||||
|
* 因此,前端获取财务数据的请求全部 404 失败,导致界面始终显示 "暂无可展示的数据",即使用户多次运行也无效。
|
||||||
|
|
||||||
|
## 2. 目标 (Goals)
|
||||||
|
|
||||||
|
1. **实现"读写穿透"缓存策略**: 数据服务在抓取前必须先检查本地数据库数据的时效性。
|
||||||
|
2. **构建事件驱动的依赖工作流**: 分析服务必须严格等待数据服务完成后触发(通过 NATS 事件链)。
|
||||||
|
3. **修复数据访问层**: 确保 API Gateway 正确暴露并转发财务数据接口,使前端可见。
|
||||||
|
4. **定义数据时效性标准**: 针对不同类型数据实施差异化的缓存过期策略。
|
||||||
|
|
||||||
|
## 3. 详细技术方案 (Technical Plan)
|
||||||
|
|
||||||
|
### 3.1. 数据时效性与缓存策略 (Data Freshness Policy)
|
||||||
|
|
||||||
|
针对基本面分析场景,不同数据的更新频率和时效性要求如下:
|
||||||
|
|
||||||
|
| 数据类型 | 内容示例 | 更新频率 | 建议 TTL (缓存有效期) | 更新策略 |
|
||||||
|
| :--- | :--- | :--- | :--- | :--- |
|
||||||
|
| **公司概况 (Profile)** | 名称、行业、简介、高管 | 极低 (年/不定期) | **30 天** | **Stale-While-Revalidate (SWR)**<br>过期后先返回旧数据,后台异步更新。 |
|
||||||
|
| **财务报表 (Financials)** | 营收、利润、资产负债 (季/年) | 季度 (4/8/10月) | **24 小时** | **Cache-Aside**<br>每次请求先查库。若 `updated_at > 24h`,则强制 Fetch;否则直接返回库中数据。<br>*注: 对于同一天内的重复请求,将直接命中缓存,0延迟。* |
|
||||||
|
| **市场数据 (Market Data)** | PE/PB/市值/价格 | 实时/日频 | **1 小时** | 基本面分析不需要秒级价格。取最近一小时内的快照即可。若需实时价格,使用专用实时接口。 |
|
||||||
|
|
||||||
|
### 3.2. 数据服务层 (Providers)
|
||||||
|
* **涉及服务**: `yfinance-provider-service`, `alphavantage-provider-service`, `finnhub-provider-service`.
|
||||||
|
* **逻辑变更**:
|
||||||
|
1. 订阅 `FetchCommand`。
|
||||||
|
2. **Step 1 (Check DB)**: 调用 `PersistenceClient` 获取目标 Symbol 数据的 `updated_at`。
|
||||||
|
3. **Step 2 (Decision)**:
|
||||||
|
* 若 `now - updated_at < TTL`: **Hit Cache**. Log "Cache Hit", 跳过外部请求,直接进入 Step 4。
|
||||||
|
* 若数据不存在 或 `now - updated_at > TTL`: **Miss Cache**. Log "Cache Miss", 执行外部 API 抓取。
|
||||||
|
4. **Step 3 (Upsert)**: 将抓取的数据存入 DB (Update `updated_at` = now)。
|
||||||
|
5. **Step 4 (Publish Event)**: 发布 `CompanyDataPersistedEvent` (包含 symbol, data_types: ["profile", "financials"])。
|
||||||
|
|
||||||
|
### 3.3. 工作流编排 (Workflow Orchestration)
|
||||||
|
* **API Gateway**:
|
||||||
|
* 移除 `POST /data-requests` 中自动触发 Analysis 的逻辑。
|
||||||
|
* 只发布 `FetchCompanyDataCommand`。
|
||||||
|
* **Report Generator**:
|
||||||
|
* **不再监听** `StartAnalysisCommand` (作为触发源)。
|
||||||
|
* 改为监听 `CompanyDataPersistedEvent`。
|
||||||
|
* 收到事件后,检查事件中的 `request_id` 是否关联了待处理的分析任务(或者简单的:收到数据更新就检查是否有待跑的分析模板)。
|
||||||
|
* *临时方案*: 为了简化,可以在 API Gateway 发送 Fetch 命令时,在 payload 里带上 `trigger_analysis: true` 和 `template_id`。Data Provider 在发出的 `PersistedEvent` 里透传这些字段。Report Generator 看到 `trigger_analysis: true` 才执行。
|
||||||
|
|
||||||
|
### 3.4. API 修复 (Fixing Visibility)
|
||||||
|
* **Backend (API Gateway)**:
|
||||||
|
* 在 `create_v1_router` 中新增路由:
|
||||||
|
* `GET /v1/market-data/financial-statements/{symbol}` -> 转发至 Data Persistence Service。
|
||||||
|
* `GET /v1/market-data/quotes/{symbol}` -> 转发至 Data Persistence Service (可选)。
|
||||||
|
* **Frontend (Next.js API Route)**:
|
||||||
|
* 修改 `frontend/src/app/api/financials/[...slug]/route.ts`。
|
||||||
|
* 将请求路径从 `${BACKEND_BASE}/market-data/...` 修正为 `${BACKEND_BASE}/v1/market-data/...` (匹配 Gateway 新路由)。
|
||||||
|
* 或者直接修正为 Data Persistence Service 的正确路径 (但最佳实践是走 Gateway)。
|
||||||
|
|
||||||
|
## 4. 执行计划 (Action Items)
|
||||||
|
|
||||||
|
### Phase 1: API & Frontend 可见性修复 (立即执行)
|
||||||
|
1. [x] **API Gateway**: 添加 `/v1/market-data/financial-statements/{symbol}` 路由。
|
||||||
|
2. [x] **Frontend**: 修正 `route.ts` 中的后端请求路径。(通过修正 Gateway 路由适配前端)
|
||||||
|
3. [ ] **验证**: 打开页面,应能看到(哪怕是旧的)财务图表数据,不再显示 404/无数据。
|
||||||
|
|
||||||
|
### Phase 2: 缓存与时效性逻辑 (核心)
|
||||||
|
4. [x] **Data Providers**: 在 `worker.rs` 中实现 TTL 检查逻辑 (Profile: 30d, Financials: 24h)。(YFinance 已实现,其他 Provider 已适配事件)
|
||||||
|
5. [x] **Persistence Service**: 确保 `get_company_profile` 和 `get_financials` 返回 `updated_at` 字段(如果还没有的话)。
|
||||||
|
|
||||||
|
### Phase 3: 事件驱动工作流 (解决 Race Condition)
|
||||||
|
6. [x] **Contracts**: 定义新事件 `CompanyDataPersistedEvent` (含 `trigger_analysis` 标记)。
|
||||||
|
7. [x] **API Gateway**: 停止直接发送 Analysis 命令,将其参数打包进 Fetch 命令。
|
||||||
|
8. [x] **Data Providers**: 完成任务后发布 `PersistedEvent`。
|
||||||
|
9. [x] **Report Generator**: 监听 `PersistedEvent` 触发分析。
|
||||||
|
|
||||||
|
## 5. 待确认
|
||||||
|
* 是否需要为每个数据源单独设置 TTL?(暂定统一策略)
|
||||||
|
* 前端是否需要显示数据的"上次更新时间"?(建议加上,增强用户信任)
|
||||||
55
frontend/Dockerfile.prod
Normal file
55
frontend/Dockerfile.prod
Normal file
@ -0,0 +1,55 @@
|
|||||||
|
FROM node:20-alpine AS base
|
||||||
|
|
||||||
|
# 1. Install dependencies only when needed
|
||||||
|
FROM base AS deps
|
||||||
|
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
|
||||||
|
RUN apk add --no-cache libc6-compat
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Install dependencies
|
||||||
|
COPY frontend/package.json frontend/package-lock.json* ./
|
||||||
|
RUN npm ci
|
||||||
|
|
||||||
|
# 2. Rebuild the source code only when needed
|
||||||
|
FROM base AS builder
|
||||||
|
WORKDIR /app
|
||||||
|
COPY --from=deps /app/node_modules ./node_modules
|
||||||
|
COPY frontend ./
|
||||||
|
|
||||||
|
# Environment variables
|
||||||
|
ENV NEXT_TELEMETRY_DISABLED=1
|
||||||
|
ENV NODE_ENV=production
|
||||||
|
|
||||||
|
RUN npm run build
|
||||||
|
|
||||||
|
# 3. Production image, copy all the files and run next
|
||||||
|
FROM base AS runner
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
ENV NODE_ENV=production
|
||||||
|
ENV NEXT_TELEMETRY_DISABLED=1
|
||||||
|
|
||||||
|
RUN addgroup --system --gid 1001 nodejs
|
||||||
|
RUN adduser --system --uid 1001 nextjs
|
||||||
|
|
||||||
|
COPY --from=builder /app/public ./public
|
||||||
|
|
||||||
|
# Set the correct permission for prerender cache
|
||||||
|
RUN mkdir .next
|
||||||
|
RUN chown nextjs:nodejs .next
|
||||||
|
|
||||||
|
# Automatically leverage output traces to reduce image size
|
||||||
|
# https://nextjs.org/docs/advanced-features/output-file-tracing
|
||||||
|
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
|
||||||
|
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
|
||||||
|
|
||||||
|
USER nextjs
|
||||||
|
|
||||||
|
EXPOSE 3000
|
||||||
|
|
||||||
|
ENV PORT=3000
|
||||||
|
ENV HOSTNAME="0.0.0.0"
|
||||||
|
|
||||||
|
CMD ["node", "server.js"]
|
||||||
|
|
||||||
@ -13,6 +13,9 @@ const nextConfig = {
|
|||||||
experimental: {
|
experimental: {
|
||||||
proxyTimeout: 300000, // 300 seconds (5 minutes)
|
proxyTimeout: 300000, // 300 seconds (5 minutes)
|
||||||
},
|
},
|
||||||
|
// Optimize for Docker deployment only in production
|
||||||
|
// 当 NODE_ENV 为 production 时开启 standalone 模式
|
||||||
|
output: process.env.NODE_ENV === 'production' ? 'standalone' : undefined,
|
||||||
};
|
};
|
||||||
|
|
||||||
export default nextConfig;
|
export default nextConfig;
|
||||||
29
frontend/src/app/api/analysis-results/route.ts
Normal file
29
frontend/src/app/api/analysis-results/route.ts
Normal file
@ -0,0 +1,29 @@
|
|||||||
|
import { NextRequest } from 'next/server';
|
||||||
|
|
||||||
|
const BACKEND_BASE = process.env.BACKEND_INTERNAL_URL || process.env.NEXT_PUBLIC_BACKEND_URL;
|
||||||
|
|
||||||
|
export async function GET(req: NextRequest) {
|
||||||
|
if (!BACKEND_BASE) {
|
||||||
|
return new Response('BACKEND_INTERNAL_URL/NEXT_PUBLIC_BACKEND_URL 未配置', { status: 500 });
|
||||||
|
}
|
||||||
|
const { searchParams } = new URL(req.url);
|
||||||
|
const symbol = searchParams.get('symbol');
|
||||||
|
|
||||||
|
if (!symbol) {
|
||||||
|
return new Response('Missing symbol parameter', { status: 400 });
|
||||||
|
}
|
||||||
|
|
||||||
|
const resp = await fetch(`${BACKEND_BASE}/analysis-results?symbol=${encodeURIComponent(symbol)}`, { cache: 'no-store' });
|
||||||
|
|
||||||
|
if (!resp.ok) {
|
||||||
|
if (resp.status === 404) {
|
||||||
|
// Return empty list if not found, to avoid UI errors
|
||||||
|
return Response.json([]);
|
||||||
|
}
|
||||||
|
return new Response(resp.statusText, { status: resp.status });
|
||||||
|
}
|
||||||
|
|
||||||
|
const data = await resp.json();
|
||||||
|
return Response.json(data);
|
||||||
|
}
|
||||||
|
|
||||||
41
frontend/src/app/api/configs/llm/test/route.ts
Normal file
41
frontend/src/app/api/configs/llm/test/route.ts
Normal file
@ -0,0 +1,41 @@
|
|||||||
|
import { NextRequest } from 'next/server';
|
||||||
|
|
||||||
|
const BACKEND_BASE = process.env.BACKEND_INTERNAL_URL || process.env.NEXT_PUBLIC_BACKEND_URL;
|
||||||
|
|
||||||
|
export async function POST(req: NextRequest) {
|
||||||
|
if (!BACKEND_BASE) {
|
||||||
|
return new Response('BACKEND_INTERNAL_URL/NEXT_PUBLIC_BACKEND_URL 未配置', { status: 500 });
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const body = await req.json();
|
||||||
|
|
||||||
|
// 将请求转发到 API Gateway
|
||||||
|
const targetUrl = `${BACKEND_BASE.replace(/\/$/, '')}/configs/llm/test`;
|
||||||
|
|
||||||
|
const backendRes = await fetch(targetUrl, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: {
|
||||||
|
'Content-Type': 'application/json',
|
||||||
|
},
|
||||||
|
body: JSON.stringify(body),
|
||||||
|
});
|
||||||
|
|
||||||
|
const backendResBody = await backendRes.text();
|
||||||
|
|
||||||
|
return new Response(backendResBody, {
|
||||||
|
status: backendRes.status,
|
||||||
|
headers: {
|
||||||
|
'Content-Type': 'application/json',
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
} catch (error: any) {
|
||||||
|
console.error('LLM测试代理失败:', error);
|
||||||
|
return new Response(JSON.stringify({ success: false, message: error.message || '代理请求时发生未知错误' }), {
|
||||||
|
status: 500,
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
@ -23,10 +23,12 @@ export async function GET(
|
|||||||
const years = url.searchParams.get('years') || '10';
|
const years = url.searchParams.get('years') || '10';
|
||||||
|
|
||||||
// Fetch financials from backend
|
// Fetch financials from backend
|
||||||
const finResp = await fetch(
|
// Corrected path to match new API Gateway route
|
||||||
`${BACKEND_BASE}/market-data/financial-statements/${encodeURIComponent(symbol)}?metrics=${encodeURIComponent('')}`,
|
const metricsParam = url.searchParams.get('metrics') || '';
|
||||||
{ cache: 'no-store' }
|
const fetchUrl = `${BACKEND_BASE}/market-data/financial-statements/${encodeURIComponent(symbol)}` +
|
||||||
);
|
(metricsParam ? `?metrics=${encodeURIComponent(metricsParam)}` : '');
|
||||||
|
|
||||||
|
const finResp = await fetch(fetchUrl, { cache: 'no-store' });
|
||||||
|
|
||||||
if (!finResp.ok) {
|
if (!finResp.ok) {
|
||||||
if (finResp.status === 404) {
|
if (finResp.status === 404) {
|
||||||
@ -52,6 +54,7 @@ export async function GET(
|
|||||||
});
|
});
|
||||||
|
|
||||||
// Fetch Company Profile to populate name/industry
|
// Fetch Company Profile to populate name/industry
|
||||||
|
// Corrected path to match new API Gateway route
|
||||||
const profileResp = await fetch(`${BACKEND_BASE}/companies/${encodeURIComponent(symbol)}/profile`, { cache: 'no-store' });
|
const profileResp = await fetch(`${BACKEND_BASE}/companies/${encodeURIComponent(symbol)}/profile`, { cache: 'no-store' });
|
||||||
let profileData: any = {};
|
let profileData: any = {};
|
||||||
if (profileResp.ok) {
|
if (profileResp.ok) {
|
||||||
@ -104,6 +107,64 @@ export async function GET(
|
|||||||
const text = await resp.text();
|
const text = await resp.text();
|
||||||
return new Response(text, { status: resp.status, headers: { 'Content-Type': 'application/json' } });
|
return new Response(text, { status: resp.status, headers: { 'Content-Type': 'application/json' } });
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// 2. Match /api/financials/{market}/{symbol}/analysis/{type}/stream
|
||||||
|
// slug length = 5
|
||||||
|
// slug[0] = market
|
||||||
|
// slug[1] = symbol
|
||||||
|
// slug[2] = 'analysis'
|
||||||
|
// slug[3] = analysisType (module_id)
|
||||||
|
// slug[4] = 'stream'
|
||||||
|
if (slug.length === 5 && slug[2] === 'analysis' && slug[4] === 'stream') {
|
||||||
|
const symbol = slug[1];
|
||||||
|
const analysisType = slug[3];
|
||||||
|
|
||||||
|
const encoder = new TextEncoder();
|
||||||
|
|
||||||
|
const stream = new ReadableStream({
|
||||||
|
async start(controller) {
|
||||||
|
// Polling logic
|
||||||
|
// We try for up to 60 seconds
|
||||||
|
const maxRetries = 30;
|
||||||
|
let found = false;
|
||||||
|
|
||||||
|
for (let i = 0; i < maxRetries; i++) {
|
||||||
|
try {
|
||||||
|
const resp = await fetch(`${BACKEND_BASE}/analysis-results?symbol=${encodeURIComponent(symbol)}&module_id=${encodeURIComponent(analysisType)}`, { cache: 'no-store' });
|
||||||
|
|
||||||
|
if (resp.ok) {
|
||||||
|
const results = await resp.json();
|
||||||
|
// Assuming results are sorted by created_at DESC (backend behavior)
|
||||||
|
if (Array.isArray(results) && results.length > 0) {
|
||||||
|
const latest = results[0];
|
||||||
|
// If result is found, send it and exit
|
||||||
|
if (latest && latest.content) {
|
||||||
|
controller.enqueue(encoder.encode(latest.content));
|
||||||
|
found = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
console.error("Error polling analysis results", e);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait 2 seconds before next poll
|
||||||
|
await new Promise(resolve => setTimeout(resolve, 2000));
|
||||||
|
}
|
||||||
|
controller.close();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
return new Response(stream, {
|
||||||
|
headers: {
|
||||||
|
'Content-Type': 'text/plain; charset=utf-8',
|
||||||
|
'Cache-Control': 'no-cache',
|
||||||
|
'Connection': 'keep-alive'
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
// 其他旧 financials 端点在新架构中未实现:返回空对象以避免前端 JSON 解析错误
|
// 其他旧 financials 端点在新架构中未实现:返回空对象以避免前端 JSON 解析错误
|
||||||
return Response.json({}, { status: 200 });
|
return Response.json({}, { status: 200 });
|
||||||
}
|
}
|
||||||
|
|||||||
@ -9,8 +9,16 @@ import { Badge } from "@/components/ui/badge";
|
|||||||
import { ScrollArea } from "@/components/ui/scroll-area";
|
import { ScrollArea } from "@/components/ui/scroll-area";
|
||||||
import { Spinner } from "@/components/ui/spinner";
|
import { Spinner } from "@/components/ui/spinner";
|
||||||
import { Separator } from "@/components/ui/separator";
|
import { Separator } from "@/components/ui/separator";
|
||||||
import { useLlmProviders, updateLlmProviders, discoverProviderModels } from '@/hooks/useApi';
|
import { useLlmProviders, updateLlmProviders, discoverProviderModels, testLlmModel } from '@/hooks/useApi';
|
||||||
import type { LlmProvidersConfig, LlmModel } from '@/types';
|
import type { LlmProvidersConfig, LlmModel } from '@/types';
|
||||||
|
import {
|
||||||
|
Dialog,
|
||||||
|
DialogContent,
|
||||||
|
DialogDescription,
|
||||||
|
DialogFooter,
|
||||||
|
DialogHeader,
|
||||||
|
DialogTitle,
|
||||||
|
} from "@/components/ui/dialog";
|
||||||
|
|
||||||
export function AIConfigTab({
|
export function AIConfigTab({
|
||||||
newProviderBaseUrl,
|
newProviderBaseUrl,
|
||||||
@ -45,6 +53,10 @@ export function AIConfigTab({
|
|||||||
const [newModelMenuOpen, setNewModelMenuOpen] = useState<Record<string, boolean>>({});
|
const [newModelMenuOpen, setNewModelMenuOpen] = useState<Record<string, boolean>>({});
|
||||||
const [newModelHighlightIndex, setNewModelHighlightIndex] = useState<Record<string, number>>({});
|
const [newModelHighlightIndex, setNewModelHighlightIndex] = useState<Record<string, number>>({});
|
||||||
|
|
||||||
|
// Test State
|
||||||
|
const [testResult, setTestResult] = useState<{ success: boolean; message: string } | null>(null);
|
||||||
|
const [isTesting, setIsTesting] = useState<string | null>(null);
|
||||||
|
|
||||||
// Refs for Auto-save
|
// Refs for Auto-save
|
||||||
const hasInitializedLlmRef = useRef(false);
|
const hasInitializedLlmRef = useRef(false);
|
||||||
const autoSaveTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
|
const autoSaveTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
|
||||||
@ -465,6 +477,24 @@ export function AIConfigTab({
|
|||||||
/>
|
/>
|
||||||
<label htmlFor={`${providerId}-${m.model_id}-active`} className="text-sm">启用</label>
|
<label htmlFor={`${providerId}-${m.model_id}-active`} className="text-sm">启用</label>
|
||||||
</div>
|
</div>
|
||||||
|
<Button
|
||||||
|
variant="outline"
|
||||||
|
size="sm"
|
||||||
|
disabled={isTesting === m.model_id}
|
||||||
|
onClick={async () => {
|
||||||
|
setIsTesting(m.model_id);
|
||||||
|
try {
|
||||||
|
const res = await testLlmModel(provider.api_base_url, provider.api_key || pendingApiKeys[providerId] || '', m.model_id);
|
||||||
|
setTestResult({ success: true, message: typeof res === 'string' ? res : JSON.stringify(res, null, 2) });
|
||||||
|
} catch (e: any) {
|
||||||
|
setTestResult({ success: false, message: e.message || 'Test failed' });
|
||||||
|
} finally {
|
||||||
|
setIsTesting(null);
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{isTesting === m.model_id ? <Spinner className="h-3 w-3" /> : 'Test'}
|
||||||
|
</Button>
|
||||||
<Button
|
<Button
|
||||||
variant="destructive"
|
variant="destructive"
|
||||||
onClick={() => {
|
onClick={() => {
|
||||||
@ -634,7 +664,23 @@ export function AIConfigTab({
|
|||||||
})}
|
})}
|
||||||
</div>
|
</div>
|
||||||
</CardContent>
|
</CardContent>
|
||||||
|
|
||||||
|
<Dialog open={!!testResult} onOpenChange={(open) => !open && setTestResult(null)}>
|
||||||
|
<DialogContent>
|
||||||
|
<DialogHeader>
|
||||||
|
<DialogTitle>{testResult?.success ? 'Test Successful' : 'Test Failed'}</DialogTitle>
|
||||||
|
<DialogDescription>
|
||||||
|
Response from model:
|
||||||
|
</DialogDescription>
|
||||||
|
</DialogHeader>
|
||||||
|
<div className="max-h-[300px] overflow-auto p-4 bg-muted rounded-md whitespace-pre-wrap font-mono text-sm">
|
||||||
|
{testResult?.message}
|
||||||
|
</div>
|
||||||
|
<DialogFooter>
|
||||||
|
<Button onClick={() => setTestResult(null)}>Close</Button>
|
||||||
|
</DialogFooter>
|
||||||
|
</DialogContent>
|
||||||
|
</Dialog>
|
||||||
</Card>
|
</Card>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -56,7 +56,10 @@ export function useAnalysisRunner(
|
|||||||
// Derive effective analysis config from template set, falling back to global config if needed
|
// Derive effective analysis config from template set, falling back to global config if needed
|
||||||
const activeAnalysisConfig = useMemo(() => {
|
const activeAnalysisConfig = useMemo(() => {
|
||||||
if (activeTemplateSet) {
|
if (activeTemplateSet) {
|
||||||
return { analysis_modules: activeTemplateSet.modules };
|
return {
|
||||||
|
...financialConfig,
|
||||||
|
analysis_modules: activeTemplateSet.modules,
|
||||||
|
};
|
||||||
}
|
}
|
||||||
return financialConfig; // Fallback to global config (legacy behavior)
|
return financialConfig; // Fallback to global config (legacy behavior)
|
||||||
}, [activeTemplateSet, financialConfig]);
|
}, [activeTemplateSet, financialConfig]);
|
||||||
|
|||||||
@ -1,5 +1,5 @@
|
|||||||
import { useParams, useSearchParams } from 'next/navigation';
|
import { useParams, useSearchParams } from 'next/navigation';
|
||||||
import { useChinaFinancials, useFinancials, useFinancialConfig, useAnalysisTemplateSets, useSnapshot, useRealtimeQuote } from '@/hooks/useApi';
|
import { useChinaFinancials, useFinancials, useFinancialConfig, useAnalysisTemplateSets, useSnapshot, useRealtimeQuote, useAnalysisResults } from '@/hooks/useApi';
|
||||||
|
|
||||||
export function useReportData() {
|
export function useReportData() {
|
||||||
const params = useParams();
|
const params = useParams();
|
||||||
@ -42,6 +42,7 @@ export function useReportData() {
|
|||||||
const { data: realtime, error: realtimeError, isLoading: realtimeLoading } = useRealtimeQuote(normalizedMarket, unifiedSymbol, { maxAgeSeconds: 30, refreshIntervalMs: 5000 });
|
const { data: realtime, error: realtimeError, isLoading: realtimeLoading } = useRealtimeQuote(normalizedMarket, unifiedSymbol, { maxAgeSeconds: 30, refreshIntervalMs: 5000 });
|
||||||
const { data: financialConfig } = useFinancialConfig();
|
const { data: financialConfig } = useFinancialConfig();
|
||||||
const { data: templateSets } = useAnalysisTemplateSets();
|
const { data: templateSets } = useAnalysisTemplateSets();
|
||||||
|
const { data: historicalAnalysis } = useAnalysisResults(unifiedSymbol);
|
||||||
|
|
||||||
return {
|
return {
|
||||||
symbol,
|
symbol,
|
||||||
@ -59,7 +60,8 @@ export function useReportData() {
|
|||||||
realtimeLoading,
|
realtimeLoading,
|
||||||
realtimeError,
|
realtimeError,
|
||||||
financialConfig: financialConfig as any,
|
financialConfig: financialConfig as any,
|
||||||
templateSets
|
templateSets,
|
||||||
|
historicalAnalysis
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -25,9 +25,11 @@ export default function ReportPage() {
|
|||||||
realtimeLoading,
|
realtimeLoading,
|
||||||
realtimeError,
|
realtimeError,
|
||||||
financialConfig,
|
financialConfig,
|
||||||
|
templateSets,
|
||||||
} = useReportData();
|
} = useReportData();
|
||||||
|
|
||||||
const {
|
const {
|
||||||
|
activeAnalysisConfig,
|
||||||
analysisTypes,
|
analysisTypes,
|
||||||
analysisStates,
|
analysisStates,
|
||||||
analysisRecords,
|
analysisRecords,
|
||||||
@ -45,7 +47,9 @@ export default function ReportPage() {
|
|||||||
retryAnalysis,
|
retryAnalysis,
|
||||||
hasRunningTask,
|
hasRunningTask,
|
||||||
isAnalysisRunning,
|
isAnalysisRunning,
|
||||||
} = useAnalysisRunner(financials, financialConfig, normalizedMarket, unifiedSymbol, isLoading, error);
|
selectedTemplateId,
|
||||||
|
setSelectedTemplateId,
|
||||||
|
} = useAnalysisRunner(financials, financialConfig, normalizedMarket, unifiedSymbol, isLoading, error, templateSets);
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="space-y-4">
|
<div className="space-y-4">
|
||||||
@ -63,6 +67,9 @@ export default function ReportPage() {
|
|||||||
onStartAnalysis={triggerAnalysis}
|
onStartAnalysis={triggerAnalysis}
|
||||||
onStopAnalysis={stopAll}
|
onStopAnalysis={stopAll}
|
||||||
onContinueAnalysis={continuePending}
|
onContinueAnalysis={continuePending}
|
||||||
|
templateSets={templateSets}
|
||||||
|
selectedTemplateId={selectedTemplateId}
|
||||||
|
onSelectTemplate={setSelectedTemplateId}
|
||||||
/>
|
/>
|
||||||
<TaskStatus
|
<TaskStatus
|
||||||
requestId={requestId}
|
requestId={requestId}
|
||||||
@ -71,7 +78,7 @@ export default function ReportPage() {
|
|||||||
elapsedSeconds={elapsedSeconds}
|
elapsedSeconds={elapsedSeconds}
|
||||||
completionProgress={completionProgress}
|
completionProgress={completionProgress}
|
||||||
currentAnalysisTask={currentAnalysisTask}
|
currentAnalysisTask={currentAnalysisTask}
|
||||||
analysisConfig={financialConfig}
|
analysisConfig={activeAnalysisConfig}
|
||||||
/>
|
/>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@ -81,7 +88,7 @@ export default function ReportPage() {
|
|||||||
<TabsTrigger value="financial">财务数据</TabsTrigger>
|
<TabsTrigger value="financial">财务数据</TabsTrigger>
|
||||||
{analysisTypes.map(type => (
|
{analysisTypes.map(type => (
|
||||||
<TabsTrigger key={type} value={type}>
|
<TabsTrigger key={type} value={type}>
|
||||||
{type === 'company_profile' ? '公司简介' : (financialConfig?.analysis_modules?.[type]?.name || type)}
|
{type === 'company_profile' ? '公司简介' : (activeAnalysisConfig?.analysis_modules?.[type]?.name || type)}
|
||||||
</TabsTrigger>
|
</TabsTrigger>
|
||||||
))}
|
))}
|
||||||
<TabsTrigger value="execution">执行详情</TabsTrigger>
|
<TabsTrigger value="execution">执行详情</TabsTrigger>
|
||||||
@ -102,7 +109,7 @@ export default function ReportPage() {
|
|||||||
financials={financials}
|
financials={financials}
|
||||||
isLoading={isLoading}
|
isLoading={isLoading}
|
||||||
error={error}
|
error={error}
|
||||||
financialConfig={financialConfig}
|
financialConfig={activeAnalysisConfig}
|
||||||
/>
|
/>
|
||||||
</TabsContent>
|
</TabsContent>
|
||||||
|
|
||||||
@ -112,7 +119,7 @@ export default function ReportPage() {
|
|||||||
analysisType={analysisType}
|
analysisType={analysisType}
|
||||||
state={analysisStates[analysisType] || { content: '', loading: false, error: null }}
|
state={analysisStates[analysisType] || { content: '', loading: false, error: null }}
|
||||||
financials={financials}
|
financials={financials}
|
||||||
analysisConfig={financialConfig}
|
analysisConfig={activeAnalysisConfig}
|
||||||
retryAnalysis={retryAnalysis}
|
retryAnalysis={retryAnalysis}
|
||||||
currentAnalysisTask={currentAnalysisTask}
|
currentAnalysisTask={currentAnalysisTask}
|
||||||
/>
|
/>
|
||||||
@ -134,5 +141,3 @@ export default function ReportPage() {
|
|||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@ -118,8 +118,9 @@ export function TradingViewWidget({
|
|||||||
if (container.isConnected) {
|
if (container.isConnected) {
|
||||||
container.appendChild(script);
|
container.appendChild(script);
|
||||||
}
|
}
|
||||||
} catch {
|
} catch (e) {
|
||||||
// 忽略偶发性 contentWindow 不可用的报错
|
// 忽略偶发性 contentWindow 不可用的报错
|
||||||
|
console.warn('TradingView widget mount error:', e);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
@ -10,24 +10,39 @@ type DialogProps = BaseProps & {
|
|||||||
onOpenChange?: (open: boolean) => void;
|
onOpenChange?: (open: boolean) => void;
|
||||||
};
|
};
|
||||||
|
|
||||||
export const Dialog: React.FC<DialogProps> = ({ children }) => {
|
export const Dialog: React.FC<DialogProps> = ({ children, open, onOpenChange }) => {
|
||||||
return <div>{children}</div>;
|
if (!open) return null;
|
||||||
|
return (
|
||||||
|
<div className="fixed inset-0 z-50 flex items-center justify-center bg-black/50">
|
||||||
|
<div className="relative bg-background p-6 rounded-lg shadow-lg max-w-lg w-full">
|
||||||
|
{children}
|
||||||
|
<button
|
||||||
|
className="absolute top-4 right-4 text-gray-500 hover:text-gray-700"
|
||||||
|
onClick={() => onOpenChange?.(false)}
|
||||||
|
>
|
||||||
|
✕
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
};
|
};
|
||||||
|
|
||||||
export const DialogContent: React.FC<BaseProps> = ({ children, className }) => {
|
export const DialogContent: React.FC<BaseProps> = ({ children, className }) => {
|
||||||
return <div className={className}>{children}</div>;
|
return <div className={className}>{children}</div>;
|
||||||
};
|
};
|
||||||
|
|
||||||
export const DialogHeader: React.FC<BaseProps> = ({ children }) => {
|
export const DialogHeader: React.FC<BaseProps> = ({ children, className }) => {
|
||||||
return <div>{children}</div>;
|
return <div className={`flex flex-col space-y-1.5 text-center sm:text-left ${className}`}>{children}</div>;
|
||||||
};
|
};
|
||||||
|
|
||||||
export const DialogTitle: React.FC<BaseProps> = ({ children }) => {
|
export const DialogTitle: React.FC<BaseProps> = ({ children, className }) => {
|
||||||
return <h3>{children}</h3>;
|
return <h3 className={`text-lg font-semibold leading-none tracking-tight ${className}`}>{children}</h3>;
|
||||||
};
|
};
|
||||||
|
|
||||||
export const DialogFooter: React.FC<BaseProps> = ({ children }) => {
|
export const DialogDescription: React.FC<BaseProps> = ({ children, className }) => {
|
||||||
return <div>{children}</div>;
|
return <p className={`text-sm text-muted-foreground ${className}`}>{children}</p>;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
export const DialogFooter: React.FC<BaseProps> = ({ children, className }) => {
|
||||||
|
return <div className={`flex flex-col-reverse sm:flex-row sm:justify-end sm:space-x-2 ${className}`}>{children}</div>;
|
||||||
|
};
|
||||||
|
|||||||
@ -351,6 +351,19 @@ export async function discoverProviderModelsPreview(apiBaseUrl: string, apiKey:
|
|||||||
return res.json();
|
return res.json();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export async function testLlmModel(apiBaseUrl: string, apiKey: string, modelId: string) {
|
||||||
|
const res = await fetch('/api/configs/llm/test', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ api_base_url: apiBaseUrl, api_key: apiKey, model_id: modelId }),
|
||||||
|
});
|
||||||
|
if (!res.ok) {
|
||||||
|
const text = await res.text().catch(() => '');
|
||||||
|
throw new Error(text || `HTTP ${res.status}`);
|
||||||
|
}
|
||||||
|
return res.json();
|
||||||
|
}
|
||||||
|
|
||||||
// --- Analysis Template Sets Config Hooks (NEW) ---
|
// --- Analysis Template Sets Config Hooks (NEW) ---
|
||||||
|
|
||||||
export function useAnalysisTemplateSets() {
|
export function useAnalysisTemplateSets() {
|
||||||
|
|||||||
287
scripts/deploy_to_harbor.sh
Executable file
287
scripts/deploy_to_harbor.sh
Executable file
@ -0,0 +1,287 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# 遇到错误立即退出
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# 配置变量
|
||||||
|
REGISTRY="harbor.3prism.ai"
|
||||||
|
PROJECT="fundamental_analysis"
|
||||||
|
VERSION="latest" # 或者使用 $(date +%Y%m%d%H%M%S) 生成时间戳版本
|
||||||
|
NAMESPACE="$REGISTRY/$PROJECT"
|
||||||
|
|
||||||
|
# 颜色输出
|
||||||
|
GREEN='\033[0;32m'
|
||||||
|
YELLOW='\033[1;33m'
|
||||||
|
RED='\033[0;31m'
|
||||||
|
NC='\033[0m' # No Color
|
||||||
|
|
||||||
|
echo -e "${GREEN}=== 开始构建并推送镜像到 $NAMESPACE ===${NC}"
|
||||||
|
|
||||||
|
# 定义服务列表及其 Dockerfile 路径
|
||||||
|
# 格式: "服务名:Dockerfile路径"
|
||||||
|
SERVICES=(
|
||||||
|
"data-persistence-service:services/data-persistence-service/Dockerfile"
|
||||||
|
"api-gateway:services/api-gateway/Dockerfile"
|
||||||
|
"alphavantage-provider-service:services/alphavantage-provider-service/Dockerfile"
|
||||||
|
"tushare-provider-service:services/tushare-provider-service/Dockerfile"
|
||||||
|
"finnhub-provider-service:services/finnhub-provider-service/Dockerfile"
|
||||||
|
"yfinance-provider-service:services/yfinance-provider-service/Dockerfile"
|
||||||
|
"report-generator-service:services/report-generator-service/Dockerfile"
|
||||||
|
"frontend:frontend/Dockerfile.prod"
|
||||||
|
)
|
||||||
|
|
||||||
|
# 总大小计数器
|
||||||
|
TOTAL_SIZE=0
|
||||||
|
|
||||||
|
for entry in "${SERVICES[@]}"; do
|
||||||
|
KEY="${entry%%:*}"
|
||||||
|
DOCKERFILE="${entry#*:}"
|
||||||
|
IMAGE_NAME="$NAMESPACE/$KEY:$VERSION"
|
||||||
|
|
||||||
|
echo -e "\n${YELLOW}>>> 正在构建 $KEY ...${NC}"
|
||||||
|
echo "使用 Dockerfile: $DOCKERFILE"
|
||||||
|
|
||||||
|
# 构建镜像
|
||||||
|
# 注意:构建上下文始终为项目根目录 (.)
|
||||||
|
docker build -t "$IMAGE_NAME" -f "$DOCKERFILE" .
|
||||||
|
|
||||||
|
# 获取镜像大小 (MB)
|
||||||
|
SIZE_BYTES=$(docker inspect "$IMAGE_NAME" --format='{{.Size}}')
|
||||||
|
SIZE_MB=$(echo "scale=2; $SIZE_BYTES / 1024 / 1024" | bc)
|
||||||
|
|
||||||
|
echo -e "${GREEN}√ $KEY 构建完成. 大小: ${SIZE_MB} MB${NC}"
|
||||||
|
|
||||||
|
# 累加大小
|
||||||
|
TOTAL_SIZE=$(echo "$TOTAL_SIZE + $SIZE_BYTES" | bc)
|
||||||
|
|
||||||
|
# 检查单个镜像大小是否异常 (例如超过 500MB 对于 Rust 微服务来说通常是不正常的,除非包含大模型)
|
||||||
|
if (( $(echo "$SIZE_MB > 500" | bc -l) )); then
|
||||||
|
echo -e "${RED}警告: $KEY 镜像大小超过 500MB,请检查 Dockerfile 是否包含不必要的文件!${NC}"
|
||||||
|
# 这里我们可以选择暂停询问用户,或者只是警告
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo -e "${YELLOW}>>> 正在推送 $KEY 到 Harbor ...${NC}"
|
||||||
|
docker push "$IMAGE_NAME"
|
||||||
|
done
|
||||||
|
|
||||||
|
TOTAL_SIZE_MB=$(echo "scale=2; $TOTAL_SIZE / 1024 / 1024" | bc)
|
||||||
|
echo -e "\n${GREEN}=== 所有镜像处理完成 ===${NC}"
|
||||||
|
echo -e "${GREEN}总大小: ${TOTAL_SIZE_MB} MB${NC}"
|
||||||
|
|
||||||
|
# 检查总大小是否超过 1GB (1024 MB)
|
||||||
|
if (( $(echo "$TOTAL_SIZE_MB > 1024" | bc -l) )); then
|
||||||
|
echo -e "${RED}警告: 总镜像大小超过 1GB,请注意远程仓库的空间限制!${NC}"
|
||||||
|
else
|
||||||
|
echo -e "${GREEN}总大小在 1GB 限制范围内。${NC}"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 生成服务器使用的 docker-compose.server.yml
|
||||||
|
echo -e "\n${YELLOW}>>> 正在生成服务器部署文件 docker-compose.server.yml ...${NC}"
|
||||||
|
|
||||||
|
cat > docker-compose.server.yml <<EOF
|
||||||
|
services:
|
||||||
|
postgres-db:
|
||||||
|
image: timescale/timescaledb:2.15.2-pg16
|
||||||
|
container_name: fundamental-postgres
|
||||||
|
command: -c shared_preload_libraries=timescaledb
|
||||||
|
environment:
|
||||||
|
POSTGRES_USER: postgres
|
||||||
|
POSTGRES_PASSWORD: postgres
|
||||||
|
POSTGRES_DB: fundamental
|
||||||
|
volumes:
|
||||||
|
- pgdata:/var/lib/postgresql/data
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U postgres -d fundamental"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 10
|
||||||
|
networks:
|
||||||
|
- app-network
|
||||||
|
|
||||||
|
nats:
|
||||||
|
image: nats:2.9
|
||||||
|
volumes:
|
||||||
|
- nats_data:/data
|
||||||
|
networks:
|
||||||
|
- app-network
|
||||||
|
|
||||||
|
data-persistence-service:
|
||||||
|
image: $NAMESPACE/data-persistence-service:$VERSION
|
||||||
|
container_name: data-persistence-service
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
HOST: 0.0.0.0
|
||||||
|
PORT: 3000
|
||||||
|
DATABASE_URL: postgresql://postgres:postgres@postgres-db:5432/fundamental
|
||||||
|
RUST_LOG: info
|
||||||
|
RUST_BACKTRACE: "1"
|
||||||
|
depends_on:
|
||||||
|
postgres-db:
|
||||||
|
condition: service_healthy
|
||||||
|
networks:
|
||||||
|
- app-network
|
||||||
|
|
||||||
|
frontend:
|
||||||
|
image: $NAMESPACE/frontend:$VERSION
|
||||||
|
container_name: fundamental-frontend
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
NEXT_PUBLIC_BACKEND_URL: http://api-gateway:4000/v1
|
||||||
|
FRONTEND_INTERNAL_URL: http://fundamental-frontend:3000
|
||||||
|
BACKEND_INTERNAL_URL: http://api-gateway:4000/v1
|
||||||
|
NODE_ENV: production
|
||||||
|
ports:
|
||||||
|
- "3001:3000"
|
||||||
|
depends_on:
|
||||||
|
api-gateway:
|
||||||
|
condition: service_healthy
|
||||||
|
networks:
|
||||||
|
- app-network
|
||||||
|
|
||||||
|
api-gateway:
|
||||||
|
image: $NAMESPACE/api-gateway:$VERSION
|
||||||
|
container_name: api-gateway
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
SERVER_PORT: 4000
|
||||||
|
NATS_ADDR: nats://nats:4222
|
||||||
|
DATA_PERSISTENCE_SERVICE_URL: http://data-persistence-service:3000/api/v1
|
||||||
|
PROVIDER_SERVICES: '["http://alphavantage-provider-service:8000", "http://tushare-provider-service:8001", "http://finnhub-provider-service:8002", "http://yfinance-provider-service:8003"]'
|
||||||
|
RUST_LOG: info,axum=info
|
||||||
|
RUST_BACKTRACE: "1"
|
||||||
|
depends_on:
|
||||||
|
- nats
|
||||||
|
- data-persistence-service
|
||||||
|
- alphavantage-provider-service
|
||||||
|
- tushare-provider-service
|
||||||
|
- finnhub-provider-service
|
||||||
|
- yfinance-provider-service
|
||||||
|
networks:
|
||||||
|
- app-network
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "curl -fsS http://localhost:4000/health >/dev/null || exit 1"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 12
|
||||||
|
|
||||||
|
alphavantage-provider-service:
|
||||||
|
image: $NAMESPACE/alphavantage-provider-service:$VERSION
|
||||||
|
container_name: alphavantage-provider-service
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
SERVER_PORT: 8000
|
||||||
|
NATS_ADDR: nats://nats:4222
|
||||||
|
DATA_PERSISTENCE_SERVICE_URL: http://data-persistence-service:3000/api/v1
|
||||||
|
RUST_LOG: info,axum=info
|
||||||
|
RUST_BACKTRACE: "1"
|
||||||
|
depends_on:
|
||||||
|
- nats
|
||||||
|
- data-persistence-service
|
||||||
|
networks:
|
||||||
|
- app-network
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "curl -fsS http://localhost:8000/health >/dev/null || exit 1"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 12
|
||||||
|
|
||||||
|
tushare-provider-service:
|
||||||
|
image: $NAMESPACE/tushare-provider-service:$VERSION
|
||||||
|
container_name: tushare-provider-service
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
SERVER_PORT: 8001
|
||||||
|
NATS_ADDR: nats://nats:4222
|
||||||
|
DATA_PERSISTENCE_SERVICE_URL: http://data-persistence-service:3000/api/v1
|
||||||
|
TUSHARE_API_URL: http://api.waditu.com
|
||||||
|
RUST_LOG: info,axum=info
|
||||||
|
RUST_BACKTRACE: "1"
|
||||||
|
depends_on:
|
||||||
|
- nats
|
||||||
|
- data-persistence-service
|
||||||
|
networks:
|
||||||
|
- app-network
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "curl -fsS http://localhost:8001/health >/dev/null || exit 1"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 12
|
||||||
|
|
||||||
|
finnhub-provider-service:
|
||||||
|
image: $NAMESPACE/finnhub-provider-service:$VERSION
|
||||||
|
container_name: finnhub-provider-service
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
SERVER_PORT: 8002
|
||||||
|
NATS_ADDR: nats://nats:4222
|
||||||
|
DATA_PERSISTENCE_SERVICE_URL: http://data-persistence-service:3000/api/v1
|
||||||
|
FINNHUB_API_URL: https://finnhub.io/api/v1
|
||||||
|
RUST_LOG: info,axum=info
|
||||||
|
RUST_BACKTRACE: "1"
|
||||||
|
depends_on:
|
||||||
|
- nats
|
||||||
|
- data-persistence-service
|
||||||
|
networks:
|
||||||
|
- app-network
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "curl -fsS http://localhost:8002/health >/dev/null || exit 1"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 12
|
||||||
|
|
||||||
|
yfinance-provider-service:
|
||||||
|
image: $NAMESPACE/yfinance-provider-service:$VERSION
|
||||||
|
container_name: yfinance-provider-service
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
SERVER_PORT: 8003
|
||||||
|
NATS_ADDR: nats://nats:4222
|
||||||
|
DATA_PERSISTENCE_SERVICE_URL: http://data-persistence-service:3000/api/v1
|
||||||
|
RUST_LOG: info,axum=info
|
||||||
|
RUST_BACKTRACE: "1"
|
||||||
|
depends_on:
|
||||||
|
- nats
|
||||||
|
- data-persistence-service
|
||||||
|
networks:
|
||||||
|
- app-network
|
||||||
|
dns:
|
||||||
|
- 8.8.8.8
|
||||||
|
- 8.8.4.4
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "curl -fsS http://localhost:8003/health >/dev/null || exit 1"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 12
|
||||||
|
|
||||||
|
report-generator-service:
|
||||||
|
image: $NAMESPACE/report-generator-service:$VERSION
|
||||||
|
container_name: report-generator-service
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
SERVER_PORT: 8004
|
||||||
|
NATS_ADDR: nats://nats:4222
|
||||||
|
DATA_PERSISTENCE_SERVICE_URL: http://data-persistence-service:3000/api/v1
|
||||||
|
RUST_LOG: info,axum=info
|
||||||
|
RUST_BACKTRACE: "1"
|
||||||
|
depends_on:
|
||||||
|
- nats
|
||||||
|
- data-persistence-service
|
||||||
|
networks:
|
||||||
|
- app-network
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "curl -fsS http://localhost:8004/health >/dev/null || exit 1"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 12
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
pgdata:
|
||||||
|
nats_data:
|
||||||
|
|
||||||
|
networks:
|
||||||
|
app-network:
|
||||||
|
EOF
|
||||||
|
|
||||||
|
echo -e "${GREEN}生成完成: docker-compose.server.yml${NC}"
|
||||||
|
echo -e "请将此文件复制到远程服务器,并执行: docker-compose -f docker-compose.server.yml up -d"
|
||||||
|
|
||||||
@ -43,6 +43,7 @@ pub fn parse_company_profile(v: Value) -> anyhow::Result<CompanyProfileDto> {
|
|||||||
"pe_ratio": v.get("PERatio"),
|
"pe_ratio": v.get("PERatio"),
|
||||||
"beta": v.get("Beta")
|
"beta": v.get("Beta")
|
||||||
})),
|
})),
|
||||||
|
updated_at: None,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -24,6 +24,16 @@ impl PersistenceClient {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub async fn get_company_profile(&self, symbol: &str) -> Result<Option<CompanyProfileDto>> {
|
||||||
|
let url = format!("{}/companies/{}", self.base_url, symbol);
|
||||||
|
let resp = self.client.get(&url).send().await?;
|
||||||
|
if resp.status() == reqwest::StatusCode::NOT_FOUND {
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
let profile = resp.error_for_status()?.json().await?;
|
||||||
|
Ok(Some(profile))
|
||||||
|
}
|
||||||
|
|
||||||
pub async fn upsert_company_profile(&self, profile: CompanyProfileDto) -> Result<()> {
|
pub async fn upsert_company_profile(&self, profile: CompanyProfileDto) -> Result<()> {
|
||||||
let url = format!("{}/companies", self.base_url);
|
let url = format!("{}/companies", self.base_url);
|
||||||
info!("Upserting company profile for {} to {}", profile.symbol, url);
|
info!("Upserting company profile for {} to {}", profile.symbol, url);
|
||||||
|
|||||||
@ -47,6 +47,44 @@ pub async fn handle_fetch_command(
|
|||||||
PersistenceClient::new(state.config.data_persistence_service_url.clone());
|
PersistenceClient::new(state.config.data_persistence_service_url.clone());
|
||||||
let symbol = command.symbol.clone();
|
let symbol = command.symbol.clone();
|
||||||
|
|
||||||
|
// Check freshness
|
||||||
|
let mut is_fresh = false;
|
||||||
|
match persistence_client.get_company_profile(&command.symbol).await {
|
||||||
|
Ok(Some(p)) => {
|
||||||
|
if let Some(updated_at) = p.updated_at {
|
||||||
|
let age = chrono::Utc::now() - updated_at;
|
||||||
|
if age < chrono::Duration::hours(24) {
|
||||||
|
info!("Data for {} is fresh (age: {}h). Skipping fetch.", command.symbol, age.num_hours());
|
||||||
|
is_fresh = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(None) => {}
|
||||||
|
Err(e) => tracing::warn!("Failed to check profile freshness: {}", e),
|
||||||
|
}
|
||||||
|
|
||||||
|
if is_fresh {
|
||||||
|
let event = FinancialsPersistedEvent {
|
||||||
|
request_id: command.request_id,
|
||||||
|
symbol: command.symbol,
|
||||||
|
years_updated: vec![],
|
||||||
|
template_id: command.template_id,
|
||||||
|
};
|
||||||
|
let subject = "events.data.financials_persisted".to_string();
|
||||||
|
publisher
|
||||||
|
.publish(subject, serde_json::to_vec(&event).unwrap().into())
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
update_task_progress(
|
||||||
|
&state.tasks,
|
||||||
|
command.request_id,
|
||||||
|
100,
|
||||||
|
"Data retrieved from cache",
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
|
||||||
// Symbol conversion for Chinese stocks
|
// Symbol conversion for Chinese stocks
|
||||||
let av_symbol = if symbol.ends_with(".SH") {
|
let av_symbol = if symbol.ends_with(".SH") {
|
||||||
symbol.replace(".SH", ".SS")
|
symbol.replace(".SH", ".SS")
|
||||||
@ -201,8 +239,9 @@ pub async fn handle_fetch_command(
|
|||||||
request_id: command.request_id,
|
request_id: command.request_id,
|
||||||
symbol: command.symbol,
|
symbol: command.symbol,
|
||||||
years_updated,
|
years_updated,
|
||||||
|
template_id: command.template_id,
|
||||||
};
|
};
|
||||||
let subject = "financials.persisted".to_string(); // NATS subject
|
let subject = "events.data.financials_persisted".to_string(); // NATS subject
|
||||||
publisher
|
publisher
|
||||||
.publish(subject, serde_json::to_vec(&event).unwrap().into())
|
.publish(subject, serde_json::to_vec(&event).unwrap().into())
|
||||||
.await?;
|
.await?;
|
||||||
|
|||||||
@ -1,7 +1,7 @@
|
|||||||
use crate::error::Result;
|
use crate::error::Result;
|
||||||
use crate::state::AppState;
|
use crate::state::AppState;
|
||||||
use axum::{
|
use axum::{
|
||||||
extract::{Path, State},
|
extract::{Path, Query, State},
|
||||||
http::StatusCode,
|
http::StatusCode,
|
||||||
response::{IntoResponse, Json},
|
response::{IntoResponse, Json},
|
||||||
routing::{get, post},
|
routing::{get, post},
|
||||||
@ -36,6 +36,11 @@ pub struct AnalysisRequest {
|
|||||||
pub template_id: String,
|
pub template_id: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Deserialize)]
|
||||||
|
pub struct AnalysisResultQuery {
|
||||||
|
pub symbol: String,
|
||||||
|
}
|
||||||
|
|
||||||
// --- Router Definition ---
|
// --- Router Definition ---
|
||||||
pub fn create_router(app_state: AppState) -> Router {
|
pub fn create_router(app_state: AppState) -> Router {
|
||||||
Router::new()
|
Router::new()
|
||||||
@ -52,7 +57,9 @@ fn create_v1_router() -> Router<AppState> {
|
|||||||
"/analysis-requests/{symbol}",
|
"/analysis-requests/{symbol}",
|
||||||
post(trigger_analysis_generation),
|
post(trigger_analysis_generation),
|
||||||
)
|
)
|
||||||
|
.route("/analysis-results", get(get_analysis_results_by_symbol))
|
||||||
.route("/companies/{symbol}/profile", get(get_company_profile))
|
.route("/companies/{symbol}/profile", get(get_company_profile))
|
||||||
|
.route("/market-data/financial-statements/{symbol}", get(get_financials_by_symbol))
|
||||||
.route("/tasks/{request_id}", get(get_task_progress))
|
.route("/tasks/{request_id}", get(get_task_progress))
|
||||||
// --- New Config Routes ---
|
// --- New Config Routes ---
|
||||||
.route(
|
.route(
|
||||||
@ -68,6 +75,7 @@ fn create_v1_router() -> Router<AppState> {
|
|||||||
get(get_data_sources_config).put(update_data_sources_config),
|
get(get_data_sources_config).put(update_data_sources_config),
|
||||||
)
|
)
|
||||||
.route("/configs/test", post(test_data_source_config))
|
.route("/configs/test", post(test_data_source_config))
|
||||||
|
.route("/configs/llm/test", post(test_llm_config))
|
||||||
// --- New Discover Routes ---
|
// --- New Discover Routes ---
|
||||||
.route("/discover-models/{provider_id}", get(discover_models))
|
.route("/discover-models/{provider_id}", get(discover_models))
|
||||||
.route("/discover-models", post(discover_models_preview))
|
.route("/discover-models", post(discover_models_preview))
|
||||||
@ -105,6 +113,7 @@ async fn trigger_data_fetch(
|
|||||||
request_id,
|
request_id,
|
||||||
symbol: payload.symbol.clone(),
|
symbol: payload.symbol.clone(),
|
||||||
market: payload.market,
|
market: payload.market,
|
||||||
|
template_id: payload.template_id.clone(),
|
||||||
};
|
};
|
||||||
|
|
||||||
info!(request_id = %request_id, "Publishing data fetch command");
|
info!(request_id = %request_id, "Publishing data fetch command");
|
||||||
@ -117,23 +126,6 @@ async fn trigger_data_fetch(
|
|||||||
)
|
)
|
||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
// If a template_id is provided, trigger the analysis generation workflow as well.
|
|
||||||
if let Some(template_id) = payload.template_id {
|
|
||||||
let analysis_command = GenerateReportCommand {
|
|
||||||
request_id,
|
|
||||||
symbol: payload.symbol,
|
|
||||||
template_id,
|
|
||||||
};
|
|
||||||
info!(request_id = %request_id, template_id = %analysis_command.template_id, "Publishing analysis generation command (auto-triggered)");
|
|
||||||
state
|
|
||||||
.nats_client
|
|
||||||
.publish(
|
|
||||||
ANALYSIS_COMMANDS_QUEUE.to_string(),
|
|
||||||
serde_json::to_vec(&analysis_command).unwrap().into(),
|
|
||||||
)
|
|
||||||
.await?;
|
|
||||||
}
|
|
||||||
|
|
||||||
Ok((
|
Ok((
|
||||||
StatusCode::ACCEPTED,
|
StatusCode::ACCEPTED,
|
||||||
Json(RequestAcceptedResponse { request_id }),
|
Json(RequestAcceptedResponse { request_id }),
|
||||||
@ -170,6 +162,15 @@ async fn trigger_analysis_generation(
|
|||||||
))
|
))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// [GET /v1/analysis-results?symbol=...]
|
||||||
|
async fn get_analysis_results_by_symbol(
|
||||||
|
State(state): State<AppState>,
|
||||||
|
Query(query): Query<AnalysisResultQuery>,
|
||||||
|
) -> Result<impl IntoResponse> {
|
||||||
|
let results = state.persistence_client.get_analysis_results(&query.symbol).await?;
|
||||||
|
Ok(Json(results))
|
||||||
|
}
|
||||||
|
|
||||||
/// [GET /v1/companies/:symbol/profile]
|
/// [GET /v1/companies/:symbol/profile]
|
||||||
/// Queries the persisted company profile from the data-persistence-service.
|
/// Queries the persisted company profile from the data-persistence-service.
|
||||||
async fn get_company_profile(
|
async fn get_company_profile(
|
||||||
@ -180,6 +181,15 @@ async fn get_company_profile(
|
|||||||
Ok(Json(profile))
|
Ok(Json(profile))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// [GET /v1/market-data/financial-statements/:symbol]
|
||||||
|
async fn get_financials_by_symbol(
|
||||||
|
State(state): State<AppState>,
|
||||||
|
Path(symbol): Path<String>,
|
||||||
|
) -> Result<impl IntoResponse> {
|
||||||
|
let financials = state.persistence_client.get_financials(&symbol).await?;
|
||||||
|
Ok(Json(financials))
|
||||||
|
}
|
||||||
|
|
||||||
/// [GET /v1/tasks/:request_id]
|
/// [GET /v1/tasks/:request_id]
|
||||||
/// Aggregates task progress from all downstream provider services.
|
/// Aggregates task progress from all downstream provider services.
|
||||||
async fn get_task_progress(
|
async fn get_task_progress(
|
||||||
@ -291,6 +301,42 @@ async fn test_data_source_config(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Deserialize, Serialize)]
|
||||||
|
struct TestLlmConfigRequest {
|
||||||
|
api_base_url: String,
|
||||||
|
api_key: String,
|
||||||
|
model_id: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn test_llm_config(
|
||||||
|
State(state): State<AppState>,
|
||||||
|
Json(payload): Json<TestLlmConfigRequest>,
|
||||||
|
) -> Result<impl IntoResponse> {
|
||||||
|
let client = reqwest::Client::new();
|
||||||
|
let target_url = format!("{}/test-llm", state.config.report_generator_service_url.trim_end_matches('/'));
|
||||||
|
|
||||||
|
let response = client
|
||||||
|
.post(&target_url)
|
||||||
|
.json(&payload)
|
||||||
|
.send()
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
if !response.status().is_success() {
|
||||||
|
let status = response.status();
|
||||||
|
let error_text = response.text().await?;
|
||||||
|
return Ok((
|
||||||
|
StatusCode::from_u16(status.as_u16()).unwrap_or(StatusCode::BAD_GATEWAY),
|
||||||
|
Json(serde_json::json!({
|
||||||
|
"error": "LLM test failed",
|
||||||
|
"details": error_text,
|
||||||
|
})),
|
||||||
|
).into_response());
|
||||||
|
}
|
||||||
|
|
||||||
|
let response_json: serde_json::Value = response.json().await?;
|
||||||
|
Ok((StatusCode::OK, Json(response_json)).into_response())
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
// --- Config API Handlers (Proxy to data-persistence-service) ---
|
// --- Config API Handlers (Proxy to data-persistence-service) ---
|
||||||
|
|
||||||
|
|||||||
@ -6,6 +6,7 @@ pub struct AppConfig {
|
|||||||
pub server_port: u16,
|
pub server_port: u16,
|
||||||
pub nats_addr: String,
|
pub nats_addr: String,
|
||||||
pub data_persistence_service_url: String,
|
pub data_persistence_service_url: String,
|
||||||
|
pub report_generator_service_url: String,
|
||||||
pub provider_services: Vec<String>,
|
pub provider_services: Vec<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -18,6 +19,8 @@ impl AppConfig {
|
|||||||
let server_port: u16 = cfg.get::<u16>("server_port")?;
|
let server_port: u16 = cfg.get::<u16>("server_port")?;
|
||||||
let nats_addr: String = cfg.get::<String>("nats_addr")?;
|
let nats_addr: String = cfg.get::<String>("nats_addr")?;
|
||||||
let data_persistence_service_url: String = cfg.get::<String>("data_persistence_service_url")?;
|
let data_persistence_service_url: String = cfg.get::<String>("data_persistence_service_url")?;
|
||||||
|
let report_generator_service_url: String = cfg.get::<String>("report_generator_service_url")
|
||||||
|
.unwrap_or_else(|_| "http://report-generator-service:8004".to_string());
|
||||||
|
|
||||||
// Parse provider_services deterministically:
|
// Parse provider_services deterministically:
|
||||||
// 1) prefer array from env (e.g., PROVIDER_SERVICES__0, PROVIDER_SERVICES__1, ...)
|
// 1) prefer array from env (e.g., PROVIDER_SERVICES__0, PROVIDER_SERVICES__1, ...)
|
||||||
@ -45,6 +48,7 @@ impl AppConfig {
|
|||||||
server_port,
|
server_port,
|
||||||
nats_addr,
|
nats_addr,
|
||||||
data_persistence_service_url,
|
data_persistence_service_url,
|
||||||
|
report_generator_service_url,
|
||||||
provider_services,
|
provider_services,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|||||||
@ -3,7 +3,7 @@
|
|||||||
//!
|
//!
|
||||||
|
|
||||||
use crate::error::Result;
|
use crate::error::Result;
|
||||||
use common_contracts::dtos::CompanyProfileDto;
|
use common_contracts::dtos::{CompanyProfileDto, TimeSeriesFinancialDto};
|
||||||
use common_contracts::config_models::{LlmProvidersConfig, DataSourcesConfig, AnalysisTemplateSets};
|
use common_contracts::config_models::{LlmProvidersConfig, DataSourcesConfig, AnalysisTemplateSets};
|
||||||
|
|
||||||
#[derive(Clone)]
|
#[derive(Clone)]
|
||||||
@ -33,6 +33,32 @@ impl PersistenceClient {
|
|||||||
Ok(profile)
|
Ok(profile)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub async fn get_financials(&self, symbol: &str) -> Result<Vec<TimeSeriesFinancialDto>> {
|
||||||
|
let url = format!("{}/market-data/financial-statements/{}", self.base_url, symbol);
|
||||||
|
let financials = self
|
||||||
|
.client
|
||||||
|
.get(&url)
|
||||||
|
.send()
|
||||||
|
.await?
|
||||||
|
.error_for_status()?
|
||||||
|
.json::<Vec<TimeSeriesFinancialDto>>()
|
||||||
|
.await?;
|
||||||
|
Ok(financials)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub async fn get_analysis_results(&self, symbol: &str) -> Result<Vec<common_contracts::dtos::AnalysisResultDto>> {
|
||||||
|
let url = format!("{}/analysis-results?symbol={}", self.base_url, symbol);
|
||||||
|
let results = self
|
||||||
|
.client
|
||||||
|
.get(&url)
|
||||||
|
.send()
|
||||||
|
.await?
|
||||||
|
.error_for_status()?
|
||||||
|
.json::<Vec<common_contracts::dtos::AnalysisResultDto>>()
|
||||||
|
.await?;
|
||||||
|
Ok(results)
|
||||||
|
}
|
||||||
|
|
||||||
// --- Config Methods ---
|
// --- Config Methods ---
|
||||||
|
|
||||||
pub async fn get_llm_providers_config(&self) -> Result<LlmProvidersConfig> {
|
pub async fn get_llm_providers_config(&self) -> Result<LlmProvidersConfig> {
|
||||||
|
|||||||
@ -11,6 +11,7 @@ pub struct CompanyProfileDto {
|
|||||||
pub industry: Option<String>,
|
pub industry: Option<String>,
|
||||||
pub list_date: Option<NaiveDate>,
|
pub list_date: Option<NaiveDate>,
|
||||||
pub additional_info: Option<JsonValue>,
|
pub additional_info: Option<JsonValue>,
|
||||||
|
pub updated_at: Option<chrono::DateTime<chrono::Utc>>,
|
||||||
}
|
}
|
||||||
|
|
||||||
// Market Data API DTOs
|
// Market Data API DTOs
|
||||||
@ -62,7 +63,7 @@ pub struct NewAnalysisResult {
|
|||||||
/// Represents a persisted analysis result read from the database.
|
/// Represents a persisted analysis result read from the database.
|
||||||
#[api_dto]
|
#[api_dto]
|
||||||
pub struct AnalysisResultDto {
|
pub struct AnalysisResultDto {
|
||||||
pub id: i64,
|
pub id: Uuid,
|
||||||
pub request_id: Uuid,
|
pub request_id: Uuid,
|
||||||
pub symbol: String,
|
pub symbol: String,
|
||||||
pub template_id: String,
|
pub template_id: String,
|
||||||
|
|||||||
@ -10,6 +10,7 @@ pub struct FetchCompanyDataCommand {
|
|||||||
pub request_id: Uuid,
|
pub request_id: Uuid,
|
||||||
pub symbol: String,
|
pub symbol: String,
|
||||||
pub market: String,
|
pub market: String,
|
||||||
|
pub template_id: Option<String>, // Optional trigger for analysis
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Command to start a full report generation workflow.
|
/// Command to start a full report generation workflow.
|
||||||
@ -35,6 +36,7 @@ pub struct FinancialsPersistedEvent {
|
|||||||
pub request_id: Uuid,
|
pub request_id: Uuid,
|
||||||
pub symbol: String,
|
pub symbol: String,
|
||||||
pub years_updated: Vec<u16>,
|
pub years_updated: Vec<u16>,
|
||||||
|
pub template_id: Option<String>, // Pass-through for analysis trigger
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@ -8,10 +8,11 @@ use axum::{
|
|||||||
};
|
};
|
||||||
use common_contracts::dtos::{AnalysisResultDto, NewAnalysisResult};
|
use common_contracts::dtos::{AnalysisResultDto, NewAnalysisResult};
|
||||||
use serde::Deserialize;
|
use serde::Deserialize;
|
||||||
use tracing::instrument;
|
use service_kit::api;
|
||||||
|
use tracing::{instrument, error};
|
||||||
use anyhow::Error as AnyhowError;
|
use anyhow::Error as AnyhowError;
|
||||||
|
|
||||||
#[derive(Debug, Deserialize)]
|
#[derive(Debug, Deserialize, utoipa::IntoParams, utoipa::ToSchema)]
|
||||||
pub struct AnalysisQuery {
|
pub struct AnalysisQuery {
|
||||||
pub symbol: String,
|
pub symbol: String,
|
||||||
pub module_id: Option<String>,
|
pub module_id: Option<String>,
|
||||||
@ -19,15 +20,18 @@ pub struct AnalysisQuery {
|
|||||||
|
|
||||||
/// Creates a new analysis result and returns the created record.
|
/// Creates a new analysis result and returns the created record.
|
||||||
#[instrument(skip(state, payload), fields(request_id = %payload.request_id, symbol = %payload.symbol, module_id = %payload.module_id))]
|
#[instrument(skip(state, payload), fields(request_id = %payload.request_id, symbol = %payload.symbol, module_id = %payload.module_id))]
|
||||||
|
#[api(POST, "/api/v1/analysis-results")]
|
||||||
pub async fn create_analysis_result(
|
pub async fn create_analysis_result(
|
||||||
State(state): State<AppState>,
|
State(state): State<AppState>,
|
||||||
Json(payload): Json<NewAnalysisResult>,
|
Json(payload): Json<NewAnalysisResult>,
|
||||||
) -> Result<impl IntoResponse, ServerError> {
|
) -> Result<impl IntoResponse, ServerError> {
|
||||||
|
// Use explicit column names to avoid issues if DB schema and struct are slightly out of sync
|
||||||
|
// Also ensure we are returning all fields needed by AnalysisResult
|
||||||
let result = sqlx::query_as::<_, AnalysisResult>(
|
let result = sqlx::query_as::<_, AnalysisResult>(
|
||||||
r#"
|
r#"
|
||||||
INSERT INTO analysis_results (request_id, symbol, template_id, module_id, content, meta_data)
|
INSERT INTO analysis_results (request_id, symbol, template_id, module_id, content, meta_data)
|
||||||
VALUES ($1, $2, $3, $4, $5, $6)
|
VALUES ($1, $2, $3, $4, $5, $6)
|
||||||
RETURNING *
|
RETURNING id, request_id, symbol, template_id, module_id, content, meta_data, created_at
|
||||||
"#
|
"#
|
||||||
)
|
)
|
||||||
.bind(&payload.request_id)
|
.bind(&payload.request_id)
|
||||||
@ -38,7 +42,10 @@ pub async fn create_analysis_result(
|
|||||||
.bind(&payload.meta_data)
|
.bind(&payload.meta_data)
|
||||||
.fetch_one(state.pool())
|
.fetch_one(state.pool())
|
||||||
.await
|
.await
|
||||||
.map_err(AnyhowError::from)?;
|
.map_err(|e| {
|
||||||
|
error!("Database error inserting analysis result: {}", e);
|
||||||
|
AnyhowError::from(e)
|
||||||
|
})?;
|
||||||
|
|
||||||
let dto = AnalysisResultDto {
|
let dto = AnalysisResultDto {
|
||||||
id: result.id,
|
id: result.id,
|
||||||
@ -56,13 +63,33 @@ pub async fn create_analysis_result(
|
|||||||
|
|
||||||
/// Retrieves all analysis results for a given symbol.
|
/// Retrieves all analysis results for a given symbol.
|
||||||
#[instrument(skip(state))]
|
#[instrument(skip(state))]
|
||||||
|
#[api(GET, "/api/v1/analysis-results", output(list = "AnalysisResultDto"))]
|
||||||
pub async fn get_analysis_results(
|
pub async fn get_analysis_results(
|
||||||
State(state): State<AppState>,
|
State(state): State<AppState>,
|
||||||
Query(query): Query<AnalysisQuery>,
|
Query(query): Query<AnalysisQuery>,
|
||||||
) -> Result<Json<Vec<AnalysisResultDto>>, ServerError> {
|
) -> Result<Json<Vec<AnalysisResultDto>>, ServerError> {
|
||||||
let results = sqlx::query_as::<_, AnalysisResult>(
|
// Use string replacement for module_id to avoid lifetime issues with query_builder
|
||||||
|
// This is safe because we're not interpolating user input directly into the SQL structure, just deciding whether to add a clause.
|
||||||
|
// However, binding parameters is better. The issue with previous code was lifetime of temporary values.
|
||||||
|
|
||||||
|
let results = if let Some(mid) = &query.module_id {
|
||||||
|
sqlx::query_as::<_, AnalysisResult>(
|
||||||
r#"
|
r#"
|
||||||
SELECT * FROM analysis_results
|
SELECT id, request_id, symbol, template_id, module_id, content, meta_data, created_at
|
||||||
|
FROM analysis_results
|
||||||
|
WHERE symbol = $1 AND module_id = $2
|
||||||
|
ORDER BY created_at DESC
|
||||||
|
"#
|
||||||
|
)
|
||||||
|
.bind(&query.symbol)
|
||||||
|
.bind(mid)
|
||||||
|
.fetch_all(state.pool())
|
||||||
|
.await
|
||||||
|
} else {
|
||||||
|
sqlx::query_as::<_, AnalysisResult>(
|
||||||
|
r#"
|
||||||
|
SELECT id, request_id, symbol, template_id, module_id, content, meta_data, created_at
|
||||||
|
FROM analysis_results
|
||||||
WHERE symbol = $1
|
WHERE symbol = $1
|
||||||
ORDER BY created_at DESC
|
ORDER BY created_at DESC
|
||||||
"#
|
"#
|
||||||
@ -70,7 +97,12 @@ pub async fn get_analysis_results(
|
|||||||
.bind(&query.symbol)
|
.bind(&query.symbol)
|
||||||
.fetch_all(state.pool())
|
.fetch_all(state.pool())
|
||||||
.await
|
.await
|
||||||
.map_err(AnyhowError::from)?;
|
};
|
||||||
|
|
||||||
|
let results = results.map_err(|e| {
|
||||||
|
error!("Database error fetching analysis results: {}", e);
|
||||||
|
AnyhowError::from(e)
|
||||||
|
})?;
|
||||||
|
|
||||||
let dtos = results
|
let dtos = results
|
||||||
.into_iter()
|
.into_iter()
|
||||||
@ -89,22 +121,31 @@ pub async fn get_analysis_results(
|
|||||||
Ok(Json(dtos))
|
Ok(Json(dtos))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
use uuid::Uuid;
|
||||||
|
|
||||||
/// Retrieves a single analysis result by its primary ID.
|
/// Retrieves a single analysis result by its primary ID.
|
||||||
#[instrument(skip(state))]
|
#[instrument(skip(state))]
|
||||||
|
#[api(GET, "/api/v1/analysis-results/{id}", output(detail = "AnalysisResultDto"))]
|
||||||
pub async fn get_analysis_result_by_id(
|
pub async fn get_analysis_result_by_id(
|
||||||
State(state): State<AppState>,
|
State(state): State<AppState>,
|
||||||
Path(id): Path<i64>,
|
Path(id_str): Path<String>,
|
||||||
) -> Result<Json<AnalysisResultDto>, ServerError> {
|
) -> Result<Json<AnalysisResultDto>, ServerError> {
|
||||||
|
let id = Uuid::parse_str(&id_str).map_err(|_| ServerError::NotFound(format!("Invalid UUID: {}", id_str)))?;
|
||||||
|
|
||||||
let result = sqlx::query_as::<_, AnalysisResult>(
|
let result = sqlx::query_as::<_, AnalysisResult>(
|
||||||
r#"
|
r#"
|
||||||
SELECT * FROM analysis_results
|
SELECT id, request_id, symbol, template_id, module_id, content, meta_data, created_at
|
||||||
|
FROM analysis_results
|
||||||
WHERE id = $1
|
WHERE id = $1
|
||||||
"#
|
"#
|
||||||
)
|
)
|
||||||
.bind(&id)
|
.bind(&id)
|
||||||
.fetch_one(state.pool())
|
.fetch_one(state.pool())
|
||||||
.await
|
.await
|
||||||
.map_err(AnyhowError::from)?;
|
.map_err(|e| {
|
||||||
|
error!("Database error fetching analysis result by id: {}", e);
|
||||||
|
AnyhowError::from(e)
|
||||||
|
})?;
|
||||||
|
|
||||||
let dto = AnalysisResultDto {
|
let dto = AnalysisResultDto {
|
||||||
id: result.id,
|
id: result.id,
|
||||||
|
|||||||
@ -40,6 +40,7 @@ pub async fn get_company_by_symbol(
|
|||||||
industry: company.industry,
|
industry: company.industry,
|
||||||
list_date: company.list_date,
|
list_date: company.list_date,
|
||||||
additional_info: company.additional_info,
|
additional_info: company.additional_info,
|
||||||
|
updated_at: Some(company.updated_at),
|
||||||
};
|
};
|
||||||
|
|
||||||
info!(target: "api", symbol = %dto.symbol, "get_company_by_symbol completed");
|
info!(target: "api", symbol = %dto.symbol, "get_company_by_symbol completed");
|
||||||
|
|||||||
@ -1,7 +1,6 @@
|
|||||||
use axum::{extract::State, Json};
|
use axum::{extract::State, Json};
|
||||||
use common_contracts::config_models::{AnalysisTemplateSets, DataSourcesConfig, LlmProvidersConfig};
|
use common_contracts::config_models::{AnalysisTemplateSets, DataSourcesConfig, LlmProvidersConfig};
|
||||||
use service_kit::api;
|
use service_kit::api;
|
||||||
use tracing::instrument;
|
|
||||||
use crate::{db::system_config, AppState, ServerError};
|
use crate::{db::system_config, AppState, ServerError};
|
||||||
|
|
||||||
#[api(GET, "/api/v1/configs/llm_providers", output(detail = "LlmProvidersConfig"))]
|
#[api(GET, "/api/v1/configs/llm_providers", output(detail = "LlmProvidersConfig"))]
|
||||||
|
|||||||
@ -29,7 +29,7 @@ pub async fn batch_insert_financials(
|
|||||||
Ok(axum::http::StatusCode::CREATED)
|
Ok(axum::http::StatusCode::CREATED)
|
||||||
}
|
}
|
||||||
|
|
||||||
#[api(GET, "/api/v1/market-data/financials/{symbol}", output(list = "TimeSeriesFinancialDto"))]
|
#[api(GET, "/api/v1/market-data/financial-statements/{symbol}", output(list = "TimeSeriesFinancialDto"))]
|
||||||
pub async fn get_financials_by_symbol(
|
pub async fn get_financials_by_symbol(
|
||||||
State(state): State<AppState>,
|
State(state): State<AppState>,
|
||||||
Path(symbol): Path<String>,
|
Path(symbol): Path<String>,
|
||||||
|
|||||||
@ -14,9 +14,11 @@ pub struct SystemConfig {
|
|||||||
|
|
||||||
#[derive(Debug, Clone, Serialize, sqlx::FromRow)]
|
#[derive(Debug, Clone, Serialize, sqlx::FromRow)]
|
||||||
pub struct AnalysisResult {
|
pub struct AnalysisResult {
|
||||||
pub id: i64,
|
pub id: Uuid,
|
||||||
|
#[sqlx(default)] // request_id is missing in some schema versions, handle gracefully or ensure migration runs
|
||||||
pub request_id: Uuid,
|
pub request_id: Uuid,
|
||||||
pub symbol: String,
|
pub symbol: String,
|
||||||
|
// template_id/module_id might be missing if schema is very old, but we rely on migrations
|
||||||
pub template_id: String,
|
pub template_id: String,
|
||||||
pub module_id: String,
|
pub module_id: String,
|
||||||
pub content: String,
|
pub content: String,
|
||||||
|
|||||||
@ -1,6 +1,5 @@
|
|||||||
//! One-time data seeding logic for initializing the database.
|
//! One-time data seeding logic for initializing the database.
|
||||||
|
|
||||||
use data_persistence_service::models::SystemConfig;
|
|
||||||
use common_contracts::config_models::{AnalysisModuleConfig, AnalysisTemplateSet, AnalysisTemplateSets};
|
use common_contracts::config_models::{AnalysisModuleConfig, AnalysisTemplateSet, AnalysisTemplateSets};
|
||||||
use sqlx::PgPool;
|
use sqlx::PgPool;
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
|
|||||||
@ -33,6 +33,7 @@ async fn test_api_upsert_and_get_company(pool: PgPool) {
|
|||||||
industry: Some("API Testing".to_string()),
|
industry: Some("API Testing".to_string()),
|
||||||
list_date: Some(chrono::NaiveDate::from_ymd_opt(2025, 1, 1).unwrap()),
|
list_date: Some(chrono::NaiveDate::from_ymd_opt(2025, 1, 1).unwrap()),
|
||||||
additional_info: None,
|
additional_info: None,
|
||||||
|
updated_at: None,
|
||||||
};
|
};
|
||||||
|
|
||||||
let request = Request::builder()
|
let request = Request::builder()
|
||||||
|
|||||||
@ -43,6 +43,7 @@ async fn test_upsert_and_get_company(pool: PgPool) {
|
|||||||
industry: Some("Testing".to_string()),
|
industry: Some("Testing".to_string()),
|
||||||
list_date: Some(chrono::NaiveDate::from_ymd_opt(2024, 1, 1).unwrap()),
|
list_date: Some(chrono::NaiveDate::from_ymd_opt(2024, 1, 1).unwrap()),
|
||||||
additional_info: Some(serde_json::json!({ "ceo": "John Doe" })),
|
additional_info: Some(serde_json::json!({ "ceo": "John Doe" })),
|
||||||
|
updated_at: None,
|
||||||
};
|
};
|
||||||
|
|
||||||
// 2. Act: Call the upsert function
|
// 2. Act: Call the upsert function
|
||||||
|
|||||||
@ -22,6 +22,7 @@ pub fn map_profile_dto(profile_raw: &FinnhubProfile, symbol: &str) -> Result<Com
|
|||||||
industry,
|
industry,
|
||||||
list_date,
|
list_date,
|
||||||
additional_info: None,
|
additional_info: None,
|
||||||
|
updated_at: None,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -73,6 +73,7 @@ pub async fn handle_fetch_command(
|
|||||||
request_id: command.request_id,
|
request_id: command.request_id,
|
||||||
symbol: command.symbol.clone(),
|
symbol: command.symbol.clone(),
|
||||||
years_updated: years_set.into_iter().collect(),
|
years_updated: years_set.into_iter().collect(),
|
||||||
|
template_id: command.template_id.clone(),
|
||||||
};
|
};
|
||||||
publisher
|
publisher
|
||||||
.publish(
|
.publish(
|
||||||
|
|||||||
274
services/report-generator-service/Cargo.lock
generated
274
services/report-generator-service/Cargo.lock
generated
@ -73,7 +73,7 @@ dependencies = [
|
|||||||
"rand 0.8.5",
|
"rand 0.8.5",
|
||||||
"regex",
|
"regex",
|
||||||
"ring",
|
"ring",
|
||||||
"rustls-native-certs",
|
"rustls-native-certs 0.7.3",
|
||||||
"rustls-pemfile",
|
"rustls-pemfile",
|
||||||
"rustls-webpki 0.102.8",
|
"rustls-webpki 0.102.8",
|
||||||
"serde",
|
"serde",
|
||||||
@ -92,6 +92,43 @@ dependencies = [
|
|||||||
"url",
|
"url",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "async-openai"
|
||||||
|
version = "0.30.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "6bf39a15c8d613eb61892dc9a287c02277639ebead41ee611ad23aaa613f1a82"
|
||||||
|
dependencies = [
|
||||||
|
"async-openai-macros",
|
||||||
|
"backoff",
|
||||||
|
"base64",
|
||||||
|
"bytes",
|
||||||
|
"derive_builder",
|
||||||
|
"eventsource-stream",
|
||||||
|
"futures",
|
||||||
|
"rand 0.9.2",
|
||||||
|
"reqwest",
|
||||||
|
"reqwest-eventsource",
|
||||||
|
"secrecy",
|
||||||
|
"serde",
|
||||||
|
"serde_json",
|
||||||
|
"thiserror 2.0.17",
|
||||||
|
"tokio",
|
||||||
|
"tokio-stream",
|
||||||
|
"tokio-util",
|
||||||
|
"tracing",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "async-openai-macros"
|
||||||
|
version = "0.1.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "0289cba6d5143bfe8251d57b4a8cac036adf158525a76533a7082ba65ec76398"
|
||||||
|
dependencies = [
|
||||||
|
"proc-macro2",
|
||||||
|
"quote",
|
||||||
|
"syn 2.0.110",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "async-trait"
|
name = "async-trait"
|
||||||
version = "0.1.89"
|
version = "0.1.89"
|
||||||
@ -176,6 +213,20 @@ dependencies = [
|
|||||||
"tracing",
|
"tracing",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "backoff"
|
||||||
|
version = "0.4.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "b62ddb9cb1ec0a098ad4bbf9344d0713fa193ae1a80af55febcff2627b6a00c1"
|
||||||
|
dependencies = [
|
||||||
|
"futures-core",
|
||||||
|
"getrandom 0.2.16",
|
||||||
|
"instant",
|
||||||
|
"pin-project-lite",
|
||||||
|
"rand 0.8.5",
|
||||||
|
"tokio",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "base64"
|
name = "base64"
|
||||||
version = "0.22.1"
|
version = "0.22.1"
|
||||||
@ -440,6 +491,16 @@ dependencies = [
|
|||||||
"libc",
|
"libc",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "core-foundation"
|
||||||
|
version = "0.10.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "b2a6cd9ae233e7f62ba4e9353e81a88df7fc8a5987b8d445b4d90c879bd156f6"
|
||||||
|
dependencies = [
|
||||||
|
"core-foundation-sys",
|
||||||
|
"libc",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "core-foundation-sys"
|
name = "core-foundation-sys"
|
||||||
version = "0.8.7"
|
version = "0.8.7"
|
||||||
@ -546,6 +607,41 @@ dependencies = [
|
|||||||
"syn 2.0.110",
|
"syn 2.0.110",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "darling"
|
||||||
|
version = "0.20.11"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "fc7f46116c46ff9ab3eb1597a45688b6715c6e628b5c133e288e709a29bcb4ee"
|
||||||
|
dependencies = [
|
||||||
|
"darling_core",
|
||||||
|
"darling_macro",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "darling_core"
|
||||||
|
version = "0.20.11"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "0d00b9596d185e565c2207a0b01f8bd1a135483d02d9b7b0a54b11da8d53412e"
|
||||||
|
dependencies = [
|
||||||
|
"fnv",
|
||||||
|
"ident_case",
|
||||||
|
"proc-macro2",
|
||||||
|
"quote",
|
||||||
|
"strsim",
|
||||||
|
"syn 2.0.110",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "darling_macro"
|
||||||
|
version = "0.20.11"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "fc34b93ccb385b40dc71c6fceac4b2ad23662c7eeb248cf10d529b7e055b6ead"
|
||||||
|
dependencies = [
|
||||||
|
"darling_core",
|
||||||
|
"quote",
|
||||||
|
"syn 2.0.110",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dashmap"
|
name = "dashmap"
|
||||||
version = "6.1.0"
|
version = "6.1.0"
|
||||||
@ -587,6 +683,37 @@ dependencies = [
|
|||||||
"serde_core",
|
"serde_core",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "derive_builder"
|
||||||
|
version = "0.20.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "507dfb09ea8b7fa618fcf76e953f4f5e192547945816d5358edffe39f6f94947"
|
||||||
|
dependencies = [
|
||||||
|
"derive_builder_macro",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "derive_builder_core"
|
||||||
|
version = "0.20.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "2d5bcf7b024d6835cfb3d473887cd966994907effbe9227e8c8219824d06c4e8"
|
||||||
|
dependencies = [
|
||||||
|
"darling",
|
||||||
|
"proc-macro2",
|
||||||
|
"quote",
|
||||||
|
"syn 2.0.110",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "derive_builder_macro"
|
||||||
|
version = "0.20.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "ab63b0e2bf4d5928aff72e83a7dace85d7bba5fe12dcc3c5a572d78caffd3f3c"
|
||||||
|
dependencies = [
|
||||||
|
"derive_builder_core",
|
||||||
|
"syn 2.0.110",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "deunicode"
|
name = "deunicode"
|
||||||
version = "1.6.2"
|
version = "1.6.2"
|
||||||
@ -716,6 +843,17 @@ dependencies = [
|
|||||||
"pin-project-lite",
|
"pin-project-lite",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "eventsource-stream"
|
||||||
|
version = "0.2.3"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "74fef4569247a5f429d9156b9d0a2599914385dd189c539334c625d8099d90ab"
|
||||||
|
dependencies = [
|
||||||
|
"futures-core",
|
||||||
|
"nom",
|
||||||
|
"pin-project-lite",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "fiat-crypto"
|
name = "fiat-crypto"
|
||||||
version = "0.2.9"
|
version = "0.2.9"
|
||||||
@ -730,9 +868,9 @@ checksum = "3a3076410a55c90011c298b04d0cfa770b00fa04e1e3c97d3f6c9de105a03844"
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "fixedbitset"
|
name = "fixedbitset"
|
||||||
version = "0.4.2"
|
version = "0.5.7"
|
||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "0ce7134b9999ecaf8bcd65542e436736ef32ddca1b3e06094cb6ec5755203b80"
|
checksum = "1d674e81391d1e1ab681a28d99df07927c6d4aa5b027d7da16ba32d1d21ecd99"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "flume"
|
name = "flume"
|
||||||
@ -854,6 +992,12 @@ version = "0.3.31"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "f90f7dce0722e95104fcb095585910c0977252f286e354b5e3bd38902cd99988"
|
checksum = "f90f7dce0722e95104fcb095585910c0977252f286e354b5e3bd38902cd99988"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "futures-timer"
|
||||||
|
version = "3.0.3"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "f288b0a4f20f9a56b5d1da57e2227c661b7b16168e2f72365f57b63326e29b24"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "futures-util"
|
name = "futures-util"
|
||||||
version = "0.3.31"
|
version = "0.3.31"
|
||||||
@ -1100,6 +1244,7 @@ dependencies = [
|
|||||||
"hyper",
|
"hyper",
|
||||||
"hyper-util",
|
"hyper-util",
|
||||||
"rustls",
|
"rustls",
|
||||||
|
"rustls-native-certs 0.8.2",
|
||||||
"rustls-pki-types",
|
"rustls-pki-types",
|
||||||
"tokio",
|
"tokio",
|
||||||
"tokio-rustls",
|
"tokio-rustls",
|
||||||
@ -1236,6 +1381,12 @@ dependencies = [
|
|||||||
"zerovec",
|
"zerovec",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "ident_case"
|
||||||
|
version = "1.0.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "idna"
|
name = "idna"
|
||||||
version = "1.1.0"
|
version = "1.1.0"
|
||||||
@ -1285,6 +1436,15 @@ dependencies = [
|
|||||||
"serde_core",
|
"serde_core",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "instant"
|
||||||
|
version = "0.1.13"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "e0242819d153cba4b4b05a5a8f2a7e9bbf97b6055b2a002b395c96b5ff3c0222"
|
||||||
|
dependencies = [
|
||||||
|
"cfg-if",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "inventory"
|
name = "inventory"
|
||||||
version = "0.3.21"
|
version = "0.3.21"
|
||||||
@ -1443,6 +1603,22 @@ version = "0.3.17"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a"
|
checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "mime_guess"
|
||||||
|
version = "2.0.5"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "f7c44f8e672c00fe5308fa235f821cb4198414e1c77935c1ab6948d3fd78550e"
|
||||||
|
dependencies = [
|
||||||
|
"mime",
|
||||||
|
"unicase",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "minimal-lexical"
|
||||||
|
version = "0.2.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "mio"
|
name = "mio"
|
||||||
version = "1.1.0"
|
version = "1.1.0"
|
||||||
@ -1469,6 +1645,16 @@ dependencies = [
|
|||||||
"signatory",
|
"signatory",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "nom"
|
||||||
|
version = "7.1.3"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
|
||||||
|
dependencies = [
|
||||||
|
"memchr",
|
||||||
|
"minimal-lexical",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "nu-ansi-term"
|
name = "nu-ansi-term"
|
||||||
version = "0.50.3"
|
version = "0.50.3"
|
||||||
@ -1665,12 +1851,14 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "petgraph"
|
name = "petgraph"
|
||||||
version = "0.6.5"
|
version = "0.8.3"
|
||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "b4c5cc86750666a3ed20bdaf5ca2a0344f9c67674cae0515bec2da16fbaa47db"
|
checksum = "8701b58ea97060d5e5b155d383a69952a60943f0e6dfe30b04c287beb0b27455"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"fixedbitset",
|
"fixedbitset",
|
||||||
|
"hashbrown 0.15.5",
|
||||||
"indexmap",
|
"indexmap",
|
||||||
|
"serde",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
@ -2046,6 +2234,7 @@ version = "0.1.0"
|
|||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"async-nats",
|
"async-nats",
|
||||||
|
"async-openai",
|
||||||
"async-trait",
|
"async-trait",
|
||||||
"axum",
|
"axum",
|
||||||
"chrono",
|
"chrono",
|
||||||
@ -2076,6 +2265,7 @@ dependencies = [
|
|||||||
"base64",
|
"base64",
|
||||||
"bytes",
|
"bytes",
|
||||||
"futures-core",
|
"futures-core",
|
||||||
|
"futures-util",
|
||||||
"http",
|
"http",
|
||||||
"http-body",
|
"http-body",
|
||||||
"http-body-util",
|
"http-body-util",
|
||||||
@ -2084,10 +2274,12 @@ dependencies = [
|
|||||||
"hyper-util",
|
"hyper-util",
|
||||||
"js-sys",
|
"js-sys",
|
||||||
"log",
|
"log",
|
||||||
|
"mime_guess",
|
||||||
"percent-encoding",
|
"percent-encoding",
|
||||||
"pin-project-lite",
|
"pin-project-lite",
|
||||||
"quinn",
|
"quinn",
|
||||||
"rustls",
|
"rustls",
|
||||||
|
"rustls-native-certs 0.8.2",
|
||||||
"rustls-pki-types",
|
"rustls-pki-types",
|
||||||
"serde",
|
"serde",
|
||||||
"serde_json",
|
"serde_json",
|
||||||
@ -2095,16 +2287,34 @@ dependencies = [
|
|||||||
"sync_wrapper",
|
"sync_wrapper",
|
||||||
"tokio",
|
"tokio",
|
||||||
"tokio-rustls",
|
"tokio-rustls",
|
||||||
|
"tokio-util",
|
||||||
"tower",
|
"tower",
|
||||||
"tower-http",
|
"tower-http",
|
||||||
"tower-service",
|
"tower-service",
|
||||||
"url",
|
"url",
|
||||||
"wasm-bindgen",
|
"wasm-bindgen",
|
||||||
"wasm-bindgen-futures",
|
"wasm-bindgen-futures",
|
||||||
|
"wasm-streams",
|
||||||
"web-sys",
|
"web-sys",
|
||||||
"webpki-roots 1.0.4",
|
"webpki-roots 1.0.4",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "reqwest-eventsource"
|
||||||
|
version = "0.6.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "632c55746dbb44275691640e7b40c907c16a2dc1a5842aa98aaec90da6ec6bde"
|
||||||
|
dependencies = [
|
||||||
|
"eventsource-stream",
|
||||||
|
"futures-core",
|
||||||
|
"futures-timer",
|
||||||
|
"mime",
|
||||||
|
"nom",
|
||||||
|
"pin-project-lite",
|
||||||
|
"reqwest",
|
||||||
|
"thiserror 1.0.69",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "ring"
|
name = "ring"
|
||||||
version = "0.17.14"
|
version = "0.17.14"
|
||||||
@ -2247,7 +2457,19 @@ dependencies = [
|
|||||||
"rustls-pemfile",
|
"rustls-pemfile",
|
||||||
"rustls-pki-types",
|
"rustls-pki-types",
|
||||||
"schannel",
|
"schannel",
|
||||||
"security-framework",
|
"security-framework 2.11.1",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "rustls-native-certs"
|
||||||
|
version = "0.8.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "9980d917ebb0c0536119ba501e90834767bffc3d60641457fd84a1f3fd337923"
|
||||||
|
dependencies = [
|
||||||
|
"openssl-probe",
|
||||||
|
"rustls-pki-types",
|
||||||
|
"schannel",
|
||||||
|
"security-framework 3.5.1",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
@ -2375,7 +2597,20 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
|||||||
checksum = "897b2245f0b511c87893af39b033e5ca9cce68824c4d7e7630b5a1d339658d02"
|
checksum = "897b2245f0b511c87893af39b033e5ca9cce68824c4d7e7630b5a1d339658d02"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"bitflags",
|
"bitflags",
|
||||||
"core-foundation",
|
"core-foundation 0.9.4",
|
||||||
|
"core-foundation-sys",
|
||||||
|
"libc",
|
||||||
|
"security-framework-sys",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "security-framework"
|
||||||
|
version = "3.5.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "b3297343eaf830f66ede390ea39da1d462b6b0c1b000f420d0a83f898bbbe6ef"
|
||||||
|
dependencies = [
|
||||||
|
"bitflags",
|
||||||
|
"core-foundation 0.10.1",
|
||||||
"core-foundation-sys",
|
"core-foundation-sys",
|
||||||
"libc",
|
"libc",
|
||||||
"security-framework-sys",
|
"security-framework-sys",
|
||||||
@ -2903,6 +3138,12 @@ dependencies = [
|
|||||||
"unicode-properties",
|
"unicode-properties",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "strsim"
|
||||||
|
version = "0.11.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "subtle"
|
name = "subtle"
|
||||||
version = "2.6.1"
|
version = "2.6.1"
|
||||||
@ -3369,6 +3610,12 @@ version = "0.1.7"
|
|||||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
checksum = "2896d95c02a80c6d6a5d6e953d479f5ddf2dfdb6a244441010e373ac0fb88971"
|
checksum = "2896d95c02a80c6d6a5d6e953d479f5ddf2dfdb6a244441010e373ac0fb88971"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "unicase"
|
||||||
|
version = "2.8.1"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "75b844d17643ee918803943289730bec8aac480150456169e647ed0b576ba539"
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "unicode-bidi"
|
name = "unicode-bidi"
|
||||||
version = "0.3.18"
|
version = "0.3.18"
|
||||||
@ -3579,6 +3826,19 @@ dependencies = [
|
|||||||
"unicode-ident",
|
"unicode-ident",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "wasm-streams"
|
||||||
|
version = "0.4.2"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "15053d8d85c7eccdbefef60f06769760a563c7f0a9d6902a13d35c7800b0ad65"
|
||||||
|
dependencies = [
|
||||||
|
"futures-util",
|
||||||
|
"js-sys",
|
||||||
|
"wasm-bindgen",
|
||||||
|
"wasm-bindgen-futures",
|
||||||
|
"web-sys",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "web-sys"
|
name = "web-sys"
|
||||||
version = "0.3.82"
|
version = "0.3.82"
|
||||||
|
|||||||
@ -41,4 +41,5 @@ thiserror = "2.0.17"
|
|||||||
anyhow = "1.0"
|
anyhow = "1.0"
|
||||||
chrono = "0.4.38"
|
chrono = "0.4.38"
|
||||||
tera = "1.19"
|
tera = "1.19"
|
||||||
petgraph = "0.6.5"
|
petgraph = "0.8.3"
|
||||||
|
async-openai = "0.30.1"
|
||||||
|
|||||||
@ -1,17 +1,22 @@
|
|||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
use axum::{
|
use axum::{
|
||||||
extract::State,
|
extract::State,
|
||||||
|
http::StatusCode,
|
||||||
response::Json,
|
response::Json,
|
||||||
routing::get,
|
routing::{get, post},
|
||||||
Router,
|
Router,
|
||||||
};
|
};
|
||||||
use common_contracts::observability::{HealthStatus, ServiceStatus, TaskProgress};
|
use common_contracts::observability::{HealthStatus, ServiceStatus, TaskProgress};
|
||||||
|
use serde::Deserialize;
|
||||||
|
use secrecy::SecretString;
|
||||||
use crate::state::AppState;
|
use crate::state::AppState;
|
||||||
|
use crate::llm_client::LlmClient;
|
||||||
|
|
||||||
pub fn create_router(app_state: AppState) -> Router {
|
pub fn create_router(app_state: AppState) -> Router {
|
||||||
Router::new()
|
Router::new()
|
||||||
.route("/health", get(health_check))
|
.route("/health", get(health_check))
|
||||||
.route("/tasks", get(get_current_tasks))
|
.route("/tasks", get(get_current_tasks))
|
||||||
|
.route("/test-llm", post(test_llm_connection))
|
||||||
.with_state(app_state)
|
.with_state(app_state)
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -41,3 +46,26 @@ async fn get_current_tasks(State(state): State<AppState>) -> Json<Vec<TaskProgre
|
|||||||
.collect();
|
.collect();
|
||||||
Json(tasks)
|
Json(tasks)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[derive(Deserialize)]
|
||||||
|
pub struct TestLlmRequest {
|
||||||
|
pub api_base_url: String,
|
||||||
|
pub api_key: String,
|
||||||
|
pub model_id: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn test_llm_connection(
|
||||||
|
State(_state): State<AppState>,
|
||||||
|
Json(payload): Json<TestLlmRequest>,
|
||||||
|
) -> Result<Json<String>, (StatusCode, String)> {
|
||||||
|
let client = LlmClient::new(
|
||||||
|
payload.api_base_url,
|
||||||
|
SecretString::from(payload.api_key),
|
||||||
|
payload.model_id,
|
||||||
|
);
|
||||||
|
|
||||||
|
let response = client.generate_text("Hello".to_string()).await
|
||||||
|
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
|
||||||
|
|
||||||
|
Ok(Json(response))
|
||||||
|
}
|
||||||
|
|||||||
@ -1,67 +1,78 @@
|
|||||||
use crate::error::ProviderError;
|
use crate::error::ProviderError;
|
||||||
use reqwest::Client;
|
use async_openai::{
|
||||||
|
config::OpenAIConfig,
|
||||||
|
types::{CreateChatCompletionRequestArgs, ChatCompletionRequestUserMessageArgs},
|
||||||
|
Client,
|
||||||
|
};
|
||||||
use secrecy::{ExposeSecret, SecretString};
|
use secrecy::{ExposeSecret, SecretString};
|
||||||
use serde::Serialize;
|
use tracing::debug;
|
||||||
|
|
||||||
#[derive(Clone)]
|
#[derive(Clone)]
|
||||||
pub struct LlmClient {
|
pub struct LlmClient {
|
||||||
client: Client,
|
client: Client<OpenAIConfig>,
|
||||||
api_url: String,
|
|
||||||
api_key: SecretString,
|
|
||||||
model: String,
|
model: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Serialize)]
|
|
||||||
struct LlmRequest {
|
|
||||||
model: String,
|
|
||||||
prompt: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
impl LlmClient {
|
impl LlmClient {
|
||||||
pub fn new(api_url: String, api_key: SecretString, model: String) -> Self {
|
pub fn new(api_url: String, api_key: SecretString, model: String) -> Self {
|
||||||
|
let api_url = api_url.trim();
|
||||||
|
// async_openai expects the base URL to NOT include /chat/completions
|
||||||
|
// It usually expects something like "https://api.openai.com/v1"
|
||||||
|
// If the user provided a full URL like ".../chat/completions", we should strip it.
|
||||||
|
|
||||||
|
let base_url = if api_url.ends_with("/chat/completions") {
|
||||||
|
api_url.trim_end_matches("/chat/completions").trim_end_matches('/').to_string()
|
||||||
|
} else if api_url.ends_with("/completions") {
|
||||||
|
api_url.trim_end_matches("/completions").trim_end_matches('/').to_string()
|
||||||
|
} else {
|
||||||
|
api_url.trim_end_matches('/').to_string()
|
||||||
|
};
|
||||||
|
|
||||||
|
debug!("Initializing LlmClient with base_url: {}", base_url);
|
||||||
|
|
||||||
|
let config = OpenAIConfig::new()
|
||||||
|
.with_api_base(base_url)
|
||||||
|
.with_api_key(api_key.expose_secret());
|
||||||
|
|
||||||
|
let client = Client::with_config(config);
|
||||||
|
|
||||||
Self {
|
Self {
|
||||||
client: Client::new(),
|
client,
|
||||||
api_url,
|
|
||||||
api_key,
|
|
||||||
model,
|
model,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
pub async fn generate_text(&self, prompt: String) -> Result<String, ProviderError> {
|
pub async fn generate_text(&self, prompt: String) -> Result<String, ProviderError> {
|
||||||
let request_payload = LlmRequest {
|
debug!("Sending request to LLM model: {}", self.model);
|
||||||
model: self.model.clone(),
|
|
||||||
prompt,
|
|
||||||
};
|
|
||||||
|
|
||||||
let res = self
|
let request = CreateChatCompletionRequestArgs::default()
|
||||||
.client
|
.model(&self.model)
|
||||||
.post(&self.api_url)
|
.messages([
|
||||||
.bearer_auth(self.api_key.expose_secret())
|
ChatCompletionRequestUserMessageArgs::default()
|
||||||
.json(&request_payload)
|
.content(prompt)
|
||||||
.send()
|
.build()
|
||||||
.await?;
|
.map_err(|e| ProviderError::LlmApi(format!("Failed to build message: {}", e)))?
|
||||||
|
.into()
|
||||||
|
])
|
||||||
|
.build()
|
||||||
|
.map_err(|e| ProviderError::LlmApi(format!("Failed to build request: {}", e)))?;
|
||||||
|
|
||||||
if !res.status().is_success() {
|
let response = self.client.chat().create(request).await
|
||||||
let status = res.status();
|
.map_err(|e| {
|
||||||
let error_text = res
|
let err_msg = e.to_string();
|
||||||
.text()
|
if err_msg.contains("<!DOCTYPE html>") || err_msg.contains("<html") {
|
||||||
.await
|
ProviderError::LlmApi(format!("LLM API request failed: Received HTML response instead of JSON. Please check your LLM Provider Base URL configuration. It might be pointing to a web page (like the Frontend) instead of the API endpoint. Original error: {}", e))
|
||||||
.unwrap_or_else(|_| "Unknown LLM API error".to_string());
|
} else {
|
||||||
return Err(ProviderError::LlmApi(format!(
|
ProviderError::LlmApi(format!("LLM API request failed: {}", e))
|
||||||
"LLM API request failed with status {}: {}",
|
}
|
||||||
status,
|
})?;
|
||||||
error_text
|
|
||||||
)));
|
if let Some(choice) = response.choices.first() {
|
||||||
|
if let Some(content) = &choice.message.content {
|
||||||
|
return Ok(content.clone());
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// This part needs to be adapted to the actual LLM provider's response format
|
Err(ProviderError::LlmApi("LLM returned no content".to_string()))
|
||||||
let response_data: serde_json::Value = res.json().await?;
|
|
||||||
let text = response_data["choices"][0]["text"]
|
|
||||||
.as_str()
|
|
||||||
.unwrap_or("")
|
|
||||||
.to_string();
|
|
||||||
|
|
||||||
Ok(text)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@ -1,12 +1,12 @@
|
|||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
|
|
||||||
use common_contracts::messages::GenerateReportCommand;
|
use common_contracts::messages::{FinancialsPersistedEvent, GenerateReportCommand};
|
||||||
use futures::StreamExt;
|
use futures::StreamExt;
|
||||||
use tracing::{error, info};
|
use tracing::{error, info};
|
||||||
|
|
||||||
use crate::{state::AppState, worker::run_report_generation_workflow};
|
use crate::{state::AppState, worker::run_report_generation_workflow};
|
||||||
|
|
||||||
const SUBJECT_NAME: &str = "analysis.commands.generate_report";
|
const SUBJECT_NAME: &str = "events.data.financials_persisted";
|
||||||
|
|
||||||
pub async fn subscribe_to_commands(
|
pub async fn subscribe_to_commands(
|
||||||
app_state: AppState,
|
app_state: AppState,
|
||||||
@ -14,27 +14,36 @@ pub async fn subscribe_to_commands(
|
|||||||
) -> Result<(), anyhow::Error> {
|
) -> Result<(), anyhow::Error> {
|
||||||
let mut subscriber = nats_client.subscribe(SUBJECT_NAME.to_string()).await?;
|
let mut subscriber = nats_client.subscribe(SUBJECT_NAME.to_string()).await?;
|
||||||
info!(
|
info!(
|
||||||
"Consumer started, waiting for commands on subject '{}'",
|
"Consumer started, waiting for events on subject '{}'",
|
||||||
SUBJECT_NAME
|
SUBJECT_NAME
|
||||||
);
|
);
|
||||||
|
|
||||||
while let Some(message) = subscriber.next().await {
|
while let Some(message) = subscriber.next().await {
|
||||||
info!("Received NATS command to generate report.");
|
info!("Received NATS event for persisted financials.");
|
||||||
let state_clone = app_state.clone();
|
let state_clone = app_state.clone();
|
||||||
tokio::spawn(async move {
|
tokio::spawn(async move {
|
||||||
match serde_json::from_slice::<GenerateReportCommand>(&message.payload) {
|
match serde_json::from_slice::<FinancialsPersistedEvent>(&message.payload) {
|
||||||
Ok(command) => {
|
Ok(event) => {
|
||||||
|
if let Some(template_id) = event.template_id {
|
||||||
info!(
|
info!(
|
||||||
"Deserialized command for symbol: {}, template: {}",
|
"Event triggered analysis for symbol: {}, template: {}",
|
||||||
command.symbol, command.template_id
|
event.symbol, template_id
|
||||||
);
|
);
|
||||||
|
let command = GenerateReportCommand {
|
||||||
|
request_id: event.request_id,
|
||||||
|
symbol: event.symbol,
|
||||||
|
template_id,
|
||||||
|
};
|
||||||
if let Err(e) = run_report_generation_workflow(Arc::new(state_clone), command).await
|
if let Err(e) = run_report_generation_workflow(Arc::new(state_clone), command).await
|
||||||
{
|
{
|
||||||
error!("Error running report generation workflow: {:?}", e);
|
error!("Error running report generation workflow: {:?}", e);
|
||||||
}
|
}
|
||||||
|
} else {
|
||||||
|
info!("Received data persisted event for {} but no template_id provided. Skipping analysis.", event.symbol);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
error!("Failed to deserialize GenerateReportCommand: {}", e);
|
error!("Failed to deserialize FinancialsPersistedEvent: {}", e);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|||||||
@ -103,7 +103,27 @@ pub async fn run_report_generation_workflow(
|
|||||||
Err(e) => {
|
Err(e) => {
|
||||||
let err_msg = format!("LLM generation failed: {}", e);
|
let err_msg = format!("LLM generation failed: {}", e);
|
||||||
error!(module_id = %module_id, "{}", err_msg);
|
error!(module_id = %module_id, "{}", err_msg);
|
||||||
generated_results.insert(module_id.clone(), format!("Error: {}", err_msg));
|
let error_content = format!("Error: {}", err_msg);
|
||||||
|
|
||||||
|
// Persist error result so frontend knows it failed
|
||||||
|
let result_to_persist = NewAnalysisResult {
|
||||||
|
request_id: command.request_id,
|
||||||
|
symbol: command.symbol.clone(),
|
||||||
|
template_id: command.template_id.clone(),
|
||||||
|
module_id: module_id.clone(),
|
||||||
|
content: error_content.clone(),
|
||||||
|
meta_data: serde_json::json!({
|
||||||
|
"model_id": module_config.model_id,
|
||||||
|
"status": "error",
|
||||||
|
"error": err_msg
|
||||||
|
}),
|
||||||
|
};
|
||||||
|
|
||||||
|
if let Err(pe) = persistence_client.create_analysis_result(result_to_persist).await {
|
||||||
|
error!(module_id = %module_id, "Failed to persist analysis error result: {}", pe);
|
||||||
|
}
|
||||||
|
|
||||||
|
generated_results.insert(module_id.clone(), error_content);
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
@ -116,7 +136,10 @@ pub async fn run_report_generation_workflow(
|
|||||||
template_id: command.template_id.clone(),
|
template_id: command.template_id.clone(),
|
||||||
module_id: module_id.clone(),
|
module_id: module_id.clone(),
|
||||||
content: content.clone(),
|
content: content.clone(),
|
||||||
meta_data: serde_json::json!({ "model_id": module_config.model_id }),
|
meta_data: serde_json::json!({
|
||||||
|
"model_id": module_config.model_id,
|
||||||
|
"status": "success"
|
||||||
|
}),
|
||||||
};
|
};
|
||||||
|
|
||||||
if let Err(e) = persistence_client.create_analysis_result(result_to_persist).await {
|
if let Err(e) = persistence_client.create_analysis_result(result_to_persist).await {
|
||||||
@ -186,8 +209,11 @@ fn create_llm_client_for_module(
|
|||||||
))
|
))
|
||||||
})?;
|
})?;
|
||||||
|
|
||||||
|
let api_url = provider.api_base_url.clone();
|
||||||
|
info!("Creating LLM client for module '{}' using provider '{}' with URL: '{}'", module_config.name, module_config.provider_id, api_url);
|
||||||
|
|
||||||
Ok(LlmClient::new(
|
Ok(LlmClient::new(
|
||||||
provider.api_base_url.clone(),
|
api_url,
|
||||||
provider.api_key.clone().into(),
|
provider.api_key.clone().into(),
|
||||||
module_config.model_id.clone(),
|
module_config.model_id.clone(),
|
||||||
))
|
))
|
||||||
|
|||||||
@ -24,6 +24,16 @@ impl PersistenceClient {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub async fn get_company_profile(&self, symbol: &str) -> Result<Option<CompanyProfileDto>> {
|
||||||
|
let url = format!("{}/companies/{}", self.base_url, symbol);
|
||||||
|
let resp = self.client.get(&url).send().await?;
|
||||||
|
if resp.status() == reqwest::StatusCode::NOT_FOUND {
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
let profile = resp.error_for_status()?.json().await?;
|
||||||
|
Ok(Some(profile))
|
||||||
|
}
|
||||||
|
|
||||||
pub async fn upsert_company_profile(&self, profile: CompanyProfileDto) -> Result<()> {
|
pub async fn upsert_company_profile(&self, profile: CompanyProfileDto) -> Result<()> {
|
||||||
let url = format!("{}/companies", self.base_url);
|
let url = format!("{}/companies", self.base_url);
|
||||||
info!("Upserting company profile for {} to {}", profile.symbol, url);
|
info!("Upserting company profile for {} to {}", profile.symbol, url);
|
||||||
|
|||||||
@ -72,6 +72,7 @@ impl TushareDataProvider {
|
|||||||
industry,
|
industry,
|
||||||
list_date,
|
list_date,
|
||||||
additional_info: None,
|
additional_info: None,
|
||||||
|
updated_at: None,
|
||||||
};
|
};
|
||||||
|
|
||||||
// Map time-series financials into DTOs
|
// Map time-series financials into DTOs
|
||||||
|
|||||||
@ -32,6 +32,65 @@ pub async fn run_tushare_workflow(
|
|||||||
};
|
};
|
||||||
|
|
||||||
// 1. Update task progress: Fetching data
|
// 1. Update task progress: Fetching data
|
||||||
|
{
|
||||||
|
let mut entry = state
|
||||||
|
.tasks
|
||||||
|
.get_mut(&task_id)
|
||||||
|
.ok_or_else(|| AppError::Internal("Task not found".to_string()))?;
|
||||||
|
entry.status = "CheckingCache".to_string();
|
||||||
|
entry.progress_percent = 5;
|
||||||
|
entry.details = "Checking local data freshness".to_string();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check freshness
|
||||||
|
let persistence_client = PersistenceClient::new(state.config.data_persistence_service_url.clone());
|
||||||
|
let mut is_fresh = false;
|
||||||
|
match persistence_client.get_company_profile(&command.symbol).await {
|
||||||
|
Ok(Some(p)) => {
|
||||||
|
if let Some(updated_at) = p.updated_at {
|
||||||
|
let age = chrono::Utc::now() - updated_at;
|
||||||
|
if age < chrono::Duration::hours(24) {
|
||||||
|
info!("Data for {} is fresh (age: {}h). Skipping fetch.", command.symbol, age.num_hours());
|
||||||
|
is_fresh = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(None) => {}
|
||||||
|
Err(e) => tracing::warn!("Failed to check profile freshness: {}", e),
|
||||||
|
}
|
||||||
|
|
||||||
|
if is_fresh {
|
||||||
|
{
|
||||||
|
let mut entry = state
|
||||||
|
.tasks
|
||||||
|
.get_mut(&task_id)
|
||||||
|
.ok_or_else(|| AppError::Internal("Task not found".to_string()))?;
|
||||||
|
entry.status = "Completed".to_string();
|
||||||
|
entry.progress_percent = 100;
|
||||||
|
entry.details = "Data retrieved from cache".to_string();
|
||||||
|
}
|
||||||
|
|
||||||
|
let nats_client = async_nats::connect(&state.config.nats_addr)
|
||||||
|
.await
|
||||||
|
.map_err(|e| AppError::Internal(format!("NATS connection failed: {}", e)))?;
|
||||||
|
|
||||||
|
let financials_event = FinancialsPersistedEvent {
|
||||||
|
request_id: command.request_id,
|
||||||
|
symbol: command.symbol.clone(),
|
||||||
|
years_updated: vec![],
|
||||||
|
template_id: command.template_id.clone(),
|
||||||
|
};
|
||||||
|
nats_client
|
||||||
|
.publish(
|
||||||
|
"events.data.financials_persisted",
|
||||||
|
serde_json::to_vec(&financials_event).unwrap().into(),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
let _ = completion_tx.send(()).await;
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
|
||||||
{
|
{
|
||||||
let mut entry = state
|
let mut entry = state
|
||||||
.tasks
|
.tasks
|
||||||
@ -57,7 +116,7 @@ pub async fn run_tushare_workflow(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// 4. Persist data
|
// 4. Persist data
|
||||||
let persistence_client = PersistenceClient::new(state.config.data_persistence_service_url.clone());
|
// persistence_client already created above
|
||||||
persist_data(
|
persist_data(
|
||||||
&persistence_client,
|
&persistence_client,
|
||||||
&profile,
|
&profile,
|
||||||
@ -158,6 +217,7 @@ async fn publish_events(
|
|||||||
request_id: command.request_id,
|
request_id: command.request_id,
|
||||||
symbol: command.symbol.clone(),
|
symbol: command.symbol.clone(),
|
||||||
years_updated: years.into_iter().collect(),
|
years_updated: years.into_iter().collect(),
|
||||||
|
template_id: command.template_id.clone(),
|
||||||
};
|
};
|
||||||
nats_client
|
nats_client
|
||||||
.publish(
|
.publish(
|
||||||
|
|||||||
@ -41,6 +41,7 @@ pub fn map_profile(summary_json: &Value, symbol: &str) -> Result<CompanyProfileD
|
|||||||
industry,
|
industry,
|
||||||
list_date: None,
|
list_date: None,
|
||||||
additional_info: if additional.is_empty() { None } else { Some(Value::Object(additional)) },
|
additional_info: if additional.is_empty() { None } else { Some(Value::Object(additional)) },
|
||||||
|
updated_at: None,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -37,6 +37,16 @@ impl PersistenceClient {
|
|||||||
Ok(config)
|
Ok(config)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub async fn get_company_profile(&self, symbol: &str) -> Result<Option<CompanyProfileDto>> {
|
||||||
|
let url = format!("{}/companies/{}", self.base_url, symbol);
|
||||||
|
let resp = self.client.get(&url).send().await?;
|
||||||
|
if resp.status() == reqwest::StatusCode::NOT_FOUND {
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
let profile = resp.error_for_status()?.json().await?;
|
||||||
|
Ok(Some(profile))
|
||||||
|
}
|
||||||
|
|
||||||
pub async fn upsert_company_profile(&self, profile: CompanyProfileDto) -> Result<()> {
|
pub async fn upsert_company_profile(&self, profile: CompanyProfileDto) -> Result<()> {
|
||||||
let url = format!("{}/companies", self.base_url);
|
let url = format!("{}/companies", self.base_url);
|
||||||
info!("Upserting company profile for {} to {}", profile.symbol, url);
|
info!("Upserting company profile for {} to {}", profile.symbol, url);
|
||||||
|
|||||||
@ -30,12 +30,56 @@ pub async fn handle_fetch_command(
|
|||||||
state.tasks.insert(task_id, common_contracts::observability::TaskProgress {
|
state.tasks.insert(task_id, common_contracts::observability::TaskProgress {
|
||||||
request_id: task_id,
|
request_id: task_id,
|
||||||
task_name: format!("yfinance:{}", command.symbol),
|
task_name: format!("yfinance:{}", command.symbol),
|
||||||
status: "FetchingData".to_string(),
|
status: "CheckingCache".to_string(),
|
||||||
progress_percent: 10,
|
progress_percent: 5,
|
||||||
details: "Fetching data from YFinance".to_string(),
|
details: "Checking local data freshness".to_string(),
|
||||||
started_at: chrono::Utc::now(),
|
started_at: chrono::Utc::now(),
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// Check freshness
|
||||||
|
let mut is_fresh = false;
|
||||||
|
match client.get_company_profile(&command.symbol).await {
|
||||||
|
Ok(Some(p)) => {
|
||||||
|
if let Some(updated_at) = p.updated_at {
|
||||||
|
let age = chrono::Utc::now() - updated_at;
|
||||||
|
if age < chrono::Duration::hours(24) {
|
||||||
|
info!("Data for {} is fresh (age: {}h). Skipping fetch.", command.symbol, age.num_hours());
|
||||||
|
is_fresh = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(None) => {}
|
||||||
|
Err(e) => tracing::warn!("Failed to check profile freshness: {}", e),
|
||||||
|
}
|
||||||
|
|
||||||
|
if is_fresh {
|
||||||
|
if let Some(mut task) = state.tasks.get_mut(&task_id) {
|
||||||
|
task.status = "Completed".to_string();
|
||||||
|
task.progress_percent = 100;
|
||||||
|
task.details = "Data retrieved from cache".to_string();
|
||||||
|
}
|
||||||
|
|
||||||
|
let financials_event = FinancialsPersistedEvent {
|
||||||
|
request_id: task_id,
|
||||||
|
symbol: command.symbol.clone(),
|
||||||
|
years_updated: vec![],
|
||||||
|
template_id: command.template_id.clone(),
|
||||||
|
};
|
||||||
|
publisher
|
||||||
|
.publish(
|
||||||
|
"events.data.financials_persisted".to_string(),
|
||||||
|
serde_json::to_vec(&financials_event).unwrap().into(),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(mut task) = state.tasks.get_mut(&task_id) {
|
||||||
|
task.status = "FetchingData".to_string();
|
||||||
|
task.progress_percent = 10;
|
||||||
|
task.details = "Fetching data from YFinance".to_string();
|
||||||
|
}
|
||||||
|
|
||||||
// Fetch
|
// Fetch
|
||||||
let (profile, financials): (CompanyProfileDto, Vec<TimeSeriesFinancialDto>) =
|
let (profile, financials): (CompanyProfileDto, Vec<TimeSeriesFinancialDto>) =
|
||||||
state.yfinance_provider.fetch_all_data(&command.symbol).await?;
|
state.yfinance_provider.fetch_all_data(&command.symbol).await?;
|
||||||
@ -69,6 +113,7 @@ pub async fn handle_fetch_command(
|
|||||||
request_id: task_id,
|
request_id: task_id,
|
||||||
symbol: command.symbol.clone(),
|
symbol: command.symbol.clone(),
|
||||||
years_updated: years.into_iter().collect(),
|
years_updated: years.into_iter().collect(),
|
||||||
|
template_id: command.template_id.clone(),
|
||||||
};
|
};
|
||||||
publisher
|
publisher
|
||||||
.publish(
|
.publish(
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user