element.chen/ai-g3sb-backman2.0
1
0
Fork 0
Code Pull Requests Activity

Refactor SQL streaming behavior in API server to default to LLM native delta transmission without secondary splitting. Update NLChatRequest to clarify optional parameters for SQL stream granularity and streaming throttle. Adjust _sse_stream_text_chunks and _chat_stream_events methods to streamline content handling and improve performance. Revise impact analysis documentation to reflect these changes.

Browse Source
This commit is contained in:
陈辅元
2026-04-16 14:29:24 +08:00
parent bff5f85d60
commit 205ee96e0c
5 changed files with 117 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files
logs/text2sql_api.log
+106
View File
@@ -1 +1,107 @@
2026-04-16 13:35:24 INFO [utils.repo_logging] repo_logging.py:79 configure_text2sql_api_logging() | 日志文件: C:\Users\24019\Desktop\backman-camel\logs\text2sql_api.log
2026-04-16 13:35:29 INFO [__main__] api_server.py:85 <module>() | [OK] 已加载配置文件: C:\Users\24019\Desktop\backman-camel\.env
2026-04-16 13:35:29 INFO [__main__] api_server.py:1120 <module>() | 启动服务: http://0.0.0.0:8041
2026-04-16 13:35:29 INFO [__main__] api_server.py:1121 <module>() | API文档: http://0.0.0.0:8041/docs
2026-04-16 13:35:29 INFO [uvicorn.error] server.py:92 _serve() | Started server process [37328]
2026-04-16 13:35:29 INFO [uvicorn.error] on.py:48 startup() | Waiting for application startup.
2026-04-16 13:35:29 INFO [__main__] api_server.py:811 lifespan() | ============================================================
2026-04-16 13:35:29 INFO [__main__] api_server.py:812 lifespan() | Text2SQL API Server 启动中...
2026-04-16 13:35:29 INFO [__main__] api_server.py:813 lifespan() | ============================================================
2026-04-16 13:35:29 INFO [main] main.py:126 setup_environment() | [OK] 环境检查通过
2026-04-16 13:35:29 INFO [main] main.py:127 setup_environment() | - Schema: data\schemas\G3SB_MCDataDictionary_table_structure.json
2026-04-16 13:35:29 INFO [main] main.py:128 setup_environment() | - LLM: openai
2026-04-16 13:35:29 INFO [main] main.py:154 load_schema() | 加载Schema: ./data/schemas/G3SB_MCDataDictionary_table_structure.json
2026-04-16 13:35:29 INFO [main] main.py:156 load_schema() | 表注释(meta): ./data/schemas/G3SB_MCDataDictionary_table_meta.json
2026-04-16 13:35:30 INFO [schema.loader] loader.py:107 load_from_json() | [OK] 加载Schema完成(G3SB schemas): G3SB_MCDataDictionary_table_structure, 共2516张表
2026-04-16 13:35:30 INFO [main] main.py:163 load_schema() | [OK] Schema加载完成: G3SB_MCDataDictionary_table_structure, 共2516张表, 59196个字段
2026-04-16 13:35:33 INFO [llm.openai_client] openai_client.py:55 __init__() | [OK] OpenAIClient初始化: model=gpt-5.4, base_url=http://113.192.49.54:9080/v1
2026-04-16 13:35:35 INFO [utils.embedding] embedding.py:144 __init__() | 使用 OpenAI Embedding API：model=text-embedding-ada-002，base_url=http://113.192.49.54:9080/v1，max_batch=100
2026-04-16 13:35:37 INFO [utils.fewshot_chroma_store] fewshot_chroma_store.py:86 __init__() | [OK] FewShotChromaStore: 磁盘 data\embeddings\chroma_fewshot collection=fewshot_samples count=50
2026-04-16 13:35:37 INFO [utils.fewshot_selector] fewshot_selector.py:128 _init_chroma_mode() | [Few-shot] 已从向量库加载（Chroma 50 条，data\embeddings\chroma_fewshot）
2026-04-16 13:35:37 INFO [utils.fewshot_selector] fewshot_selector.py:159 _init_chroma_mode() | [OK] Few-shot 使用 Chroma（50 条）
2026-04-16 13:35:37 INFO [agents.orchestrator] orchestrator.py:116 __init__() | Few-shot已启用: top_k=3, min_rating=7
2026-04-16 13:35:37 INFO [agents.orchestrator] orchestrator.py:127 __init__() | [OK] Text2SQLOrchestrator初始化完成: max_retry=2, use_vector_search=True, fewshot=on, nl→zh_norm=on
2026-04-16 13:35:37 INFO [__main__] api_server.py:132 get_orchestrator() | [OK] Orchestrator 初始化完成
2026-04-16 13:35:37 INFO [__main__] api_server.py:817 lifespan() | [OK] 服务已就绪
2026-04-16 13:35:37 INFO [uvicorn.error] on.py:62 startup() | Application startup complete.
2026-04-16 13:35:37 INFO [uvicorn.error] server.py:224 _log_started_message() | Uvicorn running on http://0.0.0.0:8041 (Press CTRL+C to quit)
2026-04-16 13:35:52 INFO [uvicorn.access] httptools_impl.py:483 send() | 127.0.0.1:23615 - "POST /g3sb/api/nl/chat/stream HTTP/1.1" 200
2026-04-16 13:35:52 INFO [__main__] api_server.py:668 _chat_stream_events() | [API/stream] 开始: user_id='anonymous' visitor_biz_id=None session_id=None service_code=None model='gpt-4o-mini' lang_code='auto' msg_chars=15 preview='所有客户账户之间的股票转移记录' dialog_context_chars=0 last_turn_was_data_query=False
2026-04-16 13:35:55 INFO [llm.openai_client] openai_client.py:55 __init__() | [OK] OpenAIClient初始化: model=gpt-4o-mini, base_url=http://113.192.49.54:9080/v1
2026-04-16 13:35:55 INFO [utils.dialog_classifier] dialog_classifier.py:261 classify_dialog() | [dialog] intent=text2sql (hybrid fast: query hint) preview='所有客户账户之间的股票转移记录'
2026-04-16 13:35:55 INFO [__main__] api_server.py:715 _chat_stream_events() | [GEN/API/stream] dialect=tsql top_k=20 dialog_context_chars=0 question_len=15 preview='所有客户账户之间的股票转移记录'
2026-04-16 13:35:57 INFO [llm.openai_client] openai_client.py:55 __init__() | [OK] OpenAIClient初始化: model=gpt-4o-mini, base_url=http://113.192.49.54:9080/v1
2026-04-16 13:36:00 INFO [agents.orchestrator] orchestrator.py:803 generate() | [GEN] 问句已归一中文：所有客户账户的股票转移记录
2026-04-16 13:36:00 INFO [agents.orchestrator] orchestrator.py:816 generate() | [GEN] 开始生成SQL: question_chars=13 preview='所有客户账户的股票转移记录' dialog_context_chars=0
2026-04-16 13:36:00 INFO [agents.orchestrator] orchestrator.py:831 generate() | 尝试 #1
2026-04-16 13:36:00 INFO [schema.indexer] indexer.py:82 __init__() | [OK] 初始化SchemaIndexer(Chroma磁盘 path=data\embeddings\chroma): collection=schema_tables, count=2516
2026-04-16 13:36:00 INFO [schema.indexer] indexer.py:106 ensure_index_for_schema() | Schema 向量索引已就绪（2516 张表），跳过向量化
2026-04-16 13:36:00 INFO [agents.orchestrator] orchestrator.py:188 _coarse_filter() | [Orchestrator] 开始向量检索: query_chars=13 query_preview='所有客户账户的股票转移记录'
2026-04-16 13:36:02 INFO [utils.embedding] embedding.py:163 _set_dim_from_vector() | [OK] Embedding 向量维度：1536
2026-04-16 13:36:03 INFO [schema.indexer] indexer.py:244 search() | Schema 向量检索: query_chars=13 命中=20（阈值=0.1）top=[('VSBHKRpt0430', 0.8264), ('VSBHKRpt0431', 0.8188), ('TSBTransferInstruction', 0.8147), ('VSBTransferInstruction', 0.8134), ('VSBHKRpt0397', 0.8126), ('VSBHKRpt0672', 0.8117), ('VSBHKRpt0090C', 0.81), ('VSBHKRpt0570', 0.8084), ('VSBHKRpt1030A', 0.8077), ('VSBRpt0999E', 0.8065), ('VSBRpt0999C', 0.8064), ('TSBAccountEntitlementRelease', 0.8063), ('VSBRpt0999A', 0.8059), ('TSBAccountInstrumentMovement', 0.8038), ('VSBTransferInstructionGenerationByAccountContract', 0.8033)]
2026-04-16 13:36:03 INFO [agents.orchestrator] orchestrator.py:204 _coarse_filter() | [Orchestrator] 向量粗筛: 命中=20 张（阈值内），表名+分: [('VSBHKRpt0430', 0.8264), ('VSBHKRpt0431', 0.8188), ('TSBTransferInstruction', 0.8147), ('VSBTransferInstruction', 0.8134), ('VSBHKRpt0397', 0.8126), ('VSBHKRpt0672', 0.8117), ('VSBHKRpt0090C', 0.81), ('VSBHKRpt0570', 0.8084), ('VSBHKRpt1030A', 0.8077), ('VSBRpt0999E', 0.8065), ('VSBRpt0999C', 0.8064), ('TSBAccountEntitlementRelease', 0.8063), ('VSBRpt0999A', 0.8059), ('TSBAccountInstrumentMovement', 0.8038), ('VSBTransferInstructionGenerationByAccountContract', 0.8033), ('VSBRpt0060', 0.8021), ('XCGatewayStockReconciliationReport', 0.802), ('TSBAccountEntitlementHold', 0.8013), ('WSBBatchLocationTransferDetail', 0.8012), ('VCAccountCashMovement', 0.8012)]
2026-04-16 13:36:07 INFO [agents.orchestrator] orchestrator.py:258 _llm_select_tables() | LLM精筛选中表：['VSBHKRpt0430', 'VSBHKRpt0431', 'TSBTransferInstruction', 'VSBTransferInstruction', 'TSBAccountInstrumentMovement'] | reasoning_chars=181 reasoning_preview='问题涉及客户账户的股票转移记录，VSBHKRpt0430和VSBHKRpt0431提供了客户股票的日常转移报告和账户工具移动报告，TSBTransferInstruction和VSBTransferInstruction则记录了账户转移指令，TSBAccountInstrumentMovement包含账户工具的移动交易信息，这些表共同覆盖了股票转移的各个方面。'
2026-04-16 13:36:07 INFO [agents.orchestrator] orchestrator.py:859 generate() | 选中表：['VSBHKRpt0430', 'VSBHKRpt0431', 'TSBTransferInstruction', 'VSBTransferInstruction', 'TSBAccountInstrumentMovement']，扩展后：['TSBTransferInstruction', 'VSBTransferInstruction', 'VSBHKRpt0431', 'VSBHKRpt0430', 'TSBAccountInstrumentMovement']
2026-04-16 13:36:08 INFO [utils.fewshot_selector] fewshot_selector.py:209 select_best_with_score() | [Few-shot] best_with_score: qid=Q2 score=0.9575 preview='列出今日所有客户账户之间的股票转移记录。'
2026-04-16 13:36:08 INFO [agents.orchestrator] orchestrator.py:432 _generate_sql_golden_adapt() | [GEN] 黄金 few-shot 条件适配: qid=Q2 score=0.9575
2026-04-16 13:36:12 INFO [agents.orchestrator] orchestrator.py:450 _generate_sql_golden_adapt() | 生成的SQL（黄金适配，chars=494）:
-- 查询所有客户账户间股票转移记录
-- 使用 TSBAccountInstrumentMovement，MovementType='T' 表示账户间转移
SELECT
m.MovementID,
m.AccountID AS FromAccountID,
m.TransferToAccountID,
m.InstrumentID,
i.Name AS InstrumentSymbol,
m.MovementType,
m.Quantity AS TransferQuantity,
m.ValueDate AS TransferDate
FROM TSBAccountInstrumentMovement m
LEFT JOIN MCInstrument i ON m.InstrumentID = i.InstrumentID
WHERE m.MovementType = 'T'
AND m.ValueDate = CAST(GETDATE() AS DATE)
ORDER BY m.ValueDate;
2026-04-16 13:36:12 INFO [db.engine] engine.py:45 get_engine() | SQLAlchemy engine initialized from database_url
2026-04-16 13:36:12 INFO [db.dbhub_tools] dbhub_tools.py:627 _execute_sql() | _execute_sql 执行语句数=1 readonly=True max_rows=1
2026-04-16 13:36:15 INFO [agents.orchestrator] orchestrator.py:754 _validate_sql() | [validate] 程序+探针+LLM 汇总: valid=True err_count=0 warn_count=0 db_execution_status=0 sql_chars=494
2026-04-16 13:36:15 INFO [agents.orchestrator] orchestrator.py:921 generate() | [OK] SQL生成与验证通过（1次尝试）
2026-04-16 13:36:15 INFO [__main__] api_server.py:776 _chat_stream_events() | [API/stream] Text2SQL 完成: valid=True attempts=1 tables_used=['TSBTransferInstruction', 'VSBTransferInstruction', 'VSBHKRpt0431', 'VSBHKRpt0430', 'TSBAccountInstrumentMovement'] sql_chars=494 sql_head="-- 查询所有客户账户间股票转移记录\n-- 使用 TSBAccountInstrumentMovement，MovementType='T' 表示账户间转移\nSELECT \n m.MovementID,\n m.AccountID AS FromAccountID,\n m.TransferToAccountID,\n m.InstrumentID,\n i.Name AS InstrumentSymbol,\n m.MovementType,\n m.Quantity AS TransferQuantity,\n m.ValueDate AS TransferDate\nFROM TSBAccountInstrumentMovement m\nLEFT JOIN MCInstrument i ON m.InstrumentID = i.InstrumentID\nWHERE m.MovementType = 'T'\n AND m.ValueDate = CAST(GETDATE() AS DATE)\nORDER BY m.ValueDate;"
2026-04-16 13:47:46 INFO [uvicorn.access] httptools_impl.py:483 send() | 127.0.0.1:27791 - "POST /g3sb/api/nl/chat/stream HTTP/1.1" 200
2026-04-16 13:47:46 INFO [__main__] api_server.py:668 _chat_stream_events() | [API/stream] 开始: user_id='anonymous' visitor_biz_id=None session_id=None service_code=None model='gpt-4o-mini' lang_code='auto' msg_chars=15 preview='所有客户账户之间的股票转移记录' dialog_context_chars=0 last_turn_was_data_query=False
2026-04-16 13:47:49 INFO [llm.openai_client] openai_client.py:55 __init__() | [OK] OpenAIClient初始化: model=gpt-4o-mini, base_url=http://113.192.49.54:9080/v1
2026-04-16 13:47:49 INFO [utils.dialog_classifier] dialog_classifier.py:261 classify_dialog() | [dialog] intent=text2sql (hybrid fast: query hint) preview='所有客户账户之间的股票转移记录'
2026-04-16 13:47:49 INFO [__main__] api_server.py:715 _chat_stream_events() | [GEN/API/stream] dialect=tsql top_k=20 dialog_context_chars=0 question_len=15 preview='所有客户账户之间的股票转移记录'
2026-04-16 13:47:52 INFO [llm.openai_client] openai_client.py:55 __init__() | [OK] OpenAIClient初始化: model=gpt-4o-mini, base_url=http://113.192.49.54:9080/v1
2026-04-16 13:47:54 INFO [agents.orchestrator] orchestrator.py:803 generate() | [GEN] 问句已归一中文：所有客户账户的股票转移记录
2026-04-16 13:47:54 INFO [agents.orchestrator] orchestrator.py:816 generate() | [GEN] 开始生成SQL: question_chars=13 preview='所有客户账户的股票转移记录' dialog_context_chars=0
2026-04-16 13:47:54 INFO [agents.orchestrator] orchestrator.py:831 generate() | 尝试 #1
2026-04-16 13:47:54 INFO [schema.indexer] indexer.py:106 ensure_index_for_schema() | Schema 向量索引已就绪（2516 张表），跳过向量化
2026-04-16 13:47:54 INFO [agents.orchestrator] orchestrator.py:188 _coarse_filter() | [Orchestrator] 开始向量检索: query_chars=13 query_preview='所有客户账户的股票转移记录'
2026-04-16 13:47:56 INFO [schema.indexer] indexer.py:244 search() | Schema 向量检索: query_chars=13 命中=20（阈值=0.1）top=[('VSBHKRpt0430', 0.8264), ('VSBHKRpt0431', 0.8188), ('TSBTransferInstruction', 0.8147), ('VSBTransferInstruction', 0.8134), ('VSBHKRpt0397', 0.8126), ('VSBHKRpt0672', 0.8117), ('VSBHKRpt0090C', 0.81), ('VSBHKRpt0570', 0.8084), ('VSBHKRpt1030A', 0.8077), ('VSBRpt0999E', 0.8065), ('VSBRpt0999C', 0.8064), ('TSBAccountEntitlementRelease', 0.8063), ('VSBRpt0999A', 0.8059), ('TSBAccountInstrumentMovement', 0.8038), ('VSBTransferInstructionGenerationByAccountContract', 0.8033)]
2026-04-16 13:47:56 INFO [agents.orchestrator] orchestrator.py:204 _coarse_filter() | [Orchestrator] 向量粗筛: 命中=20 张（阈值内），表名+分: [('VSBHKRpt0430', 0.8264), ('VSBHKRpt0431', 0.8188), ('TSBTransferInstruction', 0.8147), ('VSBTransferInstruction', 0.8134), ('VSBHKRpt0397', 0.8126), ('VSBHKRpt0672', 0.8117), ('VSBHKRpt0090C', 0.81), ('VSBHKRpt0570', 0.8084), ('VSBHKRpt1030A', 0.8077), ('VSBRpt0999E', 0.8065), ('VSBRpt0999C', 0.8064), ('TSBAccountEntitlementRelease', 0.8063), ('VSBRpt0999A', 0.8059), ('TSBAccountInstrumentMovement', 0.8038), ('VSBTransferInstructionGenerationByAccountContract', 0.8033), ('VSBRpt0060', 0.8021), ('XCGatewayStockReconciliationReport', 0.802), ('TSBAccountEntitlementHold', 0.8013), ('WSBBatchLocationTransferDetail', 0.8012), ('VCAccountCashMovement', 0.8012)]
2026-04-16 13:48:00 INFO [agents.orchestrator] orchestrator.py:258 _llm_select_tables() | LLM精筛选中表：['VSBHKRpt0430', 'VSBHKRpt0431', 'TSBTransferInstruction', 'VSBTransferInstruction', 'TSBAccountInstrumentMovement'] | reasoning_chars=182 reasoning_preview='问题涉及客户账户的股票转移记录，VSBHKRpt0430和VSBHKRpt0431提供了客户股票的日常转移报告和账户工具移动报告，TSBTransferInstruction和VSBTransferInstruction包含账户转移指令的详细信息，TSBAccountInstrumentMovement记录账户工具的移动交易。这些表共同涵盖了股票转移的相关信息。'
2026-04-16 13:48:00 INFO [agents.orchestrator] orchestrator.py:859 generate() | 选中表：['VSBHKRpt0430', 'VSBHKRpt0431', 'TSBTransferInstruction', 'VSBTransferInstruction', 'TSBAccountInstrumentMovement']，扩展后：['TSBTransferInstruction', 'VSBTransferInstruction', 'VSBHKRpt0431', 'VSBHKRpt0430', 'TSBAccountInstrumentMovement']
2026-04-16 13:48:01 INFO [utils.fewshot_selector] fewshot_selector.py:209 select_best_with_score() | [Few-shot] best_with_score: qid=Q2 score=0.9575 preview='列出今日所有客户账户之间的股票转移记录。'
2026-04-16 13:48:01 INFO [agents.orchestrator] orchestrator.py:432 _generate_sql_golden_adapt() | [GEN] 黄金 few-shot 条件适配: qid=Q2 score=0.9575
2026-04-16 13:48:06 INFO [agents.orchestrator] orchestrator.py:450 _generate_sql_golden_adapt() | 生成的SQL（黄金适配，chars=494）:
-- 查询所有客户账户间股票转移记录
-- 使用 TSBAccountInstrumentMovement，MovementType='T' 表示账户间转移
SELECT
m.MovementID,
m.AccountID AS FromAccountID,
m.TransferToAccountID,
m.InstrumentID,
i.Name AS InstrumentSymbol,
m.MovementType,
m.Quantity AS TransferQuantity,
m.ValueDate AS TransferDate
FROM TSBAccountInstrumentMovement m
LEFT JOIN MCInstrument i ON m.InstrumentID = i.InstrumentID
WHERE m.MovementType = 'T'
AND m.ValueDate = CAST(GETDATE() AS DATE)
ORDER BY m.ValueDate;
2026-04-16 13:48:06 INFO [db.dbhub_tools] dbhub_tools.py:627 _execute_sql() | _execute_sql 执行语句数=1 readonly=True max_rows=1
2026-04-16 13:48:08 INFO [openai._base_client] _base_client.py:1111 _sleep_for_retry() | Retrying request to /chat/completions in 0.481978 seconds
2026-04-16 13:48:13 INFO [agents.orchestrator] orchestrator.py:754 _validate_sql() | [validate] 程序+探针+LLM 汇总: valid=True err_count=0 warn_count=0 db_execution_status=0 sql_chars=494
2026-04-16 13:48:13 INFO [agents.orchestrator] orchestrator.py:921 generate() | [OK] SQL生成与验证通过（1次尝试）
2026-04-16 13:48:13 INFO [__main__] api_server.py:776 _chat_stream_events() | [API/stream] Text2SQL 完成: valid=True attempts=1 tables_used=['TSBTransferInstruction', 'VSBTransferInstruction', 'VSBHKRpt0431', 'VSBHKRpt0430', 'TSBAccountInstrumentMovement'] sql_chars=494 sql_head="-- 查询所有客户账户间股票转移记录\n-- 使用 TSBAccountInstrumentMovement，MovementType='T' 表示账户间转移\nSELECT \n m.MovementID,\n m.AccountID AS FromAccountID,\n m.TransferToAccountID,\n m.InstrumentID,\n i.Name AS InstrumentSymbol,\n m.MovementType,\n m.Quantity AS TransferQuantity,\n m.ValueDate AS TransferDate\nFROM TSBAccountInstrumentMovement m\nLEFT JOIN MCInstrument i ON m.InstrumentID = i.InstrumentID\nWHERE m.MovementType = 'T'\n AND m.ValueDate = CAST(GETDATE() AS DATE)\nORDER BY m.ValueDate;"