element.chen/ai-g3sb-backman2.0
1
0
Fork 0
Code Pull Requests Activity

Enhance SQL generation and streaming capabilities in the API server. Introduce optional parameters for streaming throttle and SQL stream granularity in NLChatRequest. Implement new functions for iterating SQL generation content pieces and adjusting streaming behavior based on user-defined settings. Update prompts for few-shot SQL adaptation and improve logging for SQL generation processes. Refactor orchestrator methods to support streaming responses and integrate few-shot SQL conditions. Update impact analysis documentation to reflect these changes.

Browse Source
This commit is contained in:
陈辅元
2026-04-16 13:48:44 +08:00
parent 695356a496
commit bff5f85d60
16 changed files with 586 additions and 103 deletions
Download Patch File Download Diff File Expand all files Collapse all files
logs/text2sql_api.log
+1 -67
View File
@@ -1,67 +1 @@
2026-04-16 10:32:14 INFO [utils.repo_logging] repo_logging.py:79 configure_text2sql_api_logging() | 日志文件: C:\Users\24019\Desktop\backman-camel\logs\text2sql_api.log
2026-04-16 10:32:17 INFO [__main__] api_server.py:86 <module>() | [OK] 已加载配置文件: C:\Users\24019\Desktop\backman-camel\.env
2026-04-16 10:32:17 INFO [__main__] api_server.py:1047 <module>() | 启动服务: http://0.0.0.0:8041
2026-04-16 10:32:17 INFO [__main__] api_server.py:1048 <module>() | API文档: http://0.0.0.0:8041/docs
2026-04-16 10:32:17 INFO [uvicorn.error] server.py:92 _serve() | Started server process [6728]
2026-04-16 10:32:17 INFO [uvicorn.error] on.py:48 startup() | Waiting for application startup.
2026-04-16 10:32:17 INFO [__main__] api_server.py:741 lifespan() | ============================================================
2026-04-16 10:32:17 INFO [__main__] api_server.py:742 lifespan() | Text2SQL API Server 启动中...
2026-04-16 10:32:17 INFO [__main__] api_server.py:743 lifespan() | ============================================================
2026-04-16 10:32:17 INFO [main] main.py:126 setup_environment() | [OK] 环境检查通过
2026-04-16 10:32:17 INFO [main] main.py:127 setup_environment() | - Schema: data\schemas\G3SB_MCDataDictionary_table_structure.json
2026-04-16 10:32:17 INFO [main] main.py:128 setup_environment() | - LLM: openai
2026-04-16 10:32:17 INFO [main] main.py:154 load_schema() | 加载Schema: ./data/schemas/G3SB_MCDataDictionary_table_structure.json
2026-04-16 10:32:17 INFO [main] main.py:156 load_schema() | 表注释(meta): ./data/schemas/G3SB_MCDataDictionary_table_meta.json
2026-04-16 10:32:18 INFO [schema.loader] loader.py:107 load_from_json() | [OK] 加载Schema完成(G3SB schemas): G3SB_MCDataDictionary_table_structure, 共2516张表
2026-04-16 10:32:18 INFO [main] main.py:163 load_schema() | [OK] Schema加载完成: G3SB_MCDataDictionary_table_structure, 共2516张表, 59196个字段
2026-04-16 10:32:21 INFO [llm.openai_client] openai_client.py:55 __init__() | [OK] OpenAIClient初始化: model=gpt-5.4, base_url=http://113.192.49.54:9080/v1
2026-04-16 10:32:23 INFO [utils.embedding] embedding.py:144 __init__() | 使用 OpenAI Embedding API：model=text-embedding-ada-002，base_url=http://113.192.49.54:9080/v1，max_batch=100
2026-04-16 10:32:25 INFO [utils.fewshot_chroma_store] fewshot_chroma_store.py:86 __init__() | [OK] FewShotChromaStore: 磁盘 data\embeddings\chroma_fewshot collection=fewshot_samples count=50
2026-04-16 10:32:25 INFO [utils.fewshot_selector] fewshot_selector.py:128 _init_chroma_mode() | [Few-shot] 已从向量库加载（Chroma 50 条，data\embeddings\chroma_fewshot）
2026-04-16 10:32:25 INFO [utils.fewshot_selector] fewshot_selector.py:159 _init_chroma_mode() | [OK] Few-shot 使用 Chroma（50 条）
2026-04-16 10:32:25 INFO [agents.orchestrator] orchestrator.py:116 __init__() | Few-shot已启用: top_k=3, min_rating=7
2026-04-16 10:32:25 INFO [agents.orchestrator] orchestrator.py:124 __init__() | [OK] Text2SQLOrchestrator初始化完成: max_retry=2, use_vector_search=True, fewshot=on, nl→zh_norm=on
2026-04-16 10:32:25 INFO [__main__] api_server.py:133 get_orchestrator() | [OK] Orchestrator 初始化完成
2026-04-16 10:32:25 INFO [__main__] api_server.py:747 lifespan() | [OK] 服务已就绪
2026-04-16 10:32:25 INFO [uvicorn.error] on.py:62 startup() | Application startup complete.
2026-04-16 10:32:25 INFO [uvicorn.error] server.py:224 _log_started_message() | Uvicorn running on http://0.0.0.0:8041 (Press CTRL+C to quit)
2026-04-16 10:32:42 INFO [uvicorn.access] httptools_impl.py:483 send() | 127.0.0.1:56978 - "POST /g3sb/api/nl/chat/stream HTTP/1.1" 200
2026-04-16 10:32:42 INFO [__main__] api_server.py:654 _chat_stream_events() | [API/stream] 开始: user_id='anonymous' visitor_biz_id=None session_id=None service_code=None model='gpt-4o-mini' lang_code='auto' msg_chars=15 preview='所有客户账户之间的股票转移记录' dialog_context_chars=0 last_turn_was_data_query=False
2026-04-16 10:32:45 INFO [llm.openai_client] openai_client.py:55 __init__() | [OK] OpenAIClient初始化: model=gpt-4o-mini, base_url=http://113.192.49.54:9080/v1
2026-04-16 10:32:45 INFO [utils.dialog_classifier] dialog_classifier.py:261 classify_dialog() | [dialog] intent=text2sql (hybrid fast: query hint) preview='所有客户账户之间的股票转移记录'
2026-04-16 10:32:45 INFO [__main__] api_server.py:620 _run_generate() | [GEN/API] dialect=tsql top_k=20 dialog_context_chars=0 question_len=15 preview='所有客户账户之间的股票转移记录'
2026-04-16 10:32:48 INFO [llm.openai_client] openai_client.py:55 __init__() | [OK] OpenAIClient初始化: model=gpt-4o-mini, base_url=http://113.192.49.54:9080/v1
2026-04-16 10:32:51 INFO [agents.orchestrator] orchestrator.py:636 generate() | [GEN] 问句已归一中文：所有客户账户的股票转移记录
2026-04-16 10:32:51 INFO [agents.orchestrator] orchestrator.py:649 generate() | [GEN] 开始生成SQL: question_chars=13 preview='所有客户账户的股票转移记录' dialog_context_chars=0
2026-04-16 10:32:51 INFO [agents.orchestrator] orchestrator.py:664 generate() | 尝试 #1
2026-04-16 10:32:51 INFO [schema.indexer] indexer.py:82 __init__() | [OK] 初始化SchemaIndexer(Chroma磁盘 path=data\embeddings\chroma): collection=schema_tables, count=2516
2026-04-16 10:32:51 INFO [schema.indexer] indexer.py:106 ensure_index_for_schema() | Schema 向量索引已就绪（2516 张表），跳过向量化
2026-04-16 10:32:51 INFO [agents.orchestrator] orchestrator.py:185 _coarse_filter() | [Orchestrator] 开始向量检索: query_chars=13 query_preview='所有客户账户的股票转移记录'
2026-04-16 10:32:53 INFO [utils.embedding] embedding.py:163 _set_dim_from_vector() | [OK] Embedding 向量维度：1536
2026-04-16 10:32:53 INFO [schema.indexer] indexer.py:244 search() | Schema 向量检索: query_chars=13 命中=20（阈值=0.1）top=[('VSBHKRpt0430', 0.8264), ('VSBHKRpt0431', 0.8188), ('TSBTransferInstruction', 0.8147), ('VSBTransferInstruction', 0.8134), ('VSBHKRpt0397', 0.8126), ('VSBHKRpt0672', 0.8117), ('VSBHKRpt0090C', 0.81), ('VSBHKRpt0570', 0.8084), ('VSBHKRpt1030A', 0.8077), ('VSBRpt0999E', 0.8065), ('VSBRpt0999C', 0.8064), ('TSBAccountEntitlementRelease', 0.8063), ('VSBRpt0999A', 0.8059), ('TSBAccountInstrumentMovement', 0.8038), ('VSBTransferInstructionGenerationByAccountContract', 0.8033)]
2026-04-16 10:32:53 INFO [agents.orchestrator] orchestrator.py:201 _coarse_filter() | [Orchestrator] 向量粗筛: 命中=20 张（阈值内），表名+分: [('VSBHKRpt0430', 0.8264), ('VSBHKRpt0431', 0.8188), ('TSBTransferInstruction', 0.8147), ('VSBTransferInstruction', 0.8134), ('VSBHKRpt0397', 0.8126), ('VSBHKRpt0672', 0.8117), ('VSBHKRpt0090C', 0.81), ('VSBHKRpt0570', 0.8084), ('VSBHKRpt1030A', 0.8077), ('VSBRpt0999E', 0.8065), ('VSBRpt0999C', 0.8064), ('TSBAccountEntitlementRelease', 0.8063), ('VSBRpt0999A', 0.8059), ('TSBAccountInstrumentMovement', 0.8038), ('VSBTransferInstructionGenerationByAccountContract', 0.8033), ('VSBRpt0060', 0.8021), ('XCGatewayStockReconciliationReport', 0.802), ('TSBAccountEntitlementHold', 0.8013), ('WSBBatchLocationTransferDetail', 0.8012), ('VCAccountCashMovement', 0.8012)]
2026-04-16 10:32:57 INFO [agents.orchestrator] orchestrator.py:255 _llm_select_tables() | LLM精筛选中表：['VSBHKRpt0430', 'VSBHKRpt0431', 'TSBTransferInstruction', 'VSBTransferInstruction', 'TSBAccountInstrumentMovement'] | reasoning_chars=175 reasoning_preview='问题涉及客户账户的股票转移记录，VSBHKRpt0430和VSBHKRpt0431提供了客户股票转移的日报信息，TSBTransferInstruction和VSBTransferInstruction记录账户转移指令，TSBAccountInstrumentMovement则详细记录账户的股票移动交易。这些表共同涵盖了客户账户的股票转移相关信息。'
2026-04-16 10:32:57 INFO [agents.orchestrator] orchestrator.py:692 generate() | 选中表：['VSBHKRpt0430', 'VSBHKRpt0431', 'TSBTransferInstruction', 'VSBTransferInstruction', 'TSBAccountInstrumentMovement']，扩展后：['VSBHKRpt0430', 'VSBHKRpt0431', 'TSBTransferInstruction', 'TSBAccountInstrumentMovement', 'VSBTransferInstruction']
2026-04-16 10:32:58 INFO [utils.fewshot_selector] fewshot_selector.py:271 _select_chroma() | Few-shot(Chroma)选择: 问题='所有客户账户的股票转移记录...' → 选中3个示例 (top_k=3, min_rating=7)
2026-04-16 10:32:58 INFO [agents.orchestrator] orchestrator.py:382 _generate_sql() | 已注入 3 个 few-shot 示例: qid=['Q2', 'Q40', 'Q22'] question_zh_preview=['列出今日所有客户账户之间的股票转移记录。', '列出今日执行的所有股票转移/移动。', '列出今日所有公司账户与客户账户之间的现金划转。']
2026-04-16 10:33:07 INFO [agents.orchestrator] orchestrator.py:447 _generate_sql() | 生成的SQL（chars=476）:
SELECT
m.MovementID,
m.AccountID AS FromAccountID,
m.TransferToAccountID,
m.InstrumentID,
i.Name AS InstrumentSymbol,
m.MovementType,
m.Quantity AS TransferQuantity,
m.ValueDate AS TransferDate
FROM TSBAccountInstrumentMovement m
LEFT JOIN MCInstrument i ON m.InstrumentID = i.InstrumentID
WHERE m.MovementType = 'T'
AND m.ValueDate >= CAST(GETDATE() AS DATE)
AND m.ValueDate < DATEADD(DAY, 1, CAST(GETDATE() AS DATE))
ORDER BY m.ValueDate;
2026-04-16 10:33:07 INFO [db.engine] engine.py:45 get_engine() | SQLAlchemy engine initialized from database_url
2026-04-16 10:33:07 INFO [db.dbhub_tools] dbhub_tools.py:627 _execute_sql() | _execute_sql 执行语句数=1 readonly=True max_rows=1
2026-04-16 10:33:10 INFO [agents.orchestrator] orchestrator.py:590 _validate_sql() | [validate] 程序+探针+LLM 汇总: valid=True err_count=0 warn_count=0 db_execution_status=0 sql_chars=476
2026-04-16 10:33:10 INFO [agents.orchestrator] orchestrator.py:753 generate() | [OK] SQL生成与验证通过（1次尝试）
2026-04-16 10:33:10 INFO [__main__] api_server.py:703 _chat_stream_events() | [API/stream] Text2SQL 完成: valid=True attempts=1 tables_used=['VSBHKRpt0430', 'VSBHKRpt0431', 'TSBTransferInstruction', 'TSBAccountInstrumentMovement', 'VSBTransferInstruction'] sql_chars=476 sql_head="SELECT\n m.MovementID,\n m.AccountID AS FromAccountID,\n m.TransferToAccountID,\n m.InstrumentID,\n i.Name AS InstrumentSymbol,\n m.MovementType,\n m.Quantity AS TransferQuantity,\n m.ValueDate AS TransferDate\nFROM TSBAccountInstrumentMovement m\nLEFT JOIN MCInstrument i ON m.InstrumentID = i.InstrumentID\nWHERE m.MovementType = 'T'\n AND m.ValueDate >= CAST(GETDATE() AS DATE)\n AND m.ValueDate < DATEADD(DAY, 1, CAST(GETDATE() AS DATE))\nORDER BY m.ValueDate;"
2026-04-16 13:35:24 INFO [utils.repo_logging] repo_logging.py:79 configure_text2sql_api_logging() | 日志文件: C:\Users\24019\Desktop\backman-camel\logs\text2sql_api.log