element.chen/ai-g3sb-backman2.0
1
0
Fork 0
Code Pull Requests Activity
Files
161b380e3ab9090f80c1ebb4310424d668b02992
ai-g3sb-backman2.0/backend/utils/__pycache__/fewshot_chroma_store.cpython-312.pyc
T

88 lines
12 KiB
Plaintext
Raw Normal View History

Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
Ë
Enhance SQL generation and streaming capabilities in the API server. Introduce optional parameters for streaming throttle and SQL stream granularity in NLChatRequest. Implement new functions for iterating SQL generation content pieces and adjusting streaming behavior based on user-defined settings. Update prompts for few-shot SQL adaptation and improve logging for SQL generation processes. Refactor orchestrator methods to support streaming responses and integrate few-shot SQL conditions. Update impact analysis documentation to reflect these changes.
2026-04-16 13:48:44 +08:00
¼Lài#"ãó¼—dZddlmZddlZddlZddlZddlmZddlm Z m
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
Z
m Z m Z m
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
Z
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
ddlZddlmZej$e«ZdZdZd Zd
Zd ZGd „d
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
«Zdd„Zy)u‘
Few-shot �验样本的 Chroma ��库存储与检索。
与 SchemaIndexer 共用 ``get_embedder()``。Few-shot 默认使用 ``PersistentClient``
（``FEWSHOT_CHROMA_PATH`` / ``./data/embeddings/chroma_fewshot``），与�库脚本写入目录一致；
仅当 ``FEWSHOT_CHROMA_EPHEMERAL=true`` 或显� ``persist_to_disk=False`` 时使用内存 Chroma。
集��默认 ``fewshot_samples``。
é)Ú annotationsN)ÚPath)ÚAnyÚDictÚListÚOptionalÚTuple)ÚSettingsz ./data/embeddings/chroma_fewshotÚfewshot_samplesi€>ipiÐcó~—eZdZdZ d
ddœ dd„Zdd„Zdd„Zddd œ dd
„Z d dd „Zdd „Z y)ÚFewShotChromaStoreu5Few-shot JSONL → Chroma 写入与按å�‘é‡�检索。N)Úpersist_to_diskc óì—||_|xsdj«}tjd«xsdj«}|xs
|xst|_t
|xstjd«xstj««|_tjdd«j«dv}|€ | |_
nt|«|_
|jrW|jjdd¬«tjt|j«t!d ¬
«¬ «|_n%tj$t!d ¬
«¬ «|_|j"j'|j
d
di¬«|_|jrd|j›�nd|j›d�}t*j-d|›d|j
›d|j(j/«›�«y)u¦
Args:
embedder: 编�器实例。
Enhance SQL generation and streaming capabilities in the API server. Introduce optional parameters for streaming throttle and SQL stream granularity in NLChatRequest. Implement new functions for iterating SQL generation content pieces and adjusting streaming behavior based on user-defined settings. Update prompts for few-shot SQL adaptation and improve logging for SQL generation processes. Refactor orchestrator methods to support streaming responses and integrate few-shot SQL conditions. Update impact analysis documentation to reflect these changes.
2026-04-16 13:48:44 +08:00
persist_dir: Chroma �久化根目录；内存模�下�解�为�置引用路径。
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
collection_name: 集��；�空，空则读环境��或默认。
persist_to_disk: 为 True/False 时强制�盘或内存；为 None 时默认�盘，除�环境��
``FEWSHOT_CHROMA_EPHEMERAL`` 为真。
ÚÚFEWSHOT_CHROMA_COLLECTIONÚFEWSHOT_CHROMA_PATHÚFEWSHOT_CHROMA_EPHEMERAL)Ú1ÚtrueÚyesNT)ÚparentsÚexist_okF)Úanonymized_telemetry)ÚpathÚsettings)rú
hnsw:spaceÚcosine©ÚnameÚmetadatauç£�盘 u内存(é…�置路径=Ú)z[OK] FewShotChromaStore: z collection=z count=)ÚembedderÚstripÚosÚgetenvÚCOLLECTION_NAMEÚcollection_namerÚDEFAULT_FEWSHOT_CHROMA_DIRÚ persist_dirÚlowerrÚboolÚmkdirÚchromadbÚPersistentClientÚstrÚChromaSettingsÚclientÚEphemeralClientÚget_or_create_collectionÚ
collectionÚloggerÚinfoÚcount) Úselfr"r)r'rÚexplicitÚfrom_envÚ
env_ephemeralÚbackends úJC:\Users\24019\Desktop\backman-camel\backend\utils\fewshot_chroma_store.pyÚ__init__zFewShotChromaStore.__init__"sÌ€ð !ˆŒ
à#Ò) r×0Ñ0Ó2ˆÜ—I‘IÐ9Ó:Ò@¸b×GÑGÓIˆØ'ÒF¨8ÒF´ˆÔÜØ
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
Ò
ZœBŸI™IÐ&;Ó<Ò
ZÔ@Z× aÑ aÓ có
ˆÔôŸ ™ Ð"<¸bÓA×GÑGÓIðN
ð
ˆ
ð
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
Ð "Ø'4Ð#4ˆDÕ ä#'¨Ó#8ˆDÔ Ø × Ò Ø × Ñ × "Ñ "¨4¸$Ð "Ô ?Ü"×3Ñ3ܘ×)Ñ)Ó*Ü'¸UÔCôˆD�Kô
#×2Ñ2Ü'¸UÔCôˆDŒKðŸ+™+×>Ñ>Ø×%Ñ%Ø" HÐ-ð?ó
ˆŒð ×#Ò#ð�d×&Ñ&Ð'Ñ (à'¨×(8Ñ(8Ð'9¸Ð;ð ô
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
� ‰ Ø'¨ y° ¸T×=QÑ=QÐ<RðSØ—_‘_×*Ñ*Ó,Ð-ð
/õ
ócóH—t|jj««S)N)Úintr4r7©r8s r=r7zFewShotChromaStore.count[s€Ü�4—?‘?×(Ñ(Ó*Ó+Ð+r?có—|jj«dkDri|jj|j«|jj |jddi¬«|_t jd«yy)u8删除集å�ˆå†…全部文档（用于 force rebuild）。rrrru-[OK] Few-shot Chroma 集å�ˆå·²æ¸…空并é‡�建N)r4r7r1Údelete_collectionr'r3r5r6rBs r=ÚclearzFewShotChromaStore.clear^sp€à �?‰?× Ñ Ó " QÒ &Ø �K‰K× )Ñ )¨$×*>Ñ*>Ô ?Ø"Ÿk™k×BÑBØ×)Ñ)Ø&¨Ð1ðCóˆDŒOô
�K‰KÐGÕ Hð
'r?Fé )Ú
force_rebuildÚ
batch_sizec
ó¬—|stjd«y|r#|j«dkDr|j«|sG|j«dkDr4tj d|j««|j«S|D�cgc]$}|j
xsdj
«xsd‘Œ&}}|D�cgc]}t|j«‘Œ}}g}|D]Å}|jt|j«|jxsddt|jxsddt|jxsddt|j �t#|j «nd|j$xsddd t'j(|j*xsgd
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
¬ «dd d
œ«ŒÇtj dt-|««|j.j1|t3|t-|««dd¬«}|j4j7|j9«|||¬«tj dt-|««t-|«Scc}wcc}w)uŠ
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
将已解�的样本列表写入 Chroma（��由 question_zh 编�）。
Returns:
写入�数
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
u)Few-shot Chroma: 无样本，跳过构建ruGFew-shot Chroma 已有 %s æ�¡ï¼Œè·³è¿‡æž„建（加 --force å�¯é‡�建）rÚ NéÿÿÿÿÚmediumrFF)Ú ensure_asciii )ÚqidÚ question_enÚsqlÚ explanationÚratingÚ
difficultyÚ tags_jsonu+Few-shot Chroma: 计算 %s æ�¡ embedding...T©rHÚ normalizeÚ
show_progress)Ú
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
embeddingsÚ documentsÚ metadatasÚidsu)[OK] Few-shot Chroma 索引完æˆ�: %s æ�¡)r5Úwarningr7rEr6Ú question_zhr#r/rNÚappendrOÚ
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
_MAX_QEN_METArPÚ
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
_MAX_SQL_METArQÚ_MAX_EXPLAIN_METArRrArSÚjsonÚdumpsÚtagsÚlenr"ÚencodeÚminr4ÚaddÚtolist) r8ÚsamplesrGrHÚsÚtextsr[rZrXs r=Úbuild_from_samplesz%FewShotChromaStore.build_from_sampleshs÷€ñÜ �N‰NÐFÔ GØá ˜TŸZ™Z›\¨AÒ-Ø �J‰JŒLá §¡£°Ò!1Ü �K‰KÐaÐcg×cmÑcmÓcoÔ pØ—:‘:“<Ð á?FÓG¹w¸!�!—-‘-Ò% 2×,Ñ,Ó.Ò5°#Ñ5¸wˆÐGá#*Ó+¡7˜aŒs�1—5‘5�z 7ˆÐ+Ø*,ˆ ÛˆAØ × Ñ ä˜qŸu™u›:Ø%&§]¡]Ò%8°b¸.¼=Ð$IØŸE™EšK R¨¬-Ð8Ø$%§M¡MÒ$7°RÐ9KÔ:KÐ#LØ/0¯x©xÐ/Cœc !§(¡(œmÈØ#$§<¡<Ò#;°8¸S¸bÐ"AÜ!%§¡¨A¯F©FªL°bÀuÔ!MÈeÈtÐ!Tñõ
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
ðô � ‰ ÐAÄ3ÀuÃ:ÔNØ—]‘]×)Ñ)Ø Ü˜:¤s¨5£zÓ2ØØð *ó
ˆ
ð
�‰×ÑØ!×(Ñ(Ó*ØØØð ô
ô � ‰ Ð?ÄÀSÃÔJÜ�3‹xˆùòAHùâ+s  )I Â:Icó—|j«}|dk(rgS|jj|xsdj«xsdgddd¬«}t t |d«|«}|j j|j«|¬«}g}|d r|d ds|St|d d|d
d|d d|d d«D]A\}} }
} d
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
t| «z
} t|
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
xsi«}
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
||
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
d<|j| |
| xsdf«ŒC|S)uw
Returns:
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
(score, metadata_dict, document) 列表，score � SchemaIndexer 为 1-distance
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
rrrJéTFrU)Úquery_embeddingsÚ n_resultsr[Ú distancesrZrYgð?rN)
r7r"rfr#rgÚmaxr4ÚqueryriÚzipÚfloatÚdictr^)r8rtÚtop_kÚnÚq_embÚkÚresultsÚoutÚtidÚdistÚmetaÚdocÚscoreÚms r=Ú
search_rawzFewShotChromaStore.search_raw¢s<€ð
Copy Permalink