element.chen/ai-g3sb-backman2.0
1
0
Fork 0
Code Pull Requests Activity
Files
df33657099184e3e50c94b28dfebb8c7a2f29941
ai-g3sb-backman2.0/backend/utils/__pycache__/fewshot_chroma_store.cpython-312.pyc
T

88 lines
12 KiB
Plaintext
Raw Normal View History

Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
Ë
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
/ßi#ãó¼—dZddlmZddlZddlZddlZddlmZddlm Z m
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
Z
m Z m Z m
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
Z
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
ddlZddlmZej$e«ZdZdZd Zd
Zd ZGd „d
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
«Zdd„Zy)u‘
Few-shot �验样本的 Chroma ��库存储与检索。
与 SchemaIndexer 共用 ``get_embedder()``。Few-shot 默认使用 ``PersistentClient``
（``FEWSHOT_CHROMA_PATH`` / ``./data/embeddings/chroma_fewshot``），与�库脚本写入目录一致；
仅当 ``FEWSHOT_CHROMA_EPHEMERAL=true`` 或显� ``persist_to_disk=False`` 时使用内存 Chroma。
集��默认 ``fewshot_samples``。
é)Ú annotationsN)ÚPath)ÚAnyÚDictÚListÚOptionalÚTuple)ÚSettingsz ./data/embeddings/chroma_fewshotÚfewshot_samplesi€>ipiÐcó~—eZdZdZ d
ddœ dd„Zdd„Zdd„Zddd œ dd
„Z d dd „Zdd „Z y)ÚFewShotChromaStoreu5Few-shot JSONL → Chroma 写入与按å�‘é‡�检索。N)Úpersist_to_diskc óì—||_|xsdj«}tjd«xsdj«}|xs
|xst|_t
|xstjd«xstj««|_tjdd«j«dv}|€ | |_
nt|«|_
|jrW|jjdd¬«tjt|j«t!d ¬
«¬ «|_n%tj$t!d ¬
«¬ «|_|j"j'|j
d
di¬«|_|jrd|j›�nd|j›d�}t*j-d|›d|j
›d|j(j/«›�«y)u¦
Args:
embedder: 编�器实例。
persist_dir: Chroma �久化根目录；内存模�下�解�为�置引用路径。
collection_name: 集��；�空，空则读环境��或默认。
persist_to_disk: 为 True/False 时强制�盘或内存；为 None 时默认�盘，除�环境��
``FEWSHOT_CHROMA_EPHEMERAL`` 为真。
ÚÚFEWSHOT_CHROMA_COLLECTIONÚFEWSHOT_CHROMA_PATHÚFEWSHOT_CHROMA_EPHEMERAL)Ú1ÚtrueÚyesNT)ÚparentsÚexist_okF)Úanonymized_telemetry)ÚpathÚsettings)rú
hnsw:spaceÚcosine©ÚnameÚmetadatauç£�盘 u内存(é…�置路径=Ú)z[OK] FewShotChromaStore: z collection=z count=)ÚembedderÚstripÚosÚgetenvÚCOLLECTION_NAMEÚcollection_namerÚDEFAULT_FEWSHOT_CHROMA_DIRÚ persist_dirÚlowerrÚboolÚmkdirÚchromadbÚPersistentClientÚstrÚChromaSettingsÚclientÚEphemeralClientÚget_or_create_collectionÚ
collectionÚloggerÚinfoÚcount) Úselfr"r)r'rÚexplicitÚfrom_envÚ
env_ephemeralÚbackends ú%backend\utils\fewshot_chroma_store.pyÚ__init__zFewShotChromaStore.__init__"sÌ€ð !ˆŒ
à#Ò) r×0Ñ0Ó2ˆÜ—I‘IÐ9Ó:Ò@¸b×GÑGÓIˆØ'ÒF¨8ÒF´ˆÔÜØ
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
Ò
ZœBŸI™IÐ&;Ó<Ò
ZÔ@Z× aÑ aÓ có
ˆÔôŸ ™ Ð"<¸bÓA×GÑGÓIðN
ð
ˆ
ð
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
Ð "Ø'4Ð#4ˆDÕ ä#'¨Ó#8ˆDÔ Ø × Ò Ø × Ñ × "Ñ "¨4¸$Ð "Ô ?Ü"×3Ñ3ܘ×)Ñ)Ó*Ü'¸UÔCôˆD�Kô
#×2Ñ2Ü'¸UÔCôˆDŒKðŸ+™+×>Ñ>Ø×%Ñ%Ø" HÐ-ð?ó
ˆŒð ×#Ò#ð�d×&Ñ&Ð'Ñ (à'¨×(8Ñ(8Ð'9¸Ð;ð ô
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
� ‰ Ø'¨ y° ¸T×=QÑ=QÐ<RðSØ—_‘_×*Ñ*Ó,Ð-ð
/õ
ócóH—t|jj««S)N)Úintr4r7©r8s r=r7zFewShotChromaStore.count[s€Ü�4—?‘?×(Ñ(Ó*Ó+Ð+r?có—|jj«dkDri|jj|j«|jj |jddi¬«|_t jd«yy)u8删除集å�ˆå†…全部文档（用于 force rebuild）。rrrru-[OK] Few-shot Chroma 集å�ˆå·²æ¸…空并é‡�建N)r4r7r1Údelete_collectionr'r3r5r6rBs r=ÚclearzFewShotChromaStore.clear^sp€à �?‰?× Ñ Ó " QÒ &Ø �K‰K× )Ñ )¨$×*>Ñ*>Ô ?Ø"Ÿk™k×BÑBØ×)Ñ)Ø&¨Ð1ðCóˆDŒOô
�K‰KÐGÕ Hð
'r?Fé )Ú
force_rebuildÚ
batch_sizec
ó¬—|stjd«y|r#|j«dkDr|j«|sG|j«dkDr4tj d|j««|j«S|D�cgc]$}|j
xsdj
«xsd‘Œ&}}|D�cgc]}t|j«‘Œ}}g}|D]Å}|jt|j«|jxsddt|jxsddt|jxsddt|j �t#|j «nd|j$xsddd t'j(|j*xsgd
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
¬ «dd d
œ«ŒÇtj dt-|««|j.j1|t3|t-|««dd¬«}|j4j7|j9«|||¬«tj dt-|««t-|«Scc}wcc}w)uŠ
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
将已解�的样本列表写入 Chroma（��由 question_zh 编�）。
Returns:
写入�数
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
u)Few-shot Chroma: 无样本，跳过构建ruGFew-shot Chroma 已有 %s æ�¡ï¼Œè·³è¿‡æž„建（加 --force å�¯é‡�建）rÚ NéÿÿÿÿÚmediumrFF)Ú ensure_asciii )ÚqidÚ question_enÚsqlÚ explanationÚratingÚ
difficultyÚ tags_jsonu+Few-shot Chroma: 计算 %s æ�¡ embedding...T©rHÚ normalizeÚ
show_progress)Ú
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
embeddingsÚ documentsÚ metadatasÚidsu)[OK] Few-shot Chroma 索引完æˆ�: %s æ�¡)r5Úwarningr7rEr6Ú question_zhr#r/rNÚappendrOÚ
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
_MAX_QEN_METArPÚ
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
_MAX_SQL_METArQÚ_MAX_EXPLAIN_METArRrArSÚjsonÚdumpsÚtagsÚlenr"ÚencodeÚminr4ÚaddÚtolist) r8ÚsamplesrGrHÚsÚtextsr[rZrXs r=Úbuild_from_samplesz%FewShotChromaStore.build_from_sampleshs÷€ñÜ �N‰NÐFÔ GØá ˜TŸZ™Z›\¨AÒ-Ø �J‰JŒLá §¡£°Ò!1Ü �K‰KÐaÐcg×cmÑcmÓcoÔ pØ—:‘:“<Ð á?FÓG¹w¸!�!—-‘-Ò% 2×,Ñ,Ó.Ò5°#Ñ5¸wˆÐGá#*Ó+¡7˜aŒs�1—5‘5�z 7ˆÐ+Ø*,ˆ ÛˆAØ × Ñ ä˜qŸu™u›:Ø%&§]¡]Ò%8°b¸.¼=Ð$IØŸE™EšK R¨¬-Ð8Ø$%§M¡MÒ$7°RÐ9KÔ:KÐ#LØ/0¯x©xÐ/Cœc !§(¡(œmÈØ#$§<¡<Ò#;°8¸S¸bÐ"AÜ!%§¡¨A¯F©FªL°bÀuÔ!MÈeÈtÐ!Tñõ
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
ðô � ‰ ÐAÄ3ÀuÃ:ÔNØ—]‘]×)Ñ)Ø Ü˜:¤s¨5£zÓ2ØØð *ó
ˆ
ð
�‰×ÑØ!×(Ñ(Ó*ØØØð ô
ô � ‰ Ð?ÄÀSÃÔJÜ�3‹xˆùòAHùâ+s  )I Â:Icó—|j«}|dk(rgS|jj|xsdj«xsdgddd¬«}t t |d«|«}|j j|j«|¬«}g}|d r|d ds|St|d d|d
d|d d|d d«D]A\}} }
} d
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
t| «z
} t|
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
xsi«}
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
||
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
d<|j| |
| xsdf«ŒC|S)uw
Returns:
Enhance dialog context handling for Text2SQL queries by integrating session history. Introduce new methods for summarizing previous assistant messages and determining if the last interaction was a data query. Update environment configuration for embedding options and improve error handling in SQL generation. Add user-facing delivery messages for successful SQL execution. This update supports more coherent follow-up questions and improves user experience in conversational interactions.
2026-04-14 18:02:12 +08:00
(score, metadata_dict, document) 列表，score � SchemaIndexer 为 1-distance
Enhance impact analysis and configuration for vector persistence and API integration. Update main.spec to clarify data handling for ChromaDB, ensuring vector directories are excluded from packaging. Modify Text2SQLOrchestrator to unify question normalization for consistent SQL generation across languages. Introduce SchemaIndexer improvements for persistent vector storage and optimize embedding retrieval processes. Update documentation and comments for clarity on configuration changes and behavior adjustments.
2026-04-15 11:25:19 +08:00
rrrJéTFrU)Úquery_embeddingsÚ n_resultsr[Ú distancesrZrYgð?rN)
r7r"rfr#rgÚmaxr4ÚqueryriÚzipÚfloatÚdictr^)r8rtÚtop_kÚnÚq_embÚkÚresultsÚoutÚtidÚdistÚmetaÚdocÚscoreÚms r=Ú
search_rawzFewShotChromaStore.search_raw¢s<€ð
Copy Permalink