70 lines
10 KiB
Plaintext
70 lines
10 KiB
Plaintext
|
Ë
|
|||
|
|
˜äÝi ã ó¼ — d Z ddlmZ ddlZddlZddlZddlmZ ddlm Z m
|
|||
|
|
Z
|
|||
|
|
m Z m Z m
|
|||
|
|
Z
|
|||
|
|
ddlZddlmZ ej$ e« ZdZdZd Zd
|
|||
|
|
Zd Z G d „ d
|
|||
|
|
« Zdd„Zy)uß
|
|||
|
|
Few-shot ç»�éªŒæ ·æœ¬çš„ Chroma å�‘é‡�库å˜å‚¨ä¸Žæ£€ç´¢ã€‚
|
|||
|
|
|
|||
|
|
与 SchemaIndexer 一致：使用项目统一 ``get_embedder()``，�久化目录默认
|
|||
|
|
``./data/embeddings/chroma_fewshot``，集�� ``fewshot_samples``。
|
|||
|
|
é )Ú annotationsN)ÚPath)ÚAnyÚDictÚListÚOptionalÚTuple)ÚSettingsz ./data/embeddings/chroma_fewshotÚfewshot_samplesi€> ip iÐ c ót — e Zd ZdZ d d
|
|||
|
|
d„Zdd„Zdd„Zdddœ dd „Z d dd
|
|||
|
|
„Zdd „Z y)ÚFewShotChromaStoreu5 Few-shot JSONL → Chroma 写入与按å�‘é‡�检索。Nc ó¸ — || _ |xs dj « }t j d« xs dj « }|xs
|
|||
|
|
|xs t | _ t
|
|||
|
|
|xs t j d« xs t j « « | _ | j j dd¬« t j t | j « t d¬« ¬« | _ | j j | j
|
|||
|
|
d d
|
|||
|
|
i¬ « | _ t" j% d | j | j
|
|||
|
|
| j j' « « y )
|
|||
|
|
NÚ ÚFEWSHOT_CHROMA_COLLECTIONÚFEWSHOT_CHROMA_PATHT)ÚparentsÚexist_okF)Úanonymized_telemetry)ÚpathÚsettingsú
|
|||
|
|
hnsw:spaceÚcosine©ÚnameÚmetadataz7[OK] FewShotChromaStore: path=%s collection=%s count=%s)ÚembedderÚstripÚosÚgetenvÚCOLLECTION_NAMEÚcollection_namer ÚDEFAULT_FEWSHOT_CHROMA_DIRÚ persist_dirÚmkdirÚchromadbÚPersistentClientÚstrÚChromaSettingsÚclientÚget_or_create_collectionÚ
|
|||
|
|
collectionÚloggerÚinfoÚcount)Úselfr r# r! ÚexplicitÚfrom_envs úJC:\Users\24019\Desktop\backman-camel\backend\utils\fewshot_chroma_store.pyÚ__init__zFewShotChromaStore.__init__ s € ð !ˆŒ
|
|||
|
|
à#Ò) r×0Ñ0Ó2ˆÜ—I‘IÐ 9Ó:Ò@¸b×GÑGÓIˆØ 'Ò F¨8Ò F´ˆÔ Ü Ø
|
|||
|
|
Ò
|
|||
|
|
ZœBŸI™IÐ&;Ó <Ò
|
|||
|
|
ZÔ@Z× aÑ aÓ có
|
|||
|
|
ˆÔð
|
|||
|
|
×Ñ× Ñ t°dÐ Ô;Ü ×/Ñ/Ü�T×%Ñ%Ó&Ü#¸Ô?ô
|
|||
|
|
ˆŒ ð Ÿ+™+×>Ñ>Ø×%Ñ%Ø" HÐ-ð ?ó
|
|||
|
|
ˆŒô � ‰ Ø EØ × Ñ Ø × Ñ Ø �O‰O× !Ñ !Ó #õ
|
|||
|
|
ó c óH — t | j j « « S )N)Úintr+ r. ©r/ s r2 r. zFewShotChromaStore.count> s € Ü�4—?‘?×(Ñ(Ó*Ó+Ð+r4 c ó — | j j « dkD ri| j j | j « | j j | j ddi¬« | _ t j d« yy)u8 åˆ é™¤é›†å�ˆå†…全部文档（用于 force rebuild）。r r r r u- [OK] Few-shot Chroma 集å�ˆå·²æ¸…空并é‡�建N)r+ r. r) Údelete_collectionr! r* r, r- r7 s r2 ÚclearzFewShotChromaStore.clearA sp € à �?‰?× Ñ Ó " QÒ &Ø �K‰K× )Ñ )¨$×*>Ñ*>Ô ?Ø "Ÿk™k× BÑ BØ×)Ñ)Ø&¨Ð1ð Có ˆDŒOô
|
|||
|
|
�K‰KÐGÕ Hð
|
|||
|
|
'r4 Fé )Ú
|
|||
|
|
force_rebuildÚ
|
|||
|
|
batch_sizec
|
|||
|
|
ó¬ — |st j d« y|r#| j « dkD r| j « |sG| j « dkD r4t j d| j « « | j « S |D �cg c]$ }|j
|
|||
|
|
xs dj
|
|||
|
|
« xs d‘Œ& }}|D �cg c] }t |j « ‘Œ }}g }|D ]Å }|j t |j « |j xs ddt |j xs ddt |j xs ddt |j �t# |j « nd|j$ xs ddd t' j( |j* xs g d
|
|||
|
|
¬ « dd d
|
|||
|
|
œ« ŒÇ t j dt- |« « | j. j1 |t3 |t- |« « dd¬« }| j4 j7 |j9 « |||¬« t j dt- |« « t- |« S c c}w c c}w )uŠ
|
|||
|
|
将已解æž�çš„æ ·æœ¬åˆ—è¡¨å†™å…¥ Chroma（å�‘é‡�ç”± question_zh ç¼–ç �）。
|
|||
|
|
|
|||
|
|
Returns:
|
|||
|
|
写入�数
|
|||
|
|
u) Few-shot Chroma: æ— æ ·æœ¬ï¼Œè·³è¿‡æž„å»ºr uG Few-shot Chroma 已有 %s æ�¡ï¼Œè·³è¿‡æž„å»ºï¼ˆåŠ --force å�¯é‡�建）r Ú NéÿÿÿÿÚmediumr; F)Ú ensure_asciii )ÚqidÚ question_enÚsqlÚ explanationÚratingÚ
|
|||
|
|
difficultyÚ tags_jsonu+ Few-shot Chroma: 计算 %s æ�¡ embedding...T©r= Ú normalizeÚ
|
|||
|
|
show_progress)Ú
|
|||
|
|
embeddingsÚ documentsÚ metadatasÚidsu) [OK] Few-shot Chroma 索引完æˆ�: %s æ�¡) r, Úwarningr. r: r- Ú question_zhr r' rC ÚappendrD Ú
|
|||
|
|
_MAX_QEN_METArE Ú
|
|||
|
|
_MAX_SQL_METArF Ú_MAX_EXPLAIN_METArG r6 rH ÚjsonÚdumpsÚtagsÚlenr ÚencodeÚminr+ ÚaddÚtolist) r/ Úsamplesr< r= ÚsÚtextsrP rO rM s r2 Úbuild_from_samplesz%FewShotChromaStore.build_from_samplesK s÷ € ñ Ü �N‰NÐFÔ GØá ˜TŸZ™Z›\¨AÒ -Ø �J‰JŒLá §¡£ °Ò!1Ü �K‰KÐaÐcg×cmÑcmÓcoÔ pØ—:‘:“<Ð á?FÓG¹w¸!�!—-‘-Ò% 2×,Ñ,Ó.Ò5°#Ñ5¸wˆÐGá#*Ó+¡7˜aŒs�1—5‘5�z 7ˆÐ+Ø*,ˆ ÛˆAØ × Ñ ä ˜qŸu™u›:Ø%&§]¡]Ò%8°b¸.¼=Ð$IØ ŸE™EšK R¨ ¬-Ð8Ø$%§M¡MÒ$7°RÐ9KÔ:KÐ#LØ/0¯x©xÐ/Cœc !§(¡(œmÈØ#$§<¡<Ò#;°8¸S¸bÐ"AÜ!%§¡¨A¯F©FªL°bÀuÔ!MÈeÈtÐ!Tñõ
|
|||
|
|
|
|||
|
|
ð ô � ‰ ÐAÄ3ÀuÃ:ÔNØ—]‘]×)Ñ)Ø Ü˜:¤s¨5£zÓ2ØØ ð *ó
|
|||
|
|
ˆ
|
|||
|
|
ð
|
|||
|
|
�‰×ÑØ!×(Ñ(Ó*ØØ Øð ô
|
|||
|
|
ô � ‰ Ð?ÄÀSÃÔJÜ�3‹xˆùòA Hùâ+s  )I Â: Ic ó — | j « }|dk( rg S | j j |xs dj « xs dgddd¬« }t t |d« |« }| j j |j « |¬« }g }|d r|d d s|S t |d d |d
|
|||
|
|
d |d d |d d « D ]A \ }} }
|
|||
|
|
} d
|
|||
|
|
t | « z
|
|||
|
|
} t |
|
|||
|
|
xs i « }
|
|||
|
|
||
|
|||
|
|
d< |j | |
|
|||
|
|
| xs df« ŒC |S )uw
|
|||
|
|
Returns:
|
|||
|
|
(score, metadata_dict, document) 列表，score � SchemaIndexer 为 1-distance
|
|||
|
|
r r r? é TFrJ )Úquery_embeddingsÚ n_resultsrP Ú distancesrO rN g ð?rC )
|
|||
|
|
r. r r[ r r\ Úmaxr+ Úqueryr^ ÚzipÚfloatÚdictrS )r/ ri Útop_kÚnÚq_embÚkÚresultsÚoutÚtidÚdistÚmetaÚdocÚscoreÚms r2 Ú
|
|||
|
|
search_rawz FewShotChromaStore.search_raw… s< € ð
|