Enhance environment configuration and error handling in API server and backend. Implement dynamic loading of .env files for various execution contexts, improve schema file path resolution, and refine vector search error handling in the orchestrator. Update ChromaDB integration to support memory mode and add persistence options for few-shot learning. Include additional logging for better traceability.

This commit is contained in:
lasean.zhou
2026-04-14 18:21:50 +08:00
parent ca8bc5e7de
commit 4ea3056e95
8 changed files with 161 additions and 41 deletions
+26 -11
View File
@@ -1,8 +1,9 @@
"""
Few-shot 经验样本的 Chroma 向量库存储与检索。
与 SchemaIndexer 一致:使用项目统一 ``get_embedder()``,持久化目录默认
``./data/embeddings/chroma_fewshot``,集合名 ``fewshot_samples``。
与 SchemaIndexer 一致:使用项目统一 ``get_embedder()``;默认 Chroma 为内存
(``EphemeralClient``)。离线灌库脚本可设 ``persist_to_disk=True`` 写入磁盘。
集合名默认 ``fewshot_samples``。
"""
from __future__ import annotations
@@ -34,7 +35,16 @@ class FewShotChromaStore:
embedder: Any,
persist_dir: Optional[str] = None,
collection_name: Optional[str] = None,
*,
persist_to_disk: bool = False,
):
"""
Args:
embedder: 编码器实例。
persist_dir: 磁盘模式下的持久化目录;内存模式下仍解析为配置引用路径。
collection_name: 集合名;可空,空则读环境变量或默认。
persist_to_disk: 为 True 时使用 ``PersistentClient`` 落盘(如灌库脚本)。
"""
self.embedder = embedder
# Chroma 的 collection 名;显式参数优先,否则读 FEWSHOT_CHROMA_COLLECTION,再回退默认
explicit = (collection_name or "").strip()
@@ -43,20 +53,25 @@ class FewShotChromaStore:
self.persist_dir = Path(
(persist_dir or os.getenv("FEWSHOT_CHROMA_PATH") or DEFAULT_FEWSHOT_CHROMA_DIR).strip()
)
self.persist_dir.mkdir(parents=True, exist_ok=True)
self.client = chromadb.PersistentClient(
path=str(self.persist_dir),
settings=ChromaSettings(anonymized_telemetry=False),
)
self.persist_to_disk = persist_to_disk
if persist_to_disk:
self.persist_dir.mkdir(parents=True, exist_ok=True)
self.client = chromadb.PersistentClient(
path=str(self.persist_dir),
settings=ChromaSettings(anonymized_telemetry=False),
)
else:
self.client = chromadb.EphemeralClient(
settings=ChromaSettings(anonymized_telemetry=False),
)
self.collection = self.client.get_or_create_collection(
name=self.collection_name,
metadata={"hnsw:space": "cosine"},
)
backend = f"磁盘 {self.persist_dir}" if persist_to_disk else f"内存(配置路径={self.persist_dir})"
logger.info(
"[OK] FewShotChromaStore: path=%s collection=%s count=%s",
self.persist_dir,
self.collection_name,
self.collection.count(),
f"[OK] FewShotChromaStore: {backend} collection={self.collection_name} "
f"count={self.collection.count()}"
)
def count(self) -> int: