之前知识只能"存",不能"找"。这一版把链路补上:切片 → 向量 → 打分 → 带来源返回。
- chunk 表 + 迁移 0006:正文按标题归段切、代码块整块保留、超长按句切、
相邻片重叠 80 字;RLS 与 GRANT 与其它租户表一致
- 向量提供方做成协议:内置 hash(字符二元组,零依赖可复现,作为基线)
与 OpenAI 兼容端点两种,换实现不动业务代码
- 检索是混合打分:向量 0.6 + pg_trgm 0.4。专有编号、单号这类必须原样命中的词
靠词法兜,换个说法表达的同一件事靠向量兜
- 三条纪律:只搜 published(过滤写在 SQL 里)、低于阈值宁可不返回、
每条命中都带来源标题与 item_id
- 发布时自动建索引,改完正文用 POST /knowledge/{id}/reindex 重建
- Web 知识页加检索框与命中卡片;新增 e2e/retrieval.mjs 6 项
为什么不是 pgvector:本机 Docker 起不来、fnsvr-01 没有该扩展且明确不动。
先用 real[] + SQL 余弦把链路跑通,将来换列类型与索引即可,接口不变。
后端 79 passed,e2e smoke/knowledge/richtext/retrieval 全绿。
This commit is contained in:
@@ -19,15 +19,18 @@ from datetime import datetime
|
||||
from sqlalchemy import (
|
||||
CheckConstraint,
|
||||
DateTime,
|
||||
Float,
|
||||
ForeignKey,
|
||||
Index,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import ARRAY
|
||||
from sqlalchemy.dialects.postgresql import UUID as PgUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from .models import Base, Timestamps
|
||||
from .models import Base, Timestamps, new_uuid
|
||||
|
||||
# 知识状态机(与任务/工单共用 item.status 一列,取值集合不同)
|
||||
KNOWLEDGE_STATUSES: tuple[str, ...] = ("draft", "in_review", "published", "archived")
|
||||
@@ -76,4 +79,34 @@ class KnowledgeDoc(Base, Timestamps):
|
||||
chunk_count: Mapped[int] = mapped_column(nullable=False, default=0)
|
||||
|
||||
|
||||
KNOWLEDGE_TENANT_TABLES: tuple[str, ...] = ("knowledge_doc",)
|
||||
class Chunk(Base):
|
||||
"""知识正文的切片。
|
||||
|
||||
为什么切片与文档同库:检索要按工作区过滤,跨库做权限校验就得把两边数据捞到应用里,
|
||||
既不安全也不快。向量本身是普通数组,换 pgvector 时只换列类型与索引。
|
||||
"""
|
||||
|
||||
__tablename__ = "chunk"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("item_id", "idx", name="uq_chunk_item_idx"),
|
||||
Index("ix_chunk_ws_item", "workspace_id", "item_id"),
|
||||
{"comment": "知识正文切片,检索的最小单位"},
|
||||
)
|
||||
|
||||
id: Mapped[uuid.UUID] = mapped_column(PgUUID(as_uuid=True), primary_key=True, default=new_uuid)
|
||||
workspace_id: Mapped[uuid.UUID] = mapped_column(PgUUID(as_uuid=True), nullable=False)
|
||||
item_id: Mapped[uuid.UUID] = mapped_column(
|
||||
PgUUID(as_uuid=True), ForeignKey("item.id", ondelete="CASCADE"), nullable=False
|
||||
)
|
||||
# 片内顺序,用于把命中片段按原顺序还原上下文
|
||||
idx: Mapped[int] = mapped_column(nullable=False)
|
||||
heading: Mapped[str | None] = mapped_column(String(200))
|
||||
content: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
char_count: Mapped[int] = mapped_column(nullable=False, default=0)
|
||||
# 向量。hash 提供方是 256 维,OpenAI 兼容端点通常是 1024/1536 维,
|
||||
# 所以不固定维度——换模型时老切片要么重算,要么维度不匹配(检索会拒绝并提示重建)。
|
||||
embedding: Mapped[list[float] | None] = mapped_column(ARRAY(Float))
|
||||
embed_model: Mapped[str | None] = mapped_column(String(64))
|
||||
|
||||
|
||||
KNOWLEDGE_TENANT_TABLES: tuple[str, ...] = ("knowledge_doc", "chunk")
|
||||
|
||||
Reference in New Issue
Block a user