面向 RAG 分块嵌入阶段的困惑度边界型迟 分隔策略

A Perplexity-Boundary-Based Late-Damage-Reduction Strategy for the RAG Block Embedding Stage

  • 摘要: 影响大模型实际应用效果,因此常基于RAG方法构建船舶领域专业知识库以辅助模型。然而,RAG在分块嵌入阶段存在 信息损失问题,难以完整表征原文信息,限制了其应用效果。针对这一问题,本文提出困惑度边界型迟分降损策略。具体 而言,针对RAG文本嵌入时因上下文语境缺失导致的信息嵌入偏差问题,采用迟分策略融合上下文信息,实现精准嵌入; 同时,考虑到迟分策略缺乏明确边界条件,且文本分块过程中存在边界模糊、信息不完整的问题,通过困惑度边界分块方 法划分文本块边界,提升分块合理性。最终,数据集验证结果表明,所提方法能够减少RAG分块嵌入过程中的信息损失, 提升RAG的有效性。

     

    Abstract: rapid expansion of data volume. Since simply fine-tuning a large model can easily lead to illusion and affect the actual application effect of the model,a knowledge base for the shipping field is often constructed based on the RAG method to assist the model. However, RAG has the problem of information loss in the block embedding stage,which makes it difficult to fully represent the original information and limits its application effect. To address this issue,this paper proposes a perplexity-boundary type delayed partition loss reduction strategy. Specifically,for the problem of information embedding deviation caused by the lack of context in the text embedding of RAG,the delayed partition strategy is used to integrate context information to achieve precise embedding;at the same time,considering that the delayed partition strategy lacks clear boundary conditions and there are problems of ambiguous boundaries and incomplete information during the text partition process,the text block boundaries are divided by the perplexity-boundary partition method to improve the rationality of partitioning. Finally,the results of dataset verification show that the proposed method can reduce the information loss in the RAG block embedding process and improve the effectiveness of RAG.

     

/

返回文章
返回