本文介绍了一个利用 Zephyr-7B Beta 模型作为大型语言模型以及 Langchain 和 Chainlit 的应用程序。在这里,我将研究他们各自的能力,并展示他们在开发交互式聊天应用程序方面的潜力。我将概述用户界面 (UI) 的设计、后端处理的建立以及创建完全可操作的问答应用程序所涉及的无缝集成流程。
逐步实施
该实验展示了 Zephyr-7B-Beta 和先进人工智能技术在创建复杂问答应用程序中的实际实施,展示了增强用户交互和认知响应的潜力。
技术栈
这种技术集成提供了一个强大的框架,用于开发具有高级问答功能的交互式聊天应用程序。概述的步骤和技术堆栈共同有助于实现无缝且高效的用户体验。
Zephyr-7B Beta 概述
Zephyr Beta 是 Mistral 大型语言模型的精致 7B 迭代,经过精心训练,混合了可公开访问的合成数据集。Zephyr 拥有 70 亿个参数,在 Mistral 上进行了微调,在各种基准(ARC、HellaSwag、MMLU 和 TruthfulQA)上产生与 Llama 2 70B Chat 相当的结果。
该模型不仅超越了较大的同类产品,例如 GPT-3.5 Turbo 和 Llama 70B,而且在 Alpaca 基准测试中挑战了 GPT-4 的能力,同时保持了更紧凑的外形,比 GPT 小 25 倍。 3.5型号。
Zephyr-7B Beta 和 ChatGPT 都有一个共同的目标:充当教育和研究目的的宝贵助手。然而,他们在方法论上存在分歧。Zephyr-7B Beta 展示了解决广泛查询、从网络数据和技术资源中获取见解的多功能性。相比之下,ChatGPT 采用略有不同的训练方法并利用不同的数据集。
Zephyr-7B Beta 的成功部分归功于其通过直接蒸馏进行的微调。这涉及利用基于所选奖励和人工智能反馈的模型完成结果,从而与人类偏好保持高度一致。Zephyr-7B Beta 的另一个显着特征是其分词器的聊天模板,有助于生成更精确的响应。
Chainlit 概述
Chainlit 是一个开源 Python 包,通过无缝集成业务逻辑和数据来加快类似 ChatGPT 的应用程序的开发。
关键属性包括:
可用的集成:
对于此处所示的 RAG 实现,将使用 Langchain 框架。
解释 Langchain 框架
Langchain 是一个免费提供的框架,它简化了利用大型语言模型 (LLM) 的应用程序的开发。它提供:
向量存储解释
定义
矢量存储是专门为矢量嵌入的高效存储和检索而设计的数据库。这种专业化至关重要,因为 SQL 等传统数据库无法针对处理大量矢量数据进行微调。
嵌入的作用
嵌入以高维空间内的数值向量格式表示数据,通常是文本等非结构化数据。传统的关系数据库不适合存储和检索这些向量表示。
矢量商店的主要特点
Qdrant 概述
Qdrant是一款专门的向量相似性搜索引擎,旨在通过用户友好的 API 提供生产就绪的服务。它有助于点(向量)以及附加有效负载的存储、搜索和管理。这些有效负载充当补充信息,提高搜索精度并为用户提供有价值的数据。
Qdrant 的入门是无缝的。利用 Python qdrant-client,访问 Qdrant 的最新 Docker 映像并建立本地连接,或探索 Qdrant 的云免费套餐选项,直到你准备好全面过渡。
高级 Qdrant 架构
理解语义相似性
在一组文档或术语的上下文中,语义相似性是一种基于项目含义或语义内容的相似性来衡量项目之间距离的度量,而不是依赖于词典编排的相似性。这涉及使用数学工具来评估语言单元、概念或实例之间语义关系的强度。通过此过程获得的数字描述是通过比较支持其含义或描述其性质的信息而获得的。
区分语义相似性和语义相关性至关重要。语义相关性涵盖两个术语之间的任何关系,而语义相似性具体涉及“是”关系。这种区别阐明了语义比较的细微差别及其在各种语言和概念上下文中的应用。
什么是重新排名
技术搜索过程中的重新排名涉及称为 RAG 的两阶段过程:
第一阶段——关键词搜索
第 2 阶段 — 基于语义的 Top-k 检索
在传统的语义搜索中,采用两步方法。最初,检索机制对文档集合进行近似扫描,创建初步文档列表。随后,重新排序机制获取该候选文档列表并重新组织元素。重新排序过程通过根据特定参数重新组织结果来增强模型性能。
为什么重新排名很重要?
随着更多上下文的添加,大型语言模型 (LLM) 的召回性能往往会下降,从而导致上下文窗口扩大或“上下文填充”。重新排名背后的基本概念是将文档总数精简为固定数量。重新排序器对记录进行优先级排序和重新组织,将最相关的项目放在顶部,然后将其发送给法学硕士。通过识别可能不在前三名结果中的记录并将其合并为较小的集合,然后输入法学硕士,重新排名被证明是非常有价值的。
简介:Cohere 重新排序器
Cohere 是一家加拿大初创公司,专门研究用于改善人机交互的自然语言处理模型,在其检索器中提供了重新排名端点。这建立在上下文压缩检索器中概述的概念之上。
了解上下文压缩
文档检索的一项挑战是文档存储系统在数据摄取期间将遇到的特定查询的不确定性。这通常会导致相关信息被隐藏在包含不相关文本的文档中。通过应用程序传输整个文档可能会导致 LLM 呼叫和次优响应的成本更高。
上下文压缩解决了这个问题。核心思想涉及不立即返回检索到的文档,而是使用给定查询的上下文来压缩它们。这可确保仅传递相关信息。
本文中的术语“压缩”既指压缩单个文档的内容,也指选择性地批量过滤文档。
上下文压缩的步骤
要使用上下文压缩检索器,需要以下组件:
该过程涉及以下步骤:
安装所需的依赖项
pip install langchain qdrant-client huggingface_hub sentence-transformers PyPDF2 cohere chainlit
pip install rank_bm25
pip install llama-cpp-python
创建项目文件夹
mkdir chainlit_rag
在文件夹中创建具有以下内容的.env文件
[cohere]
api_key = Your COHERE API KEY1
下载模型参数文件
https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/blob/main/zephyr-7b-beta.Q4_K_M.gguf
从上面的链接下载模型文件,并将其保存到当前工作目录,即chainlit_rag文件夹中。
代码实现
这段代码是用Python实现的。它是基于上传的PDF文档的内容来处理和回答用户问题的系统的一部分。
它使用各种自然语言处理和检索技术。让我们来分析一下关键组件和功能:
库和导入
文本分块和处理
语言模型和检索器
初始化和用户交互
消息模板和系统初始化
WebSocket事件处理程序
用户会话管理
最终答案展示
总的来说,这段代码编排了一个复杂的系统,用于处理PDF文档、检索相关信息,并通过websocket接口为用户查询提供周到的回答。
示例代码
在chainlit_rag文件夹中创建app.py文件,并粘贴如下所示的代码。
导入所需依赖项
#import required libraries
from langchain.embeddings import HuggingFaceEmbeddings,HuggingFaceBgeEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import Qdrant
from langchain.chains import RetrievalQAWithSourcesChain,RetrievalQA
from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import (ChatPromptTemplate,
SystemMessagePromptTemplate,
HumanMessagePromptTemplate)
from langchain.llms import CTransformers
from langchain.llms import LlamaCpp
from langchain.retrievers import BM25Retriever,EnsembleRetriever
from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import CohereRerank
from cohere import Client
#
import chainlit as cl
import PyPDF2
from io import BytesIO
from getpass import getpass
#
import os
from configparser import ConfigParser
env_config = ConfigParser()
从.env文件中检索一致性API密钥
# Retrieve the cohere api key from the environmental variables
def read_config(parser: ConfigParser, location: str) -> None:
assert parser.read(location), f"Could not read config {location}"
#
CONFIG_FILE = os.path.join(".", ".env")
read_config(env_config, CONFIG_FILE)
api_key = env_config.get("cohere", "api_key").strip()
os.environ["COHERE_API_KEY"] = api_key
将检索到的文本分成有重叠的小块
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000,chunk_overlap=100)
创建一个系统模板,以指导法学硕士
system_template = """Use the following pieces of context to answer the user's question.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
ALWAYS return a "SOURCES" part in your answer.
The "SOURCES" part should be a reference to the source of the document from which you got your answer.
Begin!
- - - - - - - -
{summaries}"""
messages = [SystemMessagePromptTemplate.from_template(system_template),HumanMessagePromptTemplate.from_template("{question}"),]
prompt = ChatPromptTemplate.from_messages(messages)
chain_type_kwargs = {"prompt": prompt}
Chainlit本质上支持异步操作,使代理能够同时执行任务,并促进多个用户与单个应用程序互动。这种异步编程方法是一次管理多个任务的有效方法,允许程序在不中断或阻塞的情况下运行。
async和await关键字用于在Python中定义和处理异步代码。异步函数是一种协同例程,它是一种特殊类型的函数,可以暂停其执行并稍后恢复,允许其他任务在此期间运行。
Chainlit使用异步编程来有效地处理事件和任务。在创建Chainlit代理时,我们经常需要定义异步函数来处理事件和执行操作。
#Decorator to react to the user websocket connection event.
@cl.on_chat_start
async def init():
files = None
# Wait for the user to upload a PDF file
while files is None:
files = await cl.AskFileMessage(
content="Please upload a PDF file to begin!",
accept=["application/pdf"],
).send()
file = files[0]
msg = cl.Message(content=f"Processing `{file.name}`…")
await msg.send()
# Read the PDF file
pdf_stream = BytesIO(file.content)
pdf = PyPDF2.PdfReader(pdf_stream)
pdf_text = ""
for page in pdf.pages:
pdf_text += page.extract_text()
# Split the text into chunks
texts = text_splitter.split_text(pdf_text)
# Create metadata for each chunk
metadatas = [{"source": f"{i}-pl"} for i in range(len(texts))]
# Create a Chroma vector store
model_id = "BAAI/bge-small-en-v1.5"
embeddings = HuggingFaceBgeEmbeddings(model_name= model_id,
model_kwargs = {"device":"cpu"})
#
bm25_retriever = BM25Retriever.from_texts(texts)
bm25_retriever.k=5
# Store the embeddings in the user session
cl.user_session.set("embeddings", embeddings)
docsearch = await cl.make_async(Qdrant.from_texts)(
texts, embeddings,location=":memory:", metadatas=metadatas
)
llm = LlamaCpp(streaming=True,
model_path="zephyr-7b-beta.Q4_K_M.gguf",
max_tokens = 1500,
temperature=0.75,
top_p=1,
gpu_layers=0,
stream=True,
verbose=True,n_threads = int(os.cpu_count()/2),
n_ctx=4096)
#Hybrid Search
qdrant_retriever = docsearch.as_retriever(search_kwargs={"k":5})
ensemble_retriever = EnsembleRetriever(retrievers=[bm25_retriever,qdrant_retriever],
weights=[0.5,0.5])
#Cohere Reranker
#
compressor = CohereRerank(client=Client(api_key=os.getenv("COHERE_API_KEY")),user_agent='langchain')
#
compression_retriever = ContextualCompressionRetriever(base_compressor=compressor,
base_retriever=ensemble_retriever,
)
# Create a chain that uses the Chroma vector store
chain = RetrievalQA.from_chain_type(
llm = llm,
chain_type="stuff",
retriever=compression_retriever,
return_source_documents=True,
)
# Save the metadata and texts in the user session
cl.user_session.set("metadatas", metadatas)
cl.user_session.set("texts", texts)
# Let the user know that the system is ready
msg.content = f"`{file.name}` processed. You can now ask questions!"
await msg.update()
#store the chain as long as the user session is active
cl.user_session.set("chain", chain)
对来自UI的消息作出反应的辅助函数
cl.on_message
async def process_response(res):
# retrieve the retrieval chain initialized for the current session
chain = cl.user_session.get("chain")
# Chinlit callback handler
cb = cl.AsyncLangchainCallbackHandler(
stream_final_answer=True, answer_prefix_tokens=["FINAL", "ANSWER"])
cb.answer_reached = True
print("in retrieval QA")
res = await chain.acall(res, callbacks=[cb])
print(f"response: {res}")
answer = res["result"]
sources = res["source_documents"]
source_elements = []
# Get the metadata and texts from the user session
metadatas = cl.user_session.get("metadatas")
all_sources = [m["source"] for m in metadatas]
texts = cl.user_session.get("texts")
if sources:
found_sources = []
# Add the sources to the message
for source in sources:
print(source.metadata)
try :
source_name = source.metadata["source"]
except :
source_name = ""
# Get the index of the source
text = source.page_content
found_sources.append(source_name)
# Create the text element referenced in the message
source_elements.append(cl.Text(content=text, name=source_name))
if found_sources:
answer += f"\nSources: {', '.join(found_sources)}"
else:
answer += "\nNo sources found"
if cb.has_streamed_final_answer:
cb.final_stream.elements = source_elements
await cb.final_stream.update()
else:
await cl.Message(content=answer, elements=source_elements).send()
执行Chainlit脚本
请在命令提示符中输入以下命令以调用chainlit应用程序。
chainlit run app.py
Chainlit用户界面快照
快照1
系统运行日志
in retrieval QA
Batches: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 41.66it/s]
response: {'query': 'What is the social stigma associated with Mental Health?', 'result': " Social stigma refers to the negative attitudes, beliefs, and behaviors that society assigns to individuals with mental illness. It can lead to exclusion, prejudice, and discrimination, which in turn can affect an individual's self-esteem, social relationships, and access to opportunities and resources. Stigma is a worldwide phenomenon and continues to be a significant barrier for people seeking help for their mental health concerns. Themes associated with mental illness and stigma include beliefs that individuals with mental illness are dangerous, unpredictable, to blame for their condition, difficult to communicate with, and have poor treatment outcomes. This can lead to personal fear in others, exclusion, prejudice, and discrimination against those with mental illness.", 'source_documents': [Document(page_content='In this section the aim is to explore the concepts of, and the relationship between, stigma and mental illness. One possible reason for both conceptual confusion and reluctance to seek help is that the stigmatization of mental illness continues to be a worldwide phenomenon (Jorm et al.\n1997; Crisp et al. 2000; Sartorius 2002; Gureje et al. 2005).\nDefinition of the concept of stigma\nStigma can be viewed as a social construct. Setting people apart from other members of society has a long history. In ancient Greece members of tainted groups - for example, slaves andDefining mental health and mental illness 11 traitors - were branded with a mark (Goffman 1970; Hinshaw 2005). The concept is applied in\ndiverse circumstances, including with reference to the mentally ill (Link and Phelan 2001).Additionally, stigmatization can be seen to depend on social, economic and political power andcan occur on a large and tragic scale - for example, the systematic and dreadful stigmatization', metadata={'source': '19-pl', 'relevance_score': 0.9789956}), Document(page_content='Box 1.2 Themes associated with mental illness and stigma identi fied\nby adolescents\n•Negative attitudes towards groups described as deviant - for example, the mentally\nill - were apparent by kindergarten and increased with age (Weiss 1986, 1994;\nWahl 2002).Defining mental health and mental illness 13•Words and phrases used to describe people with mental health problems or\nmental illness were largely derogatory, with the most common labels being'retarded ', 'psycho(path) ', 'spastic ', 'mental ', 'crazy ' and 'nutter ' (Bailey 1999;\nPinfold et al. 2003).\n•The most frequently cited causes of mental illness were stress, genetics and bad\nchildhood experiences (Bailey 1999).\n•Y oung people with experience of mental health problems described being met\nwith negative attitudes and reactions from other people, including professionals(Scottish Executive 2005).\n•Although adolescents stigmatized peers with both physical and mental illness,', metadata={'source': '28-pl', 'relevance_score': 0.97631055}), Document(page_content='These explanations can be elaborated as follows: people with mental illness are perceived as\ndangerous and unpredictable; there is an implied belief that the mentally ill choose to behaveas they do and have only themselves to blame for their situation; people with mental illnessare believed to respond poorly to treatment, and outcomes are poor, therefore they are anembarrassment and should be avoided; the mentally ill are seen as dif ficult to communicate\nwith and this makes for unpredictable social intercourse. These are enduring themes, provoking personal fear in others and threatening to upset the status quo (Hayward and Bright 1997;Eminson 2004).\nExplanations for the stigmatization of the mentally ill include the following ideas.\n\x7fFrom a biological perspective, a person suffering from mental illness may be viewed as a poor genetic choice in relation to reproductive potential and as a possible threat to the safety of the individual.', metadata={'source': '22-pl', 'relevance_score': 0.9651191})]}
{'source': '19-pl', 'relevance_score': 0.9789956}
{'source': '28-pl', 'relevance_score': 0.97631055}
{'source': '22-pl', 'relevance_score': 0.9651191}
快照2
系统运行日志
llama_print_timings: load time = 1093.50 ms
llama_print_timings: sample time = 37.26 ms / 130 runs ( 0.29 ms per token, 3488.81 tokens per second)
llama_print_timings: prompt eval time = 94385.15 ms / 808 tokens ( 116.81 ms per token, 8.56 tokens per second)
llama_print_timings: eval time = 27183.79 ms / 130 runs ( 209.11 ms per token, 4.78 tokens per second)
llama_print_timings: total time = 122202.95 ms
response: {'query': 'What are the Themes associated with mental illness and stigma identified\nby adolescents?', 'result': ' According to Box 1.2 in the text, some themes associated with mental illness and stigma identified by adolescents include negative attitudes toward groups described as deviant, such as those with mental health problems, increasing with age; derogatory labels used to describe people with mental health problems or mental illness, such as "retarded," "psycho(path)," "spastic," "mental," "crazy," and "nutter"; stress, genetics, and bad childhood experiences as frequently cited causes of mental illness; and negative attitudes and reactions from other people, including professionals, towards individuals with mental health problems.', 'source_documents': [Document(page_content='Box 1.2 Themes associated with mental illness and stigma identi fied\nby adolescents\n•Negative attitudes towards groups described as deviant - for example, the mentally\nill - were apparent by kindergarten and increased with age (Weiss 1986, 1994;\nWahl 2002).Defining mental health and mental illness 13•Words and phrases used to describe people with mental health problems or\nmental illness were largely derogatory, with the most common labels being'retarded ', 'psycho(path) ', 'spastic ', 'mental ', 'crazy ' and 'nutter ' (Bailey 1999;\nPinfold et al. 2003).\n•The most frequently cited causes of mental illness were stress, genetics and bad\nchildhood experiences (Bailey 1999).\n•Y oung people with experience of mental health problems described being met\nwith negative attitudes and reactions from other people, including professionals(Scottish Executive 2005).\n•Although adolescents stigmatized peers with both physical and mental illness,', metadata={'source': '28-pl', 'relevance_score': 0.9996049}), Document(page_content='Children, mental illness and stigma\nThere is also a scarcity of research examining the issue of stigma in relation to children andmental illness (Wahl 2002; Hinshaw 2005). As described previously, the high prevalence ofmental health problems in young people and their reluctance to access specialist services givescause for concern. The indications are that children develop negative attitudes towards thosewith mental illness early on (Gale 2007). Additionally, adolescents are the adults of the futureand therefore their beliefs and attitudes regarding mental health and illness will affect servicedevelopment, the quality of life of those experiencing mental health problems and the help-seeking behaviour of individuals (Armstrong et al. 2000; Hinshaw 2005).\nBox 1.2 highlights themes identi fied from work which focused on adolescents, mental illness\nand stigma, albeit to varying degrees and using different methods.\nBox 1.2 Themes associated with mental illness and stigma identi fied', metadata={'source': '27-pl', 'relevance_score': 0.99843913}), Document(page_content='From the sparse literature available, it would appear that adolescents ' attitudes towards\nmental illness tend to be negative and stigmatizing. The need for education among the public,and adolescents in particular, in order to combat the stigma of mental illness is highlightedin the literature (Davidson and Manion 1996; Armstrong et al. 1998; Esters et al. 1998;\nBailey 1999; Secker et al. 1999; Taylor 2001; Naylor et al. 2002; Pow 2003; Hinshaw 2005;\nSessa 2005b).\nEarly indicators from our own work in Nigeria are that such attitudes transcend culture\n(Dogra 2009). However, there is evidence that stigma can be tackled. We will now examinesome of the interventions undertaken to reduce stigma among children.\nInterventions to reduce stigma\nLarge-scale interventions, such as high profile campaigns, are often difficult to evaluate. In the\nUK there have been several such campaigns - for example, The Royal College of Psychiatrists '', metadata={'source': '30-pl', 'relevance_score': 0.9777138})]}
{'source': '28-pl', 'relevance_score': 0.9996049}
{'source': '27-pl', 'relevance_score': 0.99843913}
{'source': '30-pl', 'relevance_score': 0.9777138}