基于LangChain的LLM应用开发2——模型、提示和输出解析

本次会讲解LangChain的三个基本组件：模型、提示和解析器。

名词解析

模型（Models）：是指作为基础的大语言模型。LangChain中通过ChatOpenAI或者AzureChatOpenAI（部署在微软Azure的openai模型）等类来集成语言模型。

提示（Prompts）：是指给模型传递信息，让模型按要求生成我们想要的内容。LangChain中通过ChatPromptTemplate实现。

解析器（Parsers）：用于对大模型的输出进行解析，解析成更结构化的格式，如将大模型返回的json格式结果（用markdown标记）解析成Python的词典对象。LangChain中通过ResponseSchema, StructuredOutputParser实现。

我们开发LLM应用，往往都是这样的流程：重复提示模型，解析模型输出，LangChain基于此提供了更简洁优雅的封装。

下面通过代码来逐步讲解。

搭建环境

由于众所周知的原因，访问ChatGPT困难重重，特别是调用其API，想付费都无比艰难。所以这里推荐去申请微软的Azure（https://portal.azure.com/）里面的大语言模型，可以用国内的信用卡按量付费。

里面关键是进入“模型部署”部署模型，然后去“密钥和终结点”找到自己的密钥和终结点。

首先要安装python-dotenv和openai模块。使用python-dotenv库来读取.env文件初始化环境变量。

!pip install python-dotenv
!pip install openai

在你的项目目录中创建一个.env文件，并将你的OpenAI API密钥、终结点（在“密钥和终结点”里面找）等添加到该文件中。这里.env文件的概念和Docker Compose用的.env文件是一样的。

.env文件的内容如下：

OPENAI_API_BASE=https://endpoint.openai.azure.com/ #终结点
OPENAI_API_KEY=youropenapikey #密钥
OPENAI_API_TYPE=azure
OPENAI_API_VERSION=2023-07-01-preview

使用以下代码加载.env文件中的环境变量：

import openai

from dotenv import load_dotenv, find_dotenv
load_dotenv(find_dotenv()) # read local .env file

deployment = "gpt-35-turbo" # 就是上面的部署名
model = "gpt-3.5-turbo"

通过直接调用Api的方式来进行文本翻译

我们先通过“传统”的调用Api的方式来完成程序的功能：将海盗英语翻译成“优雅尊重的美式英语”。

def get_completion(prompt, model="gpt-3.5-turbo", engine="gpt-35-turbo"):
    messages = [{"role": "user", "content": prompt}]
    response = openai.ChatCompletion.create(
        model=model,
        engine=engine, #使用Azure的GPT-3.5模型要用这个
        messages=messages,
        temperature=0,
    )
    return response.choices[0].message["content"]

customer_email = """
Arrr, I be fuming that me blender lid \
flew off and splattered me kitchen walls \
with smoothie! And to make matters worse,\
the warranty don't cover the cost of \
cleaning up me kitchen. I need yer help \
right now, matey!
"""
style = """American English in a calm and respectful tone"""
prompt = f"""Translate the text \
that is delimited by triple backticks
into a style that is {style}.
text: ```{customer_email}```
"""
print(prompt)
response = get_completion(prompt)
print(response)

这里使用Python的fstring来实现提示的“参数化”。在Python世界，fstring很常用，就是多了个f感觉不太美观，不如直接用groovy的gstring的形式。

最终传给模型的Prompt是这样的：

Translate the text that is delimited by triple backticks
into a style that is American English in a calm and respectful tone.
text: \```
Arrr, I be fuming that me blender lid flew off and splattered me kitchen walls with smoothie! And to make matters worse,the warranty don't cover the cost of cleaning up me kitchen. I need yer help right now, matey!
\```

这里采用了一个小技巧来明确告诉模型要翻译的文本：把翻译的文本用三重引号（triple backticks）来括住。

模型运行返回了很漂亮的英语：I'm really frustrated that my blender lid flew off and made a mess of my kitchen walls with smoothie! And to add to my frustration, the warranty doesn't cover the cost of cleaning up my kitchen. I could really use your help at this moment, my friend.

通过LangChain的方式来进行文本翻译

下面通过LangChain的方式来完成同样的功能，看看有何不同。

同样，先安装类库：

#!pip install --upgrade langchain

这里用了—upgrade参数，这是因为LangChain还在飞速迭代，版本更新很快，碰到问题可以先考虑升级库。

from langchain.chat_models import AzureChatOpenAI
from langchain.prompts import ChatPromptTemplate

# from langchain.chat_models import ChatOpenAI #直接访问OpenAI的GPT服务
# To control the randomness and creativity of the generated
# text by an LLM, use temperature = 0.0

chat = AzureChatOpenAI(temperature=0.0, deployment_name = deployment, model_name=model)
template_string = """Translate the text \
that is delimited by triple backticks \
into a style that is {style}. \
text: ```{text}```
"""

prompt_template = ChatPromptTemplate.from_template(template_string)
print(prompt_template.messages[0].prompt.input_variables) #['style', 'text']
customer_style = """American English in a calm and respectful tone"""
customer_email = """
Arrr, I be fuming that me blender lid \
flew off and splattered me kitchen walls \
with smoothie! And to make matters worse, \
the warranty don't cover the cost of \
cleaning up me kitchen. I need yer help \
right now, matey!
"""
customer_messages = prompt_template.format_messages(
    style=customer_style,
    text=customer_email)
# print(type(customer_messages))
# print(type(customer_messages[0]))
# print(customer_messages[0])

# Call the LLM to translate to the style of the customer message
customer_response = chat(customer_messages)
print(customer_response.content)

这里很重要的组件是ChatPromptTemplate，ChatPromptTemplate会从我们的提示字符串中识别出两个输入变量/输入参数['style', 'text']，后面直接把这两个参数传进去就可以构造真正的消息。

运行程序，也能得到一样的结果：

I'm really frustrated that my blender lid flew off and made a mess of my kitchen walls with smoothie! And to make things even worse, the warranty doesn't cover the cost of cleaning up my kitchen. I could really use your help right now, my friend!

我们再进一步，假如客服收到上面客户的投诉，想用海盗英语“怼回去”：

service_reply = """Hey there customer, the warranty does not cover cleaning expenses \
for your kitchen because it's your fault that you misused your blender by forgetting \
 to put the lid on before starting the blender. Tough luck! See ya!
"""
service_style_pirate = """a polite tone that speaks in English Pirate"""
service_messages = prompt_template.format_messages(
    style=service_style_pirate,
    text=service_reply)

print(service_messages[0].content)
service_response = chat(service_messages)
print(service_response.content)

这里就可以直接重用prompt_template，只需要改变调用的参数。

最终得到很”polite”的海盗英语：Ahoy there, matey! I regret to inform ye that the warranty be not coverin' the costs o' cleanin' yer galley, as 'tis yer own fault fer misusin' yer blender by forgettin' to secure the lid afore startin' it. Aye, tough luck, me heartie! Fare thee well!

为什么要使用LangChain的Prompt Template而不直接使用Python的fstring呢？

答案是当你构建复杂的应用时，提示可能会很长很复杂，这时Prompt Template就是很好的抽象，方便你重用提示。LangChain还为一些常见操作提供了提示，如摘要、回答问题、连接到SQL数据库、连接到不同的api等等。LangChain在背后做了优化的动作，不需要再花太多时间在设计提示上面。不然我们自定义一个函数，用fstring也可以重用prompt，只是那就真的要从最底层做起，可能做着做着就做了另一个LangChain？

通过解析器来解析模型输出的结果

上面的例子模型的输出很简单，我们再来看输出复杂的结果（json格式）怎么处理。

我们让模型来帮忙处理客户的评论，针对客户的评论，用json的格式输出是否礼物、送达时间、价格/价值。

from langchain.prompts import ChatPromptTemplate

customer_review = """\
This leaf blower is pretty amazing.  It has four settings:\
candle blower, gentle breeze, windy city, and tornado. \
It arrived in two days, just in time for my wife's \
anniversary present. \
I think my wife liked it so much she was speechless. \
So far I've been the only one using it, and I've been \
using it every other morning to clear the leaves on our lawn. \
It's slightly more expensive than the other leaf blowers \
out there, but I think it's worth it for the extra features.
"""

review_template = """\
For the following text, extract the following information:

gift: Was the item purchased as a gift for someone else? \
Answer True if yes, False if not or unknown.

delivery_days: How many days did it take for the product \
to arrive? If this information is not found, output -1.

price_value: Extract any sentences about the value or price,\
and output them as a comma separated Python list.

Format the output as JSON with the following keys:
gift
delivery_days
price_value

text: {text}
"""

prompt_template = ChatPromptTemplate.from_template(review_template)
# print(prompt_template)
messages = prompt_template.format_messages(text=customer_review)
chat = AzureChatOpenAI(temperature=0.0, deployment_name = deployment, model_name=model)
response = chat(messages)
print(response.content)

运行程序，结果看起来很完美，很标准的json格式。

{
  "gift": false,
  "delivery_days": 2,
  "price_value": ["It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features."]
}

注意，这里的gift是false，如果是在ChatGPT网站上用同样的提示，GPT-3.5返回的gift也是false。但是吴恩达教授在视频中展示的结果（参考1）gift是true，是正确的。所以GPT-3.5真的退步了？

如果你打印结果的类型type(response.content)就知道返回的是str类型。这里可以通过json.loads()函数将JSON格式的字符串转换为Python的词典对象。

我们来看一下如何使用LangChain的解析器来解析模型的输出结果。

from langchain.output_parsers import ResponseSchema, StructuredOutputParser

gift_schema = ResponseSchema(name="gift",
                             description="Was the item purchased as a gift for someone else? Answer True if yes,\
                             False if not or unknown.")
delivery_days_schema = ResponseSchema(name="delivery_days",
                                      type="int", #注意这里根据需要，可能要指定类型
                                      description="How many days did it take for the product to arrive? If this  information is not found,\
                                      output -1.")
price_value_schema = ResponseSchema(name="price_value",
                                    description="Extract any\
                                    sentences about the value or \
                                    price, and output them as a \
                                    comma separated Python list.")

response_schemas = [gift_schema,
                    delivery_days_schema,
                    price_value_schema]
output_parser = StructuredOutputParser.from_response_schemas(response_schemas)
format_instructions = output_parser.get_format_instructions()
print(format_instructions)

review_template_2 = """\
For the following text, extract the following information:

gift: Was the item purchased as a gift for someone else? \
Answer True if yes, False if not or unknown.

delivery_days: How many days did it take for the product\
to arrive? If this information is not found, output -1.

price_value: Extract any sentences about the value or price,\
and output them as a comma separated Python list.

text: {text}

{format_instructions}
"""

prompt = ChatPromptTemplate.from_template(template=review_template_2)
messages = prompt.format_messages(text=customer_review,
                                  format_instructions=format_instructions)
# print(messages[0].content)
response = chat(messages)
print(response.content)
output_dict = output_parser.parse(response.content)
print(output_dict)

通过ResponseSchema来定义输出的格式，最终生成的format_instructions如下，就是让模型输出用markdown标记的json。这个可以算是LangChain的价值所在，LangChain可以持续升级优化这些输出，我们只需要跟着升级。

The output should be a markdown code snippet formatted in the following schema,
including the leading and trailing "\```json" and "\```":

\```json
{
	"gift": string  // Was the item purchased as a gift for someone else? Answer True if yes,                             False if not or unknown.
	"delivery_days": int  // How many days did it take for the product to arrive? If this  information is not found,                                      output -1.
	"price_value": string  // Extract any sentences about the value or price, and output them as a                                     comma separated Python list.
}
\```

response.content得到的是：

\```json
{
	"gift": false,
	"delivery_days": 2,
	"price_value": ["It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features."]
}
\```

再经过output_parser.parse(response.content)解析，就得到了Python的词典对象，方便我们进一步处理。

参考

1. https://learn.deeplearning.ai/langchain/lesson/2/models,-prompts-and-parsers
2. https://python.langchain.com/docs/modules/model_io/prompts/