第13章 工具增强

# 第13章 工具增强 ## 用 Python 工具增强 LLM 的推理能力 LMQL 是 Python 的超集，因此查询程序可以使用任意 Python 构造，包括函数调用。例如，下面我们先让模型生成一个简单的数学题，然后用 Python 的 eval 函数计算结果。 ```lmql # generates an arithmetic expression "A simple math problem for addition (without solution, \ without words): [MATH]" where STOPS_BEFORE(MATH, "=") # evaluate the expression and feed it back into the prompt "= {eval(MATH.strip())}" ``` ```promptdown A simple math problem for addition (without solution, without words): MATH 7 + 8 = 15 ``` 这里和 Python 的 f-string 类似，我们用 {...} 语法把 eval 函数的结果重新插入到提示词中。这样就用一个简单的计算器增强了大语言模型的推理能力。 ## 计算器 在前一个例子的基础上，我们可以构建一个能处理更复杂表达式的改进版计算器： ```lmql import re from lmql.demo import gsm8k_samples def calc(expr): expr = re.sub(r"[^0-9+\-*/().]", "", expr) return eval(expr) QUESTION = "Josh decides to try flipping a house. \ He buys a house for $80,000 and then puts in $50,000 in repairs. \ This increased the value of the house by 150%. \ How much profit did he make?" # insert few shot demonstrations "{gsm8k_samples()}" # prompt tempalte "Q: {QUESTION}\n" "Let's think step by step.\n" # reasoning loop for i in range(4): "[REASON_OR_CALC]" \ where STOPS_AT(REASON_OR_CALC, "<<") and \ STOPS_AT(REASON_OR_CALC, "So the answer") if REASON_OR_CALC.endswith("<<"): " [EXPR]" where STOPS_AT(EXPR, "=") # invoke calculator function " {calc(EXPR)}>>" elif REASON_OR_CALC.endswith("So the answer"): break # produce the final answer "is[RESULT]" ``` ```promptdown Q: Josh decides to try flipping a house. He buys a house for $80,000 and then puts in $50,000 in repairs. This increased the value of the house by 150%. How much profit did he make? Let's think step by step. REASON_OR_CALC Josh bought the house for $80,000 and put in $50,000 in repairs. The value of the house increased by 150%, so the new value of the house is $80,000 + 150% of $80,000 = << EXPR 80,000 + (80,000*1.5) = 200000.0 >> REASON_OR_CALC The profit Josh made is the difference between the new value of the house and the amount he spent on it, which is $200,000 - $80,000 - $50,000 = << EXPR 200,000 - 80,000 - 50,000 = 70000 >> REASON_OR_CALC $70,000. So the answer is RESULT $70,000. ``` 这里我们定义了一个 calc 函数，它借助内置的 re 正则表达式库，先去掉输入中的非数字字符，再调用 eval。 此外，我们还使用了 gsm8k_samples 函数，它返回 gsm8k 数据集的几个 few-shot 样本，让模型预先学会工具调用的正确格式。 ## 不止于计算器 ### Wikipedia 搜索 工具调用并不局限于计算器。下面的例子展示了如何通过文本检索（使用 Python 的 async/await 语法）来增强大语言模型的推理能力。 ```lmql async def wikipedia(q): from lmql.http import fetch try: q = q.strip("\n '.") pages = await fetch(f"https://en.wikipedia.org/w/api.php?format=json&action=query&prop=extracts&exintro&explaintext&redirects=1&titles={q}&origin=*", "query.pages") return list(pages.values())[0]["extract"][:280] except: return "No results" # ask a question "Q: From which countries did the Norse originate?\n" # prepare wikipedia call "Action: Let's search Wikipedia for the term '[TERM]\n" where STOPS_AT(TERM, "'") result = await wikipedia(TERM) # feed back result "Result: {result}\n" # generate final response "Final Answer:[ANSWER]" ``` ```promptdown Q: From which countries did the Norse originate? Action: Let's search Wikipedia for the term 'TERM Norse'. Result: Norse is a demonym for Norsemen, a Medieval North Germanic ethnolinguistic group ancestral to modern Scandinavians, defined as speakers of Old Norse from about the 9th to the 13th centuries. Norse may also refer to: Final Answer: ANSWER The Norse originated from North Germanic countries, including Denmark, Norway, Sweden, and Iceland. ``` ### 键值存储 LMQL 还能访问外围 Python 解释器的状态。为了演示这一点，我们用 assign 和 get 函数实现了一个简单的键值存储，展示如何存取数据。 ```lmql # implement a simple key value storage # with two operations storage = {} def assign(key, value): # store a value storage[key] = value; return f'{{{key}: "{value}"}}' def get(key): # retrieve a value return storage.get(key) # instructive prompt, instructing the model to how use the storage """In your reasoning you can use actions. You do this as follows: `action_name() # result: ` To remember things, you can use 'assign'/'get': - To remember something: `assign("Alice", "banana") # result: "banana"` - To retrieve a stored value: `get("Alice") # result: "banana"` Always tail calls with " # result". Using these actions, let's solve the following question.\n""" # actual problem statement """ Q: Alice, Bob, and Claire are playing a game. At the start of the game, they are each holding a ball: Alice has a black ball, Bob has a brown ball, and Claire has a blue ball. As the game progresses, pairs of players trade balls. First, Bob and Claire swap balls. Then, Alice and Bob swap balls. Finally, Claire and Bob swap balls. At the end of the game, what ball does Alice have? A: Let's think step by step. """ # core reasoning loop for i in range(32): "[REASONING]" where STOPS_AT(REASONING, "# result") and \ STOPS_AT(REASONING, "Therefore, ") if REASONING.endswith("# result"): cmd = REASONING.rsplit("`",1)[-1] cmd = cmd[:-len("# result")] "{eval(cmd)}`\n" else: break # generate final answer "Therefore at the end of the game, Alice has the[OBJECT]" \ where STOPS_AT(OBJECT, ".") and STOPS_AT(OBJECT, ",") ``` ```promptdown Model Output (...) A: Let's think step by step REASONING At the start of the game: `assign('Alice', 'black') # result {Alice: 'black'} REASONING `assign('Bob', 'brown') # result {Bob: 'brown'} REASONING `assign('Claire', 'blue') # result {Claire: 'blue'} REASONING After Bob and Claire swap balls: `assign('Bob', 'blue') # result {Bob: 'blue'} REASONING `assign('Claire', 'brown') # result {Claire: 'brown'} REASONING After Alice and Bob swap balls: `assign('Alice', 'blue') # result {Alice: 'blue'} REASONING `assign('Bob', 'black') # result {Bob: 'black'} REASONING After Claire and Bob swap balls: `assign('Claire', 'black') # result {Claire: 'black'} REASONING `assign('Bob', 'brown') # result {Bob: 'brown'} REASONING At the end of the game, Alice has a blue ball: `get('Alice') # result blue` Therefore at the end of the game, Alice has the OBJECT blue ball. ``` 如上例所示，assign 和 get 函数可以在一个简单的键值存储中存取数据。我们只需在提示词中指示模型在推理过程中使用这些函数，查询程序则负责拦截函数调用并把返回结果插入到推理过程中。这样模型就能把键值存储的状态纳入自己的推理。