【DLAI×LangChain講座⑥】 Agents

2023年7月19日 08:39

背景

LangChainは気になってはいましたが、複雑そうとか、少し触ったときに日本語が出なかったりで、後回しにしていました。

DeepLearning.aiでLangChainの講座が公開されていたので、少し前に受講してみました。その内容をまとめています。

ちなみに：ゆっくりやっていたら、DLAI×LangChain講座の2が公開されてしまいました！早速受講はしてみたのですが、なかなかグレードアップしていて感動しました。急いでいこうと思います。（けど何かおまけはつけたい。。）

第5回はこちらです。

今回は第6回Agentsについてです。ここでは、ツールの使用をLLMに計画・実行させるAgentsを紹介しています。こんだけできるなら、これもやってや、みたいな感じですね。ただ、ツールをこっちが作って使わせるのは、少し難しい印象です。少し前ではGPT-4のマイクラAgent、最近ではCode interpreterなどが話題だったように、ツールまで作らせるほうがよさそうな気がします。そうすると、新しいPythonの機能などを使えるようにするため、学習データの更新をどうするかが重要ですね。

Agents | 🦜️🔗 Langchain

アプローチ

DeepLearning.aiのLangChain講座の6の内容をまとめます。

サンプル

今回は、LangChainに内蔵されたToolsと、カスタムToolの作り方を紹介します。

import os
from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv())
import warnings
warnings.filterwarnings("ignore")

Twitterで見ましたが、この環境変数を使う方法はセキュリティ的によくないらしいですね。ログ取る際に、環境変数もまとめて送信するOSSが結構あるんだとか。でもそのツイートに、「やろうと思えば、OSSはConfigファイルでもなんでも読める」みたいなリプもあり、、難しいですね。今度少し調べてみるか。

まずは、LangChainに内蔵されたToolとして、LLM-math、Wikipedia、PythonREPLを紹介します。

Built-in LangChain tools

Wikipediaをインストールします。Wikipedia検索APIのラッパーライブラリでしょうか。

# !pip install -U wikipedia

from langchain.agents.agent_toolkits import create_python_agent
from langchain.agents import load_tools, initialize_agent
from langchain.agents import AgentType
from langchain.tools.python.tool import PythonREPLTool
from langchain.python import PythonREPL
from langchain.chat_models import ChatOpenAI

Agentに必要なモジュールを構成します。ChatOpenAIとToolsを以下のコードで作ります。Built-in Toolsは、load_toolsで以下のように簡単に呼び出せます。

llm = ChatOpenAI(temperature=0)
tools = load_tools(["llm-math","wikipedia"], llm=llm)

initialize_agentでAgentを構成します。今回は、AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTIONというZero-shotでReActを行うAgentですが、他にも様々なAgentが選べます。

Agent types | 🦜️🔗 Langchain

agent= initialize_agent(
    tools, 
    llm, 
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    handle_parsing_errors=True,
    verbose = True
)

これで、計算とWikipedia検索ができるAgentができました。早速計算をさせてみます。

agent("What is the 25% of 300?")

> Entering new  chain...
I can use the calculator tool to find the answer to this question.

Action:
```json
{
  "action": "Calculator",
  "action_input": "25% * 300"
}
\```
Observation: Answer: 75.0
Thought:The calculator tool returned the answer 75.0.
Final Answer: 75.0

> Finished chain.

{'input': 'What is the 25% of 300?', 'output': '75.0'}

計算ができていますね。```json```部分をパースし、Toolとその入力を得る方式だと思うので、ここでパースが失敗すると、全体が失敗する、ということでしょうか。

次は、Wikipediaの情報を聞いてみます。Tom M. Mitchell という、アメリカのコンピュータサイエンティストが書いたタイトルについて聞いています。

question = "Tom M. Mitchell is an American computer scientist \
and the Founders University Professor at Carnegie Mellon University (CMU)\
what book did he write?"
result = agent(question)

> Entering new  chain...
Thought: I can use Wikipedia to find out what book Tom M. Mitchell wrote.
Action:
```json
{
  "action": "Wikipedia",
  "action_input": "Tom M. Mitchell"
}
```
Observation: Page: Tom M. Mitchell
Summary: Tom Michael Mitchell (born August 9, 1951) is an American computer scientist and the Founders University Professor at Carnegie Mellon University (CMU). He is a founder and former Chair of the Machine Learning Department at CMU. Mitchell is known for his contributions to the advancement of machine learning, artificial intelligence, and cognitive neuroscience and is the author of the textbook Machine Learning. He is a member of the United States National Academy of Engineering since 2010. He is also a Fellow of the American Academy of Arts and Sciences, the American Association for the Advancement of Science and a Fellow and past President of the Association for the Advancement of Artificial Intelligence. In October 2018, Mitchell was appointed as the Interim Dean of the School of Computer Science at Carnegie Mellon.

Page: Tom Mitchell (Australian footballer)
Summary: Thomas Mitchell (born 31 May 1993) is a professional Australian rules footballer playing for the Collingwood Football Club in the Australian Football League (AFL). He previously played for the Sydney Swans from 2012 to 2016, and the Hawthorn Football Club between 2017 and 2022. Mitchell won the Brownlow Medal as the league's best and fairest player in 2018 and set the record for the most disposals in a VFL/AFL match, accruing 54 in a game against Collingwood during that season.[0m
Thought: The book that Tom M. Mitchell wrote is "Machine Learning".
Final Answer: Machine Learning

> Finished chain.

2人のTom MitchellについてのWikipediaの要約をもとに、正しいほうのTom Mitchellを選んで回答していますね。Wikipediaをベースとするのは一つありな気がしますね。検索で見つけたページよりも信頼度のばらつきが少ない気がします。

Python Agent

次に、Pythonを扱うAgentです。以下のように、create_python_agent、PythonREPLToolを使うことで簡単に構成できます。でもこのcreate_python_agent、PythonREPLTool以外を受け付けることはあるんですかね？中身を見た感じ、普通にAgent作ってるだけに見えるんですが。よくわからないので一旦置いときます。

agent = create_python_agent(
    llm,
    tool=PythonREPLTool(),
    agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True
)

カスタマーリストをLast name→First nameの順にソートし、printします。

> Entering new  chain...

Invoking: `Python_REPL` with `customers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']] 
sorted_customers = sorted(customers, key=lambda x: (x[1], x[0])) 
sorted_customers`


[['Jen', 'Ayai'], ['Harrison', 'Chase'], ['Lang', 'Chain'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Dolly', 'Too']]

> Finished chain.
"[['Jen', 'Ayai'], ['Harrison', 'Chase'], ['Lang', 'Chain'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Dolly', 'Too']]"

あれ、間違ってる？↓確認すると、やっぱり違いますね。

print(sorted([['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']], key=lambda x: (x[1], x[0])))

[['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]

「Observation: 」に何も入ってないのが問題だと思うんですが、LLMだけでそれっぽい回答が出せるので、一瞬できてると思っちゃいました。危ないですね。

調べてみると、Python REPLでは、print()された内容のみObservationに入るようです。

class PythonREPL(BaseModel):
    """Simulates a standalone Python REPL."""

    globals: Optional[Dict] = Field(default_factory=dict, alias="_globals")
    locals: Optional[Dict] = Field(default_factory=dict, alias="_locals")

    def run(self, command: str) -> str:
        """Run command with own globals/locals and returns anything printed."""
        old_stdout = sys.stdout
        sys.stdout = mystdout = StringIO()
        try:
            exec(command, self.globals, self.locals)
            sys.stdout = old_stdout
            output = mystdout.getvalue()
        except Exception as e:
            sys.stdout = old_stdout
            output = repr(e)
        return output

langchain > utilities > python.py

そのため、前回まで同様、なんとかprint()させようとプロンプトを工夫してみましたが、できませんでした。。ということで苦肉の策のGPT-4。

llm = ChatOpenAI(temperature=0,model='gpt-4')
agent = create_python_agent(
    llm,
    tool=PythonREPLTool(),
    agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True
)

customer_list = [
    ["Harrison", "Chase"], 
    ["Lang", "Chain"],
    ["Dolly", "Too"],
    ["Elle", "Elem"], 
    ["Geoff","Fusion"], 
    ["Trance","Former"],
    ["Jen","Ayai"]
]

agent.run(f"""Sort these customers by \
last name and then first name \
and print the output: {customer_list}""")

> Entering new  chain...
Invoking: `Python_REPL` with `customers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]
sorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))
print(sorted_customers)`


[['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]
The customers sorted by last name and then first name are: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']].

> Finished chain.

"The customers sorted by last name and then first name are: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]."

これは、性能の違いによるところなんでしょうか？プロンプトは結局いじってないですし、print()しろって指示を強くしてもいないんですが。。けど、GPT-3.5で動かないものがGPT-4を使うと動く、みたいなパターンは多いです。

ちなみにですが、今回のカスタマーリストだと、Last nameが全部違うので、Last nameのソートだけで終わります。↓やっぱりそうですよね。まあカスタマーリストに同じLast nameがあれば、みたいなことですね。

customer_list = [
    ["Harrison", "Chase"], 
    ["Lang", "Chain"],
    ["Dolly", "Too"],
    ["Elle", "Elem"], 
    ["Geoff","Fusion"], 
    ["Trance","Former"],
    ["Jen","Ayai"]
]
print(sorted(customer_list, key=lambda x: x[1]))
print(sorted(customer_list, key=lambda x: (x[1], x[0])))

[['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]
[['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]

View detailed outputs of the chains

次に、langchain.debug=Trueとして、中の処理を詳しく見ていきます。

import langchain
langchain.debug=True
agent.run(f"""Sort these customers by \
last name and then first name \
and print the output: {customer_list}""") 
langchain.debug=False

↓長い。。よく読んで見ると、どんなプロンプトでPythonを使っているかなどが確認できます。

[chain/start] [1:chain:AgentExecutor] Entering Chain run with input:
{
  "input": "Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]"
}
[chain/start] [1:chain:AgentExecutor > 2:chain:LLMChain] Entering Chain run with input:
{
  "input": "Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]",
  "agent_scratchpad": "",
  "stop": [
    "\nObservation:",
    "\n\tObservation:"
  ]
}
[llm/start] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] Entering LLM run with input:
{
  "prompts": [
    "Human: You are an agent designed to write and execute python code to answer questions.\nYou have access to a python REPL, which you can use to execute python code.\nIf you get an error, debug your code and try again.\nOnly use the output of your code to answer the question. \nYou might know the answer without running any code, but you should still run the code to get the answer.\nIf it does not seem like you can write code to answer the question, just return \"I don't know\" as the answer.\n\n\nPython_REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [Python_REPL]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nThought:"
  ]
}
[llm/end] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] [13.08s] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "I need to sort the list of customers first by their last name and then by their first name. I can use the built-in sorted function in Python, which allows me to specify a key function to determine the sort order. In this case, I will use a lambda function that returns a tuple of the last name and first name for each customer. This will sort the customers first by last name, and then by first name within each group of customers with the same last name. I will then print the sorted list. \nAction: Python_REPL\nAction Input: \ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\nprint(sorted_customers)",
        "generation_info": {
          "finish_reason": "stop"
        },
        "message": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "schema",
            "messages",
            "AIMessage"
          ],
          "kwargs": {
            "content": "I need to sort the list of customers first by their last name and then by their first name. I can use the built-in sorted function in Python, which allows me to specify a key function to determine the sort order. In this case, I will use a lambda function that returns a tuple of the last name and first name for each customer. This will sort the customers first by last name, and then by first name within each group of customers with the same last name. I will then print the sorted list. \nAction: Python_REPL\nAction Input: \ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\nprint(sorted_customers)",
            "additional_kwargs": {}
          }
        }
      }
    ]
  ],
  "llm_output": {
    "token_usage": {
      "prompt_tokens": 328,
      "completion_tokens": 194,
      "total_tokens": 522
    },
    "model_name": "gpt-4"
  },
  "run": null
}
[chain/end] [1:chain:AgentExecutor > 2:chain:LLMChain] [13.08s] Exiting Chain run with output:
{
  "text": "I need to sort the list of customers first by their last name and then by their first name. I can use the built-in sorted function in Python, which allows me to specify a key function to determine the sort order. In this case, I will use a lambda function that returns a tuple of the last name and first name for each customer. This will sort the customers first by last name, and then by first name within each group of customers with the same last name. I will then print the sorted list. \nAction: Python_REPL\nAction Input: \ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\nprint(sorted_customers)"
}
[tool/start] [1:chain:AgentExecutor > 4:tool:Python_REPL] Entering Tool run with input:
"customers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]
sorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))
print(sorted_customers)"
[tool/end] [1:chain:AgentExecutor > 4:tool:Python_REPL] [0.763ms] Exiting Tool run with output:
"[['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]"
[chain/start] [1:chain:AgentExecutor > 5:chain:LLMChain] Entering Chain run with input:
{
  "input": "Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]",
  "agent_scratchpad": "I need to sort the list of customers first by their last name and then by their first name. I can use the built-in sorted function in Python, which allows me to specify a key function to determine the sort order. In this case, I will use a lambda function that returns a tuple of the last name and first name for each customer. This will sort the customers first by last name, and then by first name within each group of customers with the same last name. I will then print the sorted list. \nAction: Python_REPL\nAction Input: \ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\nprint(sorted_customers)\nObservation: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]\n\nThought:",
  "stop": [
    "\nObservation:",
    "\n\tObservation:"
  ]
}
[llm/start] [1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] Entering LLM run with input:
{
  "prompts": [
    "Human: You are an agent designed to write and execute python code to answer questions.\nYou have access to a python REPL, which you can use to execute python code.\nIf you get an error, debug your code and try again.\nOnly use the output of your code to answer the question. \nYou might know the answer without running any code, but you should still run the code to get the answer.\nIf it does not seem like you can write code to answer the question, just return \"I don't know\" as the answer.\n\n\nPython_REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [Python_REPL]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nThought:I need to sort the list of customers first by their last name and then by their first name. I can use the built-in sorted function in Python, which allows me to specify a key function to determine the sort order. In this case, I will use a lambda function that returns a tuple of the last name and first name for each customer. This will sort the customers first by last name, and then by first name within each group of customers with the same last name. I will then print the sorted list. \nAction: Python_REPL\nAction Input: \ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\nprint(sorted_customers)\nObservation: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]\n\nThought:"
  ]
}
[llm/end] [1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] [6.87s] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "I now know the final answer\nFinal Answer: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]",
        "generation_info": {
          "finish_reason": "stop"
        },
        "message": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "schema",
            "messages",
            "AIMessage"
          ],
          "kwargs": {
            "content": "I now know the final answer\nFinal Answer: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]",
            "additional_kwargs": {}
          }
        }
      }
    ]
  ],
  "llm_output": {
    "token_usage": {
      "prompt_tokens": 578,
      "completion_tokens": 61,
      "total_tokens": 639
    },
    "model_name": "gpt-4"
  },
  "run": null
}
[chain/end] [1:chain:AgentExecutor > 5:chain:LLMChain] [6.87s] Exiting Chain run with output:
{
  "text": "I now know the final answer\nFinal Answer: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]"
}
[chain/end] [1:chain:AgentExecutor] [19.95s] Exiting Chain run with output:
{
  "output": "[['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]"
}

2回のAPIコールで実現されているようですね。まずは、あなたはPythonですよという指示＋問題を送り、アクションを決定させています。次に、Pythonコードの実行結果を送り、最終回答としてよいかを確認しているようです。

Define your own tool

最後に、自作ツールの使い方を紹介します。

from langchain.agents import tool
from datetime import date

@toolで関数をツール化できます。docstringで使い方を書くだけです。これがそのまま、ツールの説明としてLLMに渡されます。

@tool
def time(text: str) -> str:
    """Returns todays date, use this for any \
    questions related to knowing todays date. \
    The input should always be an empty string, \
    and this function will always return todays \
    date - any date mathmatics should occur \
    outside this function."""
    return str(date.today())

agent= initialize_agent(
    tools + [time], 
    llm, 
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    handle_parsing_errors=True,
    verbose = True)

Note:
The agent will sometimes come to the wrong conclusion (agents are a work in progress!).
If it does, please try running it again.

まだ安定しません。ということですね。

try:
    result = agent("whats the date today?") 
except: 
    print("exception on external access")

> Entering new  chain...
Question: What's the date today?
Thought: I can use the `time` tool to get the current date.
Action:
\```
{
  "action": "time",
  "action_input": ""
}
\```
Observation: 2023-07-19
Thought: I now know the final answer.
Final Answer: The date today is 2023-07-19.

> Finished chain.

うまく動作しています。入力なしでtime関数を呼び出し、今日の日付を返してくれています。

print(result)

結果は以下の形で返されます。

{'input': 'whats the date today?', 'output': 'The date today is 2023-07-19.'}

おまけ

今回のおまけでは、LangChainでのFunction calling機能の利用方法を紹介します。まずは、openaiのFunction callingの使い方復習です。

Function calling sample

import json,openai,os
from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv())
openai.api_key = os.environ['OPENAI_API_KEY']

def get_current_weather(location, unit="fahrenheit"):
    weather_info = {
        "location": location,
        "temperature": "72",
        "unit": unit,
        "forecast": ["sunny", "windy"],
    }
    return json.dumps(weather_info)

def run_conversation(input):
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo-0613",
        messages=[{"role": "user", "content": input}],
        functions=[
            {
                "name": "get_current_weather",
                "description": "指定した場所の現在の天気を取得",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "都市と州",
                        },
                        "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                    },
                    "required": ["location"],
                },
            }
        ],
        function_call="auto",
    )
    message = response["choices"][0]["message"]
    print("message>>>\n", message, "\n\n")

    if message.get("function_call"):
        function_name = message["function_call"]["name"]

        function_response = get_current_weather(
            location=message.get("location"),
            unit=message.get("unit"),
        )

        second_response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo-0613",
            messages=[
                {"role": "user", "content": input},
                message,
                {
                    "role": "function",
                    "name": function_name,
                    "content": function_response,
                },
            ],
        )
        return second_response

print("response>>>\n", run_conversation("ボストンの天気はどうですか？")["choices"][0]["message"]["content"], "\n\n")

message>>> { "role": "assistant", "content": null, "function_call": { "name": "get_current_weather", "arguments": "{\n \"location\": \"Boston\"\n}" } } response>>> ボストンの現在の天気は晴れで、風が強いです。気温は72°Fです。

天気を返すダミー関数を用意し、天気を聞いています。

Function calling in LangChain

次に、LangChainの場合です。

from langchain.chat_models import ChatOpenAI
from langchain.agents import AgentType, initialize_agent, Tool

先ほどと全く同じ関数を、Toolとして構成します。AgentType.OPENAI_FUNCTIONSとすることで、Function callingを使ってくれるようになります。

# 同じ関数
def get_current_weather(location, unit="fahrenheit"):
    weather_info = {
        "location": location,
        "temperature": "72",
        "unit": unit,
        "forecast": ["sunny", "windy"],
    }
    return json.dumps(weather_info)

tools = [
    Tool(
        name='get_current_weather',
        func=get_current_weather,
        description='Get the current weather in a given location'
    )
]

chat = ChatOpenAI(temperature=0.)
agent = initialize_agent(
    tools,chat,
    agent=AgentType.OPENAI_FUNCTIONS,
    verbose=True
)
result = agent.run("ボストンの天気はどうですか？")
print(result)

> Entering new  chain...
Invoking: `get_current_weather` with `Boston`

{"location": "Boston", "temperature": "72", "unit": "fahrenheit", "forecast": ["sunny", "windy"]}ボストンの現在の天気は72°Fで、晴れで風が強いです。

> Finished chain.
ボストンの現在の天気は72°Fで、晴れで風が強いです。

同じ回答が得られました。LangChainを使うことで簡単に実装できるようになっていますね。

しかし、Function calling機能は、LangChainのすべてのOutput Parserを置き換えるものです。LangChainの出力パースを全て置き換えると、LangChain全体の精度向上にもっと役立つはずです。

ただし、課題もあります。Function callingは構造化しかしないイメージです。一回のコールで推論と出力パースの両方をできないことです。例えば、「天気はどうですか？」と聞くと、勝手にlocationを東京としてしまいます。↓「どこですか？」と聞き返してほしいところです。同様に、食材の栄養を聞くと、全てNoneで返ってきたりします。

> Entering new  chain...

Invoking: `get_current_weather` with `{'location': '東京'}`


{"location": "\u6771\u4eac", "temperature": "72", "unit": "fahrenheit", "forecast": ["sunny", "windy"]}東京の天気は晴れで、風が強いです。現在の気温は72°Fです。

> Finished chain.
東京の天気は晴れで、風が強いです。現在の気温は72°Fです。

それはつまり、データ構造化のためにAPIコールが一回増えるということです。料金的には少しの問題ですが、全体の応答時間には大きく響きます。おそらくですが、ツールを入力らしきものが来たらツール利用の指定形式で返すようFine-tuningされたせいで、問答無用でそうするようになってしまっているのだと思います。これも、ReAct的な方法ならうまくいくのかなど、色々検証が必要な気がします。

以上、LangChainでのFunction calling機能の利用でした。

まとめ

DeepLearning.aiのLangChain講座の6の内容をまとめました。これで、LangChain講座の一つ目は全て終わりです。

今回は、Agentsについてでした。結構話題になってましたけど、gpt-3.5-turboでは難しい部分がありますね。しかし、GPT-4をほいほい使うとコスト高すぎる。悩ましいですね。リトライで使うとか、バックエンドの整備やコンテンツ生成に使うなど、価値の高い部分に当てられるような使い分けができるようにしたいですね。

Toolを使う部分は、Function callingとかぶってるなあという機能ですが、LangChainもすかさずFunction callingを取り込んでいます。ただ、今リリースされているAgentTypeとしてのOutput ParserをFunction callingに置き換え・統合が、全体の精度アップにつながると思うので、重要そうです。あまり開発は追えていないですが、進んでいるんでしょうかね。

読んでいただきありがとうございます。DLAI×LangChain2は重めですが、がんばってまとめていきたいです。

参考

【DLAI×LangChain講座⑤】 Evaluation｜harukary
Agents | 🦜️🔗 Langchain
Agent types | 🦜️🔗 Langchain

サンプルコード

https://github.com/harukary/llm_samples/blob/main/LangChain/LangChain_DLAI_1/L6-Agents.ipynb
https://github.com/harukary/llm_samples/blob/main/LangChain/LangChain_DLAI_1/plus_L6.ipynb

この記事が気に入ったらサポートをしてみませんか？