Falcon-7B で LangChain を試す

2023年7月11日 11:53

「Falcon-7B」で「LangChain」を試したのでまとめました。

1. Falcon

「Falcon」は、「Technology Innovation Institute」が開発したオープンソースのLLMです。商用利用可能なライセンスで公開されています。「Open LLM Leaderboard」でLlama系のモデルを抜いて1位を獲得したことで話題になりました。

2. Colabでの実行

Google Colab での実行手順は、次のとおりです。

(1) パッケージのインストール。

# パッケージのインストール
!pip install transformers einops accelerate langchain bitsandbytes

(2) トークナイザーとパイプラインの準備。

from langchain import HuggingFacePipeline
from transformers import AutoTokenizer, pipeline
import torch

# トークナイザーとパイプラインの準備
tokenizer = AutoTokenizer.from_pretrained(
    "tiiuae/falcon-7b-instruct"
)
pipeline = pipeline(
    "text-generation",
    model="tiiuae/falcon-7b-instruct",
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto",
    max_length=512,
    do_sample=True,
    top_k=10,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id
)
llm = HuggingFacePipeline(
    pipeline = pipeline, 
    model_kwargs = {"temperature": 0}
)

(3) プロンプトテンプレートとLLMChainの準備。

from langchain import PromptTemplate, LLMChain

# プロンプトテンプレートの準備
template = """
You are an intelligent chatbot. Help the following question with brilliant answers.
Question: {question}
Answer:"""
prompt = PromptTemplate(template=template, input_variables=["question"])

# LLMChainの準備
llm_chain = LLMChain(prompt=prompt, llm=llm)

(4) プロンプトの準備と推論の実行。

# プロンプトの準備
question = """Read the following context and answer the question.

Context : Goto Hitori is a girl who attends Shuka High School. She has loose pink hair and she always wears a jersey. She admits that she is one step closer to being withdrawn than others, and despite her strong desire for approval, she has a timid personality and is extremely bad at dealing with people. She therefore shows her emotional instability as she quickly enters her own world and becomes depressed. She has a habit of burrowing into closets and cardboard boxes, and she carries around a "Kanjyuku Mango Kamen". She is not good at sports and studying, and despite her taking classes seriously, she barely gets a red mark due to her innate lack of skill. She was keenly aware that she had nothing to offer, so when she was in middle school, she read an interview that a person with a dark personality became popular by playing in a band, and she started playing the guitar. Thanks to her daily practice, her guitar skills have become professional grade, but in the end she was not blessed with a place to show off her skills, and she graduated from junior high school without being able to make friends. She currently distributes videos under the name of "Guitar Hero". When she was looking for a guitarist, she was invited by Kika Ijichi, and she joined the "Kessoku Band" and started her own band activities. She is the guitarist and lyricist for the band. Because she is shy, she can't even go to the hairdresser, and she always hides her eyes with her bangs because she grows her hair as much as she wants. In addition to that, she is not noticed because she is wearing an unfashionable jersey, but she is actually a beautiful girl who can be admired by people of the same sex. She has the potential to be said to be "in charge of the idol office" and "in charge of visuals" if she is quietly dressed up, but her face only lasts for 10 seconds, and her usual eccentric behavior ruining everything.

Question: What is the name of the band Hitori Goto joins?"""

# 推論の実行
print(llm_chain.run(question))

The name of the band is Kessoku Band

Q: 後藤ひとりさんが加入しているバンドの名前は何ですか？
A: バンドの名前は結束バンドです

同様にいくつか質問してみます。

Question: What kind of person is Hitori Goto?

Hitori Goto is a shy, intelligent, and talented girl who is good at playing music. She has a strong desire to be accepted by others and shows emotional instability. She is a skilled guitarist and lyricist of a rock band. Her

Q: 後藤ひとりってどんな人？
A: 後藤ひとりは内気で頭が良く、音楽も得意な天才少女。他人に認められたいという気持ちが強く、情緒不安定なところがあります。彼女はロックバンドの熟練したギタリスト兼作詞家です。彼女は

Question: What is "Kanjyuku Mango Kamen"?

"Kanjyuku Mango Kamen" is a fictional band that Hitori has a crush on, and she often daydreams about their reunion.

Q: 「完熟マンゴー仮面」とは何ですか？
A: 「完熟マンゴー仮面」はひとりが憧れる架空のバンドで、よく再会を夢見ている。

これは間違いですが、コンテキスト確認したら『「完熟マンゴー仮面」を持ち歩いている。』しか書いてなく、完熟マンゴー仮面は何なのか想像で書いたみたい。

Falcon-7B で LangChain を試す

1. Falcon

2. Colabでの実行

関連