OpenAssistant Llama 13Bをgoogle colabで試してみた。

OpenAssistantとは

OpenAssistantは誰でも容易に大規模言語モデルにアクセスし、利用できるようにと開始されたProjectです。
OpenAssistantの目指しているVisionは誰でも容易に拡張することが可能かつPersonalにカスタマイズされたAI Assistantを構築したいというもののようです。
OpenAssistantの計画としては以下の３点があります。

1. 高品質な指示-達成サンプルを収集するためのクラウドソーシングプロセスを設計する。

2. 収集されたデータをランキング化して報酬モデルをトレーニングする。

3. トップ貢献者にSwagを贈呈し、コミュニティをモチベーションづける。

今回はGoogle Colabのfreeで動かせそうなLlama製の13Bモデルの4bitversionをtext-generation-webuiで利用してみたいと思います。

リンク

Colab
github

準備

Google Colabを開き、メニューから「ランタイム→ランタイムのタイプを変更」でランタイムを「GPU」に変更します。

text-generation-webuiは簡単にLLMモデルを色々な形式で試せる優れたOSSです。

環境構築

Installの手順です。

!git clone https://github.com/oobabooga/text-generation-webui
%cd text-generation-webui
!pip install -r requirements.txt
!git clone https://github.com/oobabooga/GPTQ-for-LLaMa.git -b cuda
%cd GPTQ-for-LLaMa
!python setup_cuda.py install
!cp -r /content/text-generation-webui/GPTQ-for-LLaMa/* /content/text-generation-webui/

推論

(1) モデルのダウンロード

%cd /content/text-generation-webui
!python download-model.py 4bit/oasst-llama13b-4bit-128g

(2)　WebUIの起動

%cd /content/text-generation-webui
!python3 server.py --model 4bit_oasst-llama13b-4bit-128g --wbits 4 --group 128 --chat --share --auto-devices  --model_type=llama

起動するとコンソール上にRunning on public URL: https://*****.gradio.live と出るのでそこをクリックするとText generation WebUIのページに遷移します。

推論実行後の画面です。8GBくらいで動きます。

(1)1LDK room planner
input

I want you act as a professional room creator. Please suggest some plan for comfortable room with 1LDK.

OpenAssistant LLama 13B

Sure! Here are some suggestions for creating a comfortable living space in a 1-Bedroom, 1-Living Room, 1-Dining Area (1LDK) apartment layout:

For the bedroom:

1. Place the bed against one wall, leaving enough walking space around it on all sides.
2. Install curtains or blinds that can be closed at night to block out light and create a more restful environment.
3. Consider adding decorative elements such as lamps, plants, or artwork to make the space feel cozier.
For the living area:

4. Position furniture so there's plenty of room to move around and socialize comfortably.
Include seating options like a couch, armchair, and/or stools to accommodate different types of activities (e.g., watching TV, reading, having conversations).
5. Add decor items such as potted plants, throw pillows, and other accessories to enhance the overall look and feel of the space.
For the dining area:

6. Choose chairs that are comfortable and supportive for long meals and gatherings.
7. Select tableware and serving pieces that fit your style and functional needs (e.g., plates, bowls, glasses, utensils).
Consider incorporating design elements such as table runners, centerpieces, and lighting to create a welcoming atmosphere.

chatGPT

Certainly, I'd be happy to help! Here are some suggestions for a comfortable 1LDK room:

1. First, think about the layout. For a 1LDK room, you'll want to maximize the use of the space. Consider an open-plan living room and kitchen, with a separate bedroom and bathroom.
2. Choose comfortable furniture that is appropriately sized for the space. A small sofa or sectional, a coffee table, and a TV stand can work well in the living room, while a queen or full-sized bed, a dresser, and a nightstand can fit in the bedroom. You may also want to consider storage solutions, such as a bookshelf or storage ottoman.
3. Use lighting to create a cozy atmosphere. Consider a mix of overhead lighting, table lamps, and floor lamps to provide a variety of lighting options for different moods and activities.
4. Choose a color scheme that is calming and cohesive. Soft, neutral colors like beige, gray, and white can create a serene environment, while pops of color in pillows, rugs, or artwork can add interest and personality.
5. Don't forget about the bathroom! Choose soft, fluffy towels, a shower curtain or screen, and a bath mat to make the space feel luxurious and comfortable.
I hope these suggestions are helpful in creating a comfortable 1LDK room.

遜色ない感じですね。すごくいい感じ

(2) Instruct
タスクプランニング見たいなことが出来るかを試してみましょう。
(2-1) Few shot learningのケース
input

Please do the classification task like the following examples.

Question: Can you tell me about the detail of this image? Answer: object detection+semantic segmentation+image caption

Question: Please tell me about the number of objects in this image. Answer: object detection

Question: can you extract the transcript from this audio file? Answer: speech to text + translation

Question: Please translate this sentence into English. Answer: translation

Question: Can you tell me about this video in detail? Answer:

OpenAssistant

Yes, here is some information on that image:

The image contains three objects: a car, a person and a building. The car is red and has four doors. The person is wearing a blue shirt and khaki pants. The building is white with two windows and a door. It also appears to be located near a tree and a street sign.

うーん、そういう感じのデータセットは含まれていなさそうですね。。

(2-2) zeroshot learningのケース
input

Please create task plan with AI model according to the following question. Question: Can you describe this image? Answer:

output

Sure! Here’s an example task plan for describing the image using an AI model:

Use object detection to identify all the objects present in the image (car, person, building).
For each detected object, use semantic segmentation to classify it as either background or foreground. This will help us determine which parts of the image are important for description.
Based on the importance of different regions, generate a hierarchical representation of the scene using graph neural networks. This will allow us to organize the information in a structured way.
Generate a descriptive caption by combining the region-specific features learned during step 3 with pretrained language models. We can fine-tune these models to improve their performance based on human feedback.

結構流暢な感じで返答してくるモデルですね。返答の柔軟性はchatGPTに比べて低そうです。

ちなみにキーワード抽出をさせると
input

Please extract the keyword from the above sentence

output

Extracting keywords from the above sentence could look something like this:

Object Detection: car, person, building

Semantic Segmentation: car - foreground, person - foreground, building - foreground

Graph Neural Networks: hierarchy, structure, organization

Pretrained Language Models: captions, descriptions

やっぱりズバッとは答えてくれない（笑）

最後に

今回はOpenAssistantのデータセットをLLama13Bでfinetuningされたモデルをtext-generation-webuiで利用してみました。SuggestionとかDraftを書くとかに関してはchatGPTと同等かそれ以上の性能があると思われます。ただQuestion AnsweringやText classificationといったタスクに関してはfinetuningされていないようでうまくできないようでした。
PDF文書とか食わせた時にどういう風な応答ができるかとか興味あるのでこちらもいつか検証してみたいですね。
またLLMに関して色々なモデルを試してきたのでそのモデルの実行google colabと感想まとめみたいなのも近々更新しますね。

(注)30Bの4bitもありましたがgoogle colab freeでは動かなそうでした。おうちに24GBくらいのGPUがある方は以下をお試しください。

MetaIX/OpenAssistant-Llama-30b-4bit · Hugging Face

今後ともLLM, Diffusion model, Image Analysis, 3Dに関連する試した記事を投稿していく予定なのでよろしくお願いします。