Macでllama2を最速で試す方法

llama.cppをビルドして、モデルをダウンロードしてコマンドラインで動かすまでの私的に最速の手順です。
（テスト環境：Mac book pro M1)

llama.cppをビルドする

# リポジトリからソースをダウンロードします
git clone https://github.com/ggerganov/llama.cpp.git

# ディレクトリに移動します
cd llama.cpp 

# ソースコードをビルドします
LLAMA_METAL=1 make

モデルをダウンロードする

# modelsにモデルをダウンロードするために移動します
cd models

# llama.cppに対応するGGML版を今回は使用します。
wget "https://huggingface.co/TheBloke/Llama-2-7B-chat-GGML/resolve/main/llama-2-7b-chat.ggmlv3.q4_0.bin"

試してみる

./main \
        -t 8 \
        -ngl 32 \
        -m models/llama-2-7b-chat.ggmlv3.q4_0.bin \
        --color \
        -c 2048 \
        --temp 0.7 \
        --interactive-first

上記のコマンドを入力したら質問してみましょう。
インタラクティブモードで起動していますので、以下の使い方を参考に質問を打ってみましょう

== Running in interactive mode. ==
Press Ctrl+C to interject at any time.
Press Return to return control to LLaMa.
To return control without starting a new line, end your input with '/'.
If you want to submit another line, end your input with ''.

ログ

日本一高い山の名前はなんでしょう？
 surely you can tell me the names of some famous mountains in Japan! Here are a few:

1. Mount Fuji - This is probably the most well-known mountain in Japan and is located on the island of Honshu. It's a dormant volcano that stands at 3,776 meters (12,421 feet) tall.
2. Mount Yari - Located in the Japanese Alps, this mountain stands at 3,180 meters (10,433 feet) tall and is known for its rugged terrain.
3. Mount Kita - Also located in the Japanese Alps, this mountain stands at 2,956 meters (9,731 feet) tall and is a popular destination for hiking and skiing.
4. Mount Tateyama - Located in the Chubu region of Japan, this mountain stands at 3,025 meters (9,900 feet) tall and is known for its stunning scenery, including waterfalls and hot springs.
5. Mount Zao - Located in the Yamagata prefecture, this mountain stands at 2,460 meters (8,100 feet) tall and is known for its unique snow formations that resemble a giant snowman.

These are just a few examples of the many famous mountains in Japan. Do you have any other questions about them?