llama2.mojoでもじょもじょする

noguchi-shoji

2023年9月13日 01:43

Mojoを使えるようにする

Mojo SDKのインストール

とりあえず、SDKを得るためにサインアップ。

サインアップすると、セットアップ手順が示される。

ここではWindows OSでWSL2がセットアップされているものとして進める。modular経由でMojo SDKをインストールする。

$ curl https://get.modular.com | \
  MODULAR_AUTH=mut_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX \
  sh -

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3176  100  3176    0     0  13233      0 --:--:-- --:--:-- --:--:-- 13233
Executing the  setup script for the 'modular/installer' repository ...

   OK: Checking for required executable 'curl' ...
   OK: Checking for required executable 'apt-get' ...
   OK: Detecting your OS distribution and release using system methods ...
 ^^^^: ... Detected/provided for your OS/distribution, version and architecture:
 >>>>:
 >>>>: ... distro=ubuntu  version=20.04  codename=focal  arch=x86_64
 >>>>:
   OK: Checking for apt dependency 'apt-transport-https' ...
   OK: Checking for apt dependency 'ca-certificates' ...
   OK: Checking for apt dependency 'gnupg' ...
   OK: Checking for apt signed-by key support ...
   OK: Importing 'modular/installer' repository GPG keys ...
   OK: Checking if upstream install config is OK ...
   OK: Installing 'modular/installer' repository via apt ...
   OK: Updating apt repository metadata cache ...
   OK: The repository has been installed successfully - You're ready to rock!

$ modular install mojo

Mojo CLIのセットアップ

Mojo CLIを使用できるようにするために、~/.bashrc に環境変数が必要となる。こちらは
・Modular Docs - Hello, world!
を参考に設定する。コマンドは以下。

$ echo 'export MODULAR_HOME="$HOME/.modular"' >> ~/.bashrc
$ echo 'export PATH="$/MODULAR_HOME/pkg/packages.modular.com_mojo/bin:$PATH"' >> ~/.bashrc
$ source ~/.bashrc

llama2.mojoを試してみる

llama2.mojoをclone

GitHub - tairov/llama2.mojo: Inference Llama 2 in one file of pure 🔥
こちらの手順に従い、git cloneする。

$ git clone https://github.com/tairov/llama2.mojo.git
$ cd llama2.mojo

15Mモデルを試す

ダウンロードして、

$ wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin

実行。

$ mojo llama2.mojo
num hardware threads:  12  SIMD vector width:  8
checkpoint size:  60816028
<s>
Once upon a time, there was a little girl named Lily. She loved to play outside in the sunshine. One day, she saw a big, red ball in the sky. It was the sun! She thought it was so pretty.
Lily wanted to play with the ball, but it was too high up in the sky. She tried to jump and reach it, but she couldn't. Then, she had an idea. She would use a stick to knock the ball down.
Lily found a stick and tried to hit the ball. But the stick was too short. She tried again and again, but she couldn't reach it. She felt sad.
Suddenly, a kind man came by and saw Lily. He asked her what was wrong. Lily told him about the ball. The man smiled and said, "I have a useful idea!" He took out a long stick and used it to knock the ball down. Lily was so happy! She thanked the man and they played together in the sunshine.
<s>
Once upon a time, there was a little girl named Lily. She loved to play outside in the sunshine. One day, she saw a big, red
achieved tok/s:  195.40229885057474

をー、できた。GPUも何も積んでいないPCなのだが良い感じの速さである。なお、CPUモデルは以下。

$ cat /proc/cpuinfo  | grep -e '^model name' | | uniq -c
     12 model name  : 12th Gen Intel(R) Core(TM) i7-1255U

110Mモデルを試す

110Mのモデルもあるようなので、そちらも試してみる。

$ wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories110M.bin

llama2.mojoファイルのL.689-690をコメントアウトを入れ替える。

    # let checkpoint = "stories15M.bin"   <--- こっちを#にして、
    let checkpoint = "stories110M.bin"    <--- トル#

でもって実行する。

$ mojo llama2.mojo
num hardware threads:  12  SIMD vector width:  8
checkpoint size:  438381596
<s>
Once upon a time, there was a little girl named Lily. She loved to play outside in the sunshine. One day, she saw a big, red apple on a tree. She wanted to eat it, but it was too high up.
Lily asked her friend, a little bird, "Can you help me get the apple?"
The bird said, "Sure, I can fly up and get it for you."
The bird flew up to the apple and pecked it off the tree. Lily was so happy and took a big bite. But then, she saw a bug on the apple. She didn't like bugs, so she threw the apple away.
Later that day, Lily's mom asked her to help with the laundry. Lily saw a shirt that was too big for her. She asked her mom, "Can you make it fit me?"
Her mom said, "Yes, I can make it fit you."
Lily was happy that her shirt would fit her. She learned that sometimes things don't fit, but there is always a way to make them fit.
<s>
Once upon a time, there was a little girl named Lily
achieved tok/s:  27.255237280889268

パフォーマンスは7分の1くらいに落ちました。

mojo buildしてみる

「buildしたら速くなるのかしら？」と思って、llama2.mojo（110Mモデルを使用するように修正済みの版）を

$ mojo build llama2.mojo

と叩く。buildしてできたファイルは、llama2

$ ll llama2*
-rwxr-xr-x 1 noguchi noguchi 1755640  9月 13 01:27 llama2*
-rw-r--r-- 1 noguchi noguchi   26679  9月 12 08:52 llama2.mojo

実行すると、相互運用性エラー？と出力され落ちる。

$ ./llama2
num hardware threads:  12  SIMD vector width:  8
Mojo/Python interoperability error: Unable to locate a suitable libpython, please set `MOJO_PYTHON_LIBRARY`
Segmentation fault

ふむふむ。エラーを解消しよう。
内容見るに、環境変数MOJO_PYTHON_LIBRARYに libpython の所在を設定すれば良いのだな。

$ ls $(python3 -c 'import sysconfig; print(sysconfig.get_config_var("LIBDIR"))') | grep libpython
libpython3.10.a
libpython3.10.so
libpython3.10.so.1
libpython3.10.so.1.0
$ ls -l /usr/lib/x86_64-linux-gnu/libpython3.10.so
lrwxrwxrwx 1 root root 18 Jun 11 14:26 /usr/lib/x86_64-linux-gnu/libpython3.10.so -> libpython3.10.so.1
$ echo 'export MOJO_PYTHON_LIBRARY="/usr/lib/x86_64-linux-gnu/libpython3.10.so"' >> ~/.bashrc
$ source ~/.bashrc

気を取り直して、再実行。

$ ./llama2
num hardware threads:  12  SIMD vector width:  8
checkpoint size:  438381596
<s>
Once upon a time, there was a little girl named Lily. She loved to play outside in the sunshine. One day, she saw a big, red apple on a tree. She wanted to eat it, but it was too high up.
Lily asked her friend, a little bird, "Can you help me get the apple?"
The bird said, "Sure, I can fly up and get it for you."
The bird flew up to the apple and pecked it off the tree. Lily was so happy and took a big bite. But then, she saw a bug on the apple. She didn't like bugs, so she threw the apple away.
Later that day, Lily's mom asked her to help with the laundry. Lily saw a shirt that was too big for her. She asked her mom, "Can you make it fit me?"
Her mom said, "Yes, I can make it fit you."
Lily was happy that her shirt would fit her. She learned that sometimes things don't fit, but there is always a way to make them fit.
<s>
Once upon a time, there was a little girl named Lily
achieved tok/s:  26.334813590829288

うーん、速くなったような、あんまり速くなってないような…。

この記事が気に入ったらサポートをしてみませんか？