AIによる再設計でLlama2の15倍高速!?DeciLM登場!

Llama2の15倍高速というDeciLMというのが発表された。もう何が何だかわからない。これはデコーダーモデルというやつで、4096トークンに対応

Llama 2 is Yesterday's News.

Introducing DeciLM-6B and It's Leaving Llama 2 in the Dust.

An efficient, blazing-fast text generation language model.

DeciLM-6B is 15x faster than Llama 2!

Here's what sets it apart:

⭐️ Performance Metrics that Speak Volumes:

DeciLM-6B has set… pic.twitter.com/GE5y1m2Y9W

— Hasan Toor ✪ (@hasantoxr) September 13, 2023

なぜこんなに高速なのかというと、AIが自動設計したから
AutoNAC(自動ニューラルアーキテクチャ構築)というDeci社の技術により、LLMの画期的な高性能化が実現したということらしい。まあ要はLLMは単なるデモに過ぎないということ。

Automated Neural Architecture Construction (AutoNAC)? | Deci

オフィシャルのColabを動かしたがどうもそんなに速くない。
GPUをちゃんと選択しないとダメみたい。

Google Colaboratory

仕方ないのでローカル環境で動作させる。
マシンはいつものようにドスパラ製Memeplexマシン(A6000x2)
いつもありがとうございます。

>>> model_id = 'Deci/DeciLM-6b'
>>> model = AutoModelForCausalLM.from_pretrained(model_id,
...                                              device_map="auto",
...                                              trust_remote_code=True
...                                              )
Downloading (…)lve/main/config.json: 100%|███████████████████████████████████████████████████| 831/831 [00:00<00:00, 675kB/s]
Downloading (…)figuration_decilm.py: 100%|██████████████████████████████████████████████| 1.85k/1.85k [00:00<00:00, 1.39MB/s]
A new version of the following files was downloaded from https://huggingface.co/Deci/DeciLM-6b:
- configuration_decilm.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
Downloading (…)n/modeling_decilm.py: 100%|██████████████████████████████████████████████| 12.6k/12.6k [00:00<00:00, 12.4MB/s]
A new version of the following files was downloaded from https://huggingface.co/Deci/DeciLM-6b:
- modeling_decilm.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
Downloading (…)fetensors.index.json: 100%|██████████████████████████████████████████████| 23.9k/23.9k [00:00<00:00, 17.9MB/s]
Downloading (…)of-00003.safetensors: 100%|██████████████████████████████████████████████| 4.95G/4.95G [04:11<00:00, 19.7MB/s]
Downloading (…)of-00003.safetensors: 100%|██████████████████████████████████████████████| 4.92G/4.92G [04:18<00:00, 19.0MB/s]
Downloading (…)of-00003.safetensors: 100%|██████████████████████████████████████████████| 1.56G/1.56G [01:08<00:00, 23.0MB/s]
Downloading shards: 100%|█████████████████████████████████████████████████████████████████████| 3/3 [09:42<00:00, 194.03s/it]
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████| 3/3 [00:04<00:00,  1.41s/it]
>>> model.device
device(type='cuda', index=0)
>>> tokenizer = AutoTokenizer.from_pretrained(model_id)
Downloading (…)okenizer_config.json: 100%|██████████████████████████████████████████████████| 776/776 [00:00<00:00, 80.4kB/s]
Downloading tokenizer.model: 100%|█████████████████████████████████████████████████████████| 500k/500k [00:00<00:00, 606kB/s]
Downloading (…)/main/tokenizer.json: 100%|██████████████████████████████████████████████| 1.84M/1.84M [00:00<00:00, 2.17MB/s]
>>> 
>>> tokenizer.pad_token = tokenizer.eos_token
>>> model_inputs = tokenizer("A list of colors: red, blue", return_tensors="pt").to("cuda")
>>> print(tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0])
A list of colors: red, blue, green, yellow, orange, purple, pink, black, white, and brown.
A list of shapes: circle, square, triangle, rectangle, oval, heart, star

Colabにあった通りに実行すると、ちゃんと色を数え上げ始めた。しかも速い(Colabよりずっと速い)
もう少し高度なことをやらせてみる。

>>> def generate_text(prompt:str, max_new_tokens:int, temperature:float) -> str:
...     model_inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
...     generated_ids = model.generate(**model_inputs,
...                                    max_new_tokens=max_new_tokens,
...                                    temperature=temperature,
...                                    num_beams=5,
...                                    no_repeat_ngram_size=4,
...                                    early_stopping=True
...                                    )
...     decoded_generation = tokenizer.batch_decode(generated_ids, skip_special_tokens=False)[0]
...     return print(decoded_generation)
... 
>>> prompt = """In this blog post, we're going to talk about why waking up is"""
>>> generate_text(prompt, 500, 0.25)
<s> In this blog post, we're going to talk about why waking up is one of the most important things you can do for your health and well-being. We're also going to give you some tips on how to make sure you're getting the most out of your mornings. So, if you're ready to start your day off on the right foot, let's get started!
Why is waking up important?
Waking up is important because it sets the tone for the rest of your day. If you wake up feeling groggy and unrested, it's going to be difficult to stay focused and productive throughout the day. On the other hand, waking up early and getting a good night's sleep can help you feel more energized and ready to take on the day.
How can I make sure I'm waking up on time?
If you're having trouble waking up in the morning, there are a few things you can try. One option is to set an alarm that goes off 15 minutes earlier than your usual wake-up time. This will give you a little bit of extra time to wake up and get ready for the day. Another option is to make your bedroom as dark as possible. This will help you fall asleep faster and wake up more easily in the morning.
What are some tips for making the most of my mornings?
There are a few tips you can follow to make the most of your morning routine. First, make sure you get enough sleep the night before. Second, wake up at the same time every day, even on weekends. Third, eat a healthy breakfast and drink plenty of water. Finally, take some time for yourself each morning to do something you enjoy, such as reading, meditating, or exercising. By following these tips, you'll be able to start each day feeling refreshed and ready to tackle whatever comes your way.
How do I know if I'm getting enough sleep?
There is no one-size-fits-all answer to this question, as everyone's sleep needs are different. However, as a general rule of thumb, most adults need between seven and nine hours of sleep per night. If you're not sure how much sleep you need, talk to your doctor or a sleep specialist. They can help you determine the right amount of sleep for you.

速い!
確かに体感的に速い。
意外と「体感的に速い」ということはあまりないので衝撃だ。
これはまだ本気出してないのでLlama2の3倍くらいの速さのはずだが、あきらかに「速い」と体感できる。通常のLLMの三倍だと!?

日本語が使えるか確認してみる

>>> generate_text("富士山へ登りたいのですが?", 500, 0.25)
<s> 富士山へ登りたいのですが? I want to climb Mt. Fuji.</s>
>>> generate_text("東京駅から秋葉原まではどう行けばいいですか?", 500, 0.25)
<s> 東京駅から秋葉原まではどう行けばいいですか? How do I get to Akihabara from Tokyo Station?</s>
>>> generate_text("クォータニオンについて教えてください", 500, 0.25)
<s> クォータニオンについて教えてください。</s>

日本語を入れると翻訳してしまうが、クォータニオンについては翻訳もしてくれなかった。
まあ日本語が苦手なのはLlama2も同じ
こんにちはくらいはわかるのかな?

>>> generate_text("こんにちは", 500, 0.25)
<s> こんにちは。 申し訳ありませんが、 皆さんのご意見をお待ちしております。 週末には、 年末年始に過ぎるような気持ちになっていました。 毎日繰り返していることができなくなってしまったので、 今回は、 私の感想をお届けしていきたいと思います。. このページに掲載されている情報・写真・映像などの一切の無断転載を禁じます。 Copyright( C) 1996- All Rights Reserved.
This page contains text from Wikipedia, the Free Encyclopedia -	https://wn. com/List_of_Final_Fantasy_VII_characters
Final Fantasy VII (Japanese: ファイナルファンタジーVII, Hepburn: Fainaru Fantajī Sebun) is a role- playing video game developed and published by Square for the PlayStation console. It is the seventh main installment in the Final Fantasy series. The game's story follows a mercenary named Cloud Strife as he assists an eco- terrorist group called Avalanche in their fight against the Shinra Electric Power Company, an organization that is draining the planet's life to power the city of Midgar.
The game was directed by Yoshinori Kitase and produced by Hironobu Sakaguchi, and features character designs by Tetsuya Nomura and music composed by Nobuo Uematsu. Final Fantasy VII was first released in Japan on January 31, and in North America on September 7, The game was later ported to the Microsoft Windows, PlayStation 3, PlayStation Portable, PlayStation Vita, Xbox 360, and mobile phones. A high-definition remastered version of the game was released for PlayStation 4 in

まさかの長文。
そしてFF7。なんかのユーザーサポート窓口か?
と思ったが、よく考えるとこれInstructモデルじゃなくて補完モデルだ。
だから聞くんじゃなくて続きを書いてもらう感じにしないといけない。

物語生成を試してみる

>>> generate_text("In 2103, At least mankind realize", 500, 0.7)
<s> In 2103, At least mankind realize that they are not the only intelligent species in the universe.
In 2150, the first contact with an alien species is made.
In the year 2161, the first interstellar colony is founded.
By 2200, mankind has colonized the entire galaxy.
By the year 3000, the galaxy is united under the rule of the Galactic Empire.
This page was last modified on 15 February 2019, at 19:18.</s>

翻訳

2103年、少なくとも人類は、自分たちが宇宙で唯一の知的種ではないことに気づきました。
2150年、外来種とのファーストコンタクトが行われる。
2161年、最初の星間コロニーが設立される。
2200年までに、人類は銀河系全体に植民地を築きました。
3000 年までに、銀河は銀河帝国の支配下に統一されました。
このページの最終更新日は 2019 年 2 月 15 日 19:18 です。

陳腐!
陳腐だがそれは俺のダイアログが酷いんだから仕方ない。
最後のタイムスタンプ風のものをみるとどこかの寝言ブログみたいなのを再現したのかもしれない。

ちょっとラブストーリーっぽいのも書いてみるか

>>> generate_text("one day, I saw an angel. Her name is Natalie", 500, 0.7)
<s> one day, I saw an angel. Her name is Natalie, and she's the most beautiful angel I've ever seen. She's so beautiful that I can't stop staring at her. I've never seen an angel before, so I don't know what they look like. But I know that she's an angel because she's wearing a white dress, and she has wings.
Natalie is a beautiful angel. She has long, blonde hair, and she wears a white dress. She also has wings. Natalie is an angel, but she doesn't look like one. She doesn't have a halo, or wings, or anything like that. She just looks like a normal girl. But she's not a normal girl, she's a real angel.
I'm not sure if Natalie is really an angel or not, but I think she is. She looks like an angel to me, and I think she's really beautiful. I think she looks like the kind of angel that you would see in a movie, or in a book. I think that's why I like her so much.
I think it's because she looks like a real person, and not just a character in a movie or a book. She has a real personality, and a real life, and she seems to be a real person. She seems to be the kind of person that you would want to be friends with. She seems like a really nice person, and I like her a lot.
I like Natalie because she seems like a nice person, but I also like her because she seems to have a real life. I like her because I feel like I can relate to her, and I feel like she can relate to me. I feel like we have a lot in common, and that makes me feel like we can be friends.
I feel like we could be friends because we both have a lot of the same interests. We both like to read, and we both like to play video games. We also both like to watch movies and TV shows. We both also like to go to the movies and watch movies. We also like to watch TV shows, and we also like to read books.
We also like to hang out with each other. We like to talk about our favorite movies, TV shows, books, and video games.

翻訳

ある日、私は天使を見ました。彼女の名前はナタリー、私が今まで見た中で最も美しい天使です。彼女はとても美しいので、彼女を見つめることができません。私は天使を見たことがないので、天使がどのようなものであるかわかりません。でも、白いドレスを着ていて、羽があるので、私は彼女が天使であることを知っています。
ナタリーは美しい天使です。彼女は長いブロンドの髪を持ち、白いドレスを着ています。彼女には翼もあります。ナタリーは天使ですが、天使のようには見えません。彼女には後光や翼などはありません。彼女はただの普通の女の子のように見えます。しかし、彼女は普通の女の子ではなく、本物の天使です。
ナタリーが本当に天使なのかどうかはわかりませんが、私は天使だと思います。私にとって彼女は天使のようで、本当に美しいと思います。彼女は映画や本で見るような天使に似ていると思います。それが私が彼女をとても好きな理由だと思います。
それは彼女が映画や本の登場人物ではなく、実在の人物のように見えるからだと思います。彼女は本物の性格、本物の人生を持っており、本物の人間のようです。彼女は友達になりたくなるような人のようです。彼女はとてもいい人のようで、私は彼女がとても好きです。
私はナタリーがいい人そうなので好きですが、彼女が現実的な人生を持っているように見えるという点でも好きです。私は彼女に共感できると感じ、彼女も私に共感できると感じるので彼女が好きです。私たちには共通点がたくさんあると思うので、友達になれそうな気がします。
お互い同じ興味を持っているので友達になれる気がします。私たちは二人とも読書が好きで、ビデオゲームをするのが好きです。私たちは二人とも映画やテレビ番組を見るのが好きです。私たちは二人とも映画に行ったり映画を見るのが好きです。私たちはテレビ番組を見るのも好きですし、本を読むのも好きです。
私たちはお互いにたむろするのも好きです。私たちはお気に入りの映画、テレビ番組、本、ビデオゲームについて話すのが好きです。

書き出し(プロンプト)によってこんなに結果が変わるのか。
ビジネス小説っぽいのも書いてみる。

>>> generate_text("one day, I saw an angel. Her name is Natalie", 500, 0.7)
<s> one day, I saw an angel. Her name is Natalie, and she's the most beautiful angel I've ever seen. She's so beautiful that I can't stop staring at her. I've never seen an angel before, so I don't know what they look like. But I know that she's an angel because she's wearing a white dress, and she has wings.
Natalie is a beautiful angel. She has long, blonde hair, and she wears a white dress. She also has wings. Natalie is an angel, but she doesn't look like one. She doesn't have a halo, or wings, or anything like that. She just looks like a normal girl. But she's not a normal girl, she's a real angel.
I'm not sure if Natalie is really an angel or not, but I think she is. She looks like an angel to me, and I think she's really beautiful. I think she looks like the kind of angel that you would see in a movie, or in a book. I think that's why I like her so much.
I think it's because she looks like a real person, and not just a character in a movie or a book. She has a real personality, and a real life, and she seems to be a real person. She seems to be the kind of person that you would want to be friends with. She seems like a really nice person, and I like her a lot.
I like Natalie because she seems like a nice person, but I also like her because she seems to have a real life. I like her because I feel like I can relate to her, and I feel like she can relate to me. I feel like we have a lot in common, and that makes me feel like we can be friends.
I feel like we could be friends because we both have a lot of the same interests. We both like to read, and we both like to play video games. We also both like to watch movies and TV shows. We both also like to go to the movies and watch movies. We also like to watch TV shows, and we also like to read books.
We also like to hang out with each other. We like to talk about our favorite movies, TV shows, books, and video games.
>>> generate_text("one day, Bob was sitting on the couch at the beach of Santa Monica. He never know his successful business start just from here", 500, 0.7)
<s> one day, Bob was sitting on the couch at the beach of Santa Monica. He never know his successful business start just from here. He was thinking about his life and his future. He was wondering what he should do in the future.
One day, he decided to quit his job and start his own business. He wanted to be his own boss. He had a lot of ideas and he wanted to make his dreams come true. He started his own business and it was very successful. He was very happy with his new life. He was living his dreams and he was doing what he wanted to do. He was a very successful businessman and he was living his life to the fullest. He was happy and he was enjoying every moment of his life.
He was a very happy man and he was very successful in his business. He was making a lot of money and he was able to buy a lot of things that he wanted to buy. He was able to travel to different places and see different cultures. He was also able to help a lot of people who were in need. He was doing a lot of good things in his life and he was making a difference in the world.
Bob's life was very successful and he was happy with his life. He had everything that he wanted in his life. But one day, he realized that he was not happy. He was not able to enjoy his life because he was always working. He was always thinking about his business and he was never able to take a break. He was working all the time and he was always stressed out.
Bob realized that he needed to change his lifestyle. He needed to find a way to make his life more enjoyable. He decided to take a vacation. He went to Hawaii and he spent a week there. He relaxed and he enjoyed his time there. When he came back, he was a different person. He was happier and he was more relaxed.
Bob was able to find a balance between his work and his personal life. He realized that he could have both. He could have a successful business and he could also have a life that he enjoyed. He was no longer stressed out and he was happier.
Bob is a very successful man and he is living his dream life. He is able to do what he wants to do and he is able to enjoy every moment of it. He is living a life that is full of happiness and joy. He is a very happy person and he is very successful.
Bob

翻訳

ある日、ボブはサンタモニカのビーチのソファに座っていました。彼は、ここからビジネスの成功が始まるとは決して思っていません。彼は自分の人生と将来について考えていました。彼は将来何をすべきか考えていました。
ある日、彼は仕事を辞めて自分のビジネスを始めることを決意しました。彼は自分自身の上司になりたかったのです。彼はたくさんのアイデアを持っていて、自分の夢を実現したいと思っていました。彼は自分のビジネスを始め、それは大成功しました。彼は新しい生活にとても満足していました。彼は夢を実現し、やりたいことをやっていました。彼は非常に成功したビジネスマンであり、自分の人生を最大限に生きていました。彼は幸せで、人生のあらゆる瞬間を楽しんでいた。
彼はとても幸せな人で、ビジネスでも大成功しました。彼はたくさんお金を稼いでいて、買いたいものをたくさん買うことができました。彼はさまざまな場所に旅行し、さまざまな文化を見ることができました。彼はまた、困っている多くの人々を助けることができました。彼は人生で多くの良いことをしており、世界に変化をもたらしていました。
ボブの人生は非常に成功しており、自分の人生に満足していました。彼は人生で望んでいたものをすべて持っていました。しかしある日、彼は自分が幸せではないことに気づきました。彼はいつも仕事をしていたので、人生を楽しむことができませんでした。彼は常に自分のビジネスのことを考えており、決して休むことができませんでした。彼はいつも仕事をしていて、常にストレスを感じていました。
ボブは自分のライフスタイルを変える必要があることに気づきました。彼は人生をもっと楽しくする方法を見つける必要がありました。彼は休暇を取ることに決めた。彼はハワイに行き、そこで一週間過ごしました。彼はリラックスしてそこでの時間を楽しんだ。戻ってきたとき、彼は別人になっていました。彼はより幸せになり、よりリラックスしていました。

まて、この方式なら日本語でもいけるか?

>>> generate_text("富士山は", 500, 0.7)
<s> 富士山は、静岡県北部に位置しています。
Fuji Mountain is located in the northern part of Shizuoka Prefecture, Japan. It is the highest mountain in Japan at 3,776 m (12,388 ft) and the second-highest peak of an island globally. Mount Fuji's exceptionally symmetrical cone is a well-known symbol of Japan and it is frequently depicted in art and photographs, as well as visited by sightseers, pilgrims and climbers. It is one of Japan's "Three Holy Mountains", along with Mount Tate and Mount Haku. The mountain was registered as a UNESCO World Heritage Site in 2013 as part of the "Historic Villages of Shirakawa-go and the Sacred Sites and Pilgrimage Routes in the Kii Mountain Range". Mount Fuji lies about 100 kilometres (62 mi) south-west of Tokyo, and can be seen from there on a clear day. It is also visible from several other major cities, including Nagoya, Kyoto, Osaka, Hiroshima, and Fukuoka.</s>
>>> generate_text("日本の首都は", 500, 0.7)
<s> 日本の首都は東京です。
Tokyo is the capital of Japan.
東京の首都として知られています。
It is known as the capital of Tokyo.
日本は東京を首都にしました。
Japan made Tokyo its capital.
日韓協定は1965年に締結されましたが、日本が韓国の首都であることを認めていません。
The Japan-Korea Treaty was concluded in 1966, but Japan does not acknowledge that it is the capital city of South Korea.
Japan's capital city is Tokyo, but South Korea's capital is Seoul.
Tōkyō-to (東京都) is a prefecture located in the Kantō region on the eastern side of Japan's main island, Honshū.
It is the center of the Greater Tokyo Area, the most populous metropolitan area in the world.
As of October 1, 2016, the prefecture had an estimated population of over 13.7 million, and a population density of 16,700 persons per square kilometre.
The prefecture was formed in 1868 from the merger of the city of Edo and three surrounding prefectures.
It was renamed from Tokyo Prefecture (東京府) to its current name on May 15, 1943.
The city of Tokyo is the seat of the Japanese government and the Imperial Palace, and the home of the Japanese Imperial Family.
The metropolis contains 23 central wards and 26 special wards.
The 23 special wards are collectively known as Tōkyō to (東京), which can be translated as "Greater Tokyo" or "Greater East Tokyo".
The population of the wards was 9,017,271 in 2020, and the population of the city was estimated to be 13,949,070.
The wards are further subdivided into 23 cities, towns and villages, which are governed by elected municipal assemb

すぐ英語になっちゃうの、すげーllama2っぽい
しかしすごい時代来たなあ

まあ来ることはわかっていたが。