見出し画像

README をシュッと ChatGPT に作らせてみた

bbz

どもー (/・ω・)/



アプリケーションを作るときに README って作ったりメンテするのめんどくさいのよね～



ってなわけで何の捻りもないですがシュッと ChatGPT に書かせたろ☆



という話。




実は前にも似たようなことやってたのでござる (　・ω・)





しかしながらまぁ ChatGPT 3.5-turbo を使ってたのでトークンの問題やらでなんか微妙な結果になっていたのである





駄菓子菓子




先日の ChatGPT モデルのアップデートによって扱えるトークンが 16k に増えたのでもうちょいいい感じになるのではないか？(　・ω・)




そう思った私は脳死で昔作った雑コードを走らせるのであった、、、





今回の検証（遊び）

  1. 適当な OSS の GitHub リポジトリをシュッとローカルにクローンしておく

  2. 落としてきたリポジトリコードへのパスを渡して指定拡張子のファイル内容をぐりぐり取ってくる

  3. ChatGPT に食べさせて要約させる

  4. 次のファイルコンテンツが来たら前回の要約と一緒に ChatGPT に投げる

  5. 繰り返し

  6. 最終的に README を出力する

  7. 本家の README と比較してみる




ってなわけで Let's do it (/・ω・)/





その昔作っていたコードの利用モデルを gpt-3.5-turbo-16k に変更する
（トークン力こそパワー、、、！！）

import os
import multiprocessing
import itertools
from langchain.chat_models import ChatOpenAI
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
from langchain.prompts.chat import (
    ChatPromptTemplate,
    HumanMessagePromptTemplate,
)

os.environ["OPENAI_API_KEY"] = ""

def get_repository_contents(repository_path):
    if os.path.isdir(repository_path):
        contents = []
        for entry in os.scandir(repository_path):
            if entry.is_file():
                if entry.name.endswith((".py", ".yaml", ".md")):
                    with open(entry.path, 'r', encoding='utf-8') as f:
                        contents.append(f.read())
            elif entry.is_dir():
                contents.extend(get_repository_contents(entry.path))
        return contents
    else:
        return []

def get_all_repository_contents(base_directory, batch_size=1000):
    with multiprocessing.Pool() as pool:
        paths = [os.path.join(base_directory, d) for d in os.listdir(base_directory)]
        contents = pool.imap_unordered(get_repository_contents, paths, chunksize=batch_size)
        yield from itertools.chain.from_iterable(contents)

def generate_serially(content, prompt, readme):
    chat = ChatOpenAI(temperature=0.1, model_name="gpt-3.5-turbo-16k")
    chain = LLMChain(llm=chat, prompt=prompt)
    result = readme
    try:
        res = chain.run(text=content, current_result=result)
    except Exception as e:
        print("エラーが発生しました:", str(e))
    else:
        result = res
    return result




プロンプトを綴る

prompt_template = ("""
- Instrunctions:
You are a software engineer with a detailed understanding of the application source code and documentation.
You are about to be given a series of source code and documentation information, each of which will be denoted by a ````.
You are now given a series of source code and documentation information. You are expected to understand the information in detail and create a README that will make it easy for everyone to use the application.
If you have the results to date, please add information to refine the whole thing and save it as README.md.
Previous results are provided bracketed by ###.
Please work not only with the information given, but also imagine the structure of the entire application.
The README.md should contain an overview of the application, its benefits, detailed instructions for use, and other important information.
Finally, output README.md.

- Prerequisites:.
The application name is "guidance."
The information given is part of the overall application.

- Input
```
{text}
```

- Current Result
###
{current_result}
###
""")




後は実行するだけ

human_message_prompt = HumanMessagePromptTemplate(
        prompt=PromptTemplate(
            template=prompt_template,
            input_variables=["text", "current_result"],
        )
    )
chat_prompt_template = ChatPromptTemplate.from_messages([human_message_prompt])
readme = ""
target_dir = './guidance'
for content in get_all_repository_contents(target_dir):
    readme = generate_serially(content, chat_prompt_template, readme)
path = "./output.md"
with open(path, mode="w") as f:
    f.write(readme)




特に深い意味はないですがソースのボリューム的に今回はこちらの README を作ってみようとしておるでござる (　・ω・)




でまぁ実行したら後は待つだけ☆




（ ﾟ ρ ﾟ )ﾎﾞｰｰｰｰ


（ ﾟ ρ ﾟ )ﾎﾞｰｰｰｰ


（ ﾟ ρ ﾟ )ﾎﾞｰｰｰｰ


（ ﾟ ρ ﾟ )ﾎﾞｰｰｰｰ


（ ﾟ ρ ﾟ )ﾎﾞｰｰｰｰ



はい。



しばらくほっといたら完成☆



結果

で、最終的に出来上がったものがこちら (/・ω・)/

guidance

Overview

The guidance application is a versatile tool designed to simplify the process of working with language model (LM) models from popular libraries such as OpenAI and Transformers.
This application provides a user-friendly interface with an intuitive API to perform various natural language processing tasks, including text generation, text completion, translation, and more.
By abstracting away the complexities of integrating LM models, guidance makes it easier for developers to leverage the power of these models in their projects.

Benefits

  • Seamless Integration: guidance seamlessly integrates with OpenAI and Transformers LM models, eliminating the need for developers to deal with the integration details.
    You can easily access and use these models in your application with minimal effort.

  • Caching for Improved Performance: To enhance performance, guidance incorporates a caching feature that stores loaded LM models in memory.
    This caching mechanism significantly reduces the model loading time, allowing for faster processing.
    Furthermore, the cached models can be reused across multiple requests, saving computational resources.

  • Simple API: The guidance application provides a clean and intuitive API, suitable for developers of all skill levels.
    You do not need to be an expert in natural language processing or machine learning to use this application effectively.
    With just a few lines of code, you can generate high-quality text.

  • Customization Options: The code and templates provided by guidance are highly customizable.
    You can easily modify them to suit your specific requirements and build upon them to add more advanced functionality to your projects.
    guidance serves as a solid foundation for developing sophisticated natural language processing applications.

Prerequisites

Before using the guidance application, ensure that you have the following prerequisites:

  • Python 3.8 or higher

  • Python package manager (pip)

Installation

To install the guidance application, run the following command in your terminal or command prompt:

pip install guidance

Usage

To use the guidance application, follow the instructions below:

  1. Import the necessary modules in your Python script:

import guidance

  1. Define the necessary functions and variables to support your desired use case. Refer to the provided examples and documentation for guidance.

  2. Utilize the guidance application's API to interact with LM models. Consult the documentation for detailed information on how to use the various features offered by the application.

  3. Test and validate your code using your preferred testing framework. Ensure that your application functions as expected and handles edge cases appropriately.

  4. Customize the guidance application code and templates to fit your specific needs. Add additional logic and functionality as required by your project.

# Example usage of the guidance application
template = "Hello, World!"
llm = ...  # Set your desired LLM

program = guidance(template, llm=llm)
output = program.run()

print(output)

For more in-depth information and detailed examples, please refer to the official guidance documentation.

Important Information

  • Ensure that you are aware of any usage limitations or licensing requirements associated with the LM models you are using. Consult the official OpenAI and Transformers documentation for detailed information.

Contributing

If you would like to contribute to the guidance application, please follow the guidelines provided in the CONTRIBUTING.md file.

License

The guidance application is open source and is licensed under the MIT License.

Note: The code provided in this README serves as a simplified overview of the guidance application's usage. For comprehensive instructions and more detailed examples, please refer to the official documentation.



うーん (　・ω・)



まぁ README としてはうん。



悪くはないけど。



なんかもうちょい詳しく書いて欲しいなぁという感じ。



本家の README の方が全然いいぞ☆



この辺はプロンプトの調整次第ですかなぁ(　・ω・)




あと今回は使えるトークン増えたしファイルコンテンツを特に分割せずに全部ぶっこむようにしたので



リクエスト：ファイルコンテンツ + プロンプト + 前回の要約 + その他
レスポンス：README 的な何か



となって、最終生成される README のボリュームにも影響したかもー？(/・ω・)/



ということでファイルコンテンツは適当なサイズでチャンクに分けてあげるともうちょいいい感じなるかもしれぬ。




しかしまぁそれっぽいのはできたのでよしとしましょう。



大事ななのはシュッとできたかどうかなのである☆



ということでおしまい。

この記事が気に入ったらサポートをしてみませんか？