【強化学習自動売買】~環境構築編①~
0.はじめに
先日,仮想通貨の板情報を推定する,相関0.9超えのモデルを構築しました.
しかし,取引するモジュールがなく,現在このモデルを活かしきれていません.そこで,取引するモジュールを強化学習を使ってエンドツーエンドで構築していこうと思います.
ただ,強化学習を使ったことがないので,環境構築から始まり,サンプルプログラムの実行などからやっていこうと思います.
1.ライブラリ「OpenAI Gym」のインストール
この辺の記事を参考にOpenAI Gymのインストールをしてみました
確認のためにサンプルプログラムを実行してみました.
ちゃんとレンダリングされてますね!
参考までに実行したコードを記載しておきます.
C:\Users\masaki>conda info -e
# conda environments:
#
base * C:\Users\masaki\anaconda3
C:\Users\masaki>conda create -n py_gym_v1
Collecting package metadata (current_repodata.json): done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.10.1
latest version: 4.10.3
Please update conda by running
$ conda update -n base -c defaults conda
## Package Plan ##
environment location: C:\Users\masaki\anaconda3\envs\py_gym_v1
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate py_gym_v1
#
# To deactivate an active environment, use
#
# $ conda deactivate
C:\Users\masaki>conda activate py_gym_v1
(py_gym_v1) C:\Users\masaki>pip install gym
Collecting gym
Downloading gym-0.21.0.tar.gz (1.5 MB)
|████████████████████████████████| 1.5 MB 2.2 MB/s
Requirement already satisfied: numpy>=1.18.0 in c:\users\masaki\anaconda3\lib\site-packages (from gym) (1.20.1)
Requirement already satisfied: cloudpickle>=1.2.0 in c:\users\masaki\anaconda3\lib\site-packages (from gym) (1.6.0)
Building wheels for collected packages: gym
Building wheel for gym (setup.py) ... done
Created wheel for gym: filename=gym-0.21.0-py3-none-any.whl size=1616826 sha256=e32cfc7e91a07ac7b2e03e3ee00341b819fbd11edd5793d6d2641272705953f6
Stored in directory: c:\users\masaki\appdata\local\pip\cache\wheels\27\6d\b3\a3a6e10704795c9b9000f1ab2dc480dfe7bed42f5972806e73
Successfully built gym
Installing collected packages: gym
Successfully installed gym-0.21.0
(py_gym_v1) C:\Users\masaki>pip install 'gym[atari]'
ERROR: Invalid requirement: "'gym[atari]'"
(py_gym_v1) C:\Users\masaki>pip install gym[atari]
Requirement already satisfied: gym[atari] in c:\users\masaki\anaconda3\lib\site-packages (0.21.0)
Requirement already satisfied: numpy>=1.18.0 in c:\users\masaki\anaconda3\lib\site-packages (from gym[atari]) (1.20.1)
Requirement already satisfied: cloudpickle>=1.2.0 in c:\users\masaki\anaconda3\lib\site-packages (from gym[atari]) (1.6.0)
Collecting ale-py~=0.7.1
Downloading ale_py-0.7.2-cp38-cp38-win_amd64.whl (925 kB)
|████████████████████████████████| 925 kB 1.3 MB/s
Collecting importlib-resources
Downloading importlib_resources-5.2.2-py3-none-any.whl (27 kB)
Requirement already satisfied: zipp>=3.1.0 in c:\users\masaki\anaconda3\lib\site-packages (from importlib-resources->ale-py~=0.7.1->gym[atari]) (3.4.1)
Installing collected packages: importlib-resources, ale-py
Successfully installed ale-py-0.7.2 importlib-resources-5.2.2
(py_gym_v1) C:\Users\masaki>python
Python 3.8.8 (default, Apr 13 2021, 15:08:03) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated. Libraries may fail to load. To activate this environment
please see https://conda.io/activation
Type "help", "copyright", "credits" or "license" for more information.
>>> from gym import envs
>>> envids = [spec.id for spec in envs.registry.all()]
>>> print(envids)
['ALE/Tetris-v5', 'ALE/Tetris-ram-v5', 'Adventure-v0', 'AdventureDeterministic-v0', 'AdventureNoFrameskip-v0', 'Adventure-v4', 'AdventureDeterministic-v4', 'AdventureNoFrameskip-v4', 'Adventure-ram-v0', 'Adventure-ramDeterministic-v0', 'Adventure-ramNoFrameskip-v0', 'Adventure-ram-v4', 'Adventure-ramDeterministic-v4', 'Adventure-ramNoFrameskip-v4', 'AirRaid-v0', 'AirRaidDeterministic-v0', 'AirRaidNoFrameskip-v0', 'AirRaid-v4', 'AirRaidDeterministic-v4', 'AirRaidNoFrameskip-v4', 'AirRaid-ram-v0', 'AirRaid-ramDeterministic-v0', 'AirRaid-ramNoFrameskip-v0', 'AirRaid-ram-v4', 'AirRaid-ramDeterministic-v4', 'AirRaid-ramNoFrameskip-v4', 'Alien-v0', 'AlienDeterministic-v0', 'AlienNoFrameskip-v0', 'Alien-v4', 'AlienDeterministic-v4', 'AlienNoFrameskip-v4', 'Alien-ram-v0', 'Alien-ramDeterministic-v0', 'Alien-ramNoFrameskip-v0', 'Alien-ram-v4', 'Alien-ramDeterministic-v4', 'Alien-ramNoFrameskip-v4', 'Amidar-v0', 'AmidarD
.....
2.サンプルプログラム
インスタに映っている山登りのサンプルプログラムです.
import gym
env = gym.make('CartPole-v0') # select game
observation = env.reset() # initialize
for i in range(100):
env.render()
observation, reward, done, info = env.step(1) # go right
env.env.close()
3.おわりに
次回はGoogle Colabを使った環境構築を行っていきます.
▼▼▼▼▼▼次回記事▼▼▼▼▼▼
▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲
この記事が気に入ったらサポートをしてみませんか?