RASPi4（Ubuntu）でTensorFlow環境の構築
2020年12月中頃から試行していたことが実ったのでここに残す。
結論を言うと、チュートリアルのままやれば出来ました。
しかし、OSの設定を変えないとハマります。
今回も、多数のサイトを参考にさせて頂いております。
先駆者の方々には感謝致します。
１．実験環境
・Raspberry Pi 4 Model B / 8GB
・Ubuntu-20.10 server arm64 raspi（デスクトップ環境はMATE）
・Python 3.8
２．OS（Ubuntu）の準備
画像処理環境の構築①と同じことを行う
３．サスペンド・スリープ設定
ソースからビルドするのに長時間動作させるので、サスペンドやスリープに入らないようにする（これで結構ハマりました）
（１）GUIによる設定
システム設定→電源管理→アクション、ディスプレイ共に「確認しない」
（２）コンソールによる設定
#-- ファイルを開いて二行追加
$ sudo vi /etc/gdm3/greeter.dconf-defaults
#-- Automatic suspendよりも下に追記。私は最終行に追加しました。
sleep-inactive-ac-timeout=0
sleep-inactive-battery-timeout=0４．Python環境の準備
OSに初期で入っているPython3.8をそのまま利用する
コンパイルにPython2.7が必要になるため導入する
$ python3 -V
python 3.8.6$ sudo apt -y install python
(省略)
$ python -V
python 2.7.18５．パッケージの導入
$ sudo apt -y install python3-pip
$ sudo apt -y install python3-dev
#-- パッケージ導入後
$ pip3 -V
pip 20.1.1 from /usr/lib/python3/dist-packages/pip (python 3.8)
$ sudo apt list python3-dev
一覧表示...完了
python3-dev/groovy,now 3.8.6-0ubuntu1 arm64 [インストール済み]$ pip3 install -U --user pip six numpy wheel setuptools mock
(省略)
ERROR: launchpadlib 1.10.13 requires testresources, which is not installed.
(省略)
#-- バージョン確認
$ /home/ubuntu/.local/bin/pip -V
pip 20.3.3 from /home/ubuntu/.local/lib/python3.8/site-packages/pip (python 3.8)
$ pip3 list --user
Package Version
---------- -------
mock 4.0.3
numpy 1.19.5
setuptools 51.1.2
wheel 0.36.2
$ pip3 list | grep six
six 1.15.0#-- launchpadlib 1.10.13 ERRORの確認
$ pip3 check
launchpadlib 1.10.13 requires testresources, which is not installed.
（訳：launchpadlibを動かすにはtestresourcesをインストールしてね）
#-- 足りないパッケージの導入
$ sudo apt -y install python3-testresoures
python3-testresoures2.0.1-2
#-- エラーが無いことを確認
$ pip3 check
No broken requirements found.
（訳：問題ないよ～）$ pip3 install -U --user keras_applications --no-deps
$ pip3 install -U --user keras_preprocessing --no-deps
#-- バージョン
keras_applications-1.0.8
keras_preprocessing-1.1.2６．Bazeliskをインストール
$ sudo apt -y install curl build-essential
$ curl -L -o bazel https://github.com/bazelbuild/bazelisk/releases/download/v1.7.4/bazelisk-linux-arm64
#-- ダウンロードが終わったら
$ chmod +x bazel
$ sudo mv bazel /usr/bin/
$ bazel version
(省略)
Bazelisk version: v1.7.4
(省略)７．ソース（TensorFlow）をダウンロード
$ git clone https://github.com/tensorflow/tensorflow.git
#-- ダウンロードが終わったらフォルダに移動
$ cd tensorflow８．ビルド設定
$ ./configure
#-- 設定は、全てデフォルトのまま（Enter連打）#--設定ログ
You have bazel 3.7.2 installed.
Please specify the location of python. [Default is /usr/bin/python3]:
Found possible Python library paths:
/usr/lib/python3/dist-packages
/usr/local/lib/python3.8/dist-packages
Please input the desired Python library path to use. Default is [/usr/lib/python3/dist-packages]
Do you wish to build TensorFlow with ROCm support? [y/N]:
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]:
No CUDA support will be enabled for TensorFlow.
Do you wish to download a fresh release of clang? (Experimental) [y/N]:
Clang will not be downloaded.
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -Wno-sign-compare]:
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:
Not configuring the WORKSPACE for Android builds.
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
--config=mkl # Build with MKL support.
--config=mkl_aarch64 # Build with oneDNN support for Aarch64.
--config=monolithic # Config for mostly static monolithic build.
--config=numa # Build with NUMA support.
--config=dynamic_kernels # (Experimental) Build kernels into separate shared objects.
--config=v2 # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
--config=noaws # Disable AWS S3 filesystem support.
--config=nogcp # Disable GCP support.
--config=nohdfs # Disable HDFS support.
--config=nonccl # Disable NVIDIA NCCL support.
Configuration finished９．ビルド開始
bazel build --local_cpu_resources=1 --verbose_failures //tensorflow/tools/pip_package:build_pip_package【オプションの説明】
--local_cpu_resources=1
CPU1でビルドをする
これを入れないと途中でシステムが停止する
※なぜフリーズするのか？原因は分からず。
知っている人教えて欲しいです。
--verbose_failures
エラー発生時にエラー箇所が分かるようにする
１０．ビルド完了（ログ）
約66時間かかりました（236996.899s）
#-- ビルドログ
INFO: Options provided by the client:
Inherited 'common' options: --isatty=1 --terminal_columns=181
INFO: Reading rc options for 'build' from /home/ubuntu/tensorflow/.bazelrc:
Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /home/ubuntu/tensorflow/.bazelrc:
'build' options: --apple_platform_type=macos --define framework_shared_object=true --java_toolchain=//third_party/toolchains/java:tf_java_toolchain --host_java_toolchain=//third_party/toolchains/java:tf_java_toolchain --define=tensorflow_enable_mlir_generated_gpu_kernels=0 --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --noincompatible_prohibit_aapt1 --enable_platform_specific_config --config=short_logs --config=v2
INFO: Reading rc options for 'build' from /home/ubuntu/tensorflow/.tf_configure.bazelrc:
'build' options: --action_env PYTHON_BIN_PATH=/usr/bin/python3 --action_env PYTHON_LIB_PATH=/usr/lib/python3/dist-packages --python_path=/usr/bin/python3 --config=xla --action_env TF_CONFIGURE_IOS=0
INFO: Found applicable config definition build:short_logs in file /home/ubuntu/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /home/ubuntu/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:xla in file /home/ubuntu/tensorflow/.bazelrc: --define=with_xla_support=true
INFO: Found applicable config definition build:linux in file /home/ubuntu/tensorflow/.bazelrc: --copt=-w --host_copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --config=dynamic_kernels
INFO: Found applicable config definition build:dynamic_kernels in file /home/ubuntu/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS
WARNING: Download from http://mirror.tensorflow.org/files.pythonhosted.org/packages/12/59/eaa15ab9710a20e22225efd042cd2d6a0b559a0656d5baba9641a2a4a921/gast-0.4.0.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
WARNING: Download from https://mirror.bazel.build/github.com/aws/aws-sdk-cpp/archive/1.7.336.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/bcbdeafa9cb3469a7bf367c17f32d913076a4d55.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
DEBUG: Rule 'io_bazel_rules_docker' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1556410077 -0400"
DEBUG: Repository io_bazel_rules_docker instantiated at:
/home/ubuntu/tensorflow/WORKSPACE:16:10: in <toplevel>
/home/ubuntu/tensorflow/tensorflow/workspace0.bzl:65:34: in workspace
/home/ubuntu/.cache/bazel/_bazel_ubuntu/ad1e09741bb4109fbc70ef8216b59ee2/external/bazel_toolchains/repositories/repositories.bzl:37:23: in repositories
Repository rule git_repository defined at:
/home/ubuntu/.cache/bazel/_bazel_ubuntu/ad1e09741bb4109fbc70ef8216b59ee2/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/www.sqlite.org/2020/sqlite-amalgamation-3340000.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //tensorflow/tools/pip_package:build_pip_package up-to-date:
bazel-bin/tensorflow/tools/pip_package/build_pip_package
INFO: Elapsed time: 236996.899s, Critical Path: 1269.85s
INFO: 15453 processes: 1334 internal, 14119 local.
INFO: Build completed successfully, 15453 total actions１１．パッケージをビルド
--user は必要ないかも
--nightly_flag を指定して、正しい依存関係を取得してパッケージを作成
$ ./bazel-bin/tensorflow/tools/pip_package/build_pip_package --user --nightly_flag /tmp/tensorflow_pkg#-- ビルドログ
2021年 1月 13日 水曜日 13:54:41 JST : === Preparing sources in dir: /tmp/tmp.CZOoh1JKou
~/tensorflow ~/tensorflow
~/tensorflow
~/tensorflow/bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles/org_tensorflow ~/tensorflow
~/tensorflow
/tmp/tmp.CZOoh1JKou/tensorflow/include ~/tensorflow
~/tensorflow
2021年 1月 13日 水曜日 13:57:05 JST : === Building wheel
warning: no files found matching 'README'
warning: no files found matching '*.pyd' under directory '*'
warning: no files found matching '*.pyi' under directory '*'
warning: no files found matching '*.pd' under directory '*'
warning: no files found matching '*.dylib' under directory '*'
warning: no files found matching '*.dll' under directory '*'
warning: no files found matching '*.lib' under directory '*'
warning: no files found matching '*.csv' under directory '*'
warning: no files found matching '*.h' under directory 'tensorflow/include/tensorflow'
warning: no files found matching '*.proto' under directory 'tensorflow/include/tensorflow'
warning: no files found matching '*' under directory 'tensorflow/include/third_party'
2021年 1月 13日 水曜日 13:58:49 JST : === Output wheel file is in: /tmp/tensorflow_pkg１２．h5pyのインストール
このパッケージを入れないとTensorFlowのインストール中に、PEP517のエラーが発生してERRORで終了してしまう。
回避策はサイトに記載されていたのでその通りに実施する。
$ sudo apt -y install libhdf5-dev１３．パッケージをインストール
$ pip3 install --user /tmp/tensorflow_pkg/tensorflow-2.5.0-cp38-cp38-linux_aarch64.whl#-- パッケージをインストールした後に、もう一度実行して出したログ
Processing /tmp/tensorflow_pkg/tf_nightly-2.5.0-cp38-cp38-linux_aarch64.whl
Requirement already satisfied: typing-extensions~=3.7.4 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (3.7.4.3)
Requirement already satisfied: gast==0.4.0 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (0.4.0)
Requirement already satisfied: flatbuffers~=1.12.0 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (1.12)
Requirement already satisfied: google-pasta~=0.2 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (0.2.0)
Requirement already satisfied: six~=1.15.0 in /usr/lib/python3/dist-packages (from tf-nightly==2.5.0) (1.15.0)
Requirement already satisfied: keras-preprocessing~=1.1.2 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (1.1.2)
Requirement already satisfied: tf-estimator-nightly~=2.5.0.dev in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (2.5.0.dev2021011301)
Requirement already satisfied: h5py~=3.1.0 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (3.1.0)
Requirement already satisfied: wheel~=0.35 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (0.36.2)
Requirement already satisfied: opt-einsum~=3.3.0 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (3.3.0)
Requirement already satisfied: termcolor~=1.1.0 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (1.1.0)
Requirement already satisfied: wrapt~=1.12.1 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (1.12.1)
Requirement already satisfied: tb-nightly~=2.5.0.a in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (2.5.0a20210113)
Requirement already satisfied: astunparse~=1.6.3 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (1.6.3)
Requirement already satisfied: numpy~=1.19.2 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (1.19.5)
Requirement already satisfied: absl-py~=0.10 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (0.11.0)
Requirement already satisfied: grpcio~=1.34.0 in ./.local/lib/python3.8/site-packages (from tf-nightly==2.5.0) (1.34.1)
Requirement already satisfied: protobuf>=3.9.2 in /usr/lib/python3/dist-packages (from tf-nightly==2.5.0) (3.12.3)
Requirement already satisfied: markdown>=2.6.8 in ./.local/lib/python3.8/site-packages (from tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (3.3.3)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in ./.local/lib/python3.8/site-packages (from tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (1.7.0)
Requirement already satisfied: werkzeug>=0.11.15 in ./.local/lib/python3.8/site-packages (from tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (1.0.1)
Requirement already satisfied: google-auth<2,>=1.6.3 in ./.local/lib/python3.8/site-packages (from tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (1.24.0)
Requirement already satisfied: setuptools>=41.0.0 in ./.local/lib/python3.8/site-packages (from tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (51.1.2)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in ./.local/lib/python3.8/site-packages (from tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (0.4.2)
Requirement already satisfied: requests<3,>=2.21.0 in /usr/lib/python3/dist-packages (from tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (2.23.0)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in ./.local/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (4.2.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/lib/python3/dist-packages (from google-auth<2,>=1.6.3->tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (0.2.1)
Requirement already satisfied: rsa<5,>=3.1.4 in ./.local/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (4.7)
Requirement already satisfied: requests-oauthlib>=0.7.0 in ./.local/lib/python3.8/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (1.3.0)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/lib/python3/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (3.1.0)
Requirement already satisfied: pyasn1>=0.1.3 in /usr/lib/python3/dist-packages (from rsa<5,>=3.1.4->google-auth<2,>=1.6.3->tb-nightly~=2.5.0.a->tf-nightly==2.5.0) (0.4.8)
tf-nightly is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.１４．Python3によるパッケージの確認
$ cd ~
$ python -c "import tensorflow as tf; print(tf.__version__)"
2.5.0１５．お世話になったサイト
今回も多数の先駆者様のお陰で実現することが出来ました。
何回も挫けそうになりましたが、どうにか形にすることが出来ました。
改めてお礼を申し上げます。
#-- TensorFlowとBazelisk
https://www.tensorflow.org/install/source?hl=ja
https://github.com/bazelbuild/bazelisk/releases
#-- Ubuntu自動サスペンド解除
https://qiita.com/ikesama200/items/f595293de82ddc9fa3f1
１６．愚痴
Googleさんの「ソースからのビルドする」サイトには、
「Raspberry Pi のソースからのビルド」ってのがあるのですが、これを使ってUbuntuでも出来るだろうと思いこんだのが間違いでした。
これだけで2週間くらい悩みました。
クロスコンパイル環境用にパソコンを購入しようかと迷ったほどです。
紆余曲折ありましたが、カタチになったので疲れも吹き飛びました。
ほんと、良かったです。
この記事が気に入ったらサポートをしてみませんか？