How to build tensorflow in Arch Linux

1) The instructions to build tensorflow is here https://www.tensorflow.org/install/source#ubuntu

2) It is important to use the same GCC version and Bazel version for the compilation work as listed in the guide. E.g. to build tensorflow 2.0, use the correct version in order to avoid failure.
tensorflow-2.0.0 2.7, 3.3-3.7 GCC 7.3.1 Bazel 0.26.1

3) As the compilation is very cpu intensive and may took several hours, it is also important to prepare a faster and with more cores machine for this job and avoid to use docker image.

4) The steps are as below and need to setup the Arch Linux and setup the wifi connection as in my previous post.

history.txt    Select all

Assume running as root
Connect to Wifi and check status
001 netctl list 002 netctl start wlan0-MyNetwork 003 ifconfig wlan0 Change to bash shell
005 chsh -s /bin/bash 006 exec bash Download GCC 7.3.1 and libs from
https://archive.org/download/archlinux_pkg_gcc7-libs https://archive.org/download/archlinux_pkg_gcc7/ Install GCC 7.3.1 and other dependencies
010 pacman -Sy 011 pacman -U --noconfirm https://archive.org/download/archlinux_pkg_gcc7/gcc7-7.3.1%2B20180406-2-x86_64.pkg.tar.xz https://archive.org/download/archlinux_pkg_gcc7-libs/gcc7-libs-7.3.1%2B20180406-2-x86_64.pkg.tar.xz 012 ln -sf /usr/bin/gcc-7 /usr/local/bin/gcc 013 ln -sf /usr/bin/g++-7 /usr/local/bin/g++ 014 ln -sf /usr/bin/cc-7 /usr/local/bin/cc 015 ln -sf /usr/bin/cpp-7 /usr/local/bin/cpp 016 ln -sf /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.1/cc1 /usr/local/lib/. 017 ln -sf /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.1/cc1plus /usr/local/lib/. 018 hash -r 019 gcc --version Download tensorflow code and checkout r2.0
020 cd ~ 021 pacman -S --noconfirm git patch 022 git clone https://github.com/tensorflow/tensorflow.git 023 cd tensorflow/ 024 git checkout r2.0 Check the Bazel version requirement from the tensorflow code
025 grep TF_M[AI][XN] configure.py Download Bazel Installer from https://github.com/bazelbuild/bazel/releases?after=0.27.1
026 cd ~ 027 curl -OL https://github.com/bazelbuild/bazel/releases/download/0.26.1/bazel-0.26.1-installer-linux-x86_64.sh Install Bazel 0.26.1
028 pacman -S --noconfirm unzip which curl 029 /bin/bash bazel-0.26.1-installer-linux-x86_64.sh 030 source /usr/local/lib/bazel/bin/bazel-complete.bash 031 bazel version Download and Install Anaconda python from https://repo.anaconda.com/archive/
037 cd ~ 038 curl -OL https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh Ensure using bash shell
042 echo $0 043 /bin/bash Anaconda3-2020.02-Linux-x86_64.sh 044 exec bash Set up Anaconda and install the required python packages in virtual env
050 conda config --set auto_activate_base false 051 conda deactivate 052 conda update conda 053 conda update anaconda 054 conda update python 055 conda update --all 056 conda create --name tf-py37 057 conda activate tf-py37 058 conda install pip six numpy wheel setuptools mock 'future>=0.17.1' python=3.7 059 conda install protobuf==3.6.1 --no-deps 060 pip install keras_applications --no-deps 061 pip install keras_preprocessing --no-deps Build tensorflow
062 cd ~ 063 cd tensorflow 064 git checkout r2.0 065 ./configure 070 bazel shutdown && bazel clean 071 bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package There might be error for gettid in grpc, and need to download patch and apply it. It is possible to press control-alt-F2 (mac is Control-fn-option-F2) to start another terminal while bazel building in progress.
080 cd ~ 081 curl -OL https://gist.githubusercontent.com/drscotthawley/8eb51af1b4c92c4f18432cb045698af7/raw/a4aaa020f0434c58b08e453e6d501553ceafbc81/grpc.patch 082 patch -p2 --directory='tensorflow/bazel-tensorflow/external/grpc/src' < ~/grpc.patch To review the patch before and after
085 grep -RIH 'gettid' --include="*.cc" - --exclude-dir={git,log,assets} ~/.cache/bazel/*/*/external/grpc 086 grep -RIH 'sys_gettid' --include="*.cc" --exclude-dir={git,log,assets} ~/.cache/bazel/*/*/external/grpc Alternatively, to do the patch using sed, but use with care
087 grep -RIl 'gettid' --include="*.cc" --exclude-dir={git,log,assets} ~/.cache/bazel/*/*/external/grpc | xargs sed -i 's/gettid/sys_gettid/g' Continue building the package
083 cd tensorflow 084 bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package 085 bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg Shell script utilities
100 curl -L https://tinyurl.com/BuildTensorflow | grep " [0-1][0-9][0-9] " > ~/history.txt 101 source <(grep -m 1 " 011 " ~/history.txt | cut -c8-) 102 source <(grep " 01[2-9] " ~/history.txt | cut -c8-) 103 history -ps $(grep -m 1 " 017 " ~/history.txt | cut -c8-)

5) How to use this script in Arch Linux, first create a history.txt with the content in step 4, use scp or samba to copy the history.txt to the Arch Linux environment.
Or simply download this post html page to the environment.
curl -L https://tinyurl.com/BuildTensorflow | grep " [0-1][0-9][0-9] " > ~/history.txt
or with comments
curl -L https://tinyurl.com/BuildTensorflow | grep -A105 ">Assume running as root" | sed -e 's/<[^>]*>//g' > ~/history.txt

shell script    Select all

Run the command as in line 011 of the history.txt file source <(grep -m 1 " 011 " ~/history.txt | cut -c8-)

Run several commands as in line 012 to Line 019 of the history.txt file source <(grep " 01[2-9] " ~/history.txt | cut -c8-)

Add line 017 of history.txt to bash history without execution
so as to use arrow up to edit the command history before execution. history -ps $(grep -m 1 " 017 " ~/history.txt | cut -c8-)