Saturday, December 5, 2020

How to create custom docker image for arm64

(1) When starting to use docker for arm64 architecture e.g. on M1, you might notice that there are missiing custom docker image for arm64, so there is a need to build custom image for self.

(2) When there is docker image for AMD64, you can pull them and use docker history --no-trunc to view the build commands

(3) And then create a Dockerfile to build it in your arm64 environment, it is also possibe to cross compile it in AMD64 CPU environment.

(4) For example, the creation Dockerfile to build for Quantlib juypter notebook server is as below.
P.S. You need more RAM to build using gcc, preferably 4GB to 8GB

Shell script   Select all
cd $HOME mkdir -p my-quantlib cd my-quantlib # get helloworld.ipynb wget https://raw.githubusercontent.com/lballabio/dockerfiles/master/quantlib-jupyter/Hello%20world.ipynb cat >$HOME/my-quantlib/Dockerfile_ql_1.20 <<'HEREEOF' # Dockerfile_ql_1.20 # docker build -f Dockerfile_ql_1.20 -t arm64v8/quantlib:1.20 . # docker buildx build --platform linux/arm64 -t arm64v8/quantlib:1.20 . # Build Quantlib libraries for arm64v8 ARG tag=latest FROM arm64v8/ubuntu:20.04 MAINTAINER Luigi Ballabio <luigi.ballabio@gmail.com> LABEL Description="Provide a building environment where the QuantLib Python jupyter-notebook" RUN apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential wget libbz2-dev vim git ENV boost_version=1.67.0 ENV boost_dir=boost_1_67_0 # Build boost RUN echo 'Building boost ...' #RUN wget https://dl.bintray.com/boostorg/release/${boost_version}/source/${boost_dir}.tar.gz \ RUN wget https://nchc.dl.sourceforge.net/project/boost/boost/${boost_version}/${boost_dir}.tar.gz \ && tar xfz ${boost_dir}.tar.gz \ && rm ${boost_dir}.tar.gz \ && cd ${boost_dir} \ && ./bootstrap.sh \ && ./b2 --without-python --prefix=/usr -j 4 link=shared runtime-link=shared install \ && cd .. && rm -rf ${boost_dir} && ldconfig # Build Quantlib C++ RUN echo 'Building Quantlib C++ ...' ENV quantlib_version=1.20 #RUN wget https://dl.bintray.com/quantlib/releases/QuantLib-${quantlib_version}.tar.gz \ RUN wget https://github.com/lballabio/QuantLib/releases/download/QuantLib-v1.20/QuantLib-${quantlib_version}.tar.gz \ && tar xfz QuantLib-${quantlib_version}.tar.gz \ && rm QuantLib-${quantlib_version}.tar.gz \ && cd QuantLib-${quantlib_version} \ && ./configure --prefix=/usr --disable-static CXXFLAGS=-O3 \ && make -j 4 && make check && make install \ && make clean \ && cd .. && ldconfig # && cd .. && rm -rf QuantLib-${quantlib_version} && ldconfig # Build Quantlib-Python RUN echo 'Build Quantlib-Python ...' RUN apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y swig python3 python3-pip python-dev libgomp1 # Build Quantlib for Python3 RUN echo 'Install Quantlib Python' ENV quantlib_swig_version=1.20 #RUN wget https://dl.bintray.com/quantlib/releases/QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ RUN wget https://github.com/lballabio/QuantLib-SWIG/releases/download/QuantLib-SWIG-v${quantlib_swig_version}/QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && tar xfz QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && rm QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && cd QuantLib-SWIG-${quantlib_swig_version} \ && ./configure CXXFLAGS="--param ggc-min-expand=1 --param ggc-min-heapsize=32768" PYTHON=/usr/bin/python3 \ && make -C Python && make -C Python check && make -C Python install \ && cd .. && rm -rf QuantLib-SWIG-${quantlib_swig_version} && ldconfig # Build jupyter-notebook server RUN python3 -c "print('\033[91m Building jupyter-notebook server ... \033[0m')" RUN pip3 install --no-cache-dir jupyter jupyterlab matplotlib numpy scipy pandas ipywidgets RISE RUN jupyter-nbextension install rise --py --sys-prefix RUN jupyter-nbextension install widgetsnbextension --py --sys-prefix \ && jupyter-nbextension enable widgetsnbextension --py --sys-prefix # Build Quantlib for Python2 RUN apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y python \ && apt-get clean RUN wget https://bootstrap.pypa.io/pip/2.7/get-pip.py \ && python2 get-pip.py \ && rm get-pip.py #RUN wget https://dl.bintray.com/quantlib/releases/QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ RUN wget https://github.com/lballabio/QuantLib-SWIG/releases/download/QuantLib-SWIG-v${quantlib_swig_version}/QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && tar xfz QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && rm QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && cd QuantLib-SWIG-${quantlib_swig_version} \ && ./configure CXXFLAGS="--param ggc-min-expand=1 --param ggc-min-heapsize=32768" \ && make -C Python && make -C Python check && make -C Python install \ && cd .. && rm -rf QuantLib-SWIG-${quantlib_swig_version} && ldconfig RUN pip2 install --no-cache-dir numpy EXPOSE 8888 RUN mkdir /notebooks VOLUME /notebooks COPY *.ipynb /notebooks/ # Starting jupyter-notebook server RUN python3 -c "print('\033[92m Starting jupyter-notebook server at port 8888 \033[0m')" CMD jupyter notebook --no-browser --allow-root --ip=0.0.0.0 --port=8888 --notebook-dir=/notebooks HEREEOF # build image docker build -f Dockerfile_ql_1.20 -t arm64v8/quantlib:1.20 . # run image docker run -d -p 8888:8888 --name myquantlibtesting arm64v8/quantlib:1.20 # list the token of the jupyter-notebook server docker container exec -it myquantlibtesting jupyter notebook list


(5) Testing QuantLib C++ libraries and Quantlib for Python2 and Python3
Shell script   Select all
#create and start container for testing docker run -it --rm --name myquantlib arm64v8/quantlib:1.20 /bin/bash #Create testql.cpp cd $HOME cat > testql.cpp << 'testqlEOF' #include <ql/quantlib.hpp> int main() { std::cout << "BOOST version is " << BOOST_VERSION << std::endl; std::cout << "QL version is " << QL_VERSION << std::endl; #if __x86_64__ || __WORDSIZE == 64 std::cout << "This is 64 bits" << std::endl; #elif __i386__ || __WORDSIZE == 32 std::cout << "This is 32 bits" << std::endl; #else std::cout << "This is something else" << std::endl; #endif return 0; } testqlEOF g++ testql.cpp -lQuantLib -o testql ./testql # Test QuantLib C++ Examples cd $HOME g++ /QuantLib-*/Examples/Bonds/Bonds.cpp -lQuantLib -o testBonds ./testBonds cd $HOME g++ /QuantLib-*/Examples/FRA/FRA.cpp -lQuantLib -o testFRA ./testFRA # Test python 3 QuantLib cd $HOME cat > $HOME/swap.py <<EOF from __future__ import print_function import numpy as np import QuantLib as ql print("QuantLib version is", ql.__version__) # Set Evaluation Date today = ql.Date(31,3,2015) ql.Settings.instance().setEvaluationDate(today) # Setup the yield termstructure rate = ql.SimpleQuote(0.03) rate_handle = ql.QuoteHandle(rate) dc = ql.Actual365Fixed() disc_curve = ql.FlatForward(today, rate_handle, dc) disc_curve.enableExtrapolation() hyts = ql.YieldTermStructureHandle(disc_curve) discount = np.vectorize(hyts.discount) start = ql.TARGET().advance(today, ql.Period('2D')) end = ql.TARGET().advance(start, ql.Period('10Y')) nominal = 1e7 typ = ql.VanillaSwap.Payer fixRate = 0.03 fixedLegTenor = ql.Period('1y') fixedLegBDC = ql.ModifiedFollowing fixedLegDC = ql.Thirty360(ql.Thirty360.BondBasis) index = ql.Euribor6M(ql.YieldTermStructureHandle(disc_curve)) spread = 0.0 fixedSchedule = ql.Schedule(start, end, fixedLegTenor, index.fixingCalendar(), fixedLegBDC, fixedLegBDC, ql.DateGeneration.Backward, False) floatSchedule = ql.Schedule(start, end, index.tenor(), index.fixingCalendar(), index.businessDayConvention(), index.businessDayConvention(), ql.DateGeneration.Backward, False) swap = ql.VanillaSwap(typ, nominal, fixedSchedule, fixRate, fixedLegDC, floatSchedule, index, spread, index.dayCounter()) engine = ql.DiscountingSwapEngine(ql.YieldTermStructureHandle(disc_curve)) swap.setPricingEngine(engine) print(swap.NPV()) print(swap.fairRate()) EOF # Test python3 cd $HOME python3 swap.py # Test python2 cd $HOME git clone git://github.com/mmport80/QuantLib-with-Python-Blog-Examples.git cd QuantLib-with-Python-Blog-Examples/ python2 blog_frn_example.py cd $HOME python2 swap.py