Wednesday, March 24, 2021

Differences of Mac and Linux Assembly Language

This example demo the differences of Assembly Language in Mac and Linux for x86_64 architecture. For arm64 assembly language differences, see these 2 codes in github. https://github.com/below/HelloSilicon and https://github.com/Apress/programming-with-64-bit-ARM-assembly-language

The assembly code has read and write functions. The differences of Linux and macOS are using the %ifdef MACHO64 which is the code for macOS %else is for Linux. For example the system call numbers are different and macho64 cannot use absolute addressing.
01_05-solution1.asm   Select all
; solution - answer to Chapter 1 challenge ; Read decimal input, add 10, generate output section .data prompt db "How old are you? ",0 result db "In 10 years, you will be ",0 newline db 10 ; newline, \n ten dq 10 ; used for multiplication size equ 4 ; buffer size azero equ 0x30 ; ASCII zero, '0' anine equ 0x39 ; ASCII nine, '9' nullchar equ 0x0 ; null terminator, \0 STDIN equ 0 ; standard input device STDOUT equ 1 ; standard output device %ifdef MACHO64 SYS_read equ 0x2000003 ; system call to read input macOS SYS_write equ 0x2000004 ; system call to write message macOS SYS_exit equ 0x2000001 ; system call to terminate program macOS %else SYS_read equ 0 ; system call to read input SYS_write equ 1 ; system call to write message SYS_exit equ 60 ; system call to terminate program %endif EXIT_OK equ 0 ; OK exit status TENS dq 10000 ; tens table dq 1000 dq 100 dq 10 dq 1 ; uninitialized data section .bss input_buf resb size ; age input buffer age resq 1 ; binary age value output_buf resb size ; modified age string ; code section .text global _start _start: ; output the prompt mov rsi, prompt ; prompt string output call print_string ; read input mov rbx, input_buf ; age string storage call input ; translate decimal string to binary mov rsi, input_buf ; where string value is stored call decimal2binary cmp al, 0 ; check for bogus input jz start_exit ; bail if so add rax, 10 ; add 10 to the value %ifdef MACHO64 mov [rel age], rax ; store result %else mov [age], rax ; store result %endif ; output the final prompt mov rsi, result ; First part of the string call print_string ; convert the value in 'age' to a string %ifdef MACHO64 mov rbx, [rel age] ; value %else mov rbx, [age] ; value %endif mov rsi, output_buf ; modified string call binary2decimal ; output the age-value string mov rsi, output_buf call print_value ; use this function to strip leading ; zeros ; exit program start_exit: mov rax, SYS_exit ; system exit call mov rdi, EXIT_OK syscall ; end of _start ; functions ;--------; ; output a null terminated string in rsi print_string: cmp byte [rsi], nullchar ; end-of-string test je print_string_exit ; bail on null char mov rdx, 1 ; number of characters to write mov rdi, STDOUT ; to standard output mov rax, SYS_write ; Write characters(s) syscall inc rsi ; next character jmp print_string ; keep looping print_string_exit: ret ;--------; ; grab standard input in the buffer in rbx input: mov r12, 1 ; buffer position read_char: mov rsi, rbx ; input char storage mov rdx, 1 ; character count mov rdi, STDIN ; from standard input mov rax, SYS_read ; read into rsi (rbx) syscall cmp byte [rbx], 10 ; is character read newline? je input_exit ; finish, don't store newline inc rbx ; next byte in the buffer inc r12 ; up the character count cmp r12, size ; check buffer size jl read_char ; keep looping if room ; otherwise, fall through: input_exit: mov byte [rbx], 0 ; cap the string ret ;--------; ; translate string input at rsi into binary value in rax ; if garbage input, returned value is zero decimal2binary: mov rax, 0 ; initial value d2b0: mov rbx, 0 ; initialize bax mov bl, byte [rsi] ; character, digit ; filter out non-digit values cmp bl, azero jl d2b_exit ; exit on character < '0' cmp bl, anine jg d2b_exit ; exit on character > '9' sub bl, azero ; convert from ASCII to binary %ifdef MACHO64 mul qword [rel ten] ; multiply rax by 10 %else mul qword [ten] ; multiply rax by 10 %endif add rax, rbx ; add new value inc rsi ; next char cmp byte [rsi], nullchar ; end of string? jnz d2b0 ; if not, keep looping d2b_exit: ret ;--------; ; generate a string at rsi representing the value in rbx binary2decimal: mov rdi, TENS ; reference comparision table ; values here are subtracted from ; rbx to calculate base 10 digits b2d0: xor al, al ; zero out al; al stores the character mov rcx, [rdi] ; get power of ten b2d1: or al, al ; clear carry bit (for jb) sub rbx, rcx ; subtract power of ten jb b2d2 ; if <0, the count in al is the value inc al ; decimal value++ jmp b2d1 ; keep subtracting b2d2: add al, azero ; make al ASCII add rbx, rcx ; recover from last subtraction mov byte [rsi], al ; add character to the string inc rsi ; next string position add rdi, 8 ; next value in TENS table cmp rcx, 1 ; end of table? jnz b2d0 ; loop again if not mov byte [rsi], nullchar ; terminate string ret ;--------; ; output a null-terminated string in rsi ; strip any leading zero characters print_value: cmp byte [rsi], nullchar ; always check for null char je print_value_exit ; and exit; string empty cmp byte [rsi], azero ; ASCII zero jne pv1 ; if the char isnt zero, continue inc rsi ; otherwise, check next character jmp print_value ; keep looping pv1: ; process the remaining non-zero digits cmp byte [rsi], nullchar ; null char terminator je print_value_exit ; if true, exit mov rdx, 1 ; chars to write mov rdi, STDOUT ; standard output mov rax, SYS_write ; write characters syscall inc rsi ; next char jmp pv1 ; loop until null char print_value_exit: mov rsi, newline ; newline defined as \n mov rdx, 1 ; write 1 char mov rax, SYS_write ; write character mov rdi, STDOUT ; to standard output syscall ret ; end


makefile for Linux
makefile   Select all
#makefile for Linux filename_prefix := 01_05- src_files := $(wildcard *.asm) obj_files := $(src_files:.asm=.o) prog_files := $(subst $(filename_prefix), ,$(basename $(src_files))) all: $(prog_files) $(prog_files): % : $(filename_prefix)%.o ld -o $@ $< -arch x86_64 $(filter %.o,$(obj_files)): %.o : %.asm nasm -g -f elf64 $< -o $@ .PHONY: clean clean: rm -f $(obj_files) $(prog_files)


makefile for macOS
makefile    Select all
#makefile for macOS filename_prefix := 01_05- src_files := $(wildcard *.asm) obj_files := $(src_files:.asm=.o) prog_files := $(subst $(filename_prefix), ,$(basename $(src_files))) all: $(prog_files) $(prog_files): % : $(filename_prefix)%.o ld -no_pie -macosx_version_min 11.0.0 -o $@ $< -lSystem -syslibroot `xcrun -sdk macosx --show-sdk-path` -e _start -arch x86_64 codesign --entitlement entitlements --force -s - $@ $(filter %.o,$(obj_files)): %.o : %.asm nasm -g -f macho64 -dMACHO64 $< -o $@ .PHONY: clean clean: rm -f $(obj_files) $(prog_files)


In order to debug in macOS, codesign entitlement file is needed.
entitlements   Select all
<?xml version="1.0" encoding="UTF-8"> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>com.apple.security.get-task-allow</key> <true/> </dict> </plist>


For linux, it is required to install these packages "sudo apt install build-essentials nasm gdb lldb"

For macOS, it is required to install Xcode and Command Line Tools and Homebrew package "brew install nasm"
It is possible to build, run and debug x86_64 program on M1 Macs, if Rosetta is installed.


Sunday, March 21, 2021

How to install Swift 5 compiler for Android Termux App

(1) Installation of packages
pkg install clang
pkg install swift


(2) Test Grand Central Dispatch - concurrent queue
Shell script   Select all
cd $HOME cat >$HOME/concurrentqueue.swift <<'HEREEOF' /* # Concurrent Programming with Grand Central Dispatch in Swift 5 */ import Foundation let globalQueue = DispatchQueue.global(qos: .userInitiated) func trace(task: Int) { // Generates a random integer in the [0, task] range print("Task \(task) started") sleep( UInt32.random(in: 0...UInt32(task)) ) print("Task \(task) completed") } print("Concurrent queue, synchronous execution") for i in 1...5 { print("Submitting task \(i)") globalQueue.sync { trace(task: i) } } print("\nConcurrent queue, asynchronous execution") for i in 6...9 { print("Submitting task \(i)") globalQueue.async { trace(task: i) } } sleep(10) print("Program ended") HEREEOF # test swiftc concurrentqueue.swift ./concurrentqueue


(3) Test Grand Central Dispatch - serial queue
Shell script   Select all
cd $HOME cat >$HOME/serialqueue.swift <<'HEREEOF' /* # Concurrent Programming with Grand Central Dispatch in Swift 5 */ import Foundation let serialQueue = DispatchQueue(label: "com.mycompany.demo.serial") func trace(task: Int) { // Generates a random integer in the [0, task] range print("Task \(task) started") sleep( UInt32.random(in: 0...UInt32(task)) ) print("Task \(task) completed") } print("\nSerial queue, synchronous execution") for i in 1...5 { print("Submitting task \(i)") serialQueue.sync { trace(task: i) } } print("\nSerial queue, asynchronous execution") for i in 6...9 { print("Submitting task \(i)") serialQueue.async { trace(task: i) } } sleep(25) print("Program ended") HEREEOF # test swiftc serialqueue.swift ./serialqueue


Friday, March 19, 2021

How to use m1 Mac mini to do mining

(1) download m1 ethminer here
curl -OL https://github.com/gyf304/ethminer-m1/releases/download/v0.19.0-alpha.0-m1/ethminer-m1
chmod +x ethminer-m1

(2) Download a wallet app (e.g. Blockchain) and obtain Ether Wallet Address

(3) start to mine by joining pool
./ethminer-m1 -P stratum1+tcp://0x601913a35B0f5A599271506773ae25BF1858e92b@asia1.ethermine.org:4444

If you like this article pls consider Ethereum Donations:
0x601913a35B0f5A599271506773ae25BF1858e92b

Sunday, February 28, 2021

How to cross compile using Docker Desktop experimental feature buildx

Another demo to show the building of packages using experiemtal feature of Docker desktop. Have to enable the experimental feature to cross compile in other architetcures different from the host machine. E.g. cross compile armv7 packages in amd64 or arm64 host of Mac or PC.
Shell script   Select all
cd $HOME mkdir -p my-quantlib cd my-quantlib # get helloworld.ipynb wget https://raw.githubusercontent.com/lballabio/dockerfiles/master/quantlib-jupyter/Hello%20world.ipynb cat >$HOME/my-quantlib/Dockerfile_ql_armv7_1.21 <<'HEREEOF' # Dockerfile_ql_armv7_1.21 # docker buildx build --platform linux/arm/v7 -f Dockerfile_ql_armv7_1.21 -t armv7/quantlib:1.21 . # Build Quantlib libraries for armv7 in arm64/amd64 host ARG tag=latest FROM arm32v7/ubuntu:18.04 LABEL Description="Provide a building environment where the QuantLib Python jupyter-notebook" RUN apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential wget libbz2-dev vim git ENV boost_version=1.67.0 ENV boost_dir=boost_1_67_0 # Build boost RUN echo 'Building boost ...' RUN wget --no-check-certificate https://boostorg.jfrog.i o/artifactory/main/release/${boost_version}/source/${boo st_dir}.tar.gz \ && tar xfz ${boost_dir}.tar.gz \ && rm ${boost_dir}.tar.gz \ && cd ${boost_dir} \ && ./bootstrap.sh \ && ./b2 --without-python --prefix=/usr -j 4 link=shared runtime-link=shared install \ && ./b2 --prefix=/Staging/usr install \ && cd .. && rm -rf ${boost_dir} && ldconfig # Build Quantlib C++ RUN echo 'Building Quantlib C++ ...' ENV quantlib_version=1.21 RUN wget https://github.com/lballabio/QuantLib/releases/download/QuantLib-v${quantlib_version}/QuantLib-${quantlib_version}.tar.gz \ && tar xfz QuantLib-${quantlib_version}.tar.gz \ && rm QuantLib-${quantlib_version}.tar.gz \ && cd QuantLib-${quantlib_version} \ && ./configure --prefix=/usr --disable-static CXXFLAGS=-O3 \ && make -j 4 && make install \ && make DESTDIR=/Staging install \ && make clean \ && cd .. && ldconfig # && cd .. && rm -rf QuantLib-${quantlib_version} && ldconfig # Build Quantlib-Python RUN echo 'Build Quantlib-Python ...' RUN apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y swig python3 python3-pip python-dev libgomp1 # Build Quantlib for Python3 RUN echo 'Install Quantlib Python' ENV quantlib_swig_version=1.21 RUN wget https://github.com/lballabio/QuantLib-SWIG/releases/download/QuantLib-SWIG-v${quantlib_swig_version}/QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && tar xfz QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && rm QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && cd QuantLib-SWIG-${quantlib_swig_version} \ && ./configure CXXFLAGS="--param ggc-min-expand=1 --param ggc-min-heapsize=32768" PYTHON=/usr/bin/python3 \ && make -C Python && make -C Python check && make -C Python install \ && cd .. && rm -rf QuantLib-SWIG-${quantlib_swig_version} && ldconfig # Build jupyter-notebook server RUN python3 -c "print('\033[91m Building jupyter-notebook server ... \033[0m')" RUN pip3 install --no-cache-dir jupyter jupyterlab matplotlib numpy scipy pandas ipywidgets RISE RUN jupyter-nbextension install rise --py --sys-prefix RUN jupyter-nbextension install widgetsnbextension --py --sys-prefix \ && jupyter-nbextension enable widgetsnbextension --py --sys-prefix # Build Quantlib for Python2 RUN apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y python \ && apt-get clean RUN wget https://bootstrap.pypa.io/2.7/get-pip.py \ && python2 get-pip.py \ && rm get-pip.py RUN wget https://dl.bintray.com/quantlib/releases/QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && tar xfz QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && rm QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && cd QuantLib-SWIG-${quantlib_swig_version} \ && ./configure CXXFLAGS="--param ggc-min-expand=1 --param ggc-min-heapsize=32768" \ && make -C Python && make -C Python check && make -C Python install \ && cd .. && rm -rf QuantLib-SWIG-${quantlib_swig_version} && ldconfig RUN pip2 install --no-cache-dir numpy EXPOSE 8888 RUN mkdir /notebooks VOLUME /notebooks COPY *.ipynb /notebooks/ # Starting jupyter-notebook server RUN python3 -c "print('\033[92m Starting jupyter-notebook server at port 8888 \033[0m')" CMD jupyter notebook --no-browser --allow-root --ip=0.0.0.0 --port=8888 --notebook-dir=/notebooks HEREEOF # build and load image (after experimental feature of Docker Desktop is enabled docker buildx build --platform linux/arm/v7 --memory="8g" --output "type=docker,push=false,dest=armv7_ql.tar" -f Dockerfile_ql_armv7_1.21 -t armv7/quantlib:1.21 . docker load < armv7_ql.tar # run image docker run --platform linux/arm/v7 -d -p 8888:8888 --name myquantlibarmv7testing armv7/quantlib:1.21 # list the token of the jupyter-notebook server docker container exec -it myquantlibarmv7testing jupyter notebook list


If using WSL2 docker CLI in Linux, should enable the experimental feature first.
export DOCKER_CLI_EXPERIMENTAL=enabled
docker buildx create --name mybuilder
docker buildx use mybuilder
docker buildx inspect --bootstrap
docker buildx build --platform linux/arm/v7 --memory="8g" --output "type=docker,push=false,dest=armv7_ql.tar" -f Dockerfile_ql_armv7_1.21 -t armv7/quantlib:1.21 .


Thursday, February 25, 2021

How to cross compile QuantLib in docker

This example demo using dockcross to build Quantlib armv7 library packages and cross compile in host X86_64 machine
Shell script   Select all
cd $HOME mkdir -p dockcross cd dockcross cat >$HOME/dockcross/dockcross_ql_1.21 <<'HEREEOF' # dockcross_ql_1.21 # docker build -f dockcross_ql_1.21 -t dockcross/linux-armv7/quantlib:1.21 . # Build Quantlib libraries for armv7 ARG tag=latest FROM dockcross/linux-armv7 ENV DEFAULT_DOCKCROSS_IMAGE dockcross/linux-armv7/quantlib:1.21 ENV CROSS_PREFIX /usr/xcc/armv7-unknown-linux-gnueabi/armv7-unknown-linux-gnueabi/sysroot/usr LABEL Description="Provide a building environment dockcross/linux/armv7 for the QuantLib" RUN apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential wget vim git ENV boost_version=1.67.0 ENV boost_dir=boost_1_67_0 # Install boost dependencies RUN echo 'Building boost and required packages ...' RUN dpkg --add-architecture armhf \ && apt-get update \ && apt-get download libbz2-1.0:armhf libbz2-dev:armhf liblzma-dev:armhf \ && dpkg-deb -x libbz2-1*armhf.deb ./x \ && dpkg-deb -x libbz2-dev_*armhf.deb ./x \ && dpkg-deb -x liblzma-dev_*armhf.deb ./x \ && mv ./x/usr/lib/arm-linux-gnueabihf/* ${CROSS_PREFIX}/lib/ \ && mv ./x/usr/bin/* ${CROSS_PREFIX}/bin/ \ && mv ./x/usr/include/* ${CROSS_PREFIX}/include/ \ && mv ./x/usr/share/* ${CROSS_PREFIX}/share/ \ && mv ./x/lib/arm-linux-gnueabihf/* ${CROSS_PREFIX}/lib/ \ && rm -fr ./x *.deb && ldconfig RUN wget https://sourceforge.net/projects/libpng/files/zlib/1.2.11/zlib-1.2.11.tar.gz/download -O zlib-1.2.11.tar.gz \ && tar xfz zlib-1.2.11.tar.gz \ && rm zlib-1.2.11.tar.gz \ && cd zlib-1.2.11 \ && ./configure --prefix=${CROSS_PREFIX} && make && make install \ && cd .. && rm -rf zlib-1.2.11 && ldconfig # Build boost RUN wget https://dl.bintray.com/boostorg/release/${boost_version}/source/${boost_dir}.tar.gz \ && tar xfz ${boost_dir}.tar.gz \ && rm ${boost_dir}.tar.gz \ && cd ${boost_dir} \ && ./bootstrap.sh --with-toolset=gcc --prefix=${CROSS_PREFIX} \ && touch user-config.jam \ && echo "using gcc : armv7 : ${CXX} ;" > user-config.jam \ && echo "using mpi ;" >> user-config.jam \ && ./b2 --toolset=gcc-armv7 --address-model=32 --architecture=arm --user-config=./user-config.jam --without-python --prefix=${CROSS_PREFIX} -j 4 link=shared runtime-link=shared install \ && cd .. && rm -rf ${boost_dir} && ldconfig # Build Quantlib C++ RUN echo 'Building Quantlib C++ ...' ENV quantlib_version=1.21 RUN wget https://dl.bintray.com/quantlib/releases/QuantLib-${quantlib_version}.tar.gz \ && tar xfz QuantLib-${quantlib_version}.tar.gz \ && rm QuantLib-${quantlib_version}.tar.gz \ && cd QuantLib-${quantlib_version} \ && ./configure --host=x86_64-linux-gnu --target=armv7-unknown-linux-gnueabi --with-boost-include=${CROSS_PREFIX}/include/boost --with-boost-lib=${CROSS_PREFIX}/lib --prefix=${CROSS_PREFIX} --disable-static CXXFLAGS=-O3 \ && make -j 4 && make install \ && make clean \ && cd .. && ldconfig HEREEOF # build image docker build -f dockcross_ql_1.21 -t dockcross/linux-armv7/quantlib:1.21 . # Test dockcross compile docker run --rm dockcross/linux-armv7/quantlib:1.21 > dockcross-quantlib-armv7 chmod +x dockcross-quantlib-armv7 ./dockcross-quantlib-armv7 bash -c '$CXX -v' # Copy example cpp file from image docker run -v $PWD:/opt/mount --rm --entrypoint cp dockcross/linux-armv7/quantlib:1.21 /work/QuantLib-1.21/Examples/Bonds/Bonds.cpp /opt/mount/Bonds.cpp # Use this command to copy whole directory recursively # docker run -v $PWD:/opt/mount --rm --entrypoint cp dockcross/linux-armv7/quantlib:1.21 -r /work/QuantLib-1.21/Examples /opt/mount/QLExamples # Compile and check binary ./dockcross-quantlib-armv7 bash -c '$CXX Bonds.cpp -lQuantLib -o Bonds -static' file Bonds


# install qemu and run in host
sudo apt update
sudo apt install -y qemu-user
qemu-arm Bonds

# Or test run it in armv7 machine e.g. RaspberryPi 3B+
scp Bonds mypi3b:~/.
ssh mypi3b '~/Bonds'

Sunday, February 14, 2021

How to install Windows Subsystem for Linux 2 and docker on Windows 10

Required Windows 10 update 1903 or 1909 or above
(1) Use Windows Power Shell (run as Administrator) to install WSL2
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
# for Windows update 2004 or later
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
# for Windows update 1903 or 1909
Enable-WindowsOptionalFeature -Online -FeatureName VirtualMachinePlatform -NoRestart
# Restart Windows after enable Enable-WindowsOptionalFeature curl -o $env:userprofile\Desktop\wsl_update_x64.msi https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi
msiexec.exe /I $env:userprofile\Desktop\wsl_update_x64.msi /quiet
wsl --set-default-version 2


(2) Install Ubuntu 20.04 from Windows Store

(3) Install docker (Reference : https://docs.docker.com/engine/install/ubuntu/)
sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl gnupg-agent software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io
sudo usermod -aG docker $USER

# Test docker
sudo /etc/init.d/docker start
docker run -e MYSQL_ROOT_PASSWORD=rootpassword -e MYSQL_USER=wpuser -e MYSQL_PASSWORD=wpuserpassword -e MYSQL_DATABASE=wordpressdb --name wordpressdb -d mariadb ; sudo docker run -e WORDPRESS_DB_USER=wpuser -e WORDPRESS_DB_PASSWORD=wpuserpassword -e WORDPRESS_DB_NAME=wordpressdb -p 8080:80 --link wordpressdb:mysql --name wordpress -d wordpress
# browser test enter address http://127.0.0.1:8080/


Test (4) Install docker-compose
sudo apt install python3.8 python3-pip
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 10
sudo update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 10
sudo pip3 -v install docker-compose


(5) For example, the creation Dockerfile to build for Quantlib juypter notebook server is as below.
P.S. You need 8G RAM to build thia dockerfile using gcc

Shell script   Select all
cd $HOME mkdir -p my-quantlib cd my-quantlib # get helloworld.ipynb wget https://raw.githubusercontent.com/lballabio/dockerfiles/master/quantlib-jupyter/Hello%20world.ipynb cat >$HOME/my-quantlib/Dockerfile_ql_1.21 <<'HEREEOF' # Dockerfile_ql_1.21 # docker build -f Dockerfile_ql_1.21 -t quantlib:1.21 . # Build Quantlib libraries for amd64 ARG tag=latest FROM ubuntu:20.04 MAINTAINER Luigi Ballabio LABEL Description="Provide a building environment where the QuantLib Python jupyter-notebook" RUN apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential wget libbz2-dev vim git ENV boost_version=1.67.0 ENV boost_dir=boost_1_67_0 # Build boost RUN echo 'Building boost ...' RUN wget https://dl.bintray.com/boostorg/release/${boost_version}/source/${boost_dir}.tar.gz \ && tar xfz ${boost_dir}.tar.gz \ && rm ${boost_dir}.tar.gz \ && cd ${boost_dir} \ && ./bootstrap.sh \ && ./b2 --without-python --prefix=/usr -j 4 link=shared runtime-link=shared install \ && cd .. && rm -rf ${boost_dir} && ldconfig # Build Quantlib C++ RUN echo 'Building Quantlib C++ ...' ENV quantlib_version=1.21 RUN wget https://dl.bintray.com/quantlib/releases/QuantLib-${quantlib_version}.tar.gz \ && tar xfz QuantLib-${quantlib_version}.tar.gz \ && rm QuantLib-${quantlib_version}.tar.gz \ && cd QuantLib-${quantlib_version} \ && ./configure --prefix=/usr --disable-static CXXFLAGS=-O3 \ && make -j 4 && make check && make install \ && make clean \ && cd .. && ldconfig # && cd .. && rm -rf QuantLib-${quantlib_version} && ldconfig # Build Quantlib-Python RUN echo 'Build Quantlib-Python ...' RUN apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y swig python3 python3-pip python-dev libgomp1 # Build Quantlib for Python3 RUN echo 'Install Quantlib Python' ENV quantlib_swig_version=1.21 RUN wget https://dl.bintray.com/quantlib/releases/QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && tar xfz QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && rm QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && cd QuantLib-SWIG-${quantlib_swig_version} \ && ./configure CXXFLAGS="--param ggc-min-expand=1 --param ggc-min-heapsize=32768" PYTHON=/usr/bin/python3 \ && make -C Python && make -C Python check && make -C Python install \ && cd .. && rm -rf QuantLib-SWIG-${quantlib_swig_version} && ldconfig # Build jupyter-notebook server RUN python3 -c "print('\033[91m Building jupyter-notebook server ... \033[0m')" RUN pip3 install --no-cache-dir jupyter jupyterlab matplotlib numpy scipy pandas ipywidgets RISE RUN jupyter-nbextension install rise --py --sys-prefix RUN jupyter-nbextension install widgetsnbextension --py --sys-prefix \ && jupyter-nbextension enable widgetsnbextension --py --sys-prefix # Build Quantlib for Python2 RUN apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y python2 \ && apt-get clean RUN wget https://bootstrap.pypa.io/2.7/get-pip.py \ && python2 get-pip.py \ && rm get-pip.py RUN wget https://dl.bintray.com/quantlib/releases/QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && tar xfz QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && rm QuantLib-SWIG-${quantlib_swig_version}.tar.gz \ && cd QuantLib-SWIG-${quantlib_swig_version} \ && ./configure CXXFLAGS="--param ggc-min-expand=1 --param ggc-min-heapsize=32768" \ && make -C Python && make -C Python check && make -C Python install \ && cd .. && rm -rf QuantLib-SWIG-${quantlib_swig_version} && ldconfig RUN pip2 install --no-cache-dir numpy EXPOSE 8888 RUN mkdir /notebooks VOLUME /notebooks COPY *.ipynb /notebooks/ # Starting jupyter-notebook server RUN python3 -c "print('\033[92m Starting jupyter-notebook server at port 8888 \033[0m')" CMD jupyter notebook --no-browser --allow-root --ip=0.0.0.0 --port=8888 --notebook-dir=/notebooks HEREEOF # build image docker build -f Dockerfile_ql_1.21 -t quantlib:1.21 . # run image docker run -d -p 8888:8888 --name myquantlibtesting quantlib:1.21 # list the token of the jupyter-notebook server docker container exec -it myquantlibtesting jupyter notebook list


(6) Install youtube-dl for WSL2
wget https://github.com/ytdl-org/youtube-dl/releases/download/2021.12.17/youtube-dl
sudo chmod +x youtube-dl sudo mv youtube-dl /usr/local/bin/ sudo apt install -y openssl ffmpeg
sudo apt install python-is-python3
youtube-dl -U
youtube-dl -i -f m4a https://youtu.be/e0npW4WoGmc
# use ffmpeg to resize videp https://ottverse.com/change-resolution-resize-scale-video-using-ffmpeg/ # use ffmpeg to resize image https://stackoverflow.com/questions/28806816/use-ffmpeg-to-resize-image


(7) Install X Display Server for WSL gui app
https://techcommunity.microsoft.com/t5/windows-dev-appconsult/running-wsl-gui-apps-on-windows-10/ba-p/1493242

Run wsl app batch file
https://gist.githubusercontent.com/zarinfam/eb671a82340eadb5af9026ba6e0b666b/raw/c2d0d947a488e54eeb7fdaafae5464afc778de71/wsl-app-runner.bat

vbscript to start the batch file
https://gist.github.com/zarinfam/5dbc0e10662b79468a3da9a67c107217/raw/cf4d2d3b6095cb3c42fca0df9e86e4d3b2703671/linux-gui-app-runner.vbs



To install OpenSSH Server on Windows 10 https://virtualizationreview.com/articles/2020/05/21/ssh-server-on-windows-10.aspx

To enable ssh login default to wsl ubuntu. Should use powershell with admin authority yo rmable features. https://www.hanselman.com/blog/the-easy-way-how-to-ssh-into-bash-and-wsl2-on-windows-10-from-an-external-machine

In order not to interrupt the ssh server service, should consider to disable sleep when on power adapter. And disable automatic windows update and restart.



Monday, January 18, 2021

Hello World Assembly code for Termux App

There is an article on M1 helloworld assembly language code on tge new Mac M1 hardware. https://smist08.wordpress.com/2021/01/08/apple-m1-assembly-language-hello-world/

The system call table can be referred to this in Mac /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/syscall.h
For the BSD system calls method please refer to https://sigsegv.pl/osx-bsd-syscalls/
whereas #1 is exit system call and #4 is write system call

For the complete ARM64 programming examples for M1 Mac, please refer to this.
https://github.com/below/HelloSilicon

It is important to learn debug skill through assembly language and for mac use lldb to debug, e.g.
(lldb) breakpoint set -f HelloWorld.s -l 14
(lldb) run
(lldb) step
(lldb) register read x16 x0 x1 x2

In order to debug on Mac, the program first must add -g option when compiled/asembled(as) and then must be codesigned and add this codesign command to the makefile
codesign --entitlements entitlements.plist --force -s - $@
entitlements.plist add this key.
<key>com.apple.security.get-task-allow</key>
<true/>


What about Android Termux App?
pkg install clang
wget https://raw.githubusercontent.com/matja/asm-examples/master/aarch64/hello.aarch64.linux.syscall.gas.asm
gcc -nostdlib -static -nostartfiles -Wl,--entry=_start hello.aarch64.linux.syscall.gas.asm -o hello
./hello

What about gbd debug ?
gcc -nostdlib -static -nostartfiles -Wl,--entry=_start hello.aarch64.linux.syscall.gas.asm -g -o hello
pkg install gdb
objdump -d hello
gdb hello
(gdb) break 1 # set breakpoint
(gdb) run # run
(gdb) step # step
(gdb) info reg general # exam register

ARM Architecture Basic
x0-x30 are 64-bit registers
svc 0 is the system call
x8 determines what we do, e.g. #64 write and #93 is exit (for other system call numbers please refer to document)
x8 determines what we do, e.g. #64 write and #93 is exit (for other system call numbers please refer to document)
x8 determines what we do, e.g. #64 is write and #93 is exit (for other system call numbers please refer to this document)
x0-x4 determines how we do it and the required parameters are also documented in the document above.