Wednesday, June 22, 2016

How to build Node.js REST APIs in Raspberry Pi 3

This project is based on the "Build a REST API with node.js" video tutorial from Udemy


The project file can be downloaded here : https://mega.nz/#!z9oFlTgJ!geCPHjoXJ3rpm1OdldU8s5RBSJxLKiAxh_axvTbODB4

The node.js ARMv7 version is v4.4.3 and npm 2.15.1
which (ARMv7) can be downloaded from https://nodejs.org/en/download/
The project also use packages like bcryptjs, body-parser, express, jsonwebtoken and mongoose and integrate with mongodb. - No-SQL db.

To install mongodb in Raspberrry Pi, use sudo apt-get install mongodb

The following enhancements were made to the original project code

1. add shell scripts and use curl for API testing (test1.sh ... test6.sh), as POSTMAN is only available in google chrome
2. add js code to avoid creating of duplicated user (test using test2.sh)
3. add test to user login and obtain Authorization sessionToken (test using test3.sh)
4. add js code to check Authorization before CRUD (create, read, update and delete definition record)
5. add js code to check existence of definition id before update and delete record (test using test5.sh)
6. shell script to get sessionToken in each test for instruction to further test on update and delete definition (test using test5.sh)
7. add js code to create log when create, update and delete definition record (test using test4.sh / test5.sh and then test6.sh)

To use mongo to admin user
mongo
> use workouts
> db.addUser({ user: "test", pwd: "test", roles:["readWrite", "dbAdmin"]});

or use createUser as below
> db.createUser({ user: "test", pwd: "test", roles:["readWrite", "dbAdmin"]});

To use mongo to display mongodb info
mongo
> use workouts
> show collections
> db.users.find()
> db.definitions.find().pretty()
> db.logs.find().pretty()

To use mongo to remove object
> db.definitions.remove({_id:ObjectId("573c29c0f11288dc9997d0")})

Instructions to use
-----------------------
1. First install node.js and mongodb and check that if mongod is running successfully in background. (If you use windows / OSX version of mongodb, please make sure the required data path is created before running mongod.exe / mongod)
2. use mongo to create admin user as above
3. Download the project file and unzip and install packages and start
bash    Select all
unzip nodeapi.zip cd nodeapi npm install npm start & # test ./test1.sh ./test2.sh ./test3.sh ... ./test6.sh # kill background job kill $!



Tuesday, June 21, 2016

How to run Docker for macOS

1) Download from here -> https://docs.docker.com/docker-for-mac/

2) Drag to Application Folder and double click to start and then check version in Terminal
docker --version
docker info


3) Search and Pull Docker image
E.g.
docker search kitura
docker pull ibmcom/kitura-ubuntu:latest


4) Run image with port forwarding <local port : container port>
E.g.
docker run --cap-add sys_ptrace -p 8095:8095 -t -i ibmcom/kitura-ubuntu:latest /bin/bash

5) Run / Download (if none) additional images and run as daemon
docker run -d -p 8080:8080 --name nodejs node
docker run -d -p 80:80 --name webserver nginx


6) List all downloaded or committed images
docker images

Download old images from
http://mirror.yandex.ru/mirrors/download.openvz.org/template/precreated/
and using import
cat ~/Downloads/ubuntu-10.04-x86.tar.gz | docker import - ubuntu-x86:10.04
or
docker import ubuntu-10.04-x86.tar.gz ubuntu-x86:10.04

docker run -t -i ubuntu-x86:10.04 bash


7) Run image with port forwarding <local port : container port> and bash shell
docker run -p 8095:8095 -t -i ibmcom/kitura-ubuntu:latest bash
docker run -p 80:80 -t -i nginx:latest bash
docker run -p 3000:3000 -t -i node:latest bash
docker run -t -i lballabio/quantlib:latest bash
docker run -t -i lballabio/quantlib-python:latest bash


8) List all containers with <CONTAINER ID>
docker ps -a
#export container
docker export <CONTAINER ID> > ubuntu.tar


Execute container with bash shell
docker exec -t -i <CONTAINER ID> bash

9) Commit changes to image with <newname:newtag>
docker commit <CONTAINER ID> kitura-ubuntu:v2
docker run -p 8095:8095 -it kitura-ubuntu:v2 /bin/bash

This is useful when you cannot start a container manually after docker upgraded. e.g.
docker start <CONTAINER ID> && docker attach <CONTAINER ID>


10) copy docker file to local docker start <CONTAINER ID>
docker cp <CONTAINER ID>:/root/myfile.tgz .




Docker for Mac OSX data is at
~/Library/Containers/com.docker.docker/Data



To test the kitura docker image for Swift web app
shell script    Select all
mkdir $HOME/play cd $HOME/play #swift build --init swift package init cat > $HOME/play/Package.swift <<EOF import PackageDescription let package = Package( name: "play", dependencies: [ .Package(url: "https://github.com/IBM-Swift/Kitura.git", majorVersion:0, minor:15), .Package(url: "https://github.com/OpenKitten/MongoKitten.git", majorVersion: 1, minor: 1) ] ) EOF mkdir -p $HOME/play/Sources cat > $HOME/play/Sources/main.swift <<EOF import Foundation import Kitura import KituraSys import KituraNet import MongoKitten // All Web apps need a Router instance to define routes let router = Router() // Simple Mongodb instance let dbserver: MongoKitten.Server! do { dbserver = try Server("mongodb://test:test@localhost:27017", automatically: true) let database = dbserver["workouts"] print("MongoDB is available at port 27017") } catch { // Unable to connect print("MongoDB is not available on the given host and port") } // Simple JSON Wrapper Support func JSON<KeyType, ValueType>(_ dictionary: [KeyType: ValueType]) -> String? { let bridgeValue = dictionary.bridge() do { let jsonData = try NSJSONSerialization.data(withJSONObject:bridgeValue, options: NSJSONWritingOptions.PrettyPrinted) let str = String(data:jsonData, encoding: NSUTF8StringEncoding) return(str) } catch let error as NSError{ print(error.description) return(nil) } } // Basic GET request router.get("/hello") { _, response, next in response.headers["Content-Type"] = "text/plain; charset=utf-8" do { try response.status(.OK).send("Hello World").end() } catch { print("ERROR: Failed to send response to client: \(error)") } } router.get("/today") { _, response, next in let now = NSDate() let dayTimePeriodFormatter = NSDateFormatter() dayTimePeriodFormatter.dateFormat = "E, d MMM yyyy" let dateString = dayTimePeriodFormatter.string(from:now) dayTimePeriodFormatter.dateFormat = "HH:mm:ss Z" let timeString = dayTimePeriodFormatter.string(from:now) response.headers["Content-Type"] = "text/plain; charset=utf-8" do { let payload = ["Message": "Hello World", "Date": dateString, "Time": timeString] if let json = JSON(payload) { try response.status(.OK).send(json).end() } else { try response.status(.OK).send("Does anybody really know what time it is ?").end() } } catch { print("ERROR: Failed to send response to client: \(error)") } } // Start server let port = 8095 let server = HTTPServer.listen(port:port, delegate: router) print("starting server on port \(port)") Server.run() EOF cd $HOME/play/Packages/SwiftyJSON-7.0.4/ git pull #there are 3 errors in Packages/Strand-1.4.1/Sources/Strand.swift and need to be changed before build #Strand.swift let pointer = UnsafeMutablePointer<Void>(holder.toOpaque()) #change to let pointer = UnsafeMutablePointer<Void>(OpaquePointer(bitPattern: holder)) let unmanaged = Unmanaged<StrandClosure>.fromOpaque(arg)] #change to let unmanaged = Unmanaged<StrandClosure>.fromOpaque(OpaquePointer(arg)) private func runner(arg: UnsafeMutablePointer<Void>?) -> UnsafeMutablePointer<Void>? { #change to private func runner(arg: UnsafeMutablePointer<Void>!) -> UnsafeMutablePointer<Void>! { and 1 error in Packages/MongoKitten-1.1.0/Sources/Server.swift let port: UInt16 = UInt16(url.port?.int16Value ?? 27017) # change to let port: UInt16 = UInt16(url.port?.shortValue ?? 27017) #build app cd $HOME/play swift build -Xcc -fblocks -Xlinker -rpath -Xlinker .build/debug #install and start mongodb server apt-get update apt-get install mongodb -y apt-get install lsb-release -y service mongodb start #create user in mongodb using mongo client mongo > use workouts > db.createUser( {user: "test", pwd: "test", roles: [ "readWrite", "dbAdmin" ]} ); > show users > exit #run app and use mac browser to localhost:8095/hello .build/debug/play






To test the node docker image for node.js web app
shell script    Select all
#install mongodb apt-get update apt-get install mongodb mongodb-server mongodb-clients -y #start mongdb service if not automatically service mongodb start #mongo shell script create user and to display info mongo > use workouts > db.addUser( {user: "test", pwd: "test", roles: [ "readWrite", "dbAdmin" ]} ); > show users > exit #download project file and copy to the container #the project file can be downloaded from here https://mega.nz/#!z9oFlTgJ!geCPHjoXJ3rpm1OdldU8s5RBSJxLKiAxh_axvTbODB4 docker cp ~/Download/nodeapi.zip 6d8baf06826e:/root #unzip and install apt-get install zip unzip -y cd $HOME unzip nodeapi.zip cd nodeapi npm install npm start #use docker to start another shell access to test docker exec -it 6d8baf06826e bash cd $HOME/nodeapi ./test1.sh ./test2.sh ./test3.sh ./test4.sh ./test5.sh ./test6.sh ...






To test the quantlib or quantlib-python3 docker image
shell script    Select all
apt-get update apt upgrade apt-get install python python-pip -y pip install --upgrade pip # Python 2 installation pip2 install numpy pip2 install QuantLib-Python # Python 3 installation apt-get install python3 python3-pip -y pip3 install numpy pip3 install QuantLib-Python # Install boost 1.71 export boost_version=1.71.0; export boost_dir=boost_1_71_0; wget https://dl.bintray.com/boostorg/release/${boost_version }/source/${boost_dir}.tar.gz && tar xfz ${boost_dir}.tar.gz && rm ${boost_dir}.tar.gz && cd ${boost_dir} && ./bootstrap.sh && ./b2 --without-python --prefix=/usr -j 4 link=shared runtime-link=shared install && cd .. && rm -rf ${boost_dir} && ldconfig # Install quantlib 1.17 export quantlib_version=1.17; wget https://dl.bintray.com/quantlib/releases/QuantLib-${quantlib_version}.tar.gz && tar xfz QuantLib-${quantlib_version}.tar.gz && rm QuantLib-${quantlib_version}.tar.gz && cd QuantLib-${quantlib_version} && ./configure --prefix=/usr --disable-static CXXFLAGS=-O3 && make -j 4 && make check && make install && cd .. && rm -rf QuantLib-${quantlib_version} && ldconfig # Test quantlib 1.17 cd $HOME cat > $HOME/qlversion.cpp <<EOF #include <iostream> #include <ql/version.hpp> int main() { std::cout << "Current QL Version:" << QL_LIB_VERSION << std::endl; return 0; } EOF g++ qlversion.cpp -o qlversion ./qlversion cat > $HOME/swap.py <<EOF import numpy as np import QuantLib as ql # Set Evaluation Date today = ql.Date(31,3,2015) ql.Settings.instance().setEvaluationDate(today) # Setup the yield termstructure rate = ql.SimpleQuote(0.03) rate_handle = ql.QuoteHandle(rate) dc = ql.Actual365Fixed() disc_curve = ql.FlatForward(today, rate_handle, dc) disc_curve.enableExtrapolation() hyts = ql.YieldTermStructureHandle(disc_curve) discount = np.vectorize(hyts.discount) start = ql.TARGET().advance(today, ql.Period('2D')) end = ql.TARGET().advance(start, ql.Period('10Y')) nominal = 1e7 typ = ql.VanillaSwap.Payer fixRate = 0.03 fixedLegTenor = ql.Period('1y') fixedLegBDC = ql.ModifiedFollowing fixedLegDC = ql.Thirty360(ql.Thirty360.BondBasis) index = ql.Euribor6M(ql.YieldTermStructureHandle(disc_curve)) spread = 0.0 fixedSchedule = ql.Schedule(start, end, fixedLegTenor, index.fixingCalendar(), fixedLegBDC, fixedLegBDC, ql.DateGeneration.Backward, False) floatSchedule = ql.Schedule(start, end, index.tenor(), index.fixingCalendar(), index.businessDayConvention(), index.businessDayConvention(), ql.DateGeneration.Backward, False) swap = ql.VanillaSwap(typ, nominal, fixedSchedule, fixRate, fixedLegDC, floatSchedule, index, spread, index.dayCounter()) engine = ql.DiscountingSwapEngine(ql.YieldTermStructureHandle(disc_curve)) swap.setPricingEngine(engine) print(swap.NPV()) print(swap.fairRate()) EOF python swap.py apt-get install git -y git clone git://github.com/mmport80/QuantLib-with-Python-Blog-Examples.git cd QuantLib-with-Python-Blog-Examples/ python2 blog_frn_example.py



Try docker-compose example as per https://docs.docker.com/compose/wordpress/
shell script    Select all
cd $HOME mkdir my-wordpress cd my-wordpress cat > docker-compose.yaml <<EOF version: '2' services: db: image: mysql:5.7 volumes: - "./.data/db:/var/lib/mysql" restart: always environment: MYSQL_ROOT_PASSWORD: wordpress MYSQL_DATABASE: wordpress MYSQL_USER: wordpress MYSQL_PASSWORD: wordpress wordpress: depends_on: - db image: wordpress:latest links: - db ports: - "8000:80" restart: always environment: WORDPRESS_DB_HOST: db:3306 WORDPRESS_DB_PASSWORD: wordpress EOF docker-compose up -d # then try browser http://localhost:8000/ # stop the docker-compose example docker-compose down