Sunday, October 19, 2014

How to run Android 5.0 samples in Android Studio

(1) Install Android 5.0 and Android SDK Build-tools to 21.0.1 using the latest Android Studio


(2) Download the samples from github
say from https://github.com/googlesamples/android-RecyclerView
and use Import Project in Android Studio


(3) Install HAXM if you run emulator in Windows
After you download it make sure you run the setup located in:
{SDK_FOLDER}\extras\intel\Hardware_Accelerated_Execution_Manager\intelhaxm.exe

If you get the error "VT not supported" during the installation disable Hyper-V on windows features. You can execute this command as administrator dism.exe /Online /Disable-Feature:Microsoft-Hyper-V. You will also need "Virtualization Technology" to be enabled on your BIOS.


If you are using Mac OSX 10.10 Yosemite, run this (to allow ALL UNSIGNED KEXT to be loaded)
sudo nvram boot-args="kext-dev-mode=1"
and reboot before installing the package in
{SDK_FOLDER}/extras/intel/Hardware_Accelerated_Execution_Manager/IntelHAXM_1.1.0_for_10.10.dmg


(4) setup AVD like below
Choose Intel Atom X86 and with Ram 768M (for Windows). You can use 2G for Mac


(5)change build.gradle file compileSdkVersion to 21 and buildToolsVersion to "21.0.1"
and also change AndroidManifest.xml file android:minSdkVersion= to "13"


Tuesday, October 7, 2014

CSS Selector in Chrome (Part 2) Apple On-line Store for iPhone 6

Apple On-line Store for iPhone 6 URL for HK is here
http://store.apple.com/hk/buy-iphone/iphone6

The selection flow and the CSS Selector code are listed as below


(1) Choose a model:
AOS/availability    Select all
<div id="group-1"> <div id="Item1" class="step"> <div class="selection"> <div class="small-size-buttons enabled"> <fieldset> <legend class="size-selection-header"> <h2 class="fwl"><span class="section-number step1"><span class="a11y">1</span>>/span> <span class="section-heading">Choose a model:</span> </h2> </legend> <div class="selection-buttons"> <div class="item "> <input type="radio" value="4_7inch" id="Item14_7inch" name="dimensionScreensize" class="a11y"> <label for="Item14_7inch"> <div class="details"> <div class="label"> <div class="size-label">iPhone 6 <span>4.7-inch display</span> </div> <div class="price"> <span itemtype="http://schema.org/Offer" itemprop="offers" itemscope="itemscope"> <meta itemprop="priceCurrency" content="HKD"> <span itemprop="price">From <span>HK$5,588</span></span> </span> </div> </div> </div> </label> </div> <div class="item "> <input type="radio" value="5_5inch" id="Item15_5inch" name="dimensionScreensize" class="a11y" checked=""> <label for="Item15_5inch"> <div class="details"> <div class="label"> <div class="size-label">iPhone 6 Plus <span>5.5-inch display</span> </div> <div class="price"> <span itemtype="http://schema.org/Offer" itemprop="offers" itemscope="itemscope"> <meta itemprop="priceCurrency" content="HKD"> <span itemprop="price">From <span>HK$6,388</span></span> </span> </div> </div> </div> </label> </div> </div> </fieldset> </div> </div> </div>
$("group-1 #Item1 div.selection-buttons div.item").eq(0).find("input").click();
where 0=iPhone 6; 1=iPhone 6 Plus


(2) Choose color:
$("group-1 #Item2 div.selection-buttons div.item").eq(0).find("input").click();
where 0=Silver; 1=Gold; 2=Space-Grey


(3) Choose storage:
$("group-1 #Item3 div.selection-buttons div.item").eq(0).find("input").click();
where 0=16GB; 1=64GB; 2=128GB


(4) Product Summary and Availability:
$("primary .product-title").text() + " " + $("actiontray .customer_commit_display").text();


(5) Enable Select button:
AOS/availability    Select all
<div id="actiontray" class="actiontray"> <ul class="purchase-info"> <li class="specials"></li> <li class="shipping"> Available to ship<b>:</b><br> <span class="customer_commit_display">Currently unavailable</span> <span class="freeshipping promo-text">Free Shipping</span> </li> <li> <span class="price"> <span class="current_price"> <span itemscope="itemscope" itemprop="offers" itemtype="http://schema.org/Offer"> <meta itemprop="priceCurrency" content="HKD"> <span itemprop="price"> HK$7,188 </span> </span> </span> </span> <span class="financing"></span> </li> <li class="add-to-cart"> <button class="button disabled" type="button" disabled="disabled"><span>Select</span></button> </li> <li class="shipping-lead"></li> </ul> </div>
$("actiontray .add-to-cart .button")[0].classList.remove("disabled"); $("actiontray .add-to-cart .button").removeAttr("disabled"); $("actiontray .add-to-cart .button")[0].classList.add("transactional");
note: Chrome supports HTML5 Technique for changing classes


Monitoring program see http://youtu.be/Nj1vcbEyhf4

Sunday, October 5, 2014

CSS selector in Chrome using the example of Apple iPhone 6 Availability Monitoring

The iPhone 6 Reserve Page for HK is a dynamic web content as in
https://reserve.cdn-apple.com/HK/en_HK/reserve/iPhone/availability


An example of the implementation of monitoring program uses javascript and the Chrome browser and further automation to reserve an iPhone can possibly be completed using Webdriver JS from the Selenium + chrome driver (see http://youtu.be/I1-m_qcmQLk).

To start with and illustrate, the steps for the javascript code preparation are as below:

(1) Goto the website and open the javascript console (Control-Shift-J in PC, Cmd-Option-J in Mac) in Chrome browser



(2) Click the element tab, the dynamic webpage code can be revealed.

iPhone/availability    Select all
<div id="wrapper" class="check-availability"> <div class="two-col-container"> <div class="options-container"> <div class="availability-title-container"> ... </div> <div class="storeselect select-style products" style="..."> <select name="products"> <option value="">Select a model</option> <option data-index="0">iPhone 6 Plus</option> <option data-index="1">iPhone 6</option> </select> </div> <div class="storeselect select-style stores" style="..."> <select name="stores"> <option value="">Select a store</option> <option value="R409">Causeway Bay</option> <option value="R485">Festival Walk</option> <option value="R428">ifc mall</option> </select> </div>

(3) Use the CSS style locator (to be entered into the javascript console of Chrome Browser) as below to quickly locate the model selection box and make changes to it, eq(1) being iPhone 6 plus and eq(2) is iPhone 6.
div#wrapper means the div id locator for the id wrapper.
div.two-col-container means the div class locator for the class two-col-container.
$('div#wrapper div.two-col-container div.options-container select[name="products"] option').eq(1).select().attr("selected","selected").change();
$('div#wrapper div.two-col-container div.options-container select[name="products"] option').eq(2).select().attr("selected","selected").change();


(4) Similarly, the code to select the store is
$('div#wrapper div.two-col-container div.options-container select[name="stores"] option').eq(1).select().attr("selected","selected").change();

(5) To retrieve the dynamic content of the store info, use
$("div#wrapper div.two-col-container div.dropdown-container div.results-container .store").text() + $("div#wrapper div.two-col-container div.dropdown-container div.results-container .date").text();

(6) To retrieve the dynamic content of availability of say 16GB sliver, use
" 16GB " +$("div#wrapper div.two-col-container div.dropdown-container div.results-container .device-sizes tbody tr").eq(0).attr("class") + " " + $("div#wrapper div.two-col-container div.dropdown-container div.results-container .device-sizes tbody tr").eq(0).find(".16GB .isAvailableText").text();

(6.1) Similarly, 16GB gold is
" 16GB " +$("div#wrapper div.two-col-container div.dropdown-container div.results-container .device-sizes tbody tr").eq(1).attr("class") + " " + $("div#wrapper div.two-col-container div.dropdown-container div.results-container .device-sizes tbody tr").eq(1).find(".16GB .isAvailableText").text();

(6.2) For others, just repeat and substitute '16GB' for '64GB' and '128GB' and change the eq(0) to eq(1) and eq(2) for different colors, 0=silver, 1=gold, 2=space-grey.

(7) To click the Get Started button on the right of the webpage, use
$('div.skip-wrapper div.button-container .button')[0].click();



To automate the scripts, one possible solution is to use node.js + chromedriver
Download and Put the ChromeDriver (note that it is 32 bit only for Windows/Mac) on your PATH
http://chromedriver.storage.googleapis.com/index.html?path=2.10/

and then install node.js from
http://nodejs.org/download/
and see the WebDriver wiki here https://code.google.com/p/selenium/wiki/WebDriverJs

Note: for windows 8, please create folder in "%HOMEPATH%\AppData\Roaming\npm"
before installing WebDriver using "npm install selenium-webdriver"


appleiphone6_availability.js    Select all
#!/usr/bin/env node // for bash shell // for i in {1..300}; do sleep 1s; ./appleiphone6_availability.js $i; sleep 39s ; done; // for windows command line, Use two %% if it is in a batch file // FOR /L %I IN (1,1,300) DO ("C:\Program Files\nodejs\node.exe" appleiphone6_availability.js %I; sleep 39s) var args = process.argv.slice(2); var webdriver = require('selenium-webdriver'); var driver = new webdriver.Builder().withCapabilities({ browserName : 'chrome', 'chromeOptions': { args: ['test-type'] } }).build(); var isDEBUG = false; var isAvailable = [false,false,false]; var isAvailableCount = 0; var products = ["","iPhone 6 Plus","iPhone 6"]; var stores = ["","Causeway Bay","Festival Walk","IFC Mall"]; var fs = require('fs'); function nextaction(msg, p) { if (isAvailable[p]) { return } else { isAvailable[p] = true; isAvailableCount++; var driver2 = new webdriver.Builder().withCapabilities(webdriver.Capabilities.chrome()).build(); driver2.wait(function() { driver2.get('http://admin:password@192.168.0.1/sms/newmessage.asp') if (isAvailableCount > 1) { driver2.findElement(webdriver.By.id("maincontent")).findElement(webdriver.By.name("phoneNumber")).sendKeys("12345678"); } else { driver2.findElement(webdriver.By.id("maincontent")).findElement(webdriver.By.name("phoneNumber")).sendKeys("12345678;87654321"); } driver2.findElement(webdriver.By.id("maincontent")).findElement(webdriver.By.id("draftContent")).sendKeys("IR Monitoring: " + msg + " now available"); driver2.findElement(webdriver.By.id("maincontent")).findElement(webdriver.By.id("send")).click(); return msg != null; }, 3000) .then(function() { driver2.quit(); var stream = fs.createWriteStream('availableLog.txt', {flags: 'a', encoding : 'unicode'}); stream.once('open', function(fd) { stream.write('SMS sent ' + msg + '\r\n'); stream.end(); }); console.log("SMS sent OK") }); } return } function check_availability(p,s) { driver.get('https://reserve.cdn-apple.com/HK/en_HK/reserve/iPhone/availability'); driver.executeScript('$("#wrapper .two-col-container .options-container .storeselect select[name=\'products\'] option").eq('+p+').select().attr("selected","selected").change()') .then(function() { if (isDEBUG) console.log("product selected"); }, function(err) { console.log('There was an error for product selection ! ' + err); }); driver.executeScript('$("#wrapper .two-col-container .options-container .storeselect select[name=\'stores\'] option").eq('+s+').select().attr("selected","selected").change()') .then(function() { if (isDEBUG) console.log("store selected"); }, function(err) { console.log('There was an error for store selection ! ' + err); }); if (isDEBUG) console.log('CHECKING ' + products[p] + ' ' + stores[s]); driver.wait(function() { return driver.executeScript('return $("#wrapper .two-col-container .dropdown-container .results-container .store").text() + $("#wrapper .two-col-container .dropdown-container .results-container .date").text()') .then(function(message) { console.log(args +' '+message); if (message.length < 5) { console.log('CHECKING ' + products[p] + ' ' + stores[s]); return webdriver.promise.rejected('Please come back later'); } return message != null; }); }, 3000) .then(function() { driver.wait(function() { return driver.executeScript('return $("#wrapper .two-col-container .dropdown-container .results-container .store").text() \ + $("#wrapper .two-col-container .dropdown-container .results-container .date").text() \ + " 16GB " + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(0).attr("class") + " " \ + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(0).find(".16GB .isAvailableText").text() \ + " 16GB " + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(1).attr("class") + " " \ + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(1).find(".16GB .isAvailableText").text() \ + " 16GB " + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(2).attr("class") + " " \ + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(2).find(".16GB .isAvailableText").text() \ + " 64GB " + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(0).attr("class") + " " \ + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(0).find(".64GB .isAvailableText").text() \ + " 64GB " + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(1).attr("class") + " " \ + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(1).find(".64GB .isAvailableText").text() \ + " 64GB " + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(2).attr("class") + " " \ + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(2).find(".64GB .isAvailableText").text() \ + " 128GB " + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(0).attr("class") + " " \ + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(0).find(".128GB .isAvailableText").text() \ + " 128GB " + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(1).attr("class") + " " \ + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(1).find(".128GB .isAvailableText").text() \ + " 128GB " + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(2).attr("class") + " " \ + $("#wrapper .two-col-container .dropdown-container .results-container .device-sizes tbody tr").eq(2).find(".128GB .isAvailableText").text()' ).then(function(message) { if (isDEBUG) console.log(message); if(message.indexOf(" Available") > -1) { console.log(products[p] + ' ' + stores[s] + "Available "+ message); var stream = fs.createWriteStream('availableLog.txt', {flags: 'a', encoding : 'unicode'}); stream.once('open', function(fd) { stream.write(args +' ' + products[p] + ' ' + stores[s] +' Acailable'+ '\r\n' + message + '\r\n'); stream.end(); }); vailableText = products[p] + ' ' + stores[s]+ ' (' ; var is16GBAvailable = false; var is64GBAvailable = false; var is128GBAvailable = false; if(message.indexOf("16GB gold Available") > -1) { availableText += "16G金"; is16GBAvailable = true; } if(message.indexOf("16GB space-grey Available") > -1) { if (is16GBAvailable) { availableText += "灰";} else {availableText += "16G灰";} is16GBAvailable = true; } if(message.indexOf("16GB silver Available") > -1) { if (is16GBAvailable) { availableText += "銀";} else { availableText += "16G銀";} is16GBAvailable = true; } if(message.indexOf("64GB gold Available") > -1) { availableText += " 64G金"; is64GBAvailable = true; } if(message.indexOf("64GB space-grey Available") > -1) { if (is64GBAvailable) { availableText += "灰";} else { availableText += " 64G灰";} is64GBAvailable = true; } if(message.indexOf("64GB silver Available") > -1) { if (is64GBAvailable) { availableText += "銀";} else { availableText += " 64G銀";} is64GBAvailable = true; } if(message.indexOf("128GB gold Available") > -1) { availableText += " 128G金"; is128GBAvailable = true; } if(message.indexOf("128GB space-grey Available") > -1) { if (is128GBAvailable) { availableText += "灰";} else { availableText += " 128G灰";} is128GBAvailable = true; } if(message.indexOf("128GB silver Available") > -1) { if (is128GBAvailable) { availableText += "銀";} else { availableText += " 128G銀";} is128GBAvailable = true; } nextaction(availableText+ ')',p); } return message != null; }); }, 3000) }, function(err){ console.log('There was an error checking ' + products[p] + ' ' + stores[s] + ' ' + err); }); } // node.js has non-blocking I/O and recursive, there is no need to have a loop here var p1 = 1; var s1 = 0; driver.get('https://reserve.cdn-apple.com/HK/en_HK/reserve/iPhone/availability'); driver.wait(function() { if (++s1 > 3) { s1=1; if (++p1 > 2){p1=1}} return check_availability(p1,s1); }, 180*1000) .then(function() {} ,function(err){ driver.quit(); console.log('check_availability ' + err); });


Saturday, May 3, 2014

Nokia X emulator skin

Download : http://www.2shared.com/file/XCjdetAW/NOKIAX.html

Supports: Landscape and Portrait, Power and Volume Up/Down and Back buttons
Display width : 480
Display height : 800
Installation Location : <sdk>\add-ons\addon-nokia_x_services_device-nokia-16\skins

Friday, May 2, 2014

How to install Android Studio in Mac OS X


(3) Install JDK say jdk-7u51-macosx-x64.dmg from http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html

(2) Download the latest Android Studio IDE from http://tools.android.com/download/studio/canary/latest and install to /Applications/

(3) Download the Android Studio with SDK from http://dl.google.com/dl/android/studio/ide-zips/0.4.2/android-studio-ide-133.970939-mac.zip and extract the sdk subfolder to /Applications/Andriod Studio.app/sdk

(4) Change ownership for /Applications/Android Studio.app/sdk using sudo chown -R <yourid>:staff /Applications/Android Studio.app/sdk

(5) Launch Android Studio and change Project Structure to
SDK Location : /Applications/Android Studio.app/sdk
JDK Location : /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home
(6) Tools -> Android -> SDK Manager to install the latest SDK packages Android 4.4.2 API-19

(7) Tools -> Android -> AVD Manager to add AVD

(8) Import Project to test build TouchCalculator.rar. This project has been modified from here and for test build on Android Studio API-19
Download -> http://www.2shared.com/file/XIswagFF/TouchCalculator.html



Another example on how to use Android Studio to create new app can be reference from the workflow demo in this video
http://www.youtube.com/watch?v=K2lF862TjU8
The source file in the video page was dead.

New project should be created in Android Studio with App Name as "PeamonCalculator" and packagename as "com.example.peamoncalculator.app" and using the default theme "Holo Light with Dark Action Bar"
Download the project source from here and replace the following 2 files in the newly created project and use the Nokia X skin for this project source

PeamonCalculator/app/src/main/java/com/example/peamoncalculator/app/MainActivity.java
PeamonCalculator/app/src/main/res/layout/activity_main.xml

unzip ~/Downloads/PeamonCalculator.zip -d ~/YourAndroidStudioProjectFolder/


Monday, January 6, 2014

Oracle Developer Days.ova

Virtual Box Download -> https://www.virtualbox.org/wiki/Downloads

Oracle_Developer_Day.ova -> here



Linux Server Login
username : oracle
password : oracle

Oracle Developer Days VM root password is also : oracle

Terminal
sqlplus "sys as sysdba"
password : 123
SQL> alter user hr identified by hr account unlock;



Sql Developer Connection details
Connection Name : HR_ORCL
Username : hr
Password : hr
Hostname : localhost
Port : 1521
SID : orcl



If you need to connect from outside VM, please refer to this article http://barrymcgillin.blogspot.hk/2011/12/using-oracle-developer-days-virtualbox.html on how to add Ethernet or Wifi Adapter. Then you can connect via other clients such as ssh or PLSQL to the new adapter from outside VM. If you use the NAT adapter, you should use the port forwarding setting as in the article.



This is how to show and resize the virtual hard-disk partition if needed
VBoxManage showhdinfo ~/VirtualBox\ VMs/Oracle\ Developer\ Days/Oracle_Developer_Day-disk1.vmdk
VBoxManage showhdinfo ~/VirtualBox\ VMs/Oracle\ Developer\ Days/Oracle_Developer_Day-disk2.vmdk

VBoxManage clonehd ~/VirtualBox\ VMs/Oracle\ Developer\ Days/Oracle_Developer_Day-disk2.vmdk ~/VirtualBox\ VMs/Oracle\ Developer\ Days/Oracle_Developer_Day-disk2A.vdi --format vdi

VBoxManage modifyhd ~/VirtualBox\ VMs/Oracle\ Developer\ Days/Oracle_Developer_Day-disk2A.vdi --resize 40960

VBoxManage clonehd ~/VirtualBox\ VMs/Oracle\ Developer\ Days/Oracle_Developer_Day-disk2A.vdi ~/VirtualBox\ VMs/Oracle\ Developer\ Days/Oracle_Developer_Day-disk2A.vmdk --format vmdk



Please refer to this on how to grow the linux partition using GParted http://derekmolloy.ie/resize-a-virtualbox-disk/


To create DBLINK to remote server using Easy Connect Naming (see here http://docs.oracle.com/cd/E11882_01/network.112/e10836/naming.htm#NETAG1125) :
Need to create the DBLINK using "sys as sysdba"

CREATE PUBLIC DATABASE LINK mydblink
CONNECT TO myusername IDENTIFIED BY "mypassword!"
USING '//172.1.2.3:1521/service_name';

GRANT CREATE SYNONYM, CREATE VIEW, CREATE DATABASE LINK, CREATE PUBLIC SYNONYM, DROP PUBLIC SYNONYM TO HR;


Monday, September 23, 2013

Stock analysis and calculate VaR using R

This is a demo script for stock download and analysis using R

 (1) Download time series data from Yahoo Finance (example here is to download from 2011 up to 2013), or copy the url and download in browser and save as csv files
getstockdata.sh    Select all
curl "http://ichart.finance.yahoo.com/table.csv?s=MSFT&a=0&b=01&c=2011&d=11&e=31&f=2013&g=d" > msft.csv curl "http://ichart.finance.yahoo.com/table.csv?s=AAPL&a=0&b=01&c=2011&d=11&e=31&f=2013&g=d" > aapl.csv curl "http://ichart.finance.yahoo.com/table.csv?s=^IXIC&a=0&b=01&c=2011&d=11&e=31&f=2013&g=d" > nasdaq.csv curl "http://ichart.finance.yahoo.com/table.csv?s=GOOG&a=0&b=01&c=2011&d=11&e=31&f=2013&g=d" > goog.csv ### or use wget "http://ichart.finance.yahoo.com/table.csv?s=MSFT&a=0&b=01&c=2011&d=11&e=31&f=2013&g=d" -O msft.csv

 (2) Run script in R, you can use source("stocks.r")
stocks.r    Select all
msft <- read.csv("msft.csv") aapl <- read.csv("aapl.csv") goog <- read.csv("goog.csv") nasdaq <- read.csv("nasdaq.csv") stocks <- nasdaq[,c("Date","Close")] colnames(stocks)[2] <- "NASDAQ" stocks["MSFT"] <- NA stocks$MSFT <- msft$Close stocks["AAPL"] <- NA stocks$AAPL <- aapl$Close stocks["GOOG"] <- NA stocks$GOOG <- goog$Close

 (3) Do some analysis in R
analysis.r    Select all
### plot Data plot(aapl$Date, aapl$Close) cor(goog[c("Open")],aapl[c("Open")]) summary(stocks) str(stocks) pairs(stocks[-1]) cor(stocks[-1]) cov(stocks[-1]) eigen(cor(stocks[-1])) ### Do linear regression mod1 <- lm(NASDAQ ~ MSFT + AAPL + GOOG, data=stocks) summary(mod1) ?summary.lm ### Confidence Limits on the Estimated Coefficients confint(mod1) summary.aov(mod1) ### Prediction of mean response for cases like this... predict(mod1, list(MSFT=30,AAPL=400,GOOG=800), interval="conf") ### Prediction for a single new case... predict(mod1, list(MSFT=30,AAPL=400,GOOG=800), interval="pred") mod2 = update(mod1,.~.-AAPL) summary(mod2) (prediction <- c(1,30,800) * coef(mod2)) sum(prediction) ### Compare two models anova(mod1, mod2) ### Regression Diagnostics par(mfrow=c(2,2)); plot(mod1); par(mfrow=c(1,1))

 (4) Stepwise Regression in R
stepwise.r    Select all
###retrieve more stocks data directly using R as below intc <- read.csv("http://ichart.finance.yahoo.com/table.csv?s=INTC&a=0&b=01&c=2011&d=11&e=31&f=2013&g=d") amzn <- read.csv("http://ichart.finance.yahoo.com/table.csv?s=AMZN&a=0&b=01&c=2011&d=11&e=31&f=2013&g=d") ###update model and do stepwise regression stocks$INTC <- intc$Close stocks$AMZN <- amzn$Close mod3 <- lm(NASDAQ ~ MSFT + AAPL + GOOG + INTC + AMZN, data=stocks) step(mod3, direction="both") summary(mod3) summary(step(mod3, direction="both"))

 (5) Delta Normal VaR and ES for a single asset using R
Reference (VaR using Excel) : http://www.youtube.com/watch?v=ZrKmVC-Ede8
DNVaR1.r    Select all
# get the time series of AAPL with Closed Price aapl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=AAPL&a=2&b=23&c=2012&d=2&e=23&f=2013&g=d"); # calculate log returns, note the -log sign below aapl.logreturn = c(diff(-log(aapl$Close))); # calculate VaR for 95% confidence interval, VaR is 0.03513197 qnorm(0.95)*sd(aapl.logreturn)-mean(aapl.logreturn); #alternatively, qnorm(p=0.95, sd=sd(aapl.logreturn), mean=mean(aapl.logreturn)); # with a single asset of say $1M, the calculated Delta Normal VaR is $35,131.97 sprintf("%5.2f", 1e+06*(qnorm(0.95)*sd(aapl.logreturn)-mean(aapl.logreturn))); #Expected Shortfall(ES) for Normal Distribution = $62,227.24 v <- qnorm(p=0.95, sd=sd(aapl.logreturn), mean=mean(aapl.logreturn)) tailExp <- integrate(function(x) x * dnorm(x, mean=mean(aapl.logreturn), sd=sd(aapl.logreturn)), lower=-Inf, upper=v)$value / (1-0.95) -tailExp * 1e+06

 (6) Delta Normal VaR for a portfolio of assets using R
Reference (Portfolio VaRs using Excel) : https://www.youtube.com/watch?v=GqczSHRUaDk
DNVaR2.r    Select all
# get the time series of MSFT, GOOG, AAPL, ORCL and IBM msft <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=MSFT&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") goog <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=GOOG&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") aapl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=AAPL&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") orcl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=ORCL&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") ibm <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=IBM&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") # calculate log returns from Closed Prices msft.logreturn = c(diff(-log(msft$Close))); goog.logreturn = c(diff(-log(goog$Close))); aapl.logreturn = c(diff(-log(aapl$Close))); orcl.logreturn = c(diff(-log(orcl$Close))); ibm.logreturn = c(diff(-log(ibm$Close))); # calculate the individual volatility logreturns <- data.frame(msft.logreturn, goog.logreturn, aapl.logreturn, orcl.logreturn, ibm.logreturn); sapply(logreturns,sd); # calculate the individual VaR using Matrix and with position of $1M each using correlation matrix VaR1 = (1e+06*sapply(logreturns,sd)) %*% cor(logreturns); # calculate the Dollar Variance of the Portfolio VaR2 = VaR1 %*% (1e+06*sapply(logreturns,sd)); # Square Root Portfolio Variance to calculate the Delta Normal 1-day Portfolio VaR for 95% confidence interval. The result is $119,605.5 qnorm(0.95) * sqrt(VaR2); #Alternatively, the variance-cov method is demo here v1 = cov(logreturns) %*% rep(1e+06,5); v2 = rep(1e+06,5) %*% v1; sqrt(v2)*qnorm(0.95); # Portfolio VaR for 10 days sqrt(v2)*qnorm(0.95)* sqrt(10);

 (6.1) Delta Normal VaR and ES for a portfolio of assets using R
Using : simple return instead of log return
DNVaR2.r    Select all
# get the time series of MSFT, GOOG, AAPL, ORCL and IBM msft <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=MSFT&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") yhoo <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=YHOO&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") aapl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=AAPL&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") orcl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=ORCL&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") ibm <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=IBM&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") # calculate simple returns from Closed Prices #msft.simplereturn = -diff(msft$Close)/msft$Close[-length(msft$Close)]; #yhoo.simplereturn = -diff(yhoo$Close)/goog$Close[-length(yhoo$Close)]; #aapl.simplereturn = -diff(aapl$Close)/aapl$Close[-length(aapl$Close)]; #orcl.simplereturn = -diff(orcl$Close)/orcl$Close[-length(orcl$Close)]; #ibm.simplereturn = -diff(ibm$Close)/ibm$Close[-length(ibm$Close)]; msft.simplereturn = msft$Close[-nrow(msft)] / msft$Close[-1] -1; yhoo.simplereturn = yhoo$Close[-nrow(yhoo)] / yhoo$Close[-1] -1; aapl.simplereturn = aapl$Close[-nrow(aapl)] / aapl$Close[-1] -1; orcl.simplereturn = orcl$Close[-nrow(orcl)] / orcl$Close[-1] -1; ibm.simplereturn = ibm$Close[-nrow(ibm)] / ibm$Close[-1] -1; # plot the histogram, assume different stock position simplereturns <- data.frame(msft.simplereturn,yhoo.simplereturn,aapl.simplereturn,orcl.simplereturn,ibm.simplereturn); hist(as.matrix(simplereturns) %*% c(1e+06, 1e+05, 5e+03, 9e+04, 3e+05),col="green"); # calculate the individual volatility simplereturns <- data.frame(msft.simplereturn, yhoo.simplereturn, aapl.simplereturn, orcl.simplereturn, ibm.simplereturn); sapply(simplereturns,sd); #using the variance-covariance method with different stock position to calculate VaR $35,473.23 v1 = cov(simplereturns) %*% c(1e+06, 1e+05, 5e+03, 9e+04, 3e+05); v2 = c(1e+06, 1e+05, 5e+03, 9e+04, 3e+05) %*% v1; sqrt(v2)*qnorm(0.95); # Portfolio VaR for 10 days sqrt(v2)*qnorm(0.95)* sqrt(10); #calculate the ES $21,986.07 m <- sapply(simplereturns,mean) %*% c(1e+06, 1e+05, 5e+03, 9e+04, 3e+05); v <- qnorm(p=0.95, sd=sqrt(v2), mean=m); - integrate(function(x) x * dnorm(x, mean=m, sd=sqrt(v2)), lower=-Inf, upper=v)$value / (1-0.95) # use PerformanceAnalytics to calculate VaR and ES require(PerformanceAnalytics) stockportfolio.ts <- as.xts(data.frame(msft$Close,yhoo$Close,aapl$Close,orcl$Close,ibm$Close), order.by = as.Date(msft$Date,"%Y-%m-%d")) # calculate VaR $34,289.08 VaR(R = na.omit(Return.calculate(stockportfolio.ts, method="discrete")), p = 0.95, method = "gaussian", portfolio_method = "component", weights = c(1e+06, 1e+05, 5e+03, 9e+04, 3e+05)) # calculate ES $43,300.71 ETL(R = na.omit(Return.calculate(stockportfolio.ts, method="discrete")), p = 0.95, method = "gaussian", portfolio_method = "component", weights = c(1e+06, 1e+05, 5e+03, 9e+04, 3e+05))

 (7) Historical Simulation VaR for a single asset using R
Reference (Historical Simulation using Excel) : http://www.youtube.com/watch?v=6Nolb4-iRSI
HSVaR1.r    Select all
# get the time series of GOOG with Closed Price goog <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=GOOG&a=1&b=25&c=2008&d=6&e=18&f=2008&g=d") # calculate log returns, using natural log goog.logreturn = c(diff(-log(goog$Close))); # plot histogram hist(goog.logreturn, breaks=seq(-.11,.19,by=.01), col="green"); #show the first 10 items in ascending order head(sort(goog.logreturn),n=10); # find volatility for 5% percentile, that is -4.10867% quantile(goog.logreturn,0.05); #Historical Simulation VaR for $1,000 is 4.10867 -quantile(goog.logreturn,0.05)*1000; # find historical volatility for the 102 observations sd(goog.logreturn)*sqrt(length(goog[,1]));

 (7.1) Historical Simulation VaR for a single asset using R
Reference : Using the same data as per (5) above
HSVaR11.r    Select all
# get the time series of AAPL with Closed Price aapl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=AAPL&a=2&b=23&c=2012&d=2&e=23&f=2013&g=d"); # calculate log returns, note the -log sign below aapl.logreturn = c(diff(-log(aapl$Close))); # calculate simple returns aapl.simplereturn = aapl$Close[-nrow(aapl)] / aapl$Close[-1] -1; # plot histogram and density curve hist(aapl.logreturn,breaks=seq(-.15,.1,by=.02), col="green",freq=FALSE,main="Asset Returns", xlab="Return%",xlim=range(-0.15,0.1),ylim=range(0,28));lines(density(aapl.logreturn),col="red", lty="dotted",panel.last=abline(v=quantile(aapl.logreturn, 0.05), col="darkred",lwd=3)); mtext("95% C.I.",side=3,line=-3,adj=0.40,col="darkred"); #show the first 15 items in ascending order head(sort(aapl.logreturn),n=15); # find VaR for 5% percentile, volatility is 0.03205498 = $32,054.98 -quantile(aapl.logreturn,0.05)*1e+06; # find VaR for 5% percentile, volatility is 0.03205498 = $31,546.66 -quantile(aapl.simplereturn,0.05)*1e+06; # with a single asset of say $1M, the calculated Historical Simulation VaR is $32,054.98 comparing with the result of Delta Normal VaR of $35,131.97 as per (5) above sprintf("%5.2f", 1e+06*-quantile(aapl.logreturn,0.05)); # calculate ES $46,965.9 for log returns -mean(aapl.logreturn[aapl.logreturn <= quantile(aapl.logreturn, 0.05)])*1e+6; # calculate ES $45,565.73 for simple returns -mean(aapl.simplereturn[aapl.simplereturn <= quantile(aapl.simplereturn, 0.05)])*1e+6;


 (7.2) Historical Simulation VaR for a single asset using R and Package PerformanceAnalytics
Reference : http://cran.at.r-project.org/web/packages/PerformanceAnalytics/
http://cran.r-project.org/web/packages/quantmod/
HSVaR12.r    Select all
#install.packages("PerformanceAnalytics"); require(PerformanceAnalytics) # get the time series of AAPL with Closed Price aapl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=AAPL&a=2&b=23&c=2012&d=2&e=23&f=2013&g=d"); # convert to time series aapl.ts <- as.xts(aapl$Close, order.by = as.Date(aapl$Date,"%Y-%m-%d")); # calculate VaR $32,054.98 for log returns -VaR(Return.calculate(aapl.ts, method="log"),p=.95,method="historical") * 1e+06; # calculate VaR $31,546.66 for simple returns -VaR(Return.calculate(aapl.ts, method="discrete"),p=.95,method="historical") * 1e+06; # calculate ES $46,965.9 for log returns -ETL(Return.calculate(aapl.ts, method="log"),p=.95,method="historical") * 1e+06; # calculate ES $45,565.73 for simple returns -ETL(Return.calculate(aapl.ts, method="discrete"),p=.95,method="historical") * 1e+06; #plot PerformanceSummary using Package PerformanceAnalytics charts.PerformanceSummary(Return.calculate(aapl.ts, method="log")); #plot the time series chart using Package quantmod #install.packages("quantmod"); require("quant mod"); chartSeries(aapl.ts, theme = chartTheme("white"), TA = c(addBBands(),addTA(RSI(aapl.ts))));


 (7.3) Calculate VaR for a single asset using Monte carlo simulation: Brownian motion
Require : http://cran.at.r-project.org/web/packages/PerformanceAnalytics/

MCVaR.r    Select all
#install.packages("PerformanceAnalytics"); require(PerformanceAnalytics) # get the time series of AAPL with Closed Price aapl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=AAPL&a=2&b=23&c=2012&d=2&e=23&f=2013&g=d"); # convert to time series aapl.ts <- as.xts(aapl$Close, order.by = as.Date(aapl$Date,"%Y-%m-%d")); # calculate VaR $32,054.98 for log returns -VaR(Return.calculate(aapl.ts, method="log"),p=.95,method="historical") * 1e+06; # calculate VaR $3x,xxx (will print 3 iterations) using random 250 samples log returns generated based on sd and mean of AAPL aapl.logreturn = c(diff(-log(aapl$Close))); for (i in 1:3) {print (-VaR(as.xts(rnorm(250, sd=sd(aapl.logreturn), mean=mean(aapl.logreturn)),order.by=as.Date(aapl$Date,"%Y-%m-%d")),p=.95,method="historical") * 1e+06)}; # calculate VaR using random 1000 samples log returns generated based on sd and mean of AAPL for (i in 1:3) {print (-VaR(as.xts(rnorm(1000, sd=sd(aapl.logreturn), mean=mean(aapl.logreturn)),order.by=as.Date(Sys.Date():(Sys.Date()+1000-1))),p=.95,method="historical") * 1e+06)}; # calculate VaR (will print 3 iterations) using random 250 samples log returns generated # based on the assumptions of annual return drift=0.10 and annual return volatility=0.40 # daily volatility = annual volatility / sort(T) # daily mean drift = annual drift/T - 0.5 * sd^2 # model reference from http://www.youtube.com/watch?v=e79OtCamxD0 for (i in 1:3) {print (-VaR(as.xts(rnorm(250, sd=0.40/sqrt(250), mean=(0.10/250)-0.5*((0.40/sqrt(250))^2)),order.by=as.Date(aapl$Date,"%Y-%m-%d")),p=.95,method="historical") * 1e+06)}; # calculate VaR (will print 3 iterations) using random 1000 samples log returns generated # based on the assumptions of annual return drift=0.10 and a higher annual return volatility=0.60 for (i in 1:3) {print (-VaR(as.xts(rnorm(1000, sd=0.60/sqrt(250), mean=(0.10/250)-0.5*((0.60/sqrt(250))^2)),order.by=as.Date(Sys.Date():(Sys.Date()+1000-1))),p=.95,method="historical") * 1e+06)}; # plot the simulated prices from one instance of simulation aapl.mc.logreturns = rnorm(1000, sd=0.60/sqrt(250), mean=(0.10/250)-0.5*((0.60/sqrt(250))^2)); plot(as.xts(c(1,exp(cumsum(aapl.mc.logreturns))) * tail(aapl$Close,n=1),order.by=as.Date(as.Date(c("2010-01-01")):(as.Date(c("2010-01-01"))+1000))),main="Simulated Price");



 (8) Historical Simulation VaR for a portfolio of assets using R
Reference : Using the same data as per (6) Delta Normal VaR for a portfolio of assets
HSVaR2.r    Select all
# get the time series of MSFT, GOOG, AAPL, ORCL and IBM msft <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=MSFT&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") goog <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=GOOG&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") aapl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=AAPL&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") orcl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=ORCL&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") ibm <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=IBM&a=5&b=14&c=2011&d=5&e=8&f=2012&g=d") # calculate log returns from Closed Prices msft.logreturn = c(diff(-log(msft$Close))); goog.logreturn = c(diff(-log(goog$Close))); aapl.logreturn = c(diff(-log(aapl$Close))); orcl.logreturn = c(diff(-log(orcl$Close))); ibm.logreturn = c(diff(-log(ibm$Close))); # plot the histogram and density logreturns <- data.frame(msft.logreturn, goog.logreturn, aapl.logreturn, orcl.logreturn, ibm.logreturn); hist(as.matrix(logreturns) %*% rep(1e+06,5),col="green",freq=FALSE); lines(density(as.matrix(logreturns) %*% rep(1e+06,5)),col="red") # find VaR for 5% percentile, that is $109,409.9 and comparing with the result of Delta Normal VaR of $119,605.5 as per (6) above -quantile(as.matrix(logreturns) %*% rep(1e+6,5), 0.05);

 (8.1) Historical Simulation VaR and ES for a portfolio of assets using R
Using : simple return instead of log return
Calculate : Incremental VaR, Marginal VaR and Component VaR
HSVaR250.r    Select all
# get the time series of MSFT, YHOO, AAPL, ORCL and IBM for 251 trading days msft <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=MSFT&a=3&b=3&c=2013&d=2&e=31&f=2014&g=d") yhoo <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=YHOO&a=3&b=3&c=2013&d=2&e=31&f=2014&g=d") aapl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=AAPL&a=3&b=3&c=2013&d=2&e=31&f=2014&g=d") orcl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=ORCL&a=3&b=3&c=2013&d=2&e=31&f=2014&g=d") ibm <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=IBM&a=3&b=3&c=2013&d=2&e=31&f=2014&g=d") # calculate simple returns from Closed Prices msft.simplereturn = msft$Close[-nrow(msft)] / msft$Close[-1] -1; yhoo.simplereturn = yhoo$Close[-nrow(yhoo)] / yhoo$Close[-1] -1; aapl.simplereturn = aapl$Close[-nrow(aapl)] / aapl$Close[-1] -1; orcl.simplereturn = orcl$Close[-nrow(orcl)] / orcl$Close[-1] -1; ibm.simplereturn = ibm$Close[-nrow(ibm)] / ibm$Close[-1] -1; # plot the histogram simplereturns <- data.frame(msft.simplereturn,yhoo.simplereturn,aapl.simplereturn,orcl.simplereturn,ibm.simplereturn); hist(as.matrix(simplereturns) %*% rep(1,5),col="green",freq=FALSE); lines(density(as.matrix(simplereturns) %*% rep(1,5)),col="red"); # find VaR for 5% percentile, that is $71,726.65 -quantile(as.matrix(simplereturns) %*% rep(1e+6,5), 0.05); # find Incremental VaR for each asset by removing each asset from the portfolio # Incremental VaR : MSFT $3396.407; YHOO $19522.17; AAPL $10691.23; ORCL $12628.08; IBM $10749.8 Iport = matrix(1e+06, nrow=5, ncol=5); diag(Iport)=0; for (i in 1:5) {print(quantile(as.matrix(simplereturns) %*% (Iport)[i,], 0.05) -quantile(as.matrix(simplereturns) %*% rep(1e+6,5), 0.05))}; # Marginal VaR by list out the 13th element sort by portfolio return column # msft.simplereturn yhoo.simplereturn aapl.simplereturn orcl.simplereturn ibm.simplereturn portfolio #45 -0.02118989 -0.03323661 0.008112513 -0.01670709 -0.009686039 -0.07270712 returns <- data.frame(simplereturns, portfolio=as.matrix(simplereturns) %*% rep(1,5) ) tail(head(returns[order(returns$portfolio),],n=length(returns[,1])*0.05+1),n=1) # Component VaR by multiplying Marginal VaR by the weight of each asset # msft.simplereturn yhoo.simplereturn aapl.simplereturn orcl.simplereturn ibm.simplereturn portfolio #45 -21189.89 -33236.61 8112.513 -16707.09 -9686.039 -72707.12 tail(head(returns[order(returns$portfolio),],n=250*0.05+1),n=1) * 1e+6 #find Expected Shortfall (ES) that is $103,035.5 m <- as.matrix(simplereturns) %*% rep(1e+6,5); v <- quantile(m, 0.05); -mean(m[m <= v]); # plot all hist(as.matrix(simplereturns) %*% rep(1,5),col="green",freq=FALSE,main="Portfolio Returns", xlab="Return%",xlim=range(-0.2,0.15),ylim=range(0,10));lines(density(as.matrix(simplereturns) %*% rep(1,5)),col="red", lty="dotted",panel.last=abline(v=quantile(as.matrix(simplereturns) %*% rep(1,5), 0.05), col="darkred",lwd=3)); mtext("95% C.I.",side=3,line=-3,adj=0.28,col="darkred"); #plot all plus superimposed normal distribution curve xvals <- seq(from=-0.15, to=0.15,length=100) hist(as.matrix(simplereturns) %*% rep(1,5),col="green",freq=FALSE,main="Portfolio Returns", xlab="Return%",xlim=range(-0.2,0.15),ylim=range(0,10));lines(density(as.matrix(simplereturns) %*% rep(1,5)),col="red", lty="dotted",panel.last=abline(v=quantile(as.matrix(simplereturns) %*% rep(1,5), 0.05), col="darkred",lwd=3)); mtext("95% C.I.",side=3,line=-3,adj=0.28,col="darkred");lines(xvals,dnorm(xvals,mean(as.matrix(simplereturns) %*% rep(1,5)),sd(as.vector(as.matrix(simplereturns) %*% rep(1,5)))),lwd=1,col="blue");



 (8.2) Historical Simulation 10-Days VaR and ES for a portfolio of assets using R
Using : log return for 10 days and simple return for portfolio with 4 years historical data
HSVaR10.r    Select all
# get the time series of MSFT, YHOO, AAPL, ORCL and IBM for 260 trading days msft <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=MSFT&a=2&b=20&c=2010&d=2&e=31&f=2014&g=d") yhoo <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=YHOO&a=2&b=20&c=2010&d=2&e=31&f=2014&g=d") aapl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=AAPL&a=2&b=20&c=2010&d=2&e=31&f=2014&g=d") orcl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=ORCL&a=2&b=20&c=2010&d=2&e=31&f=2014&g=d") ibm <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=IBM&a=2&b=20&c=2010&d=2&e=31&f=2014&g=d") # calculate log returns from Closed Prices and takes first 1000 samples only msft.logreturn10 = c(diff(-log(msft$Close),10)); yhoo.logreturn10 = c(diff(-log(yhoo$Close),10)); aapl.logreturn10 = c(diff(-log(aapl$Close),10)); orcl.logreturn10 = c(diff(-log(orcl$Close),10)); ibm.logreturn10 = c(diff(-log(ibm$Close),10)); logreturns10 <- data.frame(msft.logreturn10, yhoo.logreturn10, aapl.logreturn10, orcl.logreturn10, ibm.logreturn10)[1:1000,]; # convert to simple returns simplereturns10 <- (exp(logreturns10)-1) # find VaR for 5% percentile for 10 days holding period, that is $259,314.5 -quantile(as.matrix(simplereturns10) %*% rep(1e+6,5), 0.05); #find 10-Days expected Shortfall (ES) that is $343,451.5 m <- as.matrix(simplereturns10) %*% rep(1e+6,5); v <- quantile(m, 0.05); -mean(m[m <= v]); # plot all hist(as.matrix(simplereturns10) %*% rep(1,5),col="green",freq=FALSE,main="Portfolio Returns for Holding Period 10 days", xlab="Return%",xlim=range(-0.6,0.6),ylim=range(0,4));lines(density(as.matrix(simplereturns10) %*% rep(1,5)),col="red", lty="dotted",panel.last=abline(v=quantile(as.matrix(simplereturns10) %*% rep(1,5), 0.05), col="darkred",lwd=3)); mtext("95% C.I.",side=3,line=-3,adj=0.20,col="darkred"); #List the 51st item sort by portfolio return #587 0.004545455 0.00523416 -0.002571286 -0.003148254 -0.01536958 -0.2592768 returns10 <- data.frame(simplereturns, portfolio=as.matrix(simplereturns10) %*% rep(1,5)); tail(head(returns10[order(returns10$portfolio),],n=length(returns10[,1])*0.05+1),n=1);


 (8.3) Historical Simulation VaR and ES for a portfolio of assets using R and Package PerformanceAnalytics
Backtesting : if Actual Return < Theoretical VaR
constrOptim Ref : http://www.youtube.com/watch?v=MCvz-c6UUkw
HSVaR21.r    Select all
require(PerformanceAnalytics) # get the time series of MSFT, YHOO, AAPL, ORCL and IBM msft <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=MSFT&a=0&b=1&c=2012&d=2&e=31&f=2014&g=d") yhoo <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=YHOO&a=0&b=1&c=2012&d=2&e=31&f=2014&g=d") aapl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=AAPL&a=0&b=1&c=2012&d=2&e=31&f=2014&g=d") orcl <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=ORCL&a=0&b=1&c=2012&d=2&e=31&f=2014&g=d") ibm <-read.csv("http://ichart.finance.yahoo.com/table.csv?s=IBM&a=0&b=1&c=2012&d=2&e=31&f=2014&g=d") stockportfolio.ts <- as.xts(data.frame(msft$Close,yhoo$Close,aapl$Close,orcl$Close,ibm$Close), order.by = as.Date(msft$Date,"%Y-%m-%d")) # calculate VaR 0.0215341 0.01744393 0.02875233 0.02034777 0.01493424 -VaR(Return.calculate(stockportfolio.ts[1:251], method="discrete"),p=.95,method="historical") # calculate ES 0.02622294 0.02608046 0.03779529 0.02482448 0.02314859 -ETL(Return.calculate(stockportfolio.ts[1:251], method="discrete"),p=.95,method="historical") # Backtesting p=0.95 assuming equal weights # i Actual Theoretical # 2013-06-20 368 -0.128825 0.1031542 # 2013-06-21 369 -0.1229124 0.1039837 # 2013-07-19 388 -0.1754735 0.103187 # 2013-08-27 415 -0.1080645 0.1028595 # 2014-02-03 524 -0.1150287 0.1047768 for (i in 1:(nrow(stockportfolio.ts)-251)) { actual <- sum(na.omit(Return.calculate(stockportfolio.ts[(251+i-1):(251+i)]))); theoretical <- sum(-VaR(Return.calculate( stockportfolio.ts[(i):(251+i-1)],method="discrete"),p=.95,method="historical")); if (any(actual < -theoretical)) { print(cbind(stockportfolio.ts[(251+i),0],i=251+i,Actual=actual,Theoretical=theoretical)) } } # Backtesting p=0.99 assuming equal weights # i Actual Theoretical # 2013-07-19 388 -0.1754735 0.1667104 for (i in 1:(nrow(stockportfolio.ts)-251)) { actual <- sum(na.omit(Return.calculate(stockportfolio.ts[(251+i-1):(251+i)]))); theoretical <- sum(-VaR(Return.calculate( stockportfolio.ts[(i):(251+i-1)],method="discrete"),p=.99,method="historical")); if (any(actual < -theoretical)) { print(cbind(stockportfolio.ts[(251+i),0],i=251+i,Actual=actual,Theoretical=theoretical)) } } # Backtesting p=0.99 assuming weights of c(30,50,200,50,50) # i Actual Theoretical # 2013-01-24 266 -23.5823 13.22212 # 2014-01-28 520 -13.10302 12.38539 for (i in 1:(nrow(stockportfolio.ts)-251)) { actual <- sum(na.omit(Return.calculate(stockportfolio.ts[(251+i-1):(251+i)])) * c(30,50,200,50,50)); theoretical <- sum(-VaR(Return.calculate( stockportfolio.ts[(i):(251+i-1)],method="discrete"),p=.99,method="historical") * c(30,50,200,50,50)); if (any(actual < -theoretical)) { print(cbind(stockportfolio.ts[(251+i),0],i=251+i,Actual=actual,Theoretical=theoretical)) } } # Use constraint optimisation to find the weight of each share of the portfolio # optimise function g and to minimise VaR portfolioVaR <- -VaR(Return.calculate(stockportfolio.ts[1:251], method="discrete"),p=.95,method="historical"); g <- function(x) {sum(portfolioVaR* x)}; # constraint matrix A <- matrix(c(rep(1,5),rep(-1,5),diag(1,5),diag(-1,5)),nrow=2+5+5, ncol=5, byrow=TRUE); # constraint values : value of portfolio = 500 and shareholding for each stock > 0 that is no short selling b <- 500 * c(0.999999,-1.000-0.000001,rep(0.000001,5),rep(-1.000-0.000001,5)); # run constrOptim with initial value of 100 each # $par = 0.1113882 58.9182990 0.4609939 78.1495814 362.3602354 # $value = 8.045164 constrOptim(theta=rep(100,5),grad=NULL,f=g,ui=A,ci=b) # Backtesing again and plot the graph for Actual vs. Theoretical results based on the above optimised portfolio # i Actual Theoretical # 2013-04-19 325 -28.85769 12.51676 # 2013-10-17 451 -24.06681 13.6749 result.frame = data.frame() for (i in 1:(nrow(stockportfolio.ts)-251)) { actual <- sum(na.omit(Return.calculate(stockportfolio.ts[(251+i-1):(251+i)])) * c(0.1113882,58.9182990,0.4609939,78.1495814,362.3602354)); theoretical <- sum(-VaR(Return.calculate( stockportfolio.ts[(i):(251+i-1)],method="discrete"),p=.99,method="historical") * c(0.1113882,58.9182990,0.4609939,78.1495814,362.3602354)); if (any(actual < -theoretical )) { print(cbind(stockportfolio.ts[(251+i),0],i=251+i,Actual=actual,Theoretical=theoretical)); }; result.frame <- rbind(result.frame, data.frame(stockportfolio.ts[(251+i),],i=(251+i), Actual=actual,Theoretical=-theoretical)); } result.zoo <- as.zoo(result.frame) plot(x = result.zoo[,7:8], screens = 1, main="Actual vs. Theoretical VaR", xlab = "Time", ylab = "Return", col = c("darkgreen", "red")); legend(x = "bottomright", legend = c("Actual","Theoretical"),lty=1,col=c("darkgreen", "red")); # revise the optimisation function to include portfolioLogReturn and to minimise risk / return ratio #$par 1.177901 111.451867 142.695042 143.658599 101.017062 portfolioLogReturn <- colSums(Return.calculate(stockportfolio.ts[1:251], method="log"),na.rm=TRUE) g1 <- function(x) {sum(portfolioVaR * x)/sum(portfolioLogReturn * x)}; A <- matrix(c(rep(1,5),rep(-1,5),diag(1,5),diag(-1,5)),nrow=2+5+5, ncol=5, byrow=TRUE); b <- 500 * c(0.999999,-1.000-0.000001,rep(0.000001,5),rep(-1.000-0.000001,5)); constrOptim(theta=rep(100,5),grad=NULL,f=g1,ui=A,ci=b) # rerun backtesting with portfolio weights = c(1.177901,111.451867,142.695042,143.658599,101.017062) result.frame = data.frame() for (i in 1:(nrow(stockportfolio.ts)-251)) { actual <- sum(na.omit(Return.calculate(stockportfolio.ts[(251+i-1):(251+i)])) * c(1.177901,111.451867,142.695042,143.658599,101.017062)); theoretical <- sum(-VaR(Return.calculate( stockportfolio.ts[(i):(251+i-1)],method="discrete"),p=.99,method="historical") * c(1.177901,111.451867,142.695042,143.658599,101.017062)); if (any(actual < -theoretical )) { print(cbind(stockportfolio.ts[(251+i),0],i=251+i,Actual=actual,Theoretical=theoretical)); }; result.frame <- rbind(result.frame, data.frame(stockportfolio.ts[(251+i),],i=(251+i), Actual=actual,Theoretical=-theoretical)); } result2.zoo <- as.zoo(result.frame) plot(x = result2.zoo[,7:8], screens = 1, main="Actual vs. Theoretical VaR with optimised portfolio", xlab = "Time", ylab = "Return", col = c("darkgreen", "red")); legend(x = "topleft", legend = c("Actual","Theoretical"),lty=1,col=c("darkgreen", "red"));



There is one important difference between log return and simple return is that log return is time additive and simple return is portfolio additive. (Reference : http://www.youtube.com/watch?v=PtoUlt3V0CI)