1. Million Dollar Challenge
Latest Threads

Forum Statistics
  • Forum posts:226
  • Forum threads:59
  • Members:13,493
  • Latest member:g6syqjd295

  Bollinger Bands EMA Indicator

Posted by: Hassam
06-24-2018, 02:06 PM
Forum: MQL4
- Replies (1)

The default MT4 Bollinger Bands indicator does not have as option for using a EMA instead of a SMA. I have coded a custom Bollinger Bands indicator that has the EMA option. Below is a screenshot of the Bollinger Bands with EMA. I have chosen 2 standard deviation with EMA 21.

[Image: BBEMA.png]

If you want the MQL4 code for the custom Bollinger Bands with EMA option as well as the SMA option, you can copy that too from below:

Code:
//+------------------------------------------------------------------+
//|                                                       BB_EMA.mq4 |
//|                                                     Ahmad Hassam |
//|                                      https://www.doubledoji.com/ |
//+------------------------------------------------------------------+
#property copyright "Ahmad Hassam"
#property link      "https://www.doubledoji.com/"
#property version   "1.00"
#property strict
#property indicator_chart_window
#property indicator_buffers 3
#property indicator_color1 Red
#property indicator_color2 DeepSkyBlue
#property indicator_color3 DeepSkyBlue



// External parameters
//choose the number of bars to use
input int BandsPeriod = 21; //Smoothing Period
input int BandsShift = 0;   //Shift
//---choose which averaging method to use
input ENUM_MA_METHOD BandsMethod = MODE_EMA;  //Smoothing Methods
//---choose which price to use
input ENUM_APPLIED_PRICE priceBB = PRICE_CLOSE; //Applied Price

//---choose the number of standard deviations to use
input int Deviations = 2; // No Of Standard Deviations
extern int K=20; // Standard Deviation Lookback
double StdDev=0.0;
//---- buffers
double EMA[];
double upperBand[];
double lowerBand[];

//+------------------------------------------------------------------+
//| Custom indicator initialization function |
//+------------------------------------------------------------------+
int init()
{
//---- indicators
SetIndexStyle(0,DRAW_LINE);
SetIndexBuffer(0,EMA);
SetIndexLabel(0,"EMA");
SetIndexStyle(1,DRAW_LINE);
SetIndexBuffer(1,upperBand);
SetIndexLabel(1,"UpperBand");
SetIndexStyle(2,DRAW_LINE);
SetIndexBuffer(2,lowerBand);
SetIndexLabel(2,"LowerBand");
//----

return(0);
}

//+------------------------------------------------------------------+
//| Custom Bollinger Bands Indicator                              |
//+------------------------------------------------------------------+

// start() function
  int start()
 {
 int Count=0;
 int limit=0;
 int counted_bars=IndicatorCounted();
 int CalculateBars = Bars - counted_bars;
//  double StdDev;

/*
  calculate the moving average
  double  iMA(
  string       symbol,           // symbol
  int          timeframe,        // timeframe
  int          ma_period,        // MA averaging period
  int          ma_shift,         // MA shift
  int          ma_method,        // averaging method
  int          applied_price,    // applied price
  int          shift             // shift
  );
 
calculate the standard deviation  
  double  iStdDev(
  string       symbol,           // symbol
  int          timeframe,        // timeframe
  int          ma_period,        // MA averaging period
  int          ma_shift,         // MA shift
  int          ma_method,        // MA averaging method
  int          applied_price,    // applied price
  int          shift             // shift
  );
*/
// NULL means current chart symbol
// 0 means current chart timeframe
if (counted_bars==0)
     {
     limit=Bars-1;
     
     }
  if (counted_bars>0)
     {
     limit=CalculateBars-1;
     
     }

for(Count = limit-40; Count >= 0; Count--)
{
        EMA[Count] = iMA(NULL,0,BandsPeriod,BandsShift,BandsMethod,priceBB,Count);
        StdDev = iStdDev(NULL,0,BandsPeriod,BandsShift,BandsMethod,priceBB,Count);
        upperBand[Count] = EMA[Count] + (StdDev * Deviations);
        lowerBand[Count] = EMA[Count] - (StdDev * Deviations);
       
       
}
//----
  return(0);
 }
//+------------------------------------------------------------------+

Print this item

  Crude Oil Daily Market Analysis

Posted by: Hassam
06-08-2018, 06:04 AM
Forum: Commodities, Stocks And Indices
- Replies (1)

Crude oil is a very important commodity in high demand globally. Crude oil is highly susceptible to breaking news and events especially in Middle East. Right now crude oil price is hovering around $65 per barrel. I have posted the screenshot below that you should take a look at:

[Image: USOIL.H4.png]


Now do you see the red arrow in the above screenshot of US Crude OIL H4 chart? This red arrow is the time when US President Donald Trump announced that US will rescind US IRAN Nuclear deal and reimpose sanctions. You can see in the above screenshot first the market got jittery about the outcome of this annoucement. Before the announcement many analysts were skeptical that US President will rescind the nuclear deal. So crude  oil price went down. But when the announcement was made crude oil prices jumped and went above $71 per barrel. This was the start of the last month around 8th May.

Today crude oil prices are hovering around $65 per barrel. So prices have go down a little bit as Saudi Arabia has started pumping more oil. The point was to make you realize that the commodities market is highly susceptible to breaking news. Breaking news causes a lot of volatility and as a trader you should understand how to manage this volatility and profit from it.

Print this item

  Will Bitcoin Survive The Next Decade?

Posted by: Hassam
06-08-2018, 05:39 AM
Forum: Cryptocurrencies
- Replies (16)

Bitcoin has captured the imagination of millions of investors in the last year and a half. Many analysts at Wall Street however derisively call Bitcoin a bubble. BItcoin has had all the hall mark of a bubble. People have been comparing Bitcoin bubble with the Tulip Mania that gripped Netherlands in the 17th century. I have read lot of warning on CNBC and Bloomberg issued by central banks asking investors to be careful with the bitcoin mania as it has all the hall marks of a bubble. Last year Bitcoin made the staggering achievement of going above $20K for a small time. After that the market snapped and things started to unravel. Bitcoin prices plunged as fast as they had risen. Take a look at the following Bitcoin weekly chart.
[Image: BTCUSDW1.png]
You can see in the above BTCUSD weekly chart very big bullish and bearish weekly candles.

I have started this thread in which I will discuss whether bitcoin will survive into the next decade or not. Most analysts at Wall Street believe bitcoin represents nothing and its ultimate value will be $0. Right now BTCUSD is trading around $7K so it still have got a long way to go before it plunges down to nothing. Personally I was shocked to know that bitcoin mining is a very energy intensive operation and daily bitcoin mining consumes more energy that the country Ireland. If this is true than bitcoin mining has a huge environmental cost and in the long run this thing will drive its value down to zero.

Print this item

  Quadratic Discriminant Analysis With R For Algorithmic Trading

Posted by: Hassam
06-08-2018, 04:51 AM
Forum: Algorithmic Trading With R
- Replies (1)

Most of the time as traders we want to predict market movement. It is difficult to predict the precise market movement. However if we divide the market movement into classes like UP or DOWN, we can predict the market movement. This approach only tells us the market direction but it doesn't give us the magnitude. We can further improve the classification by dividing the movement as BIG and SMALL so the classes will BIG UP, SMALL UP and BIG DOWN and SMALL DOWN.

This is what I do as a forex trader. I divide the currency market movement like  this. Currency market movement less than 50 pips is considered as small and market movement above 50 pips is considered as BIG. Reading the book, Introduction to Statistical Learning, Quadratic Discriminant Analysis has been mentioned. In an example the authors use the 2 lags of DOW JONES daily returns to predict tomorrow's market direction and get an accuracy of 61%.

Quadratic Discriminant Analysis (QDA) is an improved version of Linear Discriminant Analysis (LDA) algorithm. In LDA, all classes have the same mean and covariance matrix. In QDA, we make a new assumption that all classes have a different covariance matrix. Each class has got a Gaussian distribution. I decided to test QDA for EURUSD, GBPUSD and USDJPY daily returns.


Code:
#Quadratic Discriminant Model
# Import the csv file
data1 <- read.csv("D:/Shared/MarketData/GBPUSD1440.csv",
                 header=FALSE)
colnames(data1) <- c("Date", "Time", "Open", "High",
                    "Low", "Close", "Volume")
library(quantmod)
data1 <- as.xts(data1[,-(1:2)],
               as.POSIXct(paste(data1[,1],data1[,2]),
                          format='%Y.%m.%d %H:%M'))

tail(data1)
#number of rows
x1 <- nrow(data1)

#calculate simple returns
data1$lr <- diff(log(data1$Close))

data3 <- cbind(data1$lr,data1$lr,data1$lr)
data3$lr.2 <- lag(data3$lr.2, k=1)
data3$lr.1 <- lag(data3$lr.2, k=1)

tail(data3)
data3 <- as.data.frame(data3)
data3$Direction <- ifelse(data3[, 1] >=0, 1, 0)

data3$Direction <- factor(data3$Direction, levels=c(1,0),
                         labels=c("UP","DOWN"))
tail(data3)
data3 <- na.omit(data3)
is.factor(data3$Direction)
train <- data3[1:(nrow(data3)-500),]
test <- data3[-(1:(nrow(data3)-500)),]
library(MASS)
qda.fit=qda(Direction~lr.1+lr.2 ,data=train)
qda.class =predict(qda.fit,test[,2:3])$class
table(qda.class, test$Direction)
mean(qda.class==test$Direction)

#fit logistic regression model
logistic.fit<-glm(Direction~lr.1+lr.2,
                 family=binomial(), data=train)
summary(logistic.fit)
logistic.prob <- predict(logistic.fit, test[,2:3],
                        type="response")
logistic.prob[2:4]
contrasts(train$Direction)
#test$Logistic <-rep("UP")
#test$Logistic[logistic.prob > 0.5] <- "DOWN"
logistic.pred <- ifelse(logistic.prob > 0.5, 1, 0)
test$Logistic <- factor(logistic.pred, levels=c(0,1),
                       labels=c("UP","DOWN"))
table(logistic.pred)
table(test$Logistic, test$Direction)
#calculate acccuracy on the training data
library(caret)
confusionMatrix(test$Direction, test$Logistic)

Above I have posted QDA algorithmic code. I test this QDA R script and found that it achieved 57% accuracy on EURUSD, 54% on GBPUSD and 51% on USDJPY. So I was unable to achieve above 60% winrate on any of the currency pairs. If we use Logistic Regression, we achieve a winrate of just 49% with 2 lags of daily return. So QDA did improve the predictive accuracy to above 55% on average. This is what I am thinking. Improve this QDA algorithm winrate above 65% and then use Ensemble Learning to build a few models with above 65% predictive accuracy and then combine them into one model.

Print this item

  Backtesting MACD Trading Strategy With R

Posted by: Hassam
05-30-2018, 03:26 AM
Forum: Algorithmic Trading With R
- No Replies

It is very important for you to measure the accuracy of a trading strategy on the historical data. This can clarify a lot about the trading strategy. I have an MACD Trading Strategy in mind. MACD (Moving Average Convergence/Divergence) is a powerful oscillator that is used to measure the momentum in the market. When MACD is rising we have an up momentum in the market. Similarly when MACD is decreasing we have a down momentum in the market. 

MACD Trading Strategy
First we determine the market direction on daily timeframe using MACD. If MACD is increasing on daily timeframe we will assume market direction as up. Similarly if MACD is decreasing on daily timeframe we will assume market direction as down. We will use M5 and M15 to enter into a trade in the direction of daily timeframe. You can read about this MACD Trading Strategy and download its MQL4 code in this thread.

Important question that comes to mind is what is the accuracy of this MACD Trading Strategy. First we determine the accuracy of the MACD Trading Strategy on daily timeframe. In MQL4 backtesting is a cumbersome and tedious job. With R you can easily do the backtesting on historical data in just a few minutes. Below is the R code that will backtest the MACD Trading Strategy on daily timeframe. 

> #backtesting the MACD Trading Strategy
> #Regression Model
> # Import the csv file
> data1 <- read.csv("D:/Shared/MarketData/GBPUSD1440.csv",
+                   header=FALSE)
> colnames(data1) <- c("Date", "Time", "Open", "High",
+                      "Low", "Close", "Volume")
> library(quantmod)
Loading required package: xts
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Loading required package: TTR
Version 0.4-0 included new data defaults. See ?getSymbols.
Learn from a quantmod author: https://www.datacamp.com/courses/importi...-data-in-r
> data1 <- as.xts(data1[,-(1:2)],
+                 as.POSIXct(paste(data1[,1],data1[,2]),
+                            format='%Y.%m.%d %H:%M'))
>
> tail(data1)
              Open    High     Low   Close Volume
2018-01-30 1.40719 1.41656 1.39787 1.41542  55936
2018-01-31 1.41540 1.42315 1.41206 1.41980  60953
2018-02-01 1.41983 1.42772 1.41585 1.42649  52142
2018-02-02 1.42650 1.42769 1.41017 1.41153  54029
2018-02-04 1.41074 1.41125 1.40816 1.41094   2790
2018-02-05 1.41090 1.41493 1.39986 1.40001  33006
> #number of rows
> x1 <- nrow(data1)
>
> data1$MACD <- MACD(data1$Close, 12, 26, 9, maType="EMA" )
> data1 <- data1[,-7]
> data1 <- data1[, -(1:3)]
> data1 <- data1[,-2]
> data1$Close <- lag(data1[,1], k=-1)
> data1 <- cbind(data1, data1[,2], data1[,2])
> data1 <- data1[, c(1,4,3,2)]
> data1$macd.1 <- lag(data1$macd.1, k=1)
> data1$macd <- lag(data1$macd.1, k=1)
> data1$Direction <- diff(data1$Close)
> data1 <- data1[-(1:30),]
> data1 <- as.data.frame(data1)
> data1$Direction <- ifelse(data1$Direction > 0, 1, -1)
> #downtrend
> data1$Prediction <- ifelse(((data1$macd.2 < data1$macd.1 &
+                              data1$macd < data1$macd.1) |
+                               (data1$macd.2 > data1$macd.1 &
+                                  data1$macd.1 > data1$macd)),-1,
+                     ifelse(((data1$macd.2 > data1$macd.1 &
+                         data1$macd > data1$macd.1) |
+                           (data1$macd.2 < data1$macd.1 &
+                              data1$macd.1 < data1$macd)),1 , 0))
>
> tail(data1)
             Close    macd.2   macd.1     macd Direction Prediction
2018-01-30 1.41980 1.2267670 1.231510 1.279458         1          1
2018-01-31 1.42649 1.2336942 1.226767 1.231510         1          1
2018-02-01 1.41153 1.2626800 1.233694 1.226767        -1         -1
2018-02-02 1.41094 1.1858707 1.262680 1.233694        -1         -1
2018-02-04 1.40001 1.1087917 1.185871 1.262680        -1          1
2018-02-05      NA 0.9737999 1.108792 1.185871        NA          1
> head(data1)
            Close     macd.2     macd.1       macd Direction Prediction
2011-06-23 1.5957 -0.4486519 -0.3784313 -0.3005196        -1          1
2011-06-24 1.5951 -0.5298428 -0.4486519 -0.3784313        -1          1
2011-06-26 1.5985 -0.5906509 -0.5298428 -0.4486519         1          1
2011-06-27 1.5985 -0.6149452 -0.5906509 -0.5298428        -1          1
2011-06-28 1.6064 -0.6270967 -0.6149452 -0.5906509         1          1
2011-06-29 1.6018 -0.5903830 -0.6270967 -0.6149452        -1          1
> data1 <- data1[,c(1,2,3,4,6,5)]
> library(caret)
Loading required package: lattice
Loading required package: ggplot2
Find out what's changed in ggplot2 at http://github.com/tidyverse/ggplot2/releases.
> #calculate acccuracy on the training data
> confusionMatrix(data1$Direction, data1$Prediction)
Confusion Matrix and Statistics

          Reference
Prediction  -1   1
        -1 537 503
        1  511 509
                                          
               Accuracy : 0.5078          
                 95% CI : (0.4859, 0.5296)
    No Information Rate : 0.5087          
    P-Value [Acc > NIR] : 0.5439          
                                          
                  Kappa : 0.0154          
 Mcnemar's Test P-Value : 0.8260          
                                          
            Sensitivity : 0.5124          
            Specificity : 0.5030          
         Pos Pred Value : 0.5163          
         Neg Pred Value : 0.4990          
             Prevalence : 0.5087          
         Detection Rate : 0.2607          
   Detection Prevalence : 0.5049          
      Balanced Accuracy : 0.5077          
                                          
       'Positive' Class : -1   

You can check above the accuracy of MACD Trading Strategy on daily timeframe is just 50% which is just like flipping the coin.

Print this item

  MACD Trading Strategy

Posted by: Hassam
05-30-2018, 03:16 AM
Forum: MQL4
- Replies (2)

This is a simple MACD Trading Strategy. We first determine market direction on daily timeframe then we use M5 or M15 timeframe to trade in the direction of daily trend. Below is the screenshot of this MACD Trading Strategy:
[Image: MACDStrategy.png]

Below arrow is showing the sell signal. Below is the MQL4 code for this MACD Trading Strategy:

//+------------------------------------------------------------------+
//|                                                                     Z2.mq4 |
//|                                                          Ahmad Hassam |
//|                                      https://www.doubledoji.com/ |
//+------------------------------------------------------------------+
#property copyright "Ahmad Hassam"
#property link      "https://www.doubledoji.com/"
#property version   "1.00"
#property strict
#property indicator_chart_window
#property indicator_buffers 2
#property indicator_color1 Red
#property indicator_color2 Blue
#property indicator_width1 2
#property indicator_width2 2

//Buffer Arrays
double buySignal[];
double sellSignal[];

//Higher Timeframe
int higherTimeframe=1440; //daily timeframe=1440, Weekly timeframe=10880
int higherTimeframeBar=0;

//Trend direction on the Higher Timeframe
int trendDirection = 0;

//MACD Higher Timeframe
double macdHigher1=0.0;
double macdHigher2=0.0;
double macdHigher3=0.0;

//+------------------------------------------------------------------+
//| Custom indicator initialization function                         |
//+------------------------------------------------------------------+
int OnInit()
  {
 
//custom indicator buffers mapping
IndicatorBuffers(2);

//initialize the custom indicator settings
SetIndexBuffer(0,buySignal);
SetIndexStyle(0,DRAW_ARROW);
SetIndexArrow(0,233);
SetIndexEmptyValue(0, 0.0);
SetIndexLabel(0,"Buy Signal");
SetIndexBuffer(1,sellSignal);
SetIndexStyle(1,DRAW_ARROW);
SetIndexArrow(1,234);
SetIndexEmptyValue(1,0.0);
SetIndexLabel(1,"Sell_Signal");
     
//--- indicator buffers mapping

IndicatorShortName("Trend Trader");


      
//---
   return(0);
  }
//+------------------------------------------------------------------+
//| Custom indicator iteration function                              |
//+------------------------------------------------------------------+
int OnCalculate (const int rates_total,      // size of input time series
                 const int prev_calculated,  // bars handled in previous call
                 const datetime& time[],     // Time
                 const double& open[],       // Open
                 const double& high[],       // High
                 const double& low[],        // Low
                 const double& close[],      // Close
                 const long& tick_volume[],  // Tick Volume
                 const long& volume[],       // Real Volume
                 const int& spread[]         // Spread
   )
   
   
  {
 
      
        ArraySetAsSeries(buySignal,true);
      ArraySetAsSeries(sellSignal,true);
      
      int bars = rates_total - 1;
      if (prev_calculated > 0) bars =  rates_total - (prev_calculated - 1);
      
      for ( int i =  bars; i >= 0; i--)
      {

      //determine the main trend on the daily timeframe
        higherTimeframeBar= i/96;
          
          macdHigher1=iMACD(NULL,PERIOD_D1,12,26,9,
          PRICE_CLOSE,MODE_MAIN,higherTimeframeBar+1);
          macdHigher2=iMACD(NULL,PERIOD_D1,12,26,9,
          PRICE_CLOSE,MODE_MAIN,higherTimeframeBar+2);
          macdHigher3=iMACD(NULL,PERIOD_D1,12,26,9,
          PRICE_CLOSE,MODE_MAIN,higherTimeframeBar+3);
      //downtrend on higher timeframe
      if (macdHigher3 < macdHigher2 &&
      macdHigher1 < macdHigher2)  trendDirection =-1;
      //else trendDirection =0;
      if (macdHigher3 > macdHigher2 && macdHigher2 > macdHigher1)
      trendDirection =-1;
      //else trendDirection =0;
      //uptrend on higher timeframe
      if (macdHigher3 > macdHigher2 &&
      macdHigher1 > macdHigher2)  trendDirection =+1;
     // else trendDirection =0;
      if (macdHigher3 < macdHigher2 && macdHigher2 < macdHigher1)
      trendDirection =1;
     // else trendDirection =0;    
      //determine the local trend on MACD M30
      double macdM15_1=iMACD(NULL,PERIOD_M15,12,26,9,PRICE_CLOSE,MODE_MAIN,i+1);
      double macdM15_2=iMACD(NULL,PERIOD_M15,12,26,9,PRICE_CLOSE,MODE_MAIN,i+2);
      double macdM15_3=iMACD(NULL,PERIOD_M15,12,26,9,PRICE_CLOSE,MODE_MAIN,i+3);
      //Generate buy signals
      if ( trendDirection==1 && macdM15_3 > macdM15_2 && macdM15_1 > macdM15_2 &&
      macdM15_1 < 0 && macdM15_2 < 0 && macdM15_3 < 0)
         buySignal[i]= Low[i] - 50*Point;
      
      //Generate sell signals
      if ( trendDirection==-1 && macdM15_3 < macdM15_2 && macdM15_1 < macdM15_2 &&
      macdM15_1 > 0 && macdM15_2 > 0 && macdM15_3 > 0)
         sellSignal[i]= High[i] + 50*Point;
            
      }
      //return value of prev_calculated for the next call
      return(rates_total);
  }
//+------------------------------------------------------------------+

Print this item

  Pandas For Traders

Posted by: Hassam
05-24-2018, 03:34 AM
Forum: Algorithmic Trading With Python
- Replies (9)

Pandas is a very powerful Python library that you must master if you want to use Python in algorithmic trading. Pandas provide you with the ability to slice and dice a dataframe in many ways that can make your job very easy. Pandas is written in C. It is very fast. Dataframe is a very important concept. Dataframe is basically a table with many rows and columns. Each column represents an attribute. Each row is a record. 

When developing algorithmic trading strategies, you will have to deal with dataframes a lot. Reading a dataframe is very easy with Pandas. I have written the following python code that you can use to read the data csv files into a dataframe. I define a function that reads the csv file. We just specify the currency pair and the timeframe. CSV file should be saved on your hard drive. If you have Meta Trader 4 installed on your computer, you can download the csv files for different currency pairs for different timeframes from its history center:

# Data fetching
def get_data(currency_pair, timeframe):
    link='D:/Shared/MarketData/{}{}.csv'.format(currency_pair,\
                               timeframe)
    data1 = pd.read_csv(link, header=None)
    data1.columns=['Date', 'Time', 'Open', 'High', 'Low',
                'Close', 'Volume']
    # We need to merge the data and time columns
    # convert that column into datetime object
    data1['Datetime'] = pd.to_datetime(data1['Date'] \
         + ' ' + data1['Time'])
    #rearrange the columnss with Datetime the first
    data1=data1[['Datetime', 'Open', 'High',
                 'Low', 'Close', 'Volume']]
    #set Datetime column as index
    data1 = data1.set_index('Datetime')
    return(data1)

Above function can read the csv files saved on your hard drive.

df = get_data('GBPUSD', 1440)
df.shape
df.head()

Out[12]:
                    Open    High     Low   Close  Volume
Datetime                                          
2011-05-19  1.6156  1.6241  1.6129  1.6227   12390
2011-05-20  1.6226  1.6303  1.6165  1.6229   11879
2011-05-22  1.6224  1.6228  1.6209  1.6227     663
2011-05-23  1.6226  1.6232  1.6057  1.6072   12852
2011-05-24  1.6073  1.6207  1.6067  1.6177   12635

You can see the format of the dataframe that Python has read. Datetime is the index. While defining the function, we have converted date and time into a datetime column that is picked up by Pandas and used as an index. Below is the command that you can use to check if datetime is the index or not:

isinstance(df.index, pd.DatetimeIndex)

Resampling the Dataframe
With pandas we have the ability to resample the dataframe into different frequencies mostly higher. For example, you have read the GBPUSD 15 minute Open, High, Low, Close OHLC data. We can use pandas to easily resample that dataframe into a 30 minute OHLC data, 60 minute OHLC data, 240 minute, 1440 minute and more. First we need to define a dictionary.

ohlc_dict = {
    'Open':'first',
    'High':'max',
    'Low':'min',
    'Close':'last',
    'Volume':'sum'
    }

Now we can easily resample the dataframe with the following commands:

#resample the intraday OHLC into Daily OHLC
df240Mn=df.resample('240min').agg(ohlc_dict).dropna()
dfDaily=df.resample('1D').agg(ohlc_dict).dropna()
dfDaily.head()
#resample the intraday OHLC into daily OHLC
dfWeekly=df.resample('W-Fri').agg(ohlc_dict).dropna()
dfWeekly.head()

Below we read GBPUSD 15 minute data:
Out[16]:
                        Open     High      Low    Close  Volume
Datetime                                                       
2017-11-23 14:00:00  1.33113  1.33147  1.33066  1.33078     716
2017-11-23 14:15:00  1.33080  1.33110  1.33058  1.33085     620
2017-11-23 14:30:00  1.33087  1.33113  1.33070  1.33094     476
2017-11-23 14:45:00  1.33103  1.33149  1.33038  1.33046     273
2017-11-23 15:00:00  1.33045  1.33065  1.32988  1.33042     351

Using the above code we easily converted this GBPUSD 15 minute data into GBPUSD daily data:
Out[19]:
                        Open     High      Low    Close   Volume
Datetime                                               
2017-11-23  1.33113  1.33149  1.32913  1.33045  11292.0
2017-11-24  1.33046  1.33588  1.32777  1.33365  36115.0
2017-11-26  1.33224  1.33314  1.33170  1.33229   1550.0
2017-11-27  1.33228  1.33820  1.33044  1.33199  36031.0
2017-11-28  1.33200  1.33860  1.32199  1.33634  51388.0

Everything is done by pandas internally. More on pandas tomorrow so stay tuned!

Print this item

  Creating MT5 Rest API

Posted by: Hassam
05-07-2018, 11:30 AM
Forum: MQL5
- No Replies

MT5 is the latest version of the MT4 trading platform by MetaQuotes corporation,. MQL4 programming language which was developed for MT4 had limitations. Those limitations were overcome with the development of MT5 trading platform and the accompanying MQL5 programming language. MQL5 is a object oriented programming language just like C++/Java/Python. Although MQL5 is now an object oriented programming language, it still lacks the power and fury of C++/Java/Python.

When we try to build predictive models using machine learning, we realize how limited MQL5 is. MQL5 has almost no machine learning, deep learning and reinforcement learning libraries. How to overcome this situation? We can overcome this situation by somehow finding a way to connect C++/Java/Python with MT5. Can we do that? I have started this thread in which I will show how we can convert MT5 into a REST API that C++/Java/Python can connect and exchange data with.

MT5 has a client server architecture. Each broker has the MT5 server while the trading platforms that we use on your desktops and smartphones is the client. Broker MT5 server broadcasts the price data or what we call the price data. Each client trading platform can receive that market data. We can place trading orders on the client trading platform that then get communicated to the broker MT5 server. Brokers can provide APIs. Clients don't have that privilege. But knowing a little bit of coding, we can try to convert our MT5 client into a REST API. What is a REST API? Let's start from here!

REST API
REST stands for Representational State Transfer. REST is basically a webserver that allows the clients to exchange data with the server. Most of the modern programming languages like C++/Java/Python have powerful REST libraries that we can use with the REST webserver which will be the MT5 client on your desktop in this case. MQL5 has this command function webrequest that we can use to achieve our purpose:

int  WebRequest(
   const string      method,          
// HTTP method
   const string      url,             
// URL
   const string      headers,         
// headers 
   int               timeout,         
// timeout
   const char        &data[],         
// the array of the HTTP message body
   char              &result[],       
// an array containing server response data
   string            &result_headers  
// headers of server response
   );

Print this item

  Foreign Exchange Market Microstructure

Posted by: Hassam
05-03-2018, 10:17 AM
Forum: Financial Econometrics
- Replies (1)

Currency market or what we call the foreign exchange market microstructure characteristics are not apparent when we view the low frequency data like the monthly, weekly and the daily. Foreign exchange market is divided into different geographical segments with the presence of different types of agents. FX market consists of three parts namely FX spot market, FX Forward market and the FX derivatives market. 

A large portion of the FX spot transactions go through the automated electronic order matching system that includes Electronic Brokering Services (EBS) and Reuters Dealing.  In addition to this electronic trading, over the counter FX spot market is a direct market between the banks and the brokers. So the foreign exchange market is segmented. It has a number of consequences. There can be difference in quotes. Volume tick data is also representative of that market segment instead of the whole market.

Basic spot transaction is the tick. A tick has the time stamp, bid and ask price and most of the time some information about the origin of the tick like the bank code etc. Obtaining that tick data is easy now a days. If you are using MT4/MT5 for trading, you can download tick data after writing some MQL4/MQL5 code. Just keep in mind figures can vary a little from broker to broker due to market segmentation.

FX market is a 24/5 market meaning it is a round the clock market for the week days. It gets closed late Friday and opens early Monday. So there are no limitations on the business hours and any market maker can submit new bid/ask at any time. Major centers for these currency transactions are Tokyo, London and New York. In the electronic trading market the spread is narrower as compared to the spot over the counter market. Reputation considerations are a major incentive for the market markets in the over the counter market to stick to the quoted bid/ask. Major portion of the currency transactions volume is intraday.

Print this item

  Bayesian Financial Time Series Forecasting

Posted by: Hassam
04-19-2018, 03:47 PM
Forum: Bayesian Statistics
- Replies (1)

Predicting financial time series is a challenging task. If predicting financial time series was easy, it would have been very easy to make fortunes in the financial markets. Financial time series modeling is of the most actively researched areas now a days. Time series are important in many areas that we come across. Most important areas where time series matters is the health services sector, weather forecasting, earthquake forecasting, marketing, inventory and so many other areas. Price data is a perfect example of time series data. You can open your trading platform and find price being plotted at specific time periods also known as timeframes.

Autoregressive Financial Time Series Models

Most time series have some relation with the past values meaning present value is being influenced by past values to some degree. This is precisely what we believe when we do technical analysis. We look at the chart past price values and try to predict the future price. Now I have said this dependence is not 100%. Technicians also known as chartist believe that past price can be used to predict future price. So the first financial time series model that comes to mind is the Autoregressive Models. I have used autoregressive (AR) models in trading and find that AR models do have some predictive power.

Mean Reversion Trading Strategies

The problem with the standard AR models is that they assume the time series to be stationary. Stationary is a concept in statistics that means the expected mean of the time series is constant or level. So a stationary time series will most of the time fluctuate around that level. If it strays far from that level, it will try to return to the mean. This is also known as the Reversion to Mean. Mean Reversion Trading Strategies are popular in the quant world. On mean reversion trading strategy that is very popular is the pair trading. We find two instruments like two stocks in the same sector or two currency pairs that have some correlation. We test for mean reversion and if the test is statistically significant, we wait for the two instrument to diverge from the mean. When it happens, we short the stock what is above the mean and go long the stock that is below the mean.

Autoregressive Integrated Moving Average Models
But there are many times when the price time series is not stationary at all meaning its long term mean is changing. So we have the mean level moving steadily either up or down.As I said most of the AR time series models use stationary assumption in making the forecasting. Another model that is popular is the ARIMA (Autoregressive Integrated Moving Average Model). In an ARIMA model we make the time series stationary by differencing. Differencing we detrend the time series of its trend component. I have used ARIMA models and they produce very poor financial time series forecasts. What to do? I have started this thread in which I want to discuss how we can use Bayesian Models to improve the financial time series forecasting.

Print this item


  1. Million Dollar Challenge