How To Download Tick Data in MetaTrader 4 & 5

28 March 2018
The Market Bull

This post describes how to download and save tick data offline, from both MT4 and MT5.

Functional code samples (MetaTrader indicators) have been provided via GitHub, along with instructions on how to download and use them to begin extracting tick data from MetaTrader 4 or 5.

Each tick data point extracted using this post’s implementation contains:

  1. Timestamp (in seconds or milliseconds)
  2. Bid price
  3. Ask price
  4. Bid/Ask Spread

The remainder of this post is organized as follows:

  1. The case for collecting tick data.
  2. Anatomy of a simple Tick Data indicator.
  3. Differences between MT4 and MT5 versions.
  4. Indicator Installation Instructions.

The case for collecting tick data

Traders can benefit in a number of ways by collecting tick data directly from their broker.

This ensures exact consistency (barring any unrelated issues, technical or otherwise) between tick data stored offline and that available via the MetaTrader platform.

Possible use-cases include:

1) Backtesting trading strategies that require tick-level precision.

2) Sampling price data by synchronizing it with e.g. speed of tick arrival instead of block intervals (e.g. M1, M5, etc).

This practice addresses the negative impact of time based intervals effectively ignoring the frequency of price change at different times of day.


For those interested, Mandelbrot and Taylor [1967] were among the pioneers in establishing that sampling by transaction frequency improves statistical soundness:

“Price changes over a fixed number of transactions may have a Gaussian distribution. Price changes over a fixed time period may follow a stable Paretian distribution, whose variance is infinite. Since the number of transactions in any time period is random, the above statements are not necessarily in disagreement.”

Journal link:
https://www.jstor.org/stable/168611


3) Developing strategies, tools, indicators etc. outside the MetaTrader environment (e.g. in Python, R, Julia, Java, C/C++ etc.)

4) Monitoring the evolution of Bid/Ask spreads both inside and outside MetaTrader.

5) Drawing inferences from time-weighted average bid/ask spreads (TWAS) vs. your strategy’s evolution (both in real-time and otherwise).

6) Strategies targeting small gains can benefit from optimizing execution based on the evolution of underlying asset TWAS data.

7) Connecting a real-time FX tick data feed via ZeroMQ to non-MQL trading strategies or R&D software.

.. and the list goes on.


Anatomy of a simple Tick Data Indicator

The indicators implemented in this post have the following structure:

1) Initialization -> OnInit()

This function is called when the indicator is loaded on a chart in the MetaTrader terminal.

For our purposes, the only activity required here is:

  • Open a new CSV file for writing tick data to, e.g. EURUSD_TickData.csv
  • If the file already exists, append data to it instead of creating a separate file for the same asset.

 

Code listing:

int OnInit()
 {
//--- indicator buffers mapping
 
 // Create CSV file handle in WRITE mode.
 csv_io_hnd = FileOpen(Symbol() + "_TickData.csv", FILE_CSV|FILE_READ|FILE_WRITE|FILE_REWRITE, ',');
 // If creation successful, write CSV header, else throw error
 if(csv_io_hnd > 0)
 {
 if(FileSize(csv_io_hnd) <= 5)
 FileWrite(csv_io_hnd, "time_milliseconds", "bid", "ask", "spread");
 
 // Move to end of file (if it's being written to again)
 FileSeek(csv_io_hnd, 0, SEEK_END);
 }
 else
 Alert("ERROR: Opening/Creating CSV file!");
//---
 return(INIT_SUCCEEDED);
 }

2) Tick Processing -> OnCalculate()

This function is called each time a new tick is received.

Here we’ll need to:

  • Check if a file is open for writing the tick’s bid, ask and spread values to.
  • If yes, write tick to file.
  • If no, throw an error highlighting the failure.

 

Code listing:

int OnCalculate(const int rates_total,
 const int prev_calculated,
 const datetime &time[],
 const double &open[],
 const double &high[],
 const double &low[],
 const double &close[],
 const long &tick_volume[],
 const long &volume[],
 const int &spread[])
 {
//---
 
 // If CSV file handle is open, write time, bid, ask and spread to it.
 if(csv_io_hnd > 0)
 {
 if(SymbolInfoTick(Symbol(), tick_struct))
 {
 Comment("\n[Darwinex Labs] Tick Data | Bid: " + DoubleToString(tick_struct.bid, 5) 
 + " | Ask: " + DoubleToString(tick_struct.ask, 5) 
 + " | Spread: " + StringFormat("%.05f", NormalizeDouble(MathAbs(tick_struct.bid - tick_struct.ask), 5))
 + "\n\n* Writing tick data to \\MQL5\\Files\\" + Symbol() + "_TickData.csv ..."
 + "\n(please remove the indicator from this chart to access CSV under \\MQL4\\Files.)"
 );
 FileWrite(csv_io_hnd, tick_struct.time_msc, tick_struct.bid,
 tick_struct.ask, StringFormat("%.05f", NormalizeDouble(MathAbs(tick_struct.bid - tick_struct.ask), 5)));
 } 
 else
 Print("ERROR: SymbolInfoTick() failed to validate tick."); 
 }
 
//--- return value of prev_calculated for next call
 return(rates_total);
 }

3) Termination -> OnDeinit()

This function is called when the indicator is removed from the chart, the chart is closed with the indicator still deployed on it, or the terminal is shut down correctly for any reason.

All we need to do here is check if the file being written to is still open, and close it.

* Important Note: You must remove the indicator from the chart before attempting to open the CSV file being written to. MetaTrader will have exclusive read/write access to the file while the indicator is running.

 

Code listing:

void OnDeinit(const int reason)
{ 
 // Close CSV file handle if currently open, before exiting.
 if(csv_io_hnd > 0)
 FileClose(csv_io_hnd);
 
 // Clear chart comments
 Comment("");
}

Differences between MT4 and MT5 versions

This implementation uses MetaTrader’s MqlTick structure in both the MetaTrader 4 and 5 versions.

For your reference, the definition for MqlTick is:

struct MqlTick 
  { 
   datetime     time;          // Time of the last prices update 
   double       bid;           // Current Bid price 
   double       ask;           // Current Ask price 
   double       last;          // Price of the last deal (Last) 
   ulong        volume;        // Volume for the current Last price 
   long         time_msc;      // Time of a price last update in milliseconds 
   uint         flags          // Tick flags 
  };

There is one subtle but important difference between its use in MetaTrader 4 and 5:

MqlTick.time_msc will return a timestamp in seconds (in MetaTrader 4) and in milliseconds (in MetaTrader 5)

Therefore, for MetaTrader 4 users in particular, it makes sense to use MetaTrader 5 for collecting tick data as subsecond sampling isn’t available in MetaTrader 4.

Other minor differences include using the comma (‘,’) delimiter in FileOpen() operations in MetaTrader 4, as opposed to tab (‘\t’) in MetaTrader 5.


MetaTrader Indicator Installation Instructions

We’ve uploaded functional MetaTrader 4 and 5 versions of this Indicator to our GitHub page under tools -> MQL4 and MQL5 respectively.

First, simply Right-Click and Save-As on the following link to save the file to your computer, and follow the instructions below them:

For MetaTrader 4 users:
Click here to download DLabs_TickData_ToCSV_MT4.mql4

For MetaTrader 5 users:
Click here to download DLabs_TickData_ToCSV_MT5.mql5

Instructions:

  1. Launch MetaTrader, and open your data folder (File -> Open Data Folder)

    MetaTrader 4 Open Data Folder

    MetaTrader Open Data Folder

  2. Copy the “DLabs_TickData_ToCSV_MTx.mqx into your MQLx\Indicators directory, where “x” is either 4 or 5 depending on your platform (MT4 or MT5).
  3. Restart MetaTrader and open a new chart,
  4. Double-click on DLabs_TickData_ToCSV_MTx (as above) under Indicators to load the indicator on any chart.
  5. No input parameters need configuration, therefore click OK and continue.
  6. Open a chart, the symbol for which you’d like tick data saved to disk, e.g. EURUSD.
  7. Tick data including timestamp, bid, ask and spread should now begin saving to a CSV file in the background.

If you want to open the CSV file, you must first remove the indicator from the chart.

To do this, Right-Click anywhere on your chart and select Indicator List:

MetaTrader Indicator List

MetaTrader Indicator List

 

Then select the indicator from the displayed list, and hit Delete to remove it from your chart:

Delete Indicator from MetaTrader chart

Delete Indicator from MetaTrader chart

 

To access the CSV file, click File -> Open Data Folder, and once inside, head over to the MQLx\Files (x described as above) directory and open the CSV file that will have been saved here.

Saved MetaTrader Tick Data

Saved MetaTrader Tick Data

 

In a future post, we’ll discuss how to automate tick data collection using R, MetaTrader and a VPS instance.


As always, we hope you’ve enjoyed this tutorial, and look forward to any feedback you may have for us 🙂

Code samples have been kept as simple as possible to enable the programmer in you to thrive in extensibility, and exercise your creative freedom as best as you can!

Should you have ideas/feedback on how we can extend this implementation further, please do feel free to leave a comment below – we’ll try our best to either release an update, or resolve your query directly.

Also, kindly share this post using the buttons provided, with any colleagues and/or networks you feel would benefit from the content. Or just share it anyway to help us spread the word! 🙂


Do you have what it takes? – Join the Darwinex Trader Movement!

Darwinex - The Open Trader Exchange