Posts

LVQ and Machine Learning for Algorithmic Traders – Part 3

In the last two posts, LVQ and Machine Learning for Algorithmic Traders – Part 1, and LVQ and Machine Learning for Algorithmic Traders – Part 2, we demonstrated how to use:

  1. Linear Vector Quantization
  2. Correlation testing

..to determine the relevance/importance of and correlation between strategy parameters respectively.

Yet another technique we can use to estimate the best features to include in our trading strategies or models, is called Recursive Feature Elimination, an automatic feature selection approach.


What is Automatic Feature Selection?

It enables algorithmic traders to construct multiple quantitative models using different segments of a given dataset, allowing them to identify which combination of features or strategy parameters results in the most accurate model.

Recursive Feature Elimination

Recursive Feature Elimination

One such method of automatic feature selection is Recursive Feature Elimination (RFE).

To evaluate the best feature-space for an accurate model, the technique iteratively applies a Random Forest algorithm to all possible combinations of the input feature data (strategy parameters).

The end-outcome is a list of features that produce the most accurate model.

Using RFE, algorithmic traders can refine and speed up trading strategy optimization significantly (subject to this list being smaller than the total number of input parameters of course).


R (Statistical Computing Environment)

R (Statistical Computing)

We’ll make use of the caret (Classification and Regression Training) package in R once again.

It contains functions to perform RFE conveniently, allowing us to spend more time in analysis instead of writing the functionality ourselves.

Recursive Feature Elimination – Step by Step Process

  1. As before, run “raw” backtests without any optimization, employing all features (parameters), and save your results in a suitable data structure (e.g. CSV table) + load the caret and randomForest libraries.
  2. Specify the algorithm control using a Random Forest selection method.
  3. Execute the Recursive Feature Elimination algorithm.
  4. Output the algorithm’s chosen features (strategy parameters).

 

Step 1: Load the data + “randomForest” and “caret” machine learning libraries in R

> library(caret)
> library(randomForest)
> train.blogpost <- read.csv("data.csv", head=T, nrows=1000)
> train.blogpost <- train.blogpost[,grep("feature|target",names(train.blogpost))]

Step 2: Specify the control using Random Forest selection function

> rfe.control <- rfeControl(functions=rfFuncs, method="cv", number=10)

Step 3: Execute the Recursive Feature Elimination algorithm

rfe.output <- rfe(train.blogpost[,1:21], train.blogpost[,22], sizes=c(1:21), rfeControl = rfe.control)

Step 4: Output chosen features (strategy parameters)

> print(rfe.output)
> predictors(rfe.output)
> plot(rfe.output, type=c("o", "g"))

Recursive Feature Elimination - Output Predictors

Recursive Feature Elimination – Output Predictors

Recursive Feature Elimination - RMSE Plot

Recursive Feature Elimination – RMSE Plot


Conclusion

From these results, it is easily apparent that a model with:

  1. The first two parameters only, generates the most inaccurate model.
  2. The algorithm’s 5 selected parameters (out of a total of 21) produces the most accurate model.
  3. Any number of parameters greater than 5 produces lower but comparable accuracy, therefore choosing any greater a number of parameters would add zero value to the model.

Based on this, an algorithmic trader could significantly reduce his/her optimization overhead, by culling the number of strategy parameters employed in backtesting and optimization.


Additional Resource: Measuring Investments’ Risk: Value at Risk (VIDEO)
* please activate CC mode to view subtitles.

Do you have what it takes? – Join the Darwinex Trader Movement!

Darwinex - The Open Trader Exchange

Darwinex – The Open Trader Exchange

LVQ and Machine Learning for Algorithmic Traders – Part 2

 

Highly Correlated Parameter Removal

Highly Correlated Parameter Removal

 

In LVQ and Machine Learning for Algorithmic Traders – Part 1, we discussed and demonstrated a technique (Linear Vector Quantization) to decipher the relevance and relative importance of each feature variable in the dataset under study.

In doing so, algorithmic traders would be able to isolate which of a dataset’s features (read: strategy parameters) had a minor impact on the final target, thereby aiding faster strategy optimization.

Another technique we can use for the same objective (isolating features that have little to no impact on end outcomes), involves studying the correlation between the dataset’s feature variables.


Why Correlation?

 

When a trading strategy has highly correlated parameters, algorithmic traders not only run the risk of overfitting, but also that of introducing avoidable latency in execution.

While the latter may be more of a concern for short-term / intraday traders, over the longer term it may introduce considerably higher transactional costs even for swing traders -> intended vs. actual fills.

Algorithmic traders can therefore benefit from the removal of such highly correlated parameters, prior to any optimization.

R (Statistical Computing Environment)

R (Statistical Computing Environment)

The procedure to follow for removing such redundant features is quite simple (see below).

We will once again, make use of the caret (Classification and Regression Training) package in R, that contains a suite of convenient functions for this particular task.

N.B. It is just as simple to replicate this process in C++, Java, MQL or Python.


Step by Step Process

  1. Run “raw” backtests without any optimization, employing all features (parameters), and save your results in a suitable data structure (e.g. CSV table) for further analysis.
  2. Construct a correlation matrix of the data’s features (read: strategy’s parameters).
  3. Run the correlation matrix through caret’s findCorrelation() function to determine which features (parameters) are highly correlated, and can hence be removed.
  4. Set the correlation threshold at 60% (can be higher or lower if you prefer), to remove features.

Step 1: Load the “caret” machine learning library in R.

> library(caret)

Step 2: Load and process the backtest dataset.

For the purposes of this example, we will use the same “feature|target” backtested dataset of 1,000 records employed in LVQ and Machine Learning for Algorithmic Traders – Part 1.

> train.blogpost <- read.csv("data.csv", head=T, nrows=1000)

> train.blogpost <- train.blogpost[,grep("feature",names(train.blogpost))]

Step 3: Calculate and print the correlation matrix

> correlation.matrix <- cor(train.blogpost)

> print(correlation.matrix)

Correlation Matrix using cor() in R

Correlation Matrix using cor() in R

Step 4: Detect and print highly correlated features (threshold > 60%)

> high.corr <- findCorrelation(correlation.matrix, cutoff=0.6)

> print(high.corr)

The features (parameters) printed as a result of this process, have an absolute correlation of 60% or higher, and thus should be removed before any optimization is conducted.


Additional Resource: Measuring Investments’ Risk: Value at Risk (VIDEO)
* please activate CC mode to view subtitles.

Do you have what it takes? – Join the Darwinex Trader Movement!

Darwinex - The Open Trader Exchange

Darwinex – The Open Trader Exchange

LVQ and Machine Learning for Algorithmic Traders – Part 1

Algorithmic traders across all spectra of asset classes, often face a rather daunting challenge.

What are the best inputs for an algorithmic trading strategy’s parameter space?

Different algorithmic trading strategies (whether manual or automated) will each have their own unique set of parameters that govern their behaviour.

Granted.. Genetic and Walk-Forward Optimization will help algorithmic traders establish what input values (or ranges thereof) in chosen parameter spaces, yield favourable results historically.

They will also help traders identify optimal time periods over which to re-optimize “the currently optimized parameter space”…. yes, that could indeed, get pretty messy.

While this approach may or may not yield robust parameter inputs, several questions still remain in algorithmic traders’ minds:

1) Should absolutely all parameters be optimized, or just some? If so, which ones?

2) What is the relevance and unique importance of each parameter in the trading strategy?trading strategy optimization questions

Why is this important for Algorithmic Traders?

Selecting the right parameters in your trading algorithm can be the difference between:

  • Average performance with a large number of parameters -> painfully long optimization times,
    or,
  • Fantastic performance with a smaller number of parameters -> much shorter optimization times.

What is the solution?

Selecting the most appropriate parameters is a practice known as Feature Selection in the Machine Learning world, a vast and complex area of research and development.

Needless to say it cannot be encapsulated in one single blog post, which therefore implies that there will be more blog posts on this subject in the very near future 🙂

R (Statistical Computing Environment)

R (Statistical Computing Environment)

For now, we will focus on estimating “the most important” parameters in a trading strategy, using a bit of machine learning in R.

Specifically, we will make use of the caret (short for Classification and Regression Training) package in R, as it contains excellent modeling functions to assist us with this Feature Selection problem.

Lastly, we will use a small constructed sample of 1,000 id|feature|target records as the dataset, to demonstrate Linear Vector Quantization (the solution).

 

Step 1 – Load the “caret” machine learning library in R

> library(caret)

Step 2 – Prepare the data

Construct a dataset containing 1,000 training data points in CSV form.

Making sure you’re in the directory where the training data resides, type the following commands in your R console:

> train.blogpost <- read.csv("data.csv", head=T, nrows=1000)

We need only the “feature” and “target” column values in the dataset. Type the following command in your R console to achieve this:

train.blogpost <- train.blogpost[,grep("feature|target",names(train.blogpost))]

Step 3 – Construct an LVQ Model on the data.

> model.control <- trainControl(method="repeatedcv", number=10, repeats=3)> model <- train(as.factor(target)~., data=train.blogpost, method="lvq", preProcess="scale", trControl=model.control)

Step 4 – Retrieve the “importance” of each “feature” from the computed model.

> importance <- varImp(model, scale=FALSE)> print(importance)
loess r-squared variable importance
only 20 most important variables shown (out of 21)Overall
feature2  0.011949
feature18 0.010770
feature7  0.010556
feature16 0.010522
feature5  0.010400
feature11 0.009825
feature1  0.009673
feature14 0.009672
feature3  0.009663
feature13 0.008916
feature21 0.008846
feature15 0.008737
feature10 0.008616
feature17 0.008180
feature19 0.007864
feature12 0.005575
feature9  0.005268
feature8  0.005124
feature20 0.005089
feature4  0.005052
>

Step 5 – Visualize the importance of each feature.

plot(importance)

LVQ Importance Visualization - Machine Learning in R

LVQ Importance Visualization – Machine Learning in RThe plot of “feature importance” above clearly shows that features 12, 9, 8, 20, 4 and 6 have little impact on the outcome (the “target”), compared to the rest of the features.

To put it into context – in a trading strategy, these features may well have been parameters called:

Stop Loss 1, Stop Loss 2, Take Profit 1, Take Profit 2, RSI Top, RSI Bottom.. and so on.

Conclusion

By conducting LVQ analysis on optimization results, algorithmic traders can save themselves not only time, but lost accuracy.

Machine learning techniques of this nature, greatly reduce the time a trader needs to spend on any optimization problem.

By ascertaining the relevant importance of parameters in this manner, traders can not only simplify their algorithms, but also make them more robust than previously possible with a larger number of parameters.

Additional Resource: Learn more about DARWIN Portfolio Risk (VIDEO)
* please activate CC mode to view subtitles.