Posts

capacity manager optimize investor capital

Capacity Manager: Optimize & Manage Investor Capital

This post sheds light on the new Capacity Management tool, designed from the ground up to give traders more control over the capital invested in their DARWINs, while offering total transparency to investors.


What does a Trader need to make a living off his passion?

There is no easily applicable magic formula to achieve this, since otherwise, everyone would become a trader and the indispensable zero-sum game needed to speculate in the markets would vanish into thin air.

However, there are a number of sine qua non conditions to become a professional trader such as:

—> Talent: it seems obvious that to live off your trading, you need to be systematically better than your counterparts, building a proven track record that can attract …

—> Investor Capital: enabling you to obtain a 20% performance fee on investor profits as observed in the Hall of Fame. However, if you take a look at the first DARWIN on this list – THA – you will see that it currently remains closed to new investors due to its …

—> Capacity: which measures if a DARWIN’s percentage performance falls as the number of investors grows. The better the Capacity (Cp) grade, the more likely it is that a DARWIN will be able to manage a larger volume of investment capital, without its divergence affecting performance.


In other words, once talent has been certified by our algorithms and investors begin to queue up to invest their hard earned capital in a DARWIN, traders face a new challenge:

..addressing the limitations imposed by their DARWINs’ Capacity.

Two examples of such DARWINs are THA and VTJ, assets that we had to close to new investors at some point since the capital invested in them began to cause systematically negative divergences.

This outcome broadened the difference between the DARWIN’s return and that yielded to its investors, to the detriment of both parties.

Remember, a DARWIN provider only gets paid if investors generate profits.

Therefore, it is entirely possible that certain DARWINs with positive return curves, yield returns to neither their providers nor their investors as a result of systematically negative divergence.


Tough Challenges, Innovative Solutions

After the closure of new investments to DARWINs THA and VTJ, it became apparent that:

  1. We need to offer our DARWIN providers a tool to manage the Capacity of their DARWINs autonomously,
  2. The manual process used so far was completely discretionary, which erected a barrier of opaque glass between the trader and its investors,
  3. ..we needed to break this barrier down.

Capacity Manager for DARWIN Providers

The Capacity Manager incorporates various configuration options, enabling DARWIN providers to specify the conditions under which the closing / reopening of investments in their DARWINs can take place.

These options include:

1) Maximum Divergence Accepted

Closure:

If the DARWIN exceeds the divergence threshold set by the trader, it will close to new investor capital.

However, regardless of the divergence selected by the DARWIN provider, Darwinex will still cap the maximum divergence permitted, at -1.5%.

Therefore, the trader has the option to select a maximum divergence ranging between -1.5% and -0.5% with a 1 decimal granularity.

The default value will be -1.5%.

Opening:

The DARWIN will reopen to new investments if the divergence decreases by -0.1% with respect to the threshold established by the DARWIN provider.

 

2) Maximum Investor Capital (optional)

Closure:

If the DARWIN exceeds the capital under management threshold set by its provider, it will close to new investor capital.

Traders can set the maximum amount that they want to manage with the DARWIN, in USD.

However, the value must be greater than $300,000 and in multiples of 100,000.

This condition will not be taken into account if the field remains empty, which is the default option.

Opening:

The DARWIN will reopen to new investments if the maximum capital selected by the trader decreases to 98%.

In order to manage a DARWIN’s Capacity, traders will have to go to its main profile and click on the cog wheel icon located next to the “TRADE” button, as circled in yellow in the image below:

DARWIN Configuration Settings

DARWIN Configuration Settings

Once there, they will need to click on the “divergence limit” section where the two options explained above will be displayed.

In summary,

DARWINs will close to new investor capital when at least one of the two requirements are not met, and reopen when all the conditions set out above have been fulfilled.

With the launch of the Capacity Manager, the following DARWINs have been closed to new investors.

  • PAX
  • THA (had been already closed before)
  • APZ
  • ZZQ
  • FFA
  • JJY
  • DZQ
  • JHJ
  • JFI
  • CHS

What happens when a DARWIN is closed to new investors?

Once the DARWIN is automatically closed to new investor capital, both investors – including those with a buy limit order – and DARWIN providers, will receive an email notification.

In addition, the “TRADE” button will be replaced by “SELL” and conditional orders will not be allowed.

Should a buy limit order previously placed get triggered, it will be discarded and a notification sent to the investor concerned.


Capacity Traffic Lights

The new Capacity Manager comes together with a traffic light equipped with three horizontal lines to enhance the understanding of the current state of the DARWIN’s Capacity.

Capacity Management - Traffic Lights Icon

Capacity Management – Traffic Lights Icon

Like a conventional traffic light, it will come with the following range of colours:

  • 1 green line: the DARWIN is still far from reaching the Capacity limit set by its provider.
  • 2 yellow lines: the DARWIN is close to reaching the Capacity cap set by the trader.
  • 3 red lines: the DARWIN has already reached its Capacity limit, and is closed to new investments.

You can hover the mouse over the traffic lights to learn more.


Capacity Manager: Next Steps

In addition to this release, we’re already hard at work engineering several improvements that we hope to announce as soon as possible.

What we can tell you is that the next version of the Capacity Manager will also include a tool that will show DARWIN providers the divergence per order.

This will enable them to optimize the management of their trades (fraction them into smaller trades for example), since we believe that most strategies can significantly increase their Capacity with this information.


Conclusion:

We hope that the Capacity Management tool will mark a turning point in your career as a trader and, together with your remarkable talent and discipline, help you become one of the best traders in the world.

We hope to see you soon at the very top of our Hall of Fame!


You may also wish to read:

Buying and selling DARWINs

Conditional orders

Divergence

Performance fees

WebinarManaging Capacity


The next webinar at Darwinex on May 24th, 6PM London Time will be dedicated to trading single stock CFDs – it’d be great if you attend!

Register For The Webinar

3 main advantages of becoming a DARWIN provider

3 Reasons Why You Should Have A DARWIN

In this post, we are going to explain the 3 main advantages of becoming a DARWIN provider.

But before you start reading it, let me ask you 3 simple questions:

  • Are you a trader?
  • Are you even a talented trader?
  • Do you dream of living off your trading talent but have no assets under management – AuM-?

If you’ve answered YES to all of them, you’re definitely in the right place at the right time. Keep reading and I promise that, by the end of this post, you’ll be both ecstatic and also looking for the nearest wall to hit your head pretty damn hard by the simple thought of having missing out on this golden opportunity.

Let’s get started!

In order to fully understand the main advantages of having a DARWIN, you need to know what a DARWIN is first:

A DARWIN is a financial asset listed on the DARWIN Exchange for the benefit of independent traders and investors. Investors may buy and sell DARWINs just like they can buy and sell stocks, ETFs, etc.

The DARWIN’s value is tied to an underlying trading strategy. When the trading strategy goes, for example, long EURUSD, investors who have bought that DARWIN will replicate this and will also go long EURUSD.

However, they will never know the details of this underlying trade in real time. Instead, they will track the DARWINs listed price watching their P&L in real time. As an example, if someone had bought a DARWIN at 117 and the underlying strategy had made a winning trade, the value of the DARWIN would have increased above 117.

And now, without further ado, let’s take a look at the 3 great advantages of becoming a DARWIN provider:

  • MANAGE AUM LEGALLY => Are you using a PAMM or MAM account to manage AuM? Have you thought about the legal implications to do so without being regulated?

Investments in DARWINs are managed by Darwinex, a broker and asset manager regulated by the FCA and not by the traders themselves. This way, they are free of the bureaucratic burden of becoming regulated asset managers and can focus on what they do best: trading!

Additionally, listing a DARWIN will make you automatically eligible for investors’ capital –  $47.000.000 and rising at the speed of light- benefiting from a conflict-free 20% performance fee.

FYI, we have already paid over $2 MILLION to our best DARWIN providers.

  • DARWINIA => The moment you quote a DARWIN, you’ll be automatically participating in DarwinIA where we invest 4MM notional euros every single month in the best 48 DARWINs of the month according to our rating algo. Fuerthermore, you can also earn some of our darwinian dividends.
  • INTELLECTUAL PROPERTY PROTECTION => please, read everything there’s to know about your IP being fully protected by Darwinex here.

If you are now determined to become a real pro trader, willing to capitalize on your trading talent without compromising your intellectual property, and receiving a handsome salary while you’re at it, we encourage you to sign up, open an MT4 real account or link an existing one, and list your first DARWIN. More than $50.000.000 of AuM are waiting for you!

We’re looking forward to seeing you around.

Trade safe!

P.S: You may post your comments down below or use the blue icon located at the bottom-right of this post.

If you have found this article valuable, and want to share it with your fellow traders, don’t be shy and use the icons you may find on the right hand side of this post.

 

Setting up a DARWIN Data Science Environment

How To Identify Overfit Trading Strategies

In this post, we describe the main features and behaviours of overfit trading strategies, and the risks they pose to both traders and DARWIN investors.

Overfit trading strategies typically perform well in backtesting environments, creating the illusion that they exploit the market inefficiency being targeted, really well.

However, when deployed in a live trading environment, their performance is disproportionately different to what was observed in backtesting, due primarily to their modeling historical data too closely.

This prevents such strategies from generalizing well to unseen data in future, to the detriment of both the traders (launching them live with capital) and DARWIN investors (backing them with capital).

For your convenience, this post is organized as follows:

  1. Types of Overfit Trading Strategies
  2. Typical Features & Behaviours
  3. How Traders Can Address Overfitting
  4. How Investors Can Avoid Overfit DARWINs.

Types of Overfit Trading Strategies

We can generalize them into two main categories:

  1. Model-focused – where the strategy fits historical data too closely, and exhibits high variance when tested on unseen data.
  2. Risk-focused – where the performance of a weak model is compensated for with an unrealistic, loss averse risk management rationale.

In the former (model-focused), a trading strategy will typically perform very well in backtesting, but either stagnate for lengthy periods of time (or fail entirely) in live testing.

Overfit Trading Strategies (model-focused)

Overfit Trading Strategies (model-focused)

From both a trader and investor’s perspective, such strategies are easy to identify visually; returns will at some point reach a point of inflexion where they no longer appear similar to historical performance.


In the latter (risk-focused), a trading strategy will demonstrate smooth, very consistent returns in backtesting as well as in live testing for a period of time, making such strategies more dangerous than the former as they are difficult to identify from looking at just returns charts.

Overfit Trading Strategies (risk-focused)

Overfit Trading Strategies (risk-focused)

Before & After (Risk-Focused)

Before & After (Risk-Focused)

Later in the post, we’ll discuss certain DARWIN investment attributes that assist both traders and investors in isolating such strategies.


Typical Features & Behaviours

Model-focused

It is usually quite straight-forward to identify trading strategies that overfit to historical data.

Compared to training phases in backtests, their test phases and live performance may demonstrate:

  1. Excess stagnation,
  2. Larger drawdowns, and/or
  3. An overall reversal in forecast returns.

Of the 12 DARWIN investment attributes available, this behaviour is captured best by the evolution of the following:

  1. Experience (Ex)
  2. Performance (Pf),
  3. Risk Stability (Rs),
  4. Positive/Negative Returns Consistency (R+/R-),
  5. Duration Consistency (Dc),
  6. Capacity (Cp),
  7. Loss Aversion (La),
  8. Open/Close Strategy (Os/Cs).

Low scores or unstable evolution of scores in these attributes (especially in live trading) can serve as a useful indicator in identifying a strategy as being overfit in backtests or otherwise.

When a strategy is overfit to training data, the evolution of its scores for the above attributes is likely to demonstrate high variance when subjected to test data and/or in live trading.


Typically, three combinations of scores for the above attributes demonstrate consistent performance between backtests and live trading (when accompanied by High Ex, High Mc, High Rs scores):

  1. (Low Cp | High Os/Cs | High Pf | High R-) – Once a strategy with good scores for these in the backtest is launched live, if scores for Os/Cs and R- progressively decline over time, the likelihood of the strategy being overfit increases.
  2. (Moderate Cp | High La | High Pf) – If high scores for La and Rs progressively decline upon launching live, the likelihood of the strategy being overfit increases.
  3. (High Pf | Very High R+/R- or Dc | Moderate La) – A strategy with this combination of scores in the backtest is the least likely to be overfit, and quite difficult to find. However, the same rules for monitoring declines in scores applies here too, when launched live.

Traders can therefore benefit from uploading trading strategy backtests to the Darwinex platform for analysis.

Examining the evolution of scores received provides a valuable layer of insight into how symmetric performance is likely to be in backtesting vs live trading.

For example, if steady evolution is observed over training data, but high variance in test data, the likelihood of the strategy generalizing well to unseen data in live trading is low.

Stable Evolution of Risk & Duration Consistency

Stable Evolution of Risk & Duration Consistency


Risk-focused

In trading strategies where loss averse risk management compensates for poor timing (and generates unrealistic returns in backtesting), one or more of the following behaviours can be observed:

  1. Poorly timed trades are not closed for lengthy periods of time,
  2. Additional orders are opened in the same direction as poorly timed ones in an attempt to recover the position at incrementally better prices,
  3. Excess leverage is employed per trade in an attempt to recover losing positions at incrementally better prices.

Of the 12 DARWIN investment attributes available, this behaviour is captured best by the following:

  1. Loss Aversion (La)
  2. Combination of low La and high Capacity (Cp)
  3. Risk Stability (Rs)
  4. Market Correlation (Mc)

The evolution of scores received for these investment attributes, provides valuable insight into whether a strategy will compensate for inferior timing by employing loss averse risk management practices.

Additionally, poor scores for Rs and Mc in particular add strong confirmation that risk-focused overfitting is likely the case with a strategy.

La vs Cp vs Rs (Evolution of DARWIN Investment Attributes)

La vs Cp vs Rs (Evolution of DARWIN Investment Attributes)

Additionally, strongly negative correlation with DARWIN $DWC, adds confirmation to this risk.


 

How Traders Can Address Overfitting

A recent blog post – DO’s and DONT’s of MetaTrader 4 Backtesting – details several steps traders can take to both address and eliminate overfitting from their trading strategies.


How Investors Can Avoid Overfit or Risky DARWINs

As described under “Typical Features & Behaviours” above, monitoring the evolution of a strategy’s scores for:

  1. Experience (Ex),
  2. Risk Stability (Rs),
  3. Market Correlation (Mc),
  4. Performance (Pf),
  5. Loss Aversion (La),
  6. Capacity (Cp),
  7. Open/Close Strategy (Os/Cs),
  8. Positive/Negative Returns Consistency (R+/R-), and
  9. Duration Consistency (Dc)

..can help DARWIN Investors exercise caution with (or avoid entirely), both types of overfit trading strategies discussed in this post.

More detailed information on each investment attribute is available via the Education Section.


[Additional Resources] (Video) How To Identify Overfit Trading Strategies

Do you have what it takes? – Join the Darwinex Trader Movement!

Darwinex - The Open Trader Exchange

Increasing investment capacity through leverage on the Darwinex platform

If as a DARWIN investor you have the leverage option enabled, there is a way to increase your investment amount in the event that you have a profitable open position in a given DARWIN.

 

By way of example, let’s assume that:

  1. We have €5,000 invested in a DARWIN,
  2. The open P/L is €1,000.
  3. The leverage employed is 2x.

If at this point we were to close our position, the platform would automatically retain €200 by way of performance fees.

Our unlevered investment capacity would then be equal to:

Our Initial Investment + Net P/L

i.e. €2,500 + €800 = €3,300

Using 2x leverage as previously, our available investment capacity is now increased to €6,600.

 

As such, we could reopen the same position:

i.e. €5,000 + €1,000 = €6,000

.. with €3,000, and still have another €600 left over to invest.

 

It is advisable to do this type of adjustment periodically in order to increase your investment capacity.

Remember however, that it is best to do this when the DARWIN does not have an open position in order to avoid generating unnecessary commissions.


Additional Resource: Investor Leverage
* please activate CC mode to view subtitles.

https://www.youtube.com/watch?v=xxJ4g2laPFA

Do you have what it takes? – Join the Darwinex Trader Movement!

Darwinex - The Open Trader Exchange

Darwinex – The Open Trader Exchange

$DWC 1-Minute Differenced Series

Mean Reversion Tests on DARWIN $DWC

In a previous post – Quantitative Modeling for Algorithmic Traders – we discussed the importance of Expectation, Variance, Standard Deviation, Covariance and Correlation.

In this post we’ll discuss how those concepts can be applied to DARWIN assets.

As a practical example, we will employ a series of statistical tests to assess if DARWIN $DWC is a Mean Reverting time series or otherwise.

 

These will include:

1) Hurst Exponent
2) Augmented Dickey-Fuller Test (ADF)
3) Half-life of Mean Reversion

 

In case you missed it, the mean reverting nature of DARWIN $DWC was discussed in our most recent post here.

Tests will be conducted on 1-Minute returns from $DWC, results and interpretation being published along the way. As always, please share your comments, feedback and suggestions in the comments at the end.

Note: Different statistical tests don’t always lead to similar outcomes, therefore it’s considered good practice to use at least two when evaluating mean reversion or any other statistical properties.

Before proceeding further, it’s important that we understand what Autocorrelation and Stationarity are.


Autocorrelation (Serial Correlation)

Autocorrelation (Serial Correlation)

Autocorrelation:

Also referred to as Serial Correlation.

It is a measure of the similarity or relationship between a time series and a delayed or “lagged” version of the same time series, over successive periods in time.

 

 

 

Stationary Time Series

Stationary Time Series

Stationarity:

A time series is considered stationary if its core statistical attributes remain constant over time.

These include mean, variance, standard deviation, autocorrelation, etc.

Stationary series demonstrate high predictability.

 

 

If a time series (e.g. DARWIN) can be mathematically transformed to approximately stationary, future Quotes of the time series (or trade entry direction / entries) can be reverse engineered from future points in its forecasted stationary series.

More on this in future blog posts.

Prior Assumptions:

Prior to conducting these tests on $DWC data, we are expecting to see a reasonable degree of mean reversion for the following reasons:

  1. There is visual confirmation (see below) that mean reverting tendency may exist.
  2. As $DWC behaves in relation to real time trader sentiment, it is reasonable to assume that it could exhibit cyclical behaviour.
$DWC 1-Minute Data Plot

$DWC 1-Minute Data Plot

$DWC 1-Minute Differenced Series

$DWC 1-Minute Differenced Series


Mean Reversion Test #1: Hurst Exponent

Mean Reversion in a time series can be assessed in terms of its rate of diffusion from inception.

 

For a time series X to be considered mean reverting:

Rate of Diffusion (X) < Rate of Diffusion of a Geometric Random Walk (GBM)

 

This rate of diffusion can be measured as the variance of the logarithm of the time series, at a random time interval T:

\(Var(T) = \left \langle \left | log(t + T) – log(t) \right |^{2} \right \rangle\)

 

If a time series is a GBM, then Var(T) ~ T, as T gets larger:

\(\left \langle \left | log(t + T) – log(t) \right |^{2} \right \rangle\) ~ T

 

If a time series is either trending or mean reverting, then:

\(\left \langle \left | log(t + T) – log(t) \right |^{2} \right \rangle\) ~ \(T^{2H}\)

.. where H is the Hurst Exponent, a measure of the extent to which the time series trends or mean reverts.

 

Hurst Exponent Interpretation:

If H > 0.5, the time series is TRENDING
If H < 0.5, the time series is Mean Reverting
If H = 0.5, the time series is a Geometric Random Walk

 

The DWC’s Hurst Exponent can be easily calculated in R, using the “pracma” library.

Note: For all code examples in this blog post, we have pre-loaded M1 data as “DWC.M1” to save time.

library(pracma)# Print M1 data Hurst Exponent
> hurstexp(log(DWC.M1$quote))
Simple R/S Hurst estimation: 0.8962816
Corrected R over S Hurst exponent: 0.9945418
Empirical Hurst exponent: 1.001317
Corrected empirical Hurst exponent: 0.9938308
Theoretical Hurst exponent: 0.520278

This first test shows that though this sample of DWC data is not demonstrating mean reverting behaviour (Theoretical Hurst Exponent > 0.5), it is not trending significantly either -> i.e. it is almost behaving like a GBM as per this test’s results (H = 0.520278), reducing the probability of DWC being a non-stationary random walk process.


Mean Reversion Test #2: Augmented Dickey-Fuller Test

If the $DWC time series is not a random walk (non-stationary series), then any Quote in the series will have a proportional relationship with the Quote immediately before it.

If $DWC is mean reverting, then any move higher above its mean would likely be followed by a move lower and vice versa.

The ADF Test checks for the presence of unit roots in a time series that’s autoregressive in nature, and for the tendency of a time series to mean revert.

Consider the following autoregressive model of order p:

\(\Delta x_{t} = \alpha + \beta t + \gamma x_{t-1} + \delta _{1}\Delta x_{t-1} + … + \delta _{p-1}\Delta x_{t-p+1} + \epsilon _{t}\)

The ADF test will statistically evaluate if γ = 0 (the null hypothesis) can be rejected at a given confidence interval.

If the null hypothesis can be rejected, it implies that the time series is not a random walk (non-stationary / no linear relationship between data points), and that there is a linear relationship between the current DWC Quote and the one immediately before it (stationary).

 

The ADF Test can be carried out in R quite easily, using the “urca” library.

ADF Test (1-minute DWC data):

> library(urca)
> summary(ur.df(DWC.M1$quote, type="drift", lags=1))
###############################################
# Augmented Dickey-Fuller Test Unit Root Test #
###############################################
Test regression driftCall:
lm(formula = z.diff ~ z.lag.1 + 1 + z.diff.lag)
Residuals:
Min 1Q Median 3Q Max
-1.66860 -0.01990 0.00008 0.02011 1.16945
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.0347880 0.0114228 3.045 0.00232 **
z.lag.1 -0.0003287 0.0001075 -3.057 0.00224 **
z.diff.lag -0.0365180 0.0045255 -8.069 7.22e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.04493 on 48479 degrees of freedom
Multiple R-squared: 0.001542, Adjusted R-squared: 0.001501
F-statistic: 37.45 on 2 and 48479 DF, p-value: < 2.2e-16
Value of test-statistic is: -3.0566 4.849Critical values for test statistics:
1pct 5pct 10pct
tau2 -3.43 -2.86 -2.57
phi1 6.43 4.59 3.78

 

Interpretation of ADF Test Results

Referring back to the autoregressive model earlier:

\(\Delta x_{t} = \alpha + \beta t + \gamma x_{t-1} + \delta _{1}\Delta x_{t-1} + … + \delta _{p-1}\Delta x_{t-p+1} + \epsilon _{t}\)

z.lag.1 = The value of the test-statistic γ (gamma) in the above equation.

tau2 = Critical values corresponding to the null hypothesis (γ = 0)

In order to reject the null hypothesis (γ = 0 – i.e. to reject that DWC is a non-stationary random walk), the value of the test statistic must be smaller than the critical values in tau2 (1%, 5% and 10% confidence intervals).

As z.lag.1 is -3.0566 (smaller than the critical values for the 5% and 10% confidence intervals), the null hypothesis can be rejected at the 90% and 95% confidence intervals, i.e. the probability of DWC being stationary (or not a random walk) is very high.

The tests above were also conducted on 30-minute, 1-hour, 2-hour, 4-hour and Daily precision $DWC data.

  1. Daily precision lead to the null hypothesis for the presence of a unit root being rejected at the 90% confidence interval. This test will be repeated periodically as more data is accrued over time.
  2. 30-minute, 1-hour, 2-hour and 4-hour tests all lead to the null hypothesis for the presence of a unit root being rejected at the 95% confidence interval.

Mean Reversion Test #3: Half-life of Mean Reversion

An alternative to the autoregressive linear model described above, is to consider how long any particular time series takes “to mean revert”.

By definition, a change in the next periodic value of a mean-reverting time series is proportional to the difference between the historical mean of the series and the current value.

Such time series are referred to as Ornstein-Uhlenbeck processes.

The differential of the earlier model leads us to the expected value of x(t):

\(E(x_{t}) = x_{0}e^{\gamma t} – \frac{\mu }{1 – e^{\gamma t}}\)

If DWC is a mean reverting series, and has a negative \(\gamma\), then the equation above tells us that DWC prices decay exponentially, with a half-life of \(\frac {-log(2)}{\gamma}\).

This means we now have two tasks ahead of us:

  1. Find \(\gamma\) and check if it is negative.
  2. Calculate the half-life and assess whether it is a practical length of time for traders to consider a mean reverting strategy on DWC.

Once again, we can easily conduct both steps in R.

Step 1: Calculate \(\gamma\) and check sign.

> M1.data <- as.ts(DWC.M1$quote)
> M1.data.lag <- lag(M1.data, -1)
> M1.data.delta <- diff(M1.data)
> M1.data.frame <- cbind(M1.data, M1.data.lag, M1.data.delta)
> M1.data.frame <- M1.data.frame[-1,]

> M1.regression <- lm(M1.data.delta ~ M1.data.lag, data=as.data.frame(M1.data.frame))

> gamma <- summary(M1.regression)$coefficients[2]
> print(gamma)
[1] -0.0003588994

\(\gamma\) is negative (-0.0003588994), so this $DWC 1-minute data sample can be considered mean reverting.

 

Step 2: Calculate half-life and assess practicality of mean reversion strategy.

> M1.data.half.life <- -log(2) / gamma> print(paste("Half-life: ", M1.data.half.life, " minutes, or ", M1.data.half.life/60, " Hours", sep=""))
[1] "Half-life: 1931.31306610404 minutes, or 32.1885511017341 Hours"

The half-life calculated for this $DWC 1-minute data sample is 32 hours.

 

Another important feature of the calculated half-life, is that it can be used as the period of a moving average employed in a mean reverting trading strategy[1].

If we plot a Simple Moving Average of period 1931 (in minutes, not hours), we get:

 

$DWC 1-Minute Data with SMA(1931)

$DWC 1-Minute Data with SMA(1931)


Summary:

  1. We conducted three statistical tests to ascertain the degree of mean reversion in $DWC 1-minute data, namely Hurst Exponent, Augmented Dickey-Fuller (ADF) and Half-Life of Mean Reversion.
  2. Hurst Exponent did not indicate mean reverting behaviour in the $DWC, but a rather close estimate for possible GBM behaviour.
  3. The Augmented Dickey-Fuller test results indicated stationary behaviour at the 95% confidence interval.
  4. The Half-life of Mean Reversion test indicated $DWC possesses mean reverting properties.
  5. We used the half-life calculated above as the period for a moving average, which when plotted on the chart revealed mean reverting Quote behaviour.

What are your thoughts after reading this research? ..please share in the comments section below!

 

References:

[1] Chan, Ernest, 2013. Algorithmic Trading: Winning Strategies and Their Rationale, John Wiley and Sons.


Additional Resource: Measuring Investments’ Risk: Value at Risk (VIDEO)
* please activate CC mode to view subtitles.

Do you have what it takes? – Join the Darwinex Trader Movement!

Darwinex - The Open Trader Exchange

Hidden Markov Models & Regime Change: S&P500

In this post, we will employ a statistical time series approach using Hidden Markov Models (HMM), to firstly obtain visual evidence of regime change in the S&P500.

Detecting significant, unforeseen changes in underlying market conditions (termed “market regimes“) is one of the greatest challenges faced by algorithmic traders today. It is therefore critical that traders account for shifts in these market regimes during trading strategy development.

Why use Hidden Markov Models?

Hidden Markov Models for Detecting Market Regime Change (Source: Wikipedia)

Hidden Markov Models for Detecting Market Regime Change (Source: Wikipedia)

 

Hidden Markov Models infer “hidden states” in data by using observations (in our case, returns) correlated to these states (in our case, bullish, bearish, or unknown).

They are hence a suitable technique for detecting regime change, enabling algorithmic traders to optimize entries/exits and risk management accordingly.

We will make use of the depmixS4 package in R to analyse regime change in the S&P500 Index.

 

Hidden Markov Model - State Space Model (Source: StackExchange)

Hidden Markov Model – State Space Model (Source: StackExchange)

With any state-space modelling effort in quantitative finance, there are usually three main types of problems to address:

  1. Prediction – forecasting future states of the market
  2. Filtering – estimating the present state of the market
  3. Smoothing – estimating the past states of the market

We will be using the Filtering approach.

Additionally, we will assume that since S&P500 returns are continuous, the probability of seeing a particular return R in time t, with market regime M being in state m, where the model used has parameter-set P, is described by a multivariate normal distribution with mean μ and standard deviation σ [1].

Mathematically, this can be expressed as:

\(p(R_t | M_t = m, P) = N(R_t | μ_m, σ_m)\)

As noted earlier, we will utilize the Dependent Mixture Models package in R (depmixS4) for the purposes of:

  1. Fitting a Hidden Markov Model to S&P500 returns data.
  2. Determining posterior probabilities of being in one of three market states (bullish, bearish or unknown), at any given time.

We will then use the plotly R graphing library to plot both the S&P500 returns, and the market states the index was estimated to have been in over time.

You may replicate the following R source code to conduct this analysis on the S&P500.

Step 1: Load required R libraries

library(quantmod)
library(plotly)
library(depmixS4)

Step 2: Get S&P500 data from June 2014 to March 2017

getSymbols("^GSPC", from="2014-06-01", to="2017-03-31")

Step 3: Calculate differenced logarithmic returns using S&P500 EOD Close prices.

sp500_temp = diff(log(Cl(GSPC)))
sp500_returns = as.numeric(sp500_temp)

Step 4: Plot returns from (3) above on plot_ly scatter plot.

plot_ly(x = index(GSPC), y = sp500_returns, type="scatter", mode="lines") %>%

layout(xaxis = list(title="Date/Time (June 2014 to March 2017)"), yaxis = list(title="S&P500 Differenced Logarithmic Returns"))

S&P500 Differenced Logarithmic Returns
(June 2014 to March 2017)

S&P500 Differenced Logarithmic Returns (June 2014 to March 2017)

S&P500 Differenced Logarithmic Returns (June 2014 to March 2017)

Step 5: Fit Hidden Markov Model to S&P500 returns, with three “states”

hidden_markov_model <- depmix(sp500_returns ~ 1, family = gaussian(), nstates = 3, data = data.frame(sp500_returns=sp500_returns))

model_fit <- fit(hidden_markov_model)

Step 6: Calculate posterior probabilities for each of the market states

posterior_probabilities <- posterior(model_fit)

Step 7: Overlay calculated probabilities on S&P500 cumulative returns

sp500_gret = 1 + sp500_returns
sp500_gret <- sp500_gret[-1]
sp500_cret = cumprod(sp500_gret)

plot_ly(name="Unknown", x = index(GSPC), y = posterior_probabilities$S1, type="scatter", mode="lines", line=list(color="grey")) %>%

add_trace(name="Bullish", y = posterior_probabilities$S2, line=list(color="blue")) %>%

add_trace(name="Bearish", y = posterior_probabilities$S3, line=list(color="red")) %>%

add_trace(name="S&P500", y = c(rep(NA,1), sp500_cret-1), line=list(color="black"))

S&P500 Market Regime Probabilities
(June 2014 to March 2017)

S&P500 Hidden Markov Model States (June 2014 to March 2017)

S&P500 Hidden Markov Model States (June 2014 to March 2017)

Interpretation: In any one “market regime”, the corresponding line/curve will “cluster” towards the top of the y-axis (i.e. near a probability of 100%).

For example, during a brief bullish run starting on 01 June 2014, the blue line/curve clustered near y-axis value 1.0. This correlates as you can see, with movement in the S&P500 (black line/curve). The same applies to bearish and “unknown” market states.

An interesting insight one can draw from this graphic, is how the Hidden Markov Model successfully reveals high volatility in the market between June 2014 and March 2015 (constantly changing states between bullish, bearish and unknown).

 

References:

[1] Murphy, K.P. (2012) Machine Learning – A Probabilistic Perspective, MIT Press.
https://www.cs.ubc.ca/~murphyk/MLbook/

Influences:

The honourable Mr. Michael Halls-Moore. QuantStart.com
http://www.quantstart.com/

Additional Resource: Learn more about DARWIN Portfolio Risk (VIDEO)
* please activate CC mode to view subtitles.

Do you have what it takes? – Join the Darwinex Trader Movement!

Darwinex - The Open Trader Exchange

Darwinex – The Open Trader Exchange