Juris Mindcraft: Predicting the Stock Market

As part of our R&D efforts with Mindcraft AI, we have tried to predict the movement of KLSE shares using our Artificial Intelligence engine.

To simplify the project, we decided to only use price-action-movement to predict share prices, namely:

Daily opening price
Daily closing price
Daily high
Daily low
Daily volume
Correlation with KLSE Composite Index
Various Technical Indicators such as Candlesticks, MACD, RSI, etc.

The goal was to see if we could predict whether any given stock would go up at least 5% in 2 weeks.

We felt that this goal was achievable because many traders have been successfully trading using only this information, and some traders have apparently automated their systems and also use AI. For examples of this, see Market Wizards and the newer book Hedge Fund Wizards.

KLSE data was purchased from bizfun.cc as the data is very clean and stock splits and rights issues are corrected for you.

Training was on 2011 and 2012 share price data, and then tested on data from 1995 to 2014. We used all KLSE stocks, excluding low volume stocks as these are too unpredictable.

How Mindcraft AI works

Mindcraft is a pattern recognition engine. It finds patterns in the data and assigns probabilities to how confident it thinks these patterns are. The key feature of AI is that it is able to find interesting patterns even if the data is imperfect, dirty or incomplete. A typical rule generated could look like this:

  IF Moving Average of Share Closing Price Dropped by 20-50% 
    AND Share Correlation with KLCI > 0.5 
    AND KLCI above Moving Average
    AND Share Candlestick is Tweezers Top 
  THEN BUY (67% probability)

Best Results for KLSE 1995-2014

The results for 2011 and 2012 should be good, as that was used to train the AI. The key thing to see is how good the predictions are with years it has never seen before, such as 1995-2010.

Year	Good Predictions	#Predictions	Invest/Year	Gains/Loss 14 days after purchase	% Returns (after Trx Costs)
2014	83.6%	55.0	448,447	68,847	15.4
2013	75%	12.0	139,282	12,382	8.9
2012	100%	2.0	6,668	2,440	36.6
2011	84%	25.0	280,811	27,443	9.8
2010	25%	4.0	40,000	-3,319.0	-8.3
2009	0%	1.0	11,997	-360.0	-3.0
2008	69.1%	175.0	1,393,747	258,432	18.5
2007	100%	27.0	286,422	83,031	29.0
2006	68.8%	16.0	191,002	4,946	2.6
2005	0%	3.0	11,465	-933.0	-8.1
2003	100%	4.0	46,490	3,065	6.6
2002	100%	1.0	5,434	495.0	9.1
2001	82.9%	41.0	332,359	39,277	11.8
2000	67.3%	49.0	212,641	14,671	6.9
1999	81.3%	80.0	731,509	78,097	10.7
1998	70.4%	142.0	644,163	247,561	38.4
1997	67.8%	566.0	1,752,588	432,634	24.7
1996	100%	3.0	40,000	5,511	13.8
1995	91.1%	45.0	300,384	102,644	34.2

[rn_id=560596036, cf>=0.5]

The amount invested per trade was capped at RM 10,000, with no trade allowed to be more than 1% of the total RM volume of the shares for that day (because large share purchases would move the share price). Losses were cut if the share price fell by 20%. Otherwise the shares were sold after the 14 days. We assume transaction costs of 0.5% for buy and sell combined.

Using AI for Investment

The results are currently experimental because the AI works best when there is a lot of volatility in the market. When the market is stable, the AI finds few counters to trade, so some years are very lean. We can make the AI less picky, but then accuracy will drop.

The returns can fluctuate a lot. Some years are very positive, some years negative. This strategy is only suitable if you are a risk taker.

Lastly, the model is very specific to the data provided. Testing on HK stock market counters reveals that different variables are picked up by the AI.

Overall the data is very promising and more research can be done.

Technical Notes

This is one of the largest datasets we have analysed. The KLSE data extends from 1995 to 2015 and is 3.6 million records. We used a cluster of 5 servers and ran 26 CPU cores in parallel at peak for data mining.