Indicators Input Window Length - Problems And Solutions
Introduction
Most technical indicators possesses a user defined input window length, this input affect the indicator output and for a long time, have been the cause of many innovations in technical analysis.
In this post i want to discuss the effects and particularities of indicators inputs window length, the challenges they introduce in trading and their effect when paired with machine learning forecasting models, i hope this post will be easy to read, let me know if you had difficulties understanding it.
Speed And Efficiency Problems
An input window length can involve the number of data processed by the indicator, therefore higher window length's will process more data, which result in a slower computation time, therefore in high-frequency/algorithmic trading where response time matter, maximizing the profitability might be made at the cost of the indicator response time, and even if computerized trading has been praised for its speed, small lag times can actually affect your strategy, therefore one might enter a trade at a different value than the targeted price.
Note : High frequency trading (hft) is a commonly mistaken term, one might believe that hft require the trader to open and close a large number of trades in a short amount of time, in reality hft is related to the "rate at which data is processed".
Solution - Efficient Data Processing
Solutions have been proposed in order to make certain tools more efficient. For example the simple moving average is a common tool that is the basis of many other indicators, its calculation involve summing the length last data points and diving this sum by length. In signal processing, such tool require what is called "memory", the data points must be stored in order for them to be processed, this is extremely inefficient and slow, therefore alternatives have been proposed, one of them is still mainly used in technical analysis today and is called the exponential moving average (ema), the process of computing an exponential moving average is called exponential averaging, and has the form of :
ema = sc*input+(1-sc)*(past ema value)
where sc is called the smoothing constant where 1 > sc > 0. We only need 2 data values in order to perform this computation, lets denote a moving average of period length sma(input,length), we can estimate it using exponential averaging with sc = 2/(length+1). The computation time of the exponential moving average is way lower than the one of the sma. This is the most elegant and efficient estimation of the simple moving average.
The exponential moving average is the simplest "IIR Filter", or infinite response filter, those filters are as well extremely efficient since they use recursion. Exponential averaging is also the core of many adaptive indicators. In my experience, recursion will always let you create extremely efficient tools.
Window Length And Optimization Problems
Optimization is a branch of mathematics that help us find the best parameters in order to maximize/minimize a certain function, and thanks to computers this process can be made faster. Optimizing technical indicators during backtesting involve finding the input window length (set of inputs if there are more than 1 input) that maximize the profit of a strategy.
The most common approach is brute forcing, in which we test every indicator inputs window length combination and keep the one that yield the best results. However optimization is still computationally intensive, having 2 indicators already involve a high number of combinations. This is why it is important to select a low number of indicators for your strategy. But then other problems arise, the best input window length (set of inputs) might change in the future. This is due to the fact the market price is non-stationary and one of the reasons technical indicators are looked down.
In order to deal with this problem, we can propose the following solutions :
Use indicators/Information with no input window length -> Vwap/Volume/True Range/Cumulative Mean...etc.
Study the relationship between the optimal input window length and price evolution -> Regression analysis
Forecast the optimal input window length -> Forecasting
The last two are extremely inefficient, kinda nightmarish, and would be time consuming if one use a serious backtesting procedure. However the first solution is still appealing and might actually provide a efficient result.
Machine Learning Forecasting - Performance And Technical indicators Input Window Length Dependency
Technical indicators outputs can be used as inputs for machine learning algorithms. We could think that we also need to optimize the input window length of the indicators when using machine learning (which would lead to high computations time, machine learning already involve optimization of a high number of parameters), however a research paper named "Forecasting price movements using technical indicators: Investigating the impact of varying input window length" by Yauheniya Shynkevicha, T.M. McGinnitya, Sonya A. Colemana, Ammar Belatrechec and Yuhua Li highlight an interesting phenomenon, the abstract tell us that :
"The highest prediction performance is observed when the input window length is approximately equal to the forecast horizon"
In short, if you want to forecast market price 14 step ahead with a machine learning model, you should use indicators with input window length approximately equal to 14 as inputs for the model in order to get the best performance. This would allow to skip a lot of optimizations processes regarding the technical indicators used in the model. They used 3 different type of ML algorithms, support vector machine (svm), adversarial neural networks (ann) and k nearest neighboring (knn), which reinforce their conclusion.
In the paper, we can see something interesting with the indicators they selected as inputs, they used : A simple moving average, an exponential moving average, the average true range, the Average Directional Movement Index, CCI, ROC, RSI, %R, stochastic oscillator.
First thing we can see is that they used the exponential moving average instead of the wilder moving average for certain calculations, which i think is a good choice. We can also see they used many indicators outputting the same kind of information, in this case we often talk about "Multicollinearity", for example :
The CCI, ROC, RSI, %R, Stochastic output similar information, all remove the trend in the price, the CCI and ROC are both centered around 0 and the %R, RSI and stochastic oscillator around 50. The SMA and EMA also output similar information.
In technical analysis this practice is often discouraged since the indicators will output the same kind of information, this lead to redundancy. However such practice has been seen a lot in machine learning models using technical indicators. Maybe that a higher amount of multicollinearity between indicators allow to strengthen the relationship between the forecast horizon and the indicators input window length.
Conclusion
We talked (a lot) about indicator inputs window length, what problems they cause us and how we can find solutions to those problems. Also we have seen that the forecasting performance of ML models can be higher when they use indicators outputs with input window length equal to the forecasting horizon. This can make to make the process of forecasting financial market price with ML models using technical indicators more efficient.
ML is a recurring subject in financial forecasting, those algorithms offer the hope to make technical indicators more useful, and indeed, technical indicators and ML models can benefits from each others, however it is sad to observe that classical indicators are mainly used instead of newer ones, but its also encouraging in the sense that more research can be done, using newer material/procedures.
Thanks for reading !
Most technical indicators possesses a user defined input window length, this input affect the indicator output and for a long time, have been the cause of many innovations in technical analysis.
In this post i want to discuss the effects and particularities of indicators inputs window length, the challenges they introduce in trading and their effect when paired with machine learning forecasting models, i hope this post will be easy to read, let me know if you had difficulties understanding it.
Speed And Efficiency Problems
An input window length can involve the number of data processed by the indicator, therefore higher window length's will process more data, which result in a slower computation time, therefore in high-frequency/algorithmic trading where response time matter, maximizing the profitability might be made at the cost of the indicator response time, and even if computerized trading has been praised for its speed, small lag times can actually affect your strategy, therefore one might enter a trade at a different value than the targeted price.
Note : High frequency trading (hft) is a commonly mistaken term, one might believe that hft require the trader to open and close a large number of trades in a short amount of time, in reality hft is related to the "rate at which data is processed".
Solution - Efficient Data Processing
Solutions have been proposed in order to make certain tools more efficient. For example the simple moving average is a common tool that is the basis of many other indicators, its calculation involve summing the length last data points and diving this sum by length. In signal processing, such tool require what is called "memory", the data points must be stored in order for them to be processed, this is extremely inefficient and slow, therefore alternatives have been proposed, one of them is still mainly used in technical analysis today and is called the exponential moving average (ema), the process of computing an exponential moving average is called exponential averaging, and has the form of :
ema = sc*input+(1-sc)*(past ema value)
where sc is called the smoothing constant where 1 > sc > 0. We only need 2 data values in order to perform this computation, lets denote a moving average of period length sma(input,length), we can estimate it using exponential averaging with sc = 2/(length+1). The computation time of the exponential moving average is way lower than the one of the sma. This is the most elegant and efficient estimation of the simple moving average.
The exponential moving average is the simplest "IIR Filter", or infinite response filter, those filters are as well extremely efficient since they use recursion. Exponential averaging is also the core of many adaptive indicators. In my experience, recursion will always let you create extremely efficient tools.
Window Length And Optimization Problems
Optimization is a branch of mathematics that help us find the best parameters in order to maximize/minimize a certain function, and thanks to computers this process can be made faster. Optimizing technical indicators during backtesting involve finding the input window length (set of inputs if there are more than 1 input) that maximize the profit of a strategy.
The most common approach is brute forcing, in which we test every indicator inputs window length combination and keep the one that yield the best results. However optimization is still computationally intensive, having 2 indicators already involve a high number of combinations. This is why it is important to select a low number of indicators for your strategy. But then other problems arise, the best input window length (set of inputs) might change in the future. This is due to the fact the market price is non-stationary and one of the reasons technical indicators are looked down.
In order to deal with this problem, we can propose the following solutions :
Use indicators/Information with no input window length -> Vwap/Volume/True Range/Cumulative Mean...etc.
Study the relationship between the optimal input window length and price evolution -> Regression analysis
Forecast the optimal input window length -> Forecasting
The last two are extremely inefficient, kinda nightmarish, and would be time consuming if one use a serious backtesting procedure. However the first solution is still appealing and might actually provide a efficient result.
Machine Learning Forecasting - Performance And Technical indicators Input Window Length Dependency
Technical indicators outputs can be used as inputs for machine learning algorithms. We could think that we also need to optimize the input window length of the indicators when using machine learning (which would lead to high computations time, machine learning already involve optimization of a high number of parameters), however a research paper named "Forecasting price movements using technical indicators: Investigating the impact of varying input window length" by Yauheniya Shynkevicha, T.M. McGinnitya, Sonya A. Colemana, Ammar Belatrechec and Yuhua Li highlight an interesting phenomenon, the abstract tell us that :
"The highest prediction performance is observed when the input window length is approximately equal to the forecast horizon"
In short, if you want to forecast market price 14 step ahead with a machine learning model, you should use indicators with input window length approximately equal to 14 as inputs for the model in order to get the best performance. This would allow to skip a lot of optimizations processes regarding the technical indicators used in the model. They used 3 different type of ML algorithms, support vector machine (svm), adversarial neural networks (ann) and k nearest neighboring (knn), which reinforce their conclusion.
In the paper, we can see something interesting with the indicators they selected as inputs, they used : A simple moving average, an exponential moving average, the average true range, the Average Directional Movement Index, CCI, ROC, RSI, %R, stochastic oscillator.
First thing we can see is that they used the exponential moving average instead of the wilder moving average for certain calculations, which i think is a good choice. We can also see they used many indicators outputting the same kind of information, in this case we often talk about "Multicollinearity", for example :
The CCI, ROC, RSI, %R, Stochastic output similar information, all remove the trend in the price, the CCI and ROC are both centered around 0 and the %R, RSI and stochastic oscillator around 50. The SMA and EMA also output similar information.
In technical analysis this practice is often discouraged since the indicators will output the same kind of information, this lead to redundancy. However such practice has been seen a lot in machine learning models using technical indicators. Maybe that a higher amount of multicollinearity between indicators allow to strengthen the relationship between the forecast horizon and the indicators input window length.
Conclusion
We talked (a lot) about indicator inputs window length, what problems they cause us and how we can find solutions to those problems. Also we have seen that the forecasting performance of ML models can be higher when they use indicators outputs with input window length equal to the forecasting horizon. This can make to make the process of forecasting financial market price with ML models using technical indicators more efficient.
ML is a recurring subject in financial forecasting, those algorithms offer the hope to make technical indicators more useful, and indeed, technical indicators and ML models can benefits from each others, however it is sad to observe that classical indicators are mainly used instead of newer ones, but its also encouraging in the sense that more research can be done, using newer material/procedures.
Thanks for reading !
Check out the indicators we are making at luxalgo: tradingview.com/u/LuxAlgo/
Disclaimer
The information and publications are not meant to be, and do not constitute, financial, investment, trading, or other types of advice or recommendations supplied or endorsed by TradingView. Read more in the Terms of Use.