SPDR S&P 500 ETF TRUSTEducation
Editors' picks
Fourier: Interpreting and Over-interpreting Frequency Analysis
Greetings to all!
In this post, I'd like to share some thoughts on frequency analysis based on Fourier transform.
This mathematical method breaks down data into cyclical constituents (frequency components). Then the importance of each frequency component in the original data is expressed as the square of its amplitude, that is, power. Some time ago I published a Pine Script implementation of the Fast Fourier Transform (FFT) algorithm. I made it specially designed to be used for filtering data based on its frequency content. The concept of FFT filtering is well known and quite simple (for detail, see the link to my script below).
When it comes to frequency analysis, that's a whole different story. Using the power versus FFT frequency plot (i.e., the spectrum) to study the data is an extremely popular analytical technique in science and technology. But with financial data, it is tougher than it sounds. When dealing with noisy, non-stationary and overall uncertain price data, the FFT never works as well as it does in math textbooks.
Here, to illustrate the capabilities, limitations, and some myths of frequency analysis, I generated artificial price data, applied an FFT to each price column, and plotted a power spectrum as a function of time (i.e. a spectrogram). The spectrogram shows frequency information along the vertical axis. The lowest frequency content is displayed at the bottom, the highest frequency content is displayed at the top. Frequencies are given as the number of cycles per sample size (256 bars in the above chart). Power levels are defined by the color map shown to the right. As for the generated data, it is basically a random walk with a few non-random constituents added at given time intervals (sinusoidal functions and a gap).
Now let's discuss a few aspects seen in the chart:
1. True periodicity
Starting on a high note, if there is real oscillatory behavior in the data (see the green and red areas in the above chart), the FFT can reveal this quite well. The problem is that the frequency resolution is heavily influenced by the sample size. For example, with a sample size of 64 price bars, it is impossible to resolve cycles longer than, you guessed it, 64 bars. It should also be noted that the ideal deterministic sinusoidal functions shown in the above chart can never be found in real market data.
2. Discontinuities
If there is a gap, it's bad. Gaps, that is, discontinuities in the data, are prominent features that have broad and smooth frequency spectra that can overshadow other features. If you google "Fourier transform of a step function", you can see what I mean. Thus, if a gap occurs somewhere within the sample window, the Fourier spectrum is unlikely to be reliable.
3. Random walk
This is not surprising, but even purely random data can produce rich frequency content. It is even difficult to tell from the spectrogram where the purely-random area changes into the one containg a sinusoid. (Well, if we accept the random walk theory of markets, the last comment does not make much sense. But here we are talking about syncretic data).
The situation, however, becomes much more fun if we analyze the change in data per bar (i.e., data[0] - data[1]), rather than the actual data. In the case of a random walk, these changes are purely ... well, random. And it is known from textbooks that the Fourier spectrum of random noise contains only noise. However, in areas containing sine, there is an order in the data. In other words, the data is autocorrelated. As a result, the FFT spectra of the data reveal the corresponding frequency components way above the noise level. In this sense, Fourier analysis may seem as a potential tool to to test market efficiency. (There is even a Fourier-based version of the Dickey-Fuller test for stationarity of time series, but that is a different story.) However, I am not aware if it can somehow outperform the more commonly used tests.
Conclusion
In the context of financial time series, the Fourier transform is often associated with the estimation of marked cycles. I think that's why it has become one of the most polarizing technical tools out there. While it is certainly a powerful math tool, it is helpful to know its limitations. Can one capitalize on the revealed qualitative information about the "cycle" periods? Regardless of how to think about the concept of market cycles, I don't think so. But could the Fourier transform be potentially useful in general for analyzing price data? Definitely yes!
See below for how to use FFT to filter data:
In this post, I'd like to share some thoughts on frequency analysis based on Fourier transform.
This mathematical method breaks down data into cyclical constituents (frequency components). Then the importance of each frequency component in the original data is expressed as the square of its amplitude, that is, power. Some time ago I published a Pine Script implementation of the Fast Fourier Transform (FFT) algorithm. I made it specially designed to be used for filtering data based on its frequency content. The concept of FFT filtering is well known and quite simple (for detail, see the link to my script below).
When it comes to frequency analysis, that's a whole different story. Using the power versus FFT frequency plot (i.e., the spectrum) to study the data is an extremely popular analytical technique in science and technology. But with financial data, it is tougher than it sounds. When dealing with noisy, non-stationary and overall uncertain price data, the FFT never works as well as it does in math textbooks.
Here, to illustrate the capabilities, limitations, and some myths of frequency analysis, I generated artificial price data, applied an FFT to each price column, and plotted a power spectrum as a function of time (i.e. a spectrogram). The spectrogram shows frequency information along the vertical axis. The lowest frequency content is displayed at the bottom, the highest frequency content is displayed at the top. Frequencies are given as the number of cycles per sample size (256 bars in the above chart). Power levels are defined by the color map shown to the right. As for the generated data, it is basically a random walk with a few non-random constituents added at given time intervals (sinusoidal functions and a gap).
Now let's discuss a few aspects seen in the chart:
1. True periodicity
Starting on a high note, if there is real oscillatory behavior in the data (see the green and red areas in the above chart), the FFT can reveal this quite well. The problem is that the frequency resolution is heavily influenced by the sample size. For example, with a sample size of 64 price bars, it is impossible to resolve cycles longer than, you guessed it, 64 bars. It should also be noted that the ideal deterministic sinusoidal functions shown in the above chart can never be found in real market data.
2. Discontinuities
If there is a gap, it's bad. Gaps, that is, discontinuities in the data, are prominent features that have broad and smooth frequency spectra that can overshadow other features. If you google "Fourier transform of a step function", you can see what I mean. Thus, if a gap occurs somewhere within the sample window, the Fourier spectrum is unlikely to be reliable.
3. Random walk
This is not surprising, but even purely random data can produce rich frequency content. It is even difficult to tell from the spectrogram where the purely-random area changes into the one containg a sinusoid. (Well, if we accept the random walk theory of markets, the last comment does not make much sense. But here we are talking about syncretic data).
The situation, however, becomes much more fun if we analyze the change in data per bar (i.e., data[0] - data[1]), rather than the actual data. In the case of a random walk, these changes are purely ... well, random. And it is known from textbooks that the Fourier spectrum of random noise contains only noise. However, in areas containing sine, there is an order in the data. In other words, the data is autocorrelated. As a result, the FFT spectra of the data reveal the corresponding frequency components way above the noise level. In this sense, Fourier analysis may seem as a potential tool to to test market efficiency. (There is even a Fourier-based version of the Dickey-Fuller test for stationarity of time series, but that is a different story.) However, I am not aware if it can somehow outperform the more commonly used tests.
Conclusion
In the context of financial time series, the Fourier transform is often associated with the estimation of marked cycles. I think that's why it has become one of the most polarizing technical tools out there. While it is certainly a powerful math tool, it is helpful to know its limitations. Can one capitalize on the revealed qualitative information about the "cycle" periods? Regardless of how to think about the concept of market cycles, I don't think so. But could the Fourier transform be potentially useful in general for analyzing price data? Definitely yes!
See below for how to use FFT to filter data:
DISCLAIMER: I am not a financial advisor, and my scripts are for educational purposes only. Any trades you make are at your own risk.
Disclaimer
The information and publications are not meant to be, and do not constitute, financial, investment, trading, or other types of advice or recommendations supplied or endorsed by TradingView. Read more in the Terms of Use.