Playing with (and learning from) DOW Data
November 18th, 2008 by nikkelneTonight, we dine on data.
The past few weeks I started playing with raw DJI data imported into Matlab (once again, thank you Google). Figured I’d post it for all to critique or (hopefully) credit.
My weapon of choice for starters: the Fast Fourier Transform. Why? I think a lot of information might be hidden in the signal spectrum that isn’t obvious when you’re just watching the price indices, and I’m tired of riding blind. I’m not a fan of moving averages…they’re low pass filters (weed the noise out, useful as it may be)…its like putting black tape over your side windows. They might make you react less and drive straight (which is probably their real intent, much to the boon of investment firms who get to hoard your money), but they blind you to the 20 ton truck coming in to T-bone you. I’d rather see ALL the data in its wondrous glory.
Back to the basics here for a sec…this data is sampled daily, so it tells you absolutely nothing about behavior within periods less than 2 days (Nyquist).
This first data set spans everything from Jan 1 2000 up to present. Why that period? Because I got tired of copying and pasting, and didn’t want to write a full blown robot to grab data from google just yet (but thats on the to-do list). Will probably expand it to cover the 80’s and 90’s when I’m motivated…as I’m learning, there are events in that period we need to consider for sake of comparison. After all, being able to make good investment choices is partly about knowing how to react (or not to react) to news.
So first, a plot with the entire decade up till now, and the signal content (third subplot). Ignore the second one (just a repeat of first in this one)…part of a mechanic I use later.
Clicky to get a full-sized version.
Some very important things here, right off. The signal plots are shown with period, not frequency (because its easier to think i periods of days than frequences of micro-Hertz). At first glance a VERY large quantity of the signal content resides in low frequency (high period) realm, but it turns out there are only a few points there, so I can’t quite trust it, although it does make some sense…lots of people doing lots of things seemingly at random for no apparent reasons, ie, paychecks getting invested into 401k’s bi-weekly.
Now, this one is a zoom on the signal spectra of the opening, closing, high, low, and volume data…this is zoomed way in on the lower left of the one in the first plot (ignoring the low freq stuff). The high frequency stuff is the most meaningful and useful to us.
First off, opening/closing/high/low…they all look the same. So…we’ll just use closing from now on. Note the strong 200 - 250 day periods evident. Whats cool about this one is that a) its rather close to the famous 250 day moving average indicator that many people use (thereby giving it some credibility), and that you immediately see some strong correlation between price and volume at that period (funny…the 250 day moving average could be a chicken vs. the egg problem…people use it because its there because people use it…you catch my drift?). 250 days is too damn long of a period to filter anyways…recessions are born and dead within 250 days. You go by that one, and you’ll miss the bus. I think the volume correlation kind of discredits it too, don’t you?!?
Now lower down there in the periods, there is a lot going on in the price spectrum, with no real rhyme or reason.
In the volume, I see another peak at 63 days…again, thats the other period used often in the famous moving average indicator so many use. Pretty crazy…people unknowingly inject crap like this into the market just by using a certain analysis technique!!! There’s another large volume peak at around 90 days. Guess what happens every 90 days? Quarterly earnings. Boom…you guess it. The 180 day earnings period is the strongest of all (and seems kind of blended with the 250 day effect).
What we can take from this so far (I think): a) moving averages are bullcrap, and b) pay the hell attention to quarterly earnings statements.
How to interpret the rest of that unknown mess from 5 to 150 days is the real question (and the real money maker). If you can do that, methinks you’re golden.
One observation: a number of the periods that pop out down in there in certain sections (ie, 50 to 150 days) are of similar magnitude…this could tell one they are somehow related, no? The important thing there too is that they are overlapped in time…one of those peaks could be resident to a small window of time…thats the hypothesis I’ll tackle next. I’m outta gas for now, and my DOW spectrum movie maker isn’t done yet (but gettin close).
More to come.









