Forecasting

According to the ishares blog, the Chicago Fed National Activity Index (CFNAI) was recently named the most underrated index for measuring the US economy's health.  I wholeheartedly agree.  I co-ran this beauty when I worked at the Fed in 2008-2009 and automated its graphing in Matlab.  If you look back in the archives from around then, you'll see...

I have an opening to take on more hours of client work beginning in mid-September.  Please feel free to contact me if you're looking for support in forecasting, automation, quantitative design, social network analysis---or anything I've written about here.

In Part 2, we showed how to add recession shading to a plot of American Beards over time, and did some diagnostics to check whether 19th Century Americans grew recession beards. (Spoiler alert: it appears they did not.)  In Part 1, we showed how to plot the series in the first place.  Today, we're going to look at the beardly trend over the period.  We all know about the gilded age popularity of mutton chops and sideburns, but were full beards on the rise or on the decline between 1866 and 1911?  And more importantly, what can this period tell us about beards of the future (in the past)?!

What you'll learn

• How to apply linear smoothing to a time-series plot in ggplot2
• How to interpret that and how not to interpret that.  (The difference between interpolation and extrapolation.)
• Whether beards were getting "trendier" or trending less during the tail end of the 19th Century.
• What date we're all going to have beards.

In Part I, we showed how to plot a time series of the change in American beards over time, using a dataset from Robert Hyndman's time series data library.  Today, we're going to look at whether the dramatic changes in American male beardfulness seem related to the economy.  Did Americans grow recession beards in response to the Panic of 1873?  Out of work, did they forgo their frequent trips to the barber (since Gillette didn't invent the personal safety razor until 1904 so they could do it themselves)?  Did they go on their job interviews with a face full of mutton chops and just never get called back (by telegram)?

What you'll learn

• How to apply recession shading, according to the NBER's recession dates, to a time series---easily.
• How to change the color of the recession shading
• Whether 19th century gentlemen grew recession beards after losing their jobs in algo trading, you know, just taking some time off to study graphic design for a little bit.

If you use R for time series analysis, chances are you've used Robert J. Hyndman's excellent forecast tools.  I recently stumbled on his time series data library where I found just the data set I've been looking for to show some R time series plotting tricks:

http://robjhyndman.com/tsdldata/roberts/beards.dat

It's the percentage of American men with full beards reported annually.  Nothing serious here, but absurdly perfect for a set of posts to share a couple things that took me a while to learn when plotting in R.

What you'll learn

• How to grab data from a plaintext source on the web, stripping header information
• How to convert a list of data with a known start time and end time into an xts time series object
• How to convert xts to a data.frame for plotting in ggplot2
• Aesthetics for red data points with dotted line interpolation
• How much American beardfulness there was in the late 19th century and early 20th.

Just a minor note for the observant---I'm reprinting and reposting the ARIMA sector reports since March 12.  I was using ggplot2 as my graphics package (in R), which changed the way it formatted dates in a recent upgrade.  Since March, this meant the ARIMA forecast plots had the date format "2012-03-12" rather than simply listing full months by name, such as "March."  Past data was rerun, so everything else is identical, but the format of the date axis has been updated to the original, monthly format for purely aesthetic reasons.

The sector forecasts will be down for maintenance tomorrow, Thursday, December 15.  However, for historical purposes, the Thursday forecast will be posted afterwards on Friday, December 16.

For several months now, I've been putting together an automated econometric forecasting platform.  Using a simple ARIMA model, I've created a forecast for stock sectors to inform my own short-term option trades.  Despite being a statistical model, rather than a foundational one about informationally poor stocks, I found it useful for my purposes---though I make no promises about yours---and developed this infrastructure in order to share it online.  Even if you're not interested in stock market movements, the further utility of the platform is as a demonstration of the capabilities of reports automation: this platform runs an analysis every day, makes forecasts, charts, typesets instructions and accompanying advertizements, and publishes them to PDF and in an animated gallery online.  This is an analysis that once automated, never again needs human intervention.  That's a powerful thing, especially in how it frees this human to solve new problems.

Here it is: http://www.designandanalytics.com/ARIMA-Sectors  And here is where to sign up for the newsletter providing coverage.

I've posted before about Google correlate in Google Correlate for fun and profit. It's a fantastic platform, but I have not yet discovered any practical use cases for it.  This is still the case, but after experimenting with it more, I now have a better idea of what they would need to do (and why they can't do it), to make it more useful.

...well, if you find the "profit" application, let me know.

Google labs has a new product called Google Correlate.  You can read the product's introductory description here (in comic book form).  The service takes any time-based data series you give it and matches the Google search queries that have the highest correlation with your series, then does you the additional favor of plotting it on a map---very cool.