weird data

I really liked this story on the 37signals blog yesterday where Jason Fried explained the process of seemingly serendipitous events that led to his being asked to write an opinion piece in the New York Times.  I've been working with network visualization lately and turned his story into the graphic below, which they've kindly posted back to the 37 signals blog. 

In Part 2, we showed how to add recession shading to a plot of American Beards over time, and did some diagnostics to check whether 19th Century Americans grew recession beards. (Spoiler alert: it appears they did not.)  In Part 1, we showed how to plot the series in the first place.  Today, we're going to look at the beardly trend over the period.  We all know about the gilded age popularity of mutton chops and sideburns, but were full beards on the rise or on the decline between 1866 and 1911?  And more importantly, what can this period tell us about beards of the future (in the past)?!

What you'll learn

• How to apply linear smoothing to a time-series plot in ggplot2
• How to interpret that and how not to interpret that.  (The difference between interpolation and extrapolation.)
• Whether beards were getting "trendier" or trending less during the tail end of the 19th Century.
• What date we're all going to have beards.

Checking my web analytics, I noticed that Design & Analytics is now in the coveted #4 google hit position for "weird data sets."  That means I'm probably pretty close to getting Nike sponsorship and my face on a Wheaties box in the data olympics category.

In Part I, we showed how to plot a time series of the change in American beards over time, using a dataset from Robert Hyndman's time series data library.  Today, we're going to look at whether the dramatic changes in American male beardfulness seem related to the economy.  Did Americans grow recession beards in response to the Panic of 1873?  Out of work, did they forgo their frequent trips to the barber (since Gillette didn't invent the personal safety razor until 1904 so they could do it themselves)?  Did they go on their job interviews with a face full of mutton chops and just never get called back (by telegram)?

What you'll learn

• How to apply recession shading, according to the NBER's recession dates, to a time series---easily.
• How to change the color of the recession shading
• Whether 19th century gentlemen grew recession beards after losing their jobs in algo trading, you know, just taking some time off to study graphic design for a little bit.

If you use R for time series analysis, chances are you've used Robert J. Hyndman's excellent forecast tools.  I recently stumbled on his time series data library where I found just the data set I've been looking for to show some R time series plotting tricks:

http://robjhyndman.com/tsdldata/roberts/beards.dat

It's the percentage of American men with full beards reported annually.  Nothing serious here, but absurdly perfect for a set of posts to share a couple things that took me a while to learn when plotting in R.

What you'll learn

• How to grab data from a plaintext source on the web, stripping header information
• How to convert a list of data with a known start time and end time into an xts time series object
• How to convert xts to a data.frame for plotting in ggplot2
• Aesthetics for red data points with dotted line interpolation
• How much American beardfulness there was in the late 19th century and early 20th.