fivethirtyeight Package

Albert Y. Kim, Chester Ismay, and Jennifer Chunn

2017-08-08

Motivation

We are involved in statistics and data science education, in particular at the introductory undergraduate level. As such, we are always looking for data sets that balance being

  1. Rich enough to answer meaningful questions with, real enough to ensure that there is context, and realistic enough to convey to students that data as it exists “in the wild” often needs processing.
  2. Easily and quickly accessible to novices, so that we minimize the prerequisites to research.

It has been our experience that many data sets that exist in R packages, such as the nycflights13, babynames, and gapminder packages, are of great pedagogical value as they:

It is along these lines that we present fivethirtyeight: an R package of data and code behind the stories and interactives at FiveThirtyEight.com, a data-driven journalism website founded by Nate Silver and owned by ESPN. FiveThirtyEight has been very forward thinking in making the data used in many of their articles open and accessible on GitHub, a web-based repository for collaboration on code and data.

With consultation from Andrew Flowers and Andrei Scheinkman of FiveThirtyEight, we go one step further by:

  1. Doing just enough pre-processing so that statistics and data science novices can sink their teeth into the data right away.
  2. Packaging it all in an easy to load format: package installation instead of working with CSV files.
  3. Providing easily accessible documentation: The help file for each data set includes a thorough description of the observational unit and all variables, a link to the original article, and (if listed) the data sources.

Guidelines

In order to make the data easily accessible to R novices, we pre-process the original data sets as they exist in the 538 GitHub repository to adhere to the following guidelines:

Note: The code used to pre-process the data can be found on the GitHub repository for the package in the process_data_sets.R files. These can serve as data manipulation/wrangling examples and exercises for more advanced students.

Data Sets

Data Frame Name Article Title Filed Under Author 1 Author 2 Author 3 Date URL
US_births_1994_2003 Some People Are Too Superstitious To Have A Baby On Friday The 13th Parenting Carl Bialik NA NA 2016-05-13 http://fivethirtyeight.com/features/some-people-are-too-superstitious-to-have-a-baby-on-friday-the-13th/
US_births_2000_2014 Some People Are Too Superstitious To Have A Baby On Friday The 13th Parenting Carl Bialik NA NA 2016-05-13 http://fivethirtyeight.com/features/some-people-are-too-superstitious-to-have-a-baby-on-friday-the-13th/
airline_safety Should Travelers Avoid Flying Airlines That Have Had Crashes in the Past? Airline Safety Nate Silver NA NA 2014-07-18 http://fivethirtyeight.com/features/should-travelers-avoid-flying-airlines-that-have-had-crashes-in-the-past/
antiquities_act Trump Might Be The First President To Scrap A National Monument Trump Administration Ella Koeze NA NA 2017-04-27 https://fivethirtyeight.com/features/trump-might-be-the-first-president-to-scrap-a-national-monument/
avengers Joining The Avengers Is As Deadly As Jumping Off A Four-Story Building Movies Walt Hickey NA NA 2015-05-12 http://fivethirtyeight.com/features/avengers-death-comics-age-of-ultron/
bad_drivers Dear Mona, Which State Has The Worst Drivers? Hope the Numbers Help Mona Chalabi NA NA 2014-10-24 http://fivethirtyeight.com/datalab/which-state-has-the-worst-drivers/
bechdel The Dollar-And-Cents Case Against Hollywood’s Exclusion of Women Movies Walt Hickey NA NA 2014-04-01 http://fivethirtyeight.com/features/the-dollar-and-cents-case-against-hollywoods-exclusion-of-women/
biopics Straight Outta Compton Is The Rare Biopic Not About White Dudes Movies Hannah Fingerhut NA NA 2015-08-14 http://fivethirtyeight.com/features/straight-outta-compton-is-the-rare-biopic-not-about-white-dudes/
bob_ross A Statistical Analysis of the Work of Bob Ross Happy Spreadsheets Walt Hickey NA NA 2014-04-14 http://fivethirtyeight.com/features/a-statistical-analysis-of-the-work-of-bob-ross/
cand_events_20150114 Looking For Clues: Who Is Going To Run For President In 2016? The Visible Primary Harry Enten NA NA 2015-01-14 http://fivethirtyeight.com/datalab/2016-president-who-is-going-to-run/
cand_events_20150130 Who Will Run For President: Romney Is Out The Visible Primary Harry Enten NA NA 2015-01-30 http://fivethirtyeight.com/datalab/romney-not-running-for-president/
cand_state_20150114 Looking For Clues: Who Is Going To Run For President In 2016? The Visible Primary Harry Enten NA NA 2015-01-14 http://fivethirtyeight.com/datalab/2016-president-who-is-going-to-run/
cand_state_20150130 Who Will Run For President: Romney Is Out The Visible Primary Harry Enten NA NA 2015-01-30 http://fivethirtyeight.com/datalab/romney-not-running-for-president/
classic_rock_raw_data Why Classic Rock Isn’t What It Used To Be Music Walt Hickey NA NA 2014-07-07 http://fivethirtyeight.com/features/why-classic-rock-isnt-what-it-used-to-be/
classic_rock_song_list Why Classic Rock Isn’t What It Used To Be Music Walt Hickey NA NA 2014-07-07 http://fivethirtyeight.com/features/why-classic-rock-isnt-what-it-used-to-be/
college_all_ages The Economic Guide To Picking A College Major NA Ben Casselman NA NA 2014-09-12 http://fivethirtyeight.com/features/the-economic-guide-to-picking-a-college-major/
college_grad_students The Economic Guide To Picking A College Major NA Ben Casselman NA NA 2014-09-12 http://fivethirtyeight.com/features/the-economic-guide-to-picking-a-college-major/
college_recent_grads The Economic Guide To Picking A College Major NA Ben Casselman NA NA 2014-09-12 http://fivethirtyeight.com/features/the-economic-guide-to-picking-a-college-major/
comic_characters Comic Books Are Still Made By Men, For Men And About Men Comic Books Walt Hickey NA NA 2014-10-13 http://fivethirtyeight.com/features/women-in-comic-books/
comma_survey Elitist, Superfluous, Or Popular? We Polled Americans on the Oxford Comma NA Walt Hickey NA NA 2014-06-17 http://fivethirtyeight.com/datalab/elitist-superfluous-or-popular-we-polled-americans-on-the-oxford-comma/
congress_age Both Republicans And Democrats Have an Age Problem Age Nate Silver Dhrumil Mehta NA 2014-04-28 http://fivethirtyeight.com/features/both-republicans-and-democrats-have-an-age-problem/
cousin_marriage How Many Americans Are Married To Their Cousins? Dear Mona Mona Chalabi NA NA 2015-05-15 https://fivethirtyeight.com/features/how-many-americans-are-married-to-their-cousins/
daily_show_guests Every Guest Jon Stewart Ever Had On ‘The Daily Show’ TV Walt Hickey NA NA 2015-08-06 http://fivethirtyeight.com/datalab/every-guest-jon-stewart-ever-had-on-the-daily-show/
democratic_bench Some Democrats Who Could Step Up If Hillary Isn’t Ready For Hillary 2016 Election Harry Enten NA NA 2015-03-27 http://fivethirtyeight.com/datalab/some-democrats-who-could-step-up-if-hillary-isnt-ready-for-hillary/
drinks Dear Mona Followup: Where Do People Drink The Most Beer, Wine And Spirits? Drinks Mona Chalabi NA NA 2014-08-13 http://fivethirtyeight.com/datalab/dear-mona-followup-where-do-people-drink-the-most-beer-wine-and-spirits/
drug_use How Baby Boomers Get High Public Health Anna Maria Barry-Jester Andrew Flowers NA 2015-04-23 http://fivethirtyeight.com/datalab/how-baby-boomers-get-high/
elo_blatter Blatter’s Reign At FIFA Hasn’t Helped Soccer’s Poor World Soccer Nate Silver NA NA 2015-06-09 http://fivethirtyeight.com/features/blatters-reign-at-fifa-hasnt-helped-soccers-poor/
endorsements Pols And Polls Say The Same Thing: Jeb Bush Is A Weak Front-Runner 2016 Kickoff Harry Enten NA NA 2015-06-15 http://fivethirtyeight.com/features/pols-and-polls-say-the-same-thing-jeb-bush-is-a-weak-front-runner/
fandango Be Suspicious Of Online Movie Ratings, Especially Fandango’s Movies Walt Hickey NA NA 2015-10-15 http://fivethirtyeight.com/features/fandango-movies-ratings/
fifa_audience How To Break FIFA The Beautiful Game Nate Silver NA NA 2015-05-28 http://fivethirtyeight.com/features/how-to-break-fifa/
flying 41 Percent Of Fliers Think You’re Rude If You Recline Your Seat NA Walt Hickey NA NA 2014-09-06 http://fivethirtyeight.com/datalab/airplane-etiquette-recline-seat/
food_world_cup The FiveThirtyEight International Food Association’s 2014 World Cup FiveThirtyAte Nate Silver Walt Hickey NA 2014-06-25 http://fivethirtyeight.com/features/the-fivethirtyeight-international-food-associations-2014-world-cup/
goose The Save Ruined Relief Pitching. The Goose Egg Can Fix It. MLB Nate Silver NA NA NA https://fivethirtyeight.com/features/goose-egg-new-save-stat-relief-pitchers/
hate_crimes Higher Rates Of Hate Crimes Are Tied To Income Inequality Hate Crimes Maimuna Majumder NA NA 2017-01-23 https://fivethirtyeight.com/features/higher-rates-of-hate-crimes-are-tied-to-income-inequality/
hiphop_cand_lyrics Every mention of the 2016 primary candidates in hip-hop songs NA Allison McCann NA NA 2016-07-14 https://projects.fivethirtyeight.com/clinton-trump-hip-hop-lyrics/
hist_ncaa_bball_casts The NCAA Bracket: Checking Our Work March Madness Nate Silver NA NA 2014-03-27 http://fivethirtyeight.com/datalab/the-ncaa-bracket-checking-our-work/
hist_senate_preds How The FiveThirtyEight Senate Forecast Model Works 2014 Midterms Nate Silver NA NA 2014-09-17 http://fivethirtyeight.com/features/how-the-fivethirtyeight-senate-forecast-model-works/
librarians Where Are America’s Librarians? Jobs Mona Chalabi Andrew Flowers NA 2014-04-11 http://fivethirtyeight.com/datalab/where-are-americas-librarians/
love_actually_adj The Definitive Analysis Of ‘Love Actually,’ The Greatest Christmas Movie Of Our Time Movies Walt Hickey Gus Wezerek NA 2016-12-22 http://fivethirtyeight.com/features/the-definitive-analysis-of-love-actually-the-greatest-christmas-movie-of-our-time/
love_actually_appearance The Definitive Analysis Of ‘Love Actually,’ The Greatest Christmas Movie Of Our Time Movies Walt Hickey Gus Wezerek NA 2016-12-22 http://fivethirtyeight.com/features/the-definitive-analysis-of-love-actually-the-greatest-christmas-movie-of-our-time/
mad_men Mad Men Is Ending. What’s Next For The Cast? TV Walt Hickey NA NA 2015-05-15 http://fivethirtyeight.com/datalab/mad-men-is-ending-whats-next-for-the-cast/
male_flight_attend Dear Mona, How Many Flight Attendants Are Men? Hope the Numbers Help Mona Chalabi NA NA 2014-10-03 http://fivethirtyeight.com/datalab/dear-mona-how-many-flight-attendants-are-men/
mlb_as_play_talent The Best MLB All-Star Teams Ever Baseball Neil Paine NA NA 2015-07-18 http://fivethirtyeight.com/features/the-best-mlb-all-star-teams-ever/
mlb_as_team_talent The Best MLB All-Star Teams Ever Baseball Neil Paine NA NA 2015-07-18 http://fivethirtyeight.com/features/the-best-mlb-all-star-teams-ever/
murder_2015_final A Handful Of Cities Are Driving 2016’s Rise In Murder Crime Jeff Asher NA NA 2016-10-06 http://fivethirtyeight.com/features/a-handful-of-cities-are-driving-2016s-rise-in-murders/
murder_2016_prelim A Handful Of Cities Are Driving 2016’s Rise In Murder Crime Jeff Asher NA NA 2016-10-06 http://fivethirtyeight.com/features/a-handful-of-cities-are-driving-2016s-rise-in-murders/
nba_draft_2015 Projecting The Top 50 Players In The 2015 NBA Draft Class Basketball Neil Paine Zach Bradshaw NA 2015-06-19 http://fivethirtyeight.com/features/projecting-the-top-50-players-in-the-2015-nba-draft-class/
nba_elo NA NA NA NA NA NA NA
nba_tattoos Accurately Counting NBA Tattoos Isn’t Easy, Even If You’re Up Close NA Mike Wilson NA NA 2014-06-07 http://fivethirtyeight.com/datalab/accurately-counting-nba-tattoos-isnt-easy-even-if-youre-up-close/
nfl_fav_team The Rams Are Dead To Me, So I Answered 3,352 Questions To Find A New NFL Team NFL Blythe Terrell NA NA 2016-09-08 http://fivethirtyeight.com/features/the-rams-are-dead-to-me-so-i-answered-3352-questions-to-find-a-new-team/
nfl_suspensions The NFL’s Uneven History Of Punishing Domestic Violence Suspensions Allison McCann NA NA 2014-08-28 http://fivethirtyeight.com/features/nfl-domestic-violence-policy-suspensions/
nfltix_div_avgprice Who Goes To Meaningless NFL Games And Why? Bar Fights With Walt Walt Hickey NA NA 2014-12-30 http://fivethirtyeight.com/datalab/who-goes-to-meaningless-nfl-games-and-why/
nfltix_jets_buy NA NA NA NA NA NA NA
nfltix_usa_avg Who Goes To Meaningless NFL Games And Why? Bar Fights With Walt Walt Hickey NA NA 2014-12-30 http://fivethirtyeight.com/datalab/who-goes-to-meaningless-nfl-games-and-why/
nflwr_aging_curve The Football Hall Of Fame Has A Receiver Problem Canton Neil Paine NA NA 2014-08-04 http://fivethirtyeight.com/features/the-football-hall-of-fame-has-a-receiver-problem/
nflwr_hist The Football Hall Of Fame Has A Receiver Problem Canton Neil Paine NA NA 2014-08-04 http://fivethirtyeight.com/features/the-football-hall-of-fame-has-a-receiver-problem/
nutrition_pvalues You Can’t Trust What You Read About Nutrition Nutrition Christie Aschwanden NA NA 2016-01-06 http://fivethirtyeight.com/features/you-cant-trust-what-you-read-about-nutrition
nutrition_survey NA NA NA NA NA NA NA
police_deaths The Dallas Shooting Was Among The Deadliest For Police In U.S. History Law Enforcement Carl Bialik Ritchie King Andrew Flowers 2016-07-08 https://fivethirtyeight.com/features/the-dallas-shooting-was-among-the-deadliest-for-police-in-u-s-history/
police_killings Where Police Have Killed Americans In 2015 Police Killings Ben Casselman NA NA 2015-06-03 http://fivethirtyeight.com/features/where-police-have-killed-americans-in-2015
police_locals Most Police Don’t Live In The Cities They Serve Ferguson Nate Silver NA NA 2014-08-20 http://fivethirtyeight.com/datalab/most-police-dont-live-in-the-cities-they-serve/
pres_2016_trail The Last 10 Weeks Of 2016 Campaign Stops In One Handy Gif 2016 Election Matthew Conlen NA NA 2016-12-16 https://fivethirtyeight.com/features/the-last-10-weeks-of-2016-campaign-stops-in-one-handy-gif/
pres_commencement Sitting Presidents Give Way More Commencement Speeches Than They Used To Higher Education Carl Bialik NA NA 2016-05-11 http://fivethirtyeight.com/features/sitting-presidents-give-way-more-commencement-speeches-than-they-used-to/
pulitzer Do Pulitzers Help Newspapers Keep Readers? Awards Nate Silver NA NA 2014-04-15 http://fivethirtyeight.com/datalab/do-pulitzers-help-newspapers-keep-readers/
riddler_castles2 The Battle For Riddler Nation, Round 2 The Riddler Oliver Roeder NA NA 2017-05-19 http://fivethirtyeight.com/features/the-battle-for-riddler-nation-round-2/
riddler_castles Can You Rule Riddler Nation? The Riddler Oliver Roeder NA NA 2017-02-03 https://fivethirtyeight.com/features/can-you-rule-riddler-nation/
san_andreas The Rock Isn’t Alone: Lots Of People Are Worried About ‘The Big One’ Movies Walt Hickey NA NA 2015-06-02 http://fivethirtyeight.com/datalab/the-rock-isnt-alone-lots-of-people-are-worried-about-the-big-one/
senate_polls Early Senate Polls Have Plenty to Tell Us About November 2014 Midterms Harry Enten NA NA 2014-04-16 http://fivethirtyeight.com/features/early-senate-polls-have-plenty-to-tell-us-about-november/
state_payrolls NA NA NA NA NA NA NA
steak_survey How Americans Like Their Steak FiveThirtyAte Walt Hickey NA NA 2014-05-16 http://fivethirtyeight.com/datalab/how-americans-like-their-steak/
tarantino A Complete Catalog Of Every Time Someone Cursed Or Bled Out In A Quentin Tarantino Movie Word Count Oliver Roeder NA NA 2015-12-09 http://fivethirtyeight.com/features/complete-catalog-curses-deaths-quentin-tarantino-films/
tennis_events_time Why Some Tennis Matches Take Forever Tick Tock Carl Bialik NA NA 2015-06-06 http://fivethirtyeight.com/features/why-some-tennis-matches-take-forever/
tennis_players_time Why Some Tennis Matches Take Forever Tick Tock Carl Bialik NA NA 2015-06-06 http://fivethirtyeight.com/features/why-some-tennis-matches-take-forever/
tennis_serve_time Why Some Tennis Matches Take Forever Tick Tock Carl Bialik NA NA 2015-06-06 http://fivethirtyeight.com/features/why-some-tennis-matches-take-forever/
tenth_circuit For A Trump Nominee, Neil Gorsuch’s Record Is Surprisingly Moderate On Immigration Supreme Court Kevin Cope Joshua Fischman NA 2017-03-27 https://fivethirtyeight.com/features/for-a-trump-nominee-neil-gorsuchs-record-is-surprisingly-moderate-on-immigration/?ex_cid=538twitter
trump_news How Trump Hacked The Media 2016 Election Nate Silver NA NA 2016-03-30 http://fivethirtyeight.com/features/how-donald-trump-hacked-the-media/
trump_twitter The World’s Favorite Donald Trump Tweets 2016 Election Leah Libresco NA NA 2016-06-21 https://fivethirtyeight.com/features/the-worlds-favorite-donald-trump-tweets/
unisex_names The Most Common Unisex Names In America: Is Yours One Of Them? NA Andrew Flowers NA NA 2015-06-10 http://fivethirtyeight.com/features/there-are-922-unisex-names-in-america-is-yours-one-of-them/
weather_check Where People Go To Check The Weather Weather Walt Hickey NA NA 2015-04-20 http://fivethirtyeight.com/datalab/weather-forecast-news-app-habits/