Doing Data Science

I’ve been reading an excellent book by Cathy O’Neil and Rachel Schutt. “Doing Data Science: straight talk from the frontline” (O’Reilly, 2013). The book contains many practical examples of data science, in many instances accompanied by R code. The technical detail is valuable but it is the balance and insight of the commentary that really struck me. A few of the things that resonated for me:

  • data science is special and distinct from statistics because a data product (e.g., a recommendation system) “gets incorporated back into the real world, and users interact with that product, and that generates more data, which creates a feedback loop.” (p. 42). In other words, data models can be strongly ‘performative’.
  • drawing on Josh Wills’ definition: “data scientist (noun): Person who is better at statistics than any software engineer and better at software engineering than any statistician”
  • data science isn’t just about models and algorithms, “They spend a lot more time trying to get data into shape than anyone cares to admit maybe up to 90% of their time” (p. 351). Getting data into shape involves data acquisition, data structuring, data cleaning (outliers, missing values, etc.), transformations. Many statistics courses start with fairly clean and well-structured data, spending 90% of the time on models and only 10% on herding data.
  • data scientists need to be able to ask questions, be able to say “I don’t know” and not be blinded by money: “They seek out opportunities to solve problems of social value and they try to consider the consequences of their models” (p. 355).
This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s