{"id":3660,"date":"2020-10-12T14:04:46","date_gmt":"2020-10-12T14:04:46","guid":{"rendered":"https:\/\/data-science.gotoauthority.com\/2020\/10\/12\/how-to-be-a-10x-data-scientist\/"},"modified":"2020-10-12T14:04:46","modified_gmt":"2020-10-12T14:04:46","slug":"how-to-be-a-10x-data-scientist","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2020\/10\/12\/how-to-be-a-10x-data-scientist\/","title":{"rendered":"How to be a 10x data scientist"},"content":{"rendered":"
\n

By Daoud Clarke<\/a>, Co-founder of DataPastry<\/b>.<\/p>\n

<\/p>\n

I\u2019m going to tell you what it takes to be a 10x data scientist. What is a 10x data scientist? Someone who runs ten times as many experiments as the average data scientist.<\/p>\n

Why experiments? Data scientists do other things, too: data munging, analysis, and writing implementations of machine learning algorithms for production. But experiments are what defines a data scientist. That\u2019s where the\u00a0science<\/em>\u00a0is, and it is what distinguishes them from a data analyst or a machine learning engineer.<\/p>\n

So to be a great data scientist, you have to be great at doing experiments.<\/p>\n

Why 10 times more experiments? You can never guarantee you\u2019ll get ten times better results by being cleverer or faster. But you can run more experiments. And the more experiments you run, the more likely you are to get better results, the faster you\u2019ll iterate and the faster you\u2019ll learn.<\/p>\n

Why do you want to be a 10x data scientist? I don\u2019t know. Maybe because it sounds cool. Maybe because it\u2019s fun. Maybe because you\u2019ll have more time to eat pastries. That\u2019s up to you.<\/p>\n

I\u2019m going to assume that you can run experiments correctly. You\u2019re a data scientist, right? Nevertheless, there\u2019s one thing I\u2019ve seen many data scientists get wrong. It\u2019s this:<\/p>\n

\u00a0<\/p>\n

1. Measure your uncertainty<\/h3>\n

\u00a0<\/p>\n

What\u2019s the point in improving 5% over the baseline if you don\u2019t know whether the result is statistically significant? Data scientists know (or\u00a0should<\/em>\u00a0know) statistics, but they are often too lazy to apply it to their own work.<\/p>\n

There is no shortage of options for this. My favourite method is one I learned in my physics degree: estimate the uncertainty as the\u00a0standard error in the mean<\/a>. Of course, this means that the value you report has to be the mean of something, whether it\u2019s the mean F1 score on five folds in cross-validation, or whether it\u2019s the mean precision at 10 over rankings for 1,000 different queries.<\/p>\n

You don\u2019t need to do statistical significance tests between all your results. But you need to have a handle on how uncertain your results are. The standard error in the mean gives you that \u2014 if your results are separated by more than three times the standard error, chances are the difference is significant.<\/p>\n

You probably also want to consider what\u00a0effect size<\/a>\u00a0you\u2019re looking for. If a 0.1% improvement isn\u2019t useful to you, then there\u2019s no point in running experiments that can detect this sort of change.<\/p>\n

\u00a0<\/p>\n

2. Big data is not cool<\/h3>\n

\u00a0<\/p>\n

Big data is slow. You don\u2019t want to be slow. So use small data. Most of the time, you don\u2019t\u00a0need\u00a0big data. If you think you need it, spend a bit of time rethinking to make sure you really do.<\/p>\n

You want your dataset to be big enough such that the uncertainty in your result is\u00a0small<\/em>\u00a0enough to distinguish between differences that you care about. You don\u2019t want it to be any bigger: that\u2019s just a waste of time.<\/p>\n

You don\u2019t have to use all the data you have available. Depending on your experiment, you may be able to estimate how much data you need. Otherwise, look at how the metric you care about varies with training set size. If it levels off fairly quickly, then you\u2019ll know you can get away with discarding a lot of data. Do more experiments to figure out how much data you need to make the uncertainty low enough for the insights you\u2019re looking for.<\/p>\n

The number one cause of slow experiments is using too much data. Just don\u2019t do it.<\/p>\n

\u00a0<\/p>\n

3. Don\u2019t use big data tools<\/h3>\n

\u00a0<\/p>\n

If you have small data, you don\u2019t need big data tools. Don\u2019t use Spark, it will be horribly slow, and the results will be poor compared to something like Pandas and Scikit-learn. Use that instead.<\/p>\n

\u00a0<\/p>\n

4. Use a good IDE<\/h3>\n

\u00a0<\/p>\n

Use a decent integrated development environment like\u00a0PyCharm<\/a>\u00a0\u2014 actually, just use PyCharm, as nothing really compares. Learn how to use it properly.<\/p>\n

These are the things that I find most useful:<\/p>\n