{"id":8079,"date":"2021-01-11T17:31:27","date_gmt":"2021-01-11T17:31:27","guid":{"rendered":"https:\/\/wealthrevelation.com\/data-science\/2021\/01\/11\/pycaret-2-2-efficient-pipelines-for-model-development\/"},"modified":"2021-01-11T17:31:27","modified_gmt":"2021-01-11T17:31:27","slug":"pycaret-2-2-efficient-pipelines-for-model-development","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2021\/01\/11\/pycaret-2-2-efficient-pipelines-for-model-development\/","title":{"rendered":"PyCaret 2.2: Efficient Pipelines for Model Development"},"content":{"rendered":"
\n

Data science is an exciting field, but it can be intimidating to get started, especially for those new to coding.\u00a0 Even for experienced developers and data scientists, the process of developing a model could involve stringing together many steps from many packages, in ways that might not be as elegant or efficient as one might like.\u00a0 The creator of the Caret<\/a> library in R (\u201cshort for C<\/strong>lassification A<\/strong>nd RE<\/strong>gression T<\/strong>raining\u201d) was a software engineer named Max Kuhnwho sought to improve the situation by creating a more efficient, \u201cstreamlined\u201d process for developing models<\/a>.\u00a0 Eventually, data scientist Philip Goddard<\/a> switched from R to Python and, to bring the \u201csmooth and intuitive\u201d workflows of Caret with him, he created PyCaret<\/a>.<\/p>\n

\"\"
Image from github.com\/pycaret<\/a><\/figcaption><\/figure>\n

PyCaret is a convenient entree into machine learning and a productivity tool for experienced practitioners.\u00a0 It gives scientists and analysts a simple, concise, low-code interface into many of the most popular and powerful machine learning libraries in the data science ecosystem, making it easy to start exploring new techniques or datasets.\u00a0 The clear, concise code that PyCaret users generate is also easy for collaborators and teammates to read and adapt, whether these colleagues are new to the field or experienced data scientists.\u00a0<\/p>\n

Building Blocks<\/strong><\/p>\n

Goddard initially built out PyCaret as a package that wrapped many useful Python libraries and common tasks into concise repeatable components, making it easy to construct pipelines with the minimal number of statements.\u00a0 Today, PyCaret still utilizes many modules that will be familiar to Pythonistas: Pandas and Numpy for data wrangling, Matplotlib, Plotly and Seaborn for visualization, scikit-learn and XGBoost for modeling, Gensim, Spacy and NLTK for natural language processing, among others.\u00a0<\/p>\n

\"\"
Image from pycaret.org<\/a><\/figcaption><\/figure>\n

Building a Pipeline<\/strong><\/p>\n

While the internals and usage have changed considerably from the first version to PyCaret 2.2, the experience is still rooted in the same goal: simple efficiency for the whole model development lifecycle.\u00a0 This means that you can utilize PyCaret to go from raw data through training, tuning, interpretability analysis, to model selection and experiment logging, all with just a few lines of code.\u00a0<\/p>\n

\"\"
Training and comparing models in just a few lines of code<\/figcaption><\/figure>\n

Let\u2019s walk through each step of building a classification pipeline in PyCaret.\u00a0<\/p>\n

0.\u00a0\u00a0 Installation<\/strong><\/p>\n

You can quickly try out PyCaret in the pre-configured example project here: PyCaret Project on Domino<\/a>.\u00a0 If you\u2019d like to install it in another Domino compute environment or using a Docker image elsewhere, you can add the following RUN statement to add PyCaret 2.2.2 (and dependencies required for this walk-through):<\/p>\n

\nRUN sudo apt-get purge python-numpy -y \n&amp;&amp; sudo apt-get autoremove --purge python-numpy -y \n&amp;&amp; sudo pip uninstall numpy -y \n&amp;&amp; sudo pip install numpy==1.17 &amp;&amp; sudo pip install pycaret==2.2.2 &amp;&amp; sudo pip install shap==0.36.0\n<\/pre>\n
Or you can install with pip (though you may want to do this in a virtual environment if you\u2019re not using Docker): pip install pycaret<\/code><\/strong><\/p>\n
For this example I started with a Domino Analytics Distribution base environment.\u00a0 Depending on your starting environment you may also need to install Pandas and some additional dependencies that ship with Domino Analytics Distributions.\u00a0<\/p>\n
You can verify you have everything installed by importing packages as follows:<\/p>\n
\nimport pycaret\nprint('Using PyCaret Version', pycaret.__version__)\nprint('Path to PyCaret: ', pycaret.__file__)\nimport os\nimport pandas as pd\nfrom pycaret.classification import *\nfrom pycaret import datasets\n<\/pre>\n
Accessing Data<\/strong><\/p>\n
There are two ways to register your data into PyCaret: via the repository or a Pandas dataframe.\u00a0 Let\u2019s take a look at each method.<\/p>\n
Loading a Dataframe with Pandas<\/strong><\/p>\n
The first way to get data into PyCaret is simply to load up a Pandas dataframe and then pass it to PyCaret.\u00a0<\/p>\n
\ndata_path = os.path.join(os.environ['DOMINO_WORKING_DIR'], 'mushrooms.csv')\nprint('Data path: ', data_path)\ndata = pd.read_csv(data_path)\ndata.head()\n<\/pre>\n
\"\"<\/figure>\n
Using the Data Repository<\/strong><\/p>\n
The second way of getting data, which is used in the PyCaret tutorials, is to pull in a curated dataset from the PyCaret Data Repository<\/a>. The repository helpfully includes popular sample datasets for classification, regression, clustering, NLP, etc.\u00a0 (Last I checked, the repository contained 56 datasets, a sample of which are shown here.)\u00a0 You can list all the datasets available in the repository, and see associated metadata:\u00a0<\/p>\n
\nall_datasets = pycaret.datasets.get_data('index')\n<\/pre>\n
\"\"<\/figure>\n
\nall_datasets = pycaret.datasets.get_data('index')\ndataset_name = 'heart_disease' # Replace with your desired dataset.\ndata = pycaret.datasets.get_data(dataset_name)\n<\/pre>\n
\"\"<\/figure>\n
Note<\/strong>: PyCaret 2.2 expects the data to be loaded as a dataframe in memory.\u00a0 If you are using a large dataset, PyCaret may be useful at the exploration and prototyping stage of your project if you can load a sufficient sample of your data into memory for meaningful exploration.\u00a0 If your dataset is both large and wide, such that your number of features would be close to or larger than the number of samples you can load into memory, then preprocessing to reduce the number of features or utilizing other tools would be preferable.\u00a0<\/p>\n
Experiment Setup<\/strong><\/p>\n
Many often-tedious preprocessing steps are taken care of automatically in PyCaret, which standardizes and conveniently packages fundamental data preparation steps into repeatable time-saving workflows.\u00a0 Users are able to automate cleaning (e.g. handling missing values with various imputation methods available), splitting into train and test sets, as well as some aspects of feature engineering and training.\u00a0 While many of the objects created in this process aren\u2019t explicitly shown to the user (such as train and test sets, or label vectors), they are accessible if needed or desired by more experienced practitioners.\u00a0<\/p>\n
\nclf1 = setup(data, \n             target = target_variable_name, # Use your target variable.\n             session_id=123, \n             log_experiment=True, \n             experiment_name='experiment1', # Use any experiment name.\n             silent=True # Runs the command without user input. \n            )\n<\/pre>\n
\"\"<\/figure>\n
Compare Baseline Models<\/strong><\/p>\n
Here is where we begin to see the full power of PyCaret.\u00a0 In a single line of code, we can train and compare baseline versions of all available models on our dataset:<\/p>\n
best_model = compare_models()<\/code><\/strong><\/p>\n
This trains a baseline version of each available model type and yields a detailed comparison of metrics for the trained models, and highlights the best results across models.<\/p>\n
\nbest_model = compare_models()\n<\/pre>\n
Note that we did not have to do any data preparation by hand \u2014 we just needed to make the data available as a CSV, and run the setup function.\u00a0 Behind the scenes of those two setup steps, the data was passed into PyCaret and transformed to the extent necessary to train and evaluate the available models.\u00a0 To see what models PyCaret knows about, we can run<\/p>\n
models()<\/code><\/strong><\/p>\n
which returns a dataframe of all available models, their proper names, the reference package that they\u2019re drawn from (e.g. sklearn.linear_model._logistic.LogisticRegression), and whether Turbo is supported (a mode that limits the model training time, which may be desirable for rapid comparisons).\u00a0<\/p>\n
\nmodels()\n<\/pre>\n
\"\"<\/figure>\n
Train and tune specific models<\/strong><\/p>\n
From compare_models<\/strong>, we were easily able to see the best baseline models for each metric, and select those for further investigation.\u00a0 For example, if we were looking for the model with the highest AUC above, we would have elected to continue with random forest.\u00a0 We can then save and fine tune our model using the create_model<\/strong> and tune_model<\/strong> functions.\u00a0<\/p>\n
\nrf = create_model('rf', fold = 5)\n<\/pre>\n
\"\"<\/figure>\n
\nTuned_rf = tune_model(rf) \n<\/pre>\n
\"\"<\/figure>\n
Combine Models ( Optional )<\/strong><\/p>\n
We can combine our trained models in various ways.\u00a0 First, we can create ensemble models with methods such as bagging (bootstrap aggregating) and boosting.\u00a0 Both bagging and boosting are invoked with the ensemble_model<\/strong> function.\u00a0 We can further apply blending and stacking methods to combine diverse models, or estimators \u2014 a list of estimators can be passed to blend_models<\/strong> or stack_models<\/strong>.\u00a0 If desired, one could create ensemble models and combine them via blending or stacking, all in a single line of code. \u00a0For clarity, we\u2019ll show an example in which each of these four methods is shown sequentially in its own cell, which also allows us to see the default output from PyCaret when each of these methods is used.\u00a0\u00a0<\/p>\n
Creating a bagged decision tree ensemble model:<\/p>\n
\nbagged_dt = ensemble_model(dt) \n<\/pre>\n
\"\"<\/figure>\n
Creating a boosted decision tree ensemble model:<\/p>\n
\nboosted_dt = ensemble_model(dt, method = \u2018Boosting\u2019) \n<\/pre>\n
\"\"<\/figure>\n
Blending estimators:<\/p>\n
\nblender = blend_models(estimator_list = [boosted_dt, bagged_dt, tuned_rf], method = 'soft')\n<\/pre>\n
\"\"<\/figure>\n
Stacking bagged, boosted, and tuned estimators:<\/p>\n
\nstacker = stack_models(estimator_list = [boosted_dt,bagged_dt,tuned_rf], meta_model=rf)\n<\/pre>\n
\"\"<\/figure>\n
AutoML ( Optional )<\/strong><\/p>\n
Quick and painless tuning for a particular metric can be accomplished using the AutoML feature.\u00a0<\/p>\n
\n# Select the best model based on the chosen metric\nbest = automl(optimize = 'AUC')\nbest\n<\/pre>\n
\"\"<\/figure>\n
AutoML techniques generally reduce the human oversight of the model selection process, which may not be ideal or appropriate in many contexts, they can be a useful tool to quickly identify the highest performing option for a particular purpose.\u00a0<\/p>\n
Analyze Models with Plots<\/strong><\/p>\n
Once the preferred model has been selected, whatever the method, its performance can be visualized with built-in plot options.\u00a0 For example, you can simply call plot_model<\/strong> on a random forest model to return overlayed ROC curves for each class:<\/p>\n
\nplot_model(best)\n<\/pre>\n
\"\"<\/figure>\n
Interpret Models with SHAP ( For compatible model types )<\/strong><\/p>\n
Increasingly, having a well-performing model is not enough \u2014 in many industries and applications, the model must also be explainable.\u00a0 Our Chief Data Scientist Josh Poduska has written fantastic overviews of SHAP and other explainability tools: take a look at SHAP and LIME Python Libraries: Part 1 \u2013 Great Explainers, with Pros and Cons to Both<\/a> and SHAP and LIME Python Libraries: Part 2 \u2013 Using SHAP and LIME<\/a>.\u00a0 PyCaret provides seamless integration with SHAP so you can easily add interpretation plots to your model analysis.\u00a0\u00a0<\/p>\n
\ninterpret_model(best)\n<\/pre>\n
\"\"<\/figure>\n
\ninterpret_model(best, plot = 'correlation')\n<\/pre>\n
\"\"<\/figure>\n
\ninterpret_model(best, plot = 'reason', observation = 12)\n<\/pre>\n
Predict<\/strong><\/p>\n
As we\u2019ve come to expect by now, generating predictions on our held-out test data is a cinch:<\/p>\n
\npredict_model(best)\n<\/pre>\n
\"\"<\/figure>\n
Save and Load Model<\/strong><\/p>\n
Once we\u2019re satisfied with our selected model, we can easily save it:<\/p>\n
\nsave_model(best, model_name='best-model')\n<\/pre>\n
\"\"<\/figure>\n
And finally we can load our saved model for use:<\/p>\n
\nloaded_bestmodel = load_model('best-model')\nprint(loaded_bestmodel)\n<\/pre>\n
\"\"<\/figure>\n
If you\u2019d like to see how we can put all of this together into a reproducible and shareable Domino project, please take a look at the reference project below.\u00a0 It includes everything discussed above, as well as a few other examples and resources. Anyone can browse the project and download the code files. You must be logged in (free do to \u2013 sign up here<\/a>) to run the code in Domino.<\/p>\n
We\u2019d love to hear from you about your use cases and experiences with PyCaret.\u00a0 Let us know if it\u2019s been helpful to you and if you\u2019d like to see more projects and pieces about tools in this space.\u00a0 Drop us a note or bring your questions and feedback to Domino Community<\/a> (also free \u2013 register here<\/a>).<\/p>\n
Domino Reference Project<\/strong><\/p>\n
PyCaret Project on Domino<\/a><\/p>\n
The reference project accompanies this post and provides a quick means for you to try our PyCaret and follow along on this walk-through. The project includes a classification pipeline template notebook<\/a> that addresses the above and demonstrates additional functionality.\u00a0<\/p>\n
    <\/div>\n","protected":false},"excerpt":{"rendered":"
https:\/\/blog.dominodatalab.com\/pycaret-2-2-efficient-pipelines-for-model-development\/<\/p>\n","protected":false},"author":0,"featured_media":8080,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[2],"tags":[],"_links":{"self":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/8079"}],"collection":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/comments?post=8079"}],"version-history":[{"count":0,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/8079\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media\/8080"}],"wp:attachment":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media?parent=8079"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/categories?post=8079"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/tags?post=8079"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}