Automatic Construction

Feb 15, 2023

| R bloggers February 15, 2023 In this blog, we would like to introduce you to a brand new, reorganized and restructured version of the Forrester R package. package history Responsible ML readers may already be familiar with the package name and possibly wonder why we’re describing it again. The previous version of Forrester was introduced about 1.5 years ago (authors: Anna Kozak, Szymon Szmajczynski, Thien Hoang Li) and was followed by two blogs: ‘Forrester: An AutoML R package for tree-based models’, and ‘ Guide through the jungle of models! What else is there about the Forester R package?’. Unfortunately, due to other responsibilities and new opportunities, the authors were not able to maintain the package, which lead to the point where a refactoring of the tool was required. A new scientific team (Anna Kozak, Hubert Ruzinski, Adriana Grudzien, Petrik Slowakiewicz) overtook the project and built it from scratch, learning from the mistakes of their predecessors. What is a forester? Forester is an AutoML tool in R for tabular data regression and binary classification tasks. It wraps all machine learning processes into a single train() function, which consists of: Presenting a brief data check report, preprocessing an initial dataset sufficient to train the model, training 5 tree-based models (decision tree, random forest, xgboost, catboost, lightgbm) with default parameters, random search and Bayesian optimization, evaluating them and providing a ranked list. However, this is not all that Forester has to offer. Through additional functions, the user can easily interpret the model built with the use of DALEX or generate one of the predefined reports. The main goal of the package is to keep the user interface as simple as possible, so that everyone can benefit from its possibilities. It is specially designed for: Beginners in ML to train your first model and start your modeling career. Researchers from other scientific fields to easily add ML solutions and analysis to their thesis. ML experts are faced with the ability to easily perform dataset analysis, build baseline models, and quickly explore new features. AutoML and Forrester Pipeline To fully understand what the Forrester package provides, we need to provide a brief overview of machine learning (ML) and automated machine learning (AutoML) pipelines. Classical ML pipeline starts with two pre-modelling steps which are task identification and data collection. They are undoubtedly important, however, we will focus on the steps highlighted in green, as they are the heart of the whole process. Machine Learning Pipeline. During the preprocessing phase, data scientists focus on preparing the appropriate data, so that the model can be trained later. Typical operations performed here do not include assigning values, encoding data, or deleting static columns. The feature engineering process involves more advanced methods and aims to select the most important columns from the dataset for model training. This includes for example the removal of highly correlated columns, or selection via lasso or ridge methods for regression functions. The most time consuming step is model training. At this point, the data scientist has to select model engines and manually tune a lot of hyperparameters to get the best results. Finally, comes the post-processing which involves evaluating the models by various metrics and comparing them with each other to choose the best one. As one can see, model training is an iterative process with highly repetitive steps, and is ultimately incredibly time consuming. The best way to combat this is to use an AutoML tool. As shown below, such solutions automate the ML pipeline, so data scientists can tackle the more critical cases. Automated machine learning pipeline. Why tree-based model? Some users may be surprised that all the models used inside the package are from the tree-based family and wonder whether there are any special reasons for doing so. There are definitely and the most prominent are: Tree-based models are extremely popular among winners in Kaggle competitions, which reflects their versatility. They are superior to deep learning neural networks due to their lack of bias towards highly smooth solutions. Augmenting and boosting ensembles address their main drawback, which is overfitting. Tree-based models are easy to understand for users without ML background and have already established good opinion among doctors. For further reading and a more in-depth analysis of the performance of tree-based models we recommend a paper by Leo Grisztajn ‘Why do tree-based models still outperform deep learning on tabular data?’. The visuals below come from the aforementioned publication. Benchmark on a medium sized dataset, with only numerical features. Benchmarks on medium sized datasets, with both numerical and categorical features. Package Structure and User Interface The graph presented below briefly summarizes the processes inside the main train() function and it adds information about additional features of the package. The interpret() function creates an interpretable artificial intelligence (XAI) interpreter from the DALEX package. The save() function allows the user to save the final item, and the report() generates an automatically generated report from the training process. One can also use the data_check() function, which is also present inside the preprocessing step. Forester Package Structure. In the next blog post we will describe in detail all Forrester features and we will outline what makes the package special among other AutoML solutions in R. If you’re interested in other posts about explanatory, fair, and responsible ML, follow #ResponsibleML on Medium. To view more R related content visit https://www.r-bloggers.com Forrester: An R Package for the Automated Construction of Tree-Based Models was originally published in Responsible ML on Medium, where people are continuing the conversation by highlighting and responding to this story. Connected