QFRG and DSLab seminar [19.04.2021]
19 April 2021
Dear Sir or Madam,
We invite you to the tenth seminar of the monthly series of meetings conducted jointly by QFRG (Quantitative Finance Research Group) and DSLab (Data Science Lab). The meeting will be devoted to the topic: XAI tools as a part of the best practices in model selection for business decision modelling. Example of marketing campaign success forecasting, in which Marcin Chlebus, PhD will show that limiting the criteria for selecting a predictive model only to the prediction quality measures may not allow to make an optimal choice.
QFRG is a place where research is conducted and experiences are exchanged between people engaged in examining occurrences in the world of investment from the perspective of both theory and practice, on the verge of science and business.
The activity of DSLab is focused mainly on academic projects devoted to deepening of the knowledge of DSLab team, sharing it with other people interested in Data Science issues and preparing scientific and didactic publications.
The tenth meeting will take place on April 19th, 2021 (Monday) at 17:00 online on Google Meet platform.
Link to the meeting: https://meet.google.com/odh-akxf-qbz
The presentation is scheduled for about 45 minutes, and after that we invite you to a discussion.
Please log in the latest at 4:50 PM. The presentation will start at 5:00 PM.
Joining a meeting implies consent to recording. Please turn off cameras and microphones during the presentation and send the questions to the speaker in the chat.
Complex machine learning methods are more and more popular in business decisions supporting models. Many empirical examples (in business and academia) show that one-dimensional criteria of model assessment may lead to spectacular failures of the developed models. Using Explainable Artificial Intelligence (XAI) tools to additionally evaluate model’s performance seems to be a crucial part of model assessment process. Permutated Feature Importance (PFI) and Partial Dependency Plots (PDP) become a part of best practices in this field.
Five tree-based models were compared in prediction of the success of telemarketing campaign of Portuguese bank – Random Forest, AdaBoost, GBM, XGBoost and CatBoost. XGBoost, CatBoost and GBM were performing best with respect to AUC (the highest for XGBoost, but not statistically different from the others). Further examining these models with PDP and PFI resulted in the discovery of potential overfitting of XGBoost and GBM and finally choosing CatBoost as the best model.
Thus, using only predictive/discriminatory power as model assessment measures may not lead to the optimal model selection, whereas extending the model assessment by XAI tools may help in the appropriate choice.
Posted by moponowicz