Learn how to tune machine-learning models systematically—from grid and random search to Bayesian optimization, Optuna, successive halving, and Hyperband.
This repository contains the practical notebooks for the Master Hyperparameter Optimization for Tabular Learning course. The examples focus on classification and tabular machine learning using scikit-learn, XGBoost, LightGBM, and Optuna.
Released: August 2026
Status: Actively maintained
- Select appropriate evaluation metrics and cross-validation strategies.
- Build reproducible manual, grid, and random searches.
- Understand Gaussian-process, SMAC, and TPE optimization.
- Accelerate expensive searches with pruning and multi-fidelity methods.
- Optimize scikit-learn, XGBoost, and LightGBM models with Optuna.
- Compare algorithms and hyperparameters in a single CASH workflow.
-
Hyperparameter optimization overview
- Response surfaces
- Low effective dimensionality
-
- Classification metrics: accuracy, precision, recall, ROC AUC, and more
- Regression metrics: MSE, RMSE, R², and more
- Custom metrics and optimization objectives
-
- K-fold, leave-one-out, leave-p-out, and stratified cross-validation
- Grouped and time-series cross-validation
- Nested cross-validation
-
- Manual search
- Grid search
- Random search
-
Bayesian optimization
- Gaussian processes
- Random forests and gradient-boosted models with SMAC
- Tree-structured Parzen estimators (TPE)
-
- Successive halving
- Resource allocation and early stopping
-
- Framework-agnostic search
- Search evaluation and visualization
- Successive halving and Hyperband
- Combined algorithm selection and hyperparameter optimization (CASH)
- SMAC through OptunaHub
Clone the repository, create a dedicated Python environment, and install the course dependencies:
pip install -r requirements.txtThen start Jupyter and open the notebooks in numerical order.
For lectures, explanations, and the complete learning path, visit the online course.