AwesomeAutoMLPapers
AwesomeAutoMLPapers is a curated list of automated machine learning papers, articles, tutorials, slides and projects. Star this repository, and then you can keep abreast of the latest developments of this booming research field. Thanks to all the people who made contributions to this project. Join us and you are welcome to be a contributor.
What is AutoML?
Automated Machine Learning (AutoML) provides methods and processes to make Machine Learning available for nonMachine Learning experts, to improve efficiency of Machine Learning and to accelerate research on Machine Learning.
Machine Learning (ML) has achieved considerable successes in recent years and an evergrowing number of disciplines rely on it. However, this success crucially relies on human machine learning experts to perform the following tasks:
+ Preprocess the data,
+ Select appropriate features,
+ Select an appropriate model family,
+ Optimize model hyperparameters,
+ Postprocess machine learning models,
+ Critically analyze the results obtained.
As the complexity of these tasks is often beyond nonMLexperts, the rapid growth of machine learning applications has created a demand for offtheshelf machine learning methods that can be used easily and without expert knowledge. We call the resulting research area that targets progressive automation of machine learning AutoML. As a new subarea in machine learning, AutoML has got more attention not only in machine learning but also in computer vision, natural language processing and graph computing.
There are no formal definition of AutoML. From the descriptions of most papers，the basic procedure of AutoML can be shown as the following.
AutoML approaches are already mature enough to rival and sometimes even outperform human machine learning experts. Put simply, AutoML can lead to improved performance while saving substantial amounts of time and money, as machine learning experts are both hard to find and expensive. As a result, commercial interest in AutoML has grown dramatically in recent years, and several major tech companies and startup companies are now developing their own AutoML systems. An overview comparison of some of them can be summarized to the following table.
 Company  AutoFE  HPO  NAS 
 ::  ::  ::  :: 
 4paradigm  √  √  × 
 Alibaba  ×  √  × 
 Baidu  ×  ×  √ 
 Determined AI  ×  √  √ 
 Google  √  √  √ 
 DataCanvas  √  √  √ 
 H2O.ai  √  √  × 
 Microsoft  ×  √  √ 
 MLJAR  √  √  √ 
 RapidMiner  √  √  × 
 Tencent  ×  √  × 
AwesomeAutoMLPapers includes very uptodate overviews of the breadandbutter techniques we need in AutoML:
+ Automated Data Clean (Auto Clean)
+ Automated Feature Engineering (Auto FE)
+ Hyperparameter Optimization (HPO)
+ MetaLearning
+ Neural Architecture Search (NAS)
Table of Contents
 Papers
 Surveys
 Automated Feature Engineering
 Expand Reduce
 Hierarchical Organization of Transformations
 Meta Learning
 Reinforcement Learning
 Architecture Search
 Evolutionary Algorithms
 Local Search
 Meta Learning
 Reinforcement Learning
 Transfer Learning
 Network Morphism
 Continuous Optimization
 Hyperparameter Optimization
 Bayesian Optimization
 Evolutionary Algorithms
 Lipschitz Functions
 Local Search
 Meta Learning
 Particle Swarm Optimization
 Random Search
 Transfer Learning
 Performance Prediction
 Frameworks
 Miscellaneous
 Tutorials
 Bayesian Optimization
 Meta Learning
 Articles
 Bayesian Optimization
 Meta Learning
 Slides
 Bayesian Optimization
 Books
 Meta Learning
 Projects
 Prominent Researchers
Papers
Surveys
 2019  AutoML: A Survey of the StateoftheArt  Xin He, et al.  arXiv 
PDF
 2019  Survey on Automated Machine Learning  Marc Zoeller, Marco F. Huber  arXiv 
PDF
 2019  Automated Machine Learning: StateofTheArt and Open Challenges  Radwa Elshawi, et al.  arXiv 
PDF
 2018  Taking Human out of Learning Applications: A Survey on Automated Machine Learning  Quanming Yao, et al.  arXiv 
PDF
 2020  On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice  Li Yang, et al.  Neurocomputing 
PDF
 2020  Automated Machine Learninga brief review at the end of the early years  Escalante, H. J.  arXiv 
PDF
Automated Feature Engineering
Expand Reduce
 2017  AutoLearn — Automated Feature Generation and Selection  Ambika Kaul, et al.  ICDM 
PDF
 2017  One button machine for automating feature engineering in relational databases  Hoang Thanh Lam, et al.  arXiv 
PDF
 2016  Automating Feature Engineering  Udayan Khurana, et al.  NIPS 
PDF
 2016  ExploreKit: Automatic Feature Generation and Selection  Gilad Katz, et al.  ICDM 
PDF
 2015  Deep Feature Synthesis: Towards Automating Data Science Endeavors  James Max Kanter, Kalyan Veeramachaneni  DSAA 
PDF
Hierarchical Organization of Transformations
 2016  Cognito: Automated Feature Engineering for Supervised Learning  Udayan Khurana, et al.  ICDMW 
PDF
Meta Learning
 2020  AutoML Pipeline Selection: Efficiently Navigating the Combinatorial Space  Chengrun Yang, et al.  KDD 
PDF
 2017  Learning Feature Engineering for Classification  Fatemeh Nargesian, et al.  IJCAI 
PDF
Reinforcement Learning
 2017  Feature Engineering for Predictive Modeling using Reinforcement Learning  Udayan Khurana, et al.  arXiv 
PDF
 2010  Feature Selection as a OnePlayer Game  Romaric Gaudel, Michele Sebag  ICML 
PDF
Architecture Search
Evolutionary Algorithms
 2019  Evolutionary Neural AutoML for Deep Learning  Jason Liang, et al.  GECCO 
PDF
 2017  LargeScale Evolution of Image Classifiers  Esteban Real, et al.  PMLR 
PDF
 2002  Evolving Neural Networks through Augmenting Topologies  Kenneth O.Stanley, Risto Miikkulainen  Evolutionary Computation 
PDF
Local Search
 2017  Simple and Efficient Architecture Search for Convolutional Neural Networks  Thomoas Elsken, et al.  ICLR 
PDF
Meta Learning
 2016  Learning to Optimize  Ke Li, Jitendra Malik  arXiv 
PDF
Reinforcement Learning
 2018  AMC: AutoML for Model Compression and Acceleration on Mobile Devices  Yihui He, et al.  ECCV 
PDF
 2018  Efficient Neural Architecture Search via Parameter Sharing  Hieu Pham, et al.  arXiv 
PDF
 2017  Neural Architecture Search with Reinforcement Learning  Barret Zoph, Quoc V. Le  ICLR 
PDF
Transfer Learning
 2017  Learning Transferable Architectures for Scalable Image Recognition  Barret Zoph, et al.  arXiv 
PDF
Network Morphism
 2019  AutoKeras: An Efficient Neural Architecture Search System  Haifeng Jin, et al.  KDD 
PDF
Continuous Optimization
 2018  Neural Architecture Optimization  Renqian Luo, et al.  arXiv 
PDF
 2019  DARTS: Differentiable Architecture Search  Hanxiao Liu, et al.  ICLR 
PDF
Frameworks
 2019  Auptimizer  an Extensible, OpenSource Framework for Hyperparameter Tuning  Jiayi Liu, et al.  IEEE Big Data 
PDF
 2019  Towards modular and programmable architecture search  Renato Negrinho, et al.  NeurIPS 
PDF
 2019  Evolutionary Neural AutoML for Deep Learning  Jason Liang, et al.  arXiv 
PDF
 2017  ATM: A Distributed, Collaborative, Scalable System for Automated Machine Learning  T. Swearingen, et al.  IEEE 
PDF
 2017  Google Vizier: A Service for BlackBox Optimization  Daniel Golovin, et al.  KDD 
PDF
 2015  AutoCompete: A Framework for Machine Learning Competitions  Abhishek Thakur, et al.  ICML 
PDF
Hyperparameter Optimization
Bayesian Optimization
 2020  Bayesian Optimization of Risk Measures  NeurIPS 
PDF
 2020  BOTORCH: A Framework for Efficient MonteCarlo Bayesian Optimization  NeurIPS 
PDF
 2020  Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly  JMLR 
PDF
 2019  Bayesian Optimization with Unknown Search Space  NeurIPS 
PDF
 2019  Constrained Bayesian optimization with noisy experiments 
PDF
 2019  Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning  NeurIPS 
PDF
 2019  Practical TwoStep Lookahead Bayesian Optimization  NeurIPS 
PDF
 2019  Predictive entropy search for multiobjective bayesian optimization with constraints 
PDF
 2018  BOCK: Bayesian optimization with cylindrical kernels  ICML 
PDF
 2018  Efficient High Dimensional Bayesian Optimization with Additivity and Quadrature Fourier Features  Mojmír Mutný, et al.  NeurIPS 
PDF
 2018  HighDimensional Bayesian Optimization via Additive Models with Overlapping Groups.  PMLR 
PDF
 2018  Maximizing acquisition functions for Bayesian optimization  NeurIPS 
PDF
 2018  Scalable hyperparameter transfer learning  NeurIPS 
PDF
 2016  Bayesian Optimization with Robust Bayesian Neural Networks  Jost Tobias Springenberg， et al.  NIPS 
PDF
 2016  Scalable Hyperparameter Optimization with Products of Gaussian Process Experts  Nicolas Schilling, et al.  PKDD 
PDF
 2016  Taking the Human Out of the Loop: A Review of Bayesian Optimization  Bobak Shahriari, et al.  IEEE 
PDF
 2016  Towards AutomaticallyTuned Neural Networks  Hector Mendoza, et al.  JMLR 
PDF
 2016  TwoStage Transfer Surrogate Model for Automatic Hyperparameter Optimization  Martin Wistuba, et al.  PKDD 
PDF
 2015  Efficient and Robust Automated Machine Learning 
PDF
 2015  Hyperparameter Optimization with Factorized Multilayer Perceptrons  Nicolas Schilling, et al.  PKDD 
PDF
 2015  Hyperparameter Search Space Pruning  A New Component for Sequential ModelBased Hyperparameter Optimization  Martin Wistua, et al. 
PDF
 2015  Joint Model Choice and Hyperparameter Optimization with Factorized Multilayer Perceptrons  Nicolas Schilling, et al.  ICTAI 
PDF
 2015  Learning Hyperparameter Optimization Initializations  Martin Wistuba, et al.  DSAA 
PDF
 2015  Scalable Bayesian optimization using deep neural networks  Jasper Snoek, et al.  ACM 
PDF
 2015  Sequential Modelfree Hyperparameter Tuning  Martin Wistuba, et al.  ICDM 
PDF
 2013  AutoWEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms 
PDF
 2013  Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures  J. Bergstra  JMLR 
PDF
 2012  Practical Bayesian Optimization of Machine Learning Algorithms 
PDF
 2011  Sequential ModelBased Optimization for General Algorithm Configuration(extended version) 
PDF
Evolutionary Algorithms
 2020  DeltaSTN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians  Juhan Bae, Roger Grosse  Neurips 
PDF
 2018  Autostacker: A Compositional Evolutionary Learning System  Boyuan Chen, et al.  arXiv 
PDF
 2017  LargeScale Evolution of Image Classifiers  Esteban Real, et al.  PMLR 
PDF
 2016  Automating biomedical data science through treebased pipeline optimization  Randal S. Olson, et al.  ECAL 
PDF
 2016  Evaluation of a treebased pipeline optimization tool for automating data science  Randal S. Olson, et al.  GECCO 
PDF
Lipschitz Functions
 2017  Global Optimization of Lipschitz functions  C´edric Malherbe, Nicolas Vayatis  arXiv 
PDF
Local Search
 2009  ParamILS: An Automatic Algorithm Configuration Framework  Frank Hutter, et al.  JAIR 
PDF
Meta Learning
 2019  OBOE: Collaborative Filtering for AutoML Model Selection  Chengrun Yang, et al.  KDD 
PDF
 2019  SMARTML: A Meta LearningBased Framework for Automated Selection and Hyperparameter Tuning for Machine Learning Algorithms 
PDF
 2008  CrossDisciplinary Perspectives on MetaLearning for Algorithm Selection 
PDF
Particle Swarm Optimization
 2017  Particle Swarm Optimization for Hyperparameter Selection in Deep Neural Networks  Pablo Ribalta Lorenzo, et al.  GECCO 
PDF
 2008  Particle Swarm Optimization for Parameter Determination and Feature Selection of Support Vector Machines  ShihWei Lin, et al.  Expert Systems with Applications 
PDF
Random Search
 2016  Hyperband: A Novel BanditBased Approach to Hyperparameter Optimization  Lisha Li, et al.  arXiv 
PDF
 2012  Random Search for HyperParameter Optimization  James Bergstra, Yoshua Bengio  JMLR 
PDF
 2011  Algorithms for Hyperparameter Optimization  James Bergstra, et al.  NIPS 
PDF
Transfer Learning
 2016  Efficient Transfer Learning Method for Automatic Hyperparameter Tuning  Dani Yogatama, Gideon Mann  JMLR 
PDF
 2016  Flexible Transfer Learning Framework for Bayesian Optimisation  Tinu Theckel Joy, et al.  PAKDD 
PDF
 2016  Hyperparameter Optimization Machines  Martin Wistuba, et al.  DSAA 
PDF
 2013  Collaborative Hyperparameter Tuning  R´emi Bardenet, et al.  ICML 
PDF
Miscellaneous
 2020  Automated Machine Learning Techniques for Data Streams  AlexandruIonut Imbrea 
PDF
 2018  Accelerating Neural Architecture Search using Performance Prediction  Bowen Baker, et al.  ICLR 
PDF
 2017  Automatic Frankensteining: Creating Complex Ensembles Autonomously  Martin Wistuba, et al.  SIAM 
PDF
 2018  Characterizing classification datasets: A study of metafeatures for metalearning  Rivolli, Adriano, et al.  arXiv 
PDF
 2020  Putting the Human Back in the AutoML Loop  Xanthopoulos, Iordanis, et al.  EDBT/ICDT 
PDF
Tutorials
Bayesian Optimization
 2018  A Tutorial on Bayesian Optimization. 
PDF
 2010  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning 
PDF
Meta Learning
 2008  Metalearning  A Tutorial 
PDF
Blog
 Type  Blog Title  Link 
 ::  ::  :: 
 HPO  Bayesian Optimization for Hyperparameter Tuning  Link

 MetaLearning  Learning to learn  Link

 MetaLearning  Why Metalearning is Crucial for Further Advances of Artificial Intelligence?  Link

Books
 Year of Publication  Type  Book Title  Authors  Publisher  Link 
 ::  ::  ::  ::  ::  :: 
 2009  MetaLearning  Metalearning  Applications to Data Mining  Brazdil, P., Giraud Carrier, C., Soares, C., Vilalta, R.  Springer  Download

 2019  HPO, MetaLearning, NAS  AutoML: Methods, Systems, Challenges  Frank Hutter, Lars Kotthoff, Joaquin Vanschoren   Download

 2021  Learning  Automated Machine Learning in Action  Qinquan Song, Haifeng Jin, Xia Hu  Manning Publications  Download

Projects
 Project  Type  Language  License  Link 
 ::  ::  ::  ::  :: 
 AdaNet  NAS  Python  Apache2.0  Github

 Advisor  HPO  Python  Apache2.0  Github

 AMLA  HPO, NAS  Python  Apache2.0  Github

 ATM  HPO  Python  MIT  Github

 Auger  HPO  Python  Commercial  Homepage

 auptimizer  HPO, NAS  Python (support R script)  GPL3.0  Github

 AutoKeras  NAS  Python  License
 Github

 AutoML Vision  NAS  Python  Commercial  Homepage

 AutoML Video Intelligence  NAS  Python  Commercial  Homepage

 AutoML Natural Language  NAS  Python  Commercial  Homepage

 AutoML Translation  NAS  Python  Commercial  Homepage

 AutoML Tables  AutoFE, HPO  Python  Commercial  Homepage

 HyperGBM  HPO Python  Python  Github

 HyperKeras  NAS  Python  Python  Github

 Hypernets  HPO, NAS  Python  Python  Github

 autosklearn  HPO  Python  License
 Github

 auto_ml  HPO  Python  MIT  Github

 BayesianOptimization  HPO  Python  MIT  Github

 BayesOpt  HPO  C++  AGPL3.0  Github

 comet  HPO  Python  Commercial  Homepage

 DataRobot  HPO  Python  Commercial  Homepage

 DEvol  NAS  Python  MIT  Github

 DeepArchitect  NAS  Python  MIT  Github

 Determined  HPO, NAS  Python  Apache2.0  Github

 Driverless AI  AutoFE  Python  Commercial  Homepage

 FARHO  HPO  Python  MIT  Github

 H2O AutoML  HPO  Python, R, Java, Scala  Apache2.0  Github

 HpBandSter  HPO  Python  BSD3Clause  Github

 HyperBand  HPO  Python  License
 Github

 Hyperopt  HPO  Python  License
 Github

 Hyperoptsklearn  HPO  Python  License
 Github

 Hyperparameter Hunter  HPO  Python  MIT  Github

 Katib  HPO  Python  Apache2.0  Github

 MateLabs  HPO  Python  Commercial  Github

 Milano  HPO  Python  Apache2.0  Github

 MLJAR  AutoFE, HPO, NAS  Python  MIT  Github

 mlr3automl  HPO  R  LGPL3.0  GitHub

 nasbot  NAS  Python  MIT  Github

 neptune  HPO  Python  Commercial  Homepage

 NNI  HPO, NAS  Python  MIT  Github

 Oboe  HPO  Python  BSD3Clause  Github

 Optunity  HPO  Python  License
 Github

 R2.ai  HPO   Commercial  Homepage

 RBFOpt  HPO  Python  License
 Github

 RoBO  HPO  Python  BSD3Clause  Github

 ScikitOptimize  HPO  Python  License
 Github

 SigOpt  HPO  Python  Commercial  Homepage

 SMAC3  HPO  Python  License
 Github

 TPOT  AutoFE, HPO  Python  LGPL3.0  Github

 TransmogrifAI  HPO  Scala  BSD3Clause  Github

 Tune  HPO  Python  Apache2.0  Github

 Xcessiv  HPO  Python  Apache2.0  Github

 SmartML  HPO  R  GPL3.0  Github

 MLBox  AutoFE, HPO  Python  BSD3 License  Github

 AutoAI Watson  AutoFE, HPO   Commercial  Homepage

Slides
 Type  Slide Title  Authors  Link 
 ::  ::  ::  :: 
 AutoFE  Automated Feature Engineering for Predictive Modeling  Udyan Khurana, etc al.  Download

 HPO  A Tutorial on Bayesian Optimization for Machine Learning  Ryan P. Adams  Download

 HPO  Bayesian Optimisation  Gilles Louppe  Download

Acknowledgement
Special thanks to everyone who contributed to this project.
 Name  Bio 
 ::  :: 
 Alexander Robles  PhD Student @UNICAMPBrazil 
 derekflint  
 endymecy  Senior Researcher @Tencent 
 Eric  
 Erin LeDell  Chief Machine Learning Scientist @H2O.ai 
 fwcore  
 Gaurav Mittal  
 Hernan Ceferino Vazquez  PhD, Data Science Expert @MercadoLibre 
 Kaustubh Damania  
 Lilian Besson  PhD Student @CentraleSupélec 
 罗磊  
 Marc  
 Mohamed Maher  
 Neil Conway  CTO @Determined AI 
 Richard Liaw  PhD Student @UC Berkeley
 Randy Olson  Lead Data Scientist @LifeEGX 
 Slava Kurilyak  Founder, CEO @Produvia 
 Saket Maheshwary  AI Researcher 
 shaido987  
 sophiawrightblue  
 tengben0905  
 xuehui  @Microsoft 
 Yihui He  Grad Student @CMU 
Contact & Feedback
If you have any suggestions (missing papers, new papers, key researchers or typos), feel free to pull a request. Also you can mail to:
+ Mark Lin ([email protected]).
Licenses
AwesomeAutoMLPapers is available under Apache Licenses 2.0.