Awesome AutoDL
A curated list of automated deep learning (including neural architecture search and hyper-parameter optimization) resources.
Under MIT License
By D-X-Y
A curated list of automated deep learning (including neural architecture search and hyper-parameter optimization) resources.
Under MIT License
By D-X-Y
Awesome AutoDL
A curated list of automated deep learning related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, and awesome-architecture-search.
Please feel free to pull requests or open an issue to add papers.
Awesome Blogs
Awesome AutoDL Libraies
Awesome Benchmarks
Deep Learning-based NAS and HPO
arXiv
Awesome Surveys
| Title | Venue | Code |
|:--------|:--------:|:--------:|
| NAS-Bench-101: Towards Reproducible Neural Architecture Search | ICML 2019 | GitHub |
| NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search | ICLR 2020 | Github |
| NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search | arXiv 2020 | GitHub |
| NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search | ICLR 2020 | GitHub |
| NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size | TPAMI 2021 | GitHub
| NAS-Bench-ASR: Reproducible Neural Architecture Search for Speech Recognition | ICLR 2021 | - |
| HW-NAS-Bench: Hardware-Aware Neural Architecture Search Benchmark | ICLR 2021 | |
| NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processing | arXiv 2020 | GitHub |
| Type | G | RL | EA | PD | Other |
|:------------|:--------------:|:----------------------:|:-----------------------:|:----------------------:|:----------:|
| Explanation | gradient-based | reinforcement learning | evolutionary algorithm | performance prediction | other types |
| Title | Venue | Type | Code |
|:--------|:--------:|:--------:|:--------:|
| Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator | CVPR | G | Github |
| Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition | ICCV | EA | Github |
| AutoFormer: Searching Transformers for Visual Recognition |ICCV | EA | GitHub
| LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search | CVPR | EA | GitHub |
| One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking | CVPR | EA | GitHub |
| DARTS-: Robustly Stepping out of Performance Collapse Without Indicators | ICLR | G | |
| Zero-Cost Proxies for Lightweight NAS | ICLR | O | |
| Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective | ICLR | - | GitHub |
| DrNAS: Dirichlet Neural Architecture Search | ICLR | G | GitHub |
| Rethinking Architecture Selection in Differentiable NAS | ICLR | O | |
| Evolving Reinforcement Learning Algorithms | ICLR | EA | |
| Title | Venue | Type | Code |
|:--------|:--------:|:--------:|:--------:|
| Designing Network Design Spaces | CVPR | - | GitHub |
| Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search | NeurIPS | - | GitHub |
| PyGlove: Symbolic Programming for Automated Machine Learning | NeurIPS | library | - |
| Does Unsupervised Architecture Representation Learning Help Neural Architecture Search | NeurIPS | PD | GitHub |
| RandAugment: Practical Automated Data Augmentation with a Reduced Search Space | NeurIPS | | GitHub |
| Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians | NeurIPS | G | GitHub |
| A Study on Encodings for Neural Architecture Search | NeurIPS | | GitHub |
| AutoBSS: An Efficient Algorithm for Block Stacking Style Search | NeurIPS | | |
| Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONAS | NeurIPS | G | GitHub |
| Interstellar: Searching Recurrent Architecture for Knowledge Graph Embedding | NeurIPS | | |
| Revisiting Parameter Sharing for Automatic Neural Channel Number Search | NeurIPS | | |
| Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search | NeurIPS | MCTS | GitHub |
| Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree Search | AAAI | MCTS | GitHub |
| Are Labels Necessary for Neural Architecture Search? | ECCV | G | - |
| Single Path One-Shot Neural Architecture Search with Uniform Sampling | ECCV | EA | - |
| Neural Predictor for Neural Architecture Search | ECCV | O | - |
| BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models | ECCV | G | - |
| BATS: Binary ArchitecTure Search | ECCV | - | - |
| AttentionNAS: Spatiotemporal Attention Cell Search for Video Classification | ECCV | - | - |
| Search What You Want: Barrier Panelty NAS for Mixed Precision Quantization | ECCV | - | - |
| Angle-based Search Space Shrinking for Neural Architecture Search | ECCV | - | - |
| Anti-Bandit Neural Architecture Search for Model Defense | ECCV | - | - |
| TF-NAS: Rethinking Three Search Freedoms of Latency-Constrained Differentiable Neural Architecture Search | ECCV | G | GitHub |
| Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search | ECCV | G | GitHub |
| Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search | ECCV | RL | - |
| DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search | ECCV | G | - |
| Optimizing Millions of Hyperparameters by Implicit Differentiation | AISTATS | G | - |
| Evolving Machine Learning Algorithms From Scratch | ICML | EA | - |
| Stabilizing Differentiable Architecture Search via Perturbation-based Regularization | ICML | G | GitHub |
| NADS: Neural Architecture Distribution Search for Uncertainty Awareness | ICML | - | - |
| Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data | ICML | - | - |
| Neural Architecture Search in a Proxy Validation Loss Landscape | ICML | - | - |
| UNAS: Differentiable Architecture Search Meets Reinforcement Learning | CVPR | - | GitHub |
| MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation | CVPR | G | GitHub |
| A Semi-Supervised Assessor of Neural Architectures | CVPR | PD | - |
| Binarizing MobileNet via Evolution-based Searching | CVPR | EA | - |
| Rethinking Performance Estimation in Neural Architecture Search | CVPR | - | GitHub |
| APQ: Joint Search for Network Architecture, Pruning and Quantization Policy | CVPR | G | GitHub |
| SGAS: Sequential Greedy Architecture Search | CVPR | G | Github |
| Can Weight Sharing Outperform Random Architecture Search? An Investigation With TuNAS | CVPR | RL | - |
| FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions | CVPR | G | Github |
| AdversarialNAS: Adversarial Neural Architecture Search for GANs | CVPR | G | Github |
| When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks | CVPR | G | Github |
| Block-wisely Supervised Neural Architecture Search with Knowledge Distillation | CVPR | G | Github |
| Overcoming Multi-Model Forgetting in One-Shot NAS with Diversity Maximization | CVPR | G | Github |
| Densely Connected Search Space for More Flexible Neural Architecture Search | CVPR | G | Github |
| EfficientDet: Scalable and Efficient Object Detection | CVPR | RL | - |
| NAS-BENCH-201: Extending the Scope of Reproducible Neural Architecture Search | ICLR | - | Github |
| Understanding Architectures Learnt by Cell-based Neural Architecture Search | ICLR | G | GitHub |
| Evaluating The Search Phase of Neural Architecture Search | ICLR | - | |
| AtomNAS: Fine-Grained End-to-End Neural Architecture Search | ICLR | | GitHub |
| Fast Neural Network Adaptation via Parameter Remapping and Architecture Search | ICLR | - | GitHub |
| Once for All: Train One Network and Specialize it for Efficient Deployment | ICLR | G | GitHub |
| Efficient Transformer for Mobile Applications | ICLR | - | - |
| PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search | ICLR | G | GitHub |
| Adversarial AutoAugment | ICLR | - | - |
| NAS evaluation is frustratingly hard | ICLR | - | GitHub |
| FasterSeg: Searching for Faster Real-time Semantic Segmentation | ICLR | G | GitHub |
| Computation Reallocation for Object Detection | ICLR | - | - |
| Towards Fast Adaptation of Neural Architectures with Meta Learning | ICLR | - | - |
| AssembleNet: Searching for Multi-Stream Neural Connectivity in Video Architectures | ICLR | EA | - |
| How to Own the NAS in Your Spare Time | ICLR | - | - |
| Title | Venue | Type | Code |
|:--------|:--------:|:--------:|:--------:|
| Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions | ICLR | - | - |
| DATA: Differentiable ArchiTecture Approximation | NeurIPS | - | - |
| Random Search and Reproducibility for Neural Architecture Search | UAI | G | GitHub |
| Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition | EMNLP | G | - |
| Continual and Multi-Task Architecture Search | ACL | RL | - |
| Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation | ICCV | - | - |
| Multinomial Distribution Learning for Effective Neural Architecture Search | ICCV | - | - |
| Searching for MobileNetV3 | ICCV | EA | - |
| Multinomial Distribution Learning for Effective Neural Architecture Search | ICCV | - | GitHub |
| Fast and Practical Neural Architecture Search | ICCV | | |
| Teacher Guided Architecture Search | ICCV | | - |
| AutoDispNet: Improving Disparity Estimation With AutoML | ICCV | G | - |
| Resource Constrained Neural Network Architecture Search: Will a Submodularity Assumption Help? | ICCV | EA | - |
| One-Shot Neural Architecture Search via Self-Evaluated Template Network | ICCV | G | Github |
| Evolving Space-Time Neural Architectures for Videos | ICCV | EA | GitHub |
| AutoGAN: Neural Architecture Search for Generative Adversarial Networks | ICCV | RL | github |
| Discovering Neural Wirings | NeurIPS | G | Github |
| Towards modular and programmable architecture search | NeurIPS | Other | Github |
| Network Pruning via Transformable Architecture Search | NeurIPS | G | Github |
| Deep Active Learning with a NeuralArchitecture Search | NeurIPS | - | - |
| DetNAS: Backbone Search for ObjectDetection | NeurIPS | - | - |
| SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers | NeurIPS | - | - |
| Efficient Forward Architecture Search | NeurIPS | G | Github |
| Efficient Neural ArchitectureTransformation Search in Channel-Level for Object Detection | NeurIPS | G | - |
| XNAS: Neural Architecture Search with Expert Advice | NeurIPS | G | - |
| DARTS: Differentiable Architecture Search | ICLR | G | github |
| ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware | ICLR | RL/G | github |
| Graph HyperNetworks for Neural Architecture Search | ICLR | G | - |
| Learnable Embedding Space for Efficient Neural Architecture Compression | ICLR | Other | github |
| Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution | ICLR | EA | - |
| SNAS: stochastic neural architecture search | ICLR | G | - |
| NetTailor: Tuning the Architecture, Not Just the Weights | CVPR | G | Github |
| Searching for A Robust Neural Architecture in Four GPU Hours | CVPR | G | Github |
| ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation | CVPR | - | - |
| Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search | CVPR | EA | github |
| FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search | CVPR | G | - |
| RENAS: Reinforced Evolutionary Neural Architecture Search | CVPR | G | - |
| Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation | CVPR | G | GitHub |
| MnasNet: Platform-Aware Neural Architecture Search for Mobile | CVPR | RL | Github |
| MFAS: Multimodal Fusion Architecture Search | CVPR | EA | - |
| A Neurobiological Evaluation Metric for Neural Network Model Search | CVPR | Other | - |
| Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells | CVPR | RL | - |
| Customizable Architecture Search for Semantic Segmentation | CVPR | - | - |
| Regularized Evolution for Image Classifier Architecture Search | AAAI | EA | - |
| AutoAugment: Learning Augmentation Policies from Data | CVPR | RL | - |
| Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules | ICML | EA | - |
| The Evolved Transformer | ICML | EA | Github |
| EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks | ICML | RL | - |
| NAS-Bench-101: Towards Reproducible Neural Architecture Search | ICML | Other | Github |
| On Network Design Spaces for Visual Recognition | ICCV | G | Github |
| Title | Venue | Type | Code |
|:--------|:--------:|:--------:|:--------:|
| Towards Automatically-Tuned Deep Neural Networks | BOOK | - | GitHub |
| NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications | ECCV | - | github |
| Efficient Architecture Search by Network Transformation | AAAI | RL | github |
| Learning Transferable Architectures for Scalable Image Recognition | CVPR | RL | github |
| N2N learning: Network to Network Compression via Policy Gradient Reinforcement Learning | ICLR | RL | - |
| A Flexible Approach to Automated RNN Architecture Generation | ICLR | RL/PD | - |
| Practical Block-wise Neural Network Architecture Generation | CVPR | RL | - | Efficient Neural Architecture Search via Parameter Sharing | ICML | RL | github |
| Path-Level Network Transformation for Efficient Architecture Search | ICML | RL | github |
| Hierarchical Representations for Efficient Architecture Search | ICLR | EA | - |
| Understanding and Simplifying One-Shot Architecture Search | ICML | G | - |
| SMASH: One-Shot Model Architecture Search through HyperNetworks | ICLR | G | github |
| Neural Architecture Optimization | NeurIPS | G | github |
| Searching for efficient multi-scale architectures for dense image prediction | NeurIPS | Other | - |
| Progressive Neural Architecture Search | ECCV | PD | github |
| Neural Architecture Search with Bayesian Optimisation and Optimal Transport | NeurIPS | Other | github |
| Differentiable Neural Network Architecture Search | ICLR-W | G | - |
| Accelerating Neural Architecture Search using Performance Prediction | ICLR-W | PD | - |
| Title | Venue | Type | Code |
|:--------|:--------:|:--------:|:--------:|
| Neural Architecture Search with Reinforcement Learning | ICLR | RL | - |
| Designing Neural Network Architectures using Reinforcement Learning | ICLR | RL | - | github |
| Neural Optimizer Search with Reinforcement Learning | ICML | RL | - | Large-Scale Evolution of Image Classifiers | ICML | EA | - |
| Learning Curve Prediction with Bayesian Neural Networks | ICLR | PD | - |
| Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization | ICLR | PD | - |
| Hyperparameter Optimization: A Spectral Approach | NeurIPS-W | Other | github |
| Learning to Compose Domain-Specific Transformations for Data Augmentation | NeurIPS | - | - |
| Title | Venue | Type | Code |
|:--------|:--------:|:--------:|:--------:|
| Speeding up Automatic Hyperparameter Optimization of Deep Neural Networksby Extrapolation of Learning Curves | IJCAI | PD | github |
| Title | Date | Type | Code |
|:--------|:--------:|:--------:|:--------:|
| AutoHAS: Differentiable Hyper-parameter and Architecture Search | 2020.06 | G | - |
| FBNetV3: Joint Architecture-Recipe Search using Neural Acquisition Function | 2020.06 | PD | - |
| Population Based Training of Neural Networks | 2017.11 | EA | - |
| NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search | 2018.10 | EA | - |
| Training Frankenstein’s Creature to Stack: HyperTree Architecture Search | 2018.10 | G | - |
| Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search | 2019.01 | G | github |
| Title | Venue | Year | Code |
|:--------|:--------:|:--------:|:--------:|
| Automated Machine Learning | Springer Book | 2019 | - |
| Neural architecture search: A survey | JMLR | 2019 | - |
| AutonoML: Towards an Integrated Framework for Autonomous Machine Learning | arXiv | 2020 | - |
| Taking human out of learning applications: A survey on automated machine learning | arXiv | 2018 | - |
| AutoML: A Survey of the State-of-the-Art | arXiv | 2019 | - |
| A Survey on Neural Architecture Search | arXiv | 2019 | - |
| A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions | ACM Computing Surveys | 2021 | - |
| On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice | Neurocomputing | 2020 |github |