arXiv (Cornell University) · 2019 · 24 citations · 20 references
ComputingComputer ScienceComputer VisionMachine LearningOptimizationDeep LearningGradient-based OptimizationNeural NetworksOptimization For Machine LearningStochastic OptimizationAdaptive Step-size ControlArchitecture SearchInpaintingNeural Architecture SearchOptimization AlgorithmsParameter TuningSearch Space ExplorationStochastic Gradient Descent
High sensitivity of neural architecture search (NAS) methods against their\ninput such as step-size (i.e., learning rate) and search space prevents\npractitioners from applying them out-of-the-box to their own problems, albeit\nits purpose is to automate a part of tuning process. Aiming at a fast, robust,\nand widely-applicable NAS, we develop a generic optimization framework for NAS.\nWe turn a coupled optimization of connection weights and neural architecture\ninto a differentiable optimization by means of stochastic relaxation. It\naccepts arbitrary search space (widely-applicable) and enables to employ a\ngradient-based simultaneous optimization of weights and architecture (fast). We\npropose a stochastic natural gradient method with an adaptive step-size\nmechanism built upon our theoretical investigation (robust). Despite its\nsimplicity and no problem-dependent parameter tuning, our method exhibited near\nstate-of-the-art performances with low computational budgets both on image\nclassification and inpainting tasks.\n
20
Image quality assessment: from error visibility to structural similarity
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh et al. · IEEE Transactions on Image Processing · 2004 · 56K citations
Rethinking the Inception Architecture for Computer Vision
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe et al. · 2016 · 30.9K citations
Xception: Deep Learning with Depthwise Separable Convolutions
François Chollet · 2017 · 19K citations
Automatic differentiation in PyTorch
Adam Paszke, Sam Gross, Soumith Chintala et al. · 2017 · 11.1K citations
Deep Learning Face Attributes in the Wild
Ziwei Liu, Ping Luo, Xiaogang Wang et al. · 2015 · 7.7K citations