ACM Transactions on Embedded Computing Systems · 2023 · 23 citations · 118 references
ComputingComputer EngineeringComputer ScienceArtificial IntelligenceEmbedded SystemsMachine LearningMicrocontroller ArchitectureNeural NetworksOptimization For Machine LearningContext AwarenessHardware ResourcesJoint OptimizationNeural Architecture SearchOperatorsOptimization AlgorithmsResource Allocation ConstraintsTiny Machine Learning
Machine learning at the extreme edge has enabled a plethora of intelligent, time-critical, and remote applications. However, deploying interpretable artificial intelligence systems that can perform high-level symbolic reasoning and satisfy the underlying system rules and physics within the tight platform resource constraints is challenging. In this paper, we introduce TinyNS, the first platform-aware neurosymbolic architecture search framework for joint optimization of symbolic and neural operators. TinyNS provides recipes and parsers to automatically write microcontroller code for five types of neurosymbolic models, combining the context awareness and integrity of symbolic techniques with the robustness and performance of machine learning models. TinyNS uses a fast, gradient-free, black-box Bayesian optimizer over discontinuous, conditional, numeric, and categorical search spaces to find the best synergy of symbolic code and neural networks within the hardware resource budget. To guarantee deployability, TinyNS talks to the target hardware during the optimization process. We showcase the utility of TinyNS by deploying microcontroller-class neurosymbolic models through several case studies. In all use cases, TinyNS outperforms purely neural or purely symbolic approaches while guaranteeing execution on real hardware.
118
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Laurens van der Maaten, Geoffrey E. Hinton · Journal of Machine Learning Research · 2008 · 35.7K citations
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Mark Sandler, Andrew Howard, Menglong Zhu et al. · 2018 · 25.4K citations
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Adam Paszke, Sam Gross, Francisco Massa et al. · arXiv (Cornell University) · 2019 · 16.2K citations · Full text
Learning the parts of objects by non-negative matrix factorization
Daniel D. Lee, H. Sebastian Seung · Nature · 1999 · 14.1K citations