An evaluation methodology to determine the actual limitations of a TinyML-based solution

Abstract

Tiny Machine Learning (TinyML) is an expanding research area based on pushing intelligence to the edge and bringing machine learning techniques to very small devices and embedded systems applications. TinyML reduces energy expenditure, uses low bandwidth communications technologies, and adds more privacy to the developed applications. This work, proposes an evaluation methodology to determine the limitations of a TinyML-based solution starting from creating and preparing the required dataset. Then, the training of the selected machine learning algorithms is detailed, together with the consequent evaluation, and how the experiments must be structured. Four metrics were usedto evaluate the performance of the machine learning algorithms in the various tasks: precision, recall, f1-score, and accuracy. Finally, a comparison ofthe performance of a wide range of machine learning algorithms (i.e., Random Forest, Decision Tree, Support Vector Classifier, Logistic Regression, Gaussian Naive Bayes, and Multi-Layer Perceptron) is presented.