2019 · 116 citations · 12 references
ComputingComputer ScienceComputer VisionMachine LearningMutation BreedingDeep LearningNeural NetworksPattern RecognitionSpeech RecognitionSpeech Recognition TestingModel Robustness AnalysisModel Robustness AssessmentModel Robustness TestingMutationMutation TestingRecurrent Neural NetworksRobustnessRuntime Analysis
Deep neural networks (DNNs) are increasingly expanding their real-world applications across domains, e.g., image processing, speech recognition and natural language processing. However, there is still limited tool support for DNN testing in terms of test data quality and model robustness. In this paper, we introduce a mutation testing-based tool for DNNs, DeepMutation++, which facilitates the DNN quality evaluation, supporting both feed-forward neural networks (FNNs) and stateful recurrent neural networks (RNNs). It not only enables to statically analyze the robustness of a DNN model against the input as a whole, but also allows to identify the vulnerable segments of a sequential input (e.g. audio input) by runtime analysis. It is worth noting that DeepMutation++ specially features the support of RNNs mutation testing. The tool demo video can be found on the project website https://sites.google.com/view/deepmutationpp.
12
Kexin Pei, Yinzhi Cao, Junfeng Yang et al. · 2017 · 1.2K citations · Full text
Yuchi Tian, Kexin Pei, Suman Jana et al. · 2018 · 1.2K citations · Full text
DeepMutation: Mutation Testing of Deep Learning Systems
Lei Ma, Fuyuan Zhang, Jiyuan Sun et al. · 2018 · 366 citations
DeepStellar: model-based quantitative analysis of stateful deep learning systems
Xiaoning Du, Xiaofei Xie, Yi Li et al. · 2019 · 177 citations · Full text