IEEE Signal Processing Magazine · 2009 · 3.1K citations · 66 references
EngineeringSystems EngineeringMachine LearningPerformance MeasurementPsychoacousticsDigital Signal ProcessingSignal ProcessingAlgorithm Performance EvaluationDesign CriteriaMean Squared Error
The authors argue that while MSE remains useful, researchers should consider advanced, easy‑to‑use fidelity measures—especially when perceptual criteria matter—to design and evaluate signal processing algorithms. They suggest comparing new algorithms to existing ones using multiple fidelity criteria to assess performance. They reviewed the strengths and weaknesses of MSE and surveyed emerging alternative fidelity measures and their potential applications across diverse problems.
In this article, we have reviewed the reasons why we (collectively) want to love or leave the venerable (but perhaps hoary) MSE. We have also reviewed emerging alternative signal fidelity measures and discussed their potential application to a wide variety of problems. The message we are trying to send here is not that one should abandon use of the MSE nor to blindly switch to any other particular signal fidelity measure. Rather, we hope to make the point that there are powerful, easy-to-use, and easy-to-understand alternatives that might be deployed depending on the application environment and needs. While we expect (and indeed, hope) that the MSE will continue to be widely used as a signal fidelity measure, it is our greater desire to see more advanced signal fidelity measures being used, especially in applications where perceptual criteria might be relevant. Ideally, the performance of a new signal processing algorithm might be compared to other algorithms using several fidelity criteria. Lastly, we hope that we have given further motivation to the community to consider recent advanced signal fidelity measures as design criteria for optimizing signal processing algorithms and systems. It is in this direction that we believe that the greatest benefit eventually lies.
66
Image quality assessment: from error visibility to structural similarity
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh et al. · IEEE Transactions on Image Processing · 2004 · 56K citations
Measures of the Amount of Ecologic Association Between Species
Lee R. Dice · Ecology · 1945 · 11.9K citations
Multiscale structural similarity for image quality assessment
Zhou Wang, Eero P. Simoncelli, Alan C. Bovik · 2004 · 5.9K citations
A universal image quality index
Zhou Wang, Alan C. Bovik · IEEE Signal Processing Letters · 2002 · 5.7K citations