Refined clinical scoring in comparative EAE studies does not enhance the chance to observe statistically significant differences

Abstract

Considering the 3R rules of animal experimentation, we asked if refined scoring of experimental autoimmune encephalomyelitis (EAE) in mice improves documentation of clinical EAE allowing to perform a more powerful statistical analysis. Surprisingly, refined EAE scoring failed to improve statistical outcome comparing the overall disease courses between two groups of mice. Experimental autoimmune encephalomyelitis (EAE) in rodents is frequently used as an animal model for multiple sclerosis (MS). A potential role of individual genes in MS pathogenesis is regularly investigated by comparing EAE development in gene-targeted mice with that in their wild-type littermates 1. Clinical disease in EAE models ultimately amounts to motor dysfunction, thus EAE severity is generally scored in correlation to the severity of paralysis. However, there is no international standard scoring system, which research groups would use to measure EAE severity and clinical assessment scales mostly in use reach from 0–3 points to 0–6 points 2-5. The use of different EAE scoring systems prohibits direct comparison of clinical EAE data published from different laboratories. Different scoring might include differences in the subjective bias influencing the scoring and might also lead to differences in the results reported from the different laboratories, including differences in the statistical significance reported. Refined clinical EAE scoring includes sophisticated motor skill testing, which increases mouse handling time causing increased distress for the animals. Considering the 3R rules, aiming to refine animal experiments, we here asked if extended scoring of EAE in mice will grant improved documentation of clinical EAE allowing to perform a more powerful statistical analysis and thus identify more subtle differences when comparing EAE courses in different groups of mice. To this end we compared EAE scores in wild-type SJL/J and C57BL/6J mice with different knockout littermates, respectively using a 0–3 point scale used in our laboratory 6, 7 (Materials and methods in Supporting Information) and a second, more refined scoring system 8, 9, which includes sophisticated evaluation of motor skill testing such as grid walk, a righting test (the ability of mice to turn over when they have been placed on their back) and a hanging test to determine front limb weakness. The latter score also included unilateral disease signs, which were scored as intermediate steps of 0.5 (Material and methods). Over a time period of 9 months, six different researchers evaluated the disease progression in actively induced EAE (aEAE) models in both, C57BL/6J and SJL/J mice as described before 7 as well as in an adoptive transfer EAE (tEAE) model induced by the transfer of 2D2 T cells into C57BL/6 mice 10 in comparison to their respective knockout littermates using the two scoring systems in parallel. During the time of observation a total of ten different mouse strains (C57BL/6 and SJL/J 2 wild-type mouse lines and 8 knockout mouse lines) were analyzed in three independent tEAE (n = 46) and five independent aEAE experiments (n = 157) and were included into the study. Graphical display of the disease course of not significantly different aEAE experiments (Fig. 1A), significantly different aEAE experiments (Fig. 1E), and the tEAE experiments (Fig. 1I) evaluated with a 3-point scale showed a slightly less soft curve when compared to the curve revealed by the 5.5-point scale (Fig. 1C, G, K). The area under the curve (AUC) of the overall disease severity (Fig. 1B, D, F, H, J, L) was calculated for each mouse for the 3-point and the 5.5-point scale to compare the disease course of wild-type and knockout mice using nonparametric statistical tests (Mann–Whitney test, also referred to as Wilcoxon rank-sum test). Despite a clear visual difference of the graphs with the 5.5-point scale and the graph with the 3-point scale, the obtained p-values were hardly different or even identical. Thus, we next asked whether the two scoring systems are really providing fundamentally different information. To address this we compared the discrepancies of the two scoring measurement methods by constructing Bland-Altman plots 11. In these plots, the means of the two measurement systems on the same observation (on x-axis) are plotted against the differences of the two measurements (on y-axis). For these plots we rescaled the scorings from the 5.5-point scale to have a maximum of 3 (as the 3-point scale scorings) and similarly the corresponding area under the curve values (Fig. 2A–C). Additionally we constructed Bland-Altman plots for the relative ranks of the AUC derived from the 3-point and 5.5-point scorings (Fig. 2D–F) for all three experiments. We found that there was a high agreement between the two measurement methods, suggesting that the amount of information of the two scoring systems is comparable, especially the relative rankings were very similar which then, in turn, resulted in very similar p-values when comparing wild-type with knockout mice with nonparametric statistical tests. Finally, statistical analysis of additional parameters e.g. mean day of onset, mean maximal disease score, and mean day of resolution using the two different scorings also produced similar or identical p-values (Supporting Information Fig. 1). Taken together, our results show that although a refined EAE scoring improves graphic representation of different clinical EAE scores between two groups, it failed to relevantly improve the ability to statistically establish differences between the overall disease course of the two groups of mice. Based on these observations we propose that 3-point scoring of disease progression in EAE is preferable to refined scoring systems as it comes, for almost the same amount of information, with less distress for the assessed mice and is thus more in line with the 3R guidelines. This work was funded by the Swiss National Science Foundation, the Swiss Multiple Sclerosis Society, and the EU FP7 ITN nEUROinflammation to B.E. and an UniBe Initiator Grant to S.T. The authors declare no financial or commercial conflict of interest. As a service to our authors and readers, this journal provides supporting information supplied by the authors. Such materials are peer reviewed and may be re-organized for online delivery, but are not copy-edited or typeset. Technical support issues arising from supporting information (other than missing files) should be addressed to the authors. Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.