Automated total kidney volume measurements in pre-clinical magnetic resonance imaging for resourcing imaging data, annotations, and source code

Abstract

The objective of this study was to validate a fully automated total kidney volume measurement method for pre-clinical rodent trials that is fast, accurate, reproducible, and to provide these resources to the research community. Rodent studies that involve imaging are crucial for monitoring treatment efficacy in diseases such as polycystic kidney disease. Previous studies utilize manual or semi-automated segmentations, which are time consuming and potentially biased. To develop our automated system, a total of 150 axial magnetic resonance images (MRI) from a variety of mouse models were manually segmented and used to train/validate an automated algorithm. To test the longitudinal application of the model, four mutant and four wild-type mice were imaged sequentially over three to twelve weeks via MRI. Segmentations of the kidneys (excluding the renal pelvis) were generated by the automated method and two different readers, with one reader repeating the measurements. Similarity metrics and longitudinal analysis were calculated to assess the performance of the automated compared to the manual methods. The automated approach required no user input, besides a final visual quality control step. Similarity metrics of the automated method versus the manual segmentations were on par with inter- and intra-reader comparisons. Thus, our fully automated approach described here can be safely used in longitudinal, pre-clinical trials that involve the segmentation of rodent kidneys in T2-weighted MRIs. The objective of this study was to validate a fully automated total kidney volume measurement method for pre-clinical rodent trials that is fast, accurate, reproducible, and to provide these resources to the research community. Rodent studies that involve imaging are crucial for monitoring treatment efficacy in diseases such as polycystic kidney disease. Previous studies utilize manual or semi-automated segmentations, which are time consuming and potentially biased. To develop our automated system, a total of 150 axial magnetic resonance images (MRI) from a variety of mouse models were manually segmented and used to train/validate an automated algorithm. To test the longitudinal application of the model, four mutant and four wild-type mice were imaged sequentially over three to twelve weeks via MRI. Segmentations of the kidneys (excluding the renal pelvis) were generated by the automated method and two different readers, with one reader repeating the measurements. Similarity metrics and longitudinal analysis were calculated to assess the performance of the automated compared to the manual methods. The automated approach required no user input, besides a final visual quality control step. Similarity metrics of the automated method versus the manual segmentations were on par with inter- and intra-reader comparisons. Thus, our fully automated approach described here can be safely used in longitudinal, pre-clinical trials that involve the segmentation of rodent kidneys in T2-weighted MRIs. Translational StatementThis study developed a fully automated method for measuring total kidney volume for preclinical imaging in a mutant mouse model of polycystic kidney disease as well as wild-type mice. This study also established both interreader and intrareader variabilities in the measurement of total kidney volume for preclinical imaging. Similar studies and algorithmic approaches can be used to establish methods for clinical imaging data and are needed for accurate disease prognosis and clinical decision-making. We are providing the imaging data, annotations, and source code to the research community. This study developed a fully automated method for measuring total kidney volume for preclinical imaging in a mutant mouse model of polycystic kidney disease as well as wild-type mice. This study also established both interreader and intrareader variabilities in the measurement of total kidney volume for preclinical imaging. Similar studies and algorithmic approaches can be used to establish methods for clinical imaging data and are needed for accurate disease prognosis and clinical decision-making. We are providing the imaging data, annotations, and source code to the research community. The measurement of organ volume has been shown to correlate with clinical manifestations and morbidity of diseases such as total kidney volume (TKV) in autosomal dominant polycystic kidney disease (PKD)1Grantham J.J. Torres V.E. Chapman A.B. et al.Volume progression in polycystic kidney disease.N Engl J Med. 2006; 354: 2122-2130Crossref PubMed Scopus (533) Google Scholar,2Torres V.E. Chapman A.B. Devuyst O. et al.Tolvaptan in patients with autosomal dominant polycystic kidney disease.N Engl J Med. 2012; 367: 2407-2418Crossref PubMed Scopus (856) Google Scholar and is used to ascertain the effectiveness of treatment interventions.3Caroli A. Perico N. Perna A. et al.Effect of longacting somatostatin analogue on kidney and cyst growth in autosomal dominant polycystic kidney disease (ALADIN): a randomised, placebo-controlled, multicentre trial.Lancet. 2013; 382: 1485-1495Abstract Full Text Full Text PDF PubMed Scopus (175) Google Scholar Research, clinical trials, and increasingly clinical nephrology use these measurements to monitor disease progression in both animal models4Wallace D.P. Hou Y.P. Huang Z.L. et al.Tracking kidney volume in mice with polycystic kidney disease by magnetic resonance imaging.Kidney Int. 2008; 73: 778-781Abstract Full Text Full Text PDF PubMed Scopus (29) Google Scholar and patients,5Grantham J.J. Torres V.E. The importance of total kidney volume in evaluating progression of polycystic kidney disease.Nat Rev Nephrol. 2016; 12: 667-677Crossref PubMed Scopus (55) Google Scholar evaluate the effectiveness of therapies,6Higashihara E. Torres V.E. Chapman A.B. et al.Tolvaptan in autosomal dominant polycystic kidney disease: three years’ experience.Clin J Am Soc Nephrol. 2011; 6: 2499-2507Crossref PubMed Scopus (114) Google Scholar and predict outcomes.7Irazabal M.V. Rangel L.J. Bergstralh E.J. et al.Imaging classification of autosomal dominant polycystic kidney disease: a simple model for selecting patients for clinical trials.J Am Soc Nephrol. 2015; 26: 160-172Crossref PubMed Scopus (243) Google Scholar Currently, preclinical studies are occurring at an unprecedented rate to look for new treatments to slow progression of PKD. A key advantage of magnetic resonance imaging (MRI) in animal models of PKD is the ability to use in vivo imaging, which allows for longitudinal volumetric studies that use the same animal.8Irazabal M.V. Mishra P.K. Torres V.E. et al.Use of ultra-high field MRI in small rodent models of polycystic kidney disease for in vivo phenotyping and drug monitoring.J Vis Exp. 2015; 100e52757Google Scholar Numerous studies that involve manual,4Wallace D.P. Hou Y.P. Huang Z.L. et al.Tracking kidney volume in mice with polycystic kidney disease by magnetic resonance imaging.Kidney Int. 2008; 73: 778-781Abstract Full Text Full Text PDF PubMed Scopus (29) Google Scholar,9Erokwu B.O. Anderson C.E. Flask C.A. et al.Quantitative magnetic resonance imaging assessments of autosomal recessive polycystic kidney disease progression and response to therapy in an animal model.Pediatr Res. 2018; 83: 1067-1074Crossref PubMed Scopus (4) Google Scholar, 10Doctor R.B. Serkova N.J. Hasebroock K.M. et al.Distinct patterns of kidney and liver cyst growth in pkd2(WS25/-) mice.Nephrol Dial Transplant. 2010; 25: 3496-3504Crossref PubMed Scopus (12) Google Scholar, 11Franke M. Baessler B. Vechtel J. et al.Magnetic resonance T2 mapping and diffusion-weighted imaging for early detection of cystogenesis and response to therapy in a mouse model of polycystic kidney disease.Kidney Int. 2017; 92: 1544-1554Abstract Full Text Full Text PDF PubMed Scopus (16) Google Scholar, 12Zhou X. Bao H. Takakura A. et al.Polycystic kidney disease evaluation by magnetic resonance imaging in ischemia-reperfusion injured PKD1 knockout mouse model: comparison of T2-weighted FSE and true-FISP.Invest Radiol. 2010; 45: 24-28Crossref PubMed Scopus (7) Google Scholar semiautomated,13Fei B. Flask C. Wang H. et al.Image segmentation, registration and visualization of serial MR images for therapeutic assessment of polycystic kidney disease in transgenic mice.Conf Proc IEEE Eng Med Biol Soc. 2005; 1: 467-469Crossref PubMed Google Scholar,14Almajdub M. Magnier L. Juillard L. et al.Kidney volume quantification using contrast-enhanced in vivo X-ray micro-CT in mice.Contrast Media Mol Imaging. 2008; 3: 120-126Crossref PubMed Scopus (24) Google Scholar and registration-based automated segmentations15Hadjidemetriou S. Reichardt W. Hennig J. et al.Volumetric analysis of MRI data monitoring the treatment of polycystic kidney disease in a mouse model.MAGMA. 2011; 24: 109-119Crossref PubMed Scopus (5) Google Scholar,16Gleason S.S. Sari-Sarraf H. Abidi M.A. et al.A new deformable model for analysis of X-ray CT images in preclinical studies of mice for polycystic kidney disease.IEEE Trans Med Imaging. 2002; 21: 1302-1309Crossref PubMed Scopus (17) Google Scholar of mouse kidneys have been performed previously. Many of the methods deemed as automated still require user input. A majority of these preclinical studies use manual segmentations, which are time-consuming and costly as well as introduce observer bias. Therefore, our laboratory has assessed the variability in the measurement of TKV and developed an automated analysis program to measure TKV in magnetic resonance scans of murine models of the disease. Figure 1 shows the results of the Bland-Altman analysis of TKV as measured manually by 2 readers (interobserver variance) and repeat measurements by reader 2 (intraobserver variance). When reader 1 was compared with reader 2, there was a mean percentage difference of 7.7% and a 95% confidence interval of ±4.5%. When reader 2 performed repeat measurements of the same image, there was a mean percentage difference of −0.5% and a 95% confidence interval of ±3.9%. Regression analysis indicated that there is high agreement in TKV among all methods, with an R2 value of ≥0.99. The automated method compared with each reader in relation to percentage difference of TKV was similar to that of the inter- and intraobserver variance as suggested by the Bland-Altman plots in Figure 1. When reader 1 was compared with the automated method, there was a mean percentage difference of 5.2% and a 95% confidence interval of ±5.8%. When reader 2 was compared with the automated method, there was a mean percentage difference of −2.5% and a 95% confidence interval of ±6.5%. The mean and SD TKVs were plotted at each time point for each method and separated by genotype (mutant vs. wild-type). As seen in Figure 2, the mean TKV is always smaller in wild-type mice at each time point than in mutant mice. All 3 methods (automated, reader 1, and reader 2) demonstrate a significant separation of mouse type at 9 and 12 weeks of age. The analysis of kidney volume in PKD is one of the most important metrics currently used for characterizing disease status. Before our work, there was no alternative to manually tracing kidneys in model systems of PKD. Because of the time taken to trace these structures, as well as the time required to train a person to perform these measurements, and the potential for interoperator variability, in this study we developed and validated a fully automated segmentation method for TKV. Automated segmentations are computed in a matter of minutes (depending on computing power), whereas manual segmentations take 20 to 40 minutes. Unlike manual or even semiautomated segmentation methods, this automated method will produce the same exact results each time it is applied to the same image. Preclinical trials often comprise both a control group and treatment group(s); therefore, it is essential that the automated method is sensitive enough to detect volume differences between groups appropriately.17Reichardt W. Romaker D. Becker A. et al.Monitoring kidney and renal cyst volumes applying MR approaches on a rapamycin treated mouse model of ADPKD.MAGMA. 2009; 22: 143-149Crossref PubMed Scopus (19) Google Scholar Figure 2 shows that manual segmentations and the automated segmentation both show a significant separation in the wild-type and mutant groups at 9 weeks of age. Although the overall agreement was excellent, visual comparisons suggested minor disagreement on whether to include or exclude the renal pelvis in a small subset of slices. Although it is common practice to exclude the renal pelvis, variability might decrease if readers are instructed to always include this structure. The automated method presented in this study has not yet been applied to external images. It is important to note that signal intensities vary across sites, scanners, and MRI acquisitions. It is likely that a larger training data set with more diverse cases from different MRI machines could achieve a more robust model because of the nature of deep learning algorithms. Automated algorithms such as these often need to be retrained on outside data sets because of the variations in MRI signals. We therefore provide the imaging data, annotations, and source code to the research community in order for other groups to either use the same model or develop their own. The model was trained on 100 cases and validated on 50 cases. These 150 cases consisted of mice with varied disease severity and at a range of ages. The test set is a completely held out set and is what we evaluated in this article. This study was reviewed and approved by the Mayo Clinic Institutional Animal Care and Use Committee. The cohort consisted of wild-type C57Bl6 × 129s6Svev/Tac (n = 4; 2 females/2 males) and C57Bl6 × 129s6Svev/Tac (n = 4; 2 females/2 males) mutant Pkd1RC/RC model mice. Mutant mice mirror the human manifestation of PKD1 both genetically and phenotypically.18Hopp K. Ward C.J. Hommerding C.J. et al.Functional polycystin-1 dosage governs autosomal dominant polycystic kidney disease severity.J Clin Invest. 2012; 122: 4257-4273Crossref PubMed Scopus (185) Google Scholar One of the mutant mice died mid-experiment and was replaced with another mutant mouse of the same age at week 9. Imaging was performed using an Avance DRX 700WB (Bruker BioSpin, Billerica, MA) spectrometer. Complete coverage of the kidneys was obtained by an axial TurboRARE T2-weighted acquisition reconstructed with 0.1 mm in plane resolution and 1-mm slice thickness (matrix size, 256 × 256 × Z, with Z chosen large enough to cover the full extent of the kidneys). The total scan time ranged from 5 to 10 minutes. In the testing study cohort, each mouse was imaged at 4 time points (3, 6, 9, and 12 weeks of age). The time points for each mouse were performed within 2 days of each other to ensure consistent imaging parameters and to limit environmental variations. Regions of interest were traced on each scan using an imaging software package (Analyze, version 12.0, Biomedical Imaging Resource, Mayo Clinic, Rochester, MN). Each reader was instructed to exclude the renal pelvis when the renal pelvis is not enclosed by the kidney capsule within the image slice. Manual segmentations took 20 to 40 minutes depending on the case. TKVs were calculated by first summing the number of voxels contained within the segmentation on each slice and then multiplying the number of voxels by the voxel volume obtained from the image header. For the testing data, 2 double-blind readers (1 and 2), both experienced in manual MRI segmentation, performed kidney segmentations on all scans. For intrareader analysis, reader 2 repeated the measurements at 2 time points (>3 months apart). The neural network model was adapted from our previous model for the measurement of TKV from coronal T2-weighted magnetic resonance images from clinical scans.19Kline T.L. Korfiatis P. Edwards M.E. et al.Performance of an artificial multi-observer deep neural network for fully automated segmentation of polycystic kidneys.J Digit Imaging. 2017; 30: 442-448Crossref PubMed Scopus (52) Google Scholar The source code, images, and annotations are made publicly available at: https://github.com/TLKline/AutoTKV_MouseMRI. The axial T2-weighted magnetic resonance images acquired per mouse (n = 8) at each time point (3, 6, 9, and 12 weeks) were used for the statistical analysis. A total of 32 images allowed for the comparison of different images over a wide range of ages and differences in phenotypes. For validating the fully automated method, comparison statistics were used to evaluate the ability of each method to separate wild-type and mutant groups. This was achieved by plotting the TKV according to time point and separating by mouse type. TKV measurements and growth rates for each method were also assessed using Bland-Altman and linear regression plots. All the authors declared no competing interests. This work was supported by the Mayo Clinic Robert M. and Billie Kelley Pirnie Translational Polycystic Kidney Disease Center and the National Institute of Diabetes and Digestive and Kidney Diseases (grant numbers P30DK090728 and K01DK110136 ). The authors thank Lynnae M. Henry for her administrative support in preparing and formatting this manuscript.