Calibration by differentiation - Self-supervised calibration for X-ray microscopy using a differentiable cone-beam reconstruction operator.
X-ray microscopy
computed tomography
deep learning
inverse problems
known operator learning
reconstruction
Journal
Journal of microscopy
ISSN: 1365-2818
Titre abrégé: J Microsc
Pays: England
ID NLM: 0204522
Informations de publication
Date de publication:
08 2022
08 2022
Historique:
revised:
20
04
2022
received:
14
09
2021
accepted:
22
05
2022
pubmed:
1
6
2022
medline:
14
7
2022
entrez:
31
5
2022
Statut:
ppublish
Résumé
High-resolution X-ray microscopy (XRM) is gaining interest for biological investigations of extremely small-scale structures. XRM imaging of bones in living mice could provide new insights into the emergence and treatment of osteoporosis by observing osteocyte lacunae, which are holes in the bone of few micrometres in size. Imaging living animals at that resolution, however, is extremely challenging and requires very sophisticated data processing converting the raw XRM detector output into reconstructed images. This paper presents an open-source, differentiable reconstruction pipeline for XRM data which analytically computes the final image from the raw measurements. In contrast to most proprietary reconstruction software, it offers the user full control over each processing step and, additionally, makes the entire pipeline deep learning compatible by ensuring differentiability. This allows fitting trainable modules both before and after the actual reconstruction step in a purely data-driven way using the gradient-based optimizers of common deep learning frameworks. The value of such differentiability is demonstrated by calibrating the parameters of a simple cupping correction module operating on the raw projection images using only a self-supervisory quality metric based on the reconstructed volume and no further calibration measurements. The retrospective calibration directly improves image quality as it avoids cupping artefacts and decreases the difference in grey values between outer and inner bone by 68-94%. Furthermore, it makes the reconstruction process entirely independent of the XRM manufacturer and paves the way to explore modern deep learning reconstruction methods for arbitrary XRM and, potentially, other flat-panel computed tomography systems. This exemplifies how differentiable reconstruction can be leveraged in the context of XRM and, hence, is an important step towards the goal of reducing the resolution limit of in vivo bone imaging to the single micrometre domain.
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
81-92Informations de copyright
© 2022 The Authors. Journal of Microscopy published by John Wiley & Sons Ltd on behalf of Royal Microscopical Society.
Références
Gruber, R., Pietschmann, P., & Peterlik, M. (2008). Introduction to bone development, remodelling and repair. In Radiology of osteoporosis (pp. 1-23). Springer Berlin Heidelberg.
Buenzli, P. R., & Sims, N. A. (2015). Quantifying the osteocyte network in the human skeleton. Bone, 75, 144-150.
Hannah, K. M., Thomas, C. D., Clement, J. G., De Carlo, F., & Peele, A. G. (2010). Bimodal distribution of osteocyte lacunar size in the human femoral cortex as revealed by micro-CT. Bone, 47(5), 866-871.
Grüneboom, A., Hawwari, I., Weidner, D., Culemann, S., Müller, S., Henneberg, S., Brenzel, A., Merz, S., Bornemann, L., Zec, K., Wuelling, M., Kling, L., Hasenberg, M., Voortmann, S., Lang, S., Baum, W., Ohs, A., Kraff, O., Quick, H. H., … Gunzer, M. (2019). A network of trans-cortical capillaries as mainstay for blood circulation in long bones. Nature Metabolism, 1(2), 236-250.
Rho, J.-Y., Kuhn-Spearing, L., & Zioupos, P. (1998). Mechanical properties and the hierarchical structure of bone. Medical Engineering and Physics, 20(2), 92-102.
Grüneboom, A., Kling, L., Christiansen, S., Mill, L., Maier, A., Engelke, K., Quick, H. H., Schett, G., & Gunzer, M. (2019). Next-generation imaging of the skeletal system and its blood supply. Nature Reviews Rheumatology, 15(9), 533-549.
Yu, B., Pacureanu, A., Olivier, C., Cloetens, P., & Peyrin, F. (2021). Quantification of the bone lacunocanalicular network from 3D X-ray phase nanotomography images. Journal of Microscopy, 282(1), 30-44.
Varga, P., Hesse, B., Langer, M., Schrof, S., Männicke, N., Suhonen, H., Pacureanu, A., Pahr, D., Peyrin, F., & Raum, K. (2015). Synchrotron X-ray phase nano-tomography-based analysis of the lacunar-canalicular network morphology and its relation to the strains experienced by osteocytes in situ as predicted by case-specific finite element analysis. Biomechanics and Modeling in Mechanobiology, 14(2), 267-282.
Kerschnitzki, M., Kollmannsberger, P., Burghammer, M., Duda, G. N., Weinkamer, R., Wagermaier, W., & Fratzl, P. (2013). Architecture of the osteocyte network correlates with bone material quality. Journal of Bone and Mineral Research, 28(8), 1837-1845.
Clark, D. P., & Badea, C. (2014). Micro-CT of rodents: State-of-the-art and future perspectives. Physica Medica, 30(6), 619-634.
Longo, A. B., Salmon, P. L., & Ward, W. E. (2017). Comparison of ex vivo and in vivo micro-computed tomography of rat tibia at different scanning settings. Journal of Orthopaedic Research, 35(8), 1690-1698.
Mill, L., Bier, B., Syben, C., Kling, L., Klingberg, A., Christiansen, S., Schett, G., & Maier, A. (2018). Towards in-vivo x-ray nanoscopy-The effect of motion on image quality. In Bildverarbeitung für die Medizin 2018 (pp. 115-120). Springer Berlin Heidelberg.
Huang, Y., Mill, L., Stoll, R., Kling, L., Aust, O., Wagner, F., Grüneboom, A., Schett, G., Christiansen, S., & Maier, A. (2021). Semi-permeable filters for interior region of interest dose reduction in X-ray microscopy. In Bildverarbeitung für die Medizin 2021 (pp. 61-66). Springer Fachmedien Wiesbaden.
Ghani, M. U., & Karl, W. (2018). Deep learning based sinogram correction for metal artifact reduction. Electronic Imaging, 2018(15), 4721-4728.
Ketcha, M. D., Marrama, M., Souza, A., Uneri, A., Wu, P., Zhang, X., Helm, P. A., & Siewerdsen, J. H. (2021). Sinogram + image domain neural network approach for metal artifact reduction in low-dose cone-beam computed tomography. Journal of Medical Imaging, 8(5), 1-16.
Yi, X., & Babyn, P. (2018). Sharpness-aware low-dose CT denoising using conditional generative adversarial network. Journal of Digital Imaging, 31(5), 655-669.
Patwari, M., Gutjahr, R., Raupach, R., & Maier, A. (2020). JBFnet - Low dose CT denoising by trainable joint bilateral filtering. In Proc. MICCAI (pp. 506-515). Springer.
Jacobsen, C. (2019). X-ray microscopy. Advances in Microscopy and Microanalysis. Cambridge University Press.
Niverty, S., Torbatissaraf, H., Nikitin, V., De Andrade, V., Niauzorau, S., Kublik, N., Azeredo, B., Tekawade, A., De Carlo, F., & Chawla, N. (2021). Computational imaging in 3D X-ray microscopy: Reconstruction, image segmentation and time-evolved experiments. In IEEE International Conference on Image Processing (pp. 3502-3506). IEEE.
Langer, M., & Peyrin, F. (2016). 3D X-ray ultra-microscopy of bone tissue. Osteoporosis International, 27(2), 441-455.
Andrews, J. C., Almeida, E., van der Meulen, M. C. H., Alwood, J. S., Lee, C., Liu, Y., Chen, J., Meirer, F., Feser, M., Gelb, J., Rudati, J., Tkachuk, A., Yun, W., & Pianetta, P. (2010). Nanoscale x-ray microscopic imaging of mammalian mineralized tissue. Microscopy and Microanalysis, 16(3), 327-336.
Feldkamp, L. A., Davis, L. C., & Kress, J. W. (1984). Practical cone-beam algorithm. Journal of the Optical Society of America A, 1(6), 612-619.
Zeng, G. L. (2010). Medical image reconstruction: A conceptual tutorial. Springer.
Parker, D. L. (1982). Optimal short scan convolution reconstruction for fan beam CT. Medical Physics, 9(2), 254-257.
Maier, A., Syben, C., Stimpel, B., Würfl, T., Hoffmann, M., Schebesch, F., Fu, W., Mill, L., Kling, L., & Christiansen, S. (2019). Learning with known operators reduces maximum error bounds. Nature Machine Intelligence, 1(8), 373-380.
Öktem, O., Adler, J., & Kohr, H. (2022). Operator discretization library (ODL). GitHub. Retrieved from https://github.com/odlgroup/odl
van Aarle, W., Palenstijn, W. J., Cant, J., Janssens, E., Bleichrodt, F., Dabravolski, A., Beenhouwer, J. D., Batenburg, K. J., & Sijbers, J. (2016). Fast and flexible X-ray tomography using the ASTRA toolbox. Optics Express, 24(22), 25129-25147.
Gürsoy, D., De Carlo, F., Xiao, X., & Jacobsen, C. (2014). TomoPy: A framework for the analysis of synchrotron tomographic data. Journal of Synchrotron Radiation, 21.
Syben, C., Michen, M., Stimpel, B., Seitz, S., Ploner, S., & Maier, A. K. (2019). PYRO-NN: Python reconstruction operators in neural networks. Medical Physics, 46(11), 5110-5115.
Ronchetti, M. (2020). TorchRadon: Fast differentiable routines for computed tomography. arxiv.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., … Chintala, S. (2019). PyTorch: An imperative style, high-performance deep learning library. In Proc. NeurIPS (pp. 8024-8035). Curran Associates, Inc.
Yang, X. (2021). Practical basics and applications of X-ray tomography (pp. 27-44). Springer.
Yang, X., & Schroer, C. (2021). Strategies of deep learning for tomographic reconstruction. In IEEE International Conference on Image Processing (pp. 3473-3476).
Huang, Y., Wang, S., Guan, Y., & Maier, A. (2020). Limited angle tomography for transmission X-ray microscopy using deep learning. Journal of Synchrotron Radiation, 27(2), 477-485.
Chen, H., Zhang, Y., Chen, Y., Zhang, J., Zhang, W., Sun, H., Lv, Y., Liao, P., Zhou, J., & Wang, G. (2018). LEARN: Learned experts’ assessment-based reconstruction network for sparse-data CT. IEEE Transactions on Medical Imaging, 37(6), 1333-1347.
Würfl, T., Hoffmann, M., Christlein, V., Breininger, K., Huang, Y., Unberath, M., & Maier, A. K. (2018). Deep learning computed tomography: Learning projection-domain weights from image domain in limited angle problems. IEEE Transactions on Medical Imaging, 37(6), 1454-1463.
Lagerwerf, M. J., Pelt, D. M., Palenstijn, W. J., & Batenburg, K. J. (2020). A computationally efficient reconstruction algorithm for circular cone-beam computed tomography using shallow neural networks. Journal of Imaging, 6(12), 135.
De Carlo, F., Gürsoy, D., Marone, F., Rivers, M., Parkinson, D. Y., Khan, F., Schwarz, N., Vine, D. J., Vogt, S., Gleber, S.-C., Narayanan, S., Newville, M., Lanzirotti, T., Sun, Y., Hong, Y. P., & Jacobsen, C. (2014). Scientific data exchange: A schema for HDF5-based storage of raw and analyzed data. Journal of Synchrotron Radiation, 21(6), 1224-1230.
Brooks, R. A. (1977). A quantitative theory of the Hounsfield unit and its application to dual energy scanning. Journal of Computer Assisted Tomography, 1(4), 487-493.
Schulze, R., Heil, U., Groß, D., Bruellmann, D. D., Dranischnikow, E., Schwanecke, U., & Schoemer, E. (2011). Artefacts in CBCT: A review. Dentomaxillofacial Radiology, 40(5), 265-273.
Brooks, R. A., & Di Chiro, G. (1976). Beam hardening in x-ray reconstructive tomography. Physics in Medicine and Biology, 21(3), 390-398.
Kachelrieß, M., Sourbelle, K., & Kalender, W. A. (2006). Empirical cupping correction: A first-order raw data precorrection for cone-beam computed tomography. Medical Physics, 33(5), 1269-1274.
Würfl, T., Maaß, N., Dennerlein, F., Huang, X., & Maier, A. K. (2017). Epipolar consistency guided beam hardening reduction-ECC2. In Proc. Fully3D (pp. 181-185).
Kochurov, M., Karimov, R., & Kozlukov, S. (2020). Geoopt: Riemannian optimization in PyTorch. arxiv.
Bécigneul, G., & Ganea, O.-E. (2019). Riemannian adaptive optimization methods. arxiv.
Fat, D. L., Kennedy, J., Galvin, R., O'Brien, F., Mc Grath, F., & Mullett, H. (2012). The Hounsfield value for cortical bone geometry in the proximal Humerus'an in vitro study. Skeletal Radiology, 41(5), 557-568.
Márquez-Neila, P., Baumela, L., & Alvarez, L. (2014). A morphological approach to curvature-based evolution of curves and surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(1), 2-17.
Würfl, T., Maaß, N., Dennerlein, F., Aichert, A., & Maier, A. (2018). Physical constraints for beam hardening reduction using polynomial models. In Proc. CT-Meeting (pp. 356-359).
Wagner, F., Thies, M., Gu, M., Huang, Y., Pechmann, S., Patwari, M., Ploner, S., Aust, O., Uderhardt, S., Schett, G., Christiansen, S., & Maier, A. (2022). Ultra low-parameter denoising: Trainable bilateral filter layers in computed tomography. arxiv.
Syben, C., Stimpel, B., Lommen, J., Würfl, T., Dörfler, A., & Maier, A. (2018). Deriving neural network architectures using precision learning: Parallel-to-fan beam conversion. In German Conference on Pattern Recognition (pp. 503-517). Springer.