Hi there,
I am Markus, a student of the master’s degree Medical Engineering and Analytics at the CUAS. In this post I will introduce you to the topic of my master’s thesis project.
Let’s start with a short intro: artificial intelligence (AI) isn’t – contrary to the last hype – just about large language models (LLMs). Deep learning, the subfield of AI, also includes image processing and thus enables generating synthetic images, including medical images.
And yes, that’s currently the topic I’m working on for my master’s thesis – Synthetic Magnetic Resonance Images. Well, that sounds kind of easy. And yes, by using Frameworks like PyTorch, it is quite simple to apply it. However, numerous questions arise: Which model fits best, and which criteria must the data fulfil? And what are the pitfalls? Yes, there are many if you don’t know what you’re doing.
Understanding machine learning, statistics, and some math is the key element when working with deep learning.
To the images: You are seeing a segmentation (left, 3D), anterior-posterior view, of some chosen brain areas of a synthetic image (right, one 2D slice selected), which, of course, is also done by Artificial Intelligence 😉 AI is everywhere. Please note if you show the synthetic data on the left side (MR Image) then citation may be required to state with which data Network is trained (see at the bottom).
So, basically, as the backbone, I’m using a Variational Autoencoder (VAE), also called Variational Autoencoding Bayes in the original paper – and that’s the key point: using the Bayes Theorem in the Background to learn a joint probability. Thus, we can move from a complicated image space to a simpler latent space, which is easier to manipulate. I guess you agree that it’s always good to make things easier. Right? But what about adding more cool stuff?
Interestingly, not only does dealing with medical images add a medical aspect. Artificial neurons and biological neurons, yes, you might have heard about similarities and differences, but what about the human visual system and perceptual networks – thus, perceptual discrimination between fakes and real images. And, why not, connecting GANs with VAE – cool, right? But here, attention is all you need – of course – referring just to another paper! Thus, also connecting methods of LLMs to the VAE can be achieved. For not diving too deep into this topic and in this regard, thank you for your attention.
By the way, the project is done in cooperation with the Center of Excellence High-field MRI at the Medical University of Vienna.
Best wishes,
Markus
Data used: Human Connectome Project, WU-Minn Consortium. 2017. “1200 Subjects Data Release (S1200).” https://www.humanconnectome.org/study/hcp-young-adult/document/1200-subjects-data-release
Acknowledgment: Data were provided in part by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.
