Artificial Intelligence (AI) methods (such as machine learning and deep learning) for imaging and analysis applications have proved to be powerful approaches for improving resolution, reducing noise and automating segmentation.
AI-based processing tools are now available in Thermo Scientific Amira-Avizo Software and PerGeos Software. Advanced Texture Supervised Classification tools and Color Auto Classification, powered by Machine Learning, were released in Amira-Avizo Software version 2019.2 and PerGeos Software version 2019.2 while Deep Learning modules (both training and inference) have been introduced in Amira-Avizo Software version 2019.3 and PerGeos Software version 2019.3.
The use of AI-based tools in Amira-Avizo Software and PerGeos Software is a major leap forward and enriches processing capabilities by allowing the ability to mix both traditional and AI-based algorithms.
Deep Learning in Amira-Avizo Software and PerGeos Software
Deep-learned neural networks have proved to be an invaluable tool for many research and industrial purposes in recent years. Using deep learning for processing images allows researchers to go beyond traditional image processing for greatly improved results.
Amira-Avizo Software and PerGeos Software provide ideal environments for deep learning. A pre/post-processing and segmentation toolbox allows enhanced data for both the training phase (creating some ground truth-segmented datasets) and the post-prediction phase (improving predicted data) while leveraging the actual model building, training and prediction steps from an experienced deep learning framework such as TensorFlow, Keras or PyTorch. The workflow for learning on a manually segmented sub-volume and performing the prediction on a complete dataset is as simple as using those two Python script modules.
Amira-Avizo Software and PerGeos Software's Python integration allows you to create multiple self-contained Python environments with their own Python set of packages.
An environment dedicated to deep learning can be easily set up using a new dedicated user interface.
While you can configure a deep learning environment based on your preferred Python package (Keras, TensorFlow, PyTorch) and use those libraries in Python script modules (pyscro), Amira-Avizo Software and PerGeos Software’s latest versions provide a default implementation based on Keras, a high-level neural networks API, written in Python and running on top of TensorFlow.
The deep learning training modules feature a highly configurable tool for training models using state-of-the-art architectures such as Unet. The training can occur from scratch (random weights) or from pre-trained weights.
The deep learning prediction module then allows for reuse of the trained models on data it has never seen before.
An AI-powered image data analysis solution for fast and accurate feature and statistics extraction is also available in a set of advanced applications for EM systems in the Amira-Avizo2D Software.
Applications and Use cases
Image segmentation of serial section TEM (ISBI 2012 Segmentation Challenge)
Image segmentation of mitochondria blobs
Mitochondria are difficult to segment using traditional approaches because they have connections with the outer endoplasmic reticulum and an internal membrane-like structure.
The model trained with Amira-Avizo Software’s deep learning tool allows the automatic extraction of mitochondria from a FIB/SEM stack. The training was done using only a few slices, which were segmented manually with Amira-Avizo Software’s segmentation editor. It was then possible to automatically segment the rest of the stack, saving hours of manual work.
Image denoising of SEM
For 3D serial sectioning and 2D tiling applications, time to data versus image quality has to be carefully balanced. Usually, the data is down-sampled a lot to process it. Following acquisition, conventional algorithms, such as gaussian-smoothing and non-local-means filtering, leave artifacts. Alternatively, deep learning algorithms can be tuned in such a way that they do not induce artifacts. Processing can be done relatively quickly when a deep learning model is available. Below, we highlight a model that can quickly restore SEM images.
Image super-resolution
High resolution images are often needed to clearly capture the desired structure details, while lower resolution acquisition may be imposed by exposure time and dose applied to the sample.
Super-resolution deep learning algorithms can restore realistic details from lower resolution images, dramatically facilitating image segmentation.
Color Auto Classification based on machine learning
Based on machine learning, the Color Auto Classification tool automatically segments a color image into labels. A supervised random forest method is used.
Texture Supervised Classification based on machine learning
Texture classification is a machine learning technique that relies on learning texture patterns from markers defined by the user and then classifying each pixel of the image according to its similarity to the learned patterns.
