New computational microscopy technique offers a more direct route to clear images

For hundreds of years, the clarity and magnification of microscopes were ultimately limited by the physical properties of their optical lenses. Microscope manufacturers pushed those limits by making increasingly complex and expensive arrays of lens elements. However, scientists had to decide between high resolution and a small field of view on the one hand or low resolution and a large field of view on the other.

In 2013, a team of Caltech engineers introduced a microscopy technique called FPM (for Fourier Pictographic Microscopy). This technology marked the advent of computational microscopy, the use of techniques that link the sensing of conventional microscopes with computer algorithms that process the detected information in new ways to create deeper and clearer images that cover larger areas. Since then FPM has been widely adopted for its ability to obtain high-resolution images of samples while maintaining a large field of view using relatively inexpensive equipment.

Now the same lab has developed a new method that can surpass FPM in its ability to obtain images without blur or distortion, even while taking fewer measurements. The new technique, described in a paper that appeared in the journal Nature Communicationsit could lead to advances in areas such as biomedical imaging, digital pathology and drug screening.

The new method, called APIC (for Angular Ptychographic Images by Closed Form Method), has all the advantages of FPM without what can be described as its biggest weakness – namely, that to arrive at a final image , the FPM algorithm relies on starting at one or several best guesses and then tweaking it a bit at a time to arrive at its “optimal” solution, which may not always be true with the original image.

Under the leadership of Changhuei Yang, the Thomas G. Myers Professor of Electrical Engineering, Bioengineering and Medical Engineering and an investigator with the Heritage Medical Research Institute, the Caltech team realized that it was possible to eliminate this iterative nature of the algorithm.

Rather than relying on trial and error to try to arrive at a solution, APIC solves a linear equation, detailing the deviations or distortions introduced by the microscope’s optical system. Once deviations are recognized, the system can correct them, essentially performing as if it were ideal and delivering clear images that cover large fields of view.

“We achieve a resolution of the complex field with high resolution in a closed form, as we now have a deeper understanding of what a microscope captures, what we already know and what we really need to understand, so we don’t need any replication,” says Ruizhi Cao, co-lead author on the paper, a former graduate student in Yang’s lab and now a postdoctoral researcher at UC Berkeley. “That way, we can basically guarantee that we are seeing the true final details of a champion.”

As with FPM, the new method measures not only the intensity of light seen through the microscope, but also an important property of light called “phase,” which is related to the distance the light travels. This property remains undetectable to the human eye, but it contains information that is very useful in terms of correcting aberrations.

It was in solving this phase of information that FPM relied on trial and error, explains Cheng Shen, co-lead author on the APIC paper, who also completed the work while in Yang’s lab and is now a vision algorithm engineer. computer. at Apple.

“We’ve proven that our method gives you an analytical solution and in a much more direct way. It’s faster, more accurate, and uses some deep knowledge about the optical system,” says Shen.

Beyond eliminating the iterative nature of the phase-resolving algorithm, the new technique also allows researchers to collect clear images over a large field of view without repeatedly refocusing the microscope. With FPM, if the height of the sample varied by even a few tens of microns from one section to another, the person using the microscope would have to refocus in order for the algorithm to work.

Since these computational microscopy techniques often involve stitching together more than 100 lower-resolution images to stitch together the largest field of view, this means that APIC can make the process much faster and prevent potential imaging of human error in many steps.

“We have developed a framework to correct the deviations and also improve the solution,” Cao says. “These two capabilities could be potentially fruitful for a wider range of imaging systems.”

Yang says the development of APIC is vital to the broader scope of work his lab is currently working on to optimize image data entry for artificial intelligence (AI) applications.

“Recently, my lab showed that AI can outperform expert pathologists in predicting metastatic progression from simple histopathology slides from lung cancer patients,” says Yang. “This predictive capability is critically dependent on obtaining high-quality, uniformly focused microscopy images, something APIC is well-suited for.”