Abstract
Photoacoustic imaging has been developed as a new biomedical molecular imaging modality. Due to its similarity to conventional ultrasound imaging in terms of signal detection and image generation, dual-modal photoacoustic and ultrasound imaging has been applied to visualize physiological and morphological information in biological systems in vivo. By complementing each other, dual-modal photoacoustic and ultrasound imaging showed synergistic advances in photoacoustic imaging with the guidance of ultrasound images. In this review, we introduce our recent progresses in dual-modal photoacoustic and ultrasound imaging systems at various scales of study, from preclinical small animals to clinical humans. A summary of the works reveals various strategies for combining the structural information of ultrasound images with the molecular information of photoacoustic images.
Keywords
Impact Statement
Photoacoustic and ultrasound images can be obtained in a single imaging platform because of their similar data acquisition and image reconstruction procedures. Dual-modal images can provide complementary structural and molecular information, providing more helpful information for biological system analysis rather than simply integrating two different imaging technologies. There have been reports on the synergistic guidance of ultrasound imaging for improving photoacoustic imaging performance, such as organ depth estimation, motion compensation, or multi-wavelength photoacoustic analysis. In this review, we introduce our recent reports on dual-modal photoacoustic and ultrasound imaging systems at various scales, from microscopy to tomography, that have explored the synergistic ultrasound guidance for photoacoustic analysis.
Introduction
Photoacoustic imaging (PAI) is an emerging non-invasive and non-ionizing biomedical imaging method that can achieve rich optical contrast with the spatiotemporal resolution of ultrasound imaging (USI).1–3 The key strength of PAI is that it can visualize molecular functional information of biological tissues up to a few centimeters by overcoming the shallow imaging depth of pure optical imaging techniques.4–6 Therefore, PAI has been widely applied in various biomedical fields, including hemodynamic analysis,7–12 contrast-enhanced imaging,13–16 drug delivery monitoring,17–20 and image-guided therapy.21–27
The basic principle of PAI is ultrasound (US) generation through the photoacoustic (PA) effect, which is energy transduction from light to acoustic waves. 28 When biological tissue is irradiated with a short pulsed light with a typical pulse width of a few nanoseconds, optical chromophores in biological tissues absorb light energy according to their optical absorption characteristics. Subsequently, the absorbed light energy is released in two forms: (1) light emission with a shifted wavelength, which is the principle of fluorescence, and (2) heat release, which causes thermoelastic expansion. Because of the short pulse duration, the thermoelastic expansion rapidly dissipates, and the optically absorbing chromophores shrink to their original volume, producing acoustic waves referred to as PA waves.
Based on these principles, PAI visualizes the optical absorption characteristics of biological tissues. Therefore, it can provide molecular functional information similar to that obtained using pure optical imaging techniques. 29 In addition to some endogenous chromophores, such as oxy- and deoxy-hemoglobins,30–32 melanin, 33 and lipid,34–36 exogenous agents 37–41 also have been widely applied to enhance the contrast of PA images. By visualizing the biological distribution of optical absorption characteristics, PAI has been utilized in preclinical small-animal studies42–45 and clinical human studies.46–49
In PAI, the generated PA waves can be captured using conventional US transducers and data acquisition systems. Therefore, PA and US images can be easily integrated into a single imaging platform.50–52 However, the underlying information in the two images is different, complementing each other: structural information from USI and molecular functional information from PAI (Figure 1). Therefore, dual-modal PA and US imaging (PAUSI) is significant because it can obtain more helpful information for biological system analysis rather than simply integrating two different imaging technologies.

Schematic illustration of dual-modal PA and US imaging.
USI has been a widely used clinical imaging modality due to its non-invasive and real-time imaging capabilities. One of the major limitations of USI is its inability to differentiate tissues with similar acoustic properties, which can result in misdiagnosis. In contrast, PAI is based on the absorption of light by tissue chromophores, enabling tissue differentiation based on optical properties. Additionally, PAI can provide functional and molecular information in vivo, such as oxygen saturation levels and tissue metabolism, which is not possible with USI.53,54 However, PAI cannot visualize the optically transparent tissues, thus limiting the structural information in the PA images. Therefore, the structural information in the US data can be used to compensate for the PA data, providing complementary information for more accurate analyses.
Here, we review our recent reports on US-guided PA analysis in biomedical applications, where the limitations of each image were successfully complemented based on the simultaneously achieved PA and US data. This review focuses on the implementation of PAUSI systems and their applications with improved PA data compensated by US data. From these results, we observed that dual-modal PAUSI systems have great potential in biomedical studies. In particular, US-assisted data modification can improve the quality of the PA data; therefore, a more accurate analysis can be performed.
Ultrasound-guided photoacoustic imaging with single-element transducers
Previously, we reported a PAI system that could provide whole-body PA images of small animals in vivo. 55 The system can obtain multispectral PA images with a tunable optical parametric oscillator (OPO) laser (Surelite OPO PLUS, Continuum, USA) pumped by a Q-switched Nd:YAG laser (Surelite III-10, Continuum, USA). The generated PA waves were detected by a single-element US transducer (V308, Olympus NDT, USA), amplified by a pulser/receiver (P/R) device (5072PR, Olympus NDT, USA), and stored in a data acquisition (DAQ) module (MSO 5204, Tektronix, USA). Using point-by-point scanning of the US transducer, whole-body PA responses of small animals were successfully visualized in vivo. The system has been applied in various biomedical studies, including lymphography, 56 cystography, 57 tumor imaging, 58 and image-guided therapy. 59 However, the slow imaging speed of the system, approximately 30 min for an imaging region of 60 × 40 mm2, is the main drawback of the system. The breathing motion of the mouse during scanning produced an axial distortion of the PA data. Applying maximum amplitude projection (MAP) to the data could hide the distortion, but it was not a fundamental solution. In addition, the long time interval between wavelengths makes the multispectral analysis inaccurate.
Recently, we demonstrated an updated system capable of simultaneous PA and US data acquisition. 60 While using the configuration, the operation sequence of the system was modified (Figure 2(A)). In the updated sequence, two triggers (one for USI and one for PAI) from the laser were used to synchronize the P/R and DAQ modules. PA and US images of a blood vessel-mimicking phantom showed the feasibility of complement analysis with dual-modal imaging (Figure 2(B)). The different colors of the silicon tubes in the phantom were successfully delineated using multispectral PA images, whereas US images guided the position of the tubes.

Simultaneous dual-modal PA and US imaging system: (A) schematic configuration of the system and (B) photograph, US, PA, and unmixed images of the blood vessel mimicking phantom.
In the following study, we demonstrated US-guided breath compensation using simultaneously obtained PA and US data from mice in vivo. 61 For each cross-sectional B-mode US data, the skin profile was extracted using a simple threshold method (Figure 3(A)). Spiking distortions due to the breathing motion of the mice were successfully corrected by applying a spatial low-pass filter (Figure 3(B)). By calculating the pictorial difference between the skin profiles before and after correction, the PA data could also be corrected (Figure 3(A)). The resulting three-dimensional (3D) PA images showed improved image quality without any breathing distortions (Figure 3(C)). Consequently, MAP images in the x-y, x-z, and y-z planes were successfully acquired, whereas only MAP images in the x-y plane were provided in the previous reports due to the breathing distortion in the axial direction. Moreover, the 3D distribution of hemoglobin oxygen saturation (sO2) was successfully demonstrated through pixel-wise spectral unmixing of multispectral PA data (Figure 3(D)). The successful 3D visualization with US-guided breath compensation showed great potential for analyzing the optical absorption characteristics of small animals in vivo.

US-guided breath compensation for 3D PA images: (A) schematic configuration for the process of US-guided breath compensation, (B) the surface profile of the skin layer before and after breath compensation, (C) 3D and MAP images of whole-body mice in vivo, and (D) 3D sO2 distribution at various angles of view.
Ultrasound-guided photoacoustic imaging with transparent transducers
In the previous PAUSI, the laser beam path was designed to bypass the US transducer. This made the imaging probe bulky and required repetitive beam alignment to achieve an optimized beam delivery. To overcome this limitation, some PAI systems used opto-acoustic combiners to match the paths of optical illumination and acoustic sensing.62–64 However, the proposed imaging probes are bulky and increase system complexity. In addition, opto-acoustic combiners limit the numerical aperture and degrade the signal-to-noise ratio (SNR). Moreover, it was not guaranteed that the obtained PA and US images were in the same plane because of the off-axis problem of the optical and acoustic paths. To overcome these problems, transparent US transducers (TUTs) that can directly transmit PAI excitation light have emerged as an alternative.65–68
Recently, we developed a lithium niobate (LNO) TUT and demonstrated a quadruple imaging system that can provide PAI, USI, fluorescence imaging (FLI), and optical coherence tomography (OCT). 69 In the configuration of the system, four light sources (two for multispectral PAI, one for FLI, and one for OCT) were used and synchronized (Figure 4(A)). The TUT was implemented using a piezoelectric LNO single crystal coated with transparent silver nanowires (Figure 4(B)). The resulting TUT exhibited approximately 70% light transparency with a center frequency of 30 MHz. The feasibility of the quadruple imaging system was evaluated by imaging the eyes of the chemically injured rats. Corneal neovascularization (CNV) and inflammation were induced by applying alkali burns and then quadruple images were acquired to investigate morphological and physiological responses. From the PA images before and after the alkali burn, CNV was verified by analyzing cross-sectional B-mode images and depth-encoded MAP images (Figure 5(A)). We observed that the optically transparent cornea was invisible in the PA images. In contrast, USI visualized morphological changes in the eye, including the appearance of cataracts in the lens, which may cause vision loss (Figure 5(B)). OCT also visualized morphological changes in the cornea by providing structural layers of the eye (Figure 5(C)). After alkali burn, the central corneal thickness increased by 96 ± 31 μm compared with the initial thickness. In FLI, corneal epithelial inflammation, which may cause corneal edema, was visualized by fluorescein staining (Figure 5(D)). By providing multimodal images, the proposed system shows great potential for use in various biomedical studies with a broad impact.

Quadruple imaging system with a TUT: (A) schematic configuration of the system and (B) schematic configuration and photograph of the TUT.

Quadruple images of rat’s eyes using TUT before and after alkali burn: (A) PA MAP, B-scan, and depth-encoded MAP images. The cross-sectional B-scan image showed the generation of CNV (yellow dashed region) alkali burn; (B) cross-sectional US B-scan images; (C) OCT images showing structural layers; and (D) FL images overlaid on the corresponding photographs.
Although TUTs have demonstrated superior performance compared with conventional US transducers, the two most commonly used piezoelectric materials, poly(vinylidene fluoride) (PVDF) and LNO, exhibit inherent limitations. 67 PVDF has low optical transmittance and low sensitivity due to its low electromechanical coupling coefficient and high losses, which limit its usefulness in TUTs. LNO, on the other hand, exhibits high sensitivity due to its high electromechanical coupling, but it is difficult to focus due to its physical characteristics. Therefore, the sensitivity of TUT can be maximized through the discovery of new piezoelectric materials and the establishment of a precise fabrication protocol.
Ultrasound-guided photoacoustic imaging with array transducers
The major drawback of single-element transducer systems is their low imaging speed, which is not feasible for monitoring temporal variation in a large area. To achieve fast imaging, various types of array transducers have been benchmarked. 70 Those systems are typically implemented on USI platforms to use widely available US transducers.71–73 We have also demonstrated clinical PAUSI, which consists of an FDA-approved US machine and a mobile laser system.74,75 After optimization, the system was successfully demonstrated for both clinical human and preclinical small-animal studies with real-time parameter control and a handheld operative imaging probe.76,77
In the small-animal studies, a conventional 128-element linear array US transducer was typically used to achieve both PA and US data simultaneously. By scanning the linear array transducer in the elevational direction, whole-body 3D images were also achieved.78,79 To improve the uniformity of lateral and elevation resolutions, we recently applied a 1024-element two-dimensional (2D) US transducer array for dual-modal 3D PAUSI. 80 The transducer consists of 32 × 32 piezoelectric elements with a pitch size of 300 μm in both lateral and elevational directions (Figure 6(A)). By the 2D array of transducers, MAP images of 9.6 × 9.6 mm2 area can be obtained from four laser pulses because the data acquisition system used 256 receiver channels. To enlarge the imaging area, multiple volumes were merged by scanning the transducer. A wide-field imaging capability was validated by visualizing the abdomen area (45 × 45 mm2) of rats in vivo (Figure 6(B)). To minimize phase aberration due to different sound speed in water and tissue, we applied dual-speed delay-and-sum reconstruction on PA images by estimating the water layer depth from the US images. From the overlaid PA and US images before and after transurethral injection of methylene blue solution, the position of the bladder was identified from the US images whereas PA images were not able to visualize due to the lack of optical absorption (Figure 6(C)). In contrast, PA images clearly verified the presence of methylene blue in the bladder. The results showed the feasibility of the 2D array transducer for multiplex PAUSI with a wide field of view and a uniform spatial resolution.

Dual-modal PA and US images of abdomen area in rats with a 2D matrix US transducer: (A) photograph and schematic illustration of the imaging probe, (B) photograph of the rat with the imaging area, and (C) overlaid PA and US images of the rat before and after transurethral injection of methylene blue solution.
Several clinical studies have also demonstrated that US images can synergistically improve PA images for better quantitative or structural analysis of diseases rather than simply visualizing complementary information. 81 We have applied our PAUSI system to clinical human studies including multiparametric thyroid nodule analysis, 82 melanoma boundary delineation,83,84 and triple-modal (PA, US, and magnetic resonance imaging) angiography. 85 Recently, we reported the usefulness of dual-modal PAUSI for the structural and quantitative analysis of peripheral vasculature in human feet. 86 We designed a custom-made foot scanner to obtain 3D PA and US images of the feet (Figure 7(A)). In particular, the imaging probe scanned the dorsal contours of the feet to improve the reliability of the imaging results. Contours were obtained from pre-scanned 3D US images using a thresholding-based skin detection algorithm. Contour scanning allows equalized optical illumination along the elevation contour of the foot, whereas conventional linear scanning is vulnerable to the arbitrary curvature of the foot. Another synergistic combination of PA and US images was observed from the multistructural visualization of the vasculature, bone, and skin of the feet using multiscale Frangi vesselness filtering, Log-Gabor filtering, and intensity thresholding, respectively (Figure 7(B)). In contrast to the 3D PA image that shows the microvasculature of the foot, the extracted 3D US vascular image could complementarily show large vascular networks. The extracted US bone image provided a landmark to identify each vessel, and the US skin image was used to visualize the depth of each vessel. The US skin image was also used to compensate for the depth-wise optical attenuation of the multi-wavelength PAI (Figure 7(C)). We performed a pressure cuff study to induce venous occlusion in 10 feet of 6 healthy volunteers. The results showed statistically significant changes in the total hemoglobin concentration level and PA vascular density in the big toe area.

Dual-modal PA and US images of human feet in vivo: (A) photograph of the custom-made foot scanner; (B) US skin, US bone, US vessel, and PA vessel images of a human foot in vivo; and (C) optical fluence compensation using the background PA signal segmented using the US skin contour.
Conclusions
We reviewed our results on dual-modal PAUSI systems for US-assisted PA analysis in preclinical small-animal and clinical human studies. The details of the reviewed studies are summarized in Table 1. In the single-element PAUSI system for whole-body imaging of mice, US images provided morphological information about the skin surface, which was used to estimate the depths of signals in PA images and to compensate for breath-induced motion artifacts. The TUT-based quadruple-modal PA/US/OCT/FL imaging system effectively combined all modalities, where US images aided the measurement of the structural differences for quantitative PA analysis. The clinical PAUSI system has been successfully applied in studies on thyroid cancer and peripheral vasculature imaging. The results demonstrated that US images could be used to better determine the region of interest (ROI) and compensate for the heterogeneous optical fluence when analyzing multispectral PA responses. Furthermore, 3D US images of the feet provided various structural features (vasculature, bone, and skin) to support the structural identification of the PA microvasculature and enabled feet contour scans that improved the reliability of the multispectral PAI. Based on these results, dual-modal PAUSI can be regarded as a potential research direction to synergistically improve PA images; therefore, future developments in dual-modal PAUSI techniques are strongly encouraged.
Summary of dual-modal PA and US imaging systems.
PA: photoacoustic; US: ultrasound; PRF: pulse repetition frequency; F: optical fluence; λPA: optical wavelength for PA imaging; DAQ: data acquisition; TR: transducer; TUT: transparent ultrasound transducer; fC: center frequency of TR; NE: number of element of TR; P/R: pulser-receiver; ROI: region of interest; 2D: two-dimensional; 3D: three-dimensional; MAP: maximum amplitude projection; sO2: hemoglobin oxygen saturation; OCT: optical coherence tomography; FL: fluorescence; N/A: not applicable.
The dual-modal PAUSI is currently receiving significant interest from the global research community, as evidenced by the ongoing efforts to optimize systems by utilizing various array transducers and advanced image reconstruction algorithms. Researchers have also explored specific applications, including breast cancer,87–89 thyroid,90,91 cardiovascular, 92 prostate,93,94 and brain imaging,95,96 resulting in promising findings that enable the visualization of microvascular networks, plaque detection, and monitoring of cerebral blood flow. To further enhance the accuracy and quality of PAUSI systems, deep learning methods are being employed to train models on large datasets, enabling the acquisition of complex relationships between PA and US signals.97–100 These advancements hold great potential for improving tissue differentiation and producing more precise image reconstructions.
Optimizing PA image quality has been a key focus of research, but achieving clinical-grade US image quality is also critical for the full utilization of US guidance in PA imaging. Previous studies have demonstrated degraded US image quality due to two potential factors: the presence of a deep water layer between the US transducer and the imaging target, and the limited implementation of image processing algorithms. To maximize light delivery into the tissue, we designed our system to contain a 30 mm deep water layer between the US transducer and the imaging target, allowing direct laser illumination of the target surface. However, the water layer can create heterogeneity in the speed of sound throughout the ROI, potentially inducing phase aberration. Additionally, the small lateral aperture size of matrix transducer arrays can significantly degrade image reconstruction in deep regions, which may be further compounded by the deep water layer. The US images in prior studies were typically reconstructed using basic delay-and-sum algorithms without much post-processing. To address these issues, potential solutions include optimizing light delivery to reduce water layer thickness, dual-speed image reconstruction, accelerating real-time image reconstruction software, using plane-wave mode with coherent angular compounding instead of scanline mode, or implementing speckle reduction imaging techniques such as frequency compounding or smoothing filters.
Overall, PAUSI has demonstrated significant potential for non-invasive imaging of biological tissue. The technology holds promise for various applications in disease diagnosis, monitoring, and treatment. Continued research and development of PAUSI could open up new avenues for the diagnosis and treatment of diseases such as tumors, cardiovascular diseases, and central nervous system disorders. Therefore, further investigation and studies of the dual-modal PAUSI technology are highly recommended.
Footnotes
Authors’ Contributions
All authors participated in the critical reading and writing of the manuscript. JK, BP, and CK conceived and supervised the project and wrote the manuscript. HL and WC prepared, reviewed, and analyzed previously reported articles, prepared figures, and wrote the manuscript.
Declaration Of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Chulhong Kim has a financial interest in OPTICHO, which, however, did not support this study.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the National Research Foundation (NRF) grants (2021R1A5A1032937, 2021M3C1C3097624, 2023R1A2C3004880, RS-2023-00210682), the Korea Medical Device Development Fund (1711137875, RS-2020-KD000008), the Technology Innovation Program (1415181651), and BK21 FOUR project (Pohang University of Science and Technology) funded by the Korean government (the Ministry of Science and ICT; the Ministry of Education; the Ministry of Trade, Industry, and Energy).
