Photo- or optoacoustic (OA) imaging combines optical contrast with ultrasound resolution, enabling high-resolution, real-time in vivo imaging well beyond the 1-mm penetration depth typical of microscopy methods (1, 2). OA has already provided intriguing insights into tumor heterogeneity (3), neuronal dynamics (4), psoriasis (5), and brown fat metabolism (6) based on endogenous contrast from hemoglobin and lipids (7, 8). This is complemented by theranostic research (9, 10) and clinical application (11), e.g., imaging of Crohn’s disease (12). However, OA imaging has not yet become a routine tool in life sciences because of the lack of strong OA contrast agents that can be expressed in desired cell types (13). The few transgenic labels used in OA so far (8) give weak signals that cannot rise above the strong background due to hemoglobin. Photochromic proteins that can be reversibly switched between two states by light can overcome this limitation by entirely separating the label signal, which modulates in accordance with the illumination, from the background, which remains constant (14). This concept, despite being validated in several studies (15–20), has not been implemented widely because it requires complex instrumentation and data analysis tools. Here, we introduce two reversibly switchable OA proteins (rsOAPs) and demonstrate their use with widely accessible off-the-shelf commercial imaging systems as well as our open-access machine learning (ML)–based software code for analysis. One of our new rsOAPs shows high switching speeds and dynamic range of photomodulation that allow us to resolve the signals of different cell populations labeled with differentiable rsOAPs in close proximity in the same animal, demonstrating the potential for simultaneous tracking of different cellular processes through temporal multiplexing.
Engineering of rsOAPs
Bacterial photoreceptors called bacteriophytochromes (BphPs) (21) have emerged as most suitable for rsOAP development due to their strong absorption in the near-infrared range and low photofatigue (22). To identify the most promising candidate for further development, we screened eight native BphPs (table S1) and selected the one from Rhizobium etli. A set of truncations enabled us to minimize its size and optimize its photoswitching characteristics. In brief, on the basis of existing structural data as well as homology models, we created truncations containing the minimum PAS-GAF-PHY photosensory core domains [photosensory core module (PCM)] together with extra amino acids from the annotated linkers between PHY and histidine kinase domains and tested their characteristics in regard to signal generation and photoswitching (Fig. 1, A to E, and note S1). The final variant ReBphP-PCM shows twofold larger change in OA signal (Fig. 2G), more than fivefold faster switching (Fig. 2, C and D), and greater resistance to photofatigue than other rsOAPs (Fig. 2E), while its high molar absorbance is on par with the recently described Deinococcus radiodurans DrBphP-PCM (92,000 M−1 cm−1; Fig. 2, B and G) (19). Those characteristics enable higher numbers of switching cycles per second, which improves sensitivity and allows imaging over longer timeframes. On the molecular level, this acceleration of switching speed is the result of a less stabilized Pfr state favoring the photoinduced transition to Pr. The destabilization is likely caused by an arginine present in ReBphP but not in RpBphP1 and DrBphP. This arginine, by interacting with a conserved aspartate, which, in turn, interacts with the D-ring of the Pfr state chromophore, weakens Pfr stabilization (Fig. 1, F and G, and note S1).
Our truncation strategy also proved successful in obtaining a switchable RpBphP1-PCM from Rhodopseudomonas palustris, in contrast to a previous report that truncated forms of this protein do not undergo reversible switching (19). Our engineered RpBphP1-PCM maintains the far-red state (Pfr) extinction coefficient and photochromic behavior of the parental RpBphP1 (Figs. 1E and 2B and fig. S1), and the change in its OA signal following illumination at 770 nm is similar to that of the previously described DrBphP-PCM (Fig. 2G). (Plasmid for expressing ReBphP-PCM in bacteria and eukaryotic cells or for introduction into viral vectors can be obtained from Addgene.) Both new rsOAPs are monomeric (fig. S2) and show higher expression in mammalian cells than the full-length parental proteins (fig. S3). The two developed rsOAPs and DrBphP-PCM show distinctive switching speeds, which is the reason for our ability to discriminate the proteins in vivo successfully. As a result, probes expressed in different cells in close proximity in the animal can be distinguished during high-resolution OA imaging.
OA imaging using ML-based temporal unmixing
We performed all OA imaging using an off-the-shelf, commercially available multispectral OA tomography device with a 10-Hz pulsed tunable laser and a 256-element transducer array (MSOT, iThera Medical). Off-switching of rsOAPs was achieved with light at 770 nm, which gave the highest difference in OA signal intensity between the “on” and “off” states (fig. S5A), while on switching was achieved using light at 680 nm. Lower wavelengths did not substantially improve the transition to the on state (fig. S5B). The number of laser pulses per wavelength was chosen to cover the full switching kinetics, but it can be significantly reduced using information-content analysis, which allows an estimate of the minimal number of cycles and pulses per cycle required to discern the labeled structure, thus effectively limiting imaging dwell time, which is essential for, e.g., time-resolved studies (Fig. 2H, fig. S6, and note S2). All temporal unmixing was conducted with in-house code developed to analyze time-varying patterns in the reconstructed data in the frequency and time domains using classic ML approaches (Fig. 3, Materials and Methods, and notes S3 and S4). In brief, after running fluence and motion correction on the data, a range of distinctive features was extracted from the photomodulated signal for each voxel of the tomography images. On the basis of a set of these data and corresponding histology as ground truth, a bagged random forest algorithm (23) was trained and validated on independent datasets of a different type to prevent overfitting. The ensuing model was then used to analyze all data in this study. The code for data preparation, for analysis with the model used in this work, and for generation of new models is available to the community along with graphical user interfaces.
The OA imaging scheme is shown in Fig. 4A. First, we used rsOAPs for superficial in vivo imaging. We imaged the development of 4T1 mouse mammary gland tumors coexpressing ReBphP-PCM and green fluorescent protein (GFP) after they were grafted onto the backs of FoxN1 nude mice (n = 3). The initial population of 0.8 × 106 injected cells was readily visualized separate from all background absorbers (Fig. 4B), as was the growing tumor mass at all days after injection (fig. S7, A to D). To test whether this imaging is also possible in brain tissue after light passes through the skull, we implanted 0.7 × 106 4T1 cells coexpressing ReBphP-PCM and GFP at a depth of 3.6 mm and imaged them immediately thereafter. Comparison of the OA images with fluorescence images obtained after sacrificing mice revealed perfect overlap of the labeling, confirming background-free identification of 1.4 × 105 cells deep in the mouse brain (Fig. 4C). Next, we used the same rsOAP to image deep-seated tumors of HCT116 human colon carcinoma cells implanted intraperitoneally (n = 2). From day 3 onward, we were able to visualize the growth of several individual tumor sites to a depth of ~1 cm (fig. S7, E and F). Comparison of OA images and histology obtained after sacrifice confirmed identification of all malignant tissue (Fig. 4, D to F, and fig. S7, E and G), including small tumors or metastatic patches containing less than 10,000 cells (fig. S7, I and J).
To assess the sensitivity of imaging with our rsOAPs, we imaged dorsal implants of Matrigel containing different numbers of Jurkat T lymphocytes stably coexpressing ReBphP-PCM and GFP in mice (Fig. 4G). We detected populations as small as 500 cells/μl, suggesting the potential for sensitive tracking of immune processes. Similarly, imaging of dorsal implants of Matrigel containing bacteria expressing ReBphP-PCM detected populations as small as 14,000 bacteria/μl (Fig. 4H). This sensitivity may be useful for studying and optimizing bacteria-based tumor therapies (24).
A strong advantage of photocontrollable labels is the possibility to delineate multiple labels based on their individual switching kinetics. To demonstrate this, we imaged 1-mm alginate beads filled with Escherichia coli expressing ReBphP-PCM, RpBphP1-PCM, or DrBphP-PCM. All beads were unambiguously identified on the basis of their switching kinetics (Fig. 5A). The same differentiation was achieved in vivo after implanting Jurkat T lymphocytes expressing ReBphP-PCM or DrBphP1-PCM and E. coli expressing RpBphP1-PCM into the back of mice (Fig. 5B).
Because the kinetics of photoswitching are energy dependent, fluence changes due to light attenuation by surrounding absorbers—photochromic or static—complicates temporal multiplexing (note S5). Thus, one aim of our development of the fast-switching ReBphP-PCM was to achieve a switching time constant clearly separate from other rsOAPs. We show that 4T1 tumor expressing ReBphP-PCM and GFP are readily distinguished from infiltrating DrBphP-PCM–expressing E. coli cells (intratumorally injected 108 bacteria; Fig. 5, D and E). This means that multiplexing is possible for co-registration studies and that the concentrations of the labels can be estimated based on the convoluted kinetics (fig. S8). Similarly, we show this for two populations of rsOAP labeled Jurkat T lymphocytes in a 4T1 tumor (intratumorally injected 5 × 105 cells; Fig. 5C). Hence, temporally unmixed multiplexed OA imaging of cells of the immune system enables following their function and involvement in disease mechanism in vivo, longitudinal on the organism level.
The combination of OA and transgenic rsOAP labels allows the tracking of specific cell populations in vivo, which can open up possibilities for longitudinal studies of intact animals in diverse fields such as immunology, developmental biology, neurology, and cancer research. To support these studies, we describe next-generation rsOAPs that provide faster switching and greater resistance to photofatigue than existing rsOAPs, allowing highly sensitive detection, and importantly true multiplexing, without interference from hemoglobin or other abundant absorbers in vivo. These rsOAPs can be used with off-the-shelf equipment and our ML-based open-access image processing code to detect populations of fewer than 500 cells in vivo. The approach relies entirely on a time series of images, thus making the concept translatable between different OA imaging devices. These tools will facilitate the wider use of OA imaging in life sciences, particularly for the study of cellular dynamics and interactions on the level of whole organisms.
MATERIALS AND METHODS
RpBphP1 (16) was obtained from Addgene (V. Verkhusha, plasmid no. 79845). Mammalian optimized ReBphP was synthesized as gene strings (GeneArt, Life Technologies, Regensburg, Germany). All other BphPs used in the study have been a gift from A. Möglich (University of Bayreuth, Germany).
For bacterial protein expression, the coding sequences of all BphPs used in the study except RpBphP1 were polymerase chain reaction (PCR)–amplified as a Nde I/Xho I fragment and cloned into the second multiple cloning site of the pET-Duet1 vector (Novagen, Merck Millipore). RpBphP1 was PCR-amplified as a Nde I/Pac I fragment and cloned into the second multiple cloning site of the pET-Duet1 vector. In addition, for biliverdin synthesis, the heme oxygenase (HO) of Nostoc sp. was cloned using Nco I/Hind III into the first multiple cloning site of pET-Duet1.
For equimolar mammalian expression, first, ReBphP_P2A and mCherry were PCR-amplified and then stitched using overlap PCR as an Eco RI/Xba I fragment and cloned in a pcDNA3.0 vector (Thermo Fisher Scientific). Later similar constructs for other BphPs were made by amplifying them as Eco RI/Not I fragment and inserted in place of ReBphP1-PCM in the above construct. The resulting plasmids allowed the equimolar coexpression of RpBphP1, RpBphP1-PCM, ReBphP-PCM, ReBphP-PCM, or DrBphP-PCM and mCherry proteins.
Protein expression and purification
Proteins have been expressed in E. coli strain BL21 (DE3) (New England Biolabs, #C2527). In brief, plasmids expressing BphPs and HO were transformed into the BL21 host cells. Bacterial cells were grown in LB media supplemented with ampicillin at 37°C until the culture reached OD (optical density) 0.6, followed by induction of protein expression by addition of IPTG (isopropyl-β-d-thiogalactopyranoside) and further incubation for 16 to 18 hours at 22°C. The next day, the bacterial pellet was collected by centrifugation and pellet was resuspended in phosphate-buffered saline (PBS). After cell lysis, proteins were purified by immobilized metal affinity chromatography in PBS, followed by gel filtration on a HiLoad 26/600 Superdex 75 pg (GE Healthcare Life Sciences, Freiburg, Germany).
Absorption and fluorescence spectroscopy
For absorption spectra, the purification buffer was exchanged against PBS and the proteins were measured with a Shimadzu UV-1800 spectrophotometer (Shimadzu Inc., Kyoto, Japan) using a 100-μl quartz cuvette. To measure the ON (Pfr) and OFF (Pr) spectra of respective proteins, photoswitching was carried out using 650/20-nm or 780/20-nm light-emitting diodes (Thorlabs) placed above the quartz cuvette in the spectrophotometer.
Fluorescence measurements for all BphPs were performed with a Cary Eclipse Fluorescence spectrophotometer (Varian Inc., Australia). Photoswitching was carried out as above. Fluorescence measurement was done by fixing excitation wavelength at 700 nm and emission wavelength at 720 nm. Excitation wavelength and emission slit were set to 5 nm, and the absorbance at the excitation wavelength was always equal to 0.1 to avoid inner filter effects.
Mammalian cell culture
4T1 and Jurkat cells were maintained in RPMI 1640. HeLa and HCT116 cells were maintained in Dulbecco’s modified Eagle’s medium (DMEM) and McCoy 5A medium, respectively. All media were supplemented with 10% fetal bovine serum (Invitrogen) and antibiotics [penicillin (100 U/ml) and streptomycin (100 mg/ml)]. Cells were cultivated at 37°C and 5% CO2.
Stable cell lines
Tissue culture. The Platinum-E and RD114 packaging cell lines were cultivated in cDMEM (Complete Dulbecco’s modified Eagle medium), HCT116 cell line was grown in McCoy 5A medium (Life Technologies), and 4T1 and Jurkat cells were cultured in cRPMI (Complete Roswell Park Memorial Institute)–1640 Medium. All media were supplemented with 10% fetal calf serum, 0.025% l-glutamine, 0.1% Hepes, 0.001% gentamicin, and 0.002% streptomycin.
Generation of constructs. ReBphP-PCM-IRES-GFP was amplified using specific primers (5′-ATTAGCGGCCGCGCCACCATGAGCGGCACCAGAG-3′ and 5′-ATTAGAATTCTCACTTGTACAGCTCGTCCATGCCGTGAGTG-3′) and cloned into the mP71 using Not I and Eco RI restriction sites. The mP71 vector was a gift from W. Uckert.
Generation of cell lines. For retrovirus production, Platinum-E or RD114 packaging cells were transfected with the retroviral vector mP71-ReBphP-PCM-IRES-GFP using calcium phosphate precipitation. The supernatant of the packing cells was collected at 48 and 72 hours after transfection and purified from the remaining cells by centrifugation at 1500 rpm at 4°C for 7 min. One day before transduction, non–tissue culture–treated 48-well plates were coated with RetroNectin (Clontech) according to the manufacturer’s recommendations overnight at 4°C. After washing once with PBS, virus supernatant was added and centrifuged at 3000g and 32°C for 2 hours. Virus supernatant was removed, and cell lines (4T1, HCT116, and Jurkat) were added in 400 μl of the respective medium supplemented with 1:100 LentiBOOST Solution A and 1:100 LentiBOOST Solution B (Sirion Biotech). Cells were then spinoculated at 800g at 32°C for 1.5 hours. After 5 days of culture, cells were sorted for high expression of GFP using flow cytometry.
All animal experiments were approved by the government of Upper Bavaria and were carried out in accordance with the approved guidelines. For 4T1 xenografts of stably expressing ReBphp-PCM and GFP, 0.8 × 106 cells in PBS have been implanted in the back of FoxN1 nude mice (Charles River Laboratories, Boston, USA) and maintained for 9 days. For HCT116 cells expressing ReBphP-PCM and GFP, 1.5 × 106 cells in 200 μl PBS have been injected intraperitoneally in FoxN1 nude mice and were maintained for 14 days. For intracranial injections of stably expressing ReBphP-PCM and GFP 4T1 cells, mice were first anesthetized according to the animal protocol. The head of the mouse was fixed in a Stereotaxic frame (David Kopf Instruments, model 940), an incision in the skin was made using a scalpel, and a small hole was drilled into the skull. Later, 5-μl cells (0.14 × 106 cells/μl) were injected slowly with a 10-μl Hamilton syringe (26Gs). The incision in the skin was closed using Histoacryl (B. Braun Melsungen AG). The mice were scanned in MSOT and sacrificed immediately after scanning. For Matrigel implants of Jurkat cells expressing ReBphP-PCM, different concentrations of cells ranging from 6400 to 500 cells/μl were implanted subcutaneously in the back of the mice. Similarly, bacterial cells expressing ReBphP-PCM in different concentrations (1.4 × 105 to 1.4 × 104 cells/μl) were also implanted in the back of the mice. For multiplexing experiment, bacterial cells expressing rsOAPs individually with the concentration of 1.4 × 106 cells/μl were implanted on the back of the mice in the same plane. For multiplexing experiment in vivo, intratumoral injections, bacterial cells expressing DrBphP-PCM resuspended in PBS have been injected into the 4T1 tumor expressing ReBphP-PCM and GFP using an insulin syringe with a 30-gauge needle.
For all MSOT imaging, mice have been anesthetized using 2% isoflurane in O2. Anesthetized mice were placed in the MSOT holder using ultrasound gel and water as coupling media. After termination of the experiments, all mice have been sacrificed and stored at −80°C for cryosectioning.
MSOT setup and data acquisition
Phantom and mice data were acquired using a commercially available MSOT scanner (MSOT In Vision 256-TF, iThera Medical GmbH, Munich, Germany). In brief, nanosecond pulsed light was generated from a tunable optical parametric oscillator (OPO) laser and delivered to the sample through a ring-type fiber bundle. The wavelengths, 680 and 770 nm, were used for photoswitching and imaging in phantoms and in mice. Light absorbed by the sample generates an acoustic signal that propagates through the sample and is detected outside the sample by a cylindrically focused 256-element transducer. The transducer array had a central frequency of 5 MHz (−6 dB was approximately 90%) with a radius of curvature of 40 mm and an angular coverage of 270°. Acoustic signals were detected as time series pressure readouts at 2030 discrete time points at 40 MS/s (Mega-samples per second). The acquired acoustic data were reconstructed using the ViewMSOT version 3.8.1.04 (iThera Medical GmbH, Munich, Germany) software with the following settings: 50 kHz to 6.5 MHz; trim speed of 7.
OA data analysis
All data analysis was conducted using MATLAB2018b. The data reconstructed with ViewMSOT were loaded into MATLAB by iThera MATLAB code (iThera MATLAB, version: msotlib_beta_rev75). All analyses were carried out with the code provided along with this manuscript (note S3). In brief, movement correction was done by phase correlation preliminary to optimization-based image co-registration with the intensity and nonrigid co-registration of frames of the first cycle being used as reference. For further processing, different features of the time series have been computed and are used for classification/switching label detection using an ML model. For fast Fourier transform, repetitive frequency of the whole concatenated signal for each image point is computed to identify signals corresponding to the illumination schedule. For exponential fitting, the normalized mean kinetic of all cycles is used. Then, the coefficients compared to an expected exponential kinetic are calculated and used as a quality measure. Here, positive and negative exponential are considered. Using fit coefficients and quality of fit (R2) as measures, only 77% accuracy compared to a ground truth is achieved. Thus, additional features are invoked. Overall, all analyzed features are (i and ii) the coefficient for the exponential fit (exp(b(x + 1)) and exp(b(x − 1) + 1) of the mean kinetic (mean of all cycles); (iii) R2 of the fit; (iv) the mean intensity over the concatenated signal; (v) max-min of all the data at the pixel; (vi to ix) median maximums and minimums of cycles along with SD; (x) number of cycles with positive or negative trend; (xi) the length of the part of the cycle that shows a trend, i.e., at what point the signal vanishes in the noise; and (xii) Fourier coefficient for the expected frequency defined by the photocontrol schedule. All those are used as predictor values for an unmixing model based on random forest approaches (23, 25)—for overall model, trained on 4T1 day 9 as well as highest concentration of Jurkat T lymphocytes. We used 50 trees in the ensemble, as further increase of number did not lead to out-of-bag error decrease. This approach resulted in model performance increase up to 96% of positive predictive value for ground truth (see note S4 for more details on the use of ML in this work).
For visualization, data were not further processed and are shown against the respective slice at 680 nm as anatomy information, except in the case of 4T1 injected in brain where the anatomy is shown at 900 nm. Representative slices are shown. For clustering, appropriate ranges of the kinetic parameter were chosen on the unmixed data to distinguish different labels.
After sacrificing, the mice were cryopreserved at −80°C. To detect the fluorescence in tumors, the respective part of the mouse was embedded in Tissue-Tek O.C.T. (Sakura Finetek Europe B.V., Zoeterwonde, The Netherlands). Sections (10 μm) were cut (Leica CM1950, Leica Microsystems, Wetzlar, Germany) for brain, 4T1, and HCT116 mice at the interval of 150, 250, and 500 μm, respectively, and imaged using a 482/35-nm bandpass for excitation and 535/38-nm bandpass filter for detection of GFP fluorescence. Images were taken using an Andor LucaR charge-coupled device camera (DL-604M, Andor Technology, Belfast, UK) with 10-s exposure and a gain of 10. On the basis of the histology, ground truth on co-registered images was created using a semiautomatic procedure based on anatomical markers and intensity-based multimodal co-registration together with a nonrigid spline-based method and human (two independent)–based selection of signals in the fluorescence images.
Bacterial cell immobilization by entrapment and MSOT imaging
A 2 to 4% (w/v) aqueous solution of sodium alginate was prepared in PBS. E. coli strain BL21 cells expressing rsOAPs were harvested by centrifugation (4000 rpm, 20 min) and resuspended in PBS. The cell suspensions were then mixed with sterile alginate. Beads were formed by filling the alginate cell mixtures in the syringe with 30-gauge needle, followed by centrifugation at 300 rpm, which allowed the addition of the mixtures into sterile CaCl2 (200 mM). The cell-containing beads, 1 mm in diameter, were allowed to solidify for 10 min before CaCl2 was replaced by fresh distilled water. The cell beads were then randomly distributed in the agar phantom with 1.5% (w/w) agar and 3.5% (v/v) intralipid emulsion and imaged in MSOT as described elsewhere.
OA characterization of proteins
For OA characterization of rsOAPs, custom-made experimental setup was used as described earlier (22). Briefly, nanosecond excitation pulses were generated by an OPO laser (SpitLight DPSS 250 ZHGOPO, InnoLas) running at a repetition rate of 50 Hz. Constant pulse energy was ensured using a half-wave plate in a motorized rotation stage (PRM1Z8, Thorlabs) and a polarizing beam splitter; using a lookup table and adapting the polarization with the half-wave plate, we kept the power constant at ∼1.3 mJ (otherwise mentioned) over the whole illumination schedule. Samples were injected into an acoustically coupled flow chip (μ-Slide I 0.2 Luer, hydrophobic, uncoated, ibidi) and illuminated from one side using a fiber bundle (CeramOptec) at a constant pulse energy of ∼1.3 mJ at the fiber output. Photoswitching was carried out by illuminating the sample alternatively with 650- and 780-nm light. OA signals were detected with a cylindrically focused single-element transducer (V382-SU, 3.5 MHz, Olympus) followed by signal amplification by 60 dB with a wide-band voltage amplifier (DHPVA-100, Femto) and digitized at 100 MS/s with a data acquisition card (RZE-002 400, GaGe). Dependency of Pfr → Pr conversion on 770-nm pulse energy was measured with different pulse energies (0.4, 0.7, 1.0, and 1.3 mJ). Dependency of Pfr → Pr conversion on repetition rate of laser was measured with three different laser repetition rates (10, 25, and 50 Hz). Effect of different switching ON wavelength and resulting dynamic range at 770 nm was measured using different switching ON wavelength ranging from 630 to 680 nm.