Unraveling diurnal and technical variability in cerebral hemodynamics from neurovascular 4D-Flow MRI

Abstract

Neurovascular 4D-Flow MRI enables non-invasive evaluation of cerebral hemodynamics including measures of cerebral blood flow (CBF), vessel pulsatility index (PI), and cerebral pulse wave velocity (PWV). 4D-Flow measures have been linked to various neurovascular disorders including small vessel disease and Alzheimer’s disease; however, physiological and technical sources of variability are not well established. Here, we characterized sources of diurnal physiological and technical variability in cerebral hemodynamics using 4D-Flow in a retrospective study of cognitively unimpaired older adults (N = 750) and a prospective study of younger adults (N = 10). Younger participants underwent repeated MRI sessions at 7am, 4 pm, and 10 pm. In the older cohort, having an MRI earlier on the day was significantly associated with higher CBF and lower PI. In prospective experiments, time of day significantly explained variability in CBF and PI; however, not in PWV. Test-retest experiments showed high CBF intra-session repeatability (repeatability coefficient (RPC) =7.2%), compared to lower diurnal repeatability (RPC = 40%). PI and PWV displayed similar intra-session and diurnal variability (PI intra-session RPC = 22%, RPC = 24% 7am vs 4 pm; PWV intra-session RPC = 17%, RPC = 21% 7am vs 4 pm). Overall, CBF measures showed low technical variability, supporting diurnal variability is from physiology. PI and PWV showed higher technical variability but less diurnal variability.

Keywords

4D-Flow MRI CBF pulsatility pulse wave velocity diurnal variations

Introduction

For a comprehensive characterization of cerebral hemodynamics, 4D-Flow MRI (temporally-resolved 3D PC MRI) has emerged as a powerful technology that allows measures of the entire cerebral-macro vasculature within a single MRI scan.^1
–4 In addition to cerebral blood flow (CBF), 4D-Flow enables assessment of other hemodynamic parameters including cardiac pulsations, cerebral arterial stiffness, autoregulation, and others.^5
–7 These measures have been used to visualize vascular lesions such as aneurysms⁸ and arteriovenous malformations,⁹ and to quantitatively characterize global neurovascular changes, such as identifying reduced cerebral blood and stiffer vessels in Alzheimer’s disease.^10,11 4D-Flow has also shown good agreement with transcranial Doppler (TCD) when measuring middle cerebral artery (MCA) velocity during gas challenges and have revealed aging effects on cerebral hemodynamics that were not observable using TCD.^12
–14 Despite the apparent high sensitivity of 4D-Flow to disease change, there have been few studies that aimed to understand and disentangle the sources of variation in these measures from technical and physiologic sources. Physiologic sources are of particular interest as MRI scans often require scheduling throughout the day and are acquired with patients in the supine position. In particular, diurnal variability on cerebral hemodynamics using 4D-Flow has not been previously established.

Circadian rhythms are natural physiological changes that follow a 24-hour cycle. These processes regulate cardiovascular function including diurnal changes in blood pressure and heart rate.¹⁵ Circadian rhythms affect cerebral hemodynamics and have been implicated in brain metabolite waste clearance and cerebrovascular integrity.^16
–19 For example, disruption of circadian rhythms have been associated with various neurological disorders including proteinopathies such as Alzheimer’s disease, and stroke risk and therapy outcomes.^19
–21 Accurate and precise characterization of cerebral hemodynamics in the context of circadian rhythms and diurnal variations is therefore essential to study mechanisms of pathophysiology, improved diagnosis, and treatment planning of various neurological disorders. Thus, there is both an opportunity to study cerebral hemodynamics in the context of circadian rhythms to improve understanding of disease state and the confounding influence diurnal changes might have when it is impractical to control for time of day in scanning (e.g., large-scale national clinical studies).

Our current understanding of diurnal variations effects on cerebral hemodynamics have been largely determined from studies that measured CBF velocities using TCD.^22
–25 TCD is accessible and portable, and allows for non-invasive assessment of CBF velocities. Using TCD, diurnal studies have observed variations in cerebrovascular reactivity²⁴ and CBF velocities that follow a 24-hour cycle.²⁶ However, TCD has well-known limitations including lack of CBF measures (only velocities) because it cannot inform on vessel lumen areas and is limited by the acoustic bone window restricting cerebral vessel analysis to those close to the skull (e.g., MCA). Alternatively, 2D phase contrast (PC) MRI can be used to characterize diurnal variability in CBF,²⁷ overcoming both of these limitations of TCD, but requiring manual prescription of planes orthogonal to the vessel direction and another MRI for plane placement planning.

In this work, we characterized diurnal and technical variability in cerebrovascular hemodynamics from neurovascular 4D-Flow including measures of CBF, vessel pulsatility index, and cerebral pulse wave velocity in complementary analyses of two sets of cognitively unimpaired participants. Specifically, we analyzed a large data repository of existing cross-sectional 4D-Flow data from cognitively unimpaired older adults (N = 750) who were sampled between 7am and 5 pm. We conducted additional analyses on a small, prospective test-retest study of healthy younger adults (N = 10); diurnal variability in cerebral hemodynamics was estimated from imaging studies at 7am, 4 pm, and 10 pm; in addition, the prospective study design facilitated examination of measurement variability associated with repositioning and test-retest conditions.

Materials and methods

Subjects

Two cohorts were investigated in this study including a large cohort (N = 750) of cognitively unimpaired older individuals with 4D-Flow collected as part of ongoing aging studies, and a small cohort (N = 10) of young individuals who underwent same day repeated scanning to study technical and diurnal variations. Data of the larger cohort consisted of cross-sectional observations only, limiting the study of sources of variability in 4D-Flow hemodynamics. For example, we could not rule out those who choose afternoon MRI appointment slots were systematically different in some way from those who choose morning slots. For this reason, we performed prospective studies on a smaller cohort with repeated scans throughout the day.

Existing 4D-Flow processed data from 750 cognitively unimpaired older participants (age range 46–93 y, mean 65 ± 8 y, 502 female) from the Wisconsin Alzheimer’s Disease Research Center (WADRC) and the Wisconsin Registry for Alzheimer’s Prevention (WRAP)²⁸ were re-analyzed to study diurnal variability in cerebral hemodynamics. This cohort has been used in prior studies investigating the influence of age and sex on CBF and vessel pulsatility.²⁹ Participant cognitive status was diagnostically characterized after each comprehensive neuropsychological assessment visit via each cohort’s multidisciplinary consensus conference using applicable clinical, laboratory, and imaging criteria.^30
–35 All participants were clinically diagnosed to be cognitively unimpaired at visit closest to 4D-Flow MRI collection. Exclusion criteria included significant medical conditions such as major systemic illness also at visit closest to MRI collection. Fasting and caffeine abstention were not required but recommended, and time since fasting and caffeine consumption were recorded on a participant questionnaire during the MRI visit.

In the smaller prospective study, 4D-Flow MRI data from 10 cognitively unimpaired younger participants (age range 21–32 y, mean 27 ± 3.2 y, 3 female) were collected. Participants were recommended to follow their standard daily routines on the day of the experiment. Volunteers underwent an MRI session at three different times of the day including 7am, 4 pm, and 10 pm during a weekday. During each session, test-retest imaging was also performed. Three 4D-Flow MRI scans were acquired to study the effects of repositioning and back-to-back scans (e.g., 3 scans at 7 am, 3 at 4 pm and 3 at 10 pm). After the first 4D-Flow scan, volunteers were taken outside the MRI suite for 5 minutes and then repositioned in the scanner for back-to-back 4D-Flow scans resulting in a total of 9 4D-Flow MRI scans for each participant.

The University of Wisconsin Institutional Review Board approved all study procedures and protocols following the policies and guidance established by the campus Human Research Protection Program (HRPP). Each participant signed a written informed consent before participation.

Imaging protocol

The 4D-Flow MRI protocol used in the larger cohort dataset is described elshewere²⁹ and matched to the MRI protocol used in the prospective study. Prospective MRI data were acquired on a 3.0T system (SIGNA Premier, GE Healthcare) using a noncontrast-enhanced, radially undersampled 4D-Flow acquisition^36,37 and a 48-channel head-coil (GE Healthcare). Scan parameters included: acquisition imaging volume = 22 × 22 × 16 cm³, TR/TE = 8.6/2.5 ms, number of projections ∼11,000, scan time ∼5.6 min, acquired spatial resolution = 0.7 mm isotropic, flip angle = 8°, and V_enc = 80 cm/s. Cardiac triggers were collected for each participant from a photoplethysmogram on a pulse oximeter (GE Healthcare) worn on the volunteer’s finger during the MRI exam. In repeated imaging participants, cuff-based blood pressures (Veris Vital Signs, Medrad) were measured in each session before the first and after the last scan.

Image post-processing

For both study samples, after scan acquisition, velocity, magnitude, and angiogram images were retrospectively reconstructed into 20 cardiac time frames using cardiac triggers.^38,39 Image corrections were performed including background phase offsets and velocity anti-aliasing.^40,41 A validated MATLAB-based (Matlab v2023a; Mathworks) tool (https://github.com/uwmri/QVT) was used to process 4D-Flow data.^42,43 Semi-automatic processing included global thresholding, centerline skeletonization, orthogonal cut-plane generation, in-plane k-means clustering segmentation, and hemodynamic parameter extraction at each centerline point along the segmented vessel tree. To reduce measurement noise, quantitative values were spatially averaged over five neighboring centerline points. Pulsatility index (PI) and cardiac-cycle-averaged volumetric flow rates were obtained in the cervical internal carotid arteries (ICAs), basilar artery (BA), and MCA M1 segment. Bilateral vessels measures were averaged. Vessel PI was defined as the difference between peak systolic and minimum diastolic flow rates, normalized by the mean flow over the cardiac cycle. PI is recognized as a measure of distal flow resistance and vascular wall compliance; however, is also associated with other hemodynamic variables (e.g., cerebral perfusion pressure).^44,45 Total CBF was estimated as the sum of flow in the ICAs and basilar artery. More recent technical developments have enable estimates of cerebral pulse wave velocity (PWV) measures from neurovascular 4D-Flow MRI.^5,10 PWV is recognized as the gold standard non-invasive biomarker of arterial stiffness, a significant risk factor for cardiovascular disease and mortality.^46,47 Leveraging these technical developments cerebral PWV was derived from 4D-Flow in the small prospective participants sample. Cerebral PWV were derived using a maximum likelihood estimator.⁵ PWV was fitted using velocity waveforms and phase shifts from cross-sections along the vascular length. Vascular distance was estimated from user selected seeds points to all consecutive centerline points using Euclidian distance between centerline points. Small distal arterial branches were removed keeping only the main vascular routes from the most proximal to the most distal end of the vasculature as demonstrated previously by others.⁵ Seed points were placed at the most inferior aspect of the ICAs tracking a vessel route that included ICAs, MCAs M1, M2, and M3 segments.⁵

Statistical analysis

Retrospective cohort analyses

Stepwise linear regression models were used to identify which covariates significantly explained total CBF and PI variability in the older cohort (N = 750). Covariates included age, sex, and several variables measured at the time of the scan, including heart rate, time of day, fasting time, and time of last caffeine consumption. A criterion of P < 0.05 in the F-test was used for adding terms (i.e., forward) to the final model. In secondary analyses, linear regression models and Pearson correlation coefficients were estimated for age, time of day, fasting time, and time since last caffeine consumption to study their separate contributions to total CBF and ICA PI variability. Time since last caffeine consumption was set to 24 hrs for any participant with a longer time, under the assumption that caffeine half-life is 5 hours and less than 3.6% of caffeine remains in the body after 24 hrs.⁴⁸

Prospective cohort analyses

In the younger cohort (N = 10), linear mixed effects (LME) models were used to determine fixed effects from time of day, heart rate, and pulse pressure defined as the difference between systolic and diastolic blood pressure, and random effects from intercepts grouped by participants. Likelihood ratio tests were used to determine if LME models for total CBF, PI and PWV were significantly improved by incorporating time of day as a predictor variable. To determine differences across time of day, pair-wise comparisons were performed using an F-test on the fixed-effects estimated coefficients and confidence intervals. Quantile-quantile plots of LME models residuals were used to support assumptions of normality. To determine how much of the observable variability in cerebral hemodynamic markers is between-volunteer as opposed to within-volunteer, intra-class correlation coefficients were estimated from using $\frac{σ_{volunteer}^{2}}{σ_{volunteer}^{2} + σ_{error}^{2}}$ , where $σ_{volunteer}^{2}$ and $σ_{error}^{2}$ are the variances due to volunteer and standard error from random effects. Variance differences intra- and inter-session were assessed using Levene’s test. Regressions were used to linearly fit intra- and inter-session data and Pearson correlation coefficients were estimated. Bland-Altman analyses were performed to estimate mean differences, agreement intervals, bias, and repeatability coefficients (RPC) between hemodynamic markers for intra- and inter-sessions.⁴⁹ RPC was defined as (1.96 $σ$ )*100 where $σ$ is the standard deviation of the differences (d = y1−y2) each expressed as a proportion of their mean (m = (y1 + y2)/2).⁵⁰ Statistical analyses were performed in MATLAB. P < 0.05 was set as the threshold for statistical significance.

Results

Retrospective study of older cohort

From the covariates examined, stepwise linear regressions in the older cohort revealed only age and time of day significantly explained the observed variability in total CBF and PI in the ICAs and MCAs (Table 1). Secondary analyses (Table 1 and Figure 1) examining covariates separately showed significant associations between total CBF and age (a), time of day (c), fasting time (e), and time since last caffeine (g). For ICAs PI, significant associations were observed with age (b), time of day (d) and fasting time (f), but not with time since last caffeine (h). In separate covariate analyses, MCAs PI was also significantly associated with age, time of day, and fasting time (Table 1).

Table 1.

Summary of stepwise and separate linear regressions evaluating the effects of various covariates on total cerebral blood flow and vessel pulsatility index in the older cohort (N = 750).

	Estimate	P-value	R²
Stepwise linear regression
Total Cerebral Blood Flow (mL/min)			0.14
Intercept	865 ± 31 (805, 926)	<0.001	–
Age (years)	−4 ± 0.5 (−5, −3)	<0.001	–
Time of day (hour)	−8 ± 1 (−11, −5)	<0.001	–
Vessel Pulsatility Index ICA			0.38
Intercept	0.067 ± 0.079 (−0.088, 0.222)	0.394	–
Age (years)	0.015 ± 0.001 (0.013, 0.017)	<0.001	–
Time of day (hour)	0.009 ± 0.003 (0.002, 0.015)	0.012	–
Time since last caffeine (hours)	0.001 ± 0.001 (−0.001, 0.003)	0.336	–
Vessel Pulsatility Index MCA			0.27
Intercept	0.232 ± 0.059 (0.116, 0.348)	<0.001	–
Age (years)	0.013 ± 0.001 (0.011, 0.015)	<0.001	–
Time of day (hour)	0.009 ± 0.003 (0.004, 0.014)	<0.001	–
Linear regression examining covariates separately
Total Cerebral Blood Flow (mL/min)			–
Intercept	805 ± 29 (747, 863)	<0.001	–
Age (years)	−4 ± 0.5 (−5, −3)	<0.001	0.10
Intercept	639 ± 16 (607, 671)	<0.001	–
Time of day (hour)	−10 ± 1.5 (−13, −7)	<0.001	0.06
Intercept	487 ± 8 (470, 503)	<0.001	–
Fasting time (hours)	4 ± 0.8 (3, 6)	<0.001	0.06
Intercept	483 ± 11 (461, 504)	<0.001	–
Time since last caffeine (hours)	3 ± 0.6 (1, 4)	<0.001	0.05
Vessel Pulsatility Index ICA			–
Intercept	0.163 ± 0.047 (0.070, 0.256)	<0.001	–
Age (years)	0.014 ± 0.001 (0.013, 0.016)	<0.001	0.35
Intercept	0.895 ± 0.031 (0.835, 0.955)	<0.001	–
Time of day (hour)	0.019 ± 0.003 (0.014, 0.024)	<0.001	0.06
Intercept	1.16 ± 0.02 (1.13, 1.19)	<0.001	–
Fasting time (hours)	−0.005 ± 0.001 (−0.008, −0.002)	0.001	0.02
Intercept	1.14 ± 0.02 (1.10, 1.18)	<0.001	–
Time since last caffeine (hours)	−0.001 ± 0.001 (−0.003, 0.002)	0.584	0.00
Vessel Pulsatility Index MCA			–
Intercept	0.302 ± 0.056 (0.193, 0.412)	<0.001	–
Age (years)	0.013 ± 0.001 (0.012, 0.015)	<0.001	0.26
Intercept	1.00 ± 0.03 (0.94, 1.07)	<0.001	–
Time of day (hour)	0.015 ± 0.003 (0.009, 0.021)	<0.001	0.03
Intercept	1.21 ± 0.02 (1.18, 1.25)	<0.001	–
Fasting time (hours)	−0.004 ± 0.002 (−0.007, −0.001)	0.009	0.01
Intercept	1.22 ± 0.02 (1.17, 1.26)	<0.001	–
Time since last caffeine (hours)	−0.002 ± 0.001 (−0.004, 0.001)	0.209	0.00

Covariates included age, time of day, fasting time, and time since last caffeine consumption. Estimate data are ± standard error; data in parentheses are 95% CIs. ICA: internal carotid arteries; MCA: middle cerebral arteries. Bold indicates statistical significance (P < 0.05).

Prospective study of younger cohort

Physiological measures

Heart rate and blood pressure markers acquired across time of day in the younger cohort are summarized in Supplementary Figures 1, 2, and 3. Likelihood ratio tests from LME models (Supplementary Figure 2) showed time of day significantly improve model prediction of heart rate (P = 0.008), but not of systolic and diastolic blood pressure. Heart rate was significantly lower at 4 pm (P = 0.007) and 10 pm (P = 0.007) when compared to 7am. Heart rate variance was significantly lower at 10 pm compared to 7am (P = 0.009) and 4 pm (P = 0.009).

4D-Flow MRI cerebral hemodynamic measures

Total CBF, ICA and MCA PI, and PWV measures derived from 4D-Flow MRI data across time of day in volunteer experiments are summarized in Figure 2, Table 2, and Supplementary Figure 2. Total CBF variance was significantly higher at 7am compared to 10 pm (P = 0.009) (Figure 2(a)). Variance differences in ICA and MCA PI (b, c) and PWV (d) were not significant. Likelihood ratio tests from LME models (Supplementary Figure 2) showed time of day significantly improve model prediction of total CBF (P < 0.001), ICA PI (P = 0.035), and MCA PI (P = 0.006), but not of PWV (P = 0.223). Inspection of quantile-quantile plots of the residuals supports models assumption of normality (Supplementary Figure 2). Pair-wise tests (Table 2) show significant decreases in total CBF from 7am to 4 pm (P = 0.001), and 7am to 10 pm (P < 0.001). Heart rate was positively associated to total CBF (P < 0.001) and PWV (P = 0.005). MCA PI increased from 7am to 4 pm (P = 0.012). PI decreased from 4 pm to 10 pm in the ICA (P = 0.012) and MCA (P = 0.003). Higher pulse pressure was associated with higher ICA PI (P = 0.007) and MCA PI (0.015). Intra-class correlation coefficients were 0.83, 0.77, 0.23, and 0.29 for total CBF, ICA PI, MCA PI, and PWV respectively.

Table 2.

Linear mixed effects models, intra-class correlation coefficients, and pair-wise comparisons for volunteer experiments assessing effect of time of day, heart rate, and pulse pressure on total cerebral blood flow, ICA and MCA vessel pulsatility index, and cerebral pulse wave velocity.

Parameter	Estimate	P-value	Intra-class correlation
Total Cerebral Blood Flow (mL/min)			0.83
Intercept	−12 ± 64 (−141, 116)	0.845	–
Time of day (7am vs 4 pm)	−35 ± 10 (−56, −14)	0.001	–
Time of day (7am vs 10 pm)	−41 ± 10 (−61, −20)	<0.001	–
Time of day (4 pm vs 10 pm)	–	0.546	–
HR (bpm)	9 ± 0.7 (8, 10)	<0.001	–
PP (mmHg)	1.2 ± 0.7 (−0.2, 2.7)	0.089	–
ICA Vessel Pulsatility Index			0.77
Intercept	0.754 ± 0.091 (0.573, 0.936)	<0.001	–
Time of day (7am vs 4 pm)	0.028 ± 0.015 (−0.003, 0.058)	0.075	–
Time of day (7am vs 10 pm)	−0.010 ± 0.015 (−0.040, 0.020)	0.523	–
Time of day (4 pm vs 10 pm)	–	0.012	–
HR (bpm)	0.002 ± 0.001 (0.000, 0.004)	0.103	–
PP (mmHg)	0.003 ± 0.001 (0.001, 0.005)	0.007	–
MCA Vessel Pulsatility Index			0.23
Intercept	0.981 ± 0.148 (0.688, 1.27)	<0.001	–
Time of day (7am vs 4 pm)	0.079 ± 0.031 (0.018, 0.140)	0.012	–
Time of day (7am vs 10 pm)	−0.011 ± 0.031 (−0.072, 0.050)	0.723	–
Time of day (4 pm vs 10 pm)	–	0.003	–
HR (bpm)	−0.002 ± 0.002(−0.005, 0.001)	0.215	–
PP (mmHg)	0.004 ± 0.002 (0.001, 0.007)	0.015	–
Cerebral Pulse Wave Velocity (m/s)			0.29
Intercept	5.20 ± 0.48 (4.26, 6.16)	<0.001	–
Time of day (7am vs 4 pm)	−0.007 ± 0.096 (−0.197, 0.183)	0.942	–
Time of day (7am vs 10 pm)	−0.145 ± 0.095 (−0.335, 0.045)	0.132	–
Time of day (4 pm vs 10 pm)	–	0.139	–
HR (bpm)	0.016 ± 0.005 (0.005, 0.026)	0.005	–
PP (mmHg)	−0.006 ± 0.005 (−0.016, 0.004)	0.266	–

Estimate data are ± standard error; data in parentheses are 95% CIs. HR: heart rate; ICA: internal carotid arteries; MCA: middle cerebral arteries; LME: linear mixed effects; PP: pulse pressure. Bold indicates statistical significance (P < 0.05). Estimate coefficients and CIs from the LME were used for evening vs night comparisons.

Bland-Altman analyses, linear regressions and Pearson correlations studying the effects of repositioning (i.e., scans 1 vs 2) and back-to-back on test-retest scans (i.e., scans 2 vs 3 for intra-session analyses) are summarized in Figure 3. Total CBF displayed greatest repeatability within a session (Figure 3(a) and (d)). Similar reproducibility coefficients were measured for total CBF for repositioning and back-to-back scans with values of 8.2% and 7.2%, respectively. A significant negative bias (−4.6%, P < 0.001) was measured after repositioning. PWV was less repeatable than total CBF with reproducibility coefficients of 14% (c) and 17% (f) for repositioning and back-to-back scan conditions, respectively. PI was less repeatable than total CBF and PWV (repositioning RPC = 24% (b), back-to-back RPC = 22% (e)). Summary of Bland-Altman analyses studying the effects time of day in 4D-Flow markers are displayed in Figure 4. PWV was most repeatable across time of day with RPCs of 21% (4 pm vs 7am) (c), 16% (10 pm vs 7am) (f), and 17% (10 pm vs 4 pm) (i). PI was more repeatable than total CBF across time of day (RPC = 24% 4 pm vs 7am (b), RPC = 27% 10 pm vs 7am (e), RPC = 24% 10 pm vs 4 pm (h)). Total CBF was the least repeatable marker across time of day with RPCs values of 40% (4 pm vs 7am) (a), 34% (10 pm vs 7am) (d), 32% (10 pm vs 4 pm) (g). Significant negative biases were observed for total CBF at 4 pm vs 7am (−12%, P = 0.003) (a), and 10 pm vs 7am (−12%, P < 0.001) (d). A significant positive bias was observed for PI at 4 pm vs 7am (4.5%, P = 0.006) (b), and negative bias at 10 pm vs 4 pm (−5.6%, P < 0.001) (h). Finally, a negative bias was observed for PWV at 10 pm vs 7am (−3.6%, P = 0.020) (c).

Discussion

In this work, we characterized the effects of diurnal and technical variations in human cerebral hemodynamics measures from neurovascular 4D-Flow MRI. Data from a large sample of cognitively unimpaired older adults (N = 750) from ongoing aging studies were retrospectively analyzed to study if time of day significantly explained variability in total CBF and vessel PI. Stepwise regression models showed time of day and age as the only covariates that significantly explained variability in total CBF and vessel PI in the cohort. While the relationship of CBF and PI with age was previously reported for this cohort,²⁹ the relationship with time of day is a new finding. Higher CBF and lower PI were associated with a participant having an MRI earlier on the day. However, secondary analyses also showed longer times since fasting and last caffeine consumption were significantly associated with higher total CBF. We also assessed diurnal variability in cerebral hemodynamics in a prospective study of healthy younger participants (N = 10) imaged at 7am, 4 pm, and 10 pm on the same day. Technical variability including effects of repositioning and back-to-back scans were also characterized from test-retest experiments during each session. In these studies, linear mixed effects modeling demonstrated significant fixed effects from time of day on total CBF and vessel PI but not in cerebral PWV. Other covariates such as heart rate and pulse pressure also explained variability in hemodynamic markers. Bland-Altman analyses revealed high intra-session repeatability for total CBF but substantially lower repeatability between imaging sessions throughout the day (RPC = 7.2% back-to-back, RPC = 40% 4 pm vs 7am). PI and PWV showed lower intra-session repeatability but higher repeatability throughout the day when compared to CBF (PI RPC = 22% back-to-back, RPC = 24% 4 pm vs 7am; PWV RPC = 17% back-to-back, RPC = 24% 4 pm vs 7am).

Our results suggest that diurnal variations are a significant contributing factor to the variability in cerebral hemodynamics from 4D-Flow MRI and that these variations need to be considered in the design and analysis of 4D-Flow studies. Using a state-of-the-art 4D-Flow sequence, the technical variability in CBF measures was much lower than the physiological variation across scan sessions. This alone suggests that the variability seen in studies, such as studies comparing CBF reductions in Alzheimer’s and with aging, may be significantly influenced by confounding physiologic processes. For example, from our diurnal variability data in the older cohort, a Cohen’s f² = 0.06 (using R² = 0.06) indicates a small effect size on CBF (e.g. f² >0.02 small, f² >0.15 medium, f² >0.35 large effect).^51,52 Using CBF data from studies on Alzheimer’s patients⁶ we can estimate a Cohen’s f²∼0.41, indicating having a diagnosis of dementia due to Alzheimer’s disease has a large effect size on CBF. Even though the effects of diurnal variability on CBF seem to be small in relation to disease-related effects, they are significant and should be accounted for improved precision of disease models, especially for multi-etiology dementia. Importantly, this comparison is limited by the lack of repeated measures throughout the day in the older cohort. Repeated CBF measures from the prospective study in a small number of young participants demonstrated variance levels of ∼13% attributed to diurnal variability, and ∼10% change in mean CBF between morning and evening exams. This data suggests CBF effects from diurnal and technical variability could be potentially larger than indicated by Cohen’s f² from our older cohort. This also indicates that accounting for diurnal variability would improve the statistical power of 4D-Flow studies and improve our characterization of cerebrovascular hemodynamic heterogeneity. The derived measures of vessel PI and cerebral PWV were less sensitive to diurnal variability suggesting their signal relates to vessel features that are less variable throughout the day. However, these markers displayed higher technical variability likely from processing, image noise, and other imaging artifacts. Thus, future studies are needed to reduce technical variability in PI and PWV markers including new 4D-Flow sequences, and emerging deep learning techniques^53
–55 which aim to reduce noise and improve overall image quality.

A number of studies have characterized time of day and day-to-day variations in cerebral hemodynamics including measures of CBF velocities, autoregulation, and cerebrovascular reactivity (CVR).^22
–25,27 Some studies have identified diurnal variations in CBF velocities that follow 24 hour rhythms indicative of circadian regulation.²⁶ Others have measured diurnal variability in CVR from MCA velocity response to carbogen exposure, observing highest values during evening compared with morning hours.²⁴ While most studies have measured CBF velocities with TCD, there is a limited amount of scientific literature on human studies assessing diurnal variability on cerebral hemodynamics using MRI techniques.^27,56 However, MRI systems can offer unique advantages and reveal changes in cerebral hemodynamics not observable with TCD.¹² Using MRI, diurnal variability in cerebral aqueduct and fourth ventricle cerebrospinal fluid (CSF) flow (an important component of brain fluid dynamics) have been observed in the past, with aqueduct CSF flow motion peaking at night and during sleep.^18,56 With 2D PC MRI, researchers have repeatedly measured total CBF during different days to study intra-session and day-to-day variability.²⁷ The researchers concluded intra-session and short-term day-to-day variability of total CBF is relatively low compared to between-subject variability under standardized conditions. However, researchers highlighted the need to include covariates such as hemoglobin and caffeine plasma levels in analyses as they explained 20% to 35% of total CBF variability. The effect of dietary caffeine consumption in CBF have also been studied using arterial spin labeling (ASL) MRI,⁵⁷ strongly supporting dietary caffeine consumption as confounding variable in cerebral perfusion and functional MR imaging. In those studies, time of day at which MRI data were acquired were not reported. In our study, secondary analyses also showed an association between caffeine and total CBF, with lower CBF associated with a more recent use of caffeine. To expand beyond these works, here we studied diurnal variation of cerebral hemodynamics using neurovascular 4D-Flow MRI. To the best of our knowledge, this is the first study to do so and include other hemodynamic markers beyond CBF. In our retrospective data analysis in older adults (N = 750), our primary analysis revealed a significant effect of time of day in CBF measures, with higher CBF in the morning than in the afternoon. These observations agree with previous studies that measured 24-hour circadian rhythms in CBF velocities, with higher velocities in the morning and lower velocities in the afternoon.²⁶ Distinctive from that study that measured velocities, in our study we measured blood flow. The observations in the older adult cohort are supported by results from prospective studies in younger volunteer (N = 10) scanned on the same day at 7am, 4 pm, and 10 pm with test-retest exams to assess repeatability. From those data we observed high intra-session repeatability and significant effects from time of day on total CBF. We also measured significant effects from heart rates on CBF. This result is not surprising as others have measured effects from heart rate in MCA and ICA velocities and CBF using TCD and near-infrared spectroscopy.⁵⁸

Here, we also studied diurnal variability on vessel PI and cerebral PWV (young cohort only). While CBF is an important marker to study neurovascular coupling and brain perfusion, markers that can provide a more direct assessment of cerebral vessel function are of high interest for the study of cerebrovascular diseases. For example, vessel PI has been proposed as an diagnostic marker of small vessel diease⁵⁹ and PWV is still considered the gold standard marker of arterial stiffness^46,47,60 In our study, linear mixed effects models revealed time of day and pulse pressure significantly contributed to vessel PI variability. The pulse pressure and PI relationship is not surprising as the latter is related to various hemodynamic parameters (e.g. cerebral perfusion pressure).⁴⁵ Cerebral PWV measures were not significantly affected by time of day, but were sensitive to heart rate variability. Other studies have identified associations between heart rate and aortic and brachial PWV measures in animal and human studies,^61,62 indicating a potential correlation between our cerebral PWV measures and traditional aortic and brachial PWV. Intra-class correlation coefficients from mixed effects models were lowest for MCA PI and PWV, suggesting larger within-volunteer variability in these parameters. This observation might reflect higher measurement variability in smaller arteries (e.g. MCA PI vs ICA PI), and in parameters requiring higher temporal resolution such as cerebral PWV. We also characterized intra-session variability and the effects of repositioning and back-to-back scans in CBF, vessel PI, and cerebral PWV. Repeatability coefficients from Bland-Altman analysis showed high intra-session repeatability of CBF measures, supporting low technical variability. Repositioning effects were small compared to back-to-back scans. However, we measured a negative bias in CBF from repositioning. This result is explained by a significant effect of heart rate on CBF as measured in linear mixed effects. Volunteers displayed on average a higher heart rate on the first scan compared to the second scan after repositioning (67 ± 11 vs 64 ± 10 bpm), likely due to getting accustomed to the scanning environment. Together, these findings indicate large diurnal variability in CBF measures are primarily from physiological origin. Intra-session repeatability and intra-class correlation coefficients for vessel PI and cerebral PWV were lower than for CBF, indicating PI and PWV are more sensitive to technical variability and noise. However, intra-session and diurnal (inter-session) repeatability of PI and PWV were similar and higher than diurnal repeatability of CBF, indicating PI and PWV are less sensitive to diurnal variations than CBF. These findings support vessel PI and especially cerebral PWV as markers more specific to cerebral vessel features than CBF measures. However, we have to note the smaller sample size of the prospective study (N = 10), which encourages replication in larger cohorts.

This work has a few limitations. The study on cognitively unimpaired older adults was cross-sectional; however, statistical analyses considered various covariates including age, sex, heart rate, time of day, fasting time, and time of last caffeine consumption. Blood pressure data were not collected during the MRI examination of the older participants, and PWV measures were not available for analysis. Records of time since fasting and caffeine consumption were only available in 574 and 355 of 750 participants respectively. Ideally plasma levels of caffeine would be utilized to quantify the effects of caffeine in cerebral hemodynamics; however, this was not a primary outcome of this study. Young participants were asked to maintain their daily routines on the day of the experiment. One volunteer (volunteer #7) consumed caffeine before the 7am scan, and four (#1, #4, #5, #8) before the 4 pm scan. Ideally participants would be tracked and maintained under strict similar conditions the day before and during experiments. However, the main outcome of this study was to establish the effects of technical and physiological variability on 4D-Flow based markers of cerebral hemodynamics including CBF, vessel PI, and cerebral PWV. Finally, volunteer data were acquired at 3 time points during the day. Additional time points throughout the 24-cycle including during sleeping conditions are necessary to assess the effects of the whole circadian rhythm in cerebral hemodynamics.

In this work, we utilized neurovascular 4D-Flow MRI to study diurnal variations in cerebral hemodynamics in cognitively unimpaired participants. We also investigated the effect of repositioning and back-to-back scans on measures’ repeatability. Our results revealed significant total CBF and vessel PI fluctuations throughout the day in older and younger participants. Intra- and inter-session variability analysis supports CBF fluctuations are from physiological origin. Vessel PI and cerebral PWV displayed similar variability intra-session and throughout the day indicating higher technical variability, but less sensitivity to diurnal variability compared to CBF. These findings support that experiments measuring CBF using 4D-Flow MRI may need to control for time of day or include as covariate on statistical models to account for confounding effects from diurnal variability.

Supplemental Material

sj-pdf-1-jcb-10.1177_0271678X241232190 - Supplemental material for Unraveling diurnal and technical variability in cerebral hemodynamics from neurovascular 4D-Flow MRI

Supplemental material, sj-pdf-1-jcb-10.1177_0271678X241232190 for Unraveling diurnal and technical variability in cerebral hemodynamics from neurovascular 4D-Flow MRI by Leonardo A Rivera-Rivera, Grant S Roberts, Anthony Peret, Rebecca E Langhough, Erin M Jonaitis, Lianlian Du, Aaron Field, Laura Eisenmenger, Sterling C Johnson and Kevin M Johnson in Journal of Cerebral Blood Flow & Metabolism

Footnotes

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We gratefully acknowledge research support from GE Healthcare. This work was supported by the Alzheimer’s Association (grant number AARFD-20-678095), and the National Institutes of Health (grant numbers F31AG071183, R01AG027161, R01AG075788, R01AG021155, R21AG077337, P30AG062715, UL1TR002373, and KL2TR002374).

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: S.C. Johnson served on an advisory board for Roche Diagnostics in 2018 for which he received an honorarium and is principal investigator of an equipment grant from Roche. He conducts tau imaging in NIH funded studies as well as a study funded by Cerveau Technologies using radioligand precursor material supplied by Cerveau Technologies.

Authors’ contributions

Leonardo A. Rivera-Rivera – Study design, MRI collection data, reconstruction and post processing analysis, interpreting results, editing of the manuscript and figures.

Grant S. Roberts – Data processing of larger older adults repository, editing of the manuscript

Anthony Peret – Data processing of larger older adults repository, editing of the manuscript

Rebecca E. Langhough – Design of cognitive variables for clinical characterization of older adults, statistical analyses discussions, editing of the manuscript

Erin M. Jonaitis – Design of cognitive variables for clinical characterization of older adults, statistical analyses discussions, editing of the manuscript

Lianlian Du – Statistical analyses discussions, editing of the manuscript

Aaron Field – Clinical neuroradiologist assessment of brain morphology for co-morbid diseases, interpreting the results and editing of the manuscript

Laura Eisenmenger – Clinical neuroradiologist assessment of brain morphology for co-morbid diseases, interpreting the results and editing of the manuscript

Sterling C. Johnson – Data acquisition, clinical diagnoses, interpreting results and editing of the manuscript

Kevin M. Johnson – Study design, MRI data acquisition and reconstruction, interpreting results, editing of the manuscript

Supplementary material

Supplemental material for this article is available online.

References

Turski

Scarano

Hartman

, et al. Neurovascular 4DFlow MRI (phase contrast MRA): emerging clinical applications. Neurovasc Imaging 2016; 2: 8.

Wåhlin

Eklund

Malm

4D flow MRI hemodynamic biomarkers for cerebrovascular diseases. J Intern Med 2022; 291: 115–127.

Vikner

Nyberg

Holmgren

, et al. Characterizing pulsatility in distal cerebral arteries using 4D flow MRI. J Cereb Blood Flow Metab 2019; 40: 2429–2440.

Birnefeld

Wåhlin

Eklund

, et al. Cerebral arterial pulsatility is associated with features of small vessel disease in patients with acute stroke and TIA: a 4D flow MRI study. J Neurol 2020; 267: 721–730.

Björnfot

Garpebring

Qvarlander

, et al. Assessing cerebral arterial pulse wave velocity using 4D flow MRI. J Cereb Blood Flow Metab 2021; 41: 2769–2777.

Rivera-Rivera

Turski

Johnson

, et al. 4D flow MRI for intracranial hemodynamics assessment in Alzheimer’s disease. J Cereb Blood Flow Metab 2016; 36: 1718–1730.

Rivera-Rivera

Cody

Rutkowski

, et al. Intracranial vascular flow oscillations in Alzheimer’s disease from 4D flow MRI. Neuroimage Clin 2020; 28: 102379.

Schnell

Ansari

SA.

Four-dimensional MRI flow examinations in cerebral and extracerebral vessels – ready for clinical routine?

Curr Opin Neurol 2016; 29: 419–428.

Aristova

Vali

Ansari

, et al. Standardized evaluation of cerebral arteriovenous malformations using flow distribution network graphs and dual-venc 4D flow MRI. J Magn Reson Imaging 2019; 50: 1718–1730.

10.

Rivera-Rivera

Cody

Eisenmenger

, et al. Assessment of vascular stiffness in the internal carotid artery proximal to the carotid canal in Alzheimer’s disease using pulse wave velocity from low rank reconstructed 4D flow MRI. J Cereb Blood Flow Metab 2021; 41: 298–311.

11.

Rivera-Rivera

Eisenmenger

Cody

, et al. Cerebrovascular stiffness and flow dynamics in the presence of amyloid and tau biomarkers. Alzheimers Dement (Amst) 2021; 13: e12253–e12253.

12.

Fico

Miller

Rivera-Rivera

, et al. Cerebral hemodynamics comparison using transcranial doppler ultrasound and 4D flow MRI. Front Physiol 2023; 14: 1198615.

13.

Fico

Miller

Rivera-Rivera

, et al. The impact of aging on the association between aortic stiffness and cerebral pulsatility index. Front Cardiovasc Med 2022; 9: 821151.

14.

Miller

Howery

Rivera-Rivera

, et al. Age-related reductions in cerebrovascular reactivity using 4D flow MRI. Front Aging Neurosci 2019; 11: 281–281.

15.

Chen

Yang

Recent advances in circadian rhythms in cardiovascular system. Front Pharmacol 2015; 6: 71.

16.

Xie

Kang

, et al. Sleep drives metabolite clearance from the adult brain. Science 2013; 342: 373–377.

17.

Hablitz

Plá

Giannetto

, et al. Circadian control of brain glymphatic and lymphatic fluid flow. Nat Commun 2020; 11: 4411.

18.

Fultz

Bonmassar

Setsompop

, et al. Coupled electrophysiological, hemodynamic, and cerebrospinal fluid oscillations in human sleep. Science 2019; 366: 628–631.

19.

Albers

Dichgans

, et al. Circadian biology and stroke. Stroke 2021; 52: 2180–2190.

20.

Nedergaard

Goldman

SA.

Glymphatic failure as a final common pathway to dementia. Science 2020; 370: 50–56.

21.

Fodor

Marta

Perju-Dumbravă

Implications of circadian rhythm in stroke occurrence: certainties and possibilities. Brain Sci 2021; 11: 865.

22.

Abadjiev

Toschi-Dias

Salinet

, et al. Daily rhythm of dynamic cerebral autoregulation in patients after stroke. J Cereb Blood Flow Metab 2023; 43: 989–998.

23.

Burma

Copeland

Macaulay

, et al. Comparison of diurnal variation, anatomical location, and biological sex within spontaneous and driven dynamic cerebral autoregulation measures. Physiol Rep 2020; 8: e14458.

24.

Shariffi

Lloyd

Cessac

, et al. Reproducibility and diurnal variation in middle cerebral artery blood velocity in healthy humans. Exp Physiol 2023; 108: 692–705.

25.

Labrecque

Burma

Roy

M-A

, et al. Reproducibility and diurnal variation of the directional sensitivity of the cerebral pressure-flow relationship in men and women. J Appl Physiol (1985) 2022; 132: 154–166.

26.

Conroy

Spielman

Scott

RQ.

Daily rhythm of cerebral blood flow velocity. J Circadian Rhythms 2005; 3: 3.

27.

Ismaili

ARA

Vestergaard

Hansen

, et al. Components of day-to-day variability of cerebral perfusion measurements – analysis of phase contrast mapping magnetic resonance imaging measurements in healthy volunteers. PLoS One 2018; 13: e0197807.

28.

Johnson

Koscik

Jonaitis

, et al. The Wisconsin registry for Alzheimer’s prevention: a review of findings and current directions. Alzheimers Dement (Amst) 2018; 10: 130–142.

29.

Roberts

Peret

Jonaitis

, et al. Normative cerebral hemodynamics in middle-aged and older adults using 4D flow MRI: initial analysis of vascular aging. Radiology 2023; 307: e222685.

30.

Jack

Bennett

Blennow

, Contributorset al. NIA-AA research framework: toward a biological definition of Alzheimer’s disease. Alzheimers Dement 2018; 14: 535–562.

31.

McKhann

Drachman

Folstein

, et al. Clinical diagnosis of Alzheimer’s disease: report of the NINCDS‐ADRDA work group* under the auspices of department of health and human services task force on Alzheimer’s disease. Neurology 1984; 34: 939–944.

32.

McKhann

Knopman

Chertkow

, et al. The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association Workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement 2011; 7: 263–269.

33.

Albert

DeKosky

Dickson

, et al. The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the national institute on Aging-Alzheimer’s association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement 2011; 7: 270–279.

34.

Sperling

Aisen

Beckett

, et al. Toward defining the preclinical stages of Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association Workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement 2011; 7: 280–292.

35.

Koscik

Hermann

Allison

, et al. Validity evidence for the research category, “cognitively unimpaired – declining,” as a risk marker for mild cognitive impairment and Alzheimer’s disease. Frontiers in Aging Neuroscience 2021; 13: 404.

36.

Johnson

Lum

Turski

, et al. Improved 3D phase contrast MRI with off-resonance corrected dual echo VIPR. Magn Reson Med 2008; 60: 1329–1336.

37.

Johnson

Markl

Improved SNR in phase contrast velocimetry with five-point balanced flow encoding. Magn Reson Med 2010; 63: 349–355.

38.

Griswold

Jakob

Nittka

, et al. Partially parallel imaging with localized sensitivities (PILS). Magn Reson Med 2000; 44: 602–609.

39.

Walsh

Gmitro

Marcellin

MW.

Adaptive reconstruction of phased array MR imagery. Magn Reson Med 2000; 43: 682–690.

40.

Loecher

Schrauben

Johnson

, et al. Phase unwrapping in 4D MR flow with a 4D single-step Laplacian algorithm. J Magn Reson Imaging 2016; 43: 833–842.

41.

Rivera-Rivera

Kecskemeti

Jen

M-L

, et al. Motion-corrected 4D-Flow MRI for neurovascular applications. NeuroImage 2022; 264: 119711.

42.

Schrauben

Wåhlin

Ambarki

, et al. Fast 4D flow MRI intracranial segmentation and quantification in tortuous arteries. J Magn Reson Imaging 2015; 42: 1458–1464.

43.

Roberts

Hoffman

Rivera-Rivera

, et al. Automated hemodynamic assessment for cranial 4D flow MRI. Magn Reson Imaging 2023; 97: 46–55.

44.

Roher

Garami

Tyas

, et al. Transcranial doppler ultrasound blood flow velocity and pulsatility index as systemic indicators for Alzheimer’s disease. Alzheimers Dement 2011; 7: 445–455.

45.

de Riva

Budohoski

Smielewski

, et al. Transcranial doppler pulsatility index: what it is and what it isn’t. Neurocrit Care 2012; 17: 58–66.

46.

Laurent

Cockcroft

Van Bortel

, et al. Expert consensus document on arterial stiffness: methodological issues and clinical applications. Eur Heart J 2006; 27: 2588–2605.

47.

Townsend

Wilkinson

Schiffrin

, et al. Recommendations for improving and standardizing vascular research on arterial stiffness: a scientific statement from the American heart association. Hypertension 2015; 66: 698–722.

48.

Caffeine for the sustainment of mental task performance: formulations for military operations . Institute of Medicine (US) Committee on Military Nutrition Research. Washington, DC: National Academies Press (US), 2001.

49.

Bland

Altman

DG.

Measuring agreement in method comparison studies. Stat Methods Med Res 1999; 8: 135–160.

50.

Bland

Altman

DG.

Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: 307–310.

51.

Correll

Mellinger

McClelland

, et al. Avoid Cohen’s ‘small’, ‘medium’, and ‘large’ for power analysis. Trends Cogn Sci 2020; 24: 200–207.

52.

Cohen

Statistical power analysis for the behavioral sciences. 2nd ed. New York: Routledge, 1988.

53.

Ferdian

Marlevi

Schollenberger

, et al. Cerebrovascular super-resolution 4D flow MRI – sequential combination of resolution enhancement by deep learning and physics-informed image processing to non-invasively quantify intracranial velocity, flow, and relative pressure. Med Image Anal 2023; 88: 102831.

54.

Rutkowski

Roldán-Alzate

Johnson

KM.

Enhancement of cerebrovascular 4D flow MRI velocity fields using machine learning and computational fluid dynamics simulation data. Sci Rep 2021; 11: 10240.

55.

Peper

Van Ooij

Jung

, et al. Advances in machine learning applications for cardiovascular 4D flow MRI. Front Cardiovasc Med 2022; 9: 1052068.

56.

Nilsson

Ståhlberg

Thomsen

, et al. Circadian variation in human cerebrospinal fluid production measured by magnetic resonance imaging. Am J Physiol 1992; 262: R20–24.

57.

Field

Laurienti

Yen

Y-F

, et al. Dietary caffeine consumption and withdrawal: confounding variables in quantitative cerebral perfusion studies? Radiology 2003; 227: 129–135.

58.

Lie

Hisdal

Høiseth

LØ.

Cerebral blood flow velocity during simultaneous changes in mean arterial pressure and cardiac output in healthy volunteers. Eur J Appl Physiol 2021; 121: 2207–2217.

59.

Ghorbani

Ahmadi

Shemshaki

The value of transcranial doppler derived pulsatility index for diagnosing cerebral small-vessel disease. Adv Biomed Res 2015; 4: 54.

60.

Wilkinson

Mäki-Petäjä

Mitchell

GF.

Uses of arterial stiffness in clinical practice. Arterioscler Thromb Vasc Biol 2020; 40: 1063–1067.

61.

Ghosh

Dharmarajan

Swain

, et al. Impact of cardiovascular factors on pulse wave velocity and T otal vascular resistance in different age group patients with cardiovascular disorders. Curr Aging Sci 2019; 11: 261–268.

62.

Tan

Butlin

Liu

, et al. Heart rate dependence of aortic pulse wave velocity at different arterial pressures in rats. Hypertension 2012; 60: 528–533.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.57 MB