Prevalence and correlation of human papillomavirus genotypes with clinical factors in cervical samples from Mexican women

Abstract

In the last decade, the inclusion of HPV DNA testing in cervical cancer screening has provided one of the best strategies for the prevention and timely detection of HPV. We conducted a high-throughput HPV genotyping study based on MALDI-TOF mass spectrometry to determine the prevalence of 24 HPV genotypes, including oncogenic genotypes, in Mexican women and correlated the results with cytological findings and clinical variables. We likewise identified the risk factors in patients with the HPV infection. Our study included 1000 women from Sonora, Mexico, who participated in cervical cancer screening campaigns and who underwent a Pap smear and HPV DNA test. The results showed that the overall prevalence of HPV was 27.2%, 18.5% with single, and 8.7% multiple infections. The low-risk HPV genotype 6 (8.5%) and oncogenic genotypes 31 (8.1%) and 53 (4.4%) were the most prevalent in the study population. The number of lifetime sexual partners, previous STIs, and age at first intercourse was significantly associated with HPV infection (P ≤ 0.05). Smoking (OR = 1.5609; 95% IC 1.062–2.292) and more than three lifetime sexual partners (OR = 1.609; 95% IC = 1.124–2.303) represented risk factors for HPV infection. Cytological abnormalities were found in 3.4% of the HPV-positive samples. CIN 1–3 occurred in 0.6% of high-risk HPV cases. In general, the prevalence of the HPV genotypes is high in Mexican women with normal cytological findings. This issue highlights the importance of HPV research in seemingly healthy women and could help guide screening strategies for cervical cancer prevention in Mexico.

Impact statement

We are submitting data regarding the prevalence and type distribution of the HPV infection and the risk factors associated with it, which may provide a valuable reference to reinforce screening strategies, and to maintain HPV genotype surveillance in Mexico. We discuss the overall prevalence of HPV infection as detected in normal cytological samples stratified by age, different types of infection, and oncogenic capacity. One of the most important findings was that common HPV genotypes detected in healthy women were the genotype numbers: 6, 31, 16, and 56, likewise, smoking and having a history of more than three sexual partners over their lifetime, represented the main risk factors in this study. Furthermore, we found a low frequency of cytological abnormalities and CIN 1–3 in women with HR-HPV.

Keywords

Human papillomavirus HPV DNA detection MALDI-TOF MS high risk-HPV low risk-HPV

Introduction

The HPV infection is the most common sexually transmitted infection (STI) with a global prevalence of around 11.7%,¹ and it constitutes a serious health issue extending throughout the world, due to the correlation with cervical cancer (CC), which represents the fourth most frequent neoplasm in women worldwide and the third leading cause of the most common types of cancer in Mexican women.²

In the field of papillomavirus, there are 347 annotated papillomavirus genomes in the 2017 PaVE database³; 200 types of HPV have been thoroughly identified and characterized, although only one-third of such infections predominantly, infect the genital tract.^4–7 Approximately 60 HPV genotypes have been classified according to carcinogenic potential, known as high-risk HPV (HR-HPV), such as genotype numbers 16, 18, 31, and 45, considered the main etiological agents of cervical cancer due to their key role in neoplastic transformation,⁸ and also due to the fact that they were detected in over 99% of cervical carcinomas.^9–11

On the other hand, there are low-risk HPV (LR-HPV), mostly of the genotypes 6 and 11, which are commonly present in benign lesions with a minimal risk of malignancy. The HPV natural genetic variation has been widely reported^12–14; nevertheless, high-risk HPVs 16 and 18 are by far the most prevalent in CC and high-grade squamous intraepithelial lesions (HSIL) worldwide (86.9%).^15,16

There is a common behavior worldwide, of HPV infection, showing a high incidence of HPV infection in women under the age of 25, unlike women over the age of 45; however, the percentage of persistence in this age group is higher, concerning the increasing risk of pre-neoplastic lesions.^17,18 Hence, many efforts have been the focus on primary HPV screening for cervical cancer (CC) prevention, for instance, conventional or liquid-based cytology, which have represented a successful reduction in preventing even up to 80% of CC.¹⁹

Although the Pap smear test has improved the detection of CC, there are some disadvantages, for example, high specificity but low sensitivity, therefore it is important to develop more sensitive and specific techniques to improve prevention algorithms.²⁰

One of the promising assays is the DNA HPV detection in cervical exfoliate samples; therefore, high-performance HPV genotyping in Mexican women will be essential not only for cervical cancer screening programs but also, for the development of HPV vaccination programs based on the type-specific prevalence. The present study aimed to determine the prevalence of 24 HPV genotypes in Mexican women by PCR-multiplex and Matrix-Assisted Laser Desorption/Ionization Time-Of-Flight Mass Spectrometry (MALDI-TOF MS)-based assay and correlate the results with cytological findings and epidemiological variables.

Materials and methods

Study population and recruitment procedure

The study population was selected from a representative sample from Sonora, Mexico, during a campaign for early diagnosis of cervical cancer implemented by the foundation Beltrones A.C. and the government of the state of Sonora, Mexico, during the period of January–August 2017. We calculated the sample size based on the following formula

n = \frac{(Z_{\propto / 2})^{2} p (1 - p)}{E^{2}}

where

Z_{\propto / 2}

is the 100 (1−α/2) percentile of the standard normal distribution, p is the estimated HPV prevalence rate in our sample, E is the allowed margin of error. With an estimated HPV regional prevalence of 7.0% and an estimated acceptance rate of 50%¹ at 95% confidence level, it needed a sample size of 1000 women to achieve a 5% margin of error.

This study of the cross-sectional type included 1000 female respondents, apparently healthy, who were given a Pap smear test and HPV DNA test. The participants were enrolled according to the following selection criteria: women aged 18–65 years, sexually active, without a miscarriage in the past six months, without considering education levels, number of pregnancies, drug and alcohol consumption, or infections with other pathogens with known diagnosis. Exclusion criteria were: patients treated with vaginal medicine or pregnancy at the time of recruitment, previous total hysterectomy, cervical conization, and physical or mental disability.

The clinical information was obtained by direct interview and included anthropometric, sociocultural, and sexual behavior data, as well as, a history of previous sexually transmitted infections (STI) (including: trichomonas, candida, herpes, gonorrhea and HPV), family history of cancer, use of oral contraceptives, smoking habits, and a history of HPV vaccination.

Outcome measures

The primary outcome for this study was self-reported cervical cancer screening and the presence of HPV infection, and its measure was derived from the results of the Pap Smear test and HPV DNA tests in the population of the study.

Ethical aspects

The study was registered and approved by the medical research and ethical committee of the institution. The goals of the study were communicated to the participants who were included only after providing written informed consent for the collection and storage of biological samples and the anonymous use of their data in clinical research. All protocols were performed according to the guidelines of the Declaration of Helsinki and the current health laws in Mexico.

Clinical specimens and handling

Cervical exfoliative samples for DNA HPV detection were collected in duplicate by the medical staff of the Beatriz Beltrones Foundation Mobile Clinic, gently scraping the cervix with a cervical brush and storing in Universal Viral Transport System (UTM Copan Diagnostics, USA) at room temperature until analysis. Also, for each patient, a sample of exfoliated cells from the ectocervix was collected with two wooden Ayre spatulas for the preparation of a Papanicolaou smear. Specimens were analyzed by two different approaches; DNA HPV screening at the molecular diagnostic lab, Central ADN, in Morelia, Michoacán, Mexico, and by the conventional cytology (Pap Test)²¹ at the laboratory of Beatriz Beltrones Foundation in Hermosillo, Sonora. Each case was reviewed by one cytotechnologist and one pathologist. The results were reported using the Bethesda System.

DNA extraction

The extraction of the viral and human DNA was carried out by using Instagene Matrix (Bio-Rad, USA), following the manufacturer’s instructions. Briefly, 200 uL of the sample was transferred to 200 uL tubes. Then, the samples were centrifuged at 14,000 r/min for 1 min. The pellet was washed twice with PBS-Tween 20. Finally, 50 µL of Instagene Matrix was added to the pellet and incubated at 56°C for 30 min. The sample was boiled to 100°C for 8 min. The supernatant was stored at −20°C until they were used.

Mass spectrometry-based assay

The DNA HPV detection was performed using MALDI-TOF mass spectrometry (Agena, bioscience, San Diego, CA). The panel detects 16 HR-HPV subtypes (16, 18, 31, 33, 35, 39, 45, 51, 52, 53, 56, 58, 59, 66, 67, 68, 73, 82) and 8 LR-HPV (6, 11, 34, 42, 70 y 81), mixed in one single reaction. The experiments were carried out using the complete iPlex Pro Reagent kit Medium (10160, Agena Bioscience, San Diego, CA) and following the manufacturer’s protocol. Concisely, human and viral nucleic acids were amplified by using the multiplex HPV primers in a final reaction volume of 5 uL as follows: 1× PCR buffer with 2 mM of MgCl₂, 25 mM of dNTPs mix, 0.5 uM of primers mix, 1 U of pro-enzyme (Agena, San Diego, CA, USA), and 2 µL of total DNA (∼10 ng/uL). The amplification conditions were as follows: first step 1 cycle 95°C for 2 min; second step 45 cycles 95°C 30 s, 56 °C 30 s, 72 °C 5 min.

The multipanel detection contains an internal control, known as a housekeeping gene (hGAPDH), to ensure the reliability of the DNA extraction process. The positive controls were the two major oncogenic types, HPV16 (pHPV-16 purified plasmid DNA, ATCC 45113 D) and HPV18 (pHPV-16 purified plasmid DNA (ATCC 45113 D) (10³ copies/µL), as well as positive samples to the other genotypes, all of them, proficiently detected in every batch of samples.

To remove unbound nucleotides, the reaction was treated with a shrimp-alkaline-phosphatase (SAP) mix and incubated at 37°C for 40 min followed by 85°C for 5 min. To amplify DNA strands at the target nucleotide position, an extension reaction was performed with 2 uL of the extend reaction mix (17902, Agena Bioscience, San Diego, CA) and run with the following cycle program: 30 s at 94°C for enzyme activation, 5 s at 94°C for strands denaturation, then strands were annealed at 52°C for 5 s and extended at 80°C for 5 s. The denaturing step and the five cycles of the annealing-extension step were repeated 40 times and followed by a final extension step at 72°C for 3 min. For every polymerase chain reaction (PCR) and incubation steps, the used thermal cycler was Veriti (Applied Biosystems, Foster City, CA).

Finally, a desalting step by adding 6 mg of SpectroCLEAN resin (Agena, San Diego, CA, USA) and 30 µL of water to each sample was carried out for 20 min at room temperature, approximately 15 nL was spotted on a matrix-precoated Spectro-CHIP (Nanodispenser RS1000, Agena Bioscience, San Diego, CA). The samples were analyzed by mass separation in assay-defined profiles on the MALDI-TOF mass spectrometer. The genotyping was automatically generated using Typer 4.0.22 Software (Agena, San Diego, CA, USA). The samples were clustered in positive and negative ones according to the software algorithm and to the relative intensity of each expected mass.

Statistical analysis

We used means and standard deviation (continuous variables), and weighted percentages and frequencies (categorical variables) to describe the sociodemographic, clinical, and behavioral data. The distribution of HPV genotypes was summarized using frequency distributions. Associations between HPV genotypes and epidemiological data were evaluated using χ² or Fisher's exact tests as appropriate. The risk was estimated by calculating the odds ratio (OR) as an approximation of relative risk (RR) with a 95% confidence interval (CI 95%). The analysis of different predictors was performed with logistic regression multivariate according to HPV infection status. Data were analyzed with IBM SPSS 24.0 software (IBM, Armonk, NY, USA). A P-value <0.05 was considered significant.

Results

HPV genotype distribution and prevalence

A total of 1000 women were evaluated in this study. The overall prevalence of HPV was 27.2% (272/1000). A total of 24 HPV genotypes were analyzed, of which 199 (19.9%) women showed HR-HPV genotypes (16, 18, 31, 33, 35, 39, 45, 51, 52, 53, 56, 58, 59, 66, 67, 68, 73, 82) and 73 women (7.3%) LR-HPV genotypes (6, 11, 34, 42, 70, 81). In decreasing order, the most frequent genotypes were 6 (8.5%), 31 (8.1%), 53 (4.4%), 16 (3.7%), 56 (3.7%), and 33 (3.7%) (see Table 1; Figure 1). The most prevalent HPV genotypes were LR-HPV 6 (n =85) and HR-HPV 31 (n = 81) (see Table 1).

Table 1.

Type-speciﬁc HPV prevalence.

IARC classiﬁcation	Genotype	N	Prevalence^a (%)
HR-HPV	16	37	3.70
	18	16	1.60
	31	81	8.10
	33	37	3.70
	35	12	1.20
	39	9	0.90
	45	15	1.50
	51	33	3.30
	52	22	2.20
	53	44	4.40
	56	37	3.70
	58	19	1.90
	59	13	1.30
	66	31	3.10
	67	10	1.00
	68	16	1.60
	73	6	0.60
	82	1	0.10
LR HPV	6	85	8.50
	11	18	1.80
	34	2	0.20
	42	1	0.10
	70	3	0.30
	81	24	2.40

^aCrude HPV prevalence calculated in 272 HPV-positive patients: 185 single and 87 multiple infections.

HPV: human papillomavirus; HR-HPV: high-risk HPV genotypes; LR-HPV: low-risk HPV genotypes; IARC: International Agency for Research on Cancer.

Figure 1.

High risk and low risk HPV genotypes distribution among 272 HPV-positive samples.

Multiple infections were described as more than two different HPV genotypes, either HR-HPV or LR-HPV. The positive samples showed 185 (18.5%) single and 87 (8.7%) multiple HPV infections (Figure 2).

Figure 2.

Correlation between single and multiple HPV infections and the distribution of HPV among different age groups.

Distribution of HPV among different age groups

The highest prevalence of HPV infection was found among women ranging from 36 to 50 years of age (n = 58; 5.8%), followed by women age groups 41 to 45 (n = 56; 5.6%) and 46 to 50 years (n = 56; 5.6%). It was very interesting that a decreasing rate of HPV infection was observed in women in the age groups from 51 to 55 (n = 34; 3.4%) and 56 to 65 years of age (n = 27; 2.7%) (Figures 2 and 3). The multiple HPV infection was evident in women of the 21 to 30-year group, as seen globally^22,23 with a decline at 50 years and older (Figure 2). The HR-HPV genotypes were more prevalent than LR-HPV genotypes in all age groups and as it is shown in Figure 3, the highest prevalence of HR-HPV was noticed in women aged 46–50 years of age (n = 41, 4.1%), in contrast to the age group among 31 to 35 years of age, where the LR-HPV genotypes declined (n = 19; 1.9%) (Figure 3).

Figure 3.

Correlation between HR-HPV and LR-HPV and the overall distribution among different age groups.

Socio-demographics and the correlation with HPV

The distribution of the characteristics among the population of the study is presented in Table 2. The mean age of participants was 44.42 ± 8.13 years; 41.8% (418) had middle education, of which 11.3% were HPV positive. Most of the participants were married (n = 640; 6.4%), showing the highest prevalence of HPV infection (n = 159; 15.9%) (Table 2). Only 9.4% (n = 94) were single. In the multivariate analysis, the HPV infection was evident from the age group of 31 to 35 years of age (n = 22; 2.2%), considerably increased in the age group of 36–40 years of age (n = 58; 5.8%) and gradually decreased at older ages. We also observed a significant correlation among the age groups and positive HPV cases (P < 0.0001) (Table 2).

Table 2.

Multivariate analysis of socio-demographic variables according to HPV infection status.

Variables	Overall(n = 1000)	With HPV(n = 272)	Without HPV (n = 728)	P-value*
Age group (years)				< 0.001
21–30	23 (2.3)	19 (1.9)	4 (0.4)
31–35	43 (4.3)	22 (2.2)	21 (2.1)
36–40	202 (20.2	58 (5.8)	144 (14.4)
41–45	221 (22.1)	56 (5.6)	165 (16.5)
46–50	198 (19.8)	56 (5.6)	142 (14.2)
51–55	155 (15.5)	34 (3.4)	121 (12.1)
56–65	158 (15.8)	27/2.7)	131 (13.1)
Educational level				0.868
Uneducated	8 (0.8)	3 (0.3)	5 (0.5)
Elementary school	177 (17.7)	50 (5.0)	127 (12.7)
Middle school	418 (41.8)	113 (11.3)	305 (30.5)
High school	279 (27.9)	74 (7.4)	205 (20.5)
College	109 (10.9)	31 (3.1)	78 (78.0)
Postgraduate	9 (0.9)	1 (0.1)	8 (0.8)
Marital status				0.095
Single	94 (9.4)	27 (2.7)	67 (6.7)
Married	640 (64.0)	159 (15.9)	481 (48.1)
Domestic partnership	150 (15.0)	45(4.5)	105 (10.5)
Divorced	74 (7.4)	29 (2.9)	45 (4.5)
Widowed	42 (4.2)	12 (1.2)	30 (3.0)
Occupation				0.012
Unemployed	568 (56.8)	152 (15.2)	416 (41.6)
Professional	34 (3.4)	3 (0.3)	31 (3.1)
Employee	301 (30.1)	87 (8.7)	214 (2.14)
Merchandiser	81 (8.1)	29 (2.9)	52 (5.2)
Retired	16 (1.6)	1 (0.1)	15 (1.5)

*χ² test was used to compare between participants. P-value <0.05 was considered significant.

Furthermore, our results indicated that occupational activities might play an important role to acquire HPV infections (P = 0.012). In this regard, there was a greater prevalence of HPV-positive cases in unemployed (15.2%) than in employed (8.7%) women (Table 2).

HPV prevalence correlated with gynecological variables

The prevalence of positive and negative HPV infection correlated with gynecological variables is summarized in supplementary Table A.1. The mean age for first sexual intercourse was 16 ± 3.45 years, with a range between 15 years to 30 years; in this regard, we found the highest prevalence of HPV infection among the 15 to 20 year age group (13.3%). In our multivariate models, the onset of sexual life and the number of sexual partners showed a significant association with HPV infection (P = 0.043 and P = 0.009, respectively). Further, 215 women with HPV infection (21.5%) reported one to three sexual partners and 57 (5.7%) women reported more than three. The number of pregnancies or deliveries did not show any correlation with HPV infection (P > 0.05) (supplementary Table A.1).

Risk factors and HPV infection

Risk factors associated with HPV infection in the univariate analysis are shown in supplementary Table B.2. The results suggest that current smoking is associated with an increased prevalence of HPV (OR = 1.5609; 95% CI = 1.062–2.292; P = 0.023) compared to nonsmokers. The number of sexual partners played a key role in the HPV infection, due to the fact that women with more sexual partners had a higher risk of HPV infection (OR = 1.609; 95% CI = 1.124–2.303; P = 0.009). History of previous STI/STD had a significant association with HPV infection (P = 0.042) but it did not represent a risk factor (OR = 0.5628; 95% CI = 0.276 –1.147) (supplementary Table A.2).

Furthermore, we analyzed the results of the cytological findings to determine the frequencies associated with the different entities of the HPV infection. Predominantly, we found that HPV-positive samples showed reactive cellular changes associated with inflammation (13.5%) and atrophy-associated benign reactive changes (5.3%). Also, the inflammation associated with infection with Coccobacilli was the most frequent in women positive to some genotype of HPV (5.6%) (supplementary Table A.3).

About 0.3% of women infected with HR-HPV had history of the Trichomonas infection, as opposed to women who were positive for low-risk HPV genotypes who did not have such a history. Cellular changes correlated with simple and multiple infections with HPV were found, in most cases, reactive cellular changes associated with inflammation (RCCI) (8.7% and 4.8%, respectively) and features of dysbiosis caused by Coccobacillus (4.1% and 1.5%, respectively) (supplementary Table A.3).

Cervical and cytological abnormalities associated with HPV genotypes

Abnormalities in cytology (ASCUS, LSIL, HSIL, AIS) were found in 2.4% of the samples, of which 0.8% corresponded to HR-HPV and 0.2% corresponded to LR-HPV. In women with HR-HPV, a frequency of 0.2% ASCUS, 0.2% LSIL, 0.3% HSIL, and 0.1% AIS was found. Women with LR-HPV had only an ASCUS frequency of 0.2% (Table 3).

Table 3.

Distribution of HPV genotypes in cytological and cervical abnormalities.

Variables	Negative HPVn = 728	HR-HPVn = 199	LR-HPVn = 73	Overalln = 1000
Cytological abnormalities^a
None	704 (70.4)	191 (19.1)	71 (7.0)	966 (96.6)
ASC-US	12 (1.2)	2 (0.2)	2 (0.2)	16 (1.6)
LSIL	8 (0.8)	2 (0.2)	0 (0.0)	10 (1.0)
HSIL	2 (0.2)	3 (0.3)	0 (0.0)	5 (0.5)
AIS	2 (0.2)	1 (0.1)	0 (0.0)	3 (0.3)
Cervical abnormalities^a
None	716 (71.6)	193 (19.3)	71 (7.1)	980 (98.0)
CIN1	8 (0.8)	3 (0.3)	2 (0.2)	13 (1.3)
CIN2	2 (0.2)	2 (0.2)	0 (0.0)	4 (0.4)
CIN3	2 (0.2)	1(0.1)	0 (0.0)	3 (0.3)

^aData are presented as frequency and percentages.

ASCUS: atypical squamous cells of undetermined significance; AGC: atypical glandular cells; LSIL: low-grade squamous intraepithelial lesion; HSIL: high-grade squamous intraepithelial lesion; AIS: adenocarcinoma in situ; CIN: cervical intraepithelial neoplasia.

Cervical abnormalities such as cervical intraepithelial neoplasia (CIN1-3) occurred in 2.0% of the samples, of which 0.6% and 0.2% corresponded HR-HPV and LR-HPV, respectively. Women positive for HR-HPV presented 0.3% CIN1 and 0.2% CIN2. Only one positive case to HR-HPV presented CIN3 (0.1%) (Table 3).

Discussion

Human papillomavirus is described with carcinogenic potential, due to its biological behavior. As it is well known, there is a productive and non-productive phase. The initial infection occurs in dividing cells at the basal epithelial layer, mainly in basal stem cells, where it may be dormant for weeks or months. Once HPV enters the cell, early genes "E" carries out a replication cycle of the viral genome, and these copies of the viral genome are replicated and segregated equally in daughter cells. In productive infection, HPV virions can be released as a result of the degeneration of infected exfoliated cells and eliminated through physiological processes. On the contrary, the persistence of the viral genome over several years in those dividing cells results in persistent infection.^24–26 Then, permanent research in prevalence and its oncogenic potential is mandatory.

Due to the importance of viral prevalence closed alongside the geographical variability, different associations all over the world have supported the use of DNA HPV testing, with high sensibility and specificity, especially in women older than 30 years of age and in patients with abnormal cytology.²⁰

Based on these concerning facts, the aim of our study focused on the detection of viral genotypes and correlated with cytology and clinical-epidemiological variables through mass spectrometry (MS)-based strategies, extensively employed to HPV genotyping. In this regard, this method was compared with the Inno-LiPA HPV Genotyping Assay (Innogenetics, Rome, Italy), and both procedures MS-assay and InnoLipa showed a substantial overall agreement (k = 0.86).²⁷ Different type of samples have been analyzed with this methodology, and FFPE tissue samples were compared with two hybridization assays: the LCD-array and the EuroArrayHPV system, showing an overall agreement (Cohen’s kappa value: 0.83).²⁸ Other reports of HPV genotyping by MS have shown agreement with other methods, like Proofer, with 86.0% (165/192) (kappa = 0.76; 95% CI, 0.67, 0.84).²⁹ Furthermore, MS method has been reported to have greater clinical and analytical sensitivity in HPV detection in HPV-associated disease (high-grade CIN II) than reverse dot blot hybridization (RDBH) (91.4% vs. 89.7%), with an overall concordance (kappa = 0.945).³⁰ In summary, the MS system has proved a high concordance with other platforms' type-specific identification of HPV.

This approach allowed us to find a high prevalence of the HPV infection (27.2%) in cervical samples from healthy women from Mexico, contrasting with other studies, where a higher prevalence of HPV was reported, 43.6% in clinically healthy Mexican women.³¹ Our results demonstrated that the most frequent HR-HPV genotypes were 31, 16, 33, 53, and 56, and LR-HPV type was 6, consistent with other reports with a similar high distribution of HPV genotypes 16, 53, 18, 31, 33, 39, and 45 in the Mexican population with normal cervical samples,^31–37 although the frequency of HPV 18 was not as high as reported in other studies.³² In this regard, we agree with the fact that the variation in the relative distribution of HPV genotypes may be determined by ethnic-geographical factors. In 2014, the prevalence of HPV types in Mexican women from different areas was determined by 2956 cervical samples which were tested for HPV, of which 1020 samples showed normal cytology and 931 with low-grade squamous intraepithelial lesions (LGSIL), with an 67.1% overall HPV prevalence.³⁸

In 2019, Torres-Ibarra et al.³⁹ carried out HPV-16 and HPV-18 genotyping and cytological testing in Mexico, including 36,212 Mexican participants, and the most relevant findings were 4051 women (11.2%) had high-risk HPV, of which 1109 (24.6%) had HPV-16, HPV-18, or ASC-US or worse cervical lesions. Finally, women HPV-16/HPV-18 positive with Cytological Testing as Triage improved the detection of CIN grade 2 or higher instead of doing it independently.³⁹

In a previous study, we reported a lower prevalence of HPV (15.8%) in apparently healthy women from the municipality of Cozumel, Mexico, in whom a higher frequency of high-risk genotypes 58 and 59 were observed.⁴⁰ However, these findings are not fully comparable with the results of this study, because they were populations with different demographic, geographic, and sociocultural characteristics, and Cozumel is a small Caribbean island located south of Mexico and Sonora is a larger urbanized population found near the border with the United States. However, when a regional analysis is performed, differences are found in the prevalence of HPV genotypes in these populations.³⁸ This information is parallel to the evidence in Latin America and the Caribbean, where the HPV 16 and HPV 18 were the most common type in CC and HSIL.⁴¹ Therefore, there are many other factors impacting in the incidence and prevalence of the HPV genotypes.

Another finding of our study was a high prevalence of HR- and LR-HPV observed in women among 35 to 50 years of age, with values close to 5.5%.⁴² Genetic and environmental factors are key pieces in the viral carcinogenic potential,^43,44 and one of the pathways of HPV infection is sexual intercourse. Rombaldi et al.⁴⁵ reported a higher risk of HR-HPV in women who started their sexual life before age 17 (OR = 1.6; 95% CI = 0.8–3.0),⁴⁵ a fact explained by the lacking and inequality production of cervical mucus, besides, the cellular changes caused by the estrogen’s stimulation.⁴⁶

A link has been established with the number of sexual partners as one of the most important risk factors for HPV infection, and one of our findings was that women who had sexual contact with more than three sexual partners had a higher risk of HPV infection (OR = 1.609; 95% CI = 1.124–2.303; P = 0.009), consistent facts with other studies.³⁸ Kjellberg et al.⁴⁷ reported up to a 5-fold higher risk in women who referred more than two sexual partners. Therefore, our results reinforce the concept "the higher number of sexual partners we have, the higher the risk of acquiring the HPV infection".

Also, previous sexually transmitted infections play an important role in the process of inflammation and rupture of the cervical protection barrier, facilitating the access of HPV to the epithelial cells.⁴⁸ Here, we found that women who referred a prior sexually transmitted infection showed a significant outcome in relation to the risk of HPV infection (P = 0.042), and these results may vary according to the populations studied.⁴⁰

Another documented risk factor is cigarette smoking, supported by the concept that the metabolites secreted into the cervix mucus, increase cell proliferation, inflammation, affecting the immune system.⁴⁹ Our results showed a significant association between current smoking and an increased prevalence of HPV (OR = 1.5609; 95% CI = 1.062–2.292; P = 0.023). In this sense, some studies about the impact of cigarette smoking on the early natural history of HPV infections have focused on the humoral immune response to HPV and on the prevalence and persistence of HPV infections.^50–53 Most previous studies showed some degree of the association between exposure to cigarette smoke and the increased prevalence and/or persistence of HPV infections. In an in vitro study, Alam et al. demonstrated that exposure of cervical cells to Benzo[a]pyrene, a major carcinogen in cigarette smoke, stimulated higher levels of virion synthesis in HPV-infected cell lines. Since Benzo [α] pyrene can be detected in the cervical mucus of smokers, cigarette smoke could increase the viral load of HPV infections.⁵⁴ Data from this study, with outcomes of risk factors associated to HPV prevalence, could indicate that cigarette smoking possibly affects the early natural history of HPV infections, as described in previous studies.

We did not find any association with the use of oral contraceptives, a topic with some degree of inconsistency as in some scenarios in certain populations it correlates with HPV, but not in others.^55,56

According to the Bethesda System (TBS), we evaluated the use of the category of benign cellular changes (BCC) that incorporates infections and reactive changes⁵⁷to determine the correlation with different HPV infections. In this regard, we found that a large percentage of patients infected with HPV showed diverse benign findings; however, most of them were nonspecific inflammatory processes. We also observed that women that were HR-HPV-positive had a higher frequency of inflammatory processes associated with infection by Trichomonas vaginalis, Candida, Coccobacillus, and Actinomyces, compared with women with LR-HPV genotypes. We had some limitations in our study because most of the participants did not remember their history of STI; nevertheless, the dysbiosis found in the histological examination evidenced the inflammatory microenvironment, which predisposes an individual to acquire any infection. In this sense, some authors have reported that persistent HPV infection is promoted by inflammatory processes in the cervical epithelium caused by pathogenic microorganisms of STI such as Chlamydia trachomatis, Neisseria gonorrhoeae.⁵⁸

Abnormalities in cytology such as LSIL, HSIL were more frequent in women with HR-HPV than in women with LR-HPV; nevertheless, the frequency of relevant cytological findings in women positive to HPV infection was relatively low. Several studies have shown that screening tests for HR-HPV identification are more sensitive than cytology to detect CIN 2/3, precursors of CC.^59–61 Because HPV genotyping tests are less likely to miss the cases of CIN 2 and CIN 3^60,62 compared to cytological testing, more studies are mandatory to reinforce the obligatory implementation of HPV screening tests combined with the cytological test to establish the clinical implications. The data provided here reinforce the need for a screening panel with higher sensitivity and specificity, as we could not find any association with the cytology results; therefore, it is possible that the HPV DNA test could contribute to earlier detection, as it has been widely documented.

Furthermore, our efforts in this study demonstrated the monitoring of HPV prevalence and genotypes surveillance topics are crucial to elucidate the role of HPV in HSIL and carcinogenicity, due to the fact that genotypes and their genetic variations have been associated with this potential. Therefore, the status of HPV infection in countries like Mexico, where CC presents a high incidence, the HPV surveillance impacts the strategies positively for the primary HPV screening that helps us to prevent HSIL and CC.

Conclusions

The prevalence of high-risk HPV infection in Mexico is higher in women with normal cytological findings. This issue highlights the implementation of affordable HPV DNA screening strategies to maintain the epidemiological HPV genotype surveillance and to reinforce the vaccination campaigns against HPV in women from Sonora, México.

Supplemental Material

sj-pdf-1-ebm-10.1177_1535370220959747 - Supplemental material for Prevalence and correlation of human papillomavirus genotypes with clinical factors in cervical samples from Mexican women

Supplemental material, sj-pdf-1-ebm-10.1177_1535370220959747 for Prevalence and correlation of human papillomavirus genotypes with clinical factors in cervical samples from Mexican women by Fabiola Hernández-Rosas, Erika Orozco-Hernández, Liliana Maza-Sánchez, Pamela Citlalli Salgado-García, Enrique Navarro-Vidal and Mercedes Piedad de León-Bautista in Experimental Biology and Medicine

Footnotes

Authors’ contributions

All authors participated in the design, interpretation of the studies and review of the manuscript; Sample collection: PCSG; Conceived and designed the experiments: MPDLB; Performed the experiments: LMS, MPDLB, EN, FHR; Formal analysis: MPDLB, FHR, EO; Software: MPDLB, FHR; Writing–original draft: MPDLB, FHR; Writing, reviewing, and editing: MPDLB, FHR.

ACKNOWLEDGMENTS

The authors wish to thank all the staff of the foundation´s medical center Beatriz Beltrones, and all the participants for their important contributions.

Declaration OF CONFLICTING INTERESTS

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported by Translational Medicine, Vanguard and Technology Transfer Sector, Human Health Department at Central ADN S.A. de C.V. and foundation´s medical center Beatriz Beltrones.

ORCID iD

Fabiola Hernández-Rosas

References

Bruni

Diaz

Castellsague

Ferrer

Bosch

de Sanjose

Cervical human papillomavirus prevalence in 5 continents: meta-analysis of 1 million women with normal cytological findings. J Infect Dis 2010; 202:1789–99

Ferlay

Soerjomataram

Dikshit

Eser

Mathers

Rebelo

Parkin

Forman

Bray

Cancer incidence and mortality worldwide: sources, methods and major patterns in GLOBOCAN 2012. Int J Cancer 2015; 136:E359–86

Van Doorslaer

Xirasagar

Maes

Kaminsky

Liou

Sun

Kaur

Huyen

McBride

AA.

The papillomavirus episteme: a major update to the papillomavirus sequence database. Nucleic Acids Res 2017; 45:D499–D506

Doorbar

Quint

Banks

Bravo

Stoler

Broker

Stanley

MA.

The biology and life-cycle of human papillomaviruses. Vaccine 2012; 30(Suppl 5):F55–70

Egawa

Griffin

Doorbar

Human papillomaviruses; epithelial tropisms, and the development of neoplasia. Viruses 2015; 7:3863–90

Doorbar

Egawa

Griffin

Kranjec

Murakami

Human papillomavirus molecular biology and disease association. Rev Med Virol 2015; 25(Suppl 1):2–23

de Villiers

Fauquet

Broker

Bernard

Zur Hausen

Classification of papillomaviruses. Virology 2004; 324:17–27

Schiffman

Clifford

Buonaguro

FM.

Classification of weakly carcinogenic human papillomavirus types: addressing the limits of epidemiology at the borderline. Infect Agent Cancer 2009; 4:8

Walboomers

Jacobs

Manos

Bosch

Kummer

Shah

Snijders

Peto

Meijer

Munoz

Human papillomavirus is a necessary cause of invasive cervical cancer worldwide. J Pathol 1999; 189:12–9

10.

Bosch

Munoz

The viral etiology of cervical cancer. Virus Res 2002; 89:183–90

11.

Bosch

Lorincz

Munoz

Meijer

Shah

KV.

The causal relation between human papillomavirus and cervical cancer. J Clin Pathol 2002; 55:244–65

12.

Mirabello

Yeager

Cullen

Boland

Chen

Wentzensen

Zhang

Yang

Mitchell

Roberson

Bass

Xiao

Burdett

Raine-Bennett

Lorey

Castle

Burk

Schiffman

HPV16 sublineage associations with histology-specific cancer risk using HPV whole-genome sequences in 3200 women. J Natl Cancer Inst 2016; 108:djw100

13.

Schiffman

Rodriguez

Chen

Wacholder

Herrero

Hildesheim

Desalle

Befano

Safaeian

Sherman

Morales

Guillen

Alfaro

Hutchinson

Solomon

Castle

Burk

RD.

A population-based prospective study of carcinogenic human papillomavirus variant lineages, viral persistence, and cervical neoplasia. Cancer Res 2010; 70:3159–69

14.

Arias-Pulido

Peyton

Torrez-Martinez

Anderson

Wheeler

CM.

Human papillomavirus type 18 variant lineages in United States populations characterized by sequence analysis of LCR-E6, E2, and L1 regions. Virology 2005; 338:22–34

15.

de Martel

Ferlay

Franceschi

Vignat

Bray

Forman

Plummer

Global burden of cancers attributable to infections in 2008: a review and synthetic analysis. Lancet Oncol 2012; 13:607–15

16.

Cohen

Jhingran

Oaknin

Denny

Cervical cancer. Lancet 2019; 393:169–82

17.

Pett

Coleman

Integration of high-risk human papillomavirus: a key event in cervical carcinogenesis?

J Pathol 2007; 212:356–67

18.

Williams

Filippova

Soto

Duerksen-Hughes

PJ.

HPV-DNA integration and carcinogenesis: putative roles for inflammation and oxidative stress. Future Virol 2011; 6:45–57

19.

Franceschi

The IARC commitment to cancer prevention: the example of papillomavirus and cervical cancer. Recent Results Cancer Res 2005; 166:277–97

20.

Rizzo

Feldman

Update on primary HPV screening for cervical cancer prevention. Curr Probl Cancer 2018; 42:507–20

21.

Papanicolaou

Traut

HF.

The diagnostic value of vaginal smears in carcinoma of the uterus. 1941. Arch Pathol Lab Med 1997; 121:211–24

22.

Franceschi

Herrero

Clifford

Snijders

Arslan

Anh

Bosch

Ferreccio

Hieu

Lazcano-Ponce

Matos

Molano

Qiao

Rajkumar

Ronco

de Sanjose

Shin

Sukvirach

Thomas

Meijer

Munoz

Variations in the age-specific curves of human papillomavirus prevalence in women worldwide. Int J Cancer 2006; 119:2677–84

23.

Forman

de Martel

Lacey

Soerjomataram

Lortet-Tieulent

Bruni

Vignat

Ferlay

Bray

Plummer

Franceschi

Global burden of human papillomavirus and related diseases. Vaccine 2012; 30(Suppl 5):F12–23

24.

Van Doorslaer

Tan

Xirasagar

Bandaru

Gopalan

Mohamoud

Huyen

McBride

AA.

The papillomavirus episteme: a Central resource for papillomavirus sequence data and analysis. Nucleic Acids Res 2013; 41:D571–8

25.

Graham

SV.

The human papillomavirus replication cycle, and its links to cancer progression: a comprehensive review. Clin Sci 2017; 131:2201–21

26.

Raff

Woodham

Raff

Skeate

Yan

Da Silva

Schelhaas

Kast

WM.

The evolving field of human papillomavirus receptor research: a review of binding and entry. J Virol 2013; 87:6062–72

27.

Cricca

Marasco

Alessandrini

Fazio

Prossomariti

Savini

Venturoli

Chieco

Carolis Bonafe

Garagnani

Mantovani

V. High-throughput genotyping of high-risk human papillomavirus by MALDI-TOF mass spectrometry-based method. New Microbiol 2015; 38:211–23

28.

Kriegsmann

Wandernoth

Lisenko

Casadonte

Longuespee

Arens

Kriegsmann

Detection of HPV subtypes by mass spectrometry in FFPE tissue specimens: a reliable tool for routine diagnostics. J Clin Pathol 2017; 70:417–23

29.

Basu

Chandna

Bamezai

Siddiqi

Saranath

Lear

Ratnam

MassARRAY spectrometry is more sensitive than PreTect HPV-Proofer and consensus PCR for type-specific detection of high-risk oncogenic human papillomavirus genotypes in cervical cancer. J Clin Microbiol 2011; 49:3537–44

30.

Soderlund-Strand

Dillner

Carlson

High-throughput genotyping of oncogenic human papilloma viruses with MALDI-TOF mass spectrometry. Clin Chem 2008; 54:86–92

31.

Orozco-Colin

Carrillo-Garcia

Mendez-Tenorio

Ponce-de-Leon

Mohar

Maldonado-Rodriguez

Guerra-Arias

Flores-Gil

Sotelo-Regil

Lizano

Geographical variation in human papillomavirus prevalence in Mexican women with normal cytology. Int J Infect Dis 2010; 14:e1082–7

32.

Illades-Aguiar

B, A-R

Ldel

Antonio-Vejar

Zamudio-Lopez

Sales-Linares

Flores-Alfaro

Fernandez-Tilapa

Vences-Velazquez

Munoz-Valle

Leyva-Vazquez

MA.

Prevalence and distribution of human papillomavirus types in cervical cancer, squamous intraepithelial lesions, and with no intraepithelial lesions in women from Southern Mexico. Gynecol Oncol 2010; 117:291–6

33.

Velazquez-Marquez

Paredes-Tello

Perez-Terron

Santos-Lopez

Reyes-Leyva

Vallejo-Ruiz

Prevalence of human papillomavirus genotypes in women from a rural region of Puebla, Mexico. Int J Infect Dis 2009; 13:690–5

34.

Illades-Aguiar

Cortes-Malagon

Antonio-Vejar

Zamudio-Lopez

N, A-R

Ldel

Fernandez-Tilapa

Hernandez-Sotelo

Teran-Porcayo

Flores-Alfaro

Leyva-Vazquez

MA.

Cervical carcinoma in Southern Mexico: human papillomavirus and cofactors. Cancer Detect Prev 2009; 32:300–7

35.

Sanchez-Anguiano

Alvarado-Esquivel

Reyes-Romero

Carrera

RM.

Human papillomavirus infections in women seeking cervical papanicolaou cytology of Durango, Mexico: prevalence and genotypes. BMC Infect Dis 2006; 6:27

36.

Torroella-Kouri

Morsberger

Carrillo

Mohar

Meneses

Ibarra

Daniel

Ghaffari

Solorza

Shah

KV.

HPV prevalence among Mexican women with neoplastic and normal cervixes. Gynecol Oncol 1998; 70:115–20

37.

Lazcano-Ponce

Herrero

Munoz

Cruz

Shah

Alonso

Hernandez

Salmeron

Hernandez

Epidemiology of HPV infection among Mexican women with normal cervical cytology. Int J Cancer 2001; 91:412–20

38.

Salcedo

Pina-Sanchez

Vallejo-Ruiz

Monroy-Garcia

Aguilar-Lemarroy

Cortes-Gutierrez

Santos-Lopez

Montoya-Fuentes

Grijalva

Madrid-Marina

Apresa-Garcia

Hernandez

Jave-Suarez

Romero

Poot

Salgado

Ramos-Gonzalez

Gonzalez-Hernandez

Canton

Jimenez-Aranda

Parra-Melquiadez

Paniagua

Mendoza

Arreola

Villegas

Torres-Poveda

Bahena-Roman

Gonzalez-Yebra

Taniguchi

Rodea

Mantilla-Morales

Human papillomavirus genotypes among females in Mexico: a study from the Mexican Institute for Social Security. Asian Pac J Cancer Prev 2014; 15:10061–6

39.

Torres-Ibarra

Cuzick

Lorincz

Spiegelman

Lazcano-Ponce

Franco

Moscicki

Mahmud

Wheeler

Rivera-Paredez

Hernández-López

León-Maldonado

Salmerón

, FRIDA Study Group. Comparison of HPV-16 and HPV-18 genotyping and cytological testing as triage testing within human papillomavirus-based screening in Mexico. JAMA Netw Open 2019; 2:e1915781

40.

Navarro-Vidal

Hernandez-Rosas

Rey

Flores-Peredo

Prevalence of human papillomavirus genotypes in women from cozumel,

Mexico. Asian Pac J Cancer Prev 2018; 19:2417–22

41.

Ciapponi

Bardach

Glujovsky

Gibbons

Picconi

MA.

Type-specific HPV prevalence in cervical cancer and high-grade lesions in Latin America and the Caribbean: systematic review and meta-analysis. PLoS One 2011; 6:e25493

42.

Munoz

Bosch

de Sanjose

Herrero

Castellsague

Shah

Snijders

Meijer

CJ.

International agency for research on cancer multicenter cervical cancer study G. Epidemiologic classification of human papillomavirus types associated with cervical cancer. N Engl J Med 2003; 348:518–27

43.

Chan

Wong

Chang

Chor

MY.

Epidemiologic risk profile of infection with different groups of human papillomaviruses. J Med Virol 2009; 81:1635–44

44.

Liu

Chen

SD.

Multiple sexual partners as a potential independent risk factor for cervical cancer: a meta-analysis of epidemiological studies. Asian Pac J Cancer Prev 2015; 16:3893–900

45.

Rombaldi

Serafini

Villa

Vanni

Barea

Frassini

Xavier

Paesi

Infection with human papillomaviruses of sexual partners of women having cervical intraepithelial neoplasia. Braz J Med Biol Res 2006; 39:177–87

46.

Castle

Jeronimo

Schiffman

Herrero

Rodriguez

Bratti

Hildesheim

Wacholder

Long

Neve

Pfeiffer

Burk

RD.

Age-related changes of the cervix influence human papillomavirus type distribution. Cancer Res 2006; 66:1218–24

47.

Kjellberg

Hallmans

Ahren

Johansson

Bergman

Wadell

Angstrom

Dillner

Smoking, diet, pregnancy and oral contraceptive use as risk factors for cervical intra-epithelial neoplasia in relation to human papillomavirus infection. Br J Cancer 2000; 82:1332–8

48.

Panatto

Amicizia

Bianchi

Frati

Zotti

Lai

Domnich

Colzani

Gasparini

Tanzi

Chlamydia trachomatis prevalence and chlamydial/HPV co-infection among HPV-unvaccinated young Italian females with normal cytology. Hum Vaccin Immunother 2015; 11:270–6

49.

Cigarette smoking linked to increased human papillomavirus DNA load. CA Cancer J Clin 2010; 60:137–8

50.

Wiley

Wiesmeier

Masongsong

Gylys

Koutsky

Ferris

Barr

Yu Rao

Proof Of Principle Study Investigative

Smokers at higher risk for undetected antibody for oncogenic human papillomavirus type 16 infection. Cancer Epidemiol Biomarkers Prev 2006; 15:915–20

51.

Vaccarella

Herrero

Snijders

Dai

Thomas

Hieu

Ferreccio

Matos

Posso

de Sanjose

Shin

Sukvirach

Lazcano-Ponce

Munoz

Meijer

Franceschi

Group

Smoking and human papillomavirus infection: pooled analysis of the international agency for research on cancer HPV prevalence surveys. Int J Epidemiol 2008; 37:536–46

52.

Syrjanen

Shabalova

Petrovichev

Kozachenko

Zakharova

Pajanidi

Podistov

Chemeris

Sozaeva

Lipova

Tsidaeva

Ivanchenko

Pshepurko

Zakharenko

Nerovjna

Kljukina

Erokhina

Branovskaja

Nikitina

Grunberga

Grunberg

Juschenko

Santopietro

Cintorino

Tosi

P, SS.

Smoking is an independent risk factor for oncogenic human papillomavirus (HPV) infections but not for high-grade CIN. Eur J Epidemiol 2007; 22:723–35

53.

Koshiol

Schroeder

Jamieson

Marshall

Duerr

Heilig

Shah

Klein

Cu-Uvin

Schuman

Celentano

Smith

JS.

Smoking and time to clearance of human papillomavirus infection in HIV-seropositive and HIV-seronegative women. Am J Epidemiol 2006; 164:176–83

54.

Alam

Conway

Chen

Meyers

The cigarette smoke carcinogen benzo[a]pyrene enhances human papillomavirus synthesis. J Virol 2008; 82:1053–8

55.

Kjaer

van den Brule

Bock

Poll

Engholm

Sherman

Walboomers

Meijer

CJ.

Determinants for genital human papillomavirus (HPV) infection in 1000 randomly chosen young Danish women with normal pap smear: are there different risk profiles for oncogenic and nononcogenic HPV types?

Cancer Epidemiol Biomarkers Prev 1997; 6:799–805

56.

Ylitalo

Sorensen

Josefsson

Frisch

Sparen

Ponten

Gyllensten

Melbye

Adami

HO.

Smoking and oral contraceptives as risk factors for cervical carcinoma in situ. Int J Cancer 1999; 81:357–65

57.

Kurman

Henson

Herbst

Noller

Schiffman

MH.

Interim guidelines for management of abnormal cervical cytology. The 1992 national cancer institute workshop. JAMA 1994; 271:1866–9

58.

Tamarelle

Thiebaut

ACM

de Barbeyrac

Bebear

Ravel

Delarocque-Astagneau

The vaginal microbiota and its association with human papillomavirus, Chlamydia trachomatis, Neisseria gonorrhoeae and Mycoplasma genitalium infections: a systematic review and Meta-analysis. Clin Microbiol Infect 2019; 25:35–47

59.

Cuzick

Clavel

Petry

Meijer

Hoyer

Ratnam

Szarewski

Birembaut

Kulasingam

Sasieni

Iftner

Overview of the European and North American studies on HPV testing in primary cervical cancer screening. Int J Cancer 2006; 119:1095–101

60.

Smelov

Elfstrom

Johansson

Eklund

Naucler

Arnheim-Dahlstrom

Dillner

Long-term HPV type-specific risks of high-grade cervical intraepithelial lesions: a 14-year follow-up of a randomized primary HPV screening trial. Int J Cancer 2015; 136:1171–80

61.

Cuzick

Arbyn

Sankaranarayanan

Tsu

Ronco

Mayrand

Dillner

Meijer

CJ.

Overview of human papillomavirus-based and other novel options for cervical cancer screening in developed and developing countries. Vaccine 2008; 26(Suppl 10):K29–41

62.

Castle

Schiffman

Wheeler

Wentzensen

Gravitt

PE.

Human papillomavirus genotypes in cervical intraepithelial neoplasia grade 3. Cancer Epidemiol Biomarkers Prev 2010; 19:1675–81

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.07 MB