Field performance evaluation of dual rapid HIV and syphilis tests in three antenatal care clinics in Zambia

Abstract

This cross-sectional study of 3212 pregnant women assessed the field performance, acceptability, and feasibility of two dual HIV/syphilis rapid diagnostic tests, the Chembio DPP HIV-syphilis Assay and the SD Bioline HIV/syphilis Duo in antenatal clinics. Sensitivity and specificity for HIV and syphilis were calculated compared to the rapid Determine HIV-1/2 with Uni-Gold to confirm positive results for HIV and the Treponema pallidum particle agglutination assay for syphilis. RPR titers ≥1:4 were used to define active syphilis detection. Acceptability and feasibility were assessed using self-reported questionnaires. For Chembio, the HIV sensitivity was 90.6% (95%CI = 87.4, 93.0) and specificity was 97.2% (95%CI = 96.2, 97.8); syphilis sensitivity was 68.6% (95%CI = 61.9, 74.6) and specificity was 98.5% (95%CI = 97.8, 98.9). For SD Bioline, HIV sensitivity was 89.4% (95%CI = 86.1, 92.0) and specificity was 96.3% (95%CI = 95.3, 97.1); syphilis sensitivity was 66.2% (95%CI = 59.4, 72.4) and specificity was 97.2% (95%CI = 96.4, 97.9). Using the reference for active syphilis, syphilis sensitivity was 84.7% (95%CI = 76.1, 90.6) for Chembio and 81.6% (95%CI = 72.7, 88.1) for SD Bioline. Both rapid diagnostic tests were assessed as highly acceptable and feasible. In a field setting, the performance of both rapid diagnostic tests was comparable to other published field evaluations and each was rated highly acceptable and feasible. These findings can be used to guide further research and proposed scale up in antenatal clinic settings.

Keywords

HIV Africa diagnosis women

Introduction

Mother-to-child transmission (MTCT) of HIV and syphilis remains a major global public health concern due to high associated infant morbidity and mortality.¹ In 2014, an estimated 1.2 million HIV-infected pregnant women gave birth to 170,000 infants with HIV in areas that account for 90% of the global burden.² In 2012, an estimated 930,000 pregnant women were infected with syphilis resulting in 350,000 adverse birth outcomes, of which approximately half were stillbirths or neonatal deaths.³ In 2010, the Zambia HIV antenatal prevalence was estimated at 16.4%⁴ while syphilis prevalence was reported as approximately 9.0% in a study conducted among pregnant women attending antenatal clinics (ANCs) during 2009–2010.⁵

HIV and syphilis coinfection is common in Zambia, with an estimated 24.2% of syphilis-positive pregnant women also testing HIV-positive.⁵ Observational studies suggest that infants born to pregnant women coinfected with HIV and syphilis have twice the risk of being HIV infected compared to infants of mothers infected with HIV alone.^5,6 Early screening and prompt treatment of pregnant women for HIV and syphilis can reduce the risk of MTCT of both diseases and prevent adverse birth outcomes.^7–9 Recognizing this benefit and in pursuit of dual elimination of MTCT of HIV and syphilis, the World Health Organization promotes improved diagnostic techniques such as rapid diagnostic tests (RDTs) for screening for sexually transmitted infections including HIV and syphilis during pregnancy in order to reduce barriers to testing and ensure receipt of results and prompt treatment.⁵

In this study we evaluated the field performance, acceptability, and feasibility of two dual tests, the SD BIOLINE HIV/Syphilis Duo Test (Standard Diagnostics, Korea) and the Chembio Dual Path Platform (DPP®)^10,11 HIV-Syphilis Assay (Medford, NY, USA) among pregnant women seeking antenatal care in Lusaka, Zambia.

Methods

Study design and setting

This field evaluation was conducted in three ANC clinics in Kamwala, Chipata, and Chawama primary health centers in Lusaka urban district from 1 September 2014 through 30 June 2015. Programmatic data indicate approximately 12,000 new ANC registrations are recorded annually in the three clinics. HIV prevalence ranges between 17 and 23% and RPR positivity for syphilis prevalence between 2 and 5%.The clinics are supported by the Centre for Infectious Disease Research in Zambia (CIDRZ) laboratory. CIDRZ is a nongovernmental organization-run laboratory serving public health services. They participate in external proficiency testing with One World Accuracy (Canada) and has met required scores (>80%) on the three cycles of External Quality Assessment (EQA) provided annually since enrolment. This EQA is carried out for all tests done in the CIDRZ lab including HIV and syphilis.

Participant selection

Eligible participants were pregnant women ≥18 years presenting for first ANC visit who were able to provide informed consent. Recruitment was initiated through group health talks in ANC waiting areas. Following eligibility determination, interested women completed a consent form available in two local languages, Chinyanja and Bemba. For women with limited literacy, consent was read to them in their primary language in the presence of an impartial witness and thumbprint was used to obtain consent. All consenting participants completed an interviewer-administered questionnaire (SDC 6) and underwent clinical examination.

Sample size consideration

We estimated sample size based on an assumption of 17% HIV prevalence and 5% syphilis prevalence. We elected to use a more conservative syphilis prevalence estimate (5%) for the sample size calculation to account for different testing strategies in prior studies. Using precision approach to sample size calculation, 3765 pregnant women were required to evaluate the ability of each RDT to detect syphilis at 98% sensitivity with 2% precision at 1.96 Z-score. During the course of the study, 1092 stored specimens were compromised and had to be discarded. These specimens were lost at random and not subject to any selection bias. To assess whether we had sufficient samples to complete the primary analysis, an interim analysis of the syphilis positivity rate in the study population was calculated. Based on a 9% syphilis positivity rate, it was determined that a sample size of 2100 was adequate to answer the primary outcome question under similar assumptions.

Data collection

We used structured questionnaires to collect participant data on sociodemographic characteristics including age of participant and gestational age, travel time to clinic, acceptability, and preferences for the dual tests (SDC 6). Study participants were interviewed by study staff following the testing procedure performed by the ANC staff to prevent bias. ANC staff performing the tests were interviewed on the feasibility of the dual RDTs (SDC 7).

Laboratory procedures

ANC clinic testing

All participants were tested using both RDTs under evaluation in addition to standard of care testing. ANC nurses were trained prior to study initiation to perform the RDTs according to manufacturer’s instructions.^10,11 Each result for a dual RDT was read two times. The first read at 15 min was read independently and blindly from the second read at 20 min.

An additional 5 ml of venous blood was collected in ethylenediaminetetraacetic acid anticoagulant tube, stored in cooler boxes, and sent for processing and reference testing at CIDRZ laboratory within 12 h of blood draw. These specimens were labeled with participant unique identifier, date, and time of collection.

Reference laboratory testing

When received at the laboratory, specimens were logged into the Laboratory Information Management System. The blood was centrifuged and aliquoted into 2 (1 ml) cryovials. HIV reference testing using RDTs was performed on plasma immediately upon arrival in the laboratory. The remaining plasma was frozen stored at −80°C for batch Treponema pallidum particle agglutination assay (TPPA) and rapid plasma reagin (RPR) testing, and if needed, repeat or future testing.

All reference tests were performed by technicians trained in the use of the assays for the study. The reference tests used for HIV were Determine (Alere Medical Company Limited, Japan) confirmed by Uni-Gold (Trinity Biotech, USA). The Western Blot (WB) (Biorad Laboratories) was available for use to resolve discordant results. The reference test used for syphilis was TPPA which is a treponemal test. Positive TPPA samples were then tested with RPR quantitative assays to determine whether the syphilis infection was active. For quality assurance purposes, positive and negative control runs were done daily with each batch of samples tested.

HIV samples nonreactive by Determine were considered negative; samples that were reactive by Determine were further tested using Uni-Gold. Samples with reactive results by both Determine and Uni-Gold were considered positive. Those samples with discordant results would have been further tested using WB; however, there were no discordant results.

Syphilis reference TPPA testing was conducted on batched frozen plasma at the end of the study. Samples with positive TPPA were further tested with the RPR quantitative test. Specimens with a positive TPPA test but a negative RPR were considered negative. Samples testing positive by both TPPA and RPR were considered to be probable active syphilis infections.

The details of the study procedures are depicted in online Figure 1 (SDC 1).

Statistical analysis

We assessed the diagnostic accuracy of the HIV and syphilis components separately (sensitivity and specificity) of each dual HIV/syphilis RDT against Determine (with Uni-Gold confirmation) for HIV and TPPA for syphilis. Sensitivity was defined as the proportion of RDTs testing positive among those that tested positive based on reference testing. Specificity was defined as the proportion of RDTs testing negative among those that tested negative based on reference testing. In a subanalysis of syphilis diagnostic performance of the RDTs under study, we defined a separate reference standard for ‘active syphilis’ as all cases with TPPA positive and RPR positive with a titer of $\geq$ 1:4. A Kappa statistic was used to assess the agreement of test results.

We assessed acceptability of the two RDTs in the study population. Women were asked if they preferred single versus dual RDT testing (regardless of test brand) as well as their willingness to wait for test results. To assess the operational feasibility of the two RDTs, we calculated the mean score for all staff performing each test who responded to the questionnaire for each attribute (SDC 7). All analyses were performed using Stata MP 14 (StataCorp, College Station, Texas, USA).

Ethical considerations

All procedures were conducted in accordance with local standards of care. All participants provided informed consent before any procedures were conducted. This protocol and the informed consent documents were reviewed and approved by ethics committees responsible for oversight of the study, including (1) the WHO Research Project Review Panel and the WHO Research Ethics Review Committee, (2) the University of Zambia Biomedical Research Ethics Committee, and (3) the University of North Carolina at Chapel Hill Institutional Review Board.

Results

Characteristics of participants

We enrolled 3213 pregnant women across the three clinics of the total enrollment, 2121 (66.0%) had blood samples available for the field performance of the tests analysis. Of the remaining 2121 participants, an additional two had missing laboratory results for the TPPA results (n = 2119 tests) and one had a missing laboratory result for the Determine HIV result (n = 2120) (Online Figure 2 – SDC 2),

The median age of pregnant women was 25 years (IQR = 21, 30) with median gestational age of 20 weeks (IQR = 16, 24). Over two-thirds (71%) of the participants were ≥20 weeks of gestation (Table 1).

Table 1.

Characteristics of women enrolled into the study.

Characteristics	Number of pregnant women N = 3213	Percent of total
Age (years)
Median (IQR)	25 (21, 30)
18–19	454	14.1
20–24	1146	35.7
25–29	790	24.6
30–34	527	16.4
35–45	289	9
Missing	7	0.2
Gestational age (weeks)
Median (IQR)	20 (16, 24)
<20	906	28.2
20+	2293	71.4
Missing	14	0.4
Travel time (min)
Median (IQR)	30 (20, 35)
<30 min	1581	49.2
30 min+	1615	50.3
Missing	17	0.5
Total	3213	100

IQR: interquartile range.

The HIV prevalence in the study population was 19.2% (408/2120) and the syphilis prevalence was 9.7% (205/2119).

Diagnostic performance

When compared to reference testing, the HIV diagnostic performance of the Chembio RDT showed a sensitivity of 90.6% (95%CI = 87.4, 93.0) and a specificity of 97.2% (95%CI = 96.2, 97.8). For the SD Bioline RDT the sensitivity was 89.4% (95%CI = 86.1, 92.0) and specificity was 96.3% (95%CI = 95.3, 97.1) (Table 2). The agreement of testing results as read by two testing conductors at 5 min apart was high and statistically significant: For the HIV component, the kappa statistic for Chembio was 0.98 and the kappa statistic for Bioline was 0.95 (Online Table 4 SDC 3).

Table 2.

Diagnostic accuracy of dual HIV/syphilis RDTs by brand of RDT in Lusaka, Zambia.

Test type	Comparison with Determine^a		Comparison with TPPA^b
Test type	No. positive (sensitivity, %)	No. negative (specificity, %)	No. positive (sensitivity, %)	No. negative (specificity, %)
Total (N=2121)
Chembio	434 (90.6)	1686 (97.2)	204 (68.6)	1915 (98.5)
SD Bioline	434 (89.4)	1685 (96.3)	204 (66.2)	1914 (97.2)

RDT: rapid diagnostic test.

^aDetermine = reference test for HIV.

^bTPPA = Treponema pallidum particle agglutination assay = reference test for syphilis.

The syphilis diagnostic performance of the Chembio RDT when compared to reference testing showed a sensitivity and specificity of 68.6% (95%CI = 61.9, 74.6) and 98.5% (95%CI = 97.8, 98.9), respectively. The SD Bioline RDT when compared to reference testing showed a sensitivity and specificity of 66.2% (95%CI = 59.4, 72.4) and 97.2% (95%CI = 96.4, 97.9), respectively.

When compared to an ‘active syphilis’ standard, RPR titers of 1:4 or greater, the sensitivity for detecting active syphilis was higher for both dual RDTs: 84.7% (95%CI = 76.1, 90.6) for Chembio and 81.6% (95%CI = 72.7, 88.1) (Table 3). Regarding the agreement of testing results as read by two conductors, the kappa statistic for syphilis was 0.96 for Chembio and 0.91 SD Bioline (Online Table 4 SDC 3).

Table 3.

Diagnostic accuracy for ‘active syphilis’^a of dual HIV/syphilis RDTs by brand of RDT in Lusaka, Zambia.

Brand of RDT	Comparison with active syphilis^a (combinations of TPPA and RPR)
Brand of RDT	No. positive (sensitivity, %)
Chembio	98 (84.7)
SD Bioline	98 (81.6)

RDT: rapid diagnostic test; RPR: rapid plasma reagin; TPPA: Treponema pallidum particle agglutination assay.

^aActive syphilis = TPPA positive and RPR ≥ 1:4 titer.

Acceptability and feasibility of the RDTs

For acceptability, almost all the participants preferred the dual tests (99.7%) and were willing to wait for the results (99.9%) (Online Table 5 SDC 4). About two-thirds of participating women (64.3%) reported they would be willing to wait for up to 1 h.

For feasibility, the kits were rated by ANC nurses using a questionnaire (SDC 7) with the following mean scores: for Chembio = 20.0 (SD = 3.2); for SD Bioline = 20.1 (SD = 3.2) out of a maximum obtainable score of 25 (Online Table6 SDC 5).

Discussion

Findings of this field evaluation of the dual HIV/syphilis testing kits manufactured by Chembio Diagnostic Inc. and Standard Diagnostic Inc. showed similar performance as compared to other field-based studies.^12–16 Acceptability and feasibility scores were high for both RDTs. This study is one of the few field evaluations contributing to data on the performance of the dual HIV/syphilis RDTs in field settings. These results can be considered to guide further evaluation and proposed scale up for use of RDTs in ANCs.

For HIV, compared to our laboratory-based reference, the sensitivity and specificity of the two RDTs were comparable. Results from other field evaluations of RDTs reported similar HIV sensitivity values of between 94 and 99%. The range for specificity of HIV diagnosis in these studies was 97–100% and comparable to results in this field evaluation.¹⁷

Compared to other field performance evaluations, our results for the syphilis component were also comparable.^14–16 Previously reported field-based evaluations have reported sensitivity ranging from 47 to 96% and specificity ranging from 91 to 100%.^16,18 In our study, for syphilis, using the laboratory-based TPPA reference, sensitivity of the Chembio DPP was 69% and of the SD Bioline 66%. We also found in our subanalysis, which used syphilis RPR results $\geq$ 1:4 as indicative of more recent, or ‘active’, syphilis, the sensitivity of both kits was higher, at 84.7% Chembio and 81.6% SD Bioline, compared to using TPPA only. Other studies showed similar results in identifying active syphilis infection.^15,16,18 This suggests a potential clinical advantage of using dual RDTs in ANCs where the imperative is identifying active syphilis that may affect the pregnancy rather than old, treated syphilis which might result in unnecessary treatment.

The results of our study are suggestive of what the performance of the RDTs might be outside of controlled laboratory settings. Laboratory environments are controlled settings where experienced, specially trained laboratory technologists conduct the tests and where serum rather than whole blood is used.¹⁹ Field conditions including poor lighting, limited space, high temperatures, and possible limited visual acuity of clinic staff conducting the tests may compromise results interpretation. In our study, ANC nurses were trained to conduct these tests prior to study initiation; however, they were also expected to conduct routine ANC testing. The ANC clinics are high volume and the staff-to-patient ratio is high without the extra task of performing multistep study-related testing. This would mimic the work conditions of the settings where the RDTs would be scaled up to increase HIV and syphilis screening coverage.

In our study, the kits had high scores on reader reliability, acceptability, and feasibility which have also been shown in other published studies.^16,18 Women reported preference for the dual HIV/syphilis RDTs as compared to single RDTs if available and were also willing to wait for the test results for up to 1 h. These results suggest that dual testing for HIV and syphilis is acceptable to participants and feasible for implementation in ANC settings. Incorporating dual test kits into routine ANC could potentially increase uptake of testing for HIV and syphilis among pregnant women and reduce the risk of MTCT.

This evaluation had some limitations. First, we used Determine with confirmation by Uni-Gold as reference testing rather than the gold standard of EIA for HIV. Both are rapid tests, which may not reach the performance standard of laboratory-based HIV Enzyme Immunoassay (EIA) testing or allow for comparison of the Determine versus HIV EIA testing. Second, samples were frozen and were known to have undergone one to two freeze/thaw cycles before completion of the reference testing for syphilis. The recommendation for the TPPA assay used is not more than one cycle. This may have impacted the performance accuracy for syphilis

Conclusions

Incorporating dual test kits into routine ANC could potentially increase uptake of testing and treatment for HIV and syphilis among pregnant women and reduce the risk of MTCT. The two dual RDTs evaluated in this study had comparable field performance for both HIV and syphilis when compared to other field evaluations. Both kits received high acceptability among pregnant women and were assessed as highly feasible by ANC nurses. These findings can be used to guide further research and proposed scale up in ANC settings.

Supplemental Material

Supplemental material for Field performance evaluation of dual rapid HIV and syphilis tests in three antenatal care clinics in Zambia

Supplementary Material for Field performance evaluation of dual rapid HIV and syphilis tests in three antenatal care clinics in Zambia by Margaret P Kasaro, Samuel Bosomprah, Melanie M Taylor, Ntazana Sindano, Caroline Phiri, Bushimbwa Tambatamba, Sarai Malumo, Bethany Freeman, Bertha Chibwe, Maura Laverty, Morkor N Owiredu, Lori Newman and Izukanji Sikazwe in International Journal of STD & AIDS

Footnotes

Acknowledgments

We sincerely thank the women who participated in this study. We gratefully acknowledge the study team for their work on planning, data collection, management, and statistical analysis. We wish to thank Grace Tembo-Mumba and Agnes Nondo for planning and coordinating the implementation of the study at Ministry of Community Development Mother and Child Health and at CIDRZ, respectively. The findings and conclusions in this report are those of the author(s) and do not necessarily represent the views of the Centers for Disease Control and Prevention or the World Health Organization.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Field performance evaluation of dual rapid HIV and syphilis tests in Zambia was funded by the World Health Organization Department of Reproductive Health and Research (RHR)(TSA 200920372) supported by funding from PATH (Seattle, WA, USA) and the Bill and Melinda Gates Foundation (Seattle, WA, USA). CIDRZ laboratory receives support from the U.S. President’s Emergency Plan for AIDS Relief/Centers for Disease Control and Prevention (PEPFAR/CDC) funded HIV program (grant no. NU2GGH001920).

References

Mabey

Peeling

RW.

Syphilis, still a major cause of infant mortality. Lancet Infect Dis 2011; 11: 654–655.

UNAIDS. AIDS by the numbers 2015, http://www.unaids.org/en/resources/documents/2015/AIDS_by_the_numbers_2015 (Accessed on 16 August 2017).

Wijesooriya

Rochat

Kamb

et al . Global burden of maternal and congenital syphilis in 2008 and 2012: a health systems modelling study. Lancet Glob Health 2016; 4: e525–e533.

UNICEF ZAMBIA. Prevention of mother-to-child transmission, FACT SHEETS, https://www.unicef.org/zambia/5109_8456.html 2010 (accessed 16 August 2017).

Strasser

Bitarakwate

Gill

et al . Introduction of rapid syphilis testing within prevention of mother-to-child transmission of HIV programs in Uganda and Zambia: a field acceptability and feasibility study. J Acquir Immune Defic Syndr 2012; 61: e40–e46.

Mwapasa

Rogerson

Kwiek

et al . Maternal syphilis infection is associated with increased risk of mother-to-child transmission of HIV in Malawi. AIDS 2006; 20: 1869–1877.

Swartzendruber

Steiner

Adler

et al . Introduction of rapid syphilis testing in antenatal care: a systematic review of the impact on HIV and syphilis testing uptake and coverage. Int J Gynecol Obstet 2015; 130: S15–S21.

Blencowe

Cousens

Kamb

et al . Lives saved tool supplement detection and treatment of syphilis in pregnancy to reduce syphilis related stillbirths and neonatal mortality. BMC Public Health 2011; 11: S9.

WHO. Mother-to-child transmission of HIV 2017, http://www.who.int/hiv/topics/mtct/about/en (2017, accessed 23 October 2017).

10.

CHEMBIO Diagnostic Systems INC. DPP® HIV-syphilis assay, http://chembio.com/products/human-diagnostics/dpp-hivsyphilis-assay (accessed 16 August 2017). 2006–2014.

11.

SD Biotin HIV/Syphilis Duo - Alere is now Abbott, https://www.alere.com/en/home/product-details/sd-bioline-hiv-syphilis-duo.html. 2015

12.

Bristow

Leon

Huang

et al . Field evaluation of a dual rapid diagnostic test for HIV infection and syphilis in Lima, Peru. Sex Transm Infect 2016; 92: 182–185.

13.

Bristow

Severe

Pape

et al . Dual rapid lateral flow immunoassay fingerstick wholeblood testing for syphilis and HIV infections is acceptable and accurate, Port-au-Prince, Haiti. BMC Infect Dis 2016; 16: 302.

14.

Hess

Fisher

Reynolds

GL.

Sensitivity and specificity of point-of-care rapid combination syphilis-HIV-HCV tests. PLoS One 2014; 9: e112190.

15.

Black

Williams

Maseko

et al . Field evaluation of Standard Diagnostics' Bioline HIV/Syphilis Duo test among female sex workers in Johannesburg, South Africa. Sex Transm Infect 2016;92:495–498.

16.

Bowen

Lupoli

Chipungu

et al . A bundle of health-syphilis test performance in the field evaluation of a novel dual HIV/syphilis rapid test – Malawi, 2014–2015. Sex Transm Infect 2015; 43: S223.

17.

Gliddon

HDPR

Kamb

Toskin

et al . A systematic review and meta-analysis of studies evaluating the performance and operational characteristics of dual point-of-care tests for HIV and syphilis. A systematic review and meta-analysis of studies evaluating the performance and operational characteristics of dual point-of-care tests for HIV and syphilis. Sex Transm Infect 2017; 93: 1–13.

18.

Fakile

Hoover

Delaney

et al . Evaluation of five rapid point-of-care tests for syphilis: two treponemal only, and three dual treponemal/HIV assays. Sex Transm Infect 2015; 91: (Suppl 2):A29.doi:10.1136/sextrans-2015-052270.89

19.

Mabey

Peeling

Ballard

et al . Prospective, multi-centre clinic-based evaluation of four rapid diagnostic tests for syphilis. Sex Transm Infect 2006; 82: 13–16.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.30 MB