End-user involvement in developing and field testing an online contraceptive decision aid

Study setting and research team

The DA was developed within a General Practitioners’ (GP) office in Amsterdam, the Netherlands. The research team consisted of three female researchers (T.H., J.M.M., C.M.V.D.H.), one male researcher (T.P.G.), one associate professor (F.J.M.) and one male GP (P.V.). This team complemented each other in terms of theoretical and practical research knowledge, and medical knowledge and skills. In addition, a number of other GPs served as an advisory committee in the development of the DA prototype.

Developmental stages and participant selection

Figure 1 shows how the DA was developed through several stages. These stages were based on the International Patient Decision Aids Standards (IPDAS) quality criteria of systematic development of DAs.^3,17 We thereby followed six developmental stages: set aim and audience, collecting existing information on development and content, explore preferences regarding content and structure of a contraceptive DA: needs assessment, development of a DA, field testing and revision and release of the DA.

OPEN IN VIEWER

Figure 1.

Development process of the online contraceptive decision aid, based on IPDAS quality criteria of systematic development of DAs.^3,17

Due to the exploratory nature of the study, we chose a select study population of female university students. One reason for this was that these female students were expected to visit and use an online contraceptive DA more often than other women because they are more likely to search for health information on the web. ²⁰ Female university students were recruited to participate in the needs assessment (stage 3) and the field test (stage 5). Different participants were selected for each stage. Participants were approached via email and social media. Through purposive and snowball sampling, we included female students who varied in age, educational institution and study field. No exclusion criteria were applied. Beforehand, the sample size was limited to 7–10 participants per developmental stage. This sample size was considered to be sufficient for our present goal, since our study is practical, the scope of the study is not too broad, the aim to develop a tool was obvious and clear and we were able to get high-quality data from a relatively homogeneous population of highly educated participants.^21,22

In the first stage, the aim and target audience of the DA were set. The aim is to support informed choice and shared decision making between women and healthcare providers in choosing between various contraceptives in the Netherlands. The target audiences were Dutch women, seeking support in choosing their first method of contraception or wanting to change their contraceptive method.

In the second stage, information was collected on the development process of (contraceptive) DAs by reviewing relevant documents, guidelines and articles, as well as consulting GPs of the advisory committee.^3,17,23–26 One of these documents was the IPDAS quality criteria framework: a uniform checklist of quality criteria for DAs. ²⁷ We obtained more insight into how DAs can be structured to clarify and elucidate individual values and provide evidence-based information. ²⁸ In addition, information was collected through literature search regarding contraceptive methods and their features, relevant contraceptive attributes and factors that affect contraceptive choice.^{5,15,26,29–33} Moreover, we reviewed existing Dutch online contraceptive DAs to gain insight into their structure, layout and content.

In the third stage, we conducted a needs assessment to explore decision support needs of female students concerning the preferred content and structure of the DA. Female participants were interviewed between March and May 2014 by one of the coauthors (J.M.M.). The interview guide covered two main themes: contraceptive decisions and DAs.^2,5,13,15,17 The 80–120 min semi-structured interviews started with questions about contraceptive use, factors and contraceptive attributes that (could) influence contraceptive decisions and the role of the GP and DAs in the decision making process. Participants were consequently asked to ‘think aloud’ and provide feedback (cognitive interview) while using five existing online Dutch contraceptive DAs. ³⁴ This stimulated thinking about needs for a new DA and provided the researchers insight into which elements of existing DAs (not) to include. The interview guide was piloted with female students, other than the participants. Interviews were performed at the homes of participants, at the work place or in one of the educational institutions.

We developed a Dutch online contraceptive DA prototype between February and April 2016. The content and structure of the online DA were based on the results of the literature search (stage 2) and the needs assessment (stage 3). Information presented in the DA was based on the contraceptive standard provided by the Dutch College of General Practitioners, ²⁹ and a governmental Dutch website for healthcare providers with information on medicine. ³⁰ In addition, experiences of GPs in the advisory committee regarding contraceptive consultations were taken into consideration, and they provided feedback on information in the DA. A website was chosen as medium for a DA because of common use of the Internet to gather information, accessibility and ease of updating the tool. ² The online DA was built by a web agency. The developed DA prototype was presented to participants during the field test.

The fifth stage consisted of a field test of the DA prototype among female students in April and May 2016. These women participated in an interview with one of the co-authors (T.H.).

The interview guide was based on a developed framework that combines two existing models concerning the development of a tool (see Figure 2): The IPDAS quality criteria framework and the integrative model.^27,35 The developed framework combining these two models shows that satisfaction regarding the tool is determined by comparing expectations and perceptions for the subsets content, development process and effectiveness. According to the developed framework, a user will be satisfied with a tool when expectations (prior to use) for the subsets (content, development process and effectiveness) are met by perceptions (after use). The interview guide was piloted with female students, other than the participants. Interviews were performed at the homes of participants, at the work place or in one of the educational institutions.

OPEN IN VIEWER

Figure 2.

Conceptual framework. This framework is developed by combining the IPDAS quality criteria framework of Elwyn et al. ²⁷ (indicated with striped boxes), and the integrative model of De Ruyter et al. ³⁵ (indicated with continuous arrows and boxes). The integrative model implies that satisfaction regarding a tool is determined by met or unmet prior expectations, with respect to perceptions regarding the tool after use. ³⁵ The IPDAS quality criteria framework with the format of a checklist shows which quality criteria need to be met by a DA. According to this model, these quality criteria are divided into three subsets: content, development process and effectiveness. ²⁷

The 80–120 min semi-structured interviews consisted of three parts. First, information was gathered regarding women’s contraceptive information needs and expectations towards the DA regarding the content, development process and effectiveness. Second, while participants used the developed DA prototype, we used the ‘think aloud’ technique (cognitive interview) to investigate how the DA prototype was experienced and how it could be improved. ³⁴ Third, questions were formulated about the experience with the tool, including in what way the DA complied with prior expectations regarding the content, development process and effectiveness and to identify the influence of the use of the DA on decision making.

In the sixth stage, we revised the DA based on the feedback that was provided in the field test (stage 5) and released the DA in its final format: www.keuzehulpanticonceptie.nl.

Data processing

All interviews were audio recorded and transcribed verbatim. Transcripts were processed with a thematic analysis. ³⁶ The analysis included a mix of deductive and inductive coding: codes were based on literature and the interview guides, and codes were derived from the data itself. For both needs assessment and field test, two researchers independently read and coded the first two transcripts and they discussed coding until consensus was reached on a coding frame. The coding frame was supplemented and finalized during coding of remaining transcripts by one coauthor (J.M.M. during needs assessment, T.H. during field test). Data saturation was discussed after, respectively, eight (needs assessment) and nine interviews (field test). Because most participants’ comments overlapped with collected data the researchers decided to stop recruitment.

Ethical considerations

Female participants could partake in the study on a voluntary and anonymous basis. The study process was non-invasive and it consisted of only one interview (no intervention), implying that medical integrity was not harmed. Only participants and researchers were present during the interviews. Therefore, it was not required to apply for approval from the Ethics Committee according to privacy legislation of the country where the study took place.^37,38 Nevertheless, medical ethical guidelines were followed: participants were informed extensively about the study and were asked to give their informed consent before starting the procedure. The study was completely voluntary and participants could stop at any time. To further safeguard anonymity, sound recordings were deleted after transcription. Transcripts will be kept for 15 years and will then be destroyed.

Participants

In total, 17 Dutch female students participated in this study: Eight participants were interviewed during the needs assessment (stage 3) and nine participants were interviewed during the field test (stage 5). All participants were enrolled at a University (U) or a University of Applied Sciences (UAS) in Amsterdam, the Netherlands. The participants’ age varied between 19 and 26 years. All but one of the participants used a contraceptive method at the time of the interview. Most participants were using the contraceptive pill or hormone coil. Table 1 describes the participants’ characteristics.

OPEN IN VIEWER

Table 1.

Participant characteristics. Age, educational level and current contraceptive method are shown for the participants of the needs assessment and of the field tests.

Participant	Age	Educational level	Contraceptive method
Participants during needs assessment
1	23	U	Contraceptive ring
2	24	U	Hormone coil
3	25	U	Hormone coil
4	21	U	Contraceptive pill
5	23	U	Hormone coil
6	21	UAS	Contraceptive pill
7	23	UAS	Contraceptive injection
8	23	U	Contraceptive implant
Participants during field tests
9	20	U	Contraceptive pill
10	20	UAS	Hormone coil
11	19	UAS	Contraceptive pill
12	20	UAS	Contraceptive pill
13	21	U	–
14	25	U	Hormone coil
15	22	UAS	Hormone coil
16	26	U	Contraceptive pill
17	19	UAS	Contraceptive pill

U: University; UAS: University of Applied Sciences.

Needs assessment: preferred content and structure of a DA

During the needs assessment, it became clear that none of the five existing contraceptive DAs completely complied with the participants’ needs. The needs assessment led to the identification of several aspects that needed to be taken into account during the development of the online contraceptive DA. These aspects are addressed under the themes content and structure.

Regarding the content of a DA, several contraceptive attributes were considered as important during contraceptive decision making and should therefore be included in a DA. These attributes included the reliability of contraceptive methods, ease of use (e.g. how often and how to use it) and side effects (e.g. influence on menstrual periods, acne, migraine, menstrual pain and libido). In addition, the working mechanism, amount of hormones, costs and impact on fertility and sexually transmitted diseases (STDs) were considered by women in their contraceptive decision making.

Two important structural aspects to consider in developing a DA were addressed. First, most participants preferred a contraceptive advice resulting from a DA that consists of suggestions for multiple methods instead of one, and transparency on why these methods fit you:

Not just: ‘this is the best’, but that the DA provides you some options. What I like myself […] is that you get all the options and then see which one is the best for you, and that you can see why. (respondent 6 needs assessment)

Second, information in the contraceptive DA should be extensive, in depth, reliable and easily accessible in the DA, which appeared to be more important than simplicity of the DA:

I think any way more complicated than the first few DAs. […] they were too simple. Those DAs did not address all factors that are important to me. (respondent 1 needs assessment)

Development of online contraceptive DA

The participants’ preferences that resulted from the needs assessment regarding the DA’s content (the contraceptive attributes that were considered important for contraceptive decision making) and structure (e.g. suggestions for multiple methods, transparency and the presentation of extensive and in-depth information) were included in the development of the first online contraceptive DA prototype. The DA prototype is presented as a website with different webpages:

Field testing of the DA

Overall, participants were positive about the first DA prototype; they appreciated the informative, comprehensive and clear style and expected the DA to contribute positively to contraceptive decision making. The field test resulted in positive and negative comments in the three predefined subsets of quality criteria: content, development process and effectiveness. We made minor adaptations to the DA based on the feedback, without changing its overall presentation and structure: the revised DA still consisted of the informative webpages, the questionnaire and personas as described above.

Mostly, it was stated that experiences with the content of the DA complied with or exceeded expectations:

I just didn’t expect that there would be so much information. (respondent 7 field test)

However, some participants mentioned that the DA prototype lacked certain information. For example, what to do in case I forgot to take a contraceptive pill, what to do when I start contraception for the first time, pictures of the contraceptive methods and information regarding placement of contraceptive objects. We revised the DA based on these comments. The comments were addressed as follows: an informative webpage was added with regard to ‘what to do in case I forget to take a contraceptive pill’; a contraceptive persona was added, named ‘the inexperienced woman: I have never used contraception’ and an informative webpage was added to the DA describing what the contraceptive methods look like and how contraceptive objects are placed.

Most participants lacked expectations regarding the development process, which concerns the parties involved in the development and their possible financial interests, as well as the language used in the tool. The DA was experienced as professional because it reported the sources used and parties involved. Negative comments regarding the language used in the DA included that several titles of webpages did not represent the content clearly. These titles were reformulated during revision.

In terms of effectiveness, which concerns informed and value-based decision making, we found that the perceptions of the participants were in accordance with their expectations. Participants not only mentioned that the DA would be a good addition to the contraceptive consultation with the GP, but also explicitly indicated that it could not replace the consultation:

Yes, it really helps. It improves brainstorming with your GP. You receive less new information, you are able to search for more information and you can use it to verify the information together. (respondent 5 field test)

The GP had a decisive role by either supporting the deliberation or by checking and confirming the woman’s decision. The latter includes checking if the combination with other medication is not harmful. Participants mentioned that using the DA could influence decision making and the preparation for the contraceptive consultation with the GP in multiple ways. First, the DA contributes to an increase of knowledge and information regarding available contraceptive methods:

I have more information on everything, I read new things that I never heard of anyway. (respondent 4 field test)

Second, the DA enabled participants to open up to other options than the most familiar or known methods. The DA provided insight into other available methods and their attributes and made users consider and discuss these formerly unknown or less known options:

It [DA] shows what else is there, indeed. And I do not think that if I would have gone to the GP, that she would also have told me all this. (respondent 4 field test)

Third, the DA increased participants’ awareness about specific aspects of a contraceptive method that are important to them:

Well, it makes you more aware … I now have read more of which I think, wow I didn’t know that at all, that this could also happen or that this is a side effect as well. (respondent 6 field test)

The key elements of the DA that positively contribute to decision making varied between participants. Participants mentioned different sections of the DA that they appreciated most. Some valued the directive and specific, step-by-step, information during and at the end of the questionnaire, while others valued the neutral informative texts and overview tables because of their completeness, or valued the contraceptive profiles because this stimulated considering personal values. Multiple participants liked the possibility to choose between these different sections:

There is the information that does not direct you, and then the questionnaire that leads you in a certain direction. So, you can choose to take the directive way, or not, whatever you find pleasant. (respondent 6 field test)

Strengths and limitations

This study has several limitations. First, due to the exploratory nature of this study, we chose to study a relatively homogeneous study population of Dutch female students. All participants were university (of applied sciences) students, who generally have a higher education, socioeconomic status (SES) and health-literacy level. In addition, most participants were non-religious western women. This may have caused a limited inclusion of specific information for different cultural and ethnic backgrounds and religions in the DA. However, the study population of Dutch western female students was expected to use the Internet more frequently as a source for health information than other women, and they were thus more likely to visit and use the developed DA. ²⁰ Nevertheless, future research should take into account the perspectives of a broader range of women regarding educational, cultural, religious and ethnic backgrounds, to adapt a DA to the needs of these various groups. This is important because previous research showed differences in attributes perceived important in the contraceptive decision for these various groups. ³³ In addition, Dutch research studied women who have experienced two or more abortions and found that a significant majority consists of non-western women. ¹¹ Furthermore, cultural belief systems appear to influence perceptions towards contraception and corresponding behaviour. ¹¹ A second limitation is that several participants were satisfied with their current method and not actively looking for another contraceptive method, which may have had an impact on the results concerning the expected influence of the DA. ⁴² Finally, this study provided insight into the influence of the use of a DA on decision making according to end-users, and with that, on the contraceptive consultation with the GP. However, the actual outcome of using the DA for the consultation was not tested, neither was the influence on the eventual contraceptive decision. Future research should take notice on these aspects.

The strength of the study was that the framework that we have used consisted of two models: the IPDAS quality criteria framework and the integrative model.^27,35 We incorporated the integrative model, involving expectations and perceptions regarding the DA, to enhance an open view of participants towards the DA compared to solely examining the IPDAS criteria. The IPDAS framework is presented as a uniform quality checklist for all patient DAs. However, the contraceptive choice is affected by different factors such as personal, socio-economical and geographical factors,^5,16,42 and quality criteria for DAs that vary in topic are incompatible due to the differences in contexts and objectives. ⁴³ Hence, one uniform checklist for all DAs would unlikely comprehend a high quality for every DA. The results of our study confirmed the variances of the perceived quality of the DA among participants, implying the importance of taking more steps than only including the IPDAS framework to determine quality.

Implications for health practice

Using this DA will facilitate informed and shared decision making. DA users experience increased knowledge and awareness, and open up to other options than the familiar methods.^{4,16,32,39,40} Especially in the case of contraceptive decision making, informed and shared decision making is important due to the preference sensitiveness of a contraceptive decision.^1,6 Stimulating the use of DAs within general practices can further optimize contraceptive care and improve communication between patient and provider. General practices may therefore include the use of a DA prior to contraceptive consultations to optimize customized choices following well informed decisions, which may lead to a higher satisfaction with the final choice, and better use of contraceptives.