Educating consumers in self-testing: The development of an online decision aid

Abstract

Context and objective:

Diagnostic self-tests have become available worldwide. The most frequently performed self-tests in the Netherlands are tests to detect high cholesterol and diabetes. Since these tests can be performed without professional guidance, potential consumers need to receive independent information on the pros and cons of self-testing. The aim of this study was to develop a decision aid (DA) on cholesterol and glucose self-testing to encourage informed choice and to meet consumers’ needs and expectations.

Methods:

A DA was developed based on previous research and medical practice guidelines. The first prototype was assessed for content by means of semi-structured interviews with experts (n = 13), followed by usability tests with users (n = 10), leading to improvements to the DA in an iterative process.

Results:

Comments of the experts were grouped into four categories: general comments, textual remarks, technical errors and lay-out. User comments were mainly consistent with the experts’ opinions. Important considerations that were identified concerned the safety of providing this information without direct professional counselling, whether a ranking of available tests should be provided and how strong a warning about self-testing should be.

Conclusion:

Important considerations on how to inform consumers about self-testing were discussed, and led to important changes in the DA. Future research will have to assess the actual use of the DA once it is accessible to the general public, as well as its effects on knowledge and attitude towards self-testing.

Keywords

Cardiovascular risk decision aids diabetes self-testing usability testing

Introduction

Diagnostic self-tests in the form of tests to diagnose a disease or a risk factor without requiring prior consultation with a health professional are increasingly available. Internet searches reveal that self-tests are currently available for 24 conditions, including cancer, chronic conditions and infections (Ryan et al., 2006b; Weijden et al., 2007). In 2008, 18% of the respondents in an online survey in the Netherlands had ever performed a diagnostic self-test on bodily materials, for example, to detect diabetes, high cholesterol, allergies or Chlamydia (Ickenroth et al., 2010). Of these tests, 44% had been performed as home tests, that is, without any professional support or counselling. Other forms of self-testing are the so-called streetcorner tests, direct access and home collect self-tests (Ronda et al., 2009).

Diagnostic self-tests have been a topic of much debate in recent years (Campbell and Klein, 2006; Spielberg et al., 2004). Self-testing fits in with the view of people taking responsibility for their own health, and consumers report several benefits, such as privacy or not having to see a doctor, which saves time (Ickenroth et al., 2014; Ryan et al., 2010). On the other hand, the negative aspects of self-testing should not be ignored. These include the unnecessary costs and distress when a false-positive result is followed by further investigations, or unjustified reassurance after a false-negative result (Walensky and Paltiel, 2006). Moreover, even valid results can lead to inappropriate follow-up behaviour, for example, regarding these as offering a ‘certificate of health’ when the test result is negative (Ryan et al., 2006a; Tymstra and Bieleman, 1987). Despite these considerations, consumers often show a rather straightforward response to the test result: seeing a doctor when it is positive or feeling reassured when it is normal (Ickenroth et al., 2010, 2014). While most consumers seem to have considerable faith in self-tests (Ickenroth et al., 2014), most instruction leaflets included with self-tests do not elaborate on features such as the validity of test results and the possibility of false-positive or false-negative results (Grispen et al., 2012). Available information sources are usually biased by conflicts of interest; the information from the test device producers may be biased towards selling as many self-tests as possible, while the information from clinicians may be biased towards avoiding self-tests. Consumers should therefore have access to independent information on the pros and cons of self-testing. In order to meet this need for independent information, we have developed a decision aid (DA) on self-testing for current and future consumers.

DAs are developed to support patients in medical decision-making. They have been used as adjuncts to counselling by practitioners in order to help patients recognise the value-sensitive nature of the decision and clarify the value they place on the benefits, harms and scientific uncertainties (O’Connor et al., 2009). DAs have been found to improve patients’ knowledge about the available options, including watchful waiting, and to reduce decisional conflict (O’Connor et al., 2009). Since consumers perform self-tests without professional counselling, the challenge for a DA on self-testing is to support autonomy, to guide consumers in the decision to self-test or not, to support them in the different stages of testing, to provide information about the pitfalls of self-testing and to help weigh the positive and negative aspects of self-testing, while also indicating that in some cases medical advice should be sought, independent of the result of the self-test.

In this article, we describe the development of our online DA. After the first draft of the DA had been made, it was tested for usability. The usability of an interface can be defined as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (International Organization for Standardization [ISO], 9241-11). In our study, the aim of the usability testing was to ensure that future consumers would be able to navigate easily through the website, without encountering any technical problems; that they would understand the information given on the website; and that they would be supported in making a decision whether to do a self-test or not.

Methods

Development of the DA

We distinguished two important phases in the process of developing the first draft of the DA: content specification and design (Elwyn et al., 2011). After the first draft had been developed, we tested it for usability.

Content specification phase

In this phase, we explored the experiences and information needs of people who had ever performed a self-test or who had the intention to do a self-test, and assessed the quality of available information on self-testing (Grispen et al., 2010). We found that most self-testers did not experience any problems while performing and interpreting the self-test. Consumers’ information needs were in line with the most frequently used information. The information provided with home test kits was generally of moderate to good quality, although information on test reliability and what to do with a normal or unclear test-result was often limited (Grispen et al., 2010, 2011, 2012; Ickenroth et al., 2010, 2014; Ronda et al., 2009). Next, these results were combined with current medical practice guidelines on screening for diabetes and cardiovascular disease, and used as input for the first draft of the DA. We used the International Patient Decision Aid Standards (IPDAS) (Elwyn et al., 2006) and verified whether the DA covers its dimensions. IPDAS criteria provide evidence-based information for developing DAs. The development of the DA under study was executed before the publication of the update of the IPDAS background documents (Volk et al., 2013). We were not able to meet all IPDAS criteria. The DA does not give, for example, exact probability data of a certain outcome, simply because data on the reliability of self-tests and its diagnostic yields are lacking.

Design phase

The content specification phase resulted in a storyboard (i.e. the visual lay-out of the website, including images, texts and the linkages between pages) for the website. An expert specialising in plain language use and communication reviewed our draft web text and the storyboard. The contents of the first draft of the DA are shown in Table 1. To stimulate consumers to consider the pros and cons of self-testing, we designed a value clarification tool, including two techniques for value clarification. As it is still unclear how values can best be clarified (Abhyankar et al., 2010; Fagerlin et al., 2013), we chose the most frequently used method, which is listing the pros and cons of self-testing. Our value clarification tool was modelled after the tool used by Sivell et al. (2012). Consumers can agree or disagree with the statements given, after which the image of a weighing scale shows a shift towards self-testing or consulting a doctor. Second, consumers can choose their three most important arguments for or against self-testing, which are depicted next to the results of the weighing scales (Van der Pligt et al., 2000). We chose this second technique because not all pros and cons may be of equal importance. For example, the dislike of blood sampling by patients themselves may outweigh four good reasons for self-testing.

Table 1.

Contents of the first draft of the decision aid.

- A homepage giving an introduction to visitors of the website. A warning is given that the information provided on the website is not suitable for people who are already being treated for diabetes or cardiovascular disease.
- General information on self-testing, including information on the different kinds of self-tests available, the reliability of self-tests in general and 11 cues what to check before doing a self-test.
- Information about cholesterol self-testing, information about the risk factors for cardiovascular disease, the role of cholesterol as a risk factor in cardiovascular disease, an interactive tool to determine personal risk for cardiovascular disease, information on the different kinds of cholesterol tests (tests for total cholesterol and high-density lipoprotein [HDL] cholesterol), how these self-tests have to be performed and what to do with the test result, including advise when to see a doctor, and information on lifestyle changes.
- Information about diabetes self-testing, similar to the information regarding the cholesterol self-test, including an interactive tool to assess personal risk for developing diabetes.
- Frequently asked questions (FAQ).
- A value clarification tool to weigh up the perceived benefits and disadvantages of self-testing. This tool first shows 14 propositions on self-testing, of which 7 represent advantages of self-testing, and the other 7 the disadvantages of self-testing. Examples of these propositions are as follows: ‘I think it is an advantage that I can do the test whenever I want to’ or ‘I think that it is a disadvantage of self-testing that I have to get a blood sample myself’. People can indicate whether they agree, disagree or feel neutral about these propositions. When they indicate they agree with a proposition, the weighing scales depicted next to the propositions start shifting towards doing a self-test or seeing a doctor. After respondents have filled out all the propositions, they are asked which propositions are most important to them. They are given advise whether they prefer seeing a doctor or doing a self-test.
- A sitemap, disclaimer and contact information.

Expert opinion

Experts were selected from various stakeholder groups: general practitioners, researchers in the field of cardiovascular risk and prevention, experts on health promotion, information technology (IT) and communication policy officers from a health insurance company and staff members of Diagned (Dutch organisation of manufacturers and importers of in-vitro diagnostics) and the Dutch Consumers Organisation. The experts were invited to view the DA, after which they were interviewed by one of the researchers (see Appendix 1 for the interview guide), or approached by e-mail to answer specific questions on the functionality of the website, as well as to provide a general opinion on the contents of the DA. The interviews were audio-taped and transcribed, and were subsequently – together with the e-mail answers – analysed using a simple thematic content analysis (Green and Thorogood, 2004).

Usability testing

Usability was tested using a user-based testing method, in which users use the website while thinking aloud (Bastien, 2010; Jaspers, 2009). Participants for the usability tests were recruited through posters displayed at the Maastricht University Hospital. The posters invited participation by people over 18 years of age, and explained the aims of the study and the incentive for participation (a gift voucher). People could respond by e-mail or telephone. Respondents were screened for age, gender, educational level and intensity of Internet use, in order to recruit a heterogeneous sample.

The usability tests were carried out in the usability laboratory at the Maastricht University using Morae software (http://www.techsmith.com/morae.asp) and were administered by one of the researchers (M.I.). Participants were seated behind a desk with only a laptop (15″ screen, Internet Explorer browser). The researcher was present in the same room during the whole session. Informed consent was obtained, and participants were asked to fill out a short questionnaire asking about their general characteristics. Next, participants were instructed on how to use the think-aloud method (Jaspers, 2009). They were asked to navigate freely around the website while thinking aloud and while completing three tasks that were presented in writing (Table 2). Morae software simultaneously recorded the participants’ navigation through the website and their comments. Markers were inserted in the recording when comments were made or when technical errors were found. After navigation through the website had been completed, a short interview was held with the participants (interview guide provided in Appendix 1). The comments of the users during the usability tests and the markers added in the Morae software were used to improve the DA. Changes were made after one or more usability tests, in an iterative process (Figure 1). In the case of diverging opinions, the issues were discussed in the project team until consensus was achieved. The next series of usability tests then showed whether the changes were successful or whether more improvements were needed. We continued to invite users for the usability test until a satisfactory level of usability was achieved.

Table 2.

Tasks involved in the usability test.

Task number	Description
1	You are searching for the symptoms of diabetes. Try to find this information on the website.
2	You are searching for more information on cholesterol, and about ‘good’ and ‘bad’ cholesterol. Try to find this information on the website.
3	You are considering doing a cholesterol self-test, but you are not sure whether the test is useful. Try to find out your personal risk, and if a cholesterol test is useful for you.
4	You are considering doing a diabetes self-test, but you are not sure whether the test is useful. Try to find out your personal risk, and if a diabetes test is useful for you.
5	You have bought a cholesterol self-test, and you would like to have more information on how to perform the self-test. Try to find this information on the website.
6	You are considering buying a diabetes self-test, and you would like to have more information on how to perform the test. Try to find this information on the website.
7	You are searching for more information on what to do to avoid cardiovascular disease. Try to find this information on the website.
8	You have performed a cholesterol self-test, but you do not know what to do with the test result. Try to find this information on the website.
9	You have performed a diabetes self-test, but you do not know what to do with the test result. Try to find this information on the website.
10	You would like to know more about the reliability of self-tests. Try to find this information on the website.
11	You are still in doubt whether to do a self-test or to visit a doctor. Try to compare the pros and cons of a self-test for yourself on the website.

Figure 1.

Timing of the usability tests and changes made to the decision aid.

Results

Expert opinion

In October 2010, 15 experts from Maastricht University were invited to view the DA, 11 of whom agreed to take part and were interviewed in the course of 1 week. We also received feedback from two of the external experts we had invited (staff members of a health insurance company and Diagned). The comments of the experts could be assigned to one of the following categories: general comments, textual remarks, technical errors and the lay-out of the DA. The feedback from the different stakeholders (one of which was the Dutch organisation of manufacturers and importers of in vitro diagnostics) was rather more homogeneous, and was not particularly aimed at only promoting or discouraging self-testing.

General comments

The neutrality of the central decision at stake in the DA was debated, that is, whether the DA should aim for a free choice between self-testing and seeing a doctor, or whether it should contain a strong warning about self-testing, and put greater emphasis on the role of the doctor. Suggestions were made about informing people about the dilemmas of self-testing and the goal of the website. Other general comments were made on the use of the interactive cardiovascular risk tool (e.g. whether it is medically sound to use these risk calculators and what advice should be given based on the results) and on the use of the value clarification tool (e.g. should it give users advice on whether to do a self-test or not, or should users only weigh pros and cons for themselves and draw their own conclusions?). Some of the experts asked for a ranking of the various brands of available tests, with information on aspects like reliability and user-friendliness.

Textual remarks, technical errors and lay-out

Most of the experts’ comments concerned the contents of the website, for example, parts of texts that were regarded as medically unsound, or descriptions that were too complex. Several of the experts indicated that more information should be provided on the general limitations of diagnostic testing. The information on the reliability of tests was regarded as insufficient. In addition, they wanted to see a clearer introduction to the use of the value clarification tool, and they commented that the statements used for the weighing scales were not always clear. Several technical errors were identified, such as hyperlinks that malfunctioned, too many scrollbars and the absence of an option to increase the font size of the text. Comments on the lay-out of the DA related to the position of the menu, the font used and the use of colours and images.

Results from the usability tests and changes to the DA

The usability tests for the DA involved 10 participants. Our sample included people of different ages (range = 19–64 years), educational level and Internet experience (Table 3). The usability tests were done in October and November 2010 and in March 2011 (Figure 1).

Table 3.

Personal characteristics of participants involved in the usability tests.

Participant	Age (years)	Gender	Education	Hours per week on the Internet	Ever performed self-test	Considering doing a self-test
1	62	M	Intermediate vocational education	7	No	No
2	38	M	Higher vocational education	25–30	No	Yes
3	56	F	Higher vocational education	15	No	Yes
4	29	F	Primary school	7	No	No
5	49	M	Higher vocational education	7	No	No
6	60	F	Intermediate vocational education	20	Yes	Yes
7	19	F	Higher secondary education	30	No	Yes
8	53	F	Higher vocational education	25	Yes	Yes
9	64	F	Lower secondary education	8	No	Yes
10	22	M	Higher secondary education	15	No	No

On the whole, participants had no problems understanding the web texts; they found the information about the DA to be balanced, and said it met their expectations. The first three usability tests identified problems similar to those identified by the experts and concerned similar topics or parts of the texts, and the opinions and suggestions for improvement were largely consistent. Major changes were then made, resulting in the addition of a paragraph explaining the reason for developing the DA, explaining the different views on self-testing and offering more strongly worded instruction on the importance and timing of visiting a doctor. An important change regarded the core decision at stake in the DA, which was most clearly visible in the value clarification tool: instead of weighing the pros and cons of a self-test versus consulting a doctor, the revised version asked people to weigh the pros and cons of doing a self-test or not. The value clarification tool no longer provided a direct recommendation whether to perform a self-test or to see a doctor, but instead aimed at careful weighing of the pros and cons, without giving advice. We realise that this will lead to some users being disappointed. Nevertheless, during the final usability tests, most users indicated that they thought the warnings about the pitfalls of self-testing were clear, but that they still felt free to make their own choice. Technical problems were solved, and lay-out and texts were improved.

The other seven usability tests revealed the need for minor changes, for example, hyperlinks that malfunctioned and words or text passages that were still unclear. Some of the comments did not lead to changes, for example because of differences of opinions among participants of the usability tests, because changes would not be medically sound or because of technical limitations.

Discussion

Diagnostic self-testing has elicited a great deal of debate, and many pros and cons of self-testing have been reported. By designing a DA on self-testing, we have tried to translate this debate into relatively neutral information. Our aim was to design a DA that would enable consumers to make an informed decision about self-testing based on an evaluation of all the possible consequences of all available alternatives in accordance with their values and making appropriate trade-offs.

The process of developing a DA on diagnostic self-testing confronted us with a number of challenging dilemmas, because a DA for consumers without guidance from professional health care is different from more ‘traditional’ DAs about screening options used within the doctor–patient relationship (O’Connor et al., 2009). In self-testing, consumers do not consult a doctor at all. Since a DA offers health-related advice such as information on personal risk and how to interpret a test result, it needs to be safe to use without additional professional counselling. The general opinion of the experts as well as the end-users was that a diagnostic self-test could never replace consulting a clinician. These arguments led us to include stronger statements in the website concerning the limitations of self-tests and the role of the health professional.

The second dilemma was that we were not able to meet one of the needs that was expressed, namely to help users choose ‘the best self-test’ by offering a ranking of the available tests based on quality criteria. Whereas DAs mostly present probabilities of positive or negative outcomes (Elwyn et al., 2006), these values are often not available for diagnostic self-testing devices. Self-tests are provided in various formats by different companies in a market that is constantly changing. In many cases, no information is provided on the precise validity and reliability of the tests (Grispen et al., 2012) and, to our knowledge, there have been no studies into the use of specific self-tests for cholesterol and diabetes that assessed the risk of an adverse outcome. We were therefore unable to provide this information. Although consumers as well as some experts would have preferred more information on quality, or even a quality ranking of tests, we did not want to become involved in commercial concerns in any way, so we decided not to provide such rankings.

Since the goal of the DA is to enable informed decision-making, consumers should have sufficient knowledge not only regarding the reliability of diagnostic self-tests in general, but also on the different phases of testing, in order to decide whether a self-test for a particular disease is relevant to them. Our DA aims to guide consumers through these stages: ‘Do I need the test?’ (i.e. ‘Am I at-risk for developing a particular disorder?’), ‘How should I perform the test?’ and ‘How should I interpret the results?’ (i.e. ‘What follow-up steps should I take?’). It is debatable whether the additions we made based on the usability tests, which led to quite extensive information, would affect the actual use of the website when consumers access the DA online. Further research is needed to assess whether people visit these specific pages, and in which domains they are able to increase their knowledge.

Strengths and limitations

The strengths of the development process of our DA include the thorough approach we adopted in developing the DA with an iterative process, involving experts and users. Due to practical limitations, we were not able to meet all IPDAS criteria (e.g. often exact numbers on the reliability of these tests are unknown, and therefore we were not able to provide numbers on the exact probability of, for example, receiving a false-negative or false-positive test result).

The decision at stake is whether to do a self-test or not. The latter option includes doing nothing or consulting a doctor. A potentially weak aspect of our DA is that we asked consumers to weigh the pros and cons of self-testing, which are not exactly the same as the pros and cons of the alternative options. There is evidence from the psychological literature that people do not consider information that is not explicitly presented (Abhyankar et al., 2010). The DA may therefore be improved by adding more information on the alternatives to self-testing.

By using a think-aloud method in combination with video and audio recordings of the usability tests, the study has provided us with a more in-depth understanding of the way people use our website and the problems they encounter in using the site and its interactive tools.

Almost all of the experts we invited to evaluate the website were employed by the same university as the project team. Although the background of the experts was diverse, it is unclear whether involving more experts might have elicited other views. Furthermore, we only asked the experts to view the website once, and did not ask them for further feedback after improvements had been made. Additionally, no experts on computer usability testing were directly involved in our usability tests, which meant that, for example, we did not use specific quantitative usability metrics such as task time or task level satisfaction. Such metrics could have provided more information about the usability of the DA. The sample for the usability testing consisted of 10 users. Although the level of usability seemed to be acceptable after the 10th usability test, this sample is quite small for a website that is fairly complex, and we may not have identified all of the usability issues.

Conclusion

The development and usability testing of a DA on self-testing has provided useful insights into the use of the DA and the arguments of experts and users concerning its usability and contents. It also led to some debate on the best way to inform consumers about self-testing, and to important improvements in the DA. The development of the DA still has to go through a final step, that of field testing. That is, its actual use will have to be assessed using weblog files after the DA has become accessible to the general public on the Internet. Additionally, further studies are needed to assess the actual effect of the DA on knowledge, attitudes towards self-testing, worrying, the uptake of self-tests and follow-up behaviour after self-testing (Ickenroth et al., 2012).

Footnotes

Appendix

Appendix 1

Interview guides

Experts
1	Is the information provided correct?
2	Is the information provided in a sensible way?
3	Is this a proper way of providing information to consumers?
4	What can be improved?
Users
1	What were your expectations of the decision aid?
2	Do you consider the information provided on the decision aid to be balanced?
3	Do you think the weighing scales are useful?
4	Are the texts easy to read, or too difficult?
5	Is the amount of information provided in the decision aid sufficient?
6	What do you think about the lay-out of the decision aid?
7	Do you have any suggestions for other improvements?

Funding

This work was supported by the Netherlands Organisation for Health Research and Development (ZonMW Prevention), grant number 50-50101-96-406. Supplementary financial support has been provided by the Centraal Ziekenfonds (CZ) health insurance company. None of the sources of funding influenced the study design, the writing of the manuscript or the decision to submit the manuscript for publication.

References

Abhyankar

Bekker

Summers

. (2010) Why values elicitation techniques enable people to make informed decisions about cancer trial participation. Health Expectations 14(Suppl. 1): 20–32.

Bastien

(2010) Usability testing: A review of some methodological and technical aspects of the method. International Journal of Medical Informatics 79(4): e18–e23.

Campbell

Klein

(2006) Home testing to detect human immunodeficiency virus: Boon or bane? Journal of Clinical Microbiology 44(10): 3473–3476.

Elwyn

Kreuwel

Durand

. (2011) How to develop web-based decision support interventions for patients: A process map. Patient Education and Counseling 82(2): 260–265.

Elwyn

O’Connor

Stacey

. (2006) Developing a quality criteria framework for patient decision aids: Online international Delphi consensus process. British Medical Journal 333: 417–423.

Fagerlin

Pignone

Abhyankar

. (2013) Clarifying values: An updates review. BMC Medical Informatics and Decision Making 13(Suppl. 2): S8.

Green

Thorogood

(2004) Qualitative Methods for Health Research. London: SAGE.

Grispen

Ickenroth

De Vries

. (2010) Improving behaviour in self-testing (IBIS): Study on frequency of use, consequences, information needs and use, and quality of currently available consumer information (protocol). BMC Public Health 10: 453.

Grispen

JEJ

Ickenroth

MHP

De Vries

. (2012) Quality and use of consumer information provided with home test kits: Room for improvement. Health Expectations. Epub ahead of print 19 July. DOI:10.1111/j.1369-7625.2012.00805.x.

10.

Grispen

Ronda

Dinant

. (2011) To test or not to test: A cross-sectional survey of the psychosocial determinants of self-testing for cholesterol, glucose, and HIV. BMC Public Health 11: 112.

11.

Ickenroth

Grispen

De Vries

. (2012) A single-blind randomised controlled trial of the effects of a web-based decision aid on self-testing for cholesterol and diabetes: Study protocol. BMC Public Health 12(1): 6.

12.

Ickenroth

Grispen

Ronda

. (2014) Motivation and experiences of self-testers regarding tests for cardiovascular risk factors. Health Expectations 17(1): 60–72.

13.

Ickenroth

Ronda

Grispen

. (2010) How do people respond to self-test results? A cross-sectional survey. BMC Family Practice 11: 77.

14.

Jaspers

(2009) A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. International Journal of Medical Informatics 78(5): 340–353.

15.

O’Connor

Bennett

Stacey

. (2009) Decision aids for people facing health treatment or screening decisions. Cochrane Database of Systematic Reviews 3: CD001431.

16.

Ronda

Portegijs

Dinant

. (2009) Use of diagnostic self-tests on body materials among Internet users in the Netherlands: Prevalence and correlates of use. BMC Public Health 9(1): 100.

17.

Ryan

Greenfield

McManus

. (2006a) Self-care – Has DIY gone too far? British Journal of General Practice 56(533): 907–908.

18.

Ryan

Wilson

Greenfield

. (2006b) Range of self-tests available to buy in the United Kingdom: An Internet survey. Journal of Public Health 28(4): 370–374.

19.

Ryan

Ives

Wilson

. (2010) Why members of the public self-test: An interview study. Family Practice 27(5): 570–581.

20.

Sivell

Marsh

Edwards

. (2012) Theory-based design and field-testing of an intervention to support women choosing surgery for breast cancer: BresDex. Patient Education and Counseling 86(2): 179–188.

21.

Spielberg

Levine

Weaver

(2004) Self-testing for HIV: A new option for HIV prevention? The Lancet Infectious Diseases 4(10): 640–646.

22.

Tymstra

Bieleman

(1987) The psychosocial impact of mass screening for cardiovascular risk factors. Family Practice 4(4): 287–290.

23.

Van der Pligt

de Vries

Manstead

ASR

. (2000) The importance of being selective: Weighing the role of attribute importance in attitudinal judgment. Advances in Experimental Social Psychology 32: 135–200.

24.

Volk

Llewellyn-Thomas

Stacey

. (2013) Ten years of the International Patient Decision Aid Standards Collaboration: Evolution of the core dimensions for assessing the quality of patient decision aids. BMC Medical Informatics and Decision Making 13(Suppl. 2): S1.

25.

Walensky

Paltiel

(2006) Rapid HIV testing at home: Does it solve a problem or create one? Annals of Internal Medicine 145(6): 459–462.

26.

Weijden

Ronda

Norg

. (2007) Diagnostische zelftests op lichaamsmateriaal: aanbod, validiteit en gebruik door de consument [Diagnostic Self-Tests on Body Materials: Availability, Validity and Frequency of Use]. Maastricht: CAPHRI, School for Public Health and Primary Care, Maastricht University.