Incorporating a Drug Information Consult into a Pharmacy Practice Course to Build Literature Search and Evaluation Skills Through a 3-Stage Model

Abstract

Objective:

Pharmacy students need to be equipped with skills to research and evaluate literature to effectively apply evidence-based medicine (EBM) in practice. To prepare them, a 3-stage approach to writing a drug information consult (3sDIC) was incorporated into a pharmacy course. The primary objective was to assess students’ abilities to retrieve and analyze literature pursuant to a drug information consult. Secondary objectives were to examine feasibility of faculty participation and continuation of the assignment.

Design:

Ninety students were given a clinical scenario about a patient. The assignment consisted of 3 stages incorporating use of the Population, Intervention, Comparison intervention, Outcome (PICO) method and modified systematic approach (MSA) for stage 1, evaluation of primary literature to write a draft for stage 2, and stage 3, the final consult. All 3 stages were reviewed and graded by faculty.

Assessment:

All students completed the 3sDIC, with no grade failures. The rubric employed by faculty was effective, providing students the opportunity to improve the consult. The 3sDIC was found to be feasible with adequate faculty support.

Conclusion:

The 3sDIC, although not a substitute for a complete drug information course, demonstrated a streamlined approach for Pharmacy year 2 (P2) students to acquire and develop drug information skills.

Keywords

drug information literature evaluation PICO pharmacy education

Background

Evidence-based medicine (EBM) is defined as the “conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.”¹ Practicing EBM entails health-care professionals incorporating comprehensive available literature, which includes but is not limited to a combination of randomized controlled trials, meta-analyses, case reports, practice guidelines, and review articles, which is then joined with clinical expertise into decision-making and problem-solving activities.² Health-care providers and patients often turn to pharmacists to help guide them in the therapeutic decision-making process to ensure positive health outcomes. EBM is now included in the curriculum of most colleges of pharmacy and is included in the 2016 Accreditation Standards and Key Elements for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree.³ The Center for the Advancement of Pharmaceutical Education (CAPE) outcomes set by the American Association of Colleges of Pharmacy (AACP) requires pharmacy students to also meet a set of standards that fall under providing drug information and education.⁴ For example, domain 2 in the CAPE outcomes focuses on “Essentials for Practice and Care” where some of the objectives are to ensure students are able to collect pertinent patient information and “interpret evidence” to provide evidence-based recommendations.⁴

There is a need for increased student learning experiences to build drug information skills. The teaching of literature search and evaluation skills in the curricula of pharmacy schools has been discussed for over 4 decades. Methods include librarian-led lectures and demonstrations, application of a systematic approach,⁵ formal and informal class presentation of findings, group discussions, and journal clubs.⁶ Although drug information pertaining to retrieval of health sciences information and analysis to answer “specific patient-care and or drug-related questions” is part of the current ACPE standards, no clear guidelines exist on how to implement these standards.³ Emphasis has been placed on problem solving, critical thinking, and self-directed learning in the pharmacy curricula to equip students with essential drug literature search and evaluation skills.^7,8 Studies have demonstrated that learning-centered out-of-class assignments positively impact academic performance.⁹

Drug information mastery is an expectation of all pharmacy students, especially in view of the diversification of drug information practice into specialty areas such as medication safety, informatics and pharmacoeconomics.^10,11 Therefore, the pharmacist’s role in providing drug information is more pertinent than ever.¹² This responsibility then requires pharmacists to be able to effectively practice EBM. The first step in EBM is formulating a well-focused question.¹³ The question may be one that has or has not been answered before. Once a drug information question is identified, the next step in EBM is the ability to identify and evaluate literature.¹⁴ This process requires skills to effectively search for the appropriate information; critically analyze and evaluate the information; and apply the information to the question at hand, a task that can be daunting. In order to help prepare pharmacy students to obtain these skills, the modified systematic approach (MSA) along with the Patient Intervention Comparison Outcome (PICO) framework are techniques pharmacy students and pharmacists can utilize.^2,15 Moreover, PICO is often used in clinical EBM.¹⁴

The PICO framework has been utilized to improve search strategy skills. The PICO mnemonic requires specification of the Population, Intervention, Comparison intervention, and Outcomes of interest.¹⁶ PICO essentially enables the practitioner to form the research question and assists with identification of main concepts to facilitate an effective search strategy.¹⁶ For example, using PubMed^®, the practitioner can MeSH^® the main concepts together to retrieve the appropriate literature. Schardt and colleagues observed use of the PICO template by medical interns and residents during inpatient general medicine rotations to assess the number of relevant articles retrieved when provided with a clinical question.¹³ The study found that combining PICO with the PubMed interface resulted in higher accuracy rates of appropriate citations retrieved when compared to solely using the PubMed Web interface.¹³

The MSA, initially created by Watanabe and colleagues, is used in several pharmacy schools across the country to convey the process of answering drug information questions. The approach begins with collecting the requestor’s information, retrieving background information, identifying what is being asked, performing a search strategy, formulating a response, and lastly documenting and translating the information back to the requestor.^2,15 Lavasa and colleagues demonstrated use of drug information templates utilizing the MSA. These templates significantly improved students’ abilities to obtain pertinent background information and complete references. Importantly, the templates improved students’ abilities to provide clear and evidence-based recommendations.¹⁷ Figure 1 highlights key components of the PICO/MSA method for application to drug information questions.

Although the MSA and PICO framework have both been shown to improve health information retrieval and accuracy, the concept of teaching and practicing both approaches together has not been studied. Therefore, the present study employed a 3-stage approach to writing a drug information consult (3sDIC) as a unique method to include both the MSA and the PICO framework for introducing students to the task. The primary objective of this drug information consult was to assess the efficacy of the consult and to evaluate the students’ perceptions on the approach. A second objective was to assess the feasibility of the assignment in the future and assess faculty perceptions on the assignment.

Methods

At Touro College of Pharmacy (TCOP), students have 2 years of didactic curriculum and 2 years of experiential rotations. During their P2 year, students enroll in a pharmacy practice course which, among various components, includes drug information. A total of 90 students in their P2 year enrolled in the pharmacy practice course in the fall semester were studied and given the drug information assignment. TCOP Institutional Review Board approval was obtained for the completion of the study. Permission was granted to approach students about completing the survey during a regularly scheduled class. An overview of the project and explanatory statement was presented to and acknowledged by all participants. In the year previously, the same course was offered with a drug information component of didactic lectures and laboratory activities related to drug information; however, no assignment on writing a drug information consult was included.

During the first week of class, each student was assigned a drug information question on “real-life” scenarios that occurred at the instructor’s practice. The questions were patient specific and pertained to the area of cardiovascular pharmacotherapy. Additional information about the patient’s medical history, medication use, allergies, social history, vital signs, and pertinent laboratory test results were provided. Each student was assigned to a faculty member who would be primarily responsible for reading and grading each stage of the assignment. A total of 5 drug information questions were circulated among the 90 students, allowing each faculty to advise 18 total students. The assignment was broken down into 3 stages, which was to be completed over 3 months. Grades were calculated based out of 100% using a rubric for each stage.

The first stage of the assignment entailed use of PubMed to perform a search strategy utilizing the PICO format to identify what the provider was truly asking. Students were required to identify at least 2 primary literature articles that were applicable to the drug information query. In addition to the use of primary literature, students were able to utilize tertiary resources (eg, review articles, Lexi-Comp™, Micromedex™) to support their consult. To coincide with stage 1 of the assignment, students received a didactic library module on the use of PubMed and utilization of PICO in EBM during the first week of class. Upon completion of the assignment, the faculty advisor graded the work with a standardized rubric and determined whether the student retrieved sufficient and appropriate literature to advance to the next stage. The advisor was responsible for providing feedback on the students’ search results. If it was not sufficient, students were asked to complete the task again.

The second stage required students to read through the literature and begin the process of writing their drug information consult. To guide them with this process, they received a step-by-step guideline prepared by a clinical faculty member regarding how to write their paper. Additionally, they received didactic classes on the MSA to answer drug information questions, which included how to read, analyze, and critically evaluate drug information. It should be noted that in the course, students were taught all steps of the MSA, however the students were only responsible for steps 4, 5, and 6 (Figure 1). Once completed, students submitted a first draft to their faculty advisor. The faculty advisor was responsible for grading the first draft with a standardized rubric and providing comments and feedback for the final paper.

The third stage of the assignment required students to review the comments and feedback after and make appropriate changes. The first, second, and third stages were each worth 10%, 50%, and 40%, respectively. However, the assignment contributed to a total of 10% of their final course grade, and it was emphasized that they were to work independently to complete the exercise. Students were given 2 weeks to complete the changes and develop any additional iterations of the paper.

To assess students’ perceptions of their drug information skills before and after completion of the assignment, all 90 students were provided with a survey which consisted of a 5-point Likert-type scale (1 = strongly disagree to 5 = strongly agree) to help determine whether students felt confident with their drug information searching and analytical skills after completion of the assignment. The survey was anonymous and voluntary. Additionally, students were asked if the assignment should be continued in the course and were provided the opportunity to include constructive feedback to gauge what improvement was necessary to maximize effectiveness of the assignment in the future. Faculty were also provided with a survey to identify the students’ performance level throughout each stage, determine feasibility of participating in the course assignment, and obtain impressions on continuing the assignment in the future. Quantitative responses were based on a Likert-type scale (1 = strongly disagree to 5 = strongly agree). Descriptive statistics were employed to present the results.

Results

Ninety students completed the 3sDIC, with no grade failures. An overall final cumulative average of 85.4% was calculated among the 90 students. Each faculty advisor had 1 scenario to grade with 18 students per group. Student grades were compared against each drug information question by calculating the average grade within each stage (Table 1). Average grades students earned from stage 1 through stage 3 were similar among the 5 clinical scenarios, where student grades showed an improvement between stage 2 (first rough draft) and stage 3 (final paper) with an average increase of 13 points between all 5 drug information questions (Table 1).

Table 1.

Student Grade Averages for Each Stage Among Faculty Advisors.^a

Scenario	Stage 1 grade	Stage 2 grade	Stage 3 grade	Final grade average
1	80%	75%	92%	82%
2	80%	79%	98%	87%
3	85%	78%	93%	86%
4	82%	87%	87%	85%
5	84%	80%	94%	87%
Final grade average for 90 students	85.4%

Abbreviation: PICO, Patient Intervention Comparison Outcome.

^aThere were a total of 18 students assigned each scenario (N = 90); stage 1: PICO assignment, stage 2: first draft, stage 3: final paper.

Of the 90 students, 46 (51%) completed the survey. Based on a Likert-type scale of 1 to 5, an improvement in confidence was found in performing skills associated with all 3 stages (Table 2). The survey showed the students’ level of confidence in the following areas improved before and after completion of the assignment: reading and analyzing primary literature, their ability to use tertiary drug information resources, and answering a drug information question regarding a specific patient (Table 2). Additionally, positive overall student satisfaction was demonstrated (Table 3).

Table 2.

Student Survey on Ratings of Confidence.

Pre-assignment question (n = 46)	Average rating (1-5)	Post-assignment question (n = 46)	Average rating (1-5)
Your prior level of confidence for reading and analyzing primary literature?	3.2	Post completion, your level of confidence for reading and analyzing primary literature?	4.1
Your prior level of confidence in using tertiary DI resources?	3.9	Post completion, your level of confidence using tertiary DI resources?	4.6
Your prior level of confidence in answering DI questions regarding a specific patient?	3.7	Post completion, your level of confidence answering DI questions regarding a specific patient?	4.3

Abbreviation: DI, drug information. ^aLikert-type scale: 5 = high, 4 = medium, 3 = low, 2 = very low, 1 = none.

Table 3.

Student Feedback.

Question	Average rating (1-5)
This assignment helped me understand how to break a DI question down using PICO	4.0
The feedback I received from my advisor was helpful for the research I did that resulted in the first draft of the DIC	4.0
The comments I received from my advisor helped me make revisions and thereby finalize my DIC	4.0
Overall, the 3 stage process helped me learn research skills I can apply in the future	4.2
Overall, the DIC helped me apply data and my clinical knowledge thus far to make recommendations in the future	4.1
Although the DIC requires significant time, I recommend the process be continued for future students because it provides training of important skills	4.2
Overall, I am satisfied with what I learned throughout the steps of the assignment	4.2

Abbreviations: DIC, drug information consult; DI, drug information; PICO, Patient Intervention Comparison Outcome.

In respect to the faculty response, all 5 faculty members completed 3 surveys corresponding to each stage to identify perceptions on student performance (Table 4). An overall trend was noted, where faculty’s student perception on performance improved from stage 1 to stage 2 to stage 3 (Table 4).

Table 4.

Survey to Faculty on Student Performance for Stages 1, 2, 3.^a

	Response rate (%)			Mean response rate^b
1. Faculty Perceptions of Student Performance for Stage 1	Above Average	Average	Below Average	Mean response rate^b
Q1: Overall impression on student performance completing the PICO tables appropriately	20%	80%	0%	3.2
Q2: Overall impression with regard to students identifying two resources of relevant primary literature	40%	20%	40%	3
Q3: Overall impression students demonstrated use of appropriate MESH terms	60%	40%	0%	3.6
Mean response rate for all 3 questions				3.3
2. Faculty Perceptions of Student Performance for Stage 2	Most of the Time	Sometimes	Rarely	Mean Response Rate^c
Q1: Students followed the format and instructions provided to write the first draft	80%	20%	0%	3.8
Q2: The content of the student’s paper was supported by primary literature with optional tertiary resources	75%	25%	0%	3.6
Q3: The recommendations were logical and followed evidence found in the article(s) utilized	40%	60%	0%	3.4
Mean response rate for all 3 questions				3.6
3. Faculty Perceptions of Student Performance for Stage 3	Always	Most of the Time	Sometimes	Mean Response Rate^c
Q1: Students took into account the recommendations on the draft to apply to the final paper	100%	0%	0%	5
Q2: Arguments were logical and recommendations were supported by the content written	0%	80%	20%	3.8
Mean response rate for all 2 questions				4.4
4. Faculty overall impression of the assignment	Strongly Agree	Agree	Neutral	Mean Response Rate^d
Q1: The assignment helped build student skills for analyzing a hypothetical drug information question posed by a provider	20%	80%	0%	4.2
Q2: The assignment helped build student skills in formulating a search strategy and identifying relevant primary research articles	0%	100%	0%	4
Q3: The assignment helped build student skills for interpreting research literature and synthesizing findings relative to a drug information question	0%	100%	0%	4
Q4: The assignment helped build student skills for writing recommendations that are clear, logical and based on evidence	20%	80%	0%	4.2
Q5: The 3 stage assignment was a solid introduction for helping prepare students to answer drug information questions in future settings	40%	60%	0%	4.4
Mean response rate for all 5 questions				4.16

Abbreviation: PICO, Patient Intervention Comparison Outcome.

^aN = 5.

^bMean found based on Likert-type scale 1-5: excellent = 5, above average = 4, average = 3, below average = 2, and poor = 1.

^cMean found based on Likert-type scale 1-5: always = 5, most of the time = 4, sometimes = 3, rarely = 2, and never = 1.

^dMean found based on Likert-type scale 1-5: strongly agree = 5, agree = 4, neutral = 3, disagree = 2, and strongly disagree = 1.

In terms of grading the assignment, the time spent grading each stage showed least spent on grading stage 1 (6 to 15 minutes), followed by stage 3 (11-20 minutes). Faculty noted spending the most time grading stage 2 (21-30 minutes), which was the first draft of the consult paper. All 5 faculty strongly agreed grading both stage 1 and stage 3 to be an efficient process (Likert-type rating: 5). However, only 60% of the faculty agreed grading stage 2 was an efficient process where 40% disagreed because of the time it took to grade each consult. Faculty were given an additional survey on their overall impression of the assignment (Table 4). Faculty agreed the assignment helped build student skills in formulation of search strategies and identification of relevant primary research articles, interpretation of research and literature, and skills for writing recommendations that were clear, logical, and evidence based (Table 4). Overall, faculty agreed the 3-stage assignment was a solid introduction in helping prepare students to answer drug information questions in future settings and would recommend the assignment be continued in the course (Figure 2). In comparison, student feedback with respect to continuing the assignment in the future, majority (82%) felt the assignment should be continued, where some students (13%) somewhat agreed and a minimal amount of students disagreed (4%; Figure 3).

Discussion

To build search strategy and analytical skills, introducing concepts of the MSA and PICO format together can help pharmacy students distinguish necessary information to answer a clinical drug information question. This study utilized a 3-stage approach to help students identify how to break down a clinical drug information question, determine a search strategy, and identify appropriate literature through PubMed that could be applied to the question. The study evaluated student grade performance and student perceptions of the assignment as well as faculty perceptions of student performance and feasibility of the assignment. Student grade performance reflected an improvement when moving from stage 2 to stage 3, while use of the standardized rubrics helped with grading between all 5 faculty members to deliver a consistent process. This is consistent with the results of Costa de Sousa and colleagues, showing that when a more complex activity that involved the same development process as the previous activity was performed, the students’ grades increased as a result of knowledge learned from the previous activity.¹⁸ However, we discovered faculty perceptions of student performance varied between all 3 stages. Although grade performance improved from stage 2 to stage 3, faculty felt student ability to provide recommendations and support with the evidence in a logical manner showed some improvement. This may imply students require more practice analyzing literature and applying it to a specific question.

At this stage in their career, applying true EBM would require students to have clinical experience to help support their drug information consult. Additionally, this class had not taken our pharmacoepidemiology and biostatistics course, which may have limited their ability to interpret literature. This limitation coincides with some of the students’ survey responses as they suggested having more practice in analyzing literature through either journal clubs or having the epidemiology and biostatistics course prior to fulfilling the assignment.

Faculty agreed that the 3-stage approach helped students to build their research and analytical skills and recommended the assignment should be provided for future classes. Although in the survey from stage 1, 40% of faculty felt the overall impression with regard to students identifying 2 resources of relevant primary literature was above average, it is important to note the survey was conducted on only a total of 5 faculty, making it difficult to make specific implications. However, it appears the perceptions from stage 1 to stage 3 on students’ overall performance were positive which can be correlated with the grade improvement between stage 2 (draft consult) and stage 3 (final consult). Additionally, the rubric helped standardize the grading process as students can sometimes raise concerns about the fairness surrounding how their assignments are graded and imply other graders may be easier or harder on evaluating student performance. The study also demonstrated positive student perceptions of the assignment. The level of confidence of reading and analyzing primary literature, using tertiary resources and applying information to a specific patient, increased across all 3 stages. Although, only 51% of students completed the survey, most agreed that the 3-stage assignment helped them learn and practice research skills that can be applied in the future and the assignment should be continued for future classes. A follow-up study may have determined whether the students are able to apply the skills and knowledge learned in the drug information course after they enter the professional work environment.

A number of limitations to this study exist and warrant discussion. First, the survey questions were not validated before use, and there was a low response rate where only 46 (51%) of 90 students filled out the survey. Additionally, constructive feedback provided was also limited, which was a disadvantage to provide future improvements based on the students’ viewpoint. Thus, there is a potential for respondent bias since self-report surveys were used. Although the response rate is low, some literature supports there is no specific agreed upon standard for a response rate, and some literature states a response rate of greater than 50% is suitable.^19,20

Another limitation pertains to the use of surveys to assess the primary objective of the students’ ability to attain literature search and evaluation skills which is based on the students’ perceptions. It should also be noted that although faculty feedback is a necessary step in providing education to the students, this may have helped improve the grades and not necessarily improve their ability to retrieve, analyze, and interpret evidence on their own.

The generalizability of the results is limited as well, since the study was conducted at 1 college of pharmacy with a small sample size. The students who participated may not be representative of all our pharmacy students. We also noted limitations of having only 5 faculty to grade the assignment was time consuming, especially grading stage 2, which could take up to 30 minutes per student. However, faculty did agree the grading rubric created an efficient grading process, and we found the average grade range was similar between faculty. Other limitations we noted were the lack of application of all steps of the MSA. For example, the student was not responsible for securing information about the requestor and/or collecting the background information which is part of steps 1 and 2 of the MSA (Figure 1). It was already provided to them with the assigned question (ie, past medical history, allergies, etc). Additionally, students had no opportunity to ask further questions about the patient when collecting information. This approach limits student responsibility of gathering the necessary information, which would typically occur in a “real life” situation.

The use of pre- and postsurveys upon completion of the assignment demonstrated use of the MSA and PICO format together helped train pharmacy students to improve their literature interpretation, analyzation, and evaluation skills. Further studies are needed to determine the best practice for teaching and learning literature search and evaluation skills. This study may warrant additional colleges of pharmacy to examine their strategies to teach literature search and evaluation skills.

Conclusion

Teaching students the PICO format to help refine search strategies combined with MSA had several benefits. It improved student research skills, understanding and confidence of retrieving medical information, and ability to analyze and apply literature to a specific drug information question. The tools provided to students such as a step-by-step guide on how to write a drug information consult were intended to help create a systematic structure, which can be used during future practice experiences or throughout their pharmacy career. This method is not a replacement for a drug information course but may be used to optimize learning modalities for pharmacy students to gain practice in EBM. The method may also provide a foundation on how to research information needed to effectively answer clinical drug information questions. Exploration of applying all 7 MSA steps in the 3-stage approach may be helpful in the future to ensure students practice at the skill level necessary in answering drug information questions.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Sackett

Rosenberg

Gray

. Evidence-based medicine: what it is and what it isn’t. Br Med J. 1996;312(7032):71–72.

Moores

Kee

. Evidence-based clinical practice guidelines. In: Malone

Kier

Stanovich

Malone

, eds. Drug Information: A Guide For Pharmacists. 5th ed. New York, NY: McGraw Hill; 2014: 314–316.

2016 Accreditation Standards and Key Elements for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree. Chicago, IL: Accreditation Council for Pharmacy Education; 2015.

Accreditation Council for Pharmacy Education. Center for the Advancement of Pharmaceutical Education 2013. Web site. http://www.aacp.org/resources/education/cape/Open%20Access%20Documents/CAPEoutcomes2013.pdf. Accessed June, 2015.

Wright

LeCroy

Kendrach

. A review of the three types of biomedical literature and the systematic approach to answer a drug information request. J Pharm Pract. 1998;11(3):148–162.

Arif

Gim

Nogid

. Journal clubs during advanced pharmacy practice experiences to teach literature-evaluation skills. Am J Pharm Educ. 2012;76(5):88.

Cisner

Salisbury-Glennon

Anderson-Harper

. Status of problem-based learning research in pharmacy education: a call for future research. Am J Pharm Educ. 2002;66(1):19–26.

Neil

Johnson

. An advanced pharmacy practice experience in application of evidence-based policy. Am J Pharm Educ. 2012;76(7):133.

Elliott

Suda

Hamilton

. Impact of elimination of learner-centered assignments on student’s performance in and evaluation of a drug information and literature evaluation course. Curr Pharm Teach Learn. 2014;6(4):483–487.

10.

Wang

Troutman

Seo

. Drug information education in doctor of pharmacy programs. Am J Pharm Educ. 2006;70(3):51.

11.

Cole

Berensen

. Comparison of drug information practice curriculum components in US colleges of pharmacy. Am J Pharm Educ. 2005;69(2):240–244.

12.

Ghabi

Ipema

Gabay

. ASHP Guidelines on the pharmacist’s role in providing drug information. Am J Health Syst Pharm. 2015;72(7):573–577.

13.

Schardt

Adams

Owens

. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform DecisMak. 2007;7:16.

14.

Straus

Richardson

Glasziou

. Evidence-Based Medicine. How to Practice and Teach EBM. 3rd ed. New York, NY: Elsevier; 2005: 22–25.

15.

Watanabe

Conner

. Principles of Drug Information Services. Hamilton, IL: Drug Intelligence Publications; 1978.

16.

Villanueva

Burrows

Fennessy

. Improving question formulation for use in evidence appraisal in a tertiary care setting: a randomized controlled trial. BMC Med Inform DecisMak. 2001;1:4.

17.

Lavasa

Corman

Verrico

. Effect of drug information templates on pharmacy student compliance with the modified systematic approach to answering drug information questions. Ann Pharmcother. 2009;43(11):1795–1801.

18.

de Sousa

de Lima David

Noblat Lde

. A drug information center module to train pharmacy students in evidence-based practice. Am J Pharm Educ. 2013;77(4):80.

19.

Fowler

. Survey Research Methods 3rd ed. Thousand Oaks, CA: Sage; 2002.

20.

Babbie

. Survey Research Methods. Belmont, CA: Wadsworth; 1990.