Content Analysis of Five Occupational Therapy Journals,2006–2010: Further Review of Characteristics of the Quantitative Literature

Abstract

OBJECTIVE. To analyze the content of publications in 5 occupational therapy journals to determine the strengths and weaknesses of the literature base from 2006 to 2010.

METHOD. A content analysis for 2006 through 2010 of the American Journal of Occupational Therapy (AJOT), Australian Occupational Therapy Journal (AOTJ), British Journal of Occupational Therapy (BJOT), Canadian Journal of Occupational Therapy (CJOT), and Scandinavian Journal of Occupational Therapy (SJOT) was completed.

RESULTS. AJOT and SJOT had the highest percentage of articles focusing on physical disabilities, whereas a majority of articles in AOTJ, BJOT, and CJOT focused on education. SJOT published articles with the highest median number of participants in all research designs excluding descriptive studies. The majority of the research articles were descriptive for all journals.

CONCLUSION. From 2006 to 2010, AJOT provided stronger evidence conducted at higher levels than the other journals by publishing more articles investigating interventions used to support clinical practice.

Keywords

evaluation studies evidence-based practice occupational therapy periodicals as topic research

Evidence-based practice in the field of occupational therapy has increased in importance to support the efficacy of clinical practice, meet reimbursement standards, and better serve consumers. In her Eleanor Clarke Slagle Lecture, Margo Holm (2000) suggested that the quantity of available evidence has increased at a great rate, yet the quality of the publications may be questionable; she noted that to justify occupational therapy services, quality research is needed that supports the profession’s interventions and patient outcomes. Few studies have evaluated the quality, populations studied, and methodologies used in publications among multiple journals in the field of occupational therapy. An in-depth analysis, therefore, of the recent publications in journals is warranted. The purpose of our study was to build on existing analyses of occupational therapy journals with an in-depth investigation of the publication characteristics of the quantitative literature in 5 major occupational therapy journals to determine the strengths and weakness of the literature from 2006 to 2010.

Background and Literature Review

A content analysis is a systematic review of characteristics of information found in a common area (Neuendorf, 2002). It can be considered a scientific tool that “provides new insights, increases a researcher’s understanding of particular phenomena, or informs practical actions” (Krippendorff, 2004, p. 18). The emphasis to provide more evidence-based material in the profession of occupational therapy has increased. A content analysis of recent publications in prominent journals can reveal whether the journals are publishing material that is evidence based to support treatments in occupational therapy.

The quality of occupational therapy journals was previously assessed on the basis of ratings from authors of peer-reviewed occupational therapy articles. Rodger, McKenna, and Brown (2007) identified a survey of authors of peer-reviewed occupational therapy articles from 2003 to 2005 that identified top-quality indicators of a journal and then rated the journals with the same quality indicators. The top-quality indicators identified by individuals surveyed were reputation and prestige of the journal, availability of the journal, rigor and quality of the manuscript review process, timeliness of manuscript review and publication, and ability of the journal to affect policy and practice. The journals most highly rated based on these quality indicators were the American Journal of Occupational Therapy (AJOT), Australian Occupational Therapy Journal (AOTJ), British Journal of Occupational Therapy (BJOT), Canadian Journal of Occupational Therapy (CJOT), Occupational Therapy Journal of Research (OTJR), and the Scandinavian Journal of Occupational Therapy (SJOT). These journals have been subjected to analysis of content previously and identified as major occupational therapy journals.

Some previous studies have provided an analysis of single journals across a given time period. Brown and Brown (2005) analyzed a total of 452 articles published within the first 20 yr (1981–2000) of OTJR. Categories of type of research consisted of descriptive, survey, case–field study, correlation, quasi-experimental, true experimental, practice, and qualitative. Brown and Brown found that descriptive studies made up the largest portion of publications (23.7%) followed by correlation studies (20.4%) and quasi-experimental studies (19.7%). Research was the largest subject studied in OTJR, followed by practice, theory, pediatrics, and physical dysfunction.

Mountain (1997) analyzed BJOT publications from 1986 to 1996 and found that 343 of the 569 articles focused on topics concerning clinical work, whereas the remaining articles discussed nonclinical issues. Additionally, it was found that the majority of studies authored by British occupational therapists were questionnaires or survey-type research, whereas studies performed by occupational therapists in Australia, Canada, or the United States were more likely to be carried out in collaboration with professionals in medicine or psychology and to have more complex research designs.

Likewise, Cusick (1995) analyzed the publications in AOTJ from 1987 to 1991 and found only 42% of the publications could be classified as research based. More recently, Gutman (2011) examined AJOT from 2008 to 2011 and found that studies testing treatment effectiveness and rigor of experimental design had steadily increased; however, examination of practice efficacy was lacking.

Case-Smith and Powell (2008) conducted a content analysis of 5 occupational therapy journals from 2001 to 2005 that provided the basis for our study. They examined AJOT, AOTJ, BJOT, CJOT, and OTJR in their study. A total of 1,071 articles were coded by the type of article, type of research, practice area, and professional issue in nonexclusive categories (e.g., one article could be placed into two different categories). Case-Smith and Powell found that the total number of research articles published increased from 65% to 77% across the 5-yr span. OTJR published the most research articles, followed by AJOT with 99% and 82%, whereas AOTJ, BJOT, and CJOT research articles ranged from 57% to 60%. Additionally, they found 35% of the articles focused on physical disabilities, rehabilitation, and work followed by pediatrics (29%), geriatrics (14%), mental health (15%), and assistive technology (5%).

Additionally, Case-Smith and Powell (2008) examined the research designs of the articles published across all the journals. After combining categories (e.g., experimental with quasi-experimental and outcome studies, and case studies with single-subject design), they found that 23.7% were correlational in nature, survey designs accounted for 16.4%, 12.5% were experimental and outcome research, 11.4% were systematic reviews and meta-analyses, and 24.3% were qualitative.

Previous content reviews of occupational therapy journals provide valuable information about the research published in the journals but may lack a more in-depth analysis. This type of analysis can provide a better representation of the research published. For example, Blancher, Buboltz, and Soper (2010) provided an analysis of sampling techniques used in the Journal of Counseling and Development, which resulted in evidence against the assumption that most studies in the journal used convenience sampling techniques. Likewise, Falco, Bauman, Sumnicht, and Engelstad (2011) used mutually exclusive categories for methodology used and topics discussed in their analysis of the Professional School Counseling Journal that provided a better overall representation of the journals’ publications. Additionally, mixed-methods research is increasing in rehabilitation (MacDermid & Law, 2008). Therefore, mixed-method publications must be appraised for a complete analysis.

Although previous work has analyzed the prominent occupational therapy journals, an updated investigation is needed. To provide an in-depth analysis of the journals, we examined articles in terms of treatment efficacy, levels of evidence, and the most common sampling methods as well as methodologies used in quantitative and mixed-method articles. We used the following approach:

To determine which journal published the most research articles and the type of articles published, we asked, What is the percentage of research-based articles in each journal, and what is the percentage of quantitative mixed-method studies published in each of the 5 journals?

Analysis of the area of interests most published by each journal can determine whether each journal had a different focus. Therefore, we asked, What is the percentage of articles published concerning pediatrics, geriatrics, mental health, physical disabilities, community-oriented treatment, developmental disabilities, education, and sensory processing?

Portney and Watkins (2009, p. 21) suggested that research designs might be considered on a continuum from descriptive (describing populations) to exploratory (finding relationships) to experimental (cause and effect). We categorized research designs and asked, What types of research designs were most frequently published?

More recently, evidence-based practice guidelines have been proposed to classify the strength of evidence for particular treatment approaches (MacDermid & Law, 2008). We reviewed the research in each of the journals that tested treatment efficacy in an effort to determine what levels of evidence were published in the literature over the 5-yr span.

Ottenbacher (1982) demonstrated that sample size in many studies in occupational therapy had insufficient numbers of participants to provide adequate power and avoid Type 2 error. Ottenbacher (1984) and Portney and Watkins (2009, p. 423) stated that sample size often does not provide the power necessary to find statistical significance in occupational and physical therapy research. Although sample size is not the only variable that affects power, we examined the number of participants used in each journal for quantitative studies to determine whether sample size might be an issue in the literature and because different experimental designs would require larger or smaller sample sizes.

Method

Investigators

The investigators were four graduate-level occupational therapy students who completed an evidence-based practice course within their curriculum. They completed an additional 80-hr content analysis training course led by author McPherson.

Publications Reviewed

In the content analysis, we reviewed all articles published in the journals (AJOT, AOTJ, BJOT, CJOT, SJOT) from 2006 through 2010. We initially classified articles as research based or non–research based. Research-based articles were included and analyzed completely by the investigators (the first four authors). Nonresearch articles (e.g., editorials, letters to the editor, obituaries, book review, etc.) were excluded. Qualitative findings will be disseminated in a future publication.

Operational Definitions

Operational definitions were created for each category with the caveat that they would provide mutually exclusive categorization of each article for each question asked. The operational definitions used in these guidelines were originally based on definitions provided by Portney and Watkins (2009) and previous research in this area (Case-Smith & Powell, 2008). Classification for levels of evidence was that used by the American Occupational Therapy Association (AOTA; 2011) Evidence-Based Practice Project.

Starting definitions were developed for each category of the journal articles that would be reviewed and were tested by the investigators. The starting definitions were used to rate articles and establish agreement. Definitions were reexamined and redefined when it was evident among investigators that they were unclear. Throughout this process, definitions were given specific meanings, including age ranges and conditions, to make each category mutually exclusive. Refer to Table 1 for all operational definitions.

Table 1.

Operational Definitions of Journal Article Categories

Terms	Definitions
Type of research
Research article	A written report of an experiment conducted to obtain information.
Quantitative research	A study that uses measurable data to form descriptions and identify variables or quantify behaviors, attitudes, or other measurable variables. Studies might be conducted through interviews or surveys. They may use highly structured questions, are usually performed on a large samples, and use statistical analysis to answer questions.
Qualitative research	Research that seeks to explain the complex nature of experiences on the basis of how people perceive those experiences; would not contain data subjected to statistical analysis.
Mixed method	A study that incorporates both quantitative and qualitative methods.
Focus of article
Pediatrics	Articles dealing with the treatment, description, or testing of children ranging in age from birth to 18 yr. Articles may focus on neonatal care, infant care, and treatment for groups of children with various disorders in clinics or a school setting.
Geriatrics	Articles dealing with adults who are older than age 65 who may have physical disabilities including rheumatoid arthritis, joint replacements, and orthopedic conditions; the focus of the study is age, not disability.
Physical disabilities	Articles dealing with adults age 18 to 65 who have disabilities caused by orthopedic or neurological problems such as stroke, fractures, Parkinson’s disease, or rheumatoid arthritis. The focus of the article is disability, not age.
Mental health	Articles dealing with people who have disorders that affect their lives in one or more of the following areas: cognitive functioning (skills and beliefs), communication, emotional regulation, motivation, or coping. The diagnosis is based on evidence from history, physical examination, family history, or psychosocial disturbances that hinder occupational performance. These disorders include dementia and Alzheimer’s disease.
Community	Articles dealing with treatment, descriptions, or testing of people in a specific community who may need adaptations or testing to maintain their current level of functioning. Articles may focus on driver training, health promotion programs, home modifications, smoking cessation, or obesity prevention.
Developmental disabilities	Articles dealing with adults who suffer from deficits originating at birth such as Rett syndrome, Fragile X syndrome, Down syndrome, or mental retardation.
Education	Articles dealing with the process of educating occupational therapy practitioners or students in occupational therapy practice; may include education on expectations of occupational therapy or occupational therapy assistant student supervisors, coursework, or requirements. Also may include research on how to educate clients or families about disabilities.
Sensory processing disorders	Articles about disorders that cause difficulty with processing or responding to sensory information, including visual, auditory, gustatory, tactile, olfactory, vestibular, and proprioceptive stimuli. Articles may focus on children who have problems with paying attention.
Research design
Meta-analysis	A statistical technique that reviews and summarizes the results of previous quantitative research of a number of studies to determine an overall outcome.
True experimental	A study that includes an experimental group and a control group, at least one common outcome measure, and random assignment to groups.
Quasi-experimental	A study that may include group comparisons but has no indication of random group assignment or that uses a nonequivalent or no control group. Both independent and dependent variables are identified, and there is an attempt to manipulate variables involved. Statistical analysis of the results is included.
Single-subject design	Also called single-case design. Typically uses no more than 10 subjects with repeated data collection to monitor changes over time. Data are compared at time intervals, including baseline (before intervention) and after intervention. Analysis usually consists of graphs illustrating change. May use time-series, multiple-baseline, or multiple-probe techniques.
Multiple regression	A statistical technique using multiple correlations that predicts the strength of a relationship between dependent and independent variables (uses multiple variables).
Instrument development	Research designed to develop or refine the procedures used for measures or instruments used to measure variables. Articles may focus on establishing reliability or validity or may use Rasch analysis.
Descriptive design	Studies about groups that describe characteristics of samples representing populations; may use summaries (frequencies) or correlations establishing relationships between characteristics of groups.
Case study	Studies using narrative descriptions of conditions of one or more subjects and interventions they received. It may describe outcomes.
Systematic review	A systematic review of the literature that clearly answers a question through methods that focus on relevant research.
Levels of evidence
Level I	Systematic reviews, meta-analyses, randomized controlled trials
Level II	Two-group, nonrandomized studies
Level III	One-group, nonrandomized studies
Level IV	Descriptive studies that include analysis of outcomes (e.g., single-subject design, case series)
Level V	Case reports and expert opinion that include narrative literature reviews and consensus statements

Note. All operational definitions except levels of evidence and focus of article were based on descriptions provided in Portney and Watkins (2009). Operational definitions used to identify levels of evidence were based on descriptions provided in American Occupational Therapy Association (2011).

Once investigators agreed on clarity of all operational definitions, intrarater reliability among the four raters was established. The absolute percentage of agreement among all four raters on 1 quantitative article and 1 qualitative article chosen at random was 94% and 93%, respectively. During data collection, the investigators met bimonthly to collaborate on categorizing any article when the appropriate classification was unclear. They would discuss these articles and come to consensus on the correct classification.

For every quantitative article, the focus of the article and number of participants were recorded. By identifying the focus, the areas of practice investigated most often could be identified. The identification of sampling techniques and number of participants were investigated because these variables could affect generalizability of the data (Portney & Watkins, 2009). Additionally, the research design was categorized. If the article tested treatment efficacy, then a level of evidence was assigned.

Data were analyzed descriptively using Microsoft Excel 2010 (Microsoft Corporation, Redmond, WA). For each article analyzed, data were collected for each item investigated and coded prior to input into a Microsoft Excel spreadsheet. The median rather than the average was used in most of our analyses so that outliers would not skew the findings. All results are reported in percentages that were rounded to the nearest whole number.

Results

Research-Based Articles and Percentage of Quantitative and Mixed-Method Studies

We included 1,344 total quantitative, qualitative, and mixed-method articles for analysis from the 5 journals from 2006 to 2010. SJOT had the highest percentage of research-based articles (82%), and AJOT had the second highest percentage (58%) over the 5-yr span. However, SJOT published fewer total articles (152) than AJOT (534) from 2006 to 2010. SJOT and AJOT were followed by BJOT, AOTJ, and CJOT with 51%, 41%, and 40% research-based articles, respectively. Of the research-based articles, AJOT published the highest percentage of quantitative research (80%) followed by SJOT (60%) and AOTJ (58%). Mixed methods were used least by each journal, ranging from 14% (BJOT) to 5% (AJOT, SJOT).

Article Focus

Table 2 illustrates an overview of the characteristics of the 5 journals from 2006 to 2010. The areas of interest focused on the most by the 5 journals were physical disabilities and education. Most articles published in AJOT were related to physical disabilities (25%), followed closely by pediatrics (21%). AOTJ predominantly focused on education (38%); however, pediatrics (17%) and geriatrics (15%) were relatively similar in this journal. BJOT and CJOT mainly published studies focusing on education (40% and 34%, respectively). Thirty-three percent of SJOT’s publications focused on physical disabilities, which was the more than any of the other journals.

Table 2.

Overview of the Characteristics of the Five Journals Analyzed

Type of Analysis	Journal
Type of Analysis	AJOT	AOTJ	BJOT	CJOT	SJOT
Article focus, %
Pediatrics	21	17	10	17	6
Geriatrics	7	15	5	8	15
Mental health	8	12	16	14	16
Physical disabilities	25	10	12	17	33
Community	8	8	15	7	9
Developmental disabilities	0	0	0	0	<1
Education	16	38	40	34	17
Sensory processing	14	<1	1	3	1.6
Participants, n
25th percentile	11.5	17.5	0	1	32.75
50th percentile	38	35	40	31	72
75th percentile	79	76	76	106	159.75
Type of research, %
Meta-analysis	1	1	0	0	0
Systematic review	7	9	14	19	8
True experimental	5	1	3	0	6
Quasi-experimental	19	7	8	10	13
Multiple regression	3	6	3	3	6
Instrument development	19	20	8	16	25
Descriptive	40	51	56	39	36
Single-subject design	6	1	6	6	3
Case study	7	4	2	6	1
Level of evidence, %
I	44	13	21	10	11
II	63	6	6	6	19
III	75	0	8	4	13
IV	49	13	22	11	5
V	73	18	0	9	0
Participants, by design, n
True experimental	29	32	30	0	41
Quasi-experimental	40	52	68	60	100
Multiple regression	53	49	49	83	157
Instrument development	64	26	25	60	72
Descriptive	59	56	49	91	79
Single-subject design	2	6	9	4	8
Case study	1	2	12	2	3

Note. AJOT = American Journal of Occupational Therapy; AOTJ = Australian Occupational Therapy Journal; BJOT = British Journal of Occupational Therapy; CJOT = Canadian Journal of Occupational Therapy; SJOT = Scandinavian Journal of Occupational Therapy.

Median Number of Research Participants by Journal

When distribution is not normal or data are skewed, the median is the best estimate of the measure of central tendency (Portney & Watkins, 2009, p. 391). The number of participants ranged from 1 to 74,000. 74,000 participants would skew the data; therefore, we chose to use the median. The median number of participants used in quantitative articles was similar across all journals except for SJOT, which had the highest with 72 participants. BJOT and AOTJ had drastically fewer participants in quantitative articles, with only 1 participant at the 25th percentile (i.e., 25% of articles had only 1 subject).^* The small number of participants limits the generalizability of the findings. SJOT, however, had a much higher number of participants at the 25th percentile than the other journals. In addition, SJOT had the highest number of participants in quantitative studies across all percentile ranges. This result may be interpreted to mean that results of studies in SJOT may be more generalizable and findings could be more statistically sound (Ottenbacher, 1982). At the 25th percentile for mixed methods, the number of participants ranged from 40 (CJOT) to 1 (AJOT). At the 75th percentile, all journals used high numbers of participants with SJOT (160) and CJOT (106) at the highest followed by AJOT (79), BJOT (76), and AOTJ (76).

Quantitative Research Designs

In all journals, descriptive research accounted for the highest percentage of research designs. Forty percent of the publications in AJOT used a descriptive design, followed by quasi-experimental and instrument development studies, which together accounted for 38% of the articles. AJOT published the highest percentage of quasi-experimental research. True experiments (5%), studies using multiple regression (3%), and meta-analyses (1%) were the least prevalent research designs published in AJOT from 2006 to 2010.

In AOTJ, articles of a descriptive nature accounted for a total of 51% of research designs, succeeded by instrument development research (20%) and systematic reviews at (9%). Meta-analysis (1%), true experiments (1%), and single-subject design (1%) research was less abundant.

BJOT published the highest percentage of descriptive research (56%). The high prevalence of descriptive articles was followed by systematic reviews (14%) and instrument development research and quasi-experimental research (both at 8%). True experiments (3%), multiple regression (3%), case study (2%), and meta-analytic (0%) research designs were the least prevalent.

CJOT had the highest percentage of systematic reviews (19%) over the 5-yr span. In addition, descriptive (39%) and instrument development research (16%) were among the most prevalent research designs published in the journal. CJOT published no meta-analyses and was the only journal to lack any studies of a true experimental design.

Descriptive studies accounted for 36% of the research published in SJOT, followed by instrument development studies—at 25%, this was the highest percentage of articles devoted to instrument development research among the 5 journals. SJOT also had the second highest percentage (12%) of quasi-experimental studies. The least prevalent research designs in SJOT were true experiments and multiple regression (both at 6%), single-subject design (3%), and case studies (1%). There were no meta-analyses. Compared with the other journals, SJOT had the highest percentage of true experimental studies despite the design’s low prevalence in the journal.

Treatment Efficacy and Levels of Evidence

Figure 1 presents the percentage of articles in each journal that tested treatment efficacy during this time period. We assigned a level of evidence to each article. A total of 150 articles tested treatment efficacy, resulting in 11% of the total quantitative and mixed-method articles. All the articles were combined before percentages were calculated to show a true representation of the number of articles demonstrating treatment efficacy in each journal. AJOT and SJOT accounted for the highest percentage of articles testing treatment efficacy, indicating that these journals provided a larger amount of evidence to support interventions from 2006 to 2010.

Figure 1.

Percentage of articles testing treatment efficacy, by journal, over a 5-yr span.

Because AJOT had the highest proportion of articles testing treatment efficacy, it had the highest percentage of articles that were assigned a Level I of evidence (34%). Other than AJOT, BJOT accounted for the second highest percentage of Level I evidence at 21%, whereas SJOT had the second highest percentage of Level II evidence at 19%. CJOT accounted for the lowest percentage of Level I, II, and III evidence. AJOT’s high percentage of articles testing treatment efficacy suggests that the journal is more focused on developing strong evidence for clinical practice.

Median Number of Participants by Research Design

The median number of participants in each published study was also calculated by research design in each journal. Systematic reviews and meta-analyses were omitted from this analysis. Multiple regression designs had the highest combined median number of participants (391) reported for all journals. Within the multiple regression design, SJOT had the highest number of participants (157) and AOTJ and BJOT each had the least (49). Descriptive designed studies reported the second highest combined median number of participants (334) with CJOT reporting the most (91) and BJOT again reporting the least (49). Quasi-experimental studies had the third highest combined median number of participants (320) of which SJOT reported the most (100) and AJOT reported the least (40). Instrument development studies had the fourth highest combined median number of participants (247) with SJOT reporting the highest (72) and BJOT reporting the lowest (25). True experimental studies had the fifth combined highest median number of participants (132). Of the true experimental studies, the median number of participants ranged from 29 (AJOT) to 41(SJOT). Single-subject design represents the next-to-lowest combined median number of participants (29). Of the 29 participants, BJOT had the most (9), and AJOT had the least (2). Case study represented with the least-combined median number of participants for all journals (20). The median number of participants in the case study design ranged from 1 (AJOT) to 12 (BJOT).

Discussion

Holm (2000) acknowledged that occupational therapy practitioners could not provide patients with a percentage of the effectiveness of specific interventions in relation to particular conditions. Holm also questioned whether this goal would be attainable by 2010. The current study examined the number of articles testing treatment efficacy and their levels of evidence and research methodologies in 5 prominent occupational therapy journals from 2006 to 2010. Our content analysis shows these journals published higher quantities of descriptive studies with a limited number of articles testing treatment efficacy (11%) over this 5-yr span.

This analysis revealed that SJOT published the highest proportion of research-based articles of the 5 journals analyzed. AJOT, however, published a greater number of research-based articles. Of the research-based articles, SJOT, AJOT, and AOTJ had higher proportions of their journals devoted to quantitative research. Conversely, BJOT and CJOT published higher proportions of mixed-method articles.

The area of analysis with the most clinical implications may be the investigation of the number of articles testing treatment efficacy and the levels of evidence of these articles. Gutman (2011) noted an increase in articles testing treatment efficacy within AJOT from 2008 to 2011 (our study did not include articles from 2011). Our data support this notion: AJOT had the highest percentage of articles testing treatment efficacy (34%) from 2006 to 2010. SJOT and AJOT combined to account for 54% of all articles testing interventions.

Additionally, AJOT had more than 40% of all Level I evidence and more than 60% of all Level II evidence published in the journals over the 5-yr span. Not only did AJOT produce a higher percentage of treatment efficacy articles, it also produced stronger evidence than the other 4 journals. Case-Smith and Powell (2008) found that 12.5% of the articles from 2001 to 2005 from all 5 journals they analyzed were outcome studies. Although our study did not include analysis of OTJR and instead analyzed SJOT, we found that 11% of all research-based articles tested treatment efficacy from 2006 to 2010. A direct comparison for trends cannot be made because of the differences in analysis, but this finding suggests treatment efficacy research may have been consistent over the decade.

Research designs were analyzed by the percentage of articles using each design published in each journal. It was found that the majority of research designs were descriptive in nature. These designs included correlational studies, surveys, and other articles that described the characteristics of individuals and interventions. In particular, more than 50% of AOTJ’s and BJOT’s publications were descriptive, whereas AJOT, CJOT, and SJOT were slightly under 40%. This finding is paramount for the profession because these studies can provide information regarding the association of variables and descriptions of conditions, but they cannot specifically test efficacy (Portney & Watkins, 2009).

Other than descriptive studies, instrument development studies were published at a high rate as well. These types of studies are concerned with the development or refinement of tools and assessments (Portney & Watkins, 2009).

AOTA and the American Occupational Therapy Foundation (AOTF) (2011) set forth a research agenda that outlines the areas of need for research. Assessment and measurement studies were at the top of this list, followed by intervention studies. Perhaps the high number of instrument development studies with fewer intervention studies could be because of a lack of outcome measures available for occupational therapists to use to determine the effectiveness of interventions. If this notion is true, our study suggests the profession is moving forward in creating new outcome measures that can be eventually used to establish a strong effective research base. Although percentages of true experimental studies were low for all journals, AJOT had almost twice the percentage of quasi-experimental publications as the other journals. In general, systematic reviews made up a larger portion of publications than true experimental single-subject design and multiple regression studies.

The size of the sample used in a study is a critical aspect of the accuracy of the findings. Smaller samples sizes do not provide a good representation of the target population, and statistical power is drastically reduced. We investigated the sample sizes by finding the median number of participants as well as the number of participants at the 25th and 75th percentiles. SJOT had the highest median number of participants with 72. Additionally, all journals had a high number of participants at the 75th percentile, ranging from 76 to 160. At the 25th percentile, however, all journals except SJOT used fewer than 18 participants, suggesting that 25% of the quantitative articles published in AJOT, AOTJ, BJOT, and CJOT are using very small samples sizes, which increases the risk of Type II error, resulting in unstable parametric testing and decreasing the generalizability of results (Portney & Watkins, 2009). SJOT used considerably higher numbers of participants in all studies, showing that the results of the studies published in this journal could be more generalizable.

Analysis of focus areas revealed that SJOT and AJOT focused more on physical disabilities than any other area. BJOT, CJOT, and AOTJ published a higher percentage of education articles than any other area. This finding is important because these articles primarily consist of education of occupational therapy students, curriculum, and expectations of clinical instructors and client education about disabilities. Therefore, a higher percentage of articles dealing with education of occupational therapy students would suggest fewer articles on studies testing the efficacy of treatments used by occupational therapy practitioners.

Limitations

This study was limited by not including an analysis of the trends of the research published in each journal by year. With this type of analysis, statistical procedures could have been performed to reveal any significant changes from year to year. We also did not investigate many other potential indicators of research rigor, such as statistical validity. Space limitations prevented a presentation of data from qualitative studies. This review does not reflect what is occurring today: We did not analyze publications for 2011, 2012, and 2013 or what has occurred more recently in these journals.

Implications for Occupational Therapy Practice, Education, and Research

Our study analyzed the literature base in five prominent occupational therapy journals over a 5-yr span (2006–2010). We found that 11% of all quantitative and mixed-method articles tested treatment efficacy in these journals over the 5-yr period. AJOT accounted for the highest percentage of articles testing treatment efficacy with the highest levels of evidence. However, the majority of the research articles used descriptive designs for all journals.

We suggest a continued emphasis on evidence-based practice within our profession and our educational curriculum.

A study of this nature could be conducted over each 5-yr span to assess our profession’s progress toward building a strong evidence base to reach Holm’s (2000) goal.

Conclusion

AOTA and AOTF (2011) published a research agenda emphasizing the need for studies that test treatment effectiveness to support the clinical practice of occupational therapy. Additionally, external forces such as reimbursement standards and patient expectations of treatment have caused a push for more publications to justify our services (Holm, 2000). This study of the quantitative literature, however, revealed that the majority of research published within 5 prominent occupational therapy journals from 2006 to 2010 was descriptive; only a limited amount of research tested treatment efficacy. Additionally, analysis of sample sizes revealed potential for Type II errors. Holm’s (2000) goal has not been realized.

The research base of the profession may be improving, however. For example, articles on instrument development were published at a high rate, indicating that the profession is developing and redefining assessments that can be used in clinical practice and outcome studies. Additionally, AJOT and SJOT published a large quantity of strong quantitative evidence to support occupational therapy interventions.

Footnotes

Acknowledgment

We acknowledge Ann Marie Short, associate director of the Shawnee State University library, for her hard work and dedication to ensure that we had full access to all journals analyzed in this study.

The interquartile range means that at the 75th percentile, 25% of the articles reported on research using Ns above a given number, and at the 25th percentile, 25% of the articles reported on research using Ns below the number reported.

References

American Occupational Therapy Association. (2011). Guidelines for evidence tables. Retrieved February 2011 from http://aota.org/DocumentVault/AJOT/Evidence-Guide.aspx

American Occupational Therapy Association, & American Occupational Therapy Foundation. (2011). Occupational therapy research agenda. American Journal of Occupational Therapy, 65, S4–S7. http://dx.doi.org/10.5014/ajot.2011.65S4

Blancher

A. T.

Buboltz

W. C.

Jr. , & Soper

(2010). Content analysis of the Journal of Counseling and Development: Volumes 74 to 84. Journal of Counseling and Development, 88, 139–145. http://dx.doi.org/10.1002/j.1556-6678.2010.tb00002.x

Brown

G. T.

, & Brown

(2005). Characteristics of the Occupational Therapy Journal of Research: The first twenty years. Occupational Therapy in Health Care, 19, 73–92. http://dx.doi.org/10.1080/J003v19n03_06

Case-Smith

, & Powell

C. A.

(2008). Research literature in occupational therapy, 2001–2005. American Journal of Occupational Therapy, 62, 480–486. http://dx.doi.org/10.5014/ajot.62.4.480

Cusick

(1995). Australian occupational therapy research: A review of publications 1987–91. Australian Occupational Therapy Journal, 42, 67–75. http://dx.doi.org/10.1111/j.1440-1630.1995.tb01314.x

Falco

L. D.

Bauman

Sumnicht

, & Engelstad

(2011). Content analysis of the Professional School Counseling journal: The first ten years. Professional School Counseling, 14, 271–277. http://dx.doi.org/10.5330/PSC.n.2011-14.271

Gutman

S. A.

(2011). From the Desk of the Editor—State of the journal, 2011. American Journal of Occupational Therapy, 65, 608–615. http://dx.doi.org/10.5014/ajot.2011.003145

Holm

(2000). Our mandate for the new millennium: Evidence-based practice, 2000 (Eleanor Clarke Slagle Lecture). American Journal of Occupational Therapy, 54, 575–585. http://dx.doi.org/10.5014/ajot.54.6.575

10.

Krippendorff

(2004). Content analysis: An introduction to its methodology (2nd ed.).Thousand Oaks, CA: Sage.

11.

MacDermid

, & Law

(2008). Evaluating the evidence. In Law

, & MacDermid

(Eds.), Evidence-based rehabilitation: A guide to practice (2nd ed., pp. 121–142). Thorofare, NJ: Slack.

12.

Mountain

G. A.

(1997). A review of the literature in the British Journal of Occupational Therapy, 1989–1996. British Journal of Occupational Therapy, 60, 430–435.

13.

Neuendorf

K. A.

(2002). The content analysis guide book. Thousand Oaks, CA: Sage.

14.

Ottenbacher

(1982). Statistical power and research in occupational therapy. OTJR: Occupation, Participation and Health, 2, 13–25.

15.

Ottenbacher

(1984). The significance of power and the power of significance: Recommendations for occupational therapy research. OTJR: Occupation, Participation and Health, 4, 37–50.

16.

Portney

L. G.

, & Watkins

M. P.

(2009). Foundations of clinical research: Applications to practice (3rd ed.). Upper Saddle River, NJ: Pearson Education.

17.

Rodger

McKenna

, & Brown

(2007). Quality and impact of occupational therapy journals: Authors’ perspectives. Australian Occupational Therapy Journal, 54, 174–184. http://dx.doi.org/10.1111/j.1440-1630.2007.00595.x