Pilot Study of the Effectiveness of Weighted Vests

Design

We chose an ABA design (Ottenbacher, 1986) to examine the effect of wearing a weighted vest on the attention to task of second-grade students with attention problems. We measured the effect of the intervention by comparing the participant’s on-task behavior in the no-intervention condition (A phase) with on-task behavior in the intervention condition (B phase). We also measured the effect by comparing the change in on-task behavior between the intervention group and the control group.

Participant Recruitment

The participants were students in general education classrooms at 1 of 9 elementary schools in a racially and socioeconomically diverse public school district in the greater Houston, Texas, area. None of the participants had an individualized educational program. Classroom teachers were surveyed and asked to identify students with at least three of the following four characteristics: (1) has more difficulty staying in own seat than peers; (2) has more difficulty than peers keeping eyes on teacher, board, or own work; (3) needs more frequent reminders to work on task than peers; and (4) more frequently asks irrelevant questions or talks off topic than peers. We obtained informed consent from 25 participants, who were randomly assigned to either the intervention group (n = 13) or the control group (n = 12). Because of time constraints, the primary investigator (PI; Amy Collins) was unable to review baseline (no-intervention period) videotapes for each participant and confirm eligibility for inclusion in the study until after data collection was complete. The PI found that only 11 of 25 participants met the study’s inclusion criteria by consistently exhibiting three of the four required characteristics; the other 14 participants were observed to be on task nearly 100% of the time. Data from those 14 participants were excluded from the study because inclusion would have created a potential ceiling effect. Thus, the two groups were not balanced; a total of 7 children were in the intervention group, and 4 children were in the control group.

Weighted Vests Used in the Study

To ensure consistency, all participants wore the OTvest™ (OTvest, LLC, Portage, MI). We chose this vest because it was commercially available and looks like typical children’s denim clothing. The vest comes in several sizes, each with a set amount of weight. The vest includes a removable insert, constructed with weighted disks quilted into place, that distributes the weight over the participant’s shoulders, upper chest, and upper back. For the nonweighted vests, we removed this insert and in its place used a nonweighted insert constructed with Styrofoam discs of similar shape and size. Vests appeared identical to the observer. During the intervention phase, the data collectors assisted the participants with donning the vests before beginning the seatwork activity and at least 5 min. before beginning the video recording to allow the participant to acclimate to the vest. Because the research assistants (data collectors) handled both weighted and nonweighted vests and needed to know which participant was to receive which vest, it was not feasible for them to be blinded to the study conditions.

Setting

Participants were recorded during seatwork time in their classroom. The video cameras were set on tripods approximately 5–6 ft away. Seatwork included color, cut, and paste activities; reading assignments; and working in workbooks or on worksheets. The activities were part of the classroom’s typical activities and occurred during normal daily routines.

Procedure

The PI or research assistant visited each classroom before the beginning of the videotaping. A brief explanation of the project was provided to the students. The teacher and class were told that the student or students who were chosen would be wearing a vest to see whether it helped them work better. The vest was shown to the class, but students and teachers were not given an opportunity to handle it. Then the participants’ names were drawn from a bag, making it appear to the students that selection was random.

Teachers and parents were blind to whether the participants wore a weighted or nonweighted vest. Although students could feel whether their vest was heavy or light, they were not aware that two different kinds of vests existed. As stated previously, the research assistants who videotaped the participants needed to be aware of the two study conditions and of the condition to which the participant was assigned, but they were not involved in the later data coding.

Videotaping sessions were scheduled in advance for a time when seatwork activity would be occurring. The PI provided training to five research assistants. Content included how to collect the data by videotaping, when to begin and end the taping, and what research assistants could and could not say to the participants or others in the classroom about the vests or the study. They were trained to present the vests to all participants in the same manner and to give the same instructions to all participants without regard to the group to which they were assigned. Videotaping began once the class had fully transitioned to the new task (e.g., after time to get out appropriate materials, clarify instructions). After the completion of the videotaping, participants were allowed to continue wearing the vest for the duration of the activity.

The participants were videotaped over nine sessions in 10-min increments. Three sessions took place during the baseline phase, three occurred during the intervention phase, and three took place during the intervention withdrawal phase. Each participant was recorded for a total of 90 min. Each video session occurred on a separate day. The time span for the collection of the nine sessions ranged from 3 to 6 wk. Data collection for most of the participants was completed within 3 wk. One data collector left the project during data collection, resulting in an extension of the data collection period for 3 participants.

The PI instructed teachers to keep the existing routine and expectations consistent throughout the videotaping sessions and to inform the PI if any changes occurred. Teachers were also instructed to provide the same type and amount of prompting to the participants for attending to task as they normally would. At the end of the videotaping, teachers were asked to complete a nonstandardized follow-up survey developed specifically to assess their fidelity to these expectations and to obtain anecdotal information about whether the teacher thought the vest was effective. The instrument was necessarily brief to minimize teacher burden and to enhance the district administration’s cooperation.

Although control group participants are typically given the opportunity to receive the intervention at a later time so as not to withhold some expected benefit, participants in the control group were not given the opportunity to wear the weighted vest because a few brief sessions of wearing the vest would provide no substantial advantage. Instead, all study participants were given an electronic educational toy valued at approximately $100 as compensation for completing the study.

Data Coding

Videos were coded for on-task behavior by six occupational therapists who participated in a group training session. These therapists had not been involved in the videotaping and data collection and thus were blind to the participants’ study condition. During the training, the PI explained the conceptual definition of on-task behavior and showed the group samples of students performing seatwork activities. These students were not the study participants. The PI facilitated a discussion of concrete behaviors that constituted on-task behavior until the group came to a consensus. Thus, the people responsible for applying the conceptual definition to concrete actions helped develop a realistic operationalization of the concept.

The group defined on-task behavior as the students’ having writing supplies and work materials on the desk in front of them; keeping their eyes on the teacher, the board, or their own work; listening to or working on the assignment; and asking relevant questions of the teacher or a neighbor, as appropriate. Any other behaviors were coded as off task. The group discussed and agreed that the data coders would refrain from making judgments about whether any behaviors were good or bad because those judgments might interfere with their ability to code objectively. For example, a student who dropped his or her pencil on the floor and reached down to pick it up would be coded as off task because this behavior did not fall into the operationalized definition of on-task behavior. The coder was not to try to determine whether the student dropped the pencil to avoid working or whether he or she took excessive time to retrieve it from the floor because doing so would call for a subjective judgment on the part of the coder and would likely reduce interobserver reliability. By the end of the training session, interobserver reliability for the group was calculated at .89 using the intraclass correlation (Bartko & Carpenter, 1976).

The six coders were paired, and the three pairs divided the videos to be coded. Each pair was expected to discuss each data point and come to an agreement. Because intraobserver reliability was so high during training, we determined that coding pairs should reach consensus rather than code independently for an additional reliability determination. Data were coded using a momentary interval method, in which on-task behavior was coded as yes, no, or unknown at 15-s intervals. The code unknown was used only when it was not possible to determine whether the participant was on task or off task (e.g., when the student was out of view of the camera, no task was occurring in the classroom, or the rater pair could not agree). A maximum of 40 data points were coded per video segment.

Teacher Follow-Up Survey

Teachers completed follow-up surveys, as requested, on each of the 10 participants. Surveys indicated 100% teacher fidelity to the instructions given. Teachers reported that they observed a change in the behavior of 4 of the 7 intervention group participants. One teacher commented that a participant sat in his seat more and worked on task more. Another commented that a participant improved, but her impression was that the participant improved while the camera was on him, regardless of whether he was wearing the vest. The teacher of another intervention group participant reported that the participant improved but also commented that the participant began taking medication to address attention at some point during the study. We could not obtain any additional information.

Quantitative Results

As a first step, we ascertained equivalence at baseline through t tests. At baseline, the mean (M) percentage of time on task was 66.7% for the intervention group (standard deviation [SD] = 0.131) and 66.9% for the control group (SD = 0.077). We found no significant difference between the two groups at baseline (t[8] = −0.038, p = .971). The Ms and SDs of the two groups at baseline, intervention, and withdrawal are summarized in Table 1.

After confirmation of initial group equivalence, we submitted the data to a repeated-measures analysis of variance to determine whether the within-group participants showed a change across the three testing phases; a significant interaction effect would suggest that the intervention had an effect. The test of within-group participants effects was nonsignificant (F = 0.287, df = 2, 16, p = .753). Moreover, we found a nonsignificant interaction effect (F = 0.063, df = 2, 16, p = .939). The power of the tests was extremely low (<.10), primarily because of the small sample size. However, the data given in Table 1 suggest that no statistically confirmable intervention effect was present.

Qualitative Results

Because of the small sample size and consequent low power, we turned our analysis to more qualitative methods. Figure 1 depicts the hypothesized pattern expected of an effective intervention using a weighted vest. The percentage of time on task at baseline would be relatively low. During the intervention period, the percentage of participants on task would rise sharply and then drop on withdrawal.

OPEN IN VIEWER

Figure 1.

Hypothesized treatment effect of a weighted vest intervention.

Figures 2 and 3 graph the data of each participant over the three phases. Figure 2 depicts the percentage of time on task for the intervention group participants, and Figure 3 depicts the percentage of time on task for the control group participants. As can be seen in Figure 2, only 2 intervention participants (28.6%; Participants 3 and 4) conformed to the hypothesized pattern. Moreover, as seen in Figure 3, Participants 2 and 6 (marginally) conformed to the hypothesized pattern.

OPEN IN VIEWER

Figure 2.

Treatment effect of a weighted vest intervention for intervention group participants.

OPEN IN VIEWER

Figure 3.

Effect of nonweighted vest use for control group participants.

Six of 10 participants, including intervention and control group participants, exhibited a pattern in which the percentage of time on task showed a distinct reduction, followed by a return almost to baseline. One intervention participant (Participant 9) deviated from the distraction pattern by a continued reduction of percentage of time on task even at withdrawal.

The effectiveness of the vest may be conditional. If so, the 2 intervention participants conforming to the hypothesized model (an increase in attention to task during the intervention period followed by a return to baseline levels at withdrawal) should be different from all the other participants. We examined additional background data in an attempt to differentiate the intervention group participants conforming to the hypothesized model from those who did not. Of the 2 intervention group participants who conformed to the hypothesized model, 1 was female and the other was male. The female participant had begun medication during the study, and the male participant had no record of medication. Of the intervention group participants who did not conform to the hypothesized model, 1 was reported to be receiving private counseling, and the others were not. Gender and other intervention modalities were thus eliminated as possible conditional variables. Similarly, school campus, teacher, and ethnicity did not differentiate the conforming intervention participants from the others.

Implications for Practitioners

Given that the effectiveness of weighted vests to improve attention to task is inconclusive, we recommend that weighted vests be used in conjunction with a carefully designed and implemented process of data gathering and analysis. The lack of an established treatment protocol strengthens the need to use the intervention with close clinical observation and systematic data collection. If data for an individual client do not show improvement initially but the client enjoys wearing the vest, the therapist may use his or her clinical reasoning skills to adapt the wearing time or weight of the vest while continuing to collect data. If the client continues to show no improvement in attention to task, use of the vest for therapy purposes should be discontinued.

Recommendations for Future Research

We suggest that a more homogeneous population be sampled with increased control over participant inclusion and exclusion. For example, participants might include only those with a diagnosis of an autism spectrum disorder and a special education disability designation of autism under the Individuals With Disabilities Education Improvement Act of 2004. Additional information that may be helpful would include standardized scores on cognitive ability, adaptive behavior, and sensory processing measures, as well as classroom grades. We also suggest that better information regarding potential participants’ ability to attend to task be collected, including a way to rule out inattention resulting from other factors such as boredom or poor classroom management. The inclusion of more qualitative information about the participants’ experiences when wearing the vests is also recommended to guide interpretation of results.