A Bibliography of Randomized Controlled Experiments in Social Work (1949

Abstract

A strong theme in the social work literature contends that randomized experiments are an impractical, inappropriate, unethical, and rarely undertaken research method for use in social work. In a test of this claim, the author undertook a review of the English-language published literature and prepared a provisional bibliography of primary social work studies that used experimental methodology. Astonishingly, over 740 such studies were located, with the first being published in 1949. The existence of this large experimental social work literature has been largely unrecognized, in part, because much of it has appeared in a diverse array of journals associated with other disciplines. It is no longer tenable to claim that experiments are either impractical or inappropriate as a social work research method of value in making causal inferences. This bibliography will be amended in coming years, and subsequent analyses of the types of problems and interventions, which are the focus of these studies, will add to the empirical foundations of social work practice.

Keywords

randomized experiment outcome study RCT systematic review evidence-based practice literature review methodological article

For several decades, some epistemological arguments have heated the social work literature. In a broad sense, these debates have involved the fundamental philosophical assumptions that are contended to underlay social work itself. “Are we a science or more of an art?” has been one way the debate has been framed in relation to social work practice. In research, analogous positions have been carved out, with empirically oriented scientists, grounded in the philosophy of science known as positivism, occupying one corner of the ring and another large group, variously known as constructivists or postmodernists, residing in the opposite corner. Each loudly touts the superior virtues of their particular philosophical assumptions and derivative research methods but devotes equal if not more effort in criticizing the opposing positions. One flash point in this debate has concerned the proper role of randomized experiments as a source for making causal inferences. Here is one quote representing the positivist position:

… controlled trials can provide the most convincing evidence of the impact of social work activities on the welfare of children and families. Accumulating evidence of the effectiveness of interventions … should constitute the core business of social work research. To this end, is it necessary to recognize the primacy of the randomized controlled trial in exploring the relationship between social work activities and client outcomes. (Newman & Roberts, 1997, p. 287)

True experiments are an attempt to develop empirically based causal knowledge and involve the deliberate manipulation of one or more factors (the independent variables or treatments) that are received by one group of study participants and withholding this treatment from other groups. The other groups may receive nothing, a partial treatment, an alternative treatment, treatment as usual, or, rarely, some form of placebo intervention. These groups must be created using genuine methods of random assignment, wherein at the beginning of the study every possible participant has an equal probability of being assigned to one of the planned conditions. This may be done using a simple coin toss, a table of random numbers, a computer-based allocation program, or other method. When properly conducted, such random assignment creates groups that are very likely initially equivalent on all relevant dimensions (e.g., demographics, problem severity, psychosocial strengths, etc.). Accordingly, if the groups differ on one or more outcome measures following exposure to their receipt of the interventions, it may be provisionally concluded that the treatments received were causally responsible for these differences.

Experimental methodology is an attempt to reduce the possibility of bias from affecting the results, biases on the part of the experimenter, the participants, or the environmental circumstances surrounding the participants themselves. True experiments may include the use of pretests and posttests, but sometimes posttests alone are employed. The outcome measures (dependent variables) must possess acceptable levels of reliability and validity, and the sample sizes of the groups must be sufficient. Inferential statistics and/or effect sizes are almost always used as a decision tool by researchers in determining whether meaningful changes occurred. Conclusions drawn from a single experiment are increasingly being seen as needing to be replicated by independent researchers before being provisionally accepted. Conclusions drawn from a single study obviously apply, on average, to the participants in that given study, not necessarily to every person. However, if these clients were themselves randomly selected from a larger population of interest (and this is admittedly a very rare occurrence), it may be permissible to generalize the findings from a given study’s participants to the larger population of interest. However, any generalizations from a given experiment are usually made on the basis of repeatedly replicating findings with different groups of people, rather than on the basis of randomly selecting participants from a larger population of interest, given the pragmatic difficulties of the latter method.

When true experiments are conducted to evaluate the effects of a practice intervention intended to help people, the study is called a randomized clinical trial (RCT). However, it is important to recognize that although many social work experiments are not RCTs, as in tests of educational interventions, in studies of clinical judgment or diagnostic bias, or in community-based interventions, the principles of experimental logic remain the same. There are many nuances in the design and conduct of experiments. Assessments may be made of intervention effects immediately posttreatment, or some weeks or even years later. Efforts must be made to ascertain that treatments were delivered as intended, that placebo or alternative treatments possessed credibility equivalent to that of the experimental intervention, and in some instances that assessments were conducted blindly, without assessor knowledge of what intervention the participant had received (or was assigned to). Emerging standards are that RCTs be registered in advance of their being undertaken, to help avoid the so-called file-drawer problem wherein studies with disappointing results simply disappear (Harrison & Mayo-Wilson, 2014), and that investigators proactively declare if they have any potential conflicts of interest, such as a proprietary interest in the intervention, or having received consultation fees from the study’s funders. All these features make the design and conduct of experiments a difficult undertaking. However, the payoffs in terms of scientific knowledge can be immense, which is one reason this method is so highly valued (Harrison & Thyer, 1988).

The philosophical and methodological opponents to the value of experimental studies in social work have been loud and unstinting in their critique. Table 1 contains a selected list of quotes from a number of sources, authors who have variously contended that experiments are philosophically flawed, impractical, of very limited use, unethical, reductionistic, promote scientism not true science, and incapable of yielding credible knowledge. As a result, it has been contended that “Social work has produced few trials that exemplify randomized clinical trials” (Witkin & Harrison, 2001, p. 294). In a content analysis of the research designs appearing in articles published in three major social work research journals, Holosko (2010) found that true experimental designs were used in only 2.3% of the studies. Schilling, in a similar vein, noted that “intervention research is central to the advancement of a practice profession, yet by one estimate, as few as a dozen rigorous intervention studies per year are produced by social workers” (2010, p. 550). This all suggests that experiments are an unpromising and underutilized research method among social workers.

Table 1.

Examples of Anti-Experimental Critiques^a Within Social Work.

In the laboratory experiments of these days where each factor and all the conditions are controlled, it is not such a difficult matter to analyze and evaluate results; but in experiments with human beings where the factors have not yet been clearly differentiated by science and where the conditions are only partially controlled, an analysis of results may be interpreted in a different way by each observer. (Pratt, 1921, p. 90)

She (Mrs. Sidney Webb) pointed out that science uses three methods of procedure—observation, the analysis of documents, and experiment … in experimentation the individual investigator in social science is seriously limited. (van Kleeck & Taylor, 1922, p. 167)

Many theories, hypotheses, or generalizations that hold under artificially … controlled experimental conditions … fall apart … when applied to confounding situations which social workers encounter in professional practice. (Brennan, 1973, p. 7)

Proof in the logical positivist tradition is not possible for interpersonal phenomena, given the limitless variety of variables and the rudimentary nature of the research instruments. (Vigilante, 1974, p. 112)

Past research efforts … have vividly exposed the limitations encountered when attempting to utilize experimental designs when evaluating social programs.… the experimental and quasi-experimental design models may be premature for many of the issues related to the evaluation of the effectiveness of psychotherapy (Kolevzon, 1977, pp. 211, 212)

If the group-comparison experimental model has not proved useful, then other research models must be sought … a number of social work writers have been calling for a ‘moratorium’ on group comparison evaluative research.… (Wood, 1978, p. 455)

The development of an empirical (as opposed to a theoretical, clinical-intuitive) basis for practice using experimentally derived knowledge will produce a new model of practice based upon the same interactional paradigm that governs experimental research. Practice will become frankly manipulative and practitioners will employ all the devices that experimenters now use to assure the internal validity of experiments—deception, control of idiosyncratic behavior, stifling of dialog, and the repression of two-way synergistic learning. (Saleebey, 1979, p. 267)

The belief that social work models can and should be reduced to simplified, quantified time-limited, experimentally ‘testable’ models without any loss of valuable information is an example of the logical empiricist belief in reductionism in social work research. (Heineman, 1981, p. 377)

Despite the fact that social work does not readily lend itself to empirical, scientific, experimental studies, the strong desire to believe that it does has led us to accept any research that purports to be scientific. (Guzzetta, 1980, cited in Karger, 1983, p. 201)

Saleeby argues that when used in social work, the experimental paradigm elicits an operational perspective that is predisposed toward the dispassionate manipulation of people and their situations. (Karger, 1983, p. 201)

Raynor next attacks experimental methodology, presenting this as a sort of cruel sport engaged in by academics. (Sheldon, 1984, p. 636)

… research that distorted practice with the introduction of experimental procedures was unlikely to yield information that would be relevant to their practice.… Casework practice models were viewed as unscientific unless they were put on the Procrustean bed of experimental design, a procedure that cut or stretched many casework models so that they were no longer recognizable to the practitioners … So casework models derived from experimental research are inherently no better than casework models derived from other methodologies; social worker have no ethical obligation to adopt them … heuristic researchers recognize that experimental controls simply introduce different biases.… (Tyson, 1992, pp. 542, 544, 546)

The reductionist search for truth, through the use of solely experimental designs, tends to looks for exclusively ‘black and white’ outcomes, resulting in denying all possibility of human complexity, contradiction and ambiguity.… It is rare that a fully experimental design can be used in social work situations because of moral, methodological or pragmatic considerations.… Although randomization is not impossible in social work, nevertheless there can be both ethical and practical objections to it in many instances. (Triseliotis, 1998, pp. 89, 90, 91)

… experimentation does not often serve the purposes of social work evaluation … it is often impractical, unethical and operationally difficult to control and administer treatments of variable strength in the context of human services, experimental designs alone cannot totally satisfy the social workers’ political and administrative purpose of making judicious decisions on resource allocation … the experimental paradigm applied to social work evaluation did not and probably will not add much to the theoretical foundation of this profession. (Yin-Bun, 1998, pp. 77, 81–82, 84)

The use of classic randomized control trials (RCTs) … poses continued ethical worries with the potential for service users to be randomly allocated to treatment or non-treatment groups. The practicality of conducting pure experiments on human subjects outside of laboratory conditions is also questioned, given the many potential influences on a person’s behavior.… (Gibbs, 2001, p. 696)

Research, specifically experimental research … produces at best, partial analysis, reducing complex situations to elemental components. (Witkin, 2001, p. 200)

Social work has produced few studies that exemplify randomized clinical trials. (Witkin & Harrison, 2001, p. 294)

Whilst the RCT standard might be relevant for more scientific professions such as medicine, it is not relevant in social care. (Webb, 2002, p. 49)

Random assignment of individuals to an experimental or control group may not be feasible or ethical. (Zlotnik & Galambos, 2004, p. 260)

One of the major concerns about the RCT is that it is unethical to deliver a service on the basis of chance allocation … the measures of outcome employed in psychosocial trials are too unreliable or too simple to capture the subtle changes that are meaningful to practitioners and the recipients of the service. (Rushton & Monck, 2009, pp. 21–22)

… the traditional golden standard RCT per se is an obstacle to finding interventions worth considering in the complexity of routine clinical settings. The design itself obstructs the ‘healing context’ and…the potential effect of studies’ interventions may thus be underestimated. (Salander, 2011, p. 334)

RCTs are too controlled and too disconnected from the interactions of participants and environments, due to the positivistic epistemology which underlies them. (Bonell, Fletcher, Morton, Lorenc, & Moore, 2012, p. 2302)

RCTs are … controversial within social work, seen by some as unethical, positivist, uncritically imported from other disciplines, and unable to yield the certainty they promise. (Dixon et al., 2013, p. 2)

Social work researchers and evaluators do not conduct RCTs, as it is unethical in our current environments to do so … RCTs are hardly ever used. (Holosko, Hamby, & Pettus, 2013, emphasis in original)

RCTs have a specific relevance problem: at the outset, they have high legitimacy, but their narrow focus tends to reduce their practice relevance in dynamic social work contexts. (Petersen & Olsson, 2014, p. 9)

RCTs are challenging to undertake in ‘real-world’ situations … particularly with complex social interventions. Many social scientists view them with hostility—in part because of the ethics of randomization, as one group is deprived of access to a potentially effective intervention.… (Webber, 2015, p. 12)

^aThis phrase is quoted from Karger (1983). It did not originate with the present author.

Rebuttal to Critics

A comprehensive response to such criticisms is not possible here, but a partial reply can focus on the contention that experiments in social work are rare or very limited. While these philosophical and methodological arguments whirl incessantly about our heads, seemingly incapable of satisfactory resolution, what is the hapless student, practitioner, or academic to do? One useful tactic is to turn aside from philosophical or theoretical wrangling and look at actual results. Do not debate answering the question, Are randomized experiments practical/valuable/ethical in social work practice and research? Instead, turn our attention to answering the question Are randomized experiments actually being used in social work practice and research? This bibliography of such studies provides an irrefutable answer to this question and the answer is a resounding yes! This answer has been arrived at by the philosophical technique called Solvitur ambulando, a Latin phase meaning “It is solved by walking.” It was employed by Diogenes the Cynic to refute Zeno’s paradox contending that motion is unreal, by Diogenes standing up and simply walking away. Similarly, James Boswell related how Samuel Johnson refuted George Berkeley’s contention of the nonexistence of matter by kicking a stone hard enough to cause his foot to bounce off it, saying “I refute it thus” (Patey, 1986, p. 139). And in the novel Atlas Shrugged, all the governmental experts claimed that industrialist Henry Reardon’s innovative new metal was too weak to be safely used. Ignoring his critics, Reardon built a bridge made of his metal and personally drove a high-speed train across it.

Similarly, while the philosophically minded postmodernists wailed and gnashed against the principles of experimentation within social work, a very large group of social work professional practitioners and scientists have largely ignored these arguments and did experimental studies. They have walked away from the philosophical debates and simply and laudably set to work trying to answer important causal questions and test hypotheses related to theory building and evaluation purposes. In preparing this bibliography, several important issues had to be addressed, including developing practical definitions of “social work” and of “social work research.” Below is how these were defined.

The Definition of Social Work

A satisfactory inclusive and exclusive definition of what constitutes social work remains elusive and efforts to track down a disciplinary-specific body of knowledge are unlikely to be forthcoming (Thyer, 2002). When the distinguished social work educator Joel Fischer wrestled with this problem in the early 1970s, he cited Hartman’s observation:

Because people who define themselves as caseworkers define the practice so differently, and because no one has been elected to determine the definition, I assume that we can all carve out our area, practice it, teach it, and write articles about it so long as the community, clients, universities and editors will support us. (1971, p. 419)

With this as his guide, Fischer arrived at a pragmatic definition, “In a most general sense, then, casework could be defined—at least for the purpose of reviewing studies that evaluate casework- as the services of professional caseworkers” (1973, p. 6). He reiterated this position in his later book reviewing controlled outcome studies of casework, noting “Social casework then appears to be more of a professional designation than one which describes either a specific theory or method, or specific techniques of practice … social casework is defined in this book as the services provided by professional caseworkers” (Fischer, 1976, pp. 10–11, italics in original). It was interesting to hear the distinguished Professor John Brekke (July 17, 2013) makes a related claim during his address at the 2013 Islandwood Conference on Science and Social Work: “Social work science is what social work scientists do.” Thus, in attempting to delineate criteria to be used in deciding whether or not to include a given randomized study in this bibliography of social work publications, we deemed it both modest and pragmatic to rely on these earlier definitions of our field, as further operationalized below.

Inclusionary and Exclusionary Criteria

The following criteria were used in selecting references for potential inclusion in this bibliography:

The article, book, or chapter was authored or coauthored by at least one professional social worker, for example, a holder of a bachelor of social work (BSW), master of social work (MSW), or doctorate in social work (see Rubin, 1985), or the services provided were performed by professional social workers.

The citation was published in English.

The citation pertained to a primary study involving the presentation of empirical data.

Experimental and comparison or control groups were constructed using random assignment techniques.

The author could locate the study and verify its suitability for inclusion.

The following criteria were used in rejecting potential citations for inclusion in this bibliography:

The article was primarily methodological and only discussed experimental methods, rather reporting an actual study, with data.

It was not clear that services were provided by professional social workers.

It was not clear that random assignment methods were employed.

The use of these inclusionary and exclusionary criteria nevertheless resulted in some ambiguous decisions. For example, if services were provided by professionals from several disciplines, including social workers, the study was included. If a paper was authored by a nonsocial worker who was a faculty member in an academic social work program, and the paper did not otherwise meet our inclusionary criteria, it was excluded. This resulted in the exclusion of a number of studies by distinguished social scientists, such as Richard Catalano, Lawrence Palinkas, Mary Ann Test, and Daniel Shek, who are not themselves actual professional social workers but hold positions as social work faculty. If a study was published in a social work journal, but none of the authors was a social worker, and the services were not provided by social workers, then the paper was excluded. If a paper was a methodological work which described, for example, the difficulties in implementing an RCT and how these might be (or were) overcome, the study was excluded. If a paper reported on the results from only one arm of an RCT (e.g., a retrospective predictor study from among clients receiving the experimental treatment only), or the pretreatment data only from an experiment, the study was excluded.

A single given study could be included in this bibliography more than once if data were reported in more than one publication or if expanded or follow-up data were reported in publications subsequent to the original report. Thus, the total number of citations in this bibliography is a slightly inflated reflection of the number of studies actually conducted. Offsetting this inflationary factor is that undoubtedly a large number of studies that met our inclusionary criteria were not located and hence not listed.

Limitations

This bibliography project was undertaken by the author without funding or release time, and he is not a professional information specialist. This bibliography is the result of several search methods:

Reference harvesting, extracting potential citations from the reference lists of existing studies

The use of several large-scale databases such as PsycINFO and the Web-of-Science, with suitable search terms (e.g., experimental, “random assignment,” RCT, etc.), covering all years through 2013, and located anywhere in the citation.

Contacting selected social work authors known to have published randomized experiments to assure that all their appropriate work was included.

Contacting the authors of unclear papers to see if inclusionary criteria were met and exclusionary criteria not met.

Manually searching selected social work and nonsocial work journals and reviewing article titles, abstracts, and authors.

This bibliography is decidedly NOT the result of a formal systematic review and should be viewed as incomplete. Undoubtedly, many appropriate studies were not found and should have been included (false negatives). And a few studies may have been inappropriately included (false positives), although great lengths were gone to assure that every citation appearing did meet our inclusionary criteria. There are likely large numbers of unpublished social work RCTs residing in file drawers or in the “gray literature,” for example, dissertations, conference proceedings, and websites. In acknowledging the imperfections of this published product, readers are invited to send the author corrections or additions, works to be newly added to future updates, and works currently listed which should be excluded. These amendments will be most gratefully received.

Note that this bibliography does not provide a critical review of the methodological rigor of the included studies. Many are, to be frank, methodologically quite weak, with small sample sizes yielding underpowered studies. Some investigations used outcome measures of unknown validity, did not follow-up on the long-term maintenance of any apparent improvements, and so on. Also, some arrived at negative conclusions, finding no differences following intervention and others determined that social work services seemed to produce iatrogenic results, for example, harm. Many positive studies may lack external validity, in that the favorable results do not hold up well when the tested intervention is applied to more diverse or complex client groups than those in the initial study, or when the social workers are less well trained or supervised than those in the published report. Any compilation of published studies also overlooks the file-drawer problem, the possibility that negative studies are less likely to be published, thus biasing the conclusions which may be drawn from this public literature. These are all legitimate criticisms and we cannot argue against them. However, we modestly view this bibliography as a useful service for future investigators able to undertake more detailed meta-analyses and to aid those completing genuine systematic reviews of social work intervention. This tabulation also brings together in one place over 750 social work randomized experiments, published all over the scholarly map, including many appearing in nonsocial work periodicals. Future researchers will be able to use this bibliography to analyze the types and features of these RCTs being undertaken by members of our field (e.g., related to mental health, child welfare, substance abuse, welfare policy; the various journals we publish in, types of translational research, etc.). The bibliography may be found in the Appendix.

Discussion

Several aspects of this bibliography seem worthy of highlighting, but perhaps most significant are the sheer numbers of experiments produced in our field. Frankly, it was astonishing to locate over 740 published studies describing actual experimental studies and RCTs that met the inclusionary criteria. This volume of work is well camouflaged, being spread out across an incredible array of journals, mostly outside of the social work disciplinary literature. The invisibility of our work may account, in part, for the perception that experimental studies are rarely undertaken on social work practice or authored by social workers.

Another feature we found interesting is the amazing variety of research topics addressed by social workers via experimentation. Represented are a plethora of problems, interpersonal, psychological, psychosocial, and community-based issues. Independent variables included psychosocial interventions, biological therapies, educational methods, and social welfare policies. The considerable extent to which social workers have contributed to RCTs on pharmacotherapies has been relatively underappreciated. Also evident is the extent to which social work authors function as full members of interdisciplinary teams, sometimes assuming leadership roles in investigations, as assessed by their position as senior authors.

We can see a potential criticism of this comprehensive bibliography being something along the lines of “Well, many of the investigations you cite are not really examples of social work research.” Such a criticism ignores the virtual impossibility of providing unique, comprehensive, yet exclusionary definitions of social work practice and of social work research, ones which improve upon the ones we selected—“Social work practice is what social work practitioners do” and “Social work research is what social work researchers do,” pragmatic definitions with some professional precedents. This is a long-standing issue, the existing diversity, and the ever-widening scope of social work. As far back as 1925, it was noted that “The trained social worker is prepared to find and expects to find social work extended from year to year to include activities that formerly were not considered social work at all” (Washington, 1925, p. 169) and about 50 years later to read “One of the most significant movements in the profession has been the tendency of social workers to assume an ever widening range of tasks not traditionally associated with their repertoire of responsibilities” (Brennan, 1973, p. 8). In 1949, the Journal of Consulting and Clinical Psychology published a review of two books based on conference papers delivered at the Columbia University School of Social Work and the Community Service Society of New York. It was noted that:

They show how closely psychology and social work have drawn together in recent years as each has moved toward the other’s position in emphasis on theory and service. Even much of their research becomes indistinguishable, as social and clinical psychology acquire social worker’s broad cultural frame of reference, and social work acquires the former’s refinements of technique. (Staff, 1949, p. 385)

In the inaugural issue of Social Work, Eaton noted that:

In a report prepared for the Bureau of Labor Statistics, social workers were found to be doing almost anything that aimed at helping people with social, economic, psychological, and educational problems. They were engaged in 145 different vocational functions, but had no exclusive jurisdiction over any of them. (1956, p. 11)

Around the same time, Hollis and Taylor claimed that “social work and social workers should be looked upon as evolving concepts that are yet too fluid for precise definition” (1951, p. 51). While this bibliography does not validate the 1919 recommendation of Richard C. Cabot that social workers include as a part of their practice the skill of administering enemas to clients (1919, p. 27), we clearly are intimately involved in evaluating the outcomes of psychotherapy, behavior therapy, medications, complementary and alternative medicines and therapies, herbal products, electroconvulsive therapy and other biological treatments, and welfare policy, all practice and research domains historically not always associated with our field. Arguments that the practice activities undertaken by the social work authors of these experiments are somehow not really social work lack substance until such critics can present a satisfactory definition of social work practice. We offer the definition of a psychologist provided by the American Psychological Association (APA) to illustrate the ubiquity of this issue: “Psychologists have a doctoral degree in psychology from an organized, sequential program in a regionally accredited university or professional school” (https://www.apa.org/support/about/apa/psychologist.aspx#answer). In other words, a psychologist is defined by the degree earned, not by a limited range of professional or scientific activities. This is similar to aspects of the definition of social work research contained in The Social Work Dictionary (Barker, 2014, p. 404): “It is conducted by people educated in the field of social work, with BSW, MSW, or PhD degrees ….” Another precedent for our broad inclusionary criteria was established in the Harriett M. Bartlett Practice Effectiveness Project (Videka-Sherman, 1988), which included studies if “1. The practitioners were social workers (BSW, MSW, doctorate) or an interdisciplinary team that included social workers” (Saunders, 1986, p. 3). The APA also “defines” their field as follows: “Psychology is a diverse discipline, grounded in science, but with nearly boundless applications in everyday life” (http://www.apa.org/about/). Such an expansive definition encompasses many other fields besides psychology, yet that profession progresses onward, untroubled by its shaky ontological foundations. Sociology too wrestles with this problem, and one solution was simply to assert that “Sociology is what sociologists teach” (Kennedy & Kennedy, 1942, p. 661).

Our bibliography refutes the contentions that RCTs are impractical and rarely undertaken. The careful oversight provided by human subjects review committees is especially focused on experimental studies of interventions with vulnerable and oppressed clients, hence ethical considerations are carefully attended to. Experimental enthusiasts have no hesitancy in asserting that randomized controlled trials and other experiments can be a very useful methodology, and for some research questions and hypotheses, experiments may well be the BEST research method. We unambiguously agree with Campbell and Russo (1999, p. 102) that “True experiments should almost always be preferred to quasi-experiments where both are available.” But I hasten to sooth the sensibilities of my postmodernist colleagues by freely acknowledging that best does not mean only, and by also recognizing that for many true causal questions, experiments may indeed not be a suitable approach (Sainz & Epstein, 2001). Other methods, such as quasi-experiments or regression discontinuity designs, can also yield accurate causal inferences in some situations (e.g., the link between smoking and cancer), absent RCT evidence (Thyer, 2012a). There are also many research questions in which experiments are simply not a suitable tool—descriptive studies, epidemiological investigations and etiological inquiries, psychometric assessment studies, needs assessments, client satisfaction studies, and, yes indeed, studies of clients’ reports of their inner experiences, all represent fields of inquiry in which experimental research has little to offer (Thyer, 2012b) and other legitimate methods of study are appropriate. Again, quoting Campbell and Russo, we agree that “… qualitative knowing is absolutely essential as a prerequisite foundation for quantification in any science. Without competence at the qualitative level, one’s computer printout is misleading or meaningless” (1999, p. 141).

This bibliography is offered in the hope that it will be of value to future scholars and to further the appropriate use of experimental and RCT methodologies within the field of social work. These methods have occupied a valuable and indeed honored place within our armamentarium of research techniques for almost 70 years. While it is not possible to assert that our oeuvre of experimentalist studies is comparable to those of related disciplines such as psychology and psychiatry, our historical and contemporary use of these methods is certainly in keeping with the highest traditions of rigorous scientific investigations and illustrate that these approaches are viable, valuable, and go far to validate the view of social work research as a legitimate field of social and behavioral science. To put this issue into a historical perspective, we cite Gibbons’ observation that “… the first randomized control trial in medicine (evaluating the effect of penicillin on tuberculosis) occurred in the late 1940s and psychotherapy outcome research only began to emerge in the late 1950s” (2001, p. 6). Oakley goes so far as to assert that “… the history of experimentation in social science predates that in medicine in certain key respects” (1998, pp. 1239–1240). The earliest study located employing an RCT design in social work, Powers (1949), was actually begun in 1937, which likely predates the use of these methods within clinical psychology to evaluate psychotherapy. By these benchmarks from related disciplines, the experimentalist tradition within social work holds up very favorably. It was pioneer Social Worker Edith Abbott (1931) who made the following recommendation, over 80 years ago:

The faculty and students of a professional school of social work should together be engaged in using the great method of experimental research which we are just beginning to discover in our professional education programme, and which should be as closely knit into the work of a good school of social work as research has been embodied into the program of a good medical school …. (p. 55)

Bibliographic research along the lines of this report has a long history, not only within science broadly but also with the narrower field of evidence-based practice, which particularly prizes experimental studies in part because such rigorous studies set the stage for meta-analyses and systematic reviews. Forsetlund, Chalmers, and Bjorndal (2007) used bibliographic methods to try and track down the earliest publications reporting experimental studies of social interventions. Petrosino (2003) listed numerous examples of RCTs in the field of childhood intervention in the human services, and Boruch (1974) and Boruch, McSweeney, and Soderstrom (1978) tabulated the use of RCTs in social evaluation studies. The present bibliography is offered as a further contribution along these lines in the hope that it will be of service to future scholars in the field of research on social work practice and to provide inspiration to social workers contemplating undertaking an experimental investigation. True experiments are a widely used research method within the field of social work, applicable to the diverse areas of our practice, and should be further encouraged as an important building block in the edifice of evidence-based social work practice.

Footnotes

Acknowledgments

The author gratefully acknowledges the assistance of colleagues who provided citations to experimental social work studies.

Author’s Note

Readers are welcome to contact the author with suggested additions, deletions, or corrections to this provisional bibliography.

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

Appendix

References

Abbott

(1931). Social welfare and professional education. Chicago, IL: University of Chicago Press.

Barker

R. L.

(Ed). (2014). The social work dictionary (6th ed.). Washington, DC: NASW Press.

Bonell

Fletcher

Morton

Lorenc

Moore

(2012). Realist controlled trials: A new approach to evaluating complex public health interventions. Social Science & Medicine, 75, 2299–2306.

Boruch

R. F.

(1974). Bibliography: Illustrative randomized field experiments for program planning and evaluation. Evaluation, 2, 83–87.

Boruch

R. F.

McSweeney

A. J.

Soderstrom

E. J.

(1978). Randomized field experiments for program planning, development and evaluation: An illustrative bibliography. Evaluation Quarterly, 2, 655–695.

Brekke

(2013, 7 17). A science of social work, and social work as an integrative scientific discipline: Have we gone too far, or not far enough? Paper presented at the Islandwood Roundtable on Science in Social Work, Bainbridge Island, Washington.

Brennan

W. C.

(1973). The practitioner as theoretician. Journal of Education for Social Work, 9, 5–12.

Cabot

R. C.

(1919). Social work: Essays on the meeting ground of doctor and social worker. Boston, MA: Houghton Mifflin.

Campbell

D. T.

Russo

M. J.

(1999). Social experimentation. Thousand Oak, CA: Sage.

10.

Dixon

Biehal

Green

Sinclair

Kay

Parry

(2013). Trials and tribulations: Challenges and prospects for randomized controlled trials of social work with children. British Journal of Social Work, 43, 1–19. doi:10.1093/bjsw/bct035

11.

Eaton

(1956). Whence and whither social work: A sociological analysis. Social Work, 1, 11–26.

12.

Fischer

(1973). Is casework effective? A review. Social Work, 18, 5–20.

13.

Fischer

(1976). The effectiveness of social casework. Springfield, IL: Charles C. Thomas, Publisher.

14.

Forsetlund

Chalmers

Bjorndal

(2007). When was random allocation first used to generate comparison groups in experiments to assess the effects of social interventions? Economic Innovations and New Technologies, 16, 371–384.

15.

Gibbons

(2001). Effective practice: Social work’s long history of concern about outcomes. Australian Social Work, 54, 3–13.

16.

Gibbs

(2001). The changing nature and context of social work research. British Journal of Social Work, 31, 697–704.

17.

Karger

H. J.

(1983). Science, research, and social work: Who controls the profession? Social Work, 28, 200–205.

18.

Harrison

Mayo-Wilson

(2014). Trial registration: Understanding and preventing reporting bias in social work research. Research on Social Work Practice, 24, 372–376.

19.

Harrison

D. F.

Thyer

B. A.

(1988). Doctoral research on social work practice. Journal of Social Work Education, 24, 107–114.

20.

Hartman

(1971). But what is social casework? Social Casework, 52, 411–419.

21.

Heineman

M. B.

(1981). The obsolete scientific imperative in social work research. Social Service Review, 55, 371–397.

22.

Hollis

F. V.

Taylor

A. L.

(1951). Social work education in the United States. New York, NY: Columbia University Press.

23.

Holosko

M. J.

(2010). What types of designs are we using in social work research and evaluation? Research on Social Work Practice, 20, 665–673.

24.

Holosko

M. J.

Hamby

W. A.

Pettus

J. A.

(2013, 5 13). Social work research designs: Let’s stop apologizing! Paper presented at the annual convention of the Campbell Collaboration, Chicago, IL, USA.

25.

Karger

H. J.

(1983). Science, research, and social work: Who controls the profession? Social Work, 28, 200–205.

26.

Kennedy

R. J. R.

(1942). Sociology in American colleges. American Sociological Review, 7, 661–765.

27.

Kolevzon

M. S.

(1977). Negative findings revisited: Implications for social work practice and education. Clinical Social Work Journal, 5, 210–218.

28.

Newman

Roberts

(1997). Assessing social work effectiveness in child care practice: The contribution of randomized controlled trials. Child: Care, Health, and Development, 23, 287–296.

29.

Oakley

(1998). Experimentation and social interventions. British Medical Journal, 317, 1239–1242.

30.

Patey

D. L.

(1986). Johnson’s refutation of Berkeley: Kicking the stone again. Journal of the History of Ideas, 47, 139–145.

31.

Petrosino

(2003). Estimates of randomized controlled trials across six areas of childhood intervention: A bibliometric analysis. Annals of the American Academy of Political and Social Science, 589, 190–202.

32.

Pratt

A. B.

(1921). The relation of the teacher and the social worker. Annals of the American Academy of Political and Social Science, 18, 90–96.

33.

Rubin

(1985). Practice effectiveness: More grounds for optimism. Social Work, 30, 469–476.

34.

Rushton

Monck

(2009). Enhancing adoptive parenting: A test of effectiveness. London, England: British Association for Adoption and Fostering.

35.

Sainz

Epstein

(2001). Creating experimental analogs with available clinical information: Credible alternatives to “gold-standard” experiments? Social Work in Health Care, 33, 163–183.

36.

Salander

(2011). It is futile to believe that RCT studies will steer us to Godot (letter). Psycho-Oncology, 20, 333–334.

37.

Saleebey

(1979). The tension between research and practice: Assumptions of the experimental paradigm. Clinical Social Work Journal, 7, 267–284.

38.

Saunders

B. N.

(1986). Introduction. In Videka-Sherman

(Ed.), Studies of research on social work practice: A bibliography (pp. 1–3). Silver Spring, MD: National Association of Social Workers.

39.

Sheldon

(1984). Evaluation with one eye closed: The empiricist agenda in social work research—A reply to Peter Raynor. British Journal of Social Work, 14, 635–637.

40.

Schilling

(2010). Commentary: The challenge of nonexperimental intervention studies in social work. Research on Social Work Practice, 20, 550–552.

41.

Staff. (1949). A review of social work as human relations and the family in a democratic society. Journal of Consulting and Clinical Psychology, 13, 384–385.

42.

Thyer

B. A.

(2002). Developing discipline-specific knowledge for social work: Is it possible? Journal of Social Work Education, 38, 101–113.

43.

Thyer

B. A.

(2012a). Quasi-experimental research designs. New York, NY: Oxford.

44.

Thyer

B. A.

(2012b). The scientific value of qualitative research for social work. Qualitative Social Work, 11, 115–125.

45.

Tyson

K. B.

(1992). A new approach to relevant scientific research for practitioners: The heuristic paradigm. Social Work, 37, 541–556.

46.

Triseliotis

(1998). When is evaluation a scientific activity, when is it not? Scandinavian Journal of Social Welfare, 7, 87–93.

47.

van Kleeck

Taylor

G. R.

(1922). The professional organization of social work. Annals of the American Academy of Political and Social Science, 101, 158–168.

48.

Videka-Sherman

(1988). Metaanalysis of research on social work practice in mental health. Social Work, 33, 325–388.

49.

Vigilante

J. L.

(1974). Between values and science: Education for the profession during a moral crisis or is proof truth? Journal of Education for Social Work, 10, 107–115.

50.

Washington

F. B.

(1925). What professional training means to the social worker. Annals of the American Academy of Political and Social Science, 121, 165–169.

51.

Webb

(2002). Evidence-based practice and decision analysis in social work. Journal of Social Work, 2, 45–63.

52.

Webber

(Ed). (2015). Applying research evidence in social work practice. New York, NY: Palgrave Macmillan.

53.

Witkin

S. L.

(2001). Complicating causes. Social Work, 46, 197–201.

54.

Witkin

S. L.

Harrison

W. D.

(2001). Whose evidence and for what purpose? Social Work, 46, 293–296.

55.

Wood

K. M.

(1978). Casework effectiveness: A new look at the research evidence. Social Work, 23, 437–458.

56.

Yin-Bun

(1998). The use of experimental design in social work evaluation. Asia Pacific Journal of Social Work, 8, 77–87.

57.

Zlotnik

J. L.

Galambos

(2004). Evidence-based practice in health care: Social work possibilities. Health & Social Work, 29, 259–262.

A Bibliography of Randomized Controlled Experiments in Social Work (1949–2013)

Abstract

Keywords

Rebuttal to Critics

The Definition of Social Work

Inclusionary and Exclusionary Criteria

Limitations

Discussion

Footnotes

Acknowledgments

Author’s Note

Declaration of Conflicting Interests

Funding

Appendix

References