Remedial and Special Education is soliciting manuscripts for a special series (“Advancing the Use of Evidence-Based Practices through Documentation of Negative Results”) to be guest edited by Rob Horner, Tom Kratochwill, and Joel Levin. With this announcement, the journal is recruiting manuscripts for a proposed special issue of Remedial and Special Education (RASE) targeting the value of negative results. Negative results are sometimes referred to as non-positive or “null” effects. Manuscripts are recruited with a submission date of April/May 2017 and tentative publication date of July/Aug 2017 to provide an opportunity not only for the publication of existing studies but also for the conduct of studies that may have been considered but not yet implemented. These studies may include reports of either group-randomized controlled trials research or single-case experimental intervention.
Because this special issue has an atypical focus, some clarification is appropriate. The central role of science in the field of remedial and special education is to (a) identify basic laws of nature and (b) apply those laws in the design of educational and clinical interventions/applications to achieve socially valued outcomes. The scientific process is designed to allow demonstration of specific (typically positive) outcomes and to assist in the attribution that those outcomes have to controlled variables. Although growing recognition is being given to the importance of replication in this process (e.g., Coyne, Cook, & Therrien, 2016), we believe that equal consideration should be given to the function of publishing studies documenting negative results.
Studies documenting negative results are often relegated to the “file drawers” (Rosenthal, 1979) or document files of researchers and are not included in the larger body of scientific literature. This circumstance can lead to publication bias, a major problem in the social and educational sciences (for a review and recent evidence, see Shadish, Zelinsky, Vevea, and Kratochwill, 2016). The role of negative results, however, has at least three important contributions to the emergence of scientific knowledge and documentation of evidence-based practices.
First, negative results document proposed hypotheses and interventions that do not contribute to improved outcomes (i.e., application of the proposed practice does not produce anticipated improvement in valued outcomes). Documentation of non-effects will be of greatest value in conditions where an educational or clinical procedure is highly touted, but insufficiently examined. Often research demonstrating such negative results also includes assessment of contrasting strategies or procedures that do produce desired change in valued outcomes.
Second, a more nuanced contribution of negative results is to refine the conditions under which an innovation or intervention is and is not effective (e.g., across populations, contexts, implementers). No intervention produces positive outcomes for all participants in all contexts. Defining the conditions where improvement in valued outcomes is and is not likely will be an on-going contribution of applied research.
Third, negative results may document situations where an innovation or intervention is “iatrogenic” (i.e., is functionally related to effects in the opposite direction of what prior research would have predicted). There are some cases where a particular innovation or application has strong conceptual or social value but unintentionally results in damage or negative side effects. Rigorous demonstration of iatrogenic effects is helpful to a better understanding of basic laws of nature and approaches for developing clinical technology.
We contrast the above roles of negative results from the simple absence or “disconfirmation” of anticipated positive effects due to procedural insufficiencies. Rigorous, methodologically sound intervention research is challenging to conduct. It is impressively possible for a study to be conducted without valid and reliable measurement of the dependent variable(s). If the valued outcome(s) of a study cannot be monitored with accuracy and precision, then any findings become difficult, if not impossible, to interpret. Similarly, researchers conducting any educational or clinical study have a substantial burden to document that the intervention or innovation under analysis (i.e., the independent variable) was manipulated as intended, in the context proposed, and by and with the individuals stipulated. The growing expectation that such studies measure the “integrity of treatment” is indicative of this concern.
Our goal is to publish original experimental research demonstrating the value of negative results. The special issue is not focused on discussions or commentaries of the potential role of negative results; that function is being served in other venues. Rather, the target of this special issue is on empirical research that (a) provides a rigorous research design, (b) documents valid and reliable measurement of the dependent variable(s), (c) demonstrates adherence to treatment integrity, and (d) describes the contribution of the negative results for conceptual and/or clinical advancement of the field.
This special issue of RASE will be dedicated to the memory of William R. Shadish (1949–2016), who was a constant champion of rigorous research methods and a strong believer in the valuable contribution of both positive and negative research results.
References
Coyne, M. D., Cook, B. G., & Therrien, W. J. (2016). Recommendations for replication research in special education: A framework of systematic conceptual replications. Remedial and Special Education, 37(4), 244–253. doi:10.1177/0741932516648463
Rosenthal, R. (1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86, 638–641.
Shadish, W., Zelinsky, N., Vevea, J., & Kratochwill, T. (2016). A survey of publication practices of single-case design researchers when treatments have small or large effects. Journal of Applied Behavior Analysis. Advance online publication. doi:10.1002/jaba.308
INQUIRIES/SUBMISSIONS: Please send to Robert Horner via email: robh@uoregon.edu
Sincerely,
Rob Horner, Tom Kratochwill, and Joel Levin
Editors
Remedial and Special Education