Predicting ‘It will work for us’: (Way) beyond statistics
Predicting ‘It will work for us’: (Way) beyond statistics
A great deal of attention in evidence‐based policy and practice is directed to statistical studies–especially randomized controlled trials–that support causal conclusions, which this chapter dubs ‘It‐works‐somewhere claims’. What's needed for policy and practice, however, are conclusions that the policy will work for us, as when and how we would implement it. Despite widespread recognition of the problem of external validity, it is all too easy to suppose that conclusions of the first sort provide strong evidence for those of the second sort. This chapter argues that this is not the case. Further, ‘external validity’ is the wrong way to characterize the problem. Usually the only reliable way to use an it‐works‐somewhere result as evidence for ‘It will work for us’ is via what J.S. Mill calls a ‘tendency’ claim (and the chapter calls a ‘capacity’ claim). This however points out how weak ‘It works somewhere’ is in support of ‘It will work for us’, for two reasons. (1) It takes a great deal of theory, observation and experiment, far beyond the statistical study itself, to establish a tendency/capacity claim; (2) Reliable prediction requires in addition a great deal of local knowledge supplied by neither the statistical study nor the capacity claim.
Keywords: evidence-based policy, evidence-based practice, effectiveness, efficacy, external validity, RCTs, capacities
Oxford Scholarship Online requires a subscription or purchase to access the full text of books within the service. Public users can however freely search the site and view the abstracts and keywords for each book and chapter.
Please, subscribe or login to access full text content.
If you think you should have access to this title, please contact your librarian.
To troubleshoot, please check our FAQs , and if you can't find the answer there, please contact us .
