Julian Reiss: Evidence for Use

This post was sent by Julian Reiss (Erasmus University of Rotterdam, see here) speaking to the subjects of a recent comments discussion.
————————

Analytical philosophers of science, especially those trained at an Anglo-American university, tend to ask questions that are abstract, narrow and pertain to somewhat idealized circumstances. They are abstract so that answers stand a chance of being general; they are narrow so that answers stand a chance of being precise; and they pertain to idealized circumstances so answers stand a chance of being correct. ‘Evidence for use’ can be understood as a reaction against this way of doing philosophy.

For this way of doing philosophy comes at a cost: the more abstract, narrow and ideal a question is, the less likely it is to address an issue that has broader social relevance. Proponents of evidence for use urge instead the pragmatist vision of philosophers contributing to solving the pressing social issues of the civilization they are a part of. The idea of evidence for use, then, is that philosophers of science interested in theory and evidence should ask questions and frame answers in ways that have some societal significance.

The idea has origins in Philip Kitcher’s work on the ‘well-ordered science’ (most importantly in his 2003 OUP book Science, Truth and Democracy) and Nancy Cartwright’s recent work on evidence (see for instance her paper ‘Well-Ordered Science: Evidence for Use’ that was published in Philosophy of Science in 2006). A science is well-ordered to the extent that its research priorities are such that they would be endorsed in a democratic deliberation among well-informed participants committed to engagement with the needs and aspirations of others. In other words, Kitcher demands that science should ask the right questions, and in the right ways. Cartwright’s concern is mainly with methodology: how do we devise methods so that the products of science help solving practical problems?

The recent movements of evidence-based medicine and evidence-based policy can illustrate what is at stake here. These movements demand that the causal claims on which we based our policies (such as decisions to approve a new drug or implement a new schooling program) are supported by high-quality evidence, which in their understanding means randomized controlled trials (RCTs). Indeed, RCTs can be shown to prove a causal claim, given certain assumptions. But there are two major problems: first, the assumptions required are exceedingly narrow so that their satisfaction is unlikely except under ideal conditions; second, even when satisfied, the RCT proves a narrow ‘it works somewhere’ causal claim (in Cartwright’s words), whereas what we need to know is that ‘it works for us’. Because the correctness of the claim proved by an RCT depends crucially on the characteristics of the test population, the circumstances of the test and the specific ways of administering the treatment, results are unlikely to continue to hold in the circumstances we are ultimately interested in.

Evidence for use invites us to refocus from questions we can answer easily (such as ‘How do we design an experiment so we can be reasonably certain about its result?’) to questions that matter to society (such as ‘How do we design an empirical study so we can be reasonably certain that a policy based on it will be successful?’). For a recent special issue of the journal Synthese that takes up some of these themes, see here.

Advertisements

One thought on “Julian Reiss: Evidence for Use

  1. Another problem with RCT in the social sciences and the humanities, as I see it, is that the populations on which the test is performed might not be fully informed and consenting.

    Take the poverty action lab at MIT (povertyactionlab.org). The fact that experiments take place on population stricken by poverty makes me ill at ease: are we, the scientists, depriving them of their dignity by what is in actuality, “peforming experiments” on them?
    About ethical issues, the website of the Poverty Action Lab states:
    “If there is any potential risk in participating, then everyone involved must be informed of the risks and give their consent to participate. Even if there do not appear to be risks, any experiment should get the informed consent of all participants (in both treatment and comparison groups).”
    [ http://www.povertyactionlab.org/methodology/who/who-participates-randomized-evaluations ]
    I just wonder whether, when you are poor and sick, you really have a choice when invited by an NGO or a university lab to participate in an RCT. I think not.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s