The issue of quality in qualitative research

Abstract

Abstract This article addresses the perennial issue of the criteria by which qualitative research should be evaluated. At the present time, there is a sharp conflict between demands for explicit criteria, for example in order to serve systematic reviewing and evidence‐based practice, and arguments on the part of some qualitative researchers that such criteria are neither necessary nor desirable. At issue here, in part, is what the term 'criterion' means, and what role criteria could play in the context of qualitative enquiry. Equally important, though, is the question of whether a single set of criteria is possible across qualitative research, given the fundamental areas of disagreement within it. These reflect divergent paradigms framed by value assumptions about what is and is not worth investigation. In addition, there are differences in methodological orientation: over what counts as rigorous enquiry, realism versus constructionism, and whether the goal of research is to produce knowledge or to serve other goals. Notes 1. For accounts of the history of these challenges to educational research, in the UK and the USA, see, for example, Hammersley (Citation2002) and Biesta (Citation2007), pp. 1–5. 2. I have suggested elsewhere that this is open to doubt (Hammersley, Citation2007). 3. These are lucidly reviewed and consolidated by Spencer et al. (Citation2003). See also Altheide and Johnson (Citation1994); and the recent discussion by Furlong and Oancea (Citation2005) of criteria for assessing applied and practice‐based research. An assessment of the latter is provided in Hammersley (forthcoming). There has also been much discussion of the issue within the context of the ESRC's Teaching and Learning Research Programme, focusing in particular on the issue of 'warrant': see, for example, James et al. (Citation2005). 4. There could be a position that is beyond that of Smith and Deemer (Citation2000), a position where the whole point of criteria is to transgress them! Also near this far end of the spectrum would be Feyerabend's rejection of method (Feyerabend, Citation1975). 5. Here I am of course abstracting away from the, very real, possibility that the judgements of both researchers and 'users' will be biased by their values, preferences, and interests. 6. I am not suggesting that quantitative research is entirely homogeneous; it is not. Experimental and survey research have conflicting orientations in some key respects. However, the difference does not lie at as deep a level as the differences in orientation among qualitative researchers. 7. For excellent recent discussions of Kuhn's work, see Hoyningen‐Huene (Citation1993), Bird (Citation2000) and Sharrock and Read (Citation2002). 8. For an account of some of this diversity in the field of educational research, see Hammersley (Citation2007). 9. For discussion of these orientations, see Hammersley (Citation2000, Citation2004a). 10. For a more detailed discussion of work in this area, see Hammersley (Citation1990). I have specifically avoided the other fields that have become almost standard examples in demonstrating that research can be motivated by controversial values: the case of research on racial differences in intelligence and that of research on 'effective schooling'. For a recent discussion of the latter that illustrates my point, see Clark (Citation2005). 11. Empirical work inspired by this programme in the field of education has, for example, involved detailed analysis of classroom processes: see Baker (Citation1997). 12. To a large extent, this arose from structuralism and the various moves beyond or behind it that have come to be labelled 'post‐structuralism'. There are significant parallels, as well as important differences, between ethnomethodology, inspired by phenomenology, and structuralism. 13. There are few examples of this approach within education. For what is probably the most entertaining exemplar in social science generally, see Ashmore (Citation1989). 14. The issues involved here are complex ones. See Putnam's (Citation2002, pp. 99–100) criticisms of Rorty's position. 15. The term 'activism' is not entirely satisfactory, not least because it might seem to imply an acceptance of the arguments of activists: that conventional kinds of educational research are inactive, being the pastime of those closed off from the world in their ivory towers. Needless to say, I do not accept that implication. 16. Some advocates of action research seem to come close to this position, see Elliott (Citation1988). For a critique of this, see Hammersley (Citation2003c, Citation2004b). 17. See, for example, Denzin (Citation1992), p. 131. 18. I have not been able to find any clear presentation of this argument for intervention, but an example of the approach is Miller and Whalley (Citation2005). I am not suggesting that these authors would accept my description as an accurate account of their motivation. 19. Seale (Citation1999) provides an example of this sort of position. 20. Hodkinson (Citation2004) illustrates this position, see Hammersley (Citation2005). It is worth noting that this does not have to be taken as implying that there are incommensurable paradigms, the exponents of which simply cannot understand one another. An alternative metaphor is different language communities, where while true translation may not be possible learning the other language is. This is an analogy that Kuhn uses in his later work, as against the perceptual analogy on which he relies in The Structure of Scientific Revolutions (See Kuhn, Citation1970). 21. Feuer et al. (Citation2002) exemplify this position.