Are there qualitative and quantitative cultures in social science? by Martyn Hammersley, Open University
A whole range of terms have been used to make sense of the bewildering variety of methodological orientations, practices and assumptions to be found among social scientists today. One of the most common, of course, is the distinction between qualitative and quantitative approaches, even though this is recognised to be very crude, and is frequently challenged or rejected – it seems we can’t do without it. Here I want to address this distinction in a quite specific way, via the question of to what noun the adjectives ‘qualitative’ and ‘quantitative’ should properly be applied.
There are, of course, several widely used candidates. They include ‘techniques’ or ‘methods’, but these terms are often felt to neglect the discrepant assumptions that underpin qualitative and quantitative research. As a result, various alternatives have been adopted, these being designed – at the very least – to indicate that more is at stake. They include: ‘approaches’, ‘methodologies’, ‘traditions’, and ‘paradigms’. However, these terms are themselves open to criticism. ‘Approaches’ is perhaps the least offensive, but in its blandness shares similar dangers to ‘methods’ and ‘techniques’. Meanwhile, there have been recurrent disputes over the meaning of ‘methodology’, and whether it can legitimately be used in the plural to refer to distinctive sets of assumptions about how to go about research. The problem with ‘tradition’ is that it could carry the implication that what is done on each side of the divide is justified simply because ‘that’s how we’ve always done it’. I remember hearing a senior quantitative researcher employ precisely this defence when the appropriateness of a commonly-used statistical technique was challenged by a statistician. Finally, the term ‘paradigms’ is often taken to imply internally coherent and incommensurable orientations that are adopted as a matter of faith or personal preference. Whilst some qualitative researchers relish this idea, we might wonder how social science could be scientific if it is locked into competing paradigms that rely on such foundations. And if we revert to Kuhn’s (1970) more sophisticated version of this concept, we have the problem that he regarded social science as pre- or even non-paradigmatic – ironically, precisely because it involves multiple, competing approaches. Moreover, even if we renounce the commitment to science, there remains the question of how the public funding of such ‘ideological’ paradigms could be justified, or indeed of what is the point of inquiry based on them – do they not simply trump all evidence, if you’ll pardon the expression.
Perhaps, therefore, we should welcome the suggestion from Goertz and Mahoney (henceforth G and M) to think instead of quantitative and qualitative cultures (Mahoney and Goertz 2006; Goertz and Mahoney 2012)? ‘Culture’ is, of course, very close in meaning to ‘tradition’, especially if we interpret this term in the sense used by hermeneutic philosophy (Barthold nd): one usually inherits a culture, and the implication is that engaging in practical action of any kind necessarily relies on convention, habit and judgment; on ‘what we all know to be the case’, as encapsulated in maxims, glorious exemplars and rhetorical contrasts; and even on what ‘feels’ more and less important, fruitful, acceptable, etc. So, while cultures involve values, goals, principles, etc, these are embedded in capabilities, habits, dispositions, and sensibilities. In terms of currently influential jargon, from this perspective, research is not, and cannot be, ‘evidence-based’.
One of the several oppositions in which the term ‘culture’ has been implicated, since the eighteenth century, is with ‘rationality’. Very crudely speaking, in the course of the Enlightenment, Reason came to be opposed to Tradition (for which, read ‘culture’): the task was to subject existing beliefs, practices, and institutions to rational appraisal, to counter myth, ideology, habit, custom, and ignorance, and to order society on a new basis. This is a commitment that the philosopher Michael Oakeshott (1962) disparagingly labelled ‘rationalism’. It involves an insistence on questioning all assumptions, clarifying goals, carefully assessing the effectiveness of various means for achieving those goals solely on the basis of explicit evidence, and the idea that there is one best way to be discovered for dealing with any task: a gold standard or a single ‘logic’. Any appeal to the uncertain or the unknowable, to what is less than fully explicated, or at least to what cannot be put into words, or better still numbers, in a ‘transparent’ manner, is rejected as mysticism. Here one could cite ‘Kelvin’s dictum’, which was engraved on the wall of the Social Sciences building at the University of Chicago: ‘…when you can measure what you are speaking about and can express it in numbers, you know something about it, but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind’ (to which the economist Frank Knight replied that in practice this was taken to mean: ‘if you cannot measure it, measure it anyhow’). It’s worth emphasising that, in its original form, rationalism insisted that fundamental beliefs and goals, not just particular methods and attitudes, are open to rational appraisal: utilitarianism was probably the most influential attempt to do this.
This conflict between ‘culturalism’ and ‘rationalism’ has tracked through the history of Western thought and political practice in the past two centuries, and persists today; modernism (the term is more or less synonymous with ‘rationalism’ in this context) is still alive and kicking, despite the obituaries. Indeed, it is to be found within the field of social research methodology. At first sight we find this conflict exemplified in G and M’s resistance to King et al’s (1994) (henceforth KKV) rationalist insistence that qualitative researchers must conform to ‘scientific requirements’, formulated as a single logic, in which quantitative method is effectively treated as the gold standard. On the face of it, G and M’s argument amounts to methodological multiculturalism.
However, this is not quite accurate. While G and M may appear to be anthropologists of social science who have identified two methodological cultures and set out to describe – not evaluate – them, in fact their main aim was to justify the distinctive character and value of qualitative research in the face of criticism from quantitative researchers, particularly KKV. Moreover, much like the latter, they insist that what qualitative researchers do ought to be, and is in principle, open to full explication. The point of disagreement is that they insist that there are two legitimate ways of doing social science, not just one. Moreover, while they label these ways ‘cultures’, this is not really how they formulate them in the body of their article and book; rather, they present them as sets of strategies that can be judged as more or less appropriate in pursuit of particular goals. They treat the overall goal of social science as to produce causal knowledge, and what is distinctive about qualitative research for them is that it offers a different kind of causal explanation from that of quantitative research: it explains particular events, or types of event, rather than analysing the mean effects of causes within populations. They argue that what is rational in pursuit of one of these goals is not rational in pursuit of the other; and that this is the reason for the ‘cultural’ differences. Yet cultures are not instruments devised to serve purposes; they are natural developments that are beyond anyone’s overall control. As a result, they are not open to rational appraisal in instrumental terms, at least not in any simple way.
In short, then, G and M are – to a large extent – rationalists in culturalist clothing. Indeed, they deviate from the anthropological path even in defining qualitative research. That path would imply setting out to identify how this label is used by the natives (in this case social scientists), and then investigating what seem to be the predominant core beliefs and practices amongst those who are labelled, and particularly those who label themselves, in this way; while also taking account of any subcultural variation. An obvious starting point would be to look at the contents of those journals that have phrases like ‘qualitative research’ or ‘qualitative inquiry’ in their titles. But, instead, G and M sharply restrict the term’s reference; and they do this on the basis of a distinction that they regard as methodologically, rather than anthropologically, paramount: what is included is only qualitative research that aims at causal conclusions via within-case investigations employing Boolean logic (explicitly or implicitly). In effect, what we have here is a focus on one subculture in the field of qualitative research, the one to which the authors themselves belong. They separate off the rest of qualitative research – the vast bulk of it, in terms of social science generally – as ‘interpretive’, constituting a third, non-scientific, culture. They also imply that the difference between the first two cultures is less fundamental than what divides them from the ‘interpretive’, this being effectively the division between science and the humanities – C. P. Snow’s ‘two cultures’. However, we should also note that the divisions among the various sorts of work that G and M consign to this third category – from diverse forms of discourse analysis through various kinds of ethnography to multiple versions of action research, from critical realism to postmodernism, ethnomethodology to ‘new materialisms’ – are also very deep. Clearly, then, there are more than two, or even three, cultures here.
Of course, in characterising culturalism and rationalism I have juxtaposed two caricatured positions. In my view, it would be unwise to adopt either of them exclusively. So, I am sympathetic to some sort of middling line. Indeed, I see G and M’s conception of qualitative research as an important counter to much that comes under their heading of ‘interpretive culture’, even though I wouldn’t sign up to it in toto: I wish that qualitative researchers outside of political science would give the approach that G and M champion more attention. In my view G and M are far too culturally tolerant in their treatment of ‘interpretivist’ research: here, a stronger dose of rationalism is called for! Some of this work espouses goals that are not appropriate to research; there is a tendency to engage in mere pattern-making, in the manner that has come to be labelled ‘bricolage’; and there is often bias deriving from a desire to criticise the phenomena being investigated. I also believe that, in their advocacy of multiculturalism, G and M are also too tolerant towards how standard quantitative techniques are used. May not some cultures be pathological?
So, we need to find ways of rationally appraising the diverse goals and practices to be found among qualitative researchers today, but at the same time we must recognise that this cannot be done simply on the basis of ‘logic’ or ‘evidence’: contextually-sensitive judgments are required. As part of this we could compare what qualitative researchers say they are doing with what they actually do; a strategy that, interestingly enough, has been used by anthropologists since at least the time of Malinowski. Thus, while G and M are correct that a great many qualitative researchers today deny that they are engaged in causal analysis, looking closely at their practice, and adopting a reasonably generous conception of what causal analysis is aimed at, it is hard to resist the conclusion that this is often the goal they are actually pursuing (Hammersley 2014:ch1); albeit not in the manner that G and M recommend, or – for that matter – very effectively. In fact, very often, they rely informally on the idea that various factors contribute additively to the likelihood of some type of outcome – in other words on a central idea of ‘quantitative culture’!
So, there are not two cultures – there is a much more complex field of variation, or confusion (according to whether one is wearing culturalist or rationalist spectacles). And the celebration of cultural diversity is no more appropriate than an insistence on a single ‘scientific method’. We must find a way of rationally appraising methodological ideas and practices that can distinguish between the wisdom of various local cultures and their failings.
Barthold, L. S. (nd) ‘Hans-George Gadamer (1900-2002)’, Internet Encyclopedia of Philosophy. Available at: http://www.iep.utm.edu/gadamer/
Goertz, G. and Mahoney, J. (2012) A Tale of Two Cultures, Princeton, Princeton University Press.
Hammersley, M. (2014) The Limits of Social Science, London, Sage.
King, G., Keohane, R. and Verba, S. (1994) Designing Social Inquiry: Scientific Inference in Qualitative Research, Princeton, Princeton University Press.
Kuhn, T. (1970) The Structure of Scientific Revolutions, Second edition, Chicago, University of Chicago Press.
Mahoney, J. and Goertz, G. (2006) ‘A tale of two cultures: Contrasting quantitative and qualitative research’, Political Analysis, 14, pp227–249.
Oakeshott, M. (1962) Rationalism in Politics, London, Methuen.
Snow C.P. (1998) The Two Cultures, Cambridge, Cambridge University Press
Martyn Hammersley (email@example.com) is Emeritus Professor of Educational and Social Research at The Open University, UK. He has carried out research in the sociology of education and studied the role of the media in reporting research findings. However, much of his work has been concerned with the methodological issues surrounding social science. He has written several books, including (with Paul Atkinson) Ethnography: Principles in Practice (Routledge 1997), (with Roger Gomm and Peter Foster) Case Study Method (Sage 2000), Questioning Qualitative Inquiry (Sage 2008), What is Qualitative Research? (Bloomsbury 2012), and The Limits of Social Science (Sage 2014).