What makes good quality qualitative research for me?

The first readings I had to complete for my first EdD weekend all revolved around the concept of the validity of research. How do we measure (if indeed we can?) what “good” qualitative research is? There were a range of viewpoints, depending on the philosophical and methodological viewpoints of the authors. I could see that each made sense in its own context. However, I didn’t really have any background understanding to critique the ideas adequately, and nor did I then know which way my own research would take me.

Since then, I’ve made the decision to use IPA, and feel comfortable with the relationship between this and my own views on truth and knowledge. I’ve been thinking back as to how I can ensure the quality of the research I am about to undertake. In my quantitative research past, I’ve been concerned over methods, statistics, reproducibility etc. But now, as they say, for something completely different.

Smith and the other IPA stalwarts discuss quality, but in my limited timescale and support for this small scale research, use of other people to determine whether my interpretations are appropriate is not possible. I’ve seen several papers mention Yardley (2000) in relation to quality, so I’ve had a look at her paper. Her thoughts are interesting and relevant, and above all, achievable in my timescale.

Yardley, L. (2000). Dilemmas in qualitative health research. Psychology and Health, 15(2), 215-228. http://dx.doi.org/10.1080/08870440008400302

The suggested reason for the increase in the use of qualitative methodologies (QMs) in health psychology is that it provides a detailed exploration of the experiences of the participant. Yardley clearly identifies that these are true methodologies, with differing underlying assumptions, rather than just a proliferation of methods of data collection and analysis.

The author highlights the reason for the paper as to how to evaluate qualitative research. She suggests that there is a gap in the understanding of how to evaluate qualitative studies in accredited health psychology. Yardley identifies that qualitative research is defined as ‘not quantitative’, rather than the diverse range of qualitative methodologies having a unified definition. Even within a single tradition, such as phenomenology, there is diversity in its assumptions and approaches.

Yardley suggests that it is the very subjectivity allowed by qualitative work that leads to it being used so extensively. However, because meaning is constructed and negotiate between researcher and participant, the imposition of set criteria for truth would restrict the construction of knowledge and could prejudice particular groups who subscribe to alternative criteria for truth. Therefore, a standardised procedure for performing qualitative methodologies cannot be entertained.

There is a concern in the field of health psychology that QMs most closely resembling quantitative methodologies are gaining precedence over other methods because of familiarity in terms of quality and measurements/assessments of validity. Another concern is a tendency to be drawn towards those methodologies providing a framework within which to complete research, leading to isolated methodologies without the flexibility to understand the benefits of other approaches.

This is something which does niggle away at me. I’m not really fully sure why I chose IPA as a methodology. It seemed ‘right’ for the question I wanted to answer. However, I do wonder whether I am drawn to it because, compared to other methodologies, it seems to provide a ‘process’ through which to work. The guidelines are broad and inclusive, but nevertheless, perhaps I am clinging to the set procedures of quantitative research. Perhaps in later work I should explore some in more detail other areas of phenomenology such as van Manen.

Yardley discusses the issue of quality control, suggesting that we cannot apply quantitative standards of quality, such as representative sampling, reliability and replicability, to QMs. These would be unlikely to be achievable and may also not be desirable. She believes that structured coding and interpretation fashioned by rules means that there is a loss of nuance in the rich data provided by QMs. Nevertheless, a means of ascertaining the quality of qualitative research is needed, so that the work can be judged accordingly.

The author identifies what she believes are four characteristics of good quality research, along with some examples of how these can be achieved. The four characteristics are:

1.       Sensitivity to context;

2.       Commitment and rigour;

3.       Transparency and coherence;

4.       Impact and importance.

Yardley suggests these characteristics but stresses that they are not meant to be applied rigidly. Indeed in order to be useable in a range of QMs they must be flexible.

Sensitivity to context:

The author acknowledges the vital aspect of context within qualitative research. However, whilst it is important to have an understanding of related theories and relevant literature, this knowledge must not cloud the researcher’s interpretation of the data. Areas of divergence from theory must be sufficiently explored, as well as those data which do link the specific study to more abstract theories and generalisations discussed in previous research.

The social and cultural context of participants and researchers must also be considered. Contextualisation of findings can shed new light on meanings. The social context of the data gathering needs to be considered including the shared understandings and conversations between researcher and participant and consideration of the researcher’s position in relation to the participant is vital.

A concern for the perceived power imbalance between researcher and participant is required. Whilst some QMs seek the viewpoint of the participants on the researcher’s interpretations to determine the ‘truth’ of the interpretation (perhaps an example of this is the Patricia Hill Collins reading from week 1), it must be remembered that the opinions of the participant should not over-ride the academic independence of the researcher.

Commitment:

Commitment is suggested through the longer-term involvement of the researcher with the topic being researched, as well as an appropriate ability in the methodology of research and immersion within the data.

Rigour:

This is, Yardley suggests, indicated by the completeness of the data analysis and a sufficient amount of data collection. This will vary depending on the chosen methodology – grounded theory will require more participants than IPA. Within phenomenological research, there should be sufficient depth of interpretation, not just surface level analysis.

Transparency and coherence:

This explores the persuasiveness of the research in constructing a reality. Another area to consider to provide coherence is whether the research question and the philosophical underpinnings of the research are appropriate.

Transparency is achieved through thorough documentation of the process of data collection and analysis and provision of rich data, that is, extensive excerpts. Within IPA, it’s my understanding that this is at least in part provided by the tables of themes and annotated transcripts produced as part of the analysis process. Reflexivity is also needed to demonstrate transparency – identifying the experiences and motivations which underlie the research.

Impact and importance:

The impact of the research is a key factor in determining its value. The value may be theoretical but may also have a wider impact within other fields. There may also be a socio-cultural impact to the research. Yardley suggests that an advantage for QMs in health research is their close link to practice. This would, presumably, be similar within educational research.

Yardley concludes by stressing the importance of integrity in QM, whilst taking into account the diversity of these methodologies.

Further reading?

Stern (1997) Strategies for overcoming the rage of rejection. In Morse (ed.), Completing a qualitative project: details and dialogue (pp. 135-145). This covers how to get over editors rejecting work for publication. Its relevance is, perhaps, stressing that you need to clearly demonstrate to the reviewer/reader how you expect your research to be measured against concepts of quality. If you're explicit, there's less room for misunderstanding.

The Wolf Report

This post looks at the Wolf report into vocational education. It's a recent key report into VE, and I should probably have been more aware of its contents than I was. Its content and recommendations aren't probably of specific relevance to my current work, but it's important to have an understanding of the context of the politics surrounding FE. The report stresses the relevance and employers' understanding of long-standing vocational qualifications at level 3, such as the BTEC. Where Wolf feels that students are let down are many of the lower level VQs which do not lead on to further qualifications. However, as an  overview to remind me at a later date, I've put down the key points and spent some time on the context, as I'm lacking in knowledge of the background.

Wolf, A. (2011). Review of vocational education - the Wolf Report. Retrieved from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/180504/DFE-00031-2011.pdf

Purpose: Consider how we can improve VE for 14 to 189 yos and thereby promote successful progression into the labour market and into HE and training routes.

The working definition of VE used: all qualifications, other than GCSEs, A levels, IGCSEs and IB, that incorporate clear vocational content and referencing.

Introduction: There are around 2.5 million 14-19yos, most in full- or part-time education. Education and training programmes should allow young people to progress. There is high youth unemployment. Most 14-19yos take some form of vocational qualification. Almost two thirds of young people don't take A levels - the conventional academic route. Wolf suggests that there is a failing of vocational provision, with no clear progression opportunities. Around 350,000 young people "are poorly served by current arrangements" (p. 21). This is not just a failure of education provision but of changes in labour markets. There is failure of vocational programmes due to constant reorganisations of 14-19 education by central government. She suggests we need a reduction in the centralised management of qualifications and greater involvement of employers, increased efficiency and more relevant qualifications.

The social and labour market context: There has been a reduction in the youth labour market. There are high returns on education and qualifications and high aspirations to HE. There are also high returns to work experience. Rapid economic change is having unpredictable effects on the labour market. All of these impact on 14-19 VQs.

1. Collapse of the youth labour markets: Apprenticeships are a valuable route into employment throughout the EU. The number of young people in employment is low. Those not in education and training face high levels of unemployment. England has very high youth unemployment, up to 25yos. People are pushed into education due to the lack of jobs. even before the 2009 recession, youth unemployment was rising and the number of NEETs increasing.

The nature of the labour market: There is a high level of 'over-qualification' for jobs, but shortages in very specific areas - as demonstrated by the returns for specific qualifications, especially quantitative ones. It is now harder for 16 -17 yos to find employment due to legislation making it more difficult (H&S). The employer perception of 16-17yos leaving education is one of 'low achievers'.

2. High returns to education and qualifications: Education and qualifications 'pay'. Low level VQs have very low returns - they have little or no labour market value (level 2 qualifications). Employers look for qualifications with which they are familiar, rather than trying to keep up with constant qualification reforms.

3. High returns to apprenticeship and employment: Employers value work experience - this explains the high returns associated with apprenticeships. "The best predictor of being employed in the future is being employed now" (p. 34).

4. Occupational change: There is rapid economic and occupational change and a decline in skilled manual jobs.

Implications of change (A): YPs entry into the English labour market:- YP are very likely to change both jobs and occupations in the first years of employment, There is considerable 2churn2 in and out of education and employment.

Implications of change (B): How education systems are adapting - facts and misconceptions:- Most developed countries now delay specialisation to later stages. England pushes for earlier and more complete specialisation in both academic and vocational tracks.

Implications of change (C): Challenges for VE:- VE needs to take into account the varying job histories YP can expect to experience. Needs to take into account aspirations for higher study.

The educational context:
Issues: Young people taking VQs which the labour market does not reward; and also established VQs valued by industry being denied accreditation and funding by the government. YP are being encouraged to take VQs which will reduce opportunities for progression. There are high drop out rates and 'churning'. The funding systems discourage further English/maths courses post-16. There is a marked decrease in returns to post-16 VQs. The causes of these issues are complex.

There has been rapid and repeated change in VE over the past 25 years, for example: new 'non-academic' qualifications such as the Diploma, designed by central government; increasing regulation of qualifications for 14-19yos; redesign of VQs to a specific vocational focus; changes in performance management by central government; apprenticeship reform; changes in funding formulae for 14-19yos; changes/redesign of maths and English, especially for those on VQs.

Key issues A: Key Stage 4: Until recently, VQs were only a small proportion of a 14-15yos timetable. In the 1990s, GNVQs were introduced, at first for post 16s only. Then came GCSE equivalencies for VQs such as BTEC Firsts. The QCA's programme of 'equivalencies' to ensure all qualifications at a particular level are treated as substantively equivalent (and with parity).

Key Issues B: Upper 2nd, age 16-19: No major changes, unlike KS4. Most study for AS and A levels - the 'sixth form' pathway. BTEC awards also attract substantial numbers and are well recognised by HE; these are teacher assessed and awarding body verified.

Part 4: Audit of current provision
In 2009 11.4% of UK applicants accepted for HE entry had BTEC ND and no A levels. A further 1.7% had BTEC national plus A levels. 37.1%  had A levels alone. In 1999 only 4.9 of acceptances had only BTEC nationals. 14-19 education should equip young people to follow different routes successfully and not operate as a tracking system.

1. There is a mismatch between labour market requirements and VE provision. The content of many current VQs is not valued by employers and the labour market. Level 2 courses allow limited progression and lead to churning.
2. The labour market recognises qualifications that are stable and familiar - academic qualifications have been relatively stable. YPs employment patterns imply a need for more general rather than highly specific VQs; however, YP are increasingly being offered only highly specific VQs.

The report discusses the mismatches in VQs and the labour market in detail as well as the role of apprenticeships. There are a large number of recommendations, which the 2015 review returns to.


Need to: Review the Government's response to the Wolf report and also any updates (2015) - for future work rather than of relevance now.

Revisiting Eisner

Eisner was one of my Week 1 readings. As I gradually start to write the assignment, I find myself revisiting Sparkes etc., but have only just returned to Eisner.

See here for the original post.

My thoughts on this second reading are as follows. I've basically summarised each paragraph, in order t0 gather key points.

Eisner starts the paper by giving an overview of what is meant by objectivity - setting the coconuts up for him to later knock down. He discusses the hierarchy of objectivity over subjectivity - something he comes back to later, in relation to our culture and history. After discussing the struggle for objectivity, he sets his argument against it before defining objectivity in relation to ontology, and linking it to the correspondence theory of truth. Eisner brings in Rorty when he talks about there having to be a correspondence in perception, understanding and representation. Rorty is discussed by the use of his analogy of this representation being a mirror to nature.

Procedural objectivity is dismissed by Eisner, suggesting its inhumanity. He then summarises the argument against subjectivity, setting the argument up for attack in the following paragraph. How comes his argument - and how he will knock down the arguments for objectivity. He suggests that we cannot actually know that we know 'the truth'. Just because we can predict, doesn't mean that we know the truth.

It is important to realise that perception is always based on personal frameworks, so cannot be objective. Quote, p. 12: "What we come to see depends upon what we seek, and what we seek depends upon what we know how to say". This is something I'd like to discuss in the assignment.

The way we represent knowledge will also affect how it is perceived; whatever we do, we will conceal as well as reveal. Group perceptions of reality, through schemata, again  mean we cannot be objective. Procedural objectivity only suggests agreement rather than truth.

So, having destroyed the idea of objectivity, what does Eisner suggest?

Historically and culturally, we are programmed to see objectivity as a requirement and higher than subjectivity. He suggests we look for an alternative. Subjectivity is not 'anything goes' but is based on our personal frames of reference, from our own experiences that we make. If there is lack of a common framework, then communication will be impossible.

Eisner suggests that we should not be afraid of the plurality of worlds that his ideas suggest. Our understanding of truth changes as we make it. We can also consider alternative means fro perceiving truth, e.g. through fiction rather than 'science'.

He then starts to summarise his argument, that objectivity is unachievable. Science and the world have changed. We  must look to recognise the plurality of truth, and use reason to determine truth, not rely on correspondence.

My thoughts now:

I can use the quote and its surrounding information to support my view of subjectivity being appropriate and linked to the research paradigm I'm using. I can also bring in the frameworks area and link, perhaps to reflexivity.

I still can't find the original to Philips' work, but having re-read the blog post, it seems that there is a lot of agreement between Philips and Eisner. However, Philips is convinced that objectivity is achievable through peer review and effective procedures in research. The crux of the argument appears to be around the definition of objectivity, with Eisner linking it clearly to truth. Philips, on the other hand, talks about research being objective but not necessarily true. However, this is my interpretation of someone else's view of the paper, so I need to be careful.

It was useful to revisit this paper and to think about how I can apply the information to my research.

Weekend 1 aditional reading: Hermeneutic Listening

This paper related to the session we had on interviewing. This is an interesting concept. Rather than laying aside our prejudices when we talk to an 'other', we acknowledge those prejudices (positive and negative), and work towards a common understanding. We do not need to change others' minds but find areas of common ground.

The notes I scribbled on the text whilst in the session were:
- We bring prejudices with us when we listen to others.
- We need to consider what someone is saying. In an interview, this happens in the moment - we need to listen carefully in order to respond in an appropriate way.

Kimball, S. & Garrison, J. (1996). Hermeneutic listening: An approach to understanding in multicultural situations. Studies in Philosophy and Education, 15, 151-159.

The article starts with a quotation by Gadamer, who was key in the development of hermeneutics. The quotation concerns conversation being a means of understanding. Conversation encourages consideration of others' viewpoints and also highlights new ways of interpreting our own positions. This is helical - consideration of others leads to greater understanding of ourselves, which leads to improved consideration of others, etc.....

The authors home in on multicultural conversations - for multiculturalism to work, there must be respect for others' experiences (and their interpretation of the experiences), and to create new understanding.

The authors warn against both 'passive listening', which can lead to the listener being 'assimilated' by a dominant culture, and also against 'empathetic listening'. The authors prefer a concept of listening as 'the art of interpretation'. Hermeneutics can be used to interpret the 'text' of a conversation to achieve understanding between individuals.

Within hermeneutics, meaning is developed within a particular context, amongst particular participants - the role of hermeneutics is to produce new understanding in both conversants. It is important to understand what a person says rather than understanding the person.

From empathy to ontological hermeneutics

There is, as mentioned, a difference between empathetic listening and hermeneutics. Empathetic listeners acknowledge prejudices and their role in understanding, but claim to be able to put them to one side in order to understand another person as they 'really are'. However, this claim is, according to hermeneutic listeners, very limiting. There is the assumption that prejudice is "bad", but Gadamer shows us that this assumption is incorrect. There has only been this negative connotation to the word since the Enlightenment. Prejudices are just 'pre-judgements' which allow us to get through everyday life.

One must recognise that we are "conditioned by historical circumstances" - we are the product of our experiences, values etc. We cannot eliminate this; to think that we can do so is false. Our participation in different communities (family, school, state, nation, race, gender etc.) shapes how we interpret the world and our pace within it. This is what makes us 'us'.

So, hermeneutic conversation requires us not to rid ourselves of these prejudices but to examine them, and free ourselves of those which hinder our efforts to understand others. It is only when we encounter difference that we can recognise our own prejudices and question them. Then understanding can be developed.

We do not necessarily have to take on the other's views when we examine our prejudices, but change of some sort will occur. "By coming into contact with different beliefs, ...[etc.], we become aware of our own prejudices". If we can interrogate these prejudices in light of the newly perceived alternatives, then new understanding is opened up both with others and in ourselves.

In this way, the authors suggest that the hermeneutic process is compatible with multiculturalism. Empathetic listening is not, as it presupposes that differences between people act as barriers to understanding which must be overcome. Prejudices cannot just be set aside - to suggest as much is to reduce the need for reflection on our pre-judgements. Empathy seeks to 'reproduce' the speaker's original meaning. This way, new understandings cannot be reached. Hermeneutics produces new, multicultural understandings.

Openness

'Openness' is more than being open to what the other means so that the understanding can be reproduced. Instead, the listener is open to new meanings and understanding that are being developed through the conversation. Meanings are produced rather than repeated. Meaning relies on more than one person within a specific socio-historical context having a discussion and jointly assigning meaning.

With regard to multicultural conversations, this openness can be thought of as an "openness to alternative interpretation", including self and culture. We reinterpret our experiences differently after time has passed, and also after encountering new people and cultures. This leads us to new meaning. What is said, what is not said (i.e. omitted), the tone of voice, all reflect our prejudices. By listening to ourselves as we speak to others, we can achieve greater awareness of our identity and what frames it.

How does this differ from empathy? A person committed to hermeneutic listening/understanding acknowledges that each person in the conversation is conditioned by different historical circumstances even if they share the same race, culture, gender, etc. It is important to acknowledge these differences rather than to eradicate them and use our own culturally conditioned prejudices to imagine another person's experiences. By responding with a question providing an analogy to the experience, the comparison may not be exact but new meanings can be made through this exploration.

Fusion of horizons in multicultural conversations

"So far, we have described hermeneutic listening as a continuous, cyclical process by which we become more aware of ourselves as well as the unfamiliar other." This leads to the "mutual creation" of new understandings and meaning. Gadamer calls these points where new understandings are reached as a 2fusion of horizons" - opening up a new horizon of understanding through conversation with others of different beliefs, culture, values, etc. Horizons are ever-moving. Where we enter situations that test our prejudices, our horizons will be redefined.

2The act of understanding involves the fusion of horizons" - through looking at other people's lives we broaden our own horizons. Within hermeneutics, it is important to accept the possibility of tension between different sets of prejudices but we can draw this out to create common understanding, a "social construction" shared by all.

The object is to neither have empathy for not to subordinate the other but to redefine our mutual horizons, whilst still acknowledging difference. This builds a common ground to continue the mutual dialogue.

Why listen?

Why should we go through the process of questioning our prejudices and shifting our horizons? Sometimes, we cannot avoid it. However mostly it is a conscious decision for at least one of three reasons:

1. education is to challenge familiarity with the novel/uncomfortable, to reach new understanding;
2. We can only come to know ourselves through our encounters with others. Otherwise we are unaware of the prejudices which shape our views;
3. when we wish to change ourselves, we need to interact with 'others'. Change in ourselves is a social process.

To sustain conversations with others can lead to new understanding which can be beneficial to our personal growth.

The writers finish by discussing that throughout the paper, their thoughts are underpinned by their own cultural prejudices.

My own thoughts?

Once again, this paper is about having an awareness of the lenses through which we see the world. In interpretivism, it is important not to try to put these lenses to one side. This is impossible. But what we must do is acknowledge that they exist.

Other thoughts: How does this link to Lyotard? The talk about Gadamer here suggests there is the reaching of a mutual understanding. However, if I remember correctly, Lyotardian paralogy goes further than that - moving towards a continuous expansion of knowledge and understanding.

Another thought (irrelevant but I wanted to capture it): when it spoke of change in ourselves being a social process, I was very struck by the way this links to Slimming World. when we wish to change, we need to interact with others. Nothing here to do with multiculturalism, but discussion amongst others to change an individual's mindset.

Interviewees are not automatons

Reading from:

Foddy, W. (1993). Interviews and questionnaires: Theory and practice in social research. Cambridge, United Kingdom: Cambridge University Press.

This information is taken from chapter 2 of this book. The chapter is entitled: A theoretical framework.

This paper links to the session we undertook on questionnaires and interviews. In it, Foddy discusses the issues associated with both a quantitative method of survey and also qualitative. Foddy introduces symbolic interactionism (SI) as a way of social actors in a social situation negotiating a shared definition of the situation.

An overview

Foddy introduces the positivist approach which attempts to discover the 'real world' 'out there'. To do this they use a 'stimulus/response' model of questioning, with carefully controlled questions and also answers. They aim for standardised understanding and standardised responses, to provide validity.

Foddy discusses ten key assumptions associated with positivistic surveys, primarily that the researcher provides clear definitions and the respondent is able to provide appropriate standardised answers in the specific situation, and that answers from different respondents can be meaningfully compared.

Pawson (n.d.) suggests that questions and answers are simplified to a 'lowest common denominator' approach. This is a very behaviourist approach. Control is by the researcher, trying to formulate standardised questions and limiting the respondent to standardised answers.

In contrast, 'qualitative field researchers' are interested in how human beings 'experience' their world, for example through the use of non-directive, open questions. They are committed to understanding the respondent's 'meaning' so use near-to-naturalistic unstructured interviews. The researcher and respondent should come to a joint construction of meaning. The data are narrative. Questions have been asked about he 'validity' of qualitative researchers' data, and the fact that it is difficult to replicate studies.

Within quantitative research, they still cannot control for the respondent not understanding the question 'correctly'. This is equally true for qualitative research, with the respondent often looking to the interviewer to show evidence of arriving at a "shared understanding" of questions and answers.
[Note: may need to consider Lyotardian paralogy here, to go past this shared understanding - see my notes on Lather].

Symbolic Interactionist (SI) theory

SI was coined by Blumer (1967, 1969).
- Humans interpret and define each other's actions; they do not react in a stimulus/response way.
- Humans can be the objects of their own attention - the concept of 'self';
- Conscious social behaviour is intentional. We construct and rehearse different possible lines of action before choosing how to act in a given social situation;
- These are ongoing processes, occurring at every stage of a social interaction. Both parties take part in this. Each social actor takes their view of the other into account but also the other's view of themselves, when constructing and choosing possible lines of action.
- Human intelligence is, in part reflexive, for example when you 'take the role' of the other.

SIs claim that social actors in any social situation are constantly negotiating a shared definition of the situation.

Implications of SI theory for social research

Survey researchers and qualitative field researchers have paid little attention to respondents 'taking the role' of the researcher when framing answers. Similarly the reason for asking the question. SI theory suggests that respondents will constantly try to reach a mutually shared definition of the situation with the researcher. Respondents search for clues if the information they require are not forthcoming. Different respondents may attach to different 'clues' and so differently interpret a question - so there is little reason for comparing respondents' different answers.

There are at least four additional sources of response variability that the researcher should keep in mind when formulating questions. Different respondents can interpret the same question in many different ways and give many different answers to it.

My thoughts

In the questionnaire exercise in the first weekend, it was clear to see that including free text answers in such a questionnaire led to a wide range of interpretations of the question. The questionnaire wasn't designed particularly well, but even so the relatively straightforward questions were open to different interpretations. It's probable that the same respondent could answer differently at a different time. So, quantitative work is subject to this issue.

The SI viewpoint seems to be to accept/embrace this multitude of potential answers. The participant will take their cues from the researcher and respond to the researcher to try and reach concordance.

Relevance to me? Again, I think this highlights the importance of thinking about my own frames of reference and being aware of them throughout an interview process, from writing the questions to the questioning itself and then on to the analysis. Just to be aware that the respondent is not only responding to the question I ask but more particularly the question they think I asked, and looking to me to help them provide the answer they think I want. Difficult!

 

Bringing together the week 1 reading

Even before the weekend, I'd started to get an understanding of why we'd been given the articles to read. They all revolve around the idea of validity. How do we know that research is good research? Do we actually need to know this?

Different paradigms raise different interpretations of what is meant by 'good' research and how this can be determined.

At the weekend session, each group was given a paper to discuss. we ended up with Lather, which was rather mind-blowing.

What was said about each author?

Hammersley: A post-empiricist, believing in 'subtle realism'. He believes that rigorously conducted method is a guarantor of truth, from a scientific viewpoint. "Validity is a synonym of truth and method its guarantor" (p. 69). This is the correspondence theory of truth. For Hammersley, research is science.

Smith: Outlines three perspectives that challenge the dominant view. He suggests that all judgements are based on practical, ethical issues - we are all post-foundationalists. We make judgements based on social interaction. The criteria aren't fixed in advance (compare to Hammersley). Think about how you judge art. You engage with the piece - you ask questions of it, it asks questions of you - a "fusion of horizons" - dialectical criteria.

Collins: Structures in society perpetuate the dominant truth. Concrete experience is required.

Lather: Not quite 'anything goes' but suggests a more open view of what can be used to consider validity. Be aware that even in alternative views of validity, the shadowy hand of positivism remains.
 Use of reflexive poetry, reflecting on participants' feedback. All needs t be considered. And maybe all's been done before...Erasmus Darwin  - scientific prose in poetry! It seems that, although Lather suggested that some of her ideas were undoubtedly ephemeral; nevertheless, some seem to have ensured over 20 years... http://www.tandfonline.com/doi/abs/10.1080/08893675.2015.1051293

Week 1 reading - heaven knows, (almost) anything goes!

Lather, P. (1993). Fertile obsession: Validity after poststructuralism. The Sociological Quarterly, 34(4), 673-693.

This is the final piece we were asked to read before the first EdD weekend. This was, quite possibly, the most difficult work to read. I've read and re-read it and still can barely get my head around the concepts. There may be some interpretations of her work that make it clearer to a beginner in this area - I'll have to have a look. Usual caveats apply with this overview of the article - there's a lot of direct material and little interpretation. I will need to go back to the original article if I want to use anything at a later date :-/

(Note: maybe have a read through this: https://books.google.co.uk/books?id=Nf_FBQAAQBAJ&pg=PT64&lpg=PT64&dq=what+is+%22antifoundational+discourse+theory%22&source=bl&ots=TOjElHemfF&sig=JXGS-k2qOQECK6kQwxJTIVqNk9Q&hl=en&sa=X&ved=0CC8Q6AEwA2oVChMI6Mab1v7IyAIVg4UaCh2sFgZn#v=onepage&q=what%20is%20%22antifoundational%20discourse%20theory%22&f=false)

Lather suggests that, at the time of the article, everything is up for discussion, to develop a new way of thinking or rather new ways. She defines validity as "the conditions of the legitimisation of knowledge in contemporary postpositivism" (p. 673).  "Rethinking validity in light of antifoundational discourse theory". She wants to continue to use the word "validity" (with all its associated baggage) but to use it in a more subversive way.

Lather is a poststructuralist feminist and suggests that out of the current void in terms of validity will come innovative ways of imagining it, powered by practice. She muses on the development of validity criteria which are sensitive to context. She suggests that even within the realms of postpositivism, the positivist code still maintains its grip. Think differently is her watchword!

The masks of methodology

Lather suggests we need to see "...what frames out seeing." (p.675). A range of methods have been used to try to resolve the issue of validity, such as member checks, peer review and triangulation, but ultimately these raise more questions than answers. "...to not revert to the dominant, foundational formulaic and readily available codes of validity requires the invention counter discourses/practices of legitimisation" (p. 676).

Transgressive validity

"What do you do with validity once you've met poststructuralism?" Lather provides four "framings" of validity that take antifoundational discourse theory into account.

Counter-practices of authority

Lather introduces a "dispersed validity" (p. 677) suggesting alternative ways of considering validity. She fights against the ideas of "policing sociological sciences" and the development of another "regime of truth". She suggests that her ideas are ephemeral, to open us up to possibilities.

Frame 1: Validity as simulacra/ironic validity
Baudrillard suggests that we have "shifted from a culture of representations to one of simulacra. Neither original nor copy". Ironic validity acknowledges that text will always fail to represent what it points towards but can never reach. Her example (Agee & Evans, 1988) looks at the struggle of an "I" to become an "eye" that both inscribes and interrupt normalising power/knowledge.

Frame 2: Lyotardian paralogy/neopragmatic validity
Legitimising research through 'paralogy' does away with the "Habermasian drive for consensus". Having briefly read about this, my understanding is that Habermas saw discussion as ultimately leading to consensus. However, Lyotard, whilst acknowledging that consensus was one stage of discussion, but ultimately it should lead to 'paralogy' - the development of ideas through conversation which consequently leads to an unending development of new ideas. Lather uses Woodbrooks' (1991) study of African-American women in leadership positions in HE. Woodbrooks used two standard methods of validity in qualitative research - member checks and peer debriefing. After reflecting on the feedback from the participants, she amended her work and sent a second draft to the participants. What resulted was the juxtaposition of the voice of the white female researcher with those of African-American female participants. She held her own frame/lens up for scrutiny and her privilege as a middle-class white female was identified. She had missed out on key aspects of the African-American female experience.

Frame 3: Derridean rigour/rhizomatic validity
Deleuze and Guattari (1983) suggest that the tree is a modernist model of knowledge, but the rhizome as the model for post-modernist knowledge. Arbitrary branching systems of knowledge - networks.

Frame 4: Voluptuous validity/Situated validity
Lather suggests that authority "comes from engagement and self-reflexivity, not distanced 'objectivity'". She discusses Richardson's (1992) essay on a study conducted on unmarried mothers. She writes sociological research as poetry - a "disruptive excess".

Lather suggests that her offerings are given as "more problem than solution" (p. 683). Within the gap offered from the death of positivism lies potential.

My thoughts

I suppose, from a gardening viewpoint, if you pull up a well-established plant which is no longer of use or performing as it should, it opens up clear ground. On the clear ground, seeds can germinate and grow. Not all will reach maturity but in the mean time they are valid competitors which have the potential to grow to maturity.

I can't imagine using these methods of validating research at the moment, but maybe this is because they are so distant from my comfort zone. Of greatest interest out of the four is that of Lyotardian paralogy. It makes sense that conversation should not need to stop at consensus but continue, to see where it does lead.

Actions based on this reading:

• Maybe read about Lyotard some more

 

Weekend 1 reading: Patricia Hill Collins - afrocentric feminist epistemology

Collins, P. H. (1991). Towards an afrocentric feminist epistemology. In: P. H. Collins (ed.). Black feminist thought. London, United Kingdom: Routledge.

Black feminist thought, like all specialised thought, reflects the interests and standpoints of its creators. Tracing the origin and diffusion of any body of specialised thought reveals its affinity to the power of the group that created it. Because elite white men and their representatives control structures of knowledge validation, white male interests pervade the thematic content of traditional scholarship. As a result, black females' experiences with work, family, motherhood, political activism and sexual politics have been routinely distorted in or excluded from traditional academic discourse.

"Black feminist thought as specialised thought reflects the thematic content of African-American women's experiences. Subordinate groups have to use alternative ways to create independent self-definitions and self-valuations. African-American women have developed a distinctive Black female's standpoint and have done so by using alternative ways of producing and validating knowledge.

"Epistemology: the study of the philosophical problems in concepts of knowledge and truth."
To investigate this work, Collins consulted established bodies of academic research but also her own experiences and those of other Af-Am females - this is illustrated by the use of the word 2our" rather than "their" within the research. Few statistics are reported; instead. the voices of Black women are used. This allows Collins to explore the thematic content of Black feminist work in a way that doesn't violate its basic epistemological framework.

Black women intellectuals often encounter two distinct epistemologies - elite white male interests, and Afrocentric feminist concerns.

Eurocentric masculinist knowledge validation process

The Eurocentric masculinist knowledge validation process: the institutions, paradigms and other elements of the knowledge validation process controlled by elite white men.
Two political criteria influence the knowledge validation process:

1. Knowledge claims are evaluated by a community of experts whose members represent the standpoints of the groups from which they originate;
2. Each community of experts must maintain its credibility as defined by the larger group in which it is situated and from which it draws its taken-for-granted knowledge.

Therefore scholars challenging the basic beliefs of the culture at large are deemed less credible than those supporting popular perspectives. This has suppressed Black feminist thought. Accepting a few "safe" outsiders legitimises keeping outsiders out. Those who challenge the taken-for-granted assumptions run the risk of being ostracised.

Af-Am female academics face potential rejection of knowledge claims on epistemological grounds (theories of knowledge). Rather than using Eurocentric, masculinist criteria for assessing knowledge, they may use other criteria. These methods of validating knowledge claims must be acceptable to the group controlling the knowledge validation process.

For example, positivist approaches of validating knowledge do so by producing objective generalisations, distanced from values, interests and emotions of the researcher. Positivist methodological approaches:

1. Distance the researcher from the 'object' of study;
2. Show an absence of emotions;
3. Ethics and values are deemed inappropriate;
4. Adversarial debates are the preferred method of ascertaining truth.

For these reasons, it's unlikely that Black women would use a positivist epistemological stance in rearticulating a Black female standpoint. Back females are more likely to choose an alternative epistemology, using different standards consistent with Black females' criteria for substantiated knowledge.

So, what does this look like? The core African value system that existed prior to racial oppression, and also a sharing of the common experience of oppression leads to a distinctive Afrocentric epistemology. This is similar to the feminist advance of a history of gender oppression which led to the feminist epistemology. There is an alternative epistemology for Back women - an Afrocentric feminist viewpoint. There is a discussion of the overlap between the feminist and Afrocentric standpoints - with Black women belong to 2both/and" - there is some overlap but also some separate, in addition. The Afrocentric feminist epistemology is rooted in the everyday experiences of AfAm women.

What are the dimensions of an Afrocentric feminist epistemology?

1. Concrete experience as a criterion of meaning

AfAs use wisdom to assess knowledge, and an understanding of their oppression through race, gender and probably class. "Knowledge without wisdom is adequate for the powerful but wisdom is essential to the survival of the subordinate". Those individuals who have lived through the experience about which they claim to be experts are more believable and credible than those who have merely read or thought about such experiences; i.e. the 'lived experience'.

Some scholars claim that women, too, as a group are more likely than men to use concrete knowledge in assessing knowledge claims. Therefore, in valuing the concrete, AfAMFems invoke not only an Afrocentric tradition but a woman's tradition as well. There is considerable institutional support in traditional AfAm communities for valuing concrete experience - sharing it through the 2sisterhood".

2. The use of dialogue in assessing knowledge claims

"Dialogue is a talk between two subjects, not speech of subject and object - it is a humanising speech, one that challenges and resists domination."
Through dialogue, connectedness rather than separation is an essential component of the knowledge validation process. The use of dialogue has Afrocentric roots, seeking harmony. Different to adversarial debate, it has its basis in African-based oral tradition. The use of call-and-response discourse needs the active participation of all. For ideas to be tested and validated, everyone in the group must participate. Black female centrality in families and community organisations provides AfAm women with a high degree of support for invoking dialogue as a dimension of an Afrocentric feminist epistemology. This ties in with a female way of knowing.

3. The ethic of caring

"Talking with the heart" taps the ethic of caring - another dimension of an alternative epistemology used by AfAm females.

• Emphasis on individual uniqueness, rooted in a tradition of African humanism;
• Use of emotions in dialogues - emotion shows a speaker believes in the validity of an argument;
• capacity for empathy.

There is growing evidence that the ethic of caring may be part of women's' experience as well.

4. The ethic of personal accountability

People are expected to be accountable for their knowledge claims. AfAms believe it is essential for individuals to have personal positions on issues and assume full responsibility for arguing their validity. Knowledge claims made by individuals respected for their moral and ethical connections to their ideas will carry more weight than those offered by less respected figures.

It seems (with limited research) that there is a female model for moral development, therefore there is another convergence between Afrocentric and feminist institutions.

"An alternative epistemology challenges...knowledge an opens up the question of whether what has been taken to be true can stand the test of alternative ways of validating truth".

My thoughts

Very interesting. I've heard talk of intersectionality on Twitter, and not really understood it (or taken much notice of it) until now. It seems clear to me that experiences and understandings will vary. What a middle class white woman conceives of as their feminist viewpoint will be different, though no more or less valid, than that of a Black woman. However, the Black/transgendered/with impairments will have additional barriers of oppression. It was fascinating to learn of different ways in which research can be viewed as 'valid', based on cultural roots. It's obvious that this should be so, but my own view has been blinkered by the standard white Eurocentric masculinist approach that it is very hard to conceive of alternative voices towards validity, as suggested here. Something to read  more about. It's unlikely to be something I develop, but it has made me more aware of the frames within which I operate, as being a privileged, white, cis-, able-bodied woman.

If I had to categorise Collins, it would be as a critical theorist, with one key concern being from a feminist viewpoint, though of course taking in other oppressed sections of society, particularly that of African-Americans. As a CT, she would espouse 'transformation'.

Weekend 1 reading: Smith and the problem of criteria

Smith, J. K. (1993). The problem of criteria. In: J. K. Smith (ed.). After the demise of empiricism. The problem of judging social and educational enquiry. Norwood, NJ: Ablex.

This is another of the weekend 1 readings. Smith is an interpretivist. Again, as I don't really yet understand the topic, a lot of this is verbatim and will need to be paraphrased or quoted if I use it in work.

PEs, CTs and interpretivists can all agree:

• There's no definitive way to distinguish knowledge from opinion or from false claims;
• There's no definitive way to sift good research from bad.

All three paradigms agree that the empirical theory of knowledge is undermined by the fact that theory-free observation is impossible, and so on.

The end of the common ground

However, as the three theories have developed, so have major differences in their positions. The differences lie in the depth to which the PE ideas of theory-free knowledge, abandonment of direct realism and collapse of fact-value and subject-object distinctions are taken.

PEs: To develop a "modified" version of empiricism. They are unwilling to abandon completely the concepts of truth and objectivity as ideals. They suggest that, without these ideals, it is impossible to determine knowledge from opinion and good research from bad. They view the task as to modify empiricism rather than abandon it.

PEs have sought to develop the philosophical doctrine of realism, from 'naïve/direct' to 'sophisticated/indirect' realism. They also argue for the importance of certain values key to the scientific process, e.g. being guided by the 'search for truth'. They realise that certitude is impossible but one isn't railroaded into relativism; just because research cannot definitively be judged on its quality does not mean that some research is not more valid than others. "They are looking for a non-foundational theory of knowledge" between certitude and 'anything goes relativism' and between definitive criteria for judgement and no criteria at all.

CTs have attempted to develop an alternative foundation for knowledge, with alternative conceptions for truth, objectivity etc. Within CT, the ideal is to make transparent the historical socio-political forces that have led to false consciousness, distorted communication, etc. The knowledge must then be put to use so that the exploited can transform themselves and society. "CTs make no pretence that their theory is neutral or divorced from practical action".

The key task for CTs has been to develop a normative foundation for their position. Doing this, they've adopted a critical realism, interpreted objectivity and truth within a historical viewpoint, elaborated a theory of communicative action and rationality, etc. They do this knowing that certitude is impossible but that relativism  is unacceptable. They realise that judging research definitively is impossible but state that some research is more objective and valid than others. Their task has been to establish alternative criteria, alternative to empiricism, post-empiricism and interpretivism.

Interpretivists are guided by human solidarity. hey heavily discuss the 'no theory-free knowledge' concept, the collapse of subject/object dualism, the end of the distinction between facts and values, etc. They push further than PEs and CTs - they redefine traditional concepts such as subjective, objective, truth, relativism etc in non-epistemological terms.

The key position of interpretivists is that researchers do not have a unique hold on knowledge. We cannot judge, in a foundational or epistemological sense, knowledge against opinion. The best we can do is to describe forms of justification for knowledge that are common to a particular group or society at a particular time. Any judgements must be framed by practices and moral concerns, not epistemological ones.

Interpretivists believe that researchers are not privy to privileged knowledge about social life. A research study is thought of as another narrative account of our social lives, alongside other (research and lay) narrative accounts.

However, that is not to say that they believe all research is equally important or justified - they're not 'anything goes' relativists. However, this judgement is not made as to whether some research is more objective (giving an accurate depiction) but because some accounts make sense to us given our interests at this time and place. Judgements we make about knowledge against opinion and good against bad research are practical and moral tasks, not epistemological ones. Interpretivists must elaborate what lies beyond epistemology and beyond the thought that there are special abstract criteria for judging research quality.

1. The need to fill the vacuum left by empiricism with another theory of knowledge: PEs and CTs agree with this; we must be able to say something special about genuine knowledge against false claims and about good against bad research. Good research is defined in terms of objectivity. Interps do not desire to develop a theory of knowledge and believe that there is no vacuum to fill. The absence of a theory of knowledge does not mean that all claims to knowledge and all research are equal, but not better because they are more objective but what we can agree on at any given time/place.
2. Theories of knowledge differ between PEs and CTs. PEs want modified empiricism but CTs want an alternative basis for knowledge. Both have adopted realism and make claims to objectivity but have different concepts of these definitions. PEs see objectivity as true to reality as it exists. CTs seem objectivity in terms of historical distortions that have led to the present false consciousness. PEs believe that objectivity is a matter of detachment. CTs believe objectivity can only be understood in terms of commitment to the emancipation and empowerment of those who are unaware of the reality of their situations.

These differences are clearest when it comes to the interpretation of meaning, i.e. in hermeneutics.

PE make a distinction between meaning and significance. Meaning has independent existence and can be known (in principle if not in practice) at any given moment, as it actually is, separate from the interests and purposes of the interpreter. An objective account of what an author meant is one that has accurately captured that meaning. Meaning is given this status as an external referent point against which to assess interpretation; therefore we can assess the extent to which an interpreter has got it right or wrong.

Making this interpretation is not arbitrary. Using the regulative ideal of objectivity the author makes a hypothesis with a systematic approach. The extent to which the interpreter applies data collection and analysis can be judged. Furthermore, they explain how their subjectivities are made explicit and how discomfiting evidence was sought.

CTs make good on their versions of realism and objectivity with a critical or depth hermeneutics. Interpretation doesn't mean an accurate representation of the author's meaning, because the author may be mistaken. An objective interpretation points out to the victim of false consciousness the reasons for their illusions/distorted communication. In other words, an interpreter must depict what an author meant and also assess that meaning in light of objective historical conditions. This will lead to true meaning and the possibility that empowerment and emancipation will follow. Therefore, good critical research is that in which the researcher has been reflexive, clarified historical conditions, collaborated with those studied and educated meaning as to the true nature of their conditions.

Interps don't see hermeneutics in terms of theories, as they are not interested in theories of knowledge. They believe meaning only comes into being as a result of dialogue between the interpreter and that to be interpreted. There is therefore no objective or 'right' interpretation. However, this doesn't mean that there is no good or bad research - one interpretation is often agreed to be better than another. All one can do to convince another to accept their interpretation through providing materials that best support it. Interpretivists see criteria not as abstract standards but as open-ended evolving lists of traits that characterise what we think research should do and be like.

Criteria for judging social and educational research

PEs and CTs, based on their interest in a theory of knowledge, have both attempted to establish some form of abstract, general and universal criteria for distinguishing the quality of research.

PEs: Their realism leads them to judge that research which more accurately depicts reality (ie is more objective) from less accurate depictions (less objective). Whilst mistakes can be made, over time, with more research and analysis, more accurate judgements can be made. PEs differentiate between valid research and important/relevant research - this has been carried over from empiricism. Research can be of no practical or theoretical importance but can be valid through employing appropriate data collection and analysis and through thorough research. A view of importance is, necessarily, more subjective.

To CTs, good research must be both theoretically well developed and lead to emancipation and empowerment (i.e practically based). These are addressed through validity. Judgement is the extent to which the researcher accurately captures how people interpret their own expressions and those of others. This also needs to be understood in terms of the historical conditions that have led to these interpretations. Finally, good critical research must have catalytic validity - it must lead to or inspire practical action. Because of this, CTs are less likely to make a distinction between good research and important research. A researcher could objectively depict a person's interpretation and determine that the interpretation is mistaken or an illusion but not inspire people to act on that knowledge; therefore meeting the criterion of theoretical success but not that of practical agency, so it is invalid and unimportant.

Interpretivists: As the criteria problem is not an epistemological problem, criteria are not conceptualised as abstract standards. Instead, they are characterising traits. To argue that some research is good is to advance an argument for the particular traits that you think should characterise that study. Therefore decisions about the quality of an interpretive study draw on exemplars of that research tradition, involve judgemental interpretation and are a practical-moral affair.

Practical consequences

Within empiricism it was thought (in principle) that an objectivity of judgement about quality of research was possible. However, in reality, this was only ever loose and imprecise. PEs believe that although mistakes in judgement of research do occur, in the long haul, it will if it *is* good research, come to the forefront. This has implications for peer review of research journal articles.

P. 158: "It appears that there is little prospect for universal criteria to distinguish good research from bad" therefore there's little sense in the notion of a community of all researchers.

Potential concern related to the problem of criteria is the status of social and educational research/ers - no longer such as 'special or privileged knowledge', although it does still occur in PEs and CTs because of their self-reflection, openness to criticism, systematicness in thinking than non-researchers. Interpretivists shy away from this idea of a 'hierarchy of knowledge'. In the absence of a theory of knowledge there is no point worrying about a hierarchy of knowledge. There's no special knowledge about knowing and therefore no special, abstract way of distinguishing knowledge from opinion and good from bad research. Research is therefore knowledge that is different from, and expressed differently from the knowledge and language of the lay person. However, it is not superior knowledge - research is another voice in the conversation.

Summary

If PEs and CTs are correct that there is a theory of knowledge to replace the empiricist theory of knowledge, then there are abstract criteria/standards for distinguishing knowledge from belief and good from bad research. These criteria are loose and imprecise but are sufficient for the task of distinguishing good/bad research. Therefore these criteria can also serve as the basis for the claim that research knowledge is superior to other forms of knowledge - because it can be judged.

If there is no theory of knowledge (I.e. interpretivism) then there are no (epistemological) criteria for distinguishing good research from bad and knowledge from belief. Instead, there are characterising traits which are expressions of our values. The attempt to distinguish knowledge from opinion and good research from bad is a practical and moral task, not an epistemological one.

What  next? Questions to answer.

PEs: Standing between the correspondence theory of truth and the coherence theory of truth, they have yet to come up with a convincing theory which draws from both. They also acknowledge their uncertainty about the role of empirical evidence in theory choice in light of there being no theory-free observation.

CTs: Can they justify that their critique is more than just another possibility (amongst many others) for understanding the social and educational world? Is their critique of ideology only another ideology? Have *they* escaped the historical distortions and false consciousness that everyone else is enveloped within?

Interps: Have they actually moved beyond a theory of knowledge, or just side-stepped it? "A judgement that a research study is good is never more than an expression of the fact that people, guided by the ideal of human solidarity, have come to agreement about the quality of that study".

What this means???

There's no agreement on the criteria that are used to consider what is good/bad research. Different groups use different means or even believe that there is no clear way except through agreement on characteristic traits at that moment in time.

CTs believe that change should occur. PEs do not seem that different from empiricists.

More discussions on criteria for assessing research... Hammersley

... and more week 1 reading. I think even I am beginning to see the theme in this early reading.

Hammersley, M. (1992). what's wrong with ethnography? London, United Kingdom: Routledge. (Chapter 4: By what criteria should ethnographic research be judged?)

Hammersley introduces the main issue with ethnography in social sciences which is, which criteria should be used to assess ethnographic studies. He suggests that there are three basic position:

1. we should use the same criteria as are used by quantitative researchers. In this way, ethnography is one of many methods and is not associated with a distinctive methodological philosophy.
2. Most believe that ethnography is within an alternative paradigm to positivism - a methodology which is more appropriate to the study of the social world than the quantitative methodology of the physical sciences. Therefore the criteria for judging ethnography will differ to those used by quantitative researchers. There is little agreement as to what "appropriate criteria" are.
3. Some researchers believe that there can be no criteria that can be used to judge qualitative research - assessing research through criteria is incompatible with the nature of the social world and how we understand it.

Hammersley starts by looking at viewpoint three:

The rejection of criteria

Smith (1984) believes that establishing 'non-arbitrary' criteria for ethnographic research is marked by inconsistency and confusion. With the qualitative tradition being 'idealist', the definition of 'validity' is one of agreement on an interpretation. Because we're unable to have certain knowledge of an independent truth the correspondence theory of reality is not appropriate. This undermines the idea that there exist criteria by which we should judge research. "To accept that social reality is mind-constructed and that there are multiple realities is to deny that there are any 'givens' upon which to found knowledge. If one accepts these assumptions, different claims about reality result not from incorrect procedures but may simply be a case of one investigator's interpretation of reality against another's."

Hammersley unpicks Smith' argument:

1. The claim that assessment of ethnographic research requires judgement, so there is always the potential for disagreement about the application of proposed criteria;
2. The argument that there are no criteria whose validity is certain, so that assessing claims leads to the possibility of disagreement and potential change;
3. The suggestion that there are no criteria in the sense of judgements that assume the reality of the phenomena studied to be independent of the researcher. Instead, the considerations in terms of which assessment is and should be made just refer to very particular, historically-located forms of social practice. The only legitimacy is that they belong to such a practice and that they are agreed on by those who engage in that practice.

Hammersley agrees with Smith's first two points - there are no criteria whose validity is a given. Applying criteria requires JUDGEMENT and the criteria are open to challenge and reformulation. Hammersley doesn't accept Smith's third argument. He disagrees with Smith's anti-realist position of rejecting that our knowledge can correspond to the phenomena it is intended to represent and that achieving such correspondence is one of the aims of inquiry: Hammersley develops his refutation of Smith's final point by affirming that we have no way of determining the certainty of our understanding of the world, but that this does not necessarily mean the rejection of the truth as correspondence.

This lack of certainty also fails to undermine the idea that some methods are more effective than others in producing knowledge of reality. It similarly doesn't counter the thought that there are criteria that we can use to judge empirical claims. All of these suppositions only follow (as per Smith's argument) if, when we discuss knowledge, it can only be known with ABSOLUTE CERTAINTY, that to be of any value, the methods used must ENSURE true findings and that the criteria must produce assessments that are BEYOND ALL POSSIBLE DOUBT.

But these assumptions are effectively irrelevant - Smith suggests you can only have realism (naïve realism) or idealism. However, there is a wider range of positions than this polar view. Many of these positions are philosophically defensible even though none can be established beyond all doubt.

Smith defines truth (within the interpretive tradition) as 'what is agreed upon' ; Hammersley states that this inhibits any rational discussion. He also believes that it is logically impossible and subject to a circular definition. He also suggests that Smith falsely suggests that choice of paradigm is down to personal preference - quantitative paradigms have criteria, but if you prefer the interpretive paradigm, you cannot have foundational criteria. Hammersley suggests that criteria for assessment should be applied but should be HEURISTIC and subject to debate.

Ethnographic criteria

Some researchers (e.g. Lofland & Lofland, 1984) suggest criteria for assessment, for the 'generic' style of ethnography. He suggests looking for several of the following features:

• a GENERIC conceptual framework, applicable to a wide range of social phenomena. Can the research be applied to other areas/relationships or is it specific to one location/social phenomenon?
• Is the framework NOVEL?
• Is the framework ELABORATED? Is it more than a skeleton frame? rather, does it draw out implications and show major variations?
• Is the framework 'EVENTFUL'? Is it illustrated by a richness of events and concrete episodes?
• The framework should be interpenetrated with the empirical materials - i.e. both theoretical framework and the empirical account enrich each other.

This generic style differs from very specific ethnographic studies which do not produce generalisations, or contain generalised description rather than detailed empirical data.

Athens (1984) produced a similar set of criteria for assessment:

• Does it generate formal theory? refining existing theory is less valuable. Formal theory relates to Lofland's 'generic' theory.
• Is it empirically grounded? Are the theoretical concepts consistent with empirical observations from which they are derived? Observations need to be included in the report so the reader can make this assessment.
• Is it scientifically credible? Is an account of the research process included?

Only this final criterion seems distinct from Lofland's.

Hammersley discusses another attempt to specify criteria for judging ethnographic research, that of Guba and Lincoln (1981 and 1985). They believe that the NATURALISTIC approach is needed for studying human social life. They believe that the four major traditional criteria (truth value, applicability, consistency and neutrality) can be used but must be formulated in a way that differs from the rationalist paradigm, as follows:

1. Truth value: concerned with credibility; do the people studied find the account produced to be true?
2. Applicability is related to transferability; although naturalists reject 'generalisability', they believe that there can be some transferability, if enough 'thick descriptive' can be produced.
3. Consistency equates to dependability; because of the emergent nature of research design in naturalistic research, replication is impossible.
4. Neutrality takes the form of 'confirmability'; is the analysis 'grounded in the data'? Are any inferences based on the data logical and of high utility?

This viewpoint adds to Lofland and to Athens. Hammersley synthesises these to bring together a new list:

1. the degree to which generic/formal theory is generated;
2. the degree of development of the theory;
3. the novelty of the claims made;
4. the consistency of the claims with empirical observation and the inclusion of representative examples of the latter in the report;
5. the credibility of the account to readers and/or to those studied;
6. the extent to which the findings are transferable to other settings;
7. the reflexivity of the account: the degree to which the effects of the findings of the researcher and of the research strategies employed are assessed and/or the amount of information about the research process that is provided to readers.

Hammersley suggests that there are questions that need to be asked about these criteria:

1. are they applied to all ethnographic research? For example, should all ethnographic research be concerned with developing formal theory? *Er, No!* We need to modify the application of criteria according to the intended product of the research.
2. Credibility shouldn't be define as whether readers or people studied judge the account to be true. The researched may not want to  acknowledge the truth. The studied do not have privileged access to the truth. The responses to the ethnographic accounts are useful data sources but agreement with these accounts should not be a criterion for assessing research. If it were, bland, agreeable results would be produced.

Applying quantitative criteria

Evaluation for quantitative methods centre on:

• internal and external validity
• reliability and validity.

The first subsumes the second.

Hammersley suggests that application of these criteria to ethnography is unsound, even to quantitative research. Campbell's development of the concepts of internal and external validity takes the 'quasi-experiment' as its research model, but it's been used for other research. The fcus is on designing research to rule out threats to validity.

INTERNAL VALIDITY: whether, in the specific experiment, manipulation of the treatmentproduced variation in the outcome.

EXTERNAL VALIDITY: whether a relationship discovered in an experiment can be generalised to other situations.

You can have internal validity without external validity.

Hammersley believes that this talk of different types of validity is misleading. The findings of a study are either valid or not - they can't be valid in one sense but not in another.

The other framework from quantitative methodology that is often applied to ethnography is the distinction between reliability and validity. There are ambiguities in the definitions.

VALIDITY: Accuracy with which a description of particular events represents the theoretical category that it is intended to represent and captures the relevant features of those events.

RELIABILITY: The degree of consistency with which instances are assigned to the same category by different observers or the same observer on different occasions. It provides evidence about validity and tells us about the usefulness of the particular strategy used.

Validity and reliability are appropriate for judging methods and products of research, but they are not sufficient. We need to consider alternatives for internal/external validity and the question of relevance.

A reformulation of the criteria

Some of what Guba and Lincoln identify as criteria are means by which the validity of qualitative research may be assessed, rather than criteria themselves - definitely 'credibility', 'consistency', 'neutrality' and possibly 'transferability'.

Hammersley concentrates on criteria of assessment and standards by which research results should be assessed. we need to first consider: what is the PURPOSE of such research? What goal is it intended to serve? Hammersley suggests that research is "to provide information and relevant to some legitimate public concern". Based on this definition, we need to judge on two criteria: TRUTH (validity) and RELEVANCE. These apply to both qualitative and quantitative research.

The importance of validity is obvious, if not unproblematic. However, relevance is important, including it being of importance to those outside the research community. Sound research requires the criteria of validity and relevance.

Validity

Hammersley suggests that he is a 'subtle' realist, using validity as a synonym for 'truth'. "An account is valid/true if it accurately represents those features of the phenomena that it is intended to describe, explain or theorise". This assumes a correspondence theory of truth. Since we cannot know the extent to which an account is true, we have to judge validity on the basis of the adequacy of evidence used to support them. We must recognise that judgements on the truth of knowledge claims rely on assumptions, many of which we are not consciously aware of, and many of which have not been subjected to rigorous testing.

We must be aware that potentially we need to keep challenging each new piece of evidence and so on ad infinitum, so where does this stop? we make this sort of judgement on a daily basis in real life, taking into account such considerations as: judgements about what is beyond reasonable doubt; likely costs of error; the scope for acquiring further evidence before a decision has to be made; the likely value of that evidence; the costs of getting this evidence; and so on...

Limits on time and resources as well as an understanding of 'beyond reasonable doubt' will set the bar for sufficiency of evidence. Hammersley suggests there are three important considerations with regard to sufficiency of evidence:

1. Issues of plausibility/credibility: are the claims made sufficiently plausible, given our existing knowledge? If so, accept them. If not, is the claim reasonably likely to be accurate (i.e. is it credible), given what we know about the circumstances of the research? researchers must ensure their findings are sufficiently plausible/credible including through anticipating the likely judgements of fellow researchers. They must provide sufficient evidence to counter any concerns over the research.
2. The evidence should relate to the level of centrality of the claim to the researcher's argument. A claim that is central to the researcher's argument will require more convincing evidence than a more marginal claim.
3. The type of claim made: we need to distinguish between: definitions; descriptions (claims about what happened at a particular time/location; explanations; theory (relationships between different types of phenomena, wherever instances of those types occur). The validity of claims for supporting theories is more complex than that of descriptions, therefore needs more evidence, via descriptions of a range of examples. We'd also need evidence to show any confounding factors could not cause the link. This requires the use of a variety of sources of information.

Hammersley suggests that the assessment of validity involves identifying the main claims made by a study, noting the types of claims these represent, then comparing the evidence provided with each claim with what is judged to be necessary, given the claim's plausibility and credibility.

Relevance

Work should not only "just" provide 'truth' but also be judged in terms of its relevance. But, relevant to what/whom? - A limted audience, generally. We need to consider the 'audience': bear in mind the researcher's own specialisations of topic but also different methodologies and theories. Hammersley believes that the primary emphasis should be on substantive relevance.

Relevance to researchers can be considered through:

1. Importance of the topic: the centrality of the topic studied to a substantive field; the ideas about importance should reflect wider societal values and circumstances.
2. Contribution ot the literature: mere confirmation of previous research is of little value. It must make a significant contribution to established knowledge. This links to Lofland's and Athens' concerns with novely and theoretical development.

we may also need ot consider relevance of the work to practitioners, as well as researchers, e.g. application of findings to practice; this i not so relevant in ethnography. Hammersley suggests that the role of research influencing practice is limited - we need to consider the same two aspects of relevance, though with diffferent implications.

1. Importance of the topic: will the research help the practitioner with a current problem? A short term and specific judgement.
2. Contribution of findings: confirmation of what is already known is of little use to the practitioner too. However, the researcher can question what the practitioner takes for granted.

Which is the most important audience for ethnographic research - other researchers, or practitioners? Assessments of practical relevance of research are continually changing. Judgements of the relevance of studies rely on prior value judgements. How can we justify those judgements? Hammersley believes there are imprtant questions about the ability of ethnography to produce relevant results. Situations studied by ethnographers rarely have intrinsic relevance; they have to be made relevant by generalisation  or theoretical inference.

Summary: Two over-arching criteria that must govern assessment of the value of social research: validity and relevance.

Who is Martin Hammersley?

Professor of Educational and Social Research at the Centre for Childhood, Development and Learning at The Open University. He has carried out research in the sociology of education and the sociology of the media. However, much of his work has been concerned with the methodological issues surrounding social enquiry. http://www.bloomsbury.com/author/martyn-hammersley#sthash.L4aMeWUO.dpuf

Any critiques of his work? I can't find any with a quick look.


OK, so what do I think? It makes sense. Sort of. I'll think some more after the first weekend session.

More reading... Sparkes on the Paradigms Debate

Alternative title: Help me, my brain is melting...

This was a big article. I got lucky with Eisner as my first read - it was an easier read and made intuitive sense. This overview of three key research paradigms, however, was not so easy.

This wasn't helped by coming at it piecemeal - 20 minutes here, half an hour in the soft play centre there, and so on.

Much of what I'm going to write (and there's lots because I found it so difficult) is pretty much verbatim from the text, as I don't yet have the knowledge to be able to condense. As I always tell students asking about paraphrasing, if you don't understand the text, you can't effectively paraphrase it. So, this is mostly direct from text.

Sparkes, A. C. (1992). The paradigm debate: An extended review and a celebration of difference. In: A. C. Sparks (ed.). Research in physical education and sport. (pp. 9-60). London, United Kingdom: Falmer.

There has been a revolution in philosophical paradigms, including in Sparkes' specialism (PE), where empirical predominance is being rethought. [Bear in mind that this was written in 1992, so there has probably been a lot more re-thinking since then!]. The purpose of the paper is as a framework to aid understanding of the debate that is taking place. It covers three paradigms, interpretive, critical and positivist.

There is a discussion on the nature of paradigms but effectively that they are a shared conception of problems and the methods shared within them. They can be described as a shared world view, a way of breaking down the complexity of the world. Paradigms are deeply embedded within the researcher's own socialisations, experiences and framework. Herein lies their strength and their weakness. The strength is that they make action possible without thinking about *every* angle of research. Their weakness is that there are hidden, unquestioned assumptions within each paradigm - an awareness of them is needed to ensure an understanding of the unseen frameworks within which the researcher operates. Paradigms act as a theoretical lens through which we experience the world.

To add to what Sparkes says, and to link to Eisner: these frameworks and assumptions within each paradigm don't just affect how we think and act but at a more fundamental level it will act as a barrier as to whether we *see* specific aspects of what we observe. I'm not sure that's clear, but I think I know what I mean there. We are blinkered, so we don't know that we can't see something, so we have no ability to research/understand it, because to us, it's not there!

I suppose it makes me uncomfortable that Sparkes states "the individual must not only learn the content of the field but also a particular way of seeing the world that eventually becomes not only unquestioned but unquestionable." Is this a healthy way of working? Perhaps so, If the alternative is a philosophical dam against which we ultimately can't move because of the inertia of philosophical vacillation.

During the 'socialisation' process into a paradigm the researcher absorbs assumptions on ontology and epistemology. Social scientists need to question whether there is an external 'reality' or one coloured by individual consciousness and experience - ONTOLOGY.

External realists of positivism vs the internal idealists of interpretivism.

The epistemology - how we know what is true or false - differs too. The objectivist view is that knowledge is capable of being transmitted in a tangible form and is 'hard' and 'real'. The subjectivist epistemology believes that the truth/knowledge is subjective and as such subject to interpretation through our individual lenses or the lenses employed when working within a specific paradigm.

There is a third set of assumptions, based on human nature:
Deterministic: people react to their environment and are products of it;
Voluntaristic: people have control over at least some of their environment and exhibit autonomy, "actively creating their environment".

The standpoint of the researcher on these three sets of assumption affects how they gather date:

1. External realist vs internal idealist
2. Objectivist vs subjectivist
3. Deterministic vs voluntaristic.

Sparkes discusses these assumptions to show that all researchers make assumptions relating to ontology, epistemology, human nature and methodology. These assumptions cluster and are given coherence within the frameworks of particular paradigms. No researcher approaches research as a blank slate - ontological assumptions give rise to epistemological assumptions which have methodological implications for the choices made regarding particular techniques of data collection, interpretation of findings and the ways they are understood and discussed in texts or orally presented.

Essentially, those working within different paradigms view the world in different ways, investigate situations in different ways and report the results of their investigations in different ways. The researchers' basic assumptions concerning ontology (reality), epistemology (truth), the physical world and the social world affect all aspects of research. Confusion is increased through the inaccurate use of words such as methods, research methods, and methodology. Sparkes uses the following definitions:
Methodology: the philosophical underpinning of the investigation
Research strategy: the design and carrying out of the investigation
Research techniques: specific methods to yield data, e.g. questionnaire construction, statistical analysis.
Methods cannot be independent of the philosophical underpinning and similarly the interpretation of results and their recording. "Techniques of data collection do not constitute the uniqueness of a paradigm" - the same data collection method can be use din both a positivist and behavioural way but the meaning put on the findings will differ, as well as the content gathered.

In investigating various paradigms, Sparkes explains that it is important to avoid caricatures. It must be remembered that within each paradigm there is heterogeneity - different traditions within each form of paradigm.

The Positivist Paradigm

This has a  long history, and uses the notions of science as a framework. It is historically important - Sparkes discusses its role within PE and that it offers "almost unquestionable respectability". For example, objective measurement of teaching and learning, the use of standardised data collection instruments. Inter-related assumptions listed by Popkewitz (1984) shape the positivist paradigm:

1. theories are universal and not affected by the values of the researchers;
2. the science is "disinterested" and not affected by the values of the researchers;
3. the social world can be reduced to variables that can be studied independently;
4. concepts can be formalised and defined so as to provide dependent and independent variables to manipulate;
5. use of quantitative analysis to reduce ambiguity.

Ontologically: the social world is ontologically objective and concepts such as intelligence, self esteem exist separate from the individual. The researcher observes nature in such a way as to not affect nature's answers. Sources of bias must be recognised and controlled. Internal and external validity, as well as reliability must be provided by the research techniques.

VALIDITY can be established when the extent to which conclusions represent empirical reality, and also when assessing whether the constructs devised by researchers measure the categories of human experience that occur. "A judgement is 'true' when it corresponds to this external reality and 'false' when it does not - the correspondence theory of truth. Observation (empirical verification) is needed so that we can judge whether a statement is true. To achieve objectivity, positivists follow the 'scientific method'.

The interpretive paradigm

In contrast to the positivist paradigm is the interpretive paradigm, an umbrella term within which sit a whole range of methodologies. We cannot lump them all together as 'qualitative vs quantitative' as some interpretive methodologies are at least partially quantitative. It only cam einto more common usage in the late 1980s, and developed as a reaction against positivism, as it was suggested that studying the social world couldn't be achieved objectively.

The interpretive paradigm adopts an 'internalist idealist' ontology and a subjective epistemology and prefer an IDEOGRAPHIC methodology. For example, facts only exist within the context of a mental framework (construct); i.e. reality is only viewed through a window tinted by values. Knowledge is the consequence of human activity, i.e it is a human construct and can never be agreed as ultimately true. The paradigm takes a position of RELATIVISM - there are multiple realities existing within different individual's minds, and are open to subjective interpretation. The mind of the researcher doesn't "create" the findings, but colours the interpretation of the findings, assigning meaning and intentions makes the social reality.

The paradigm rejects the positivist idea of an independently existing reality that can be found through specific methods. Even 'objective' methods are value-laden and open to interpretation. It takes an EMIC approach. Sparkes discusses authors' views on ethnography, explaining social reality from within the group, as opposed to the positivist view, detachedly looking in.

Interpretive research and the 'researcher as instrument'
Sparkes discusses researchers viewing the role of the researcher within ethnographic studies as being *the* most important research instrument - it is "engagement of the self". Within ethnographic studies, the data that are not gathered, as well as those that are, shape the research - the framework within the researcher will affect the data gathered.

In interpretivism, truth/validity isn't a matter of correspondence, it is a matter of COHERENCE - what is true is what we can agree (within the confines of that time/context) is true. 2Within a coherence theory of truth, a proposition is judged to be true if it COHERES (connects consistently) with other propositions in a scheme or network at a particular time - COHERENCE is a matter of internal relations as opposed to the DEGREE OF CORRESPONDENCE with some external reality.

Popkewitz describes objectivity in this area is the result of inter-subjective consensus through social interaction. He also states that this also applies to the scientific community - knowledge of science is considered valid and truthful only insofar as it reflects the consensus of the scholarly community.

The "truth" of an ethnographic report depends on how well it "rings true" to natives and colleagues within the field. However, interpretation remains an INTERPRETATION of a set of events; credibility is not necessarily is altered by agreement or disagreement by the subjects. The subjects will have their own interpretation of their social world (first order constructs) which often differ from those of the researcher (second order constructs). It is still important for the researcher to check their findings and that certain words were used/events took place. It's also important to discuss interpretations even though there may be disagreements - these, in themselves, can be of use and provide further data for interpretation - REFLEXIVE ELABORATION. Furthermore, there may be disagreements amongst scholars studying the same group. These differences are generally those of emphasis or orientation. "qualitative research cannot be made researcher-proof". Because multiple interpretations can be made even on the same group, there can be 'many truths' available. So, with multiple interpretations possible, which is 'best' or 'most nearly correct'?

There can be multiple truths through multiple interpretations, but we must make sure that we don't lose our critical abilities - not all interpretations stand up to scrutiny. This is true, even though interpretivism is relativistic. There are three main views of relativism (Rorty, 1985):

1. every belief is as good as any other;
2. "true" is an equivocal term, with as many meanings as there are procedures of justification;
3. There is nothing to be said about truth or rationality apart from descriptions of the familiar procedures or justifications which our society uses in areas of inquiry.

Most interpretivists adhere to the third view - judgements of truth are relative to a particular framework, paradigm or point of view. Therefore, not everything goes; researchers within an interpretive paradigm differentiate between 'good' and 'bad' research, according to their own framework.

Some researchers have discussed how constructivist research can best be judged, in this case. e.g. trustworthiness, credibility, transferability, dependability and authenticity. Under these come sub-lists - these checklists help to judge the 'goodness' of constructivist research.

Harris states that at least three levels of interpretation are needed for 'good' qualitative research:

1. Grounded in shared understandings about the culture developed between the researcher and members of the group being examined.
2. Must include the researcher’s insights of the culture that are not well-articulated by the members of the group;
3. Must include theoretical generalisations that go beyond the group culture to link to relevant parts of other research.

Athens discusses here criteria for evaluating qualitative studies.

1. Theoretical import – the contribution towards new theories or refinement of existing theories;
2. Whether the scientific concepts are empirically grounded;
3. “Scientific credibility” – the researcher must make the study credible by providing an account of the research along with a description of the results. An account is a story told by the researcher about how they performed the research in question.

It is important to remember that different researchers using different frameworks and theoretical underpinnings, when observing the same group, may well provide different interpretations using sound interpretive research.

To try to consider which research is ‘stronger’ we are tied by HERMENEUTICS – truth is constructed by the rhetoric of the researchers. “A good explanation makes sense of the behaviour, but to appreciate the good explanation, one must agree on what makes good sense. What makes good sense is a function of ones readings, and these in turn are based on the kind of sense one understands.” ”Truth is what we agree to be true at any one time”.

However, different interpretations can co-exist, providing a richer view of a culture.

The Critical Paradigm

There is no such thing as ‘critical theory’; it’s an umbrella term for a range of different theories. The key commonly shared assumption is EMANCIPATION – enabling people to gain knowledge and power in order to take control of their lives. There is a dismissal of positivism as a means of enforcing the status quo whilst producing theories that are “often trivial and useless”.

Critical theory developed from the work of the Frankfurt School; it is overtly political, highlighting substantive social issues, uncovering the oppression within these issues and, importantly, doing something about it.

With regard to ontology and epistemology, there are two strands running through the critical paradigm:

1. Associated with positivism – a RADICAL STRUCTURALIST strand with an external realist ontology, an objectivist epistemology and a deterministic view of people. It concentrates on STRUCTURAL relationships within a realist social world via analysis of deepseated internal contradictions and the analysis of power relationships.
2. Associated with radical humanism – similar to the interpretive paradigm, it has in internal-idealist ontology, a subjectivist epistemology and a voluntaristic view of people. It is believed that reality is socially constructed, knowledge is context specific and value-laden.

Critical theorists see interpretive paradigm research as having major weaknesses as they do not consider the wider sociohistorical, political and economic movements on the theories.

One major concern within education research (PE) is that interpretivist research has not taken into account the way individual and group behaviour is influenced by the way society is organised. Findings are determined within a social and organisational context permeated by the inequal power relationships (cf Foucault?) Anderson (1989a) suggests these following critical research questions regarding the nature of knowledge in organisations:

1. What counts as knowledge?
2. How is what counts as knowledge organised?
3. How is what counts as knowledge transmitted?
4. How is access to what counts as knowledge determined?
5. What are the processes of control?
6. What ideological appeals justify the system?

The central emphasis is the way human consciousness is shaped and controlled by existing social arrangements to serve some groups in society at the expense of others (INEQUAL POWER RELATIONSHIPS). Relational analysis is often used to analyse these relationships – viewing the historical, social and cultural constructs surrounding the practice under study.

Griffin (1990): “

1. Society is made of groups with power and privilege and those without;
2. Social institutions in a society perpetuate the status quo of this power imbalance;
3. The powerful and privileged have a vested interest in maintaining the status quo (their power and privilege);
4. The powerless and disadvantaged have a vested interest in social change;
5. The competing interests result in conflict and tension, often below the surface of apparent harmony and consensus;
6. The critical perspective brings to light the contradictions between apparent harmony/consensus and conflict/tension, in order to problematize the status quo;
7. A critical perspective is concerned with ‘why/why not’ questions – whose interest is served? The intention is to change the world, not describe it;
8. A critical perspective believes in the importance of changing individual and group consciousness in creating social change.

Critical researchers may use data collection techniques from the interpretive paradigm, but go further, to investigate the inequal power relationships – critical ethnography. This can be done in three ways:

1. Consider the subject group in a wider social context. This is weak as the critical analysis can be omitted if it just analyses functional relationships between the subject group and the wider social milieu.
2. Critical ethnography through focussing on the wider structural relations and examining how social processes in the subject group are mediated by structural relations.
3. The strongest form is to incorporate ethnography into a DIALECTICAL analysis – the understandings from the ethnographic study are analysed in relation to the social structures that shape the lives of people.

This final method begins with structural relationships and then undertakes an ethnographic study in order to facilitate a structural analysis.

Where critical ethnographers differ is that they claim that the subject’s perceptions of social reality are permeated with meanings that sustain powerlessness. The conscious models used by people exist to ensure the continuance of the social phenomena.

Researchers investigate the process by which certain meaning structures become accepted as the status quo. They then consider whose interest the status quo benefits. They actively engage with the social group to elicit transformation through understanding and action. They seek to change the world. Examples of the critical paradigm are: feminist research and neo-Marxist critical ethnography.

For transformation to occur, research must be done with the full participation of the people under study. The researcher provides those researched with insights that might act as the basis for change. The researched are participants in the process not subjects to be studied.

Validity has a very different definition in critical research. It needs to be understood as being relevant to the practitioner’s situation, and potentially transformative, to be valid. “Validity in critical research relates not only to the trustworthiness and credibility of the interpretation but also how effective the research process has been in empowering the participants and enabling them to create change.” This is very different to the positivist view of researcher neutrality. Therefore, validity’s definition changes significantly depending on the paradigm within which you work.

Comment by Sparkes

Validity derives its meaning from different sets of assumptions, theories and purposes within the different paradigms. Each paradigm needs to be understood in its own terms. It is important to recognise that each paradigm can help develop our understanding of the social world, and also that new paradigms will be developed. You must judge research carried out within a paradigm according to that paradigm’s own criteria and assumptions. It is important to be aware of the assumptions within each paradigm, even if you do not work within it.

What does all this mean to me?

Not a lot at the moment…

Firstly, this paper was written a long time ago, now. I’m sure the arguments have moved on and that there are new paradigms that have taken over as the most in touch with the ‘zeitgeist’.
Secondly – the long words… my brain hurts…
Thirdly, whilst I get the gist of critical research, I doubt I could get involved in it. The thought of being transformative in the research seems such a huge jump, especially with my background in empiricism and the ‘detached scientist’ viewpoint. Interpretivism doesn’t seem quite such a huge leap, though. We’ll see. Lots more to read, and I need to go back to look at some of the terms I don’t understand.