Printable PDF Version   Full Symposium         Just this article


Issues of Validity: Behavioral Concepts, Their Derivation and Interpretation

Janice M. Morse, Judith E. Hupcey, Janice Penrod, Jude A. Spiers, Charlotte Pooler, and Carl Mitcham

Janice M. Morse, RN, PhD, FAAN, Director, International Institute for Qualitative Methodology, Professor, Faculty of Nursing, University of Alberta, Canada

Judith E. Hupcey, RN, EdD, Assistant Professor, School of Nursing, The Pennsylvania State University, USA

Janice Penrod, RN, PhD, Assistant Professor, School of Nursing, The Pennsylvania State University, USA

Jude A. Spiers, RN, PhD, CIHR & Killam Postdoctoral Fellow, Assistant Professor, Faculty of Nursing, University of Alberta, Canada

Charlotte Pooler, RN, MN, Predoctoral student, Faculty of Nursing, University of Alberta, Canada

Carl Mitcham, PhD, Professor, Liberal Arts and International Studies, Colorado School of Mines, USA

Abstract:
Qualitative inquiry that commences with the concept, rather than the phenomenon itself, is subject to violating the tenet of induction, thus is exposed to particular threats of invalidity. In this symposium, using the examples of the concepts of uncertainty, trust, vulnerability and suffering, and interview and videotaped data, we discuss strategies to maintain the inductive thrust, and hence validity, during data analysis. The authors present the use of a skeletal framework and scaffold as techniques to “frame” the concept, while, at the same time, continuing to further develop the concept.

Key words: Concept development, qualitative validity, induction, video analysis, scaffold, skeletal framework, trust, uncertainty, vulnerability, suffering.


Contents:

Introduction

Exploring Qualitatively-derived Concepts: Inductive-Deductive Pitfalls
Janice M. Morse and Carl Mitcham

The Pink Elephant Paradox (or, Avoiding the Misattribution of Data)
Jude A. Spiers

Maintaining Validity: The Development of the Concept of Trust
Judith E. Hupcey

Advancing Uncertainty: Untangling and Discerning Related Concepts
Janice Penrod

Analysis of Videotaped Data: Methodological Considerations
Janice M. Morse and Charlotte Pooler

Conclusions

References


Introduction

The anathema of qualitative inquiry is essentially one of validity. While much literature exists on methods of controlling or countering threats to validity when the goal of research is description, these problems are compounded when one begins working abstractly. Not only is the research most at risk with this research approach, but these problems have been poorly addressed in the methodological literature.

We consider the goal of qualitative science twofold: first to develop concepts in order to get a better grasp on the phenomena represented by the concepts themselves and, second, from this, to develop generalizable and valid theories. We believe it is these tasks, essentially those involving interpretation, conceptualization and abstraction, that will eventually provide qualitative inquiry with a legitimate place in the social sciences, and ultimately earn its respect and contribution to knowledge.

Presently, ways of controlling threats to inductive validity with descriptive research are only partially successful. Briefly, strategies used prior to commencement of data analysis such as bracketing (Janesick, 2000, pp.390-391), rejection of preconceived theoretical frameworks (Miles & Huberman, 1994), or techniques of verification used during the conduct of inquiry (Meadows & Morse, 2001) demand that inquiry begins from the data with each new project, and do not facilitate the incremental compounding of research projects. Post hoc methods to ensure validity, such as testing results by implementation and subsequent inquiry (Morse, 2001), while important, occur too late in the process of inquiry to expedite the process of inquiry itself. While these checks and balances guide inquiry towards validity, there is a need to explore the problem of conducting qualitative inquiry using concepts as a starting point within the analytic processes of induction/deduction, and to bring to the fore ways that more advanced inquiry implicitly proceeds. In particular, there is a need to explore the problem in instances in which inquiry begins with a concept itself, rather than commencing with basic description. Thus, in this symposium, we have attempted to identify and to formalize techniques by which inductive processes may be sustained (and deductive tendencies avoided) when commencing inquiry at the conceptual level. We use four research projects to illustrate these solutions.


Exploring Qualitatively-derived Concepts: Inductive—Deductive Pitfalls

Janice M. Morse and Carl Mitcham

Citation information for this section:
Morse, J. M., & Mitcham, C. (2002). Exploring qualitatively-derived concepts: Inductive—deductive pitfalls . International Journal of Qualitative Methods, 1(4). Article 3. Retrieved DATE from http://www.ualberta.ca/~ijqm.


Analytic induction is a sacred tenet of qualitative inquiry.1 Therefore, when one begins a project focusing on concept of interest (rather than allowing the concepts to emerge from the data per se), how does one maintain a valid approach? When commencing inquiry with a chosen concept or phenomena of interest, rather than with a question from the data per se about what is going on, how does one control deductive tendencies to see what one desires to see and which threaten validity?

Difficulties stem from the nature of induction itself – Is analytic induction an impossible operation in qualitative research, as Popper (1963/65) suggests? In this section, we first discuss Popper’s concern, followed by a discussion of two major threats that may prevent an inductive approach in qualitative research.2 The first threat is the “pink elephant paradox;” the second is the avoidance of conceptual tunnel vision or, specifically, how does the researcher decontextualize the concept of interest from the surrounding context and thereby avoid the tendency to consider all data to be pertinent to the concept of interest? As we explore each of these pitfalls, and we present methodological strategies to maintain both the integrity of the concept and the integrity of the research.

The myth of induction

Popper (1963/65, p. 46) identified the most well-known threat to inductive soundness, which has become the Archille’s heel of qualitative inquiry.3 Popper summed up his challenge to the notion of induction with an example of a group of physics students in Vienna in the 1940’s:

Take a pencil and paper; carefully observe, and write down what you have observed!’ They asked, of course, what I wanted them to observe. Clearly the instruction, ‘Observe!’ is absurd. (p. 46)

With this example, Popper is implying that just as observation is ‘always selective,’ induction is not presuppositionless. From this criticism, fear of violating inductive processes has resulted in researchers’ reluctance to focus on a concept until it ‘emerges,’ and some researchers even avoid the literature before commencing fieldwork (see Glaser, 1992).

But because Popper has removed the process of induction from the context of research itself, we suggest that Popper’s concern is unwarranted. Let us explain, and at the same time consider the history of the development of this problem, which we call the myth of induction.

The problem of induction is already hinted at in the 4th century BCE by Aristotle (2000), although his approach is not so much to reject what will not fit into a tight logical box as to explain how something like induction, which obviously takes place, must in fact be able to do so. In an important passage from On Interpretation (Aristotle, 2000), he suggests that the formation of concepts is a little like what goes on as an army retreats under attack, constantly falling back here and then there looking for a place to make a firm stand. The passage easily reminds one of Piaget's (1959) notion of equilibration, of how concepts are developed through trial and error engagement with phenomena. In both cases, induction is accepted as a real process, and one that is not subject to deductive logical formulation. This is not to deny that some skill-based rules of thumb might help guide induction, although it has been left to later phenomenologists and qualitative researchers to attempt to formulate such rules or guidelines.

When Hume (1976) formulated the classic riddle of induction (albeit, applied to propositions), the upshot is simply to note that thinking involves two different kinds of concepts: those which can be linked or connected by necessity and those which cannot. But there is no need to deny the reality of concepts that cannot be connected by necessity. The fact that the concept of a triangle necessitates that the sum of the interior angles be 180 degrees, whereas the concept of a dog does not with the same necessity mean that it is a mammal, in no way requires that the concepts of dog, mammal, and that dogs are mammals be rejected as being unsound or illegitimate concepts.

Thus, when Popper goes so far as to reject induction as a myth and to replace it with capricious conjecture, which we simply accept as long as we cannot empirically refute it by finding some phenomenon that falsifies it, he reveals his own inherently rationalist biases. It may well be true that this is how some sciences, especially the highly mathematicized ones, tend to work. But it is certainly not how all science has to work, or in fact does work. Biology, for instance, clearly proceeds in its classification of organisms more like a well organized army faced with ever new experiences.

In this way, Popper's argument is itself unsound precisely because he has removed the process of induction from its real-world context in different kinds of research. Consider another example: A race is defined by certain characteristics or parameters (that is, there must be a start and a finish, something to race against such as more than one competitor or time, there must be a system of measuring whatever is being challenged, and so forth), and without these characteristics one cannot have a race. Similarly, research has defining characteristics, one of which is a focus of inquiry. You cannot have research without something to be inquiring about. Thus taken in the context of research, Popper’s classic criticism of induction in qualitative inquiry (‘What shall I observe?’) is in itself invalid.

The issue is not if the inductive process can be used in qualitative research, but how induction should be used.

Nevertheless, our concerns regarding the pink elephant paradox remain, and are concerns that the concept of bracketing does not resolve. Bracketing works very well for formal knowledge, but less well in instances when the threat to induction is less conscious, as may occur with conceptual tunnel vision. The alternative offered, a priori theoretical frameworks that prescribe coding schemes, have been rightly discarded as a source of invalidity for qualitative inquiry.

The pink elephant paradox

“Don’t think of a pink elephant!” is an impossible instruction, for once the idea of a pink elephant is mentioned, it cannot be erased from one’s consciousness. The pink elephant paradox raises the possibility that one could think an idea or concept that one was trying to avoid, and indeed confirm the existence of phenomena to which the concept refers, since once a person starts to think of pink elephants the person also easily starts not just to think them but also to believe in them. It is possible, for instance, that the mere adoption of some particular coding (or theoretical) framework might lead one to “prove anything”, as Popper and others have noted.

We argue that pink elephants are less of a risk in sound qualitative inquiry because they are controlled, to some extent, by processes of saturation, replication and verification. At the same time, by accruing multiple examples of the same event/relationship/phenomenon in the data, from different times or different circumstances, by asking critical questions of these data, and by constantly looking for alternative explanations, the risk of misattribution or miscategorization is reduced. Thus, the risk of pink elephants is greatest in thin data sets.

However, to some extent, the risk always remains and we admit vast pink elephant problems have occurred in social science research, both qualitative and quantitative. One historical example is the theory that masturbation causes madness, which was experimentally "confirmed" repeatedly, and “treated” with treatments such as clitoridectomy (Engelhardt, 1974).

Conceptual tunnel vision

Conceptual tunnel vision exemplifies the researcher’s problem in deciding which data do and which do not pertain to a concept, or are and are not examples of the concept. Conceptual tunnel vision is the over-categorization of data, assigning more data to one category than actually belongs, or seeing or justifying most things as being related to, or considered examples of, the concept being investigated. This problem is inflated with the value in qualitative inquiry on holism, so that the process of encompassing all data–and the fear of missing something–is embedded in this problem. The questions that the researcher must struggle with are: What is and what is not pertinent to inquiry? And how can I be certain?

When conducting research into a concept, tunnel vision becomes the analytic anathema and over-attribution inflates both the contents and the role of a concept in the results.

How can this problem be controlled? One method is to bring critical inquiry out into the open and demanding that categories earn their way into the analytic scheme. For instance, in Morse’s research program on comfort, we ask: Is thus and so an example of caring or comforting? What is the relationship between caring and comforting? Is caring a part of comfort, or comfort a part of caring? Do they share attributes, or are their attributes distinct? In this way, by constantly being alert to hidden and underlying assumptions, and by only allowing legitimate facts and relationships to be used, we control the use of poorly linked or irrelevant contextual characteristics into the developing theory.

Exploring qualitatively-derived concepts: Inductive techniques

We now discuss intermediate solutions or approaches to controlling validity. These strategies are probably already used in qualitative inquiry, but have not been yet formalized and described. We will identify these strategies, and in the other four parts of this article we will illustrate the use of these strategies in the context of completed projects. Because research is a process, each of these methods identified are best used at different stages of inquiry according to the maturity of the project itself.

Deconstruction: Techniques of concept analysis

The first step is using the literature to conduct a concept analysis of the concept. While we disagree with Glaser (1978, 1992) that one enter qualitative inquiry without using the knowledge of others, either conceptual or substantive, we also disagree with the process of simplistic bracketing. Rather, the researcher should act as an informed consumer when using this literature, assume that it is correct, and critically analyze it all as a whole, deconstructing the concept to identify the attributes or characteristics, assumptions, gaps, limitations, differing perspectives (including way the concept has been developed in different contexts or disciplines), and different forms of the concept for different functions. Then, once this analysis is completed, the researcher is working wisely, perhaps selectively bracketing, perhaps using this information to refine one’s proposal, perhaps using this information as a comparative template in the process of data collection. Regardless of how the information is used, knowledge makes one’s questioning of data smarter as data collection proceeds. The researcher is not with blinded by ignorance, or by the present ‘partly line’ of theories, models and myths that seem pervasive in the literature.

Jude Spiers’ analysis in Part II of this symposium is particularly interesting, as she subsequently conceived vulnerability not as an internal state, but as something that could be negotiated in the nurse-patient interactions, and therefore observed. In part III, Judith Hupcey will briefly describe how she built her study of trust through an interdisciplinary exploration of the concept; Janice Penrod (in part IV) describes a careful assessment of uncertainty. We concede that Popper was correct when he stated that inquiry does not begin from nothing, but by using concept analysis as described elsewhere, (Morse, 2000) qualitative inquiry begins its inductive processes by deconstructing all the implicit assumptions, building from a carefully inspected base, by an informed researcher.

Focusing: Development of a skeletal framework

Inquiry then proceeds depending on the ‘maturity’ of the concept (Morse, Mitcham, Hupcey & Tasón, 1996). When concepts are immature or little is known about the concept, the next step in inquiry is to identify and develop a skeletal framework.

How do you proceed? Normally with ethnographic research, data collection begins as a comprehensive and complete ‘fishing trip’–the holistic approach, or ‘scoping’ (Morse & Richards, 2002). Indeed, broad ‘maps’ are available to ensure such comprehensive data collection, such as Leininger’s (1988) Sunrise Model or Spradley’s (1980) Descriptive Question Matrix. Basically, these schemata ensure that inquiry is broad, so that necessary data are available when, later in the study, the researcher focuses on a particular topic of inquiry. It is a way to ensure validity—by ensuring a complete data set is available, by ensuring that the concept developed is comprehensive and complete, and by ensuring that ‘premature closure’ has not occurred. For instance, Leininger’s Sunrise Model includes categories such as technological factors, religious and philosophical factors, kinship and social factors, and so forth, and how these broad categories influence care patterns and health. Spradley’s model is more particular and action oriented, and includes categories such as space, object, act, activity, event, actor, goal, and feelings (1980, pg. 82). Each topic is linked in a matrix to every other topic but, again, these data must be placed within the context of the question asked. In our case, the careful conceptual analysis work preceding the stage of data collection reduces this fumbling, and enables the researcher to move more quickly through the fieldwork. This background work allows the researcher to focus more quickly, thus expediting the research process.

Note that the researcher is only partially rescued from the invalidity dilemma. We discussed what to call the ‘level of theory’ developed from this type of semi-focused observations and interview, and decided that the analogy of the skeleton best summed up what we were trying to convey. From the concept analysis, we have some information about the essential characteristics or attributes of the concept, so we know where to direct our attentions but much still remains unknown. As an archaeologist does when discovering a skeleton, we knew roughly the shape of the original dinosaur–and perhaps even how it moved and worked–but we only had a general idea of its actual appearance. As the concept boundaries remain unclear, the risk of omission in data collection remains. To compensate for this risk of missing, ignoring, or omitting essential data, the scope of data collection needs to remain somewhat broader than the actual concept. Thus, researchers should initially sample more data than is required, and refine focus as the study proceeds. However, we avoid the mistake of assuming that all data are relevant–to conduct such a fishing trip is not using inductive principles for inquiry. Only by collecting rich and relevant data around the bare bones of what is known, using principles of saturation and verification, can we recognize the pertinent data from other data.

In summary, a skeletal framework serves to sensitize the researcher and facilitate focusing the inquiry at an early stage. It provides internal structure to study, thus enabling observations, interviews and analysis to proceed. As an archaeologist tries to piece bones together, the inductive puzzle of inquiry is maintained, and, as inquiry proceeds, falls into place, the skeletal framework is padded, and provides the emerging model with indices of purpose and function.

Towards verification: Using a scaffold

When using a scaffold, one is reasonably confident of the type of concept, either from the literature or from previous inquiry, and the concept may be considered at least partially mature. In this way, the investigator may recognize that a particular setting will provide the researcher with a good example of exploring a particular concept. The investigator is reasonably confident about the domain of the concepts–of what is and what is not an example of the concept. Boundaries have been established, so that the scope of the concept is known (Morse & Richards, 2002). However, the researcher may still have questions about the attributes or characteristics that comprise the concept. Thus, a scaffold delineates a concept, but still enables inductive exploration of the internal compositions of the concept to take shape.

When using a scaffold, the boundaries of the concept may be known, thereby focusing sampling and data collection. However, the internal structures require further investigation. Compared to the previous skeletal framework, sampling is more focused, data are collected in increasing depth, and event sampling may be used. Internally, the researcher holds loosely held assumptions about the attributes. These are inductively explored, with what is already known drawn as a comparative template over the emerging scheme. Thus, previous work, while focusing inquiry, still enables the internal structure of the concept to be malleable and ‘emerge’. Data collection proceeds inductively, with the investigator seeking new insights, verification, and saturation. Once the work is completed, the scaffold is dismantled, and the theory stands on its own.

From our previous work, developed from interviews, we had an understanding of reports of enduring and emotional suffering emotions and behaviors, but we did not know if we could differentiate these states observationally. We also had little information about the interaction between family members who were also enduring or emotionally suffering. In this case, we recognized the pink elephant, but explored it closely to collect rich and detailed behavioral descriptions.

Theoretical frameworks?

Once a concept has been explored and described in depth, inquiry has proceeded to the level that quantitative inquiry and a theoretical framework may, at this stage, be used. A theoretical framework organizes a coding scheme, and it is this structure that deductively prescribes the form of data collection instruments, measurements, and even types of analysis. Note, however, that inquiry has now moved to the deductive quantitative stages.

Summary

To summarize, the systematic exploration of concepts, using interview or observational methods, progresses sequentially from deconstruction of concept analysis of the literature to the use of these data as a skeleton, or to using prior knowledge as a scaffold. All of these stages continue to use induction, but in different ways and in varying degrees. Awareness of the stage of development of the concept, and of how you are using previous inquiry, will expedite inquiry and enhance, rather than threaten, validity. These frameworks have not been previously placed in the context of inquiry into behavioral concepts, and we will use the next articles to illustrate the utility of this approach.

Notes for Section I:

1. In the well-established distinction between qualitative and quantitative research, qualitative research is often criticized for failing to meet the standards generally applicable in quantitative work, when in fact different standards apply. The present argument is an effort to clarify this situation and to defend qualitative research against the kinds of criticisms leveled particularly at the closely related work of concept formation and inductive generalization.

Prefatory to our argument it is useful to review the contrast between qualitative and quantitative research. As has been summarized in Morse (1995), qualitative is typically used to explore new or little known, previously unconceptualized or adequately understood phenomena, or when an investigator suspects the adequacy of or a bias in present knowledge. Qualitative methods are especially appropriate as well in order to approach phenomena from the emic perspective, that is, from the perspective of a non-experimenter or non-observer. As a result qualitative research is usually conducted in a naturalistic setting rather than in a controlled, laboratory situation. In the course of qualitative research hypotheses and theories emerge from data, while data collection is in process or in the course of data analysis. Finally, qualitative research typically uses a small data set investigated in depth.

By contrast, quantitative research approaches more or less well defined phenomena in search of causal relations described from the etic or external world-view perspective of a non-participant observer. The investigator's observations take precedence over the lived experiences of any experimental subjects. Quantitative research thus typically begins not with an exploration of phenomena or data collection, but with the analytic formulation of a hypothesis about causal relations existing in the phenomena and the establishment experimental controls for confirming or falsifying the hypothesis. Quantitative research also uses statistics to determine an appropriately large data set, which will then only be investigated from the perspective predetermined by the hypothesis under investigation.

As this comparison no doubt already suggests, qualitative research is peculiarly appropriate to field work, as in conservation biology or geology, and to investigations focusing on the psychological or personal experiences of human subjects, whereas quantitative research is peculiarly appropriate when doing controlled laboratory experiments on objects or persons insofar as they may be treated as objects. The contrast between these two types of research may thus be summarized in Table 1.

Qualitative Research

Quantitative Research

Used to conceptualize and explore new phenomena

Used to determine causal relations among phenomena

Emic perspective

Etic perspective

Naturalistic setting

Laboratory setting

Hypotheses emerge in the process of data collection

Hypotheses formulated prior to data collection

Small number of samples studies in depth

Large statistically determined sample of subjects studied only in relation to predetermined hypotheses

Especially appropriate to psychological research focused on personal experiences

Especially appropriate to research on physical objects

Qualitative research thus brings into play two of the least clarified and contested processes in scientific practice: concept formation and induction. Almost all analyses of scientific method begin where quantitative research begins, with the assumption or stipulative assertion of concepts or definitions, proceeds from there to the formulation of propositions that link these concepts in hypotheses to be investigated, deduces from the hypotheses phenomena that would or would not be the case if the hypotheses were true, and then proceeds to use appropriately structured empirical investigations to see whether in fact phenomena are or are not such as have been predicted. In the standard philosophies of science concept formation is ignored; the same philosophies typically argue that there are no methods of induction. In the present instance, however, we will make an attempt to clarify the process of concept formation and to defend the process of induction, because both are central to qualitative research. Nevertheless, it will not be necessary to accept all of our claims or arguments to appreciate the case study examples that follow. As Arthur Kaplan has suggested, "differences in epistemology do not prevent acceptance of the same body of scientific truths" (1983, p. 90). Back to text

2. At this point we need to differentiate between analytic induction and abstraction. Analytic induction includes process of testing propositions or less formally, processes of asking questions and seeking the answers in the data, or processes of constant verification as analyses progresses. Abstraction, on the other hand, is a process of analyzing by identifying common properties in the concept. Back to text

3. Popper also defines research narrowly, as refutation rather than discovery, and this perspective also challenges qualitative inquiry, which of course, does not proceed using hypotheses and the classical scientific method. Back to text


Continue to Article II

References