Psychological Methods 2002, Vol. 7, No. 1, 126–138
Copyright 2002 by the American Psychological Association, Inc. 1082-989X/02/$5.00 DOI: 10.1037//1082-989X.7.1.126
The Role of Qualitative Research in Psychological Journals Sean A. Kidd University of Windsor The acceptance of qualitative research in 15 journals published and distributed by the American Psychological Association (APA) was investigated. This investigation included a PsycINFO search using the keyword qualitative, an analysis of 15 APA journals for frequency of qualitative publication, a content analysis of the journal descriptions, and the results of qualitative interviews with 10 of the chief editors of those journals. The results indicate that there exists a substantial amount of interest in the potential contribution of qualitative methods in major psychological journals, although this interest is not ubiquitous, well defined, or communicated. These findings highlight the need for APA to state its position regarding the applicability of qualitative methods in the study of psychology.
coln, 2000b; Smith, Harre, & Langenhove, 1995a.) The present study was designed to examine the degree to which qualitative research is being accepted and published in mainstream psychology journals. In many articles and texts, it is mentioned that there are increasing numbers of qualitative publications (e.g., Elliot et al., 1999; Strean, 1998) and that we are now in a state of transition (Shank, 1994). Few efforts, however, have been made to support such statements or to examine the underlying issues. Studies that addressed questions in this area were literature searches done by Krahn et al. (1995), who found only 30 qualitative articles in psychology journals from 1987 to 1993 and Rennie (2000), who noted a dramatic increase in qualitative publications in the 1980s and 1990s. Additionally, Keeley, Shemberg, and Zaynor (1988) found an increase in nonquantitative clinical psychology PhD dissertations between 1965 (2.5%) and 1985 (9.8%). Finally, a recent article in the APA Monitor reported that a group of editors, primarily in the child development area, was calling for an increase in the recognition of qualitative research (Azar, 1999). Qualitative studies are typically conducted in the field, are usually broadly focused initially without hypotheses or predefined response categories, and concentrate on individuals or small groups. Individual experience (i.e., meaning) is the material studied and interpreted and is obtained through a wide variety of means including interviews, conversations, observation, personal experience, and textual analysis. Depth takes precedence over breadth, and contact might
The question of whether qualitative research has a place in psychology is one that is encountered more and more frequently in recent years (Goldman, 1993; Lee, Mitchell, & Sablynski, 1999; Rogers, 2000). Psychological journals are receiving increasing numbers of qualitative submissions (Azar, 1999; Banyard & Miller, 1998), and qualitative work has become firmly established in related fields such as education, nursing, social work, medicine, and sociology (Denzin & Lincoln, 2000a; Krahn, Hohn, & Kime, 1995; Lee et al., 1999; Rennie, 1999). Psychological journals that had previously published only quantitative work are now, in small numbers, publishing qualitative findings. This trend has been noted in community, developmental, sport, organizational, vocational, and clinical psychology (Banyard & Miller, 1998; Burman, 1996; Elliot, Fischer, & Rennie, 1999; Goldman, 1993; Lee et al., 1999; Rogers, 2000; Strean, 1998). The discussion of qualitative methodology in this article focuses primarily on its current and ongoing development in the field of psychology. (For a more comprehensive discussion, see Denzin & Lin-
Appreciation is extended to Dana Barratt for assistance with coding, Michael Kral for his advice and support on this project, and Eleanor MacKenzie for proofreading the manuscript. Correspondence concerning this article should be addressed to Sean A. Kidd, Department of Psychology, University of Windsor, 401 Sunset, Windsor, Ontario N9B 3P4, Canada. E-mail:
[email protected]
126
QUALITATIVE RESEARCH IN PSYCHOLOGICAL JOURNALS
range from participant observation to a single interview. Statistical analyses are usually not incorporated. Numerous methods are available to researchers conducting qualitative inquiry. One example is grounded theory in which interview transcripts are reorganized into categories and themes in an effort to form theory and hypotheses inductively from the data. Another commonly practiced type of qualitative inquiry is ethnography. Central to ethnography is participant observation in which the ethnographer immerses her- or himself in the context of the phenomenon being studied, conducts formal and informal interviews, and examines related documents. The high degree of personal involvement in this type of fieldwork ideally results in “a detailed depiction and analysis of social relations and culture” (Stewart, 1998, p. 7). Theoretical paradigms such as feminism, Marxism, cultural studies, and constructivism serve as frameworks within which qualitative research is conceived and developed. For a more detailed description, see Stiles (1993); Smith, Harre, and Langenhove (1995b); or Denzin and Lincoln (2000a). It is important to note that qualitative research is no more a single, unitary phenomenon than is quantitative research (Krahn et al., 1995; Strean, 1998). Under the umbrella term of qualitative research, there are multiple methods and theoretical approaches, not all of which are compatible. The arguments and questions raised in this article, however, are applicable and relevant to methodologies ranging from ethnographic field studies to qualitative analyses of texts or brief interviews. The question currently being faced is whether qualitative approaches have something to offer the field of psychology. The issues pertinent to this question are multiple and complex, ranging from epistemological clashes to limited journal space. At the root of the current dilemma is the clash between postpositivist and constructivist theoretical paradigms. These two terms are used in this article in a broad sense for the sake of clarity to represent a spectrum with postpositivism at one end and constructivist approaches at the other (see Denzin & Lincoln, 2000b, for discussion of the paradigms situated along this dimension). Postpositivism is used rather than positivism because the majority of psychologists would likely not subscribe to the more extreme positivist stance. The following discussion is derived from Lincoln and Guba (2000). Each paradigm is described in terms of ontology (nature of reality), epistemology (relationship between inquirer and the phenomenon of inquiry), methodology, and research purpose. Onto-
127
logically, postpositivists ascribe to a critical realism. It is emphasized that there is a reality outside of the inquirer that can be studied, but acknowledged that only an approximate probabilistic understanding of reality can be obtained. Epistemologically, postpositivists strive to study and know their subject matter from an objective standpoint, but are aware that the ideal of objectivity is seldom if ever completely achieved. The primary methodology used by postpositivists is the experimental model that is evaluated according to models of validity and reliability. Typically, the goal is explanation and prediction through a process controlled by the investigator. Constructivist ontology is relativist. Each person’s reality is formed as she or he interprets her or his world through a preconstructed system of ideas, theories, values, and attitudes. An emphasis is placed on accessing the subjective experience of the participants and the meaning they give to their experience. For example, gay identity would be viewed not as an inherent characteristic, but rather as a culturally constructed belief/meaning system that is open to change. Epistemologically, constructivists view inquiry as a process in which the findings are a culmination of both the participant’s and researcher’s meaning systems interacting and the idea of objectivity is rejected. The methodology most often used is qualitative and traditional understandings of validity and generalizability have been altered or rejected. The goals of constructivism are typically to develop understanding and, with maximal involvement of the participants, help them to deconstruct destructive beliefs. An example of this would be breaking apart the belief of “gay identity as a flaw or sickness” and helping to establish the belief of “gay identity as a strength.” Most qualitative researchers subscribe to paradigms that would be placed toward the constructivist end of the previously mentioned dimension. By no means, however, are all qualitative researchers constructivists. As methodological newcomers, qualitative researchers must prove themselves everywhere from their psychology department to editorial boards. A review of what has been written about qualitative methodologies’ position in psychology reveals that while a clash of epistemology likely plays a large role in its slow acceptance, its novelty for both quantitative researchers and many psychologists conducting qualitative research contributes to the problem. Currently, concerns are being voiced that training in qualitative methodology is often inadequate with stu-
128 dent interest exceeding faculty expertise (Elliot et al., 1999; Krahn et al., 1995). Additionally, descriptions of what constitutes good qualitative research in the context of psychology are lacking, with considerable disagreement among the sources available (Goldman, 1993; Krahn et al., 1995; Lee et al., 1999; Shank, 1994). The result has been increasing numbers of publications that propose criteria for good qualitative research in psychology derived from fields in which qualitative research has been longer established. This has been driven in part by editors and reviewers struggling to deal with the increasing number of qualitative papers crossing their desks (Elliot et al., 1999; Fiese & Bickham, 1998; Lee et al., 1999; Merrick, 1999; Stiles, 1993). Developing criteria for qualitative research is not a problem unique to psychology. Because of the relativist ontology underlying many forms of qualitative inquiry, establishing one set of broadly applicable criteria is likely impossible (Schwandt, 1996). Qualitative researchers in the field of psychology often occupy a marginalized position. This stance as the “outsider” appears to lead many psychologists to become defensive in their search for legitimization and to use excessive criticism of positivism or, alternately, to undercut their research as being merely ancillary to quantitative work (Banyard & Miller, 1998; Burman, 1996; Strean, 1998). There are, however, reasons for this defensive attitude considering the substantial difficulties qualitative researchers face in getting their work recognized and published. The major problem, and closely linked to the epistemologically unpalatable nature of qualitative research to mainstream psychologists, appears to be a tendency to apply quantitative criteria to qualitative studies (Elliot et al., 1999; Goldman, 1993; Rogers, 2000). Given the defining characteristics of qualitative research, the result is a miserable failure when qualitative studies are evaluated using quantitative concepts of validity and reliability. This lack of fit extends to issues such as the difficulty in presenting qualitative work in accordance with American Psychological Association (APA) guidelines for publication, and the larger amount of journal space required to adequately present qualitative material (Goldman, 1993; Rogers, 2000; Zeller & Farmer, 1999). Related to the abovementioned problems is the way in which qualitative research is frequently used to communicate the experiences of politically marginal individuals or groups, possibly creating resistance from dominant cultural groups (Lather, 1991).
KIDD
The resistance of natural scientific epistemology and methodology, combined with a lack of knowledge regarding qualitative methodology, is coming up against a rapidly growing recognition of the strengths of qualitative research. In the past decade, there have been repeated calls in the various subfields of psychology for further investigation into the potential contribution of qualitative studies (Banyard & Miller, 1998; Elliot et al., 1999; Fiese & Bickham, 1998; Goldman, 1993; Krahn et al., 1995; Lee et al., 1999; Shank, 1994; Stiles, 1993; Rogers, 2000; Strean, 1998). This trend has extended to quantitative textbooks that are now devoting more space to the discussion of the utility of qualitative research (e.g., Kazdin, 1998). Often cited as a strength is the ability of qualitative methods to access personal experience and meaning, cultural diversity, contextual factors, theory/ hypothesis generation and elaboration, rare cases/ conditions, and exploration of a topic in a depth not possible in quantitative approaches. Qualitative methodology has been conceptualized as being a useful tool for inquiry (Lee et al., 1999), and not a new one. Early in the history of psychology, significant contributions were made by using qualitative methodology in the work of eminent psychologists such as Kurt Lewin, Henry Murray, Erik Erikson, Jean Piaget, and as far back as William James and Willhelm Wundt (Langenhove, 1995; Rogers, 2000). While the debate about the utility and applicability of qualitative methods and their underlying philosophies is only recently gaining momentum in psychology, similar controversy was heard and to varying degrees resolved in other fields. Most notable are anthropology and sociology, in which the majority of the history and development of qualitative methodologies occurred (LeCompte, Millroy, & Preissle, 1992; Vidich & Lyman, 2000). In the 1970s, through a movement spearheaded by Clifford Geertz, anthropology came to reject as inadequate ethnographies written as objective accounts of the rites and rituals of the “primitives” or “others” that they were studying (Denzin & Lincoln, 2000a; Vidich & Lyman, 2000). The subjective worlds of participants were regarded as important, with the ethnographer’s worldview being an integral part of the information generated. This shift occurred at many levels including dissatisfaction with methods, clashes of epistemology, and reaction against racist and ethnocentric assumptions underlying much of the earlier work. Since then, there has been an enormous amount of activity in the development of qualitative methodologies and the advance-
QUALITATIVE RESEARCH IN PSYCHOLOGICAL JOURNALS
ment of paradigmatic perspectives, although the question of quality in qualitative research remains a major source of debate in anthropology (Denzin & Lincoln, 2000a). In other fields with more empirical scientific traditions, the recognition of qualitative methodologies took place as part of a reaction against scientific method as the only valid method of inquiry. This recognition occurred in sociology in the 1970s (Vidich & Lyman, 2000) and expanded into other fields such as social work, nursing, and education throughout the 1980s. Although there was resistance in all of these fields to a set of methods that appeared to be different from the scientific model, by means of demonstrating the utility of the qualitative methods, scholarly debate, and endorsement by major figures, these qualitative methodologies became widely recognized (Streubert & Carpenter, 1995). In the present study, I incorporated several elements in an effort to address the question of the frequency of qualitative publication and the issues surrounding the publication of qualitative material. First, the frequency of research articles indexed through PsycINFO in which qualitative methodology had been used for the years 1989, 1994, and 1999 was determined. These years were chosen because it has been in the past 10 years that the majority of current interest in qualitative methods in psychology became evident (Elliot et al., 1999). Second, the frequency of qualitative publication was determined for 15 journals published and/or distributed by APA for the years 1989, 1994, and 1999. Third, a content analysis was made of the mission statements of these journals to determine which appear to be open to qualitative submissions. Fourth, 10 chief editors of these journals participated in a brief semistructured phone interview focusing on their thoughts and opinions regarding qualitative research. These interviews were then analyzed by using a qualitative content analysis procedure.
Method PsycINFO Search A search was performed on the PsycINFO database in which the search term qualitative was used for the years 1989, 1994, and 1999. The identified listings were then reviewed to determine the number of research studies in which qualitative methods were used. These research papers included studies incorporating both quantitative and qualitative methods, dissertation abstracts, and articles that would not accu-
129
rately be called “psychological.” While it is recognized that there exists a margin of error in such a database search, it was felt that such a search would serve as a rough indicator of a major trend in the social sciences as accessed by most psychology researchers, practitioners, and students.
APA Journal Analysis For the purpose of this analysis, I identified 15 journals published and/or distributed by APA (see Table 1) in which it appeared that qualitative studies could potentially make a contribution. These were research journals that had as their foci areas in which human experience could be and is a substantial component. This strategy is a variation of critical-case sampling (Patton, 1980). By selecting journals that are the most likely to publish qualitative material, one can logically generalize that if these journals are not publishing qualitative work, other journals are likely publishing such work with similar if not lower frequencies. Additionally, generalizability is likely, given that the sample of journals chosen represent more than one third of the total number of journals published and/or distributed by APA. For these 15 journals, the number of articles incorporating qualitative research was determined for the years 1989, 1994, and 1999. The definition of what constitutes qualitative is consistent with that given at the beginning of this article. In addition to purely qualitative research articles, studies in which both quantitative and qualitative components had been incorporated were counted.
Mission Statement Content Analysis The mission statements for each of the 15 journals were independently analyzed by two coders to determine how open these publications appeared to be to the acceptance of qualitative submissions. Three categories were developed. First was the exclusive category, which included mission statements specifying that only or primarily empirical or scientific studies would be considered. Second was the neutral category, in which neither discouraging nor encouraging terms were used regarding qualitative submissions. Journals that called for case reports/studies were also placed in the neutral category; although case studies can be qualitative, they have a long tradition in clinical psychology and may not indicate openness toward other forms of qualitative work. Third was the inclu-
130
KIDD Table 1 Results of Content Analysis for Studies Incorporating Qualitative Methods Journal title
1989
1994
1999
Total
Cultural Diversity and Ethnic Minority Psychology Developmental Psychology European Psychologist Health Psychology Journal of Abnormal Psychology Journal of Consulting and Clinical Psychology Journal of Counseling Psychology Journal of Educational Psychology Journal of Family Psychology Journal of Occupational Health Psychology Journal of Personality and Social Psychology Professional Psychology: Research and Practice Psychology and Aging Psychology of Addictive Behavior Rehabilitation Psychology Total
— 0 — 1 0 0 1 0 1 — 0 1 1 0 0 5
— 0 — 0 0 0 8 1 1 — 0 0 0 1 2 13
6 0 0 0 0 0 4 0 0 1 0 7 0 0 2 14
6 0 0 1 0 0 13 1 2 1 0 8 1 1 4 32
Note: Dashes indicate new journals for which there was no information available regarding earlier years. Total number of research articles including qualitative articles was 2,402.
sive category, which included mission statements that requested qualitative submissions.
Interviews With Chief Editors Ten chief editors of the 15 journals were contacted by phone for semistructured interviews on the topic of qualitative research. After briefly defining qualitative research, two central questions were asked: “What are your thoughts on qualitative research?” and “Does qualitative work have a place in a journal such as yours. Why or why not?” These interviews were conducted in a flexible manner consistent with semistructured interviewing (Smith, 1995) and ranged in length from 5 to 25 min. Seven of the 10 interviews ran 5 to 10 min in length, with the remaining 3 lasting 15 to 25 min. The brevity of most of these interviews was not due to a limitation placed on the interview length, but rather that most of the editors admittedly did not have a great deal of experience with qualitative research and as such, they did not have a great deal to say on the matter. Ten of the 15 chief editors were contacted and readily agreed to participate in the study; the analysis was conducted as the interviews progressed. I was unable to contact the remaining 5 editors despite repeated attempts spanning over a period of 6 weeks. Interviews were transcribed and analyzed by using a qualitative content analysis procedure (for a clear and concise description of such a method of analysis, see Smith, 1995, or Rennie, Phillips, & Quartaro, 1988). Qualitative analysis is a process developed
to access and understand the meanings that a participant or participants give to the phenomena of interest. Through the interview process, analysis, and writing, the researcher keeps memos/field notes in which ideas, reactions, observations, and so forth are documented. Keeping memos is a crucial task as it aids in the development of the analysis and theory. In addition, “memoing” helps researchers understand their positions and opinions regarding the topic/participants and facilitates the researchers’ ability to communicate how their worldview affected the direction of the study. During the process of data analysis, the text of each interview is thoroughly examined several times as the analyst identifies “meaning units.” These may be only a few words or several sentences. For example, “qualitative research is good at generating hypotheses” would be considered a meaning unit. In the coding procedure related meaning units are placed together to form categories. Using the example given previously, every meaning unit indicating how qualitative research is good at generating hypotheses is placed together in the same category. This is a process referred to as open categorizing, which involves creating a new category for each meaning unit that does not fit in with any previous category (a meaning unit may also be placed in more than one category). One major way of validating the analysis lies in ensuring that the categories created are based largely upon what the participant says without the researcher making inferences and interpretations that may move the
QUALITATIVE RESEARCH IN PSYCHOLOGICAL JOURNALS
analysis away from the meaning systems of the participants. As the coding procedure progresses, the analyst moves into higher levels of abstraction, beginning to organize and manipulate the categories themselves. This may involve developing a hierarchy of categories, in which a weaknesses of qualitative research category would be subsumed under the method issues category. The analyst might also place together categories that appear to exist along the same dimension, bringing a number of categories together under one overarching dimensional category. This is the stage of the analysis in which potential hypotheses regarding causal conditions, contextual factors, interactions, and consequences are formulated. There is a constant interplay between generating an idea and bringing it back to the data where that idea might be revised, supported, or refuted. The categories are organized and reorganized conceptually as the analyst audits original meaning units to determine if that organization accurately reflects the meanings communicated by the participants. Throughout this process, descriptive statistics can be helpful in highlighting patterns within the data and assessing the amount of evidence supporting conclusions (Maxwell, 1998). At this stage, there may appear in the data a central theme that cuts across all or most categories. Such a central theme can be used in conjunction with findings of category relationships and processes to develop a theory (although not readily applicable in the present study because of the focused nature of the interviews; see Glaser & Strauss, 1967, or Rennie et al. 1988, for a description of “grounded theory procedures”). Transcripts were categorized independently by a research collaborator in what has been called “peer debriefing” (Stiles, 1993). Stiles also suggested that the qualitative investigator disclose her or his orientation, a procedure that helps put the interpretation in perspective. In this case, I (primary investigator) am in my second year of doctoral studies in clinical psychology. The primary methodology I employ in my own research is qualitative, although I have completed the required training in quantitative methods and have conducted numerous quantitative investigations. As such, I am both interested in the development of qualitative methodology within psychology and appreciative of the value of and enormous contribution made through quantitative research. The research collaborator is in his first year of doctoral studies in applied social psychology, has conducted primarily quantitative investigations, and
131
has had little exposure to qualitative methods. The technique of peer debriefing offers the assurance that the interpretation has been found convincing by other investigators who have reviewed all of the source material. Because of the focused and brief nature of the present interviews, this process was atypically uncomplicated with agreement regarding both category content and structure.
Results PsycINFO Search The frequencies of research articles generated through the PsycINFO search using the keyword qualitative for the years 1989, 1994, and 1999 can be found in Figure 1. The results were 155 for 1989, 365 for 1994, and 1,024 for 1999. Given that these numbers more than doubled over each interval, it appears that the amount of qualitative research being conducted and published in the social sciences is rapidly increasing.
APA Journal Analysis Overall, the results indicate that qualitative research is rarely seen in APA journals. Thirty-two qualitative research articles were identified from a total of 2,402 (1.3%). If one removes clinical case studies (n ⳱ 14), which are considered qualitative but have a tradition in psychology, one is left with 18 qualitative studies (< 1%). These 18 studies consist of 2 Q-Sort procedures, 5 grounded theory studies, 1 ethnographic study, and 10 studies that incorporate multirater coding strategies, 4 of which mix quantitative and qualitative findings. Although the number of
Figure 1. Number of research studies incorporating qualitative methods generated by using a PsycINFO search with the keyword qualitative.
132 qualitative studies marginally increased over the 10year span, the frequencies were too small to make any conclusive statements.
Mission Statement Content Analysis The following are the results of the coding of journal mission statements on the basis of their apparent inclusiveness or exclusiveness toward qualitative methods (see Table 2). The categories developed were exclusive, neutral, and inclusive. Exclusive mission statements included material such as “journal devoted to furthering an understanding of scientific relationships between behavioural principles on the one hand and physical health and illness on the other.” An example of an inclusive guideline is “both quantitative and qualitative methods are appropriate.” Agreement between the two coders was 93%, with disagreement subsequently negotiated. Four journals fell into the exclusive category; 10 journals were placed in the neutral category, and 1 journal fell into the inclusive category.
Interviews With Chief Editors Data quality and representation. As mentioned previously, 7 of the 10 interviews were brief by qualitative standards. This was likely due to many of the editors’ admitted unfamiliarity with the topic, leaving them without a lot to say about the method and related issues. This lack of familiarity is a notable finding. Although not verbose on the matter, all of the editors had opinions and described their beliefs regarding the Table 2 Results of Mission Statement Content Analysis Exclusive Developmental Psychology Health Psychology Journal of Family Psychology Journal of Personality and Social Psychology Neutral Journal of Abnormal Psychology European Psychologist Journal of Consulting and Clinical Psychology Journal of Educational Psychology Journal of Occupational Health Psychology Professional Psychology: Research and Practice Psychology and Aging Cultural Diversity and Ethnic Minority Psychology Psychology of Addictive Behaviors Rehabilitation Psychology Inclusive Journal of Counseling Psychology
KIDD
place of qualitative methods in psychology and in their journals. All editors reported an interest in the project and appeared to make an effort to attend to the topics raised to the best of their ability. Another potential concern is that 10 out of 15 editors were interviewed. The intention to interview all 15 editors could not be accomplished because of an inability to contact them. The reasons for their unavailability are unclear, and it must be acknowledged that a sampling bias may exist. The analysis. The results of the content analysis of the editors’ comments on qualitative research are organized as follows: Presented first are categories and subcategories into which the editors’ thoughts on qualitative research were placed. These categories are (a) the value of qualitative research, (b) weaknesses of qualitative research, (c) improvement on current practices in qualitative research, and (d) role in editor’s journal. Following the presentation of these categories, the theme that runs across the categories is discussed. This theme represents the editors’ overall attitude toward qualitative research. The value of qualitative research. This category was divided into two subcategories: (a) qualitative research has strengths in areas in which quantitative research is limited and (b) qualitative research is advantageous as a part of a larger research program. Within the first subcategory were two concepts. Six participants regarded qualitative research as potentially “richer” and useful in finding relationships/ variables and meaning that are difficult to access with quantitative methods. The general message was that these editors thought that qualitative methodologies provided a type of inquiry that could discover and illuminate areas that quantitative methodologies cannot access because of inherent limitations: I think that in using qualitative research it is possible to find relationships and variables you can’t typically get at with more quantitative research. It is a better way of getting at meaning, at how people construe their experiences and what those experiences mean to them. That’s often difficult to capture statistically or with more quantitative methods.
Five participants stated that qualitative research was useful for special populations/problems. The primary issue was that generating large numbers of participants or appropriate experimental conditions (i.e., quantitative designs) was either impractical or impossible in many clinical and other applied situations.
QUALITATIVE RESEARCH IN PSYCHOLOGICAL JOURNALS
One editor expressed his frustration that there exists a large amount of valuable information in clinical settings that is never used because the nature of that information would not be considered valid in the context of quantitative designs: “It is especially valuable, given the wide variety of categories of disability that we deal with and the many different personality characteristics that are often very difficult to get in a sizeable group.” The second subcategory falling under the “value of qualitative research” is the premise that qualitative research can serve a useful purpose as a part of a larger research program that includes quantitative analysis. This point arose from the categories that indicate that qualitative research has strengths in areas in which quantitative methods are limited. As a result, the two approaches to inquiry can complement one another in a manner that can greatly strengthen the research program as a whole. Seven participants communicated this point. According to one, “It certainly has a role. I think that it is always good to use many kinds of methods to look at a problem . . . dovetailing methods using qualitative procedures.” Another participant said, “I see it as a very good source for generating hypotheses which would then be tested with more systematic and more paradigmatically acceptable methodologies.” Weaknesses of qualitative research. Another category that arose involved the participants’ thoughts about problems that have a substantial impact on the usefulness and applicability of qualitative research. The two subcategories that emerged were methodological issues and paradigmatic differences. On the subject of methodology, the two major areas of criticism were generalizability (brought up by 4 participants), and questions about bias/validity and related paradigmatic issues (2 participants). Regarding generalizability, the concern raised was how useful qualitative results were beyond the small and select group of participants in a given study. One participant said, “The major concern I would have here is that the results of such research without either statistics or careful description of the sample might prove to be rather non-generalizeable.” Another participant commented, “The weakness I see is that it tends not to use large samples, and regardless of whether it is qualitative or quantitative you run into potential problems of representativeness and generality.” Questions of bias and validity were framed as a guardedness regarding whether information not obtained through scientific inquiry could be trusted as
133
being more than just a reflection of the beliefs and goals of the researcher. The 2 participants who voiced this concern linked the validity problem with more general paradigmatic clashes. For these editors, it appeared that the identity and the foundation of psychology were closely linked to the quantitative scientific paradigm and to consider qualitative approaches would be to compromise such foundational principles and beliefs. According to one of these editors, As a psychologist by definition, I think we are a science of human behavior. As a science I think we need to pay attention to human bias. And I think if we are going to pay attention to making sense of data and be cautious to avoid the potential unwanted human bias, we are going to need large sample size and the benefit of some statistical testing, and reliability and validity.
Another editor said, I think the history and tradition of psychology is in publishing empirical work and statistical method . . . at this point I would not be ready to publish it because it would be a fairly significant change in the type of material traditionally published.
Improvement on current practices in qualitative research. This category arose as some of the participants spoke about changes that they felt would improve the position of qualitative research in psychology. Although only a few participants brought up these points, they were felt to be valuable given the current state of development of qualitative methodologies in psychology. There were three subcategories under this heading. First, 2 participants spoke of the need to develop more sophisticated methods. They felt that the current level of knowledge and practice of qualitative methods in psychology needed to be developed both in the classroom and in general practice. One of these participants said, “I think that the development of more sophisticated methods will play a big role. I know that it is being taught more in graduate programs now.” One editor’s comments formed the second subcategory, which was the need for investigators to link their qualitative work with current theory in a clear and creative manner. This participant felt that if it could be demonstrated how qualitative research can contribute to and help advance existing psychological theory, this would help in developing a case for the acceptance of such methods in psychology: It is a matter of in a couple of areas having investigators use it creatively so that its worth can become more salient or clear. It has to be married with a creative use of
134 theory so that the information can be understood, and the importance can be made more salient.
Finally, the third subcategory was formed by 1 participant’s opinion that a major step toward advancing the position of qualitative methodologies in psychology would be the endorsement of such methods by a recognized official body such as APA: To my mind it [call for qualitative submissions] required a policy change in terms of the type of research that the journal would publish, the kind of change that would require approval by APA. I was not prepared to request that kind of policy change.
Role in editor’s journal. This category arose as the participants discussed the place qualitative research had or did not have in the journal they were editing. The first point that arose in this category was the opinion of 4 editors that among the psychological research community there was a growing acceptance or openness toward the use of qualitative methodology. One editor felt that this was due at least in part to a less rigid adherence to the values and practices of the scientific paradigm: “I think that in general it is becoming more acceptable. It is certainly not in the mainstream psychological literature very much, but I think it is gaining acceptance.” Another editor said that “the role of qualitative methods in psychology is increasing as psychology becomes less rigid.” The second subcategory concerned the quality of the qualitative work received. Six editors stated that they did not feel that qualitative research was inferior or inherently flawed and would treat such submissions in exactly the same manner they would quantitative work. In other words, if the work met the standards of good research practice as based on criteria developed within qualitative theory, they would not hesitate to publish it: “We are in no ways biased against qualitative research. We in no way demand that people do research of a positivist nature. We just insist that it be good research no matter what type it is.” Finally, 4 participants stated that they did not think that their journals were appropriate forums for qualitative work. They did not entirely dismiss qualitative research, but thought that paradigmatic and methodological differences could not be negotiated or reconciled. According to one of the participants, I think if you had a qualitative paper that was somehow going to spark incredible interest and be generative and exciting and ignite a whole new area that I would expect would then be much more quantitative in focus. It would probably be a tough sell.
KIDD
Another participant indicated the following: Unless there is a real strong rationale, and real favor among reviewers, it is not likely to get published in our journal. That would be like sending a narrative description to Science. The reviewers would say nice work, but it doesn’t belong here.
Central theme. The theme that ran across the 10 interviews was a general acceptance of the usefulness of qualitative methods with reservations of varying degrees influencing whether they believed that such work should find a forum in their journals. Of the 10 editors, 5 felt that qualitative methods had significant strengths, were relevant, and said that they would publish such work in their journals. One editor believed that the weaknesses were a liability but would consider publishing such work. Four editors expressed that while such studies have strengths, the weaknesses limit the applicability of qualitative methods to psychology and that they would not consider publishing such work. No participant stated that they believed that qualitative methods were fatally flawed. All agreed that this approach could be conceived of as a useful source of information.
Discussion The results of the present study bring to light a contradiction between the status of qualitative research in APA publications and the field at large. As the journal content analysis indicated, few qualitative studies are being published in APA journals and there is little indication that this situation has changed in the past 10 years. The overall percentage of qualitative publication in the selected journals was approximately 1% and 5 of the 15 journals examined have not published any qualitative work. This continued paucity of qualitative work in APA publications exists despite an apparently dramatic increase in the number of qualitative studies being conducted in psychology and related social science fields. The number of qualitative studies generated by the PsycINFO search more than doubled between 1989 and 1994 and, again, more than doubled between 1994 and 1999. This finding supports the often stated but seldom studied hypothesis that the amount of qualitative research being conducted and published is rapidly increasing in the social sciences (Elliot et al., 1999; Rennie, 2000; Strean, 1998). This lack of qualitative publication exists despite the finding that several chief editors of these APA journals appeared interested in and willing to publish
QUALITATIVE RESEARCH IN PSYCHOLOGICAL JOURNALS
research studies in which qualitative methods are used. The interviews revealed that all of the chief editors interviewed recognized a number of the strengths of qualitative designs, with 6 editors believing that these strengths were substantial enough to warrant inclusion in their journals. These editors saw qualitative research as being a valid way of accessing meaning/experience, relationships, and variables difficult to illuminate with quantitative designs. Furthermore, several editors recognized the strength of qualitative methods in addressing special problems and populations, and they particularly saw qualitative research’s potential as part of a larger research program. The interest in qualitative research expressed by the editors in the current study appears to be consistent with the increasing interest commented on elsewhere (Krahn et al., 1995; Lee et al., 1999; Stiles, 1993). The discrepancy between a lack of qualitative publication and editor interest is likely influenced to a large degree by an apparent lack of clarity in these journals’ (and APA’s) position regarding qualitative research. Qualitative researchers are probably not going to be inclined to submit their work to journals that have in the past published little if any qualitative work and that do not encourage such submissions in their mission statements. Most of the mission statements examined in the present study did not indicate whether they were interested in qualitative submissions. Indeed, four of the mission statements analyzed in the present study indicated a heavy emphasis on empirical and scientific study, despite the fact that two of them have published qualitative work. Only 1 editor interviewed in the present study reported receiving qualitative submissions in substantial numbers. Such a small number of submissions is obviously going to limit the number of publications, regardless of the enthusiasm of the reviewers for qualitative research. Additionally, such small numbers of qualitative submissions are not likely to demonstrate to more skeptical editors and reviewers the potential of such research. Thus, there exists what appears to be a large gap between the impression given by a review of the journals and the interests and opinions of several of the chief editors. This argument must be qualified, however, by the fact that there are relatively few researchers using qualitative methods compared with the number of researchers conducting quantitative studies (Lee et al., 1999). To expect, therefore, large numbers of qualitative studies to be published in psychology journals would not represent the current state of psychological research.
135
Although the previously indicated findings may be encouraging to qualitative researchers wishing to publish in APA journals, the interviews revealed a considerable amount of criticism of the methodology. Four of the editors interviewed felt that the weaknesses of qualitative work were problematic enough to make it unlikely that they would ever accept a qualitative submission. The problematic areas were identified as (a) a lack of generalizability, (b) concerns about bias and validity, and (c) paradigmatic differences. The paradigmatic or epistemological differences between positivist and more constructivist philosophies have been extensively commented on as being the central impediment to an acceptance of qualitative methodology (Langenhove, 1995). Stemming from this metaphysical basis for disagreement are more specific concerns such as generalizability and validity. Regarding the lack of generalizability, this appears to be related to the previously noted tendency to apply quantitative criteria to qualitative studies (Elliot et al., 1999; Goldman, 1993; Rogers, 2000). Most qualitative researchers believe that generalizability, as it is understood in quantitative designs in terms of external validity, is not relevant to qualitative research (Janesick, 2000; Kazdin, 1998; Merrick, 1999). The criticism regarding generalizability is also inappropriate because many qualitative researchers are interested in generalizability of their findings, although in a different way than what is meant in the quantitative context. Qualitative researchers are often interested in if and how the themes generated in their studies might apply to other types and groups of individuals. In-depth descriptions of findings are encouraged to aid in the process of determining how such findings or elements may be found among or transferred to another individual(s) (Merrick, 1999). As such, this criticism is not only inappropriate to this kind of research but also ill informed and likely stems from epistemological differences. The concerns about bias and validity are shared by many qualitative researchers from a different perspective (for a more detailed description, see Gergen & Gergen, 2000; Merrick, 1999). In recent years, there has been a proliferation of articles and books describing procedures and practices designed to address this issue (Maxwell, 1998; Stiles, 1993; Strauss & Corbin, 1990). Most qualitative researchers struggle to develop and utilize methodologies that result in more accurate and truthful representations of the experiences of their participants. For 2 of the editors interviewed, however, this problem appeared to be deeply
136 embedded within qualitative design and again likely indicates an epistemological clash rather than a particular methodological question. There appears to be a growing number of researchers in the field of psychology who are utilizing qualitative methods and working within the theoretical paradigms associated with these methods. This movement is complemented by an openness and interest in established quarters of psychology as seen in the comments and views of several chief editors of major journals in the field. The continuing development of qualitative methodology in psychology is, however, being hampered by skepticism and doubt regarding the validity of such methods and also by the small amount of qualitative work being sent to journals where it would be received and reviewed as potentially informative and important. This is a situation found in the histories of several other fields (Denzin & Lincoln, 2000a; Vidich & Lyman, 2000) from which psychology, by examining such histories, could benefit. First, there must exist a substantial body of qualitative research meeting high standards so that its utility and possibilities can be explored. In psychology, the interest and practice appear to exist but likely need to be matched with improved instruction in universities (Elliot et al., 1999; Krahn et al., 1995) and the development of a body of literature describing how to conduct such research within psychology. Second, the skepticism toward such methods within the field must be addressed, and established entities and organizations within the field of psychology must develop a clearer position on the topic. This will likely involve debate and demonstration of research in conferences, the publication in major forums of studies of the highest quality that clearly demonstrate the value of such work, endorsement and practice of qualitative research by major figures in the field, and policy and position statements made by bodies such as APA. If the histories of other disciplines may be applied to psychology, the above points will be important elements in the establishment of qualitative methods in psychology. In summary, in the present study I have found that there exists a substantial amount of interest in mainstream psychology regarding the potential contribution of qualitative research. Qualitative studies, if only in small numbers, are being published in APA journals, and 6 of the 10 chief editors interviewed stated that they would seriously consider qualitative submissions for publication. There are, however, several elements likely impeding a more thorough exploration
KIDD
of the potential of qualitative methodologies in psychology. First, the position of mainstream psychology on the subject appears to be unclear. Both acceptance and serious reservations are found readily with journal descriptions, and the number of qualitative publications currently shed little light on the situation. This ambiguity may be due, in part, to the problem that most psychologists are unfamiliar with qualitative methods and, therefore, are unsure of how to approach research of this nature. Second, there appears to exist a significant and, for some, insurmountable paradigmatic clash between traditional empirical psychological study and the more interpretivist philosophies of qualitative work. Third, and likely related to the above two points, insufficient qualitative research is being sent to these journals in order for it to be adequately represented or assessed as to the potential contribution of such methods. The amount of qualitative research being conducted is rapidly increasing. Concurrent to this is an interest in this research as expressed by several major psychological journals and their editors. An appeal could be made to approach the problem as scientists. This will entail looking past one’s personal philosophical differences and testing the degree to which this tool can contribute to the study of human behavior. Observing a significant number of such studies and thoroughly investigating their strengths and weaknesses is essential. It is only after engaging in such a process that a strong case can be made for supporting or refuting this set of methods.
References Azar, B. (1999, February). Consortium of editors pushes shift in child research method. Retrieved February 28, 2000, from http://www.apa.org/monitor/feb99/qual.html Banyard, V. L., & Miller, K. E. (1998). The powerful potential of qualitative research for community psychology. American Journal of Community Psychology, 26, 485– 505. Burman, E. (1996). Continuities and discontinuities in interpretive and textual approaches in developmental psychology. Human Development, 39, 330–345. Denzin, N. K., & Lincoln, Y. S. (2000a). Introduction: The discipline and practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (2nd ed.), Handbook of qualitative research (pp. 1–29). Thousand Oaks, CA: Sage. Denzin, N. K., & Lincoln, Y. S. (2000b) Handbook of qualitative research. Thousand Oaks, CA: Sage.
QUALITATIVE RESEARCH IN PSYCHOLOGICAL JOURNALS Elliot, R., Fischer, C. T., & Rennie, D. L. (1999). Evolving guidelines for publication of qualitative research studies in psychology and related fields. British Journal of Clinical Psychology, 38, 215–229. Fiese, B. H., & Bickham, N. L. (1998). Qualitative inquiry: An overview for pediatric psychology. Journal of Pediatric Psychology, 23, 79–86. Gergen, M. M., & Gergen, K. J. (2000). Qualitative inquiry: Tensions and transformations. In N. K. Denzin & Y. S. Lincoln (2nd ed.), Handbook of qualitative research (pp. 1025–1046). Thousand Oaks, CA: Sage. Glaser, B. J., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago: Sociology Press. Goldman, L. (1993). Reaction: A broader scientific base for professional psychology. Professional Psychology: Research and Practice, 24, 252–253. Janesick, V. J. (2000). The choreography of qualitative research design: Minuets, improvisations, and crystallization. In N. K. Denzin & Y. S. Lincoln (2nd ed.), Handbook of qualitative research (pp. 379–400). Thousand Oaks, CA: Sage. Kazdin, A. E. (1998). Research design in clinical psychology. Boston: Allyn & Bacon. Keeley, S. M., Shemberg, K. M., & Zaynor, L. (1988). Dissertation research in clinical psychology: Beyond positivism? Professional Psychology: Research and Practice, 19, 216–222. Krahn, G. L., Hohn, M. F., & Kime, C. (1995). Incorporating qualitative approaches into clinical child psychology research. Journal of Clinical Child Psychology, 24, 204– 213. Lather, P. (1991). Getting smart: Feminist research and pedagogy within the postmodern. New York: Routledge. Langenhove, L. V. (1995). The theoretical foundations of experimental psychology and its alternatives. In J. A. Smith, R. Harre, & L. V. Langenhove (Eds.), Rethinking psychology (pp. 10–23). Thousand Oaks, CA: Sage.
137
Denzin & Y. S. Lincoln (2nd ed.), Handbook of qualitative research (pp. 163–188). Thousand Oaks, CA: Sage. Maxwell, J. A. (1998). Designing a qualitative study. In L. Bickman & D. S. Rog (Eds.), Handbook of applied social research methods (pp. 69–100). Thousand Oaks, CA: Sage. Merrick, E. (1999). An exploration of quality in qualitative research. In M. Kopala & L. A. Suzuki (Eds.), Using qualitative methods in psychology (pp. 25–36). Thousand Oaks, CA: Sage. Patton, M. Q. (1980). Qualitative evaluation methods. Thousand Oaks, CA: Sage. Rennie, D. L. (1999). Qualitative research: A matter of hermeneutics and the sociology of knowledge. In M. Kopala & L. A. Suzuki (Eds.), Using qualitative methods in psychology (pp. 3–14). Thousand Oaks, CA: Sage. Rennie, D. L. (2000, June). The rise of qualitative research in psychology. Paper presented at the annual meeting of the Canadian Psychological Association, Ottawa, Ontario, Canada. Rennie, D. L., Phillips, J. R., & Quartaro, G. K. (1988). Grounded theory: A promising approach to conceptualization in psychology? Canadian Psychology, 29, 139– 150. Rogers, A. G. (2000). When methods matter: Qualitative research issues in psychology. Harvard Educational Review, 70, 75–85. Schwandt, T. A. (1996). Farewell to criteriology. Qualitative Inquiry, 2, 59–73. Shank, G. (1994). Shaping qualitative research in educational psychology. Contemporary Educational Psychology, 19, 340–359. Smith, J. A. (1995). Semi-structured interviewing and qualitative analysis. In J. A. Smith, R. Harre, & L. V. Langenhove (Eds.), Rethinking methods in psychology (pp. 9–26). Thousand Oaks, CA: Sage. Smith, J. A., Harre, R., & Langenhove, L. V. (1995a). Rethinking methods in psychology. Thousand Oaks, CA: Sage.
LeCompte, M. D., Millroy, W. L., & Preissle, J. (1992). Preface. In M. D. LeCompte, W. L. Millroy, & J. Preissle (Eds.), The handbook of qualitative research in education (pp. xv–xvi). London: Academic Press.
Smith, J. A., Harre, R., & Langenhove, L. V. (1995b). Introduction. In J. A. Smith, R. Harre, & L. V. Langenhove (Eds.), Rethinking methods in psychology (pp. 1–8). Thousand Oaks, CA: Sage.
Lee, T. W., Mitchell, T. R., & Sablynski, C. J. (1999). Qualitative research in organizational and vocational psychology. Journal of Vocational Behavior, 55, 161– 187.
Stewart, A. (1998). The ethnographer’s method. Thousand Oaks, CA: Sage. Stiles, W. B. (1993). Quality control in qualitative research. Clinical Psychology Review, 13, 593–618.
Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, contradictions, and emerging confluences. In N. K.
Strauss, A., & Corbin, J. (1990). Basics of qualitative research. Newbury Park, CA: Sage.
138 Strean, W. B. (1998). Possibilities for qualitative research in sport psychology. The Sport Psychologist, 12, 333–345. Streubert, H. J., & Carpenter, D. R. (1995). Qualitative research in nursing. Philadelphia: Lippincott. Vidich, A. J., & Lyman, S. M. (2000). Qualitative methods: Their history in sociology and anthropology. In N. K. Denzin & Y. S. Lincoln (2nd ed.), Handbook of qualitative research (pp. 37–84). Thousand Oaks, CA: Sage.
KIDD Zeller, N., & Farmer, F. M. (1999). “Catchy, clever titles are not acceptable”: Style, APA, and qualitative reporting. International Journal of Qualitative Studies in Education, 12, 3–19.
Received October 24, 2000 Revision received August 21, 2001 Accepted August 21, 2001 ■
