International Journal of Qualitative Methods 5 (1) March 2006

Printable PDF Version

Processes of Metastudy: A Study of Psychosocial Adaptation to Childhood Chronic Health Conditions

David B. Nicholas, Judith Globerman, Beverley J. Antle, Ted McNeill, and Lucyna M. Lach


David B. Nicholas, PhD, Academic and Clinical Specialist, Department of Social Work; Associate Scientist, Research Institute, The Hospital for Sick Children, Toronto, Canada

Judith Globerman, PhD, Executive Leader, Care Services, Dr. Irving & Phyliss Snider Campus for Jewish Seniors; Associate Professor, Institute for Health Promotion Research, University of British Columbia, Vancouver, Canada

Beverley J. Antle, PhD, Academic & Clinical Specialist, Department of Social Work; Director, PKU Program, The Hospital for Sick Children, Toronto, Canada

Ted McNeill, PhD, Director, Department of Social Work, The Hospital for Sick Children, Toronto, Canada

Lucyna M. Lach, PhD, Assistant Professor, School of Social Work, McGill University, Montreal, Canada

 

Abstract: Metastudy introduces a systematically aggregated interpretive portrayal of a body of literature, based on saturation and the synthesis of findings. In this metastudy, the authors examined qualitative studies addressing psychosocial adaptation to childhood chronic health conditions, published over a 30-year period (1970-2000). They describe metastudy processes, including study identification, strategies for study search and retrieval, adjudication of difference in study design and rigor, and analysis of findings. They also illustrate metastudy components through examples drawn from this project and discuss implications for practice and recommendations.

Keywords: metastudy, pediatrics, chronic health conditions, psychosocial adaptation


Authors’ note
The authors gratefully acknowledge the funding from the Social Sciences and Humanities Research Council of Canada.

Citation
Nicholas, D. B., Globerman, J., Antle, B. J., McNeill, T., & Lach, L. M. (2006). Processes of metastudy: A study of psychosocial adaptation to childhood chronic health conditions. International Journal of Qualitative Methods, 5(1), Article 5. Retrieved [date] from http://www.ualberta.ca/~iiqm/backissues/5_1/html/nicholas.htm


Introduction

A relatively recent addition to the qualitative research literature is metastudy procedures. Metastudy offers a synthesis of individual qualitative studies within a selected substantive area. In this project, a metastudy of qualitative literature addressing psychosocial adaptation to childhood chronic health conditions, we reviewed research published over a 30-year period, 1970-2000. In this article, we describe methodological processes and summary findings from this comprehensive review.

Background

Recent efforts to operationalize metalevel review of qualitative literature have appeared under various headings, including systematic review of nonrandom and qualitative research literature (Lemmer, Grellier, & Steven, 1999), meta-ethnography (Noblit & Hare, 1988), and metastudy (Paterson, Thorne, Canam, & Jillings, 2001; Zhao, 1991). These notions comprise systematic aggregate reviews of individual research sources for the purpose of evaluating the comparability of studies (Jensen & Allen, 1996), and developing new and cumulative understandings (Thorne & Paterson, 1998). Accordingly, Noblit and Hare (1988), Thorne and Paterson (1998), and, later, Paterson and colleagues (2001) introduced multifactorial reviews of studies on the basis of theoretical orientation, content, methodology, and contextual notions such as geographical, sociohistorical, and political environments; author discipline; and funding source.

This multidimensional review process offers diverse vantage points for the aggregate review of individual studies. Accordingly, metastudy overcomes inherent limits of applicability within single qualitative studies. For instance, the lack of generalizability in a qualitative study is tempered by the emergence of comparable findings within diverse samples and settings. Variation can be accounted for, and the theoretical bases of studies, including epistemology, philosophical stance, subjectivity, reflexivity, and researcher background and affiliation, can be critically examined.

Inherent in metalevel procedures are assumptions about the interlinking and influencing nature of theory and context in the yield and interpretation of qualitative findings. Paterson and colleagues (2001) asserted that metastudy encompasses not only a systematic review of qualitative research results but also inherent and, perhaps more cogently, sociohistoric, paradigmatic, tangential, and idiosyncratic perspectives imposed on “understandings” of a topic at a given point in time and location.

Although metastudy offers promise in seeking new knowledge based on aggregated findings, limitations to this approach also exist. By nature of its secondary level of analysis, metastudy invites heightened abstraction and dissociation from the words, ways of expression, and emotionality of study participants. The readership of a metastudy, therefore, is limited by multiple levels of abstraction and interpretation; first, that imposed by the primary researcher on the researched and, second, further abstraction imposed by the metastudy reviewers of the published findings (Paterson et al., 2001).

Given vast inconsistencies in approaches, methodologies, sample details, data, and other characteristics, possible vagueness or confusion can emerge in metalevel findings. Notwithstanding this risk, a literature base intuitively invites cumulative evidence for knowledge advancement and best practices. These aims entail the synthesis of qualitative evidence, for which metastudy offers a set of procedures.

This metastudy examined the literature addressing psychosocial adaptation to pediatric chronic health conditions. It entailed a formidable process comprising identification, retrieval, and analysis of relevant qualitative studies over a 30-year time frame (1970-2000). In this article, we outline specific methodological components undertaken in conducting this metastudy and present examples of findings for each metastudy component. Key components were (a) delineation of study inclusion, (b) study search and retrieval, and (c) analysis. Systematic implementation of metastudy components ultimately culminates in the hermeneutical portrayal of a literature involving review of presented data, cumulative interpretation, and synthesis of findings. Each component is outlined and exemplified below.

Delineation of study inclusion

In preparation for the selection of studies to be included in a metastudy, the operationalization of key variables and relevant retrieval strategies is required. Careful thought is crucial, as key terms, definitions, and parameters for study eligibility constitute the bedrock for how the metastudy will unfold. In our current metastudy examining pediatric psychosocial adaptation, we determined that peer-reviewed journals were required to ensure a level of researcher accountability. The substantive focus was defined as pediatric psychosocial adaptation to chronic conditions, including physical illness or disability lasting in excess of 3 months. Studies in which the focus was primarily on developmental disability or cognitive delay were excluded because of clinical concerns that these experiences and processes of adaptation might substantially differ from those experienced by children with primarily physical conditions. Initial constructs and keywords associated with psychosocial adaptation were based on solicited expert opinion among relevant clinicians and researchers. Following this determination of key constructs, we initiated electronic database searches.

Study search and retrieval

Article search and retrieval involve systematic processes of identifying and accessing relevant studies for subsequent review. An extensive process of sifting an unwieldy amount of literature is invariably part of metastudy, particularly if the substantive area is relatively broad in scope, as was the case in the current project. Specifically in the pediatric adaptation metastudy, electronic searches yielded 18,722 peer-reviewed journal articles based on searches within relevant electronic retrieval databases (PsychInfo, 3,889; MedLine, 4,284; CINAHL, 31; Social Work Abstracts, 1,125; Embase, 9,443).1 Article abstracts were reviewed for substantive relevance and qualitative design. If it met inclusion criteria, the article was obtained and subsequently reviewed.

This review process was monumental in scope and required several weeks of devoted research assistant time as well as close supervision by the investigative team. Of the literature reviewed, 112 studies met our eligibility criteria, which required topic relevance, qualitative method, and the inclusion of data within the study (e.g., themes, quotes). Established criteria reflected both methodological and substantive considerations. In this metastudy, the research team attempted to avoid conceptually based commentaries and case studies in favor of studies based on a minimum of four participants. These criteria encompassed commonly used qualitative approaches, including phenomenology, ethnography, and grounded theory.

Included studies were reviewed for demographic content (e.g., author discipline, geographic region, sample, population identifiers), qualitative research approach, data collection and analysis methods, rigor standards, and findings. Given the volume of articles and the extensive study-by-study detail being extracted and managed, findings were entered into a database that had been developed to capture and ultimately contrast findings.

Metastudy analysis

Analysis required multilevel review of both content and processes of the study. Reviews were completed by a senior research assistant with graduate-level qualitative research training and extensive health research experience, and this review was closely supervised by the research team. To increase consistency in reviews, the research assistant and three experienced pediatric qualitative researchers independently reviewed a sample of articles. A 95% agreement rate was achieved in coding, and ultimately, 100% agreement in a coding scheme was achieved through a peer-debriefing Delphi process (Dawson & Bruker, 2001) in which initial differences were examined and recoded. This consensus strategy provided a coding framework for study-by-study review and elicited demographic and sample information, standards of rigor, and substantive findings, as outlined below.

Demographics and the sample. Review of demographic and sample data allowed for illumination of populations studied, trends over time, and emergent gaps in the literature. As an example from our metastudy of pediatric adaptation, the accumulation of demographic information revealed that the majority of reviewed studies had originated in North America. Within these North America studies, Canadian samples tended to be overrepresented relative to the population distribution in North America.

This analysis of demographic data highlights that a substantial proportion of this literature comes from a relatively stable, Western democratic society in which health care is universally accessible. From a global resource distribution perspective, speculation could be raised about the disproportionate, hence inequitable, sample distribution in this literature. Questions can be raised about what emerges as a marginalization or non-identification of voices from non-North American populations. Identifying such trends allows for critical review of the literature and recommendations for further research development.

Standards of rigor. A conundrum facing metastudy researchers involves how to grapple with varying standards of rigor. It cannot be assumed that all studies are completed with comparable qualitative research acumen, resulting in the apparent need to consider rigor within study-by-study analysis. Yet the task of subjecting studies to review of trustworthiness and authenticity is controversial, given shifts in perceptions of quality within naturalistic inquiry (Lincoln, 1995; Lincoln & Guba, 1985). Toward this end, Glaser and Strauss (1967), in their groundbreaking work on developing the grounded theory approach, introduced elements of rigor, namely theoretical sampling and saturation. Lincoln and Guba (1985) extended this discussion of quality through their conceptual development of trustworthiness, and, more recently, these criteria have been critiqued and expanded to encompass priorities of voice, context, and transparency, and to ensure authentic benefits to research participants (Lincoln, 1995; Ristock & Pennell, 1996; Rodwell & Woody, 1994).

To identify key criteria for evaluating rigor in the current study, a Delphi panel (Dawson & Bruker, 2001) of experienced qualitative health researchers was established to review the methodological literature and recommend relevant indicators of rigor. Consensus was achieved about key elements to include in this study-by-study review, based on a balance of established elements of trustworthiness and authenticity (Erlandson, Harris, Skipper, & Allen, 1993; Lincoln, 1995; Lincoln & Guba, 1985) and a simultaneous avoidance of overly narrow conceptualizations of rigor. In Table 1, we have itemized identified elements of rigor used in this review. However, all relevant studies were included regardless of differences in the application of rigor standards. Accordingly, the purpose of rigor assessment was to understand the application of rigor indicators within this literature, not to impose a tool to judge the inherent value of a given study or group of studies.

Element of Rigor

Definition of Each Element

Evidence “Indicator” of Rigor within an Article

Face validity

Work resonates with itself, addresses what is indicated

• Addresses the substantive area it indicated it would; consistency and logical flow of arguments—can follow the story

Trustworthiness

Credibility

• Prolonged engagement: engages in culture of participants—aware of participants
• Persistent observation—interviews of at least 1 hour or interviews over time
• Referential adequacy—tapes, transcriptions
• Negative case analysis—search for opposites or disconfirming situations, data, and/or literature
• Member checking—researcher went back to participants with findings
• Peer debriefing—coding and analysis discussed with others
• Confirmability—quotes used

 

Transferability

• Sample size explained and justified; purposive sampling explained
• Findings presented appropriately, e.g., percentages not used incorrectly; does not overgeneralize (sticks to data)
• Thick description—detailed description of sample and context; saturation addressed

 

Dependability/ confirmability

• Articulation of who collected data, when data were collected, and who analyzed data
• Form of data collection identified
• Explanation of method
• Audit trail: (a) record of memos; (b) can draw a picture of what researchers did, i.e., decisions made, processes

 

Triangulation

• Findings contrasted to other literature/data; variety of data collection methods, sources, or types of data; more than one person reviewed data

Reflexivity

Subjectivity, cultural review

• Self-awareness, bias, perspective of researcher articulated; reflexive journaling

Authenticity

 

• “Voice” is articulated, i.e., who speaks and for whom
• Stakeholders are involved in the project, e.g., advisory committee; participant is a “true” participant in the research process

Fairness

 

• Respect for/reciprocity with participants/stakeholders demonstrated
• All stakeholders have equal access to research process/benefits

Table 1. Elements of rigor, their definitions, and evidence of their presence in studies

Substantial differences were found in the use of rigor among the studies reviewed in the pediatric psychosocial adaptation metastudy. Trustworthiness was most frequently reported; and reflexivity, fairness, and authenticity were seldom identified. Methodological diversity within studies was also commonly noted. Inconsistencies between tenets of the identified research approach (e.g., grounded theory, ethnography, phenomenology) and the actual study design were found. As an example, grounded theory studies often failed to present evidence of the constant comparative method, theoretical sampling, and/or theoretical saturation.

Given that indicators of trustworthiness were the most prominent indicators of rigor in this literature, we focused on the extent to which trustworthiness was evident. In Table 2, we outline the extent to which elements of trustworthiness were present in the reviewed studies. We tallied a score by calculating the total number of indicators determining the presence of an element of rigor (in Table 1). For instance, the trustworthiness element of face validity had two potential indicators determining its presence within a given study (article addresses substantive area indicated; logic [consistency and logical flow of arguments]). The median face validity score was 1.5, and the mean was 2, suggesting a midpoint face validity of 1.5 out of a potential total of 2.0, and a mode (the most frequently occurring score) of 2.0. Accordingly, this body of literature frequently demonstrates face validity.

Element of Trustworthiness

Indicators Determining the Presence of Element

Median (Mode) Score

Face validity (2 potential indicators)

Addresses substantive area indicated

1.5 (2)

Logic: Consistency and logical flow of arguments

   

Credibility (7 potential indicators)

Persistent observation
Prolonged engagement
Referential adequacy: Taped sessionws
Referential adequacy 2: Transcription of taped sessions
Negative case analysis
Member checking
Peer debriefing

5 (6)

Triangulation (5 potential indicators

Findings compared and contrasted with other literature or data
Variety of data collection methods
Variety of data sources
Variety of types of data
More than one person reviewed data

3(2)

Dependability/confirmability (8 potential indicators)

Articulation of who collected the data
Articulation of when data were collected
Articulation of form of data collection
Explanation of method
Articulation of who conducted analysis

6 (7)

Transferability (4 potential indicators)

Sample size explained
Sample size justified
Saturation addressed
Thick description

2 (2)

Reflexivity (2 potential indicators)

Self-awareness: Perspective of researcher articulated
Reflexive journaling

0 (0)

 

Reflexive journaling

 

Fairness (3 potential indicators)

Respect for/reciprocity with participants or stakeholders
All participants have equal access to research process and benefits
Participants are partners in the research process

1 (1)

Authenticity (7 potential indicators)

Voice is articulated
Stakeholders are involved in the project
Participants are "true" participants in the research process
Ontological: Stakeholders have an enhanced understanding of their own reality
Educative: Stakeholders have increased their sensitivity to others' realities
Catalytic: Research goals include change in participants
Tactical: Stakeholders are empowered to "act" as a result of the research process.

1 (1)

Total RIGOR Score (38 potential indicators/total score)

Median of total from each domain

20 (20)

Table 2. Overview of domains for rating trustworthiness. The median (or midpoint of score) was used because of variability. The mode reflects most frequently occurring score.

The next element of trustworthiness reviewed was credibility. The presence of credibility in a single study could be demonstrated by a potential total of seven indicators (persistent observation, prolonged engagement, referential adequacy based on tape recording, referential adequacy based on transcriptions, negative case analysis, member checking, and peer debriefing). A median of 5 credibility indicators and a mode of 6 indicators were found based on this metalevel review. As illustrated, this metalevel analysis permitted us to determine that there is a relatively high presence of credibility in this literature. Furthermore, more traditional elements of qualitative research rigor, such as validity, credibility, triangulation, and dependability, have higher median scores. Transferability was much less evident in the articles, and recent concepts such as reflexivity, fairness, and authenticity were largely absent within this literature (see Table 2).

Substantive findings. Each study was reviewed for abstracted themes, which were imported into a text-based database and subsequently analyzed for themes, using NVivo qualitative research software (Richards, 1999). In Figure 1, we present two examples of abstracted themes from included articles.

Figure 1. Examples of themes abstracted from qualitative studies
Figure 1

McCracken’s (1988) “long interview” multiphased approach was adapted for analysis. Specifically, this involved coding for article themes, contrasting themes within and across articles, reduction of redundancy, and thematic/conceptual development. Emergent themes convey a rich, complex, nuanced, and multilayered perspective of pediatric psychosocial adaptation. The subjective experiences of living with a health condition were often conveyed using participant quotes and emotionality, characteristic of qualitative research.

To conduct content analysis for individual studies, the research team remained close to the words and ways of expression within individual study texts. Notwithstanding this commitment, the subjectivity of the metastudy research team could not help but influence this secondary examination. In disclosing author affiliation, the research team is made up of PhDs with a background in pediatric social work, nursing, physical therapy, and psychology. Cumulatively, they represent decades of experience as health care practitioners and/or academics specifically within pediatric health. They are located in Canada and the United States. Supervision of the research assistant, an MSW-trained (masters in social work) qualitative health researcher, involved close monitoring by key members of the research team.

These data cumulatively illuminated pediatric psychosocial adaptation as an ongoing process of constructively yet tenuously living with physical difference. This entails both living with uncertainty yet managing it, as well as deriving value and appreciated meanings despite the pervasive and difficult experience of a chronic condition. Theoretically derived critical approaches elucidated structural barriers and resource inequities experienced by patients and families. Interpretive approaches tended to be less critical in perspective, yet, regardless of author standpoint, most studies recognized tension between adversity and adaptation. Specific processes that appear to emerge in this lived experience include balancing the “good” and “bad,” grief, anger, hope, relief, continual cognitive reappraisal, emotional responsiveness, and action (e.g., tasks of care, relating to others, persevering). Accordingly, this literature heavily reflects dwelling in or “living in” the experience of adaptation.

Figure 2. Emerging framework of living with difference: Children with chronic health conditions and their families

Figure 1

In contrast, empirical and quantitative studies traditionally yield outcomes or indicators of adaptation such as academic performance, behavioral adjustment, and emotional well-being (Statistics Canada, 1996). These outcomes might be interpreted as the internalization of externally based achievements based on normative standards of child development. Yet, ill or disabled children and their parents in this qualitative literature identify a sense of perceived satisfaction and “okayness” with not achieving traditionally sought milestones. Rather, they describe living with adversity—and therein, find meaning, value and quality of life (see Figure 2), despite the imposed barriers and structural impediments.

This qualitative literature, therefore, appears to uphold and celebrate the integration of children’s lives with difference, such that the child and her or his family are accepted and unique, and have a viable sense of self. This sense of self integrates (a) interests and aptitudes; (b) barriers and challenges; (c) difference, discomfort, and inconvenience; (d) uncertainty; and (e) value and acceptance. This perspective appears to convey the chronically ill or disabled child and her or his family as active and engaged in their life and social world. This qualitative portrayal advances current models about outcomes, risks, and capacities of children with chronic health conditions and their families. It confronts stereotypes of diminished quality of life among ill or disabled children and their family, yet amplifies challenges and structural barriers faced by this population. Accordingly, this qualitative literature illuminates meanings, challenges, and strategies used by children and families as well as processes of living with chronic illness or disability.

Implications

Over the past several decades, models of pediatric adaptation to chronic health conditions have been presented in the literature (Kazak, 1987; Moos & Tsu, 1977; Pless & Pinkerton, 1975; Rutter, 1987; Thompson, Gill, Burbach, Keith, & Kinney, 1993; Thompson & Gustafson, 1996; Wallander, Varni, Babani, Banis, & Wilcox, 1989). These models often depict determinants and/or risk factors such as academic achievement, lack of behavioral problems, and emotional well-being. Although these common notions provide elements associated with adaptation, evidence from this metastudy also identifies the value of constructively “living with” illness, disability, and difference rather than focusing on the challenge of overcoming adversity. Accordingly, this metastudy demonstrates the need for conceptual frameworks that do not limit and distill adaptation to mere elements of impediment versus achievement. Psychosocial adaptation emerges as a life that is lived; one of incalculable value and meaning despite or along with varying adversity. Within the application of such frameworks in clinical practice, ameliorative efforts and resources must continually seek the removal of barriers yet, in doing so, barriers should not supercede emphasis on the personhood, development, and growth of these children and families.

From a methodological perspective, it is important to note that trustworthiness criteria, developed in the 1980s or earlier, are generally present within this literature. More recent authenticity criteria, such as participant benefit, reciprocity, and respect, have been introduced in the social sciences and, furthermore, fit well within a health paradigm. Yet, to date, they remain relatively absent in pediatric health qualitative research. Integrating these standards of authenticity and participant benefit is therefore warranted, which perhaps invites the advancement of models for rigor or accountability within pediatric health research.

Conclusion

Metastudy procedures offer substantial value in assessing and critically reviewing a body of literature. Specifically, metastudy permits the process of accumulating data, sifting through them, and drawing conclusions from a large and disparate body of literature. The ability to review systematically a diverse array of studies constitutes a formidable contribution to qualitative research methodology. Established findings as well as gaps and discontinuities in the literature can be postulated and the field thereby advanced.    

Toward these ends, we have demonstrated in this article metastudy processes as applied within a broad substantive area of pediatric psychosocial adaptation. The capacity of this metastudy for distilling new understandings, trends, and gaps within the literature has been exemplified. Based on this applied example, metastudy clearly offers promise for systematically and critically analyzing a large and diverse body of qualitative research.

Notes

1. Keywords used in electronic searches were developed based on clinical experience, and review of content and key words in articles. They included adjustment, well-being, quality of life, hardiness, psychological endurance, depression, anxiety, behavior, adaptive behavior, resilience, helplessness, body image, locus of control, self perception, self esteem, self confidence, self concept, self efficacy, motivation, achievement, independence, social adjustment, social acceptance, social discrimination, peer relations, friendships, social dating, social desirability, stigma, bullying/teasing, social integration, coping behavior, cognitive ability, social networks, psychological stress, social stress, social networks, family relations, marital relations, parent/child relations, academic achievement, performance, and school phobia. The search was then limited by combining the above key terms with (a) chronic illness, disabled/handicapped, physical disorders and (b) children, adolescents, young adults or family. back to text

References

Boyd, J. R., & Hunsberger, M. (1998). Chronically ill children coping with repeated hospitalizations: Their perceptions and suggested interventions. Journal of Pediatric Nursing: Nursing Care of Children & Families, 13(6), 330-342.

Dawson, M. D., & Bruker, P. S. (2001). The utility of the Delphi method in MFT research. American Journal of Family Therapy, 29(2), 125-140.

Erlandson, D., Harris, E., Skipper, B. L., & Allen, S. D. (1993). Doing naturalistic inquiry. Newbury Park, CA: Sage.

Glasser, B., & Strauss, A. (1967). The discovery of grounded theory. Chicago: Aldine.

Jensen, L. A., & Allen, M. N. (1996). Meta-synthesis of qualitative findings. Qualitative Health Research, 6, 553-560.

Kazak, A. E. (1987). Families with disabled children: Stress and social networks in three samples. Journal of Abnormal Child Psychology, 15(10), 137-146.

Lemmer, B., Grellier, R., & Steven, J. (1999). Systematic review of nonrandom and qualitative research literature: Exploring and uncovering an evidence base for health visiting and decision making. Qualitative Health Research, 9, 315-328.

Lincoln, Y. (1995). Emerging criteria for quality in qualitative and interpretive research. Qualitative Inquiry, 1(3), 275-289.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.

McCracken, G. (1988). The long interview. Thousand Oaks, CA: Sage.

Moos, R. H., & Tsu, U. D. (1977). The crisis of physical illness: An overview. In R. H. Moos (Ed.), Coping with physical illness (pp. 3-21). New York: Plenum.

Noblit, G. W., & Hare, R. D. (1988). Who’s afraid of meta-theory. Millennium, 23, 387-393.

Paterson, B. L., Thorne, S. E., Canam, C., & Jillings, C. (2001). Meta study of qualitative health research: A practical guide to meta analysis and synthesis. Thousand Oaks, CA: Sage.

Pless, I. B., & Pinkerton, P. (1975). Chronic childhood disorders: Promoting patterns of adjustment. Chicago: Year-Book Medical.

Richards, L. (1999). Using NVivo in qualitative research. Bundoora, Australia: SCOLARIS, Sage Publication Software.

Ristock, J. L., & Pennell, J. (1996). Community research as empowerment: Feminist links, postmodern interruptions. Toronto, Canada: Oxford University Press.

Robinson, C. A. (1985). Parents of hospitalized chronically ill children: Competency in question. Nursing Papers, 17(2), 59-68.

Rodwell, M. K., & Woody, D. (1994). Constructivist evaluation: The policy/practice context. New York: Columbia University Press.

Rutter, M. (1987). Psychosocial resilience and protective mechanisms. American Journal of Orthopsychiatry, 57, 316-331.

Statistics Canada. (1996). National longitudinal child and youth study. Ottawa: Government of Canada.

Thompson, R. J. J., Gill, K. M., Burbach, D. J., Keith, B. R., & Kinney, T. R. (1993). Psychological adjustment of mothers of children and adolescents with sickle cell disease: The role of stress, coping methods and family functioning. Journal of Pediatric Psychology, 18, 549-559.

Thompson, R. J. J., & Gustafson, K. E. (1996). Adaptation to chronic childhood illness. Washington, DC: American Psychological Association.

Thorne, S., & Paterson, B. (1998). Shifting images of chronic illness. Image: Journal of Nursing Scholarship, 30(2), 173-178.

Wallander, J. L., Varni, J. W., Babani, L., Banis, H. T., & Wilcox, K. T. (1989). Family resources as resistance factors for psychological maladjustment in chronically ill and handicapped children. Journal of Pediatric Psychology, 14(2), 157-173.

Zhao, S. (1991). Meta-theory, meta-method, meta data analysis: What, why and how? Sociological Perspectives, 34, 377-390.

 International Journal of Qualitative Methods 5 (1) March 2006