Printable PDF Version| International Journal of Qualitative Methods 4 (2) June 2005 |
Virtual Focus Groups: New Frontiers in Research
Lyn Turney and Catherine Pocknee
Lyn Turney, PhD, Lecturer in Sociology, Australian Centre for Emerging Technologies and Society, Swinburne University of Technology, Hawthorn, Victoria, Australia.
Catherine Pocknee, MEd Candidate, BEd, GDSE, Educational Development Advisor, Swinburne University of Technology, Hawthorn, Victoria, Australia.
Abstract: New information and communication technologies in the form of learning management systems provide unique and inventive opportunities for qualitative researchers. Their intrinsic ability to record discursive data in text format accurately and to provide safe, secure, and anonymous environments for participants makes them amenable for use as advanced research tools. In this article, the authors report on a collaborative project that tested the potential of online discussion boards for use in virtual focus groups. What the researchers found was that not only was the method theoretically sound, it actually enhanced their ability to connect with difficult-to-access populations that were disparately spread.
Keywords: virtual focus group, Blackboard, Internet research, qualitative research, difficult-to-access populations
Citation
Turney, L., & Pocknee, C. (2005). Virtual focus groups: New frontiers in research. International Journal of Qualitative Methods, 4(2), Article 3. Retrieved [insert date] from http://www.ualberta.ca/~iiqm/backissues/4_2/html/turney.htm
Authors’ note
We would like to acknowledge the valuable contribution of other members of the research teams: Professor Michael Gilding, Dr. Christine Critchley, Dr. Karen Farquarhson, Lisa Bakacs, Kerrie-Anne Butler, and Penelope Shields. In particular, the fine work of Lisa Bakacs and Penelope Shields in setting up both online projects requires special mention. They also freely contributed their thoughts and ideas about the process for this article.
Information and communications technologies (ICTs) in universities are predominantly language based and hence provide a myriad of opportunities for qualitative research (Burgess, 1995). They can accurately record synchronous and asynchronous dialogue and sort, retrieve, and analyze qualitative data using programs such as Qualpro, Ethnograph, and Nudist (Lim & Tan, 2001).
In this article, we report on a collaborative project that examined the potential of online discussion boards in Blackboard as virtual focus groups, a term used by Adler and Zarchin (2002), as a tool for formal qualitative research. In particular, we report on whether online discussion boards can be used to carry out theoretically sound focus group research; whether an online data collection method can facilitate the involvement in attitudinal research of difficult-to-access groups who are disparately spread and often hesitant to respond; and whether the method allows sensitive data collection through the provision of a secure, safe, and anonymous environment.1
We set out to identify a method for conducting the research that met the predetermined criteria of the broader research projects. In the first instance, this involved a review of the technical options available; followed by the development of the online environment, the identification and induction of participants into the online environment, and the sharing of the technical and online facilitation skills required to conduct the research.
Social researchers have begun to recognize the opportunities the Internet provides to recruit participants for their research projects. Online survey research has proliferated over the past few years (Pocknee & Robbie, 2002), enabling many more people to have their say and providing larger samples for market researchers, pollsters, and, more recently, academic researchers. Market researchers, the developers of the traditional face-to-face focus group method, have recently turned their attention to online, or virtual, focus group techniques (for example, Greenbaum, 2002). To date, social scientists, despite their enthusiastic uptake of face-to-face focus group method and their development and theorizing about it, have been slower to adopt this particular online method, yet using the Internet to conduct focus group interviews with particular interest groups has enormous potential for qualitative social research.
The virtual focus groups were 3 of 12 focus groups conducted following the Swinburne National Technology and Society Monitors in 2003 and 2004, two large random surveys of public attitudes toward a range of new technologies (ACETS, 2003, 2004). The 2003 survey included a raft of questions on DNA paternity testing, and the 2004 survey asked a similar number of questions about stem cell research. We followed up each survey up with six focus group interviews to gain a more detailed understanding of public perceptions. The virtual focus groups were used with participants who had a stake or special interest in the biotechnologies, whereas the face-to-face groups were members of the general public. The latter are not reported here.2 In each study, the researchers wanted, among other factors, to explore differences in attitudes among individuals who had a particular interest in each technology, DNA paternity testing or stem cell research. Accordingly, for the paternity testing study, we interviewed online a group of men’s rights leaders who advocate unrestricted and direct access to paternity testing. The second paternity testing group was mothers whose estranged partners had denied paternity. For the stem cell research project, the two interest groups involved were those with a particular interest in, or stance on, stem cell research. The first consisted of a religious group with strongly held views on abortion, and the second was a group of patients who had a medical condition that might be helped or cured by stem cell research. The latter was conducted online, but the religious focus group was conducted face-to-face and is not reported here. The virtual focus groups thus consisted of two groups, one male and one female, that had a personal stake in DNA paternity testing and a patient special interest group from the stem cell study. Both studies were approved by the University Human Research Ethics Committee.
We adopted the online method in the three groups because they all involved difficult-to-access populations. Participants in each group were geographically separated, so we employed various strategies, including e-mail, telephone, and snowball recruitment, to access them from around Australia.
Recruitment to the paternity testing stakeholder focus groups was particularly challenging. The researchers were advised that men who were involved in fathers’ rights groups were generally unprepared to speak to outsiders, and early attempts to recruit men for whom access paternity testing was of particular importance confirmed this to be the case. Eventually, we recruited these men from across Australia through a direct approach to a father’s rights group leader, purposive snowballing from this contact, and posting notices on relevant online sites and in chat groups.
The women were recruited through publicity related to the survey, a flyer issued through the Council for Single Mothers and Their Children, and notices in relevant online chat groups. The mothers and their children had been compelled to undergo testing to have the father’s name on the child’s birth certificate and/or to meet the requirements for claiming child support payments. All of the women were single mothers with responsibilities for young babies, which affected their ability to participate in a face-to-face focus group.
The stem cell study patient group was also recruited through e-mail and the posting of notices on relevant online sites and chat groups. The participants all had a medical condition that the outcomes of stem cell research promise to alleviate or cure. They were people living either with Parkinson’s disease or with spinal injury and were young enough to benefit personally from any promising developments in the near future. Their restricted mobility, medication, and need for care rendered it nearly impossible to meet in a face-to-face setting.
During the initial setup of each group, we had multiple direct contacts by telephone and/or e-mail with each participant (using their real identity) to ensure that they met the requirements for group membership, to negotiate time and availability, and to nominate a pseudonym password for their online persona. Therefore, although participants were anonymous to each other, they were each aware that the researcher mediated the space between their online and offline identities, leaving little scope for identity deception. Hine (2000) has argued, “The internet is only a space for identity play as far as the boundary between online and offline is sustained. If this boundary is broken down, the internet loses its radical potential [to create and use other personas]” (p. 120). The researcher precluded any such boundary by vetting identity and regulating entry to the site, so that participants could assume mutuality, authenticity, and a nonthreatening environment.
The three virtual focus groups were conducted through an Internet discussion board on Blackboard, involving asynchronous communication over a 1-week period. Computer-mediated communication allowed the researchers to cross time and space barriers (Mann & Stewart, 2000) to reach populations who had a particular stake in the technology. In all three cases, the online environment made it easy for the members of each group to speak openly and anonymously about a politically and personally sensitive issue in a way that would have been more difficult to achieve in face-to-face focus groups.
Given the difficulties associated with recruitment, we considered a number of options when the focus group planning took place. In the early stages of the project, the research team had expressed concerns about the security of data, citing some recent worries over participant responses to online surveys hosted by commercial companies. There are security measures that can be employed when using sites on commercial servers to collect data, such as vetting potential participants for authenticity, password protection, and the use of online pseudonyms (Oringderff, 2004). However, data collected in this way are not protected from unauthorized access by technical staff (Oregon Health & Science University [OHSU], 2001). The research group agreed that confidentiality and anonymity were of key importance if we were to gain meaningful responses from the participants as well as meet the National Health and Medical Research Council (NHMRC) requirements for the conduct of ethical research. To overcome these issues, an online approach seemed appropriate only if we could set up a secure site.
In consultation with the relevant departments, we decided to use existing university infrastructure. Our university, and, indeed, 41 out of 42 Australian universities, currently use a learning management system such as Blackboard or WebCT (numbers compiled from Bell, Bush, Nicholson, O’Brien, & Tran 2002; Smissen, 2002). We use Blackboard, which is run by the university’s Information Technology department and is programmed to allow access only to university-approved personnel. In its routine use, it provides academics and students with an environment that has extensive communication facilities, password protection, automated data collection, Internet access, and security features. For security reasons, the Blackboard system is hosted on internal university servers, and all data collected are subject to the university’s quality, security, and privacy procedures.
Given the confidential nature of the data to be collected for the trial studies, we decided that supervisory access would be limited to the research group, with personal e-mail details known only the research assistant who made the initial contact and the principal researcher. Participants were thus guaranteed that their personal information and views would be secure and in no way accessible by unauthorized personnel. Thus, two password-protected Web sites were developed in 2003, one for male participants and the other for female participants. We established a third site for the stem cell research group in early 2004. Each Web site had a discussion forum, an online chat room, and e-mail and announcement facilities. The coordinating researcher and moderators were registered in the sites as instructors, which gave them the opportunity to communicate fully with participants while still working at a higher technical level for data collation, record keeping, and system management. Participants were populated into the appropriate sites through self-nominated “name tags” (for example, Sandpiper, Coolcat), which ensured participant anonymity throughout the data collection phase. Each virtual focus group was moderated by two researchers, one who posted questions and probes and the other to respond to individual technical difficulties, should they arise.
We ran a number of preparatory sessions to train researchers and moderators in how online communication can be used effectively to collect data for research. There was extensive debate within the group on the key characteristics of focus groups (Krueger 1994) and how those characteristics could be transferred to an online environment. Once the participants had been inducted into the project, we provided them with detailed instructions on how to use the Blackboard system, discussion forums, and chat facilities well before the focus groups were run. This included a trial discussion in advance of the virtual focus group study proper, enabling them to have an experimental run to explore the site, test the rules and guidelines, and report to the moderator any technical difficulties they might be experiencing. We decided that the discussion forum would act as the virtual focus group, with the moderator posting a series of text based questions and probes, which would give the participants the opportunity to respond in text format. Responses were later downloaded for analysis. Participants had full access to the site for a 1-week period to post their responses to the questions, read other participants’ and the moderator’s probes, and reflect and post secondary comments. All participants (male, female, and patient groups) did take up the opportunity to reenter the site to read other participants’ and moderator postings and make further comment, although the patient group did so to a lesser degree.
To evaluate the efficacy of the virtual focus group method, we used as a measure Krueger’s (1994) criteria for constitutive components of a proper focus group. Broadly defined, a focus group has “the primary aim of describing and understanding perceptions, interpretations, and beliefs of a select population to gain understanding of a particular issue from the perspectives of the group’s participants” (Khan & Manderson, 1992, p. 57). According to Maykut and Morehouse (1994), a focus group interview brings several different perspectives into contact through a process that is open and emergent. Internet-mediated focus group discussions might use a different medium from face-to-face focus groups, but it is our contention that they share all but one of the key features of a focus group outlined by Krueger and Morgan (1988). Krueger delineated a set of six main characteristics of the focus group interview, the combination of which sets it apart from other group processes. He claimed that other groups might contain one or more of the characteristics but not all six. The six key characteristics identified by Krueger are that focus groups involve people; they are conducted in a series; participants are reasonably homogenous and unfamiliar with each other; they are methods of data collection; the data are qualitative; and they constitute a focused discussion. These criteria will be evaluated in turn in relation to the virtual focus groups that we conducted.
The focus groups comprised 4 (men’s rights activists), 6 (women who had been paternity tested), and 9 (stem cell study patients) people, respectively. Krueger (1994) argued that the group needs to be small enough to share insights but large enough to allow for diversity of views. Consistent with his view, that larger groups have the tendency to fragment because participants compete for the opportunity to comment, we found that the size of the group did seem to matter. Although there were only three groups, the smaller groups seemed to offer a better opportunity to share ideas. There are three possible reasons for this: First, it might be that the largest group (patients), whose members were overall less vocal, had less social cohesion and did not feel the same level of responsibility to respond. Alternatively, it might be that their relationship to the issue was not as personal, so their views were not as passionately and volubly expressed as were those associated with the paternity testing study. Finally, it might be that the patient group members’ restricted concentration, dexterity, and technical ability due to their medical condition made their participation more challenging. The latter point, though, highlights a particular value of this method: that is, without online methods, this patient group would not have been able to participate in a discussion about stem cell research, a debate that is directly relevant to them, making it essential to include their views in a study on attitudes toward the use of this biotechnology. Virtual focus groups thus foster democracy and democratic participation in research, enabling inclusion of groups whose pertinent views otherwise would have been overlooked. In both of these studies, individuals represented stakeholder groups who provided views crucial to understanding the social aspects of each biotechnology.
That focus groups need to be conducted in a series is the one criterion that the virtual groups did not meet. Krueger (1994), like other researchers who undertake evaluation research, has insisted that multiple groups with similar participants need to be run to optimize the detection of patterns and trends across the groups, a form of data validation. Together with Hurworth (1996), Krueger argued that it is important to run at least three focus groups, particularly when evaluating community-based projects for funding purposes. Both said that single focus groups can result in the collection of extraordinary results due to a variety of factors, such as a dominant personality in the group, a community event, incendiary comments by an individual, or the reluctance of a group to participate. However, the purpose of the virtual focus groups reported here, and many other focus groups, is substantially different from that of evaluation research. The in-depth focus group method used in our studies aimed to give an insight into a particular issue from the viewpoint of those with an interest or stake in the technology. Thus, it is our contention that single, stand-alone focus groups can and should be used for this purpose. If their objective is not generalizability but depth understanding from an insider, or emic (Pike, 1967), viewpoint, interest groups are an important and legitimate source of, and target for, research.
Furthermore, the factors causing extraordinary results to which Krueger (1994) referred are practically eliminated in the absence of the normal verbal and visual cues that indicate and enable hierarchy and dominance of views within a face-to-face setting. Because participants were unknown to each other but had a similar interest in the use of each technology under investigation, the synergistic group effect identified by Stewart and Shamdasani (1990) was facilitated. Although spontaneity was somewhat restricted by the asynchronicity of responses, participants were prepared to challenge each other if they disagreed. Thus, Krueger’s (1994) idea of being part of a community group whose solidarity on particular issues might be at stake simply did not apply. Interactive responses did, however, vary among the groups, with the women<@146>s (paternity testing) group much more practiced in techniques that enhance group dynamics. This might be explained by generation, gender, and technical competence: They were all younger, text-expressive mothers of a young child who were accustomed to online communication. Increased familiarity with computer-mediated communication, in particular, suggests the future potential of virtual focus groups for research, as the gender divide in interactive communication continues to break down and the size of the Internet-savvy cohort increases.
Krueger’s (1994) criterion that focus group participants need to be recruited on the basis of similarity to each other was met in each of the virtual focus groups. They each included participants who had a particular stake in the issue of paternity testing or stem cell research. The selection of participants aimed at homogeneity within groups in terms of a shared relationship to the issue. Krueger (1994) insists that shared interest, once known to the participants, serves to enhance the permissive or nonthreatening environment in which they can express controversial or private views. In each group, the participants were not known to each other, a factor Krueger (1994) claimed important, as familiarity tends to inhibit disclosure. Furthermore, their anonymity and virtual engagement within the group was likely to have been enhanced by the absence of social context cues and the use of anonyomizing techniques afforded by the Blackboard technology itself. A virtual identity enabled participants to find commonality beyond the usual social and physical barriers to communication, such as socioeconomic status, gender, age, ethnicity, and disability status.
The absence of immediacy in computer-mediated communication is said by some theorists to have a negative effect on the intensity of participant interaction, with text-based communication more formal, neutral, and lacking in any of the affective nuances contained in interpersonal verbal exchanges. However, other theorists (cited in Rourke, Anderson, Garrison, & Archer 2001) have argued that, far from being impersonal, computer-mediated communication can be hyperpersonal, characterized by an uninhibited openness and solidarity. There was evidence in our studies of an assumed community in these focus groups, particularly in the paternity-testing study. Far from being deterred by the formality of language, exchanges were candid and sometimes personal, with participants appearing to use a different set of cues that were specific to the online context.
Krueger (1988) has argued that focus groups are created with the specific goal of collecting data. He said that unlike other group discussions, the goal is not to reach consensus, make decisions, or provide recommendations. Rather, focus groups are narrow in their purpose and center their attention on the perceptions of users and consumers toward solutions, products, and services. This study found that the virtual focus group was a superior mechanism for data collection on attitudes, especially for these groups that each had a particular interest in the technology under investigation. The text-based nature of the Blackboard Learning Management System was particularly helpful in allowing researchers to download participants’ responses to questions and probes. Because the virtual focus group was run using text as the communication medium, participants’ responses were accurately and automatically recorded. An online text-based approach eliminated the need for the oral recording and transcription of participants’ responses, so there was little margin for error. Comments made using the asynchronous online chat facility can be archived and downloaded in rich text (rtf format), either by “threaded discussion” response or by individual participant response. The system’s flexibility also allows the researchers to track the reading and response patterns of participants. It is important to note, however, that participants with low levels of computer literacy and connectivity can be marginalized by any form of computer-based data collection (Dillman, & Bowker 2001; Dix & Anderson, 2002; Pocknee & Robbie, 2002).
Focus groups are designed to provide insight into attitudes, perceptions, and opinions of participants (Krueger 1994). These insights are gained through the recording of participant responses to a series of predetermined, open-ended questions. The questions asked in the virtual focus groups in our studies were essentially the same as those asked of the face-to-face participants, but the wording was altered slightly and presented in a different order to suit the specific group, as well as the asynchronous nature of the online environment. In an attempt to encourage participants to set the direction of the discussion in the virtual focus groups, we constructed our questions so that they were more open ended and less directive than those asked in the face-to-face groups. Knowledge of the issues related to each biotechnology was assumed by the nature of each group’s relationship to it, so we used Glaser and Strauss’s (1967) grounded theory approach. In all groups, a saturation point was reached for inclusion of new data by the end of the preset 7-day period. This was further made evident when the moderators extended the time frame for several days in the paternity-testing groups and nothing of a substantive nature was added to the discussions after the original close time. We therefore recommend that researchers adhere to close dates as indicated in the initial researcher-participant transaction.
The final criterion set by Krueger (1994) was the requirement for a focused discussion. By its very nature, the virtual method is a focused approach. Participants enter the discussion site with the specific intent of contributing their views on a particular topic. As in telephone interviewing, having no extraneous distractions, participants did not diverge from the topic, so there was a distilled response to the questions. What was lost in the immediacy of real-time group dynamics was gained in interactive responses that were measured and considered, because participants had the opportunity to reflect on their answers. They had time to think through what information they were prepared to share, something very important when discussing sensitive issues. Questions were posed in a way that encouraged expression of attitudes but still allowing for personal experience to be shared if participants chose to include this. They were therefore free to provide information in general terms, such as “People in my situation think . . . ,” or as a personal experience. Personal information was shared by some participants in each group as a way of establishing their “credentials” with the group.
In the virtual setting, participants had a choice of responding to question or deciding to withhold response, thus providing a noncoercive and truly democratic discursive environment. One main difference with the virtual focus group method was the changed role of the moderator. Once we had constructed the environment for discussion and set the rules of engagement prior to the opening of the virtual focus group, we found that the moderator role was less interventionist and less directive than in conventional focus groups. Observation, virtual listening, and the insertion of probes and additional questions replaced the steering role of the face-to-face moderator.
Although this inquiry was relatively limited in its scope, there are a number of findings and recommendations we feel confident to make regarding the three major areas of inquiry we addressed. As a formal research method, the virtual focus group, in the context of this trial, was theoretically sound and met the key criteria of traditional focus group methods as outlined by Krueger (1994) and Morgan (1988). We therefore recommend that researchers use the online method more regularly and evaluate its usage in a variety of contexts to confirm these findings. Furthermore, we suggest that the method be used in conjunction with other forms of qualitative data collection for both completeness and validation of the data collected (Jick, 1979).
From a technology perspective, we highly recommend the use of university learning management systems such as Blackboard and WebCT for conducting virtual focus group research. Posting and facilitating virtual focus groups using this technology takes advantage of existing university infrastructure with which academic researchers are familiar, suggesting the transferability of this online method will be straightforward. Furthermore, the uptake of an inventive extension of a readily available system reduces the need for extensive training, although we do advise that researchers conducting virtual focus groups receive appropriate training in online communications skills. In particular, we encourage other researchers to develop innovative methods that enhance participant interaction and build social context through strong moderator leadership skills. Using an existing learning management system also improves the cost-effectiveness of conducting online research, including the elimination of commercial site costs and the purchase of expensive online qualitative research software. Most important, however, by taking advantage of university quality, security, and privacy procedures, this form of hosting meets stringent ethical requirements by optimizing participant anonymity and confidentiality.
In the studies reported here, we found that there are several contexts and particular populations for which the virtual focus group appears to be particularly well suited as an effective vehicle for qualitative data collection. We recommend that this method be considered for use whenever populations are difficult to recruit and/or access. In particular, the method should be taken into account in designs for studies that involve individuals or groups that are, for whatever reason, hesitant or unable to participate in face-to-face focus groups. This could be when physical mobility is limited, such as in health-related studies (for instance, the stem cell research study), but also simply because virtual groups can involve “regular” workers, shift workers, homemakers, and parents, including those in different time zones, at times dictated by their personal needs and timetables. The ability of participants to be involved in research from the comfort and privacy of their homes is an attractive option for them. Their comfort is enhanced by the secure, safe, and anonymous environment afforded by the online technology. The main limitation to participation is the requirement for moderate-to-high levels of connectivity and computer literacy.
When employing an online method, researchers need to consider a number of key access and equity issues. Online information communication tools (ICTs) favor articulate, script-based, opinionated participants who have high levels computer literacy (Salmon, 2003). Participants need to be able to follow detailed written instructions and perform advanced script-based discursive functions. Low levels of formal literacy and lack of prior exposure to online ICTs could, in some cases, marginalize and inhibit participation. This is particularly the case if synchronous communication tools are adopted. However, if asynchronous ICTs are used, participants who have physical, educational, and social limitations have a greater opportunity to take their time and to reflect on what they intend to say. They can develop and compose their responses in an environment in which time and linguistic skills are not such a pressure.
In the reported studies, we used simplified, user-friendly, pictorial, electronic instruction sheets to overcome some of the abovementioned constraints and barriers to participation. The adoption of an online method also proved to be an effective way to recruit and access participants who have poor mobility, are spread geographically, and are hesitant to participate in face-to-face environments. However, we do suggest that issues surrounding equity and opportunity be fully considered when developing the broader research method, as the digital divide itself automatically omits people who have limited or no access to computers.
New technologies and ICTs provide unique and inventive opportunities for qualitative researchers. Their intrinsic ability to record written, discursive data accurately and provide safe, secure, and anonymous environments for participants makes them amenable to testing a variety of qualitative research methods. This inquiry addressed the use of virtual focus groups, but it opens up the possibilities for future use of technology in research to provide such things as accurate videorecordings of in-depth interviews from a variety of locations around the world. However, when researchers are planning such studies, they need to consider the theoretical underpinnings of these methods, as well as the specific access and equity issues that might be pertinent to their use. These matters need to be measured carefully against the obvious advantages new technologies provide for connecting researchers with populations who would otherwise be unable to participate in a research project.
1. A version of this article emphasizing the educational advantages of the virtual focus group method was published in Atkinson, McBeath, Jonas-Dwyer, and Phillips (2004). back to text
2. The results of the empirical study on paternity testing are reported in Turney et al. (2003). back to text
Adler, C. L., & Zarchin, Y. R. (2002). The “virtual focus group”: Using the Internet to reach pregnant women on home bed rest. Journal of Obstetric, Gynecologic, and Neonatal Nursing, 31(4), 418-427.
ACETS (Australian Centre for Emerging Technologies and Society). (2003). Swinburne national technology and society monitor 2003. Hawthorn, Australia: Swinburne University of Technology.
ACETS (Australian Centre for Emerging Technologies and Society). (2004). Swinburne national technology and society monitor 2004. Hawthorn, Australia: Swinburne University of Technology. Retrieved June 21, 2005, from http://www.swin.edu.au/sbs/acets/
Atkinson, R., McBeath, C., Jonas-Dwyer, D., & Phillips, R. (Eds.), Beyond the comfort zone: Proceedings of the 21st ASCILITE (Australasian Society for Computers in Learning in Tertiary Education) Conference (pp. 905-912). Perth, Australia: ASCILITE. Retrieved June 16, 2005, from http://www.ascilite.org.au/conferences/perth04/procs/turney.html
Bell, M., Bush, D., Nicholson, P., O’Brien, D., & Tran, T. (2002). Universities online. Canberra, Australia: Department of Education Science & Training.
Burgess, R. G. (1995). Introduction. In R. G. Burgess (Ed.), Studies in qualitative methodology: computing and qualitative research (pp. ix-xiii). Greenwich, CT: JAI.
Dillman, D., & Bowker, D. (2001). The Web questionnaire challenge to survey methodologists in dimensions of Internet science. Lengerich, Germany: Science Publishers.
Dix, K., & Anderson, J. (2000). Distance no longer a barrier: Using the Internet as a survey tool in educational research. International Education Journal, 1(2), 83-93.
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. New York: Aldine.
Greenbaum, T., (2002) The case against internet focus groups. MRA Alert Newsletter, 40(4). Retrieved July 9, 2004, from http://www.groupsplus.com/pages/case2.htm
Hurworth, R. (1996). Qualitative methodology: Common questions about running focus groups during evaluations. Evaluation News and Comments, 5(1), 40-52.
Hine, C. (2000). Virtual ethnography. London: Sage.
Jick, T. (1979). Mixing quantitative and qualitative methods: Triangulation in action. Administrative Science Quarterly, 24, 602-611.
Khan, M. E., & Manderson, L. (1992). Focus groups in tropical diseases research. Health Policy and Planning, 7(1), 56-66.
Krueger, R. A. (1988). Focus groups: A practical guide for applied research. London: Sage.
Krueger, R. A. (1994). Focus groups: A practical guide for applied research (2nd ed.). Thousand Oaks, CA: Sage.
Lim, P. C., & Tan, S.C. (2001). Online discussion boards for focus group interviews: An exploratory study. Journal of Educational Enquiry, 2(1). Retrieved July 9, 2004, from
Mann, C., & Stewart, F. (2000). Internet communication and qualitative research. London: Sage.
Maykut, P., & Morehouse, R. (1994). Beginning qualitative research. A philosophic and practical guide. London: Falmer.
Moloney, M. F., Dietrich, A. S., Strickland, O., & Myerburg, S. (2003). Using Internet discussion boards as virtual focus groups. Advances in Nursing Science, 26(4), 274-286.
Montoya-Weiss, M. M., Massey, A. P., & Clapper, D. L. (1998). On-line focus groups: Conceptual issues and a research tool. European Journal of Marketing, 32(7/8), 713-723.
Morgan, D. L. (1988). Focus Groups as Qualitative Research. Newbury Park, CA: Sage.
Murray, P. J. (1997). Using virtual focus groups in qualitative research. Qualitative Health Research, 7(4), 542-549.
Oregon Health & Science University. (2001, March 11). The ethics of research. Retrieved December 17, 2004, from http://www.journeyofhearts.org/jofh/jofh_old/minf_528/esearch.htm
Oringderff, J. (2004). “My way”: Piloting an online focus group. International Journal of Qualitative Methods, 3(3). Retrieved December 17, 2004, from http://www.ualberta.ca/~ijqm/english/engframeset.html
Pike, K. (1967). Language in relation to a unified theory of the structure of human behaviour. Glendale, CA: Glendale Summer Institute of Linguistics.
Pocknee, C., & Robbie, D. (2002, December). Surveyor: A case study of a web-based survey tool for academics. Paper presented at the Winds of Change in the Sea of Learning: Proceedings from the 19th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education, Auckland, New Zealand. Retrieved January 19, 2003, from http://www.ascilite.org.au/conferences/auckland02/proceedings/papers/140.pdf
Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education/Revue de l’enseignement à distance, 14(2). Retrieved November 10, 2004, from http://cade.athabascau.ca/vol14.2/rourke_et_al.html
Salmon, G. (2003). E-moderating: The key to teaching and learning online. London: Routledge Falmer.
Smissen, I. (2002, January 8). Learning management systems at other Australian universities. Retrieved December 17, 2004, from http://www.deakin.edu.au/lms_evaluation/old/otherInstitutions.htm
Stewart, D. W., & Shamdasani, P. N. (1990). Focus groups: Theory and practice. Newbury Park, CA: Sage.
Turney, L., Gilding, M., Critchley, C., Shields, P., Bakacs, L., & Butler, K. A. (2003). DNA paternity testing: Public perceptions and the influence of gender. Australian Journal of Emerging Technologies and Society, 1(1). Retrieved 4 October, 2003, from http://www.swin.edu.au/sbs/ajets/journal/issue1/paternity.pdf
| International Journal of Qualitative Methods 4 (2) June 2005 |