Audience Response Systems for Medical Teaching
Robyn Latessa, MD; David Mouw, MD
Background and Objectives: Little data exist about the use of an Audience Response System (ARS) as an interactive educational tool in medical teaching. The goals of our pilot study were to determine whether an ARS can enhance educational experiences of health care providers. Methods: The learners in the study were mainly physicians, and the educational topic was treating self, family, and friends. Results: The learners reported that the ARS made the presentation more fun, helped them be more attentive, and allowed them to learn more than in traditional lecture formats. Conclusions: An ARS has potential as a teaching tool in this setting.
An audience response system (ARS) has been praised as an effective teaching tool, primarily because it transforms a lecture into an interactive learning experience.1-4 With this system, each participant in the audience registers a response on a key pad, and the responses are instantly tallied and displayed on screen. In a review by Davis et al, the authors concluded that interactive teaching strategies that enhance participant activity are more effective in changing professional practice (and possibly health care outcomes) than traditional didactic methods.5 Little data exist regarding the effectiveness and preferences for the ARS in particular. In a randomized controlled trial, Miller et al found no increase in knowledge between ARS intervention and control groups, but health care providers in the ARS group rated the quality of the presentation, the quality of the speaker, and their level of attention more highly than the control group did.2
If the amount learned is equal for ARS and control groups, the ARS might still be useful if it augmented the enthusiasm and attentiveness of the audience. The goal of our pilot study was to determine whether an ARS can enhance educational experiences. Additional goals were to add to the small body of literature on ARS in medical teaching and encourage medical educators to consider using the ARS as a teaching tool.
Methods
The first author presented a lecture to the statewide faculty meeting of the University of North Carolina-affiliated family medicine residency programs. The setting was a retreat center at Wild Acres, NC. The audience consisted of 46 participants, which included 30 faculty physicians, one family nurse practitioner, two PharmDs, six behavioral medicine faculty, and seven other staff. The topic was “A Personal Decision: When Should Physicians Treat Themselves, Family, or Friends.” The technology used was Audience Response Systems, Inc, wireless response system and Power Poll software.
The practice of treating self and “nonpatients†(people not established in physician’s practice and treated outside the office setting) is widespread but rarely written or talked about.6-14 After a brief introduction to the topic, participants were asked to use the ARS to respond to questions about: (1) the clarity and comfort in their own practices, (2) their likelihood of documenting any treatments, and (3) what they would do in six hypothetical situations. Immediately after each question, the audience viewed a slide summarizing the responses of everyone in the audience. After this “pretest,” the learners heard a 20-minute lecture on this topic and then again used the ARS to respond to the same questions (“posttest”). Finally, the audience was asked to evaluate the effectiveness of the ARS, compared with traditional lecture formats. At the conclusion, an audience discussion gave participants a chance to talk about other aspects of their experience with the ARS.
Except for the discussion, all responses were anonymous. The software allowed for the downloading of the ARS data into an Excel spreadsheet for analysis. The paired t test was used to compare pre- and post-lecture responses for each individual.
Results
We used mean responses on a 4-point Likert scale to compare the pre- and post-lecture responses of the participants with prescribing privileges (all 33 responded). We found a significant increase in both the clarity (P<.001) and comfort (P<.01) of their guidelines following the presentation. Before the lecture,lecture, 20% of practitioners said they were “very likely” or “likely” to document encounters with “nonpatients,” compared to 69% after the lecture (P<.005).
All 46 learners responded to questions about the effectiveness of the ARS. Most of the participants felt that the ARS made them more attentive, allowed them to learn more than in traditional lecture formats, and made the lecture more fun. Further, most of the audience said they were likely to consider using the ARS in their work (Table 1).
The audience stayed for a 15-minute discussion after the lecture, and they noted several advantages of the ARS. Audience participation was enhanced and broadened by the fact that the ARS encouraged participants to commit to an answer and that the responses were anonymous. The immediate feedback about colleagues’ views or knowledge was appreciated. The ARS was noted to have the potential to generate more enthusiasm for topics that are not normally exciting for learners (ie, office visit coding). The ARS also could be used in situations other than lectures, like voting or polling or for research purposes with focus groups and other audiences.
| “A Lot” | “Some” | “Little” | “None” | |
|---|---|---|---|---|
| To what degree did the ARS make this lecture more fun that traditional lecture formats? | 84% | 16% | 0% | 0% |
| To what degree did the ARS make you more attentive than traditional lecture formats? | 67% | 33% | 0% | 0% |
| To what degree did the ARS help you learn more than traditional lecture formats? | 22% | 63% | 15% | 0% |
| Financial considerations aside, how likely are you to consider using the ARS in your work? | 44% | 33% | 15% | 8% |
Discussion
The audience in our study clearly felt that the ARS enhanced their educational experience. Moreover, they said that they would consider using the ARS in their work. Our data confirm and amplify the findings of Miller et al2 that learners feel more attentive when the ARS is used.
There are some potential barriers to the use of the ARS. More preparation time might be needed for the first-time ARS presenter. However, the speakers in Miller’s study rated the ease of use of the ARS system highly at 4.58 on a 5-point scale (with a higher score indicating a more positive opinion). Some tips for using an ARS include making questions short and simple, allowing time for discussion, using questions sparingly, and allowing enough time for set up.15 The largest disadvantage of using the ARS is the cost. The hardware and software can currently run anywhere from $5,000–$25,000.
Limitations
Several limitations of this study deserve mention. This is a limited intervention of a single lecture with small numbers of participants. The study was not designed to quantitatively compare learning in lecture plus ARS to lecture alone, since all subjects were in the former group. Since there was no control group, the learning might have been similar with other teaching methods.
Conclusions
This pilot study shows that using an ARS enhances participants’ attention, enjoyment, and overall learning. Further studies with other types of learners, other subject matter, larger numbers, and control groups are needed before the method can be more strongly endorsed.
Acknowledgments: We thank Cynthia Janes, PhD, for help with conceptualization of this project, Mike Buechler, MD, for use of ARS hardware and software, and Norma Beaty, MS, MEd, for editing assistance.
Corresponding Author: Address correspondence to Dr. Latessa, Mountain AHEC, 118 W.T. Weaver Boulevard, Asheville, NC 28804. 828-771-3429. Fax: 828-257-4738. robynl@mtn.ncahec.org.
Dr. Latessa is a contributor to Family Medicine Journal, the official journal of the Society of Teachers of Family Medicine.
REFERENCES
- Copeland HL, Longworth DL, Hewson MG, Stoller JK. Successful lecturing: a prospective study to validate attributes of the effective medical lecture. J Gen Intern Med 2000;15:366-71.
- Miller RG, Ashar BH, Getz KJ. Evaluation of an audience response system for the continuing education of health professionals. J Contin Educ Health Prof 2003;23:109-15.
- Nasmith L, Steinert Y. The evaluation of a workshop to promote interactive learning. Teach Learn Med 2001;13(1):43-8.
- Turpin DL. Enhance learning with an audience response system. Am J Orthod Dentofacial Orthop 2003;124:607.
- Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999;282(9):867-74.
- Aboff BM, Collier VU, Farber NJ, Ehrenthal DB. Residents’ prescription writing for nonpatients. JAMA 2002;288(3):381-5.
- Chambers R, Belcher J. Self-reported health care over the past 10 years: a survey of general practitioners. Br J Gen Pract 1992; 42:153-6.
- Christie JD, Rosen IM, Bellini LM, et al. Prescription drug use and self-prescription among resident physicians. JAMA 1998;280(14):1253-5.
- Hughes PH, Brandenburg N, Baldwin DC Jr, et al. Prevalence of substance use among US physicians. JAMA 1992;267(17):2333-9.
- La Puma J, Priest ER. Is there a doctor in the house? An analysis of the practice of physicians treating their own families. JAMA 1992;267(13):1810-2.
- La Puma J, Stocking CB, LaVoie D, Darling CA. When physicians treat members of their own families: practices in a community hospital. N Engl J Med 1991;325 (18):1290-4.
- Reagan B, Reagan P, Sinclair A. Common sense and a thick hide: physicians providing care to their own family members. Arch Fam Med 1994;3:599-604.
- Wachtel TJ, Wilcox VL, Moulton AW, Tammaro D, Stein MD. Physicians’ utilization of health care. J Gen Intern Med 1995:10:261-5.
- Westfall JM, McCabe J, Nicholas RA. Personal use of drug samples by physicians and office staff. JAMA 1997;278(2):141-3.
- Robertson LJ. Twelve tips for using a computerized interactive audience response system. Med Teach 2000;22(3):237-40.