Randomized Trial of a Program to Increase Staff Influenza Vaccination in Primary Care Clinics ============================================================================================= * Zvi Howard Abramson * Ohad Avni * Orit Levi * Ian Nigel Miskin ## Abstract **PURPOSE** Although vaccination of health care workers against influenza is widely recommended, vaccination uptake is low. Data on interventions to increase staff immunization in primary care are lacking. We examine the effect of a promotional and educational intervention program, not addressing vaccine availability, to raise the influenza vaccination rate among staff in primary care clinics. **METHODS** The study included all 344 staff members with direct patient contact (physicians, nurses, pharmacists, and administrative and ancillary staff) in 27 primary care community clinics in the Jerusalem area during the 2007–2008 influenza season. Thirteen clinics were randomly selected for an intervention that consisted of a lecture session given by a family physician, e-mail-distributed literature and reminders, and a key figure from the local staff who personally approached each staff member. **RESULTS** Influenza immunization rate was 52.8% (86 of 163) in the intervention group compared with 26.5% (48 of 181) in the control group (*P*<.001). When compared with the rate of immunization for the previous season, the absolute increase in immunization rate was 25.8% in the intervention clinics and 6.6% in the control clinics. Multivariate analysis showed a highly significant (*P*<.001) independent association between intervention and immunization, with an odds ratio of 3.51 (95% confidence interval, 2.03–6.09). **CONCLUSION** We have developed an effective intervention program to increase previously low vaccination rates among primary health care workers. This simple intervention could be reproduced easily in other clinics and organizations with an expected substantial increase in influenza immunization rates. * Influenza/prevention & control * immunization * health personnel * primary health care * promotional program ## INTRODUCTION Because of their proximity to patients, vaccination of health care workers against influenza is widely recommended.1,2 Studies have shown that immunization of health care workers protects their patients3–5 and themselves6 from influenza infection, and probably reduces staff absenteeism during the influenza season.7 The low immunization uptake among health care workers8–10 and the lack of increase in immunization rates11 have led to a number of studies, conducted mainly in hospitals and long-term care facilities, that examined the workers knowledge, attitudes, and reasons for not receiving vaccination. Influenza vaccination in primary health care workers was studied in health maintenance organization (HMO) clinics in Jerusalem after the influenza season of 2006–2007.12 Groups for whom vaccinations were recommended in Israel included adults aged 50 years and older and health care workers in contact with chronically ill patients. That Jerusalem study, which formed the basis for the intervention described in this article, found low compliance with immunization recommendations, with less than one-third of health care workers reporting vaccination. Immunization was higher among physicians and also positively associated with previous year’s vaccination, age, knowledge, attitude, and having received a physician’s recommendation. Previous studies have reported the effectiveness of various campaigns to vaccinate health care workers against influenza.2,13 Most were before-after studies, and only very few were randomly controlled.14–17 Only 1 of all these studies15 included primary health care workers. The vaccination campaign described in that study, however, in contrast to most others, did not substantially increase uptake rates. Our randomly controlled trial examines an intervention program intended to increase influenza vaccination among the staff of primary care clinics. Previously successful campaigns, including those of 2 controlled studies,14,16 were based on increasing vaccine availability among health care workers. This approach, which was effective in these hospitals studies, is less appropriate for primary care clinics, where the vaccine is easily accessible. In primary care clinics, attitudinal barriers or lack of motivation may be more important. The intervention examined in our study was therefore educational and promotional only. We chose an intervention simple enough so that, if proven effective, could easily be adopted by other clinics and organizations. ## METHODS The study participants consisted of the staff of 27 primary care community clinics in the Jerusalem area belonging to Clalit Health Services, an HMO serving the majority of the Israeli population. All 344 permanent workers with direct patient contact—physicians, nurses, pharmacists, and administrative and ancillary staff—were included. The clinics were randomly allocated for the 2007–2008 influenza season into intervention and control groups. Sample size was calculated correcting for cluster intraclass correlation using a coefficient of 0.017, imputed from the previous year’s vaccination data. A sample of 156 health care workers in each group was calculated to have a power of 90% for detection of a relative increase in immunization rates from an expected rate of 0.3 to a rate of 0.5 with a significance level of .05. As in previous years, all clinics received the HMO recommendations for influenza vaccination of health care workers. There was no HMO managerial participation in the program beyond its general approval. The clinics were not autonomous and did not have individual policies concerning staff immunization. The only involvement of the clinic management was in facilitating the lecture session. Immunization was free and easily available in all clinics. In concordance with previous trials in other settings showing that programs with more than 1 intervention method are more successful,2 and with the established value of a local vaccination “champion”17 and of a physician’s recommendation to receive vaccination,12 the intervention (in 13 of the 27 clinics) included a lecture session to the staff given in the clinic by a family physician (O.A. or I.N.M.), e-mail distributed reminders and relevant literature, and recruitment of a key figure from the local staff (physician or nurse) who personally approached each staff member. The data gathered in the study clinics in the survey after the previous influenza season12 enabled the lecturer to report on the magnitude of undervaccination and to address concerns and misconceptions. Data, including influenza immunization status, were extracted from the HMO’s computerized database. We also included knowledge and attitude factors measured by the questionnaire at the end of the previous influenza season, before the intervention. Data were analyzed using SPSS version 15.0 for Windows (Statistical Package for the Social Sciences, SPSS Inc, Chicago, Illinois) and WINPEPI statistical programs for epidemiologists.18 Data were examined at both the clinic and individual level. Intraclass correlation coefficients and adjustment for clustering were computed by WINPEPI using, as do other widely used statistical software packages, the Donald-Donner method.19 Negative values (a result of the within-cluster mean square exceeding the between-cluster mean square) are taken as zero in adjusting for clustering. Bivariate, Mantel-Haenszel, and logistic regression analyses were performed, and first-degree interactions were examined. The study was approved by the Helsinki ethics committee of the Meir Hospital, Kfar Saba, Israel. ## RESULTS Data were available for all 344 permanent staff members, with the exception of data from the previous season’s questionnaire, which were available for 67.2% of the staff (231 of 344). No statistically significant differences in influenza vaccination, age, sex, chronic disease, and intervention or control membership were found between responders and nonresponders to the questionnaire. Characteristics of the staff in the intervention and control clinics are displayed and compared in Table 1⇓. No statistically significant differences were found, including for characteristics previously found to be associated with immunization.12 The intervention group had a somewhat higher rate of staff who had been immunized for the previous season, as well as of physicians and older members. The control clinics had a higher rate of those who knew, according to the previous year’s survey, that the vaccine cannot cause influenza. View this table: [Table 1.](http://www.annfammed.org/content/8/4/293/T1) Table 1. **Baseline Characteristics of Intervention and Control Groups** The influenza immunization rate for the intervention season was 52.8% (86 of 163) in the intervention group compared with 26.5% (48 of 181) in the control group. This difference was highly significant, *P* <.001, with a rate ratio of 1.99 (95% confidence interval [CI], 1.50–2.64). The absolute increase in immunization rate compared with that of the previous season was 25.8% for the intervention clinics (increased from 27.0% to 52.8%) and 6.6% for the control clinics (19.9% to 26.5%). Staff vaccination in the different intervention clinics ranged from 31.3% to 77.8% (mean, 54.2%) compared with 0.0% to 50.0% in the control clinics (mean, 26.5%). The difference between the means was highly significant (Welche’s *t* test, *P* <.001; CI, 17.2%–38.4%). Twelve of the 13 intervention clinics and only 7 of the 14 control clinics had an increase in staff vaccination rates compared with the previous year’s rates. The mean absolute increase in clinic vaccination was 27.5% in control clinics and 7.6% in intervention clinics. This difference of 19.9% was statistically significant (*P* = .004; 95% CI, 6.9–32.9%). Intracluster (clinic) correlations were negative: −.018 in the intervention clinics and −.015 in the control clinics. Table 2⇓ shows that all examined subgroups had consistently higher rates of immunization in the intervention group. The differences in vaccination rates between intervention and control were statistically significant for staff who had both received and not received vaccination in the previous year. The difference was significant also in all subgroups of the other examined characteristics except among workers with chronic disease and pharmacists and administrative staff, the subgroup with the smallest rate difference between intervention and control groups. View this table: [Table 2.](http://www.annfammed.org/content/8/4/293/T2) Table 2. **Influenza Immunization According to Staff Characteristics** The results of the Mantel-Haenszel procedure show that the difference between the intervention and control groups remained highly significant (*P* <.001) when controlling for each of the other variables. The adjusted rate ratios were consistently about 2. Results of the multivariate analysis, displayed in Table 3⇓, shows a highly significant (*P* <.001) independent association between intervention and subsequent immunization while controlling for the other variables listed in the table, with an odds ratio of 3.51 (compared with 3.09 in the crude data). Other variables independently and positively associated with vaccination were having been vaccinated during the previous season, having a chronic disease, and being a nurse or physician. Adding questionnaire variables to the model showed that those who believed (before the intervention) that it was desirable to immunize primary health care workers were also independently and significantly more likely to be immunized. There were no significant interactions between the intervention and any of the other variables in their effect on immunization. View this table: [Table 3.](http://www.annfammed.org/content/8/4/293/T3) Table 3. **Associations with Influenza Immunization: Logistic Regression (n = 311)a** Examination of immunization rates among those who were not vaccinated in the previous season indicated that the rates differed according to the reasons given for not receiving a vaccination in the preintervention questionnaire. An increase in immunization with intervention was apparent when the reasons for not having previously been vaccinated were a perceived low risk of contracting severe influenza (7 of 18 in intervention clinics and 2 of 21 in control clinics), a belief in low efficacy of the vaccine (5 of 11 and 2 of 14, respectively), or lack of time (6 of 9 and 5 of 13, respectively). Rates were not considerably increased when the reasons given for not receiving immunization were fear that immunization would cause influenza (1 of 8 in intervention clinics and 0 of 14 in control clinics), fear of other side effects (3 of 18 and 3 of 21, respectively), or a general objection to vaccinations (2 of 11 and 1 of 17, respectively). We recorded lecture attendance in 4 of the intervention clinics. The data showed that 66% of the staff attended the lecture. Immunization was significantly higher (*P* = .009) among lecture participants (66% among participants compared with 30% among nonparticipants). Lecture attendance was significantly higher among workers who had been immunized the previous year, among physicians and nurses, and among those who believed it desirable to immunize primary health care workers. There was no significant difference in participation rates according to reasons given for not previously immunizing. ## DISCUSSION Our study, clearly and for the first time in primary care, shows that an intervention program can substantially increase influenza vaccination among health care workers. We did not detect any other differences between the intervention and control groups that could explain the doubling of immunization rate in intervention clinics when compared with control clinics. The true effect of the program may have been even larger, as it is probable that some of the educational efforts spread to the control clinics. The increase in control staff immunization (19.9% to 26.5%, rate ratio = 1.33) could be the result of differences between the years, specifically the result of a media scare the previous year12 that reduced immunization rates. The intervention staff’s increase in immunization rate was much larger, however (52.8% vs 27.0% in the previous year, rate ratio = 1.96), and analysis showed that the intervention was highly effective beyond this temporal effect. The intervention proved successful both among staff who had been immunized and those who had not been immunized the previous season. Among those not immunized the previous year, however, the intervention appeared not to be effective among staff who previously had an objection to immunization. It was more effective among those who gave other reasons for not receiving immunization, mainly belief that there was insufficient reason to immunize or lack of time. In addition to the effect of the intervention, regression analysis also found the expected strong association between having received an immunization in the previous year and present immunization status. Profession (physicians and nurses), chronic disease, and the belief that it is desirable to immunize primary health care workers were also associated with higher rates of immunization. The associations of these characteristics, independent of the association with the previous year’s vaccination (which would already include the general effect of these characteristics on vaccination), may have resulted from a greater reduction of the effect of the previous year’s media scare in these groups.12 Previous controlled interventions that proved to be effective14,16,17 were in hospital settings and consisted primarily of increasing vaccine availability (vaccine day or vaccine clinics, where vaccine was offered freely at the work sites).14,16 The effectiveness of our campaign is more remarkable in that it was promotional and educational only and was delivered in a setting where vaccine was already highly available (for both intervention and control clinics). A vaccine-day intervention, effective in a hospital setting, would not be expected to be very beneficial in primary health care clinics where vaccines are regularly and freely available every day. Immunization, which was correlated with lecture attendance (examined in 4 of the clinics), was not necessarily the effect of the lecture but possibly of the choice to attend the lecture; participation was significantly higher among workers with characteristics previously found to be associated with higher immunization rates.12 About two-thirds of the intervention staff attended the lecture, and possibly even fewer read the educational material sent by e-mail. It is probable that the specifics of the lecture and educational material played only a minor part in the success of the intervention. Our impression is that the program’s success resulted from the general effect of raising the immunization issue and recommendation repeatedly and from different directions: medical literature, a familiar family physician with expertise, and a local staff member. The central role of unit vaccination champions is supported by a recent hospital-based study17 showing their effectiveness in increasing staff vaccination. The success of our program, compared with the failure in the 1 previous controlled trial that included primary care staff,15 in which the intervention was performed by a visiting public health nurse, may partly have resulted from this multifaceted approach. Management not being involved in the program may have possibly decreased resistance and increased staff responsiveness. We have shown the effectiveness of a simple intervention program in increasing the influenza vaccination rate in primary health care workers. The program, though far from achieving full vaccination coverage, affected an absolute increase in immunization of approximately 20%. It could be easily replicated in other clinics with low immunization rates. The intervention requires little investment of time and resources and has already been adopted by the control clinics for the following influenza immunization period. To retain the program’s effectiveness in the clinics where it has already been implemented, selected local staff members will continue to personally approach their colleagues during the vaccination period in coming years. Additional research is recommended to determine whether intervention programs aimed at increasing staff influenza immunization also substantially increase patient immunization. Such benefit would further support wide implementation of these programs. ## Acknowledgments We thank the 13 vaccination champions for their important voluntary contribution to the intervention program. ## Footnotes * *Conflicts of interest: none reported* * Received for publication May 24, 2009. * Revision received December 20, 2009. * Accepted for publication January 15, 2010. * © 2010 Annals of Family Medicine, Inc. ## REFERENCES 1. van Essen GA, Palache AM, Forleo E, Fedson DS. Influenza vaccination in 2000: recommendations and vaccine use in 50 developed and rapidly developing countries. Vaccine. 2003;21(16):1780–1785. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/S0264-410X(03)00072-0&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12686094&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000182575500009&link_type=ISI) 2. Centers for Disease Control and Prevention (CDC). Influenza vaccination of health-care personnel. Recommendations of the health-care infection control practices advisory committee (HICPAC) and the advisory committee on immunization practices (ACIP). MMWR Recomm Rep. 2006;55(RR02):1–16.[Erratum in *MMWR Morb Mortal Wkly Rep* 2006;55(09):252.] [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16498385&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) 3. Potter J, Stott DJ, Roberts MA, et al. Influenza vaccination of health care workers in long-term-care hospitals reduces the mortality of elderly patients. J Infect Dis. 1997;175(1):1–6. [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiamluZmRpcyI7czo1OiJyZXNpZCI7czo3OiIxNzUvMS8xIjtzOjQ6ImF0b20iO3M6MjI6Ii9hbm5hbHNmbS84LzQvMjkzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 4. Carman WF, Elder AG, Wallace LA, et al. Effects of influenza vaccination of health-care workers on mortality of elderly people in long-term care: a randomised controlled trial. Lancet. 2000;355 (9198):93–97. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/S0140-6736(99)05190-9&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=10675165&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000085557200009&link_type=ISI) 5. Hayward AC, Harling R, Wetten S, et al. Effectiveness of an influenza vaccine programme for care home staff to prevent death, morbidity, and health service use among residents: cluster randomised controlled trial. BMJ. 2006;333(7581):1241. 10.1136/bmj.39010. 581354.55. [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMzMvNzU4MS8xMjQxIjtzOjQ6ImF0b20iO3M6MjI6Ii9hbm5hbHNmbS84LzQvMjkzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 6. Ito Y, Sumi H, Kato T. Evaluation of influenza vaccination in health-care workers, using rapid antigen detection test. J Infect Chemother. 2006;12(2):70–72. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s10156-005-0424-9&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16648945&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) 7. Chan SS. Does vaccinating ED health care workers against influenza reduce sickness absenteeism? Am J Emerg Med. 2007;25(7):808–811. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.ajem.2007.02.002&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17870487&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) 8. King WD, Woolhandler SJ, Brown AF, et al. Brief report: Influenza vaccination and health care workers in the United States. J Gen Intern Med. 2006;21(2):181–184. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s11606-006-0255-0&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16606378&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000235867800017&link_type=ISI) 9. Jiménez-García R, Hernández-Barrera V, Carrasco-Garrido P, Sierra-Moros MJ, Martinez-Hernandez D, de Miguel AG. Influenza vaccination coverages among Spanish children, adults and health care workers. Infection. 2006;34(3):135–141. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1007/s15010-006-5627-1&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16804656&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) 10. Leitmeyer K, Buchholz U, Kramer M, et al. Influenza vaccination in German health care workers: effects and findings after two rounds of a nationwide awareness campaign. Vaccine. 2006;24(47–48): 7003–7008. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.vaccine.2006.04.040&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16730866&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000242444200024&link_type=ISI) 11. Walker FJ, Singleton JA, Lu P, Wooten KG, Strikas RA. Influenza vaccination of healthcare workers in the United States, 1989–2002. Infect Control Hosp Epidemiol. 2006;27(3):257–265. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1086/501538&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16532413&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000249035800007&link_type=ISI) 12. Abramson ZH, Levi O. Influenza vaccination among primary health-care workers. Vaccine. 2008;26(20):2482–2489. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.vaccine.2008.03.011&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=18407385&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) 13. Burls A, Jordan R, Barton P, et al. Vaccinating healthcare workers against influenza to protect the vulnerable—is it a good use of healthcare resources? A systematic review of the evidence and an economic evaluation. Vaccine. 2006;24(19):4212–4221. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/j.vaccine.2005.12.043&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16546308&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000238016200021&link_type=ISI) 14. Ohrt CK, McKinney WP. Achieving compliance with influenza immunization of medical house staff and students. A randomized controlled trial. JAMA. 1992;267(10):1377–1380. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.1992.03480100083036&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=1740861&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=A1992HG67800033&link_type=ISI) 15. Dey P, Halder S, Collins S, Benons L, Woodman C. Promoting uptake of influenza vaccination among health care workers: a randomized controlled trial. J Public Health Med. 2001;23(4):346–348. [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoianBobSI7czo1OiJyZXNpZCI7czo4OiIyMy80LzM0NiI7czo0OiJhdG9tIjtzOjIyOiIvYW5uYWxzZm0vOC80LzI5My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 16. Kimura AC, Nguyen CN, Higa JI, Hurwitz EL, Vugia DJ. The effectiveness of vaccine day and educational interventions on influenza vaccine coverage among health care workers at long-term care facilities. Am J Public Health. 2007;97(4):684–690. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.2105/AJPH.2005.082073&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17329659&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000255647500026&link_type=ISI) 17. Slaunwhite JM, Smith SM, Fleming MT, Strang R, Lockhart C. Increasing vaccination rates among health care workers using unit “champions” as a motivator. Can J Infect Control. 2009:159–164. 18. Abramson JH. WINPEPI (PEPI-for-Windows): computer programs for epidemiologists. Epidemiol Perspect Innov. 2004;1(1):6. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1186/1742-5573-1-6&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15606913&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) 19. Donald A, Donner A. Adjustments to the Mantel-Haenszel chi-square statistic and odds ratio variance estimator when the data are clustered. Stat Med. 1987;6(4):491–499. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1002/sim.4780060408&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=3629050&link_type=MED&atom=%2Fannalsfm%2F8%2F4%2F293.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=A1987J075900007&link_type=ISI)