We conducted a randomized trial among dental practices from Alabama, Georgia, Florida, and North Carolina, identified using mailing lists from dental licensure and the Dental PBRN, a dental practice-based research network [22]. PBRNs are "groups of primary care clinicians and practices working together to answer community-based health care questions and translate research findings into practice. PBRNs engage clinicians in quality improvement activities and an evidence-based culture in primary care practice" (Agency for Healthcare Research and Quality). The community-based dental practices have a varying number of providers and are based in a variety of settings (rural and urban). Beginning in January 2005 through February 2006, dental practices were recruited using a letter addressed to the dentist which advertised the study. For blinding purposes the letter did not mention tobacco control but identified the study as an evaluation of an “Online Study Club for Oral Cancer Prevention”. Face-to-face study clubs are frequent in dentistry and usually refer to a group of dental providers who gather to discuss clinical practice and the dental literature [23-25]. Eligible practices included general dentistry or periodontal practices which reported having Internet access in their practice (requirement of the study) and indicated an interest in participating.

Accounting for clustering of patients within practices, we calculated a sample size of 130 practices (65 per arm) would be needed to detect a difference of 10%, comparing intervention and control. Anticipating an attrition rate of 30%, our targeted recruitment goal was 190 practices. Dental practices which initially agreed to participate were required to complete a run-in phase of baseline data collection, including patient and practice data, and then they were randomized. From our initial recruitment pool, we randomized the first 190 practices which returned the baseline data. Practices were randomized to the intervention described below, or a control group using a permuted block randomization sequence generated by our biostatistician. As practices returned baseline data, allocation to intervention or control was performed using the predetermined randomization sequence by an analyst blinded to the results of the baseline data. The protocol was approved by the University of Alabama at Birmingham Institutional Review Board.

After the run-in phase and randomization, the dentist and staff of intervention practices were sent a letter with information about the website and log-on instructions. We then tracked each practice to determine who from the practice logged on, when they logged on, and the amount of time they spent visiting the site. To encourage participation, emails were sent to the intervention participants alerting them about new website content and updates in the field of dental tobacco control. Once a practice logged on to the website, patient education materials about tobacco use were mailed to the practice for in-office use.

We used a delayed-intervention control group. Control practices continued to provide the usual care that they delivered to patients during the intervention period while still completing baseline and follow-up data collection. Control practices did not receive access to the intervention until all data collection was complete.

We developed an Internet-delivered educational intervention designed to support oral cancer prevention in dentistry. The development team included a hygienist, dentist, and tobacco control and health informatics experts. Prior to the development of the website, we conducted 3 Nominal Group Technique (NGT) meetings, 2 with a total of 13 dentists, and 1 with 10 hygienists participating. The NGT is a structured approach to collecting and prioritizing input from stakeholders [26]. The question for NGT discussion (“What sorts of things could be done to ensure that as a routine part of every dental visit all patients are asked about their tobacco use and/or advised to quit using tobacco?”) was identified through numerous brainstorming sessions with the investigative team. The dentists and hygienists identified 76 potential strategies for promoting tobacco control, including 9 distinct educational issues. Based on the NGT findings, the investigative team along with programmers met weekly for 12 months to develop both the content and format of the Internet intervention, which resulted in an interactive, multi-component website with supporting emails. Usability testing was conducted to confirm ease of navigation. The site was designed to be accessed longitudinally over 8 months and be frequently updated with new content.

The final Oral Cancer Prevention product was comprised of 3 educational cases, patient education and practice tools, a forum for chatting, opportunities to ask questions, and presentation of headlines (see Multimedia Appendix 1 for sitemap and screenshots). The dentist could spend from as little as a few minutes up to hours on the website. All course materials were updated as needed and the 3 cases were released at 2-month intervals.

The interactive educational cases were interspersed at key decision points with questions, and we provided targeted feedback based on user responses. In addition, references and literature were available at critical points to support the course material. Dentists and hygienists could access downloadable, patient education materials and practice tools, including brochures and posters. A discussion forum allowed the dentists and dental staff to post questions and receive feedback/responses from other dental staff and practitioners. The “ask a question” feature allowed any participant to submit a question related to oral cancer prevention and receive a direct response from the investigative team. In addition, we emailed all participants bi-weekly with “headlines” presenting new research findings to the group and/or with “questions of the week” asking challenging questions related to tobacco control.

Participants received one continuing education unit for each of the cases completed. As cues to log on, we provided the practices with calendars, pens, and squeeze balls that had the website address and the project name. The intervention was available over an 8-month period for each practice.

Data were collected from the practices at baseline before randomization. The baseline practice survey included an assessment of the number of dentists, hygienists, and dental assistants in the practice; the number of years employed at that practice; and current oral cancer prevention-related activities.

Once randomized, user authentication was required for all providers as they logged onto the intervention. This allowed use of server tracking logs linked to site visits to measure participation. The administrative portal of the study website tracked type of page visited, volume of pages, number of visits by practice and individual, date of access, time of access, name, and practice identification of each participant who logged onto the site [27]. We used the total number of pages of website content accessed as a marker for overall participation.

Our main outcome measures were based on the patient reports of guideline-compliant provider performance of tobacco control for the first 2 components of the 5A’s (ASK and ADVISE) [4]. The 2 main outcomes were the proportion of patients asked if they were tobacco users (ASK), and among the tobacco users, the proportion who were advised to quit (ADVISE). Accordingly, we collected patient reports of provider behaviors using patient exit cards.

After completing the practice survey, practices were provided a set of 100 patient exit cards. The patient exit cards, brief post-card sized surveys, were completed by adult patients at the end of their appointments prior to leaving the office. The exit cards were developed using principles of ecological momentary assessment (EMA) [28-30]. First, EMA is completed as close in time to the exposure as possible to avoid faulty recall. Second, EMA is designed to be brief and unobtrusive to maximize participation rates and diffusion.

The exit cards were designed to be completed in 1 to 2 minutes while the patient was awaiting follow-up instructions and completing payment. Each practice was provided with instructions to hand out these exit cards to 100 consecutive adult patients after their visit. Each patient was provided a pen to complete the survey, and they were allowed to take the pen as a gift. Patients completing cards then deposited them in a sealed collection box. When all 100 cards were distributed, the dental practice returned the collection box to our coordinating center.

We used the patient exit card to assess patient tobacco use, age, and gender. Patients indicated whether they had been asked about tobacco use and, if a tobacco user, whether they had been advised to quit. To blind the patient and practice to the outcome of interest, the exit card also included questions related to alcohol use and counseling, as well as dietary intake and counseling, received at the practice. Patients indicated on the card if they were willing to be contacted for a follow-up call and, if so, provided their name and telephone number. A sample of 150 patients from 6 practices was called to evaluate the reliability of the patient exit card data [31]. Agreement between card and telephone interview responses on whether the patient was a tobacco user was high (99%), with only 2 disagreements. Agreement rates for patient age and gender comparing immediate and delayed were also high (97% and 100%, respectively).

In this trial, the unit of randomization was the dental practice, and both dentists and their staff were the targets of the intervention. Our analysis used an intent-to-treat design including all practices with follow-up data available. Intervention practices were analyzed as randomized regardless of whether they actually used the intervention. As noted, dependent variables for this study are patient-reported provider performance measures (ASK and ADVISE) collected as binary variables at the patient level. As patients were clustered within practices and the unit of randomization was at the practice level, we used a modeling approach appropriate to hierarchical data. Common approaches to clustered data include generalized estimating equations and generalized linear mixed models [32,33]. As the number of smokers per practice varied, we chose to use a generalized linear mixed model approach with adaptive quadrature with a logit link for binary outcomes because this approach is more robust to variations in intra-class correlation coefficient and cluster size [34,35]. This analysis was implemented using the Generalized Linear Latent and Mixed Models (GLLAMM) procedure in the STATA software package and verified using the SAS software package.

To assess the impact of the intervention, we first calculated the unadjusted proportion of patients who were asked and smokers who were advised pre- and post-intervention. Then, separately for intervention and control, we assessed the difference pre- versus post-intervention. For each indicator (ASK and ADVISE), we developed 2 models (1 for intervention, 1 for control). Finally, significance of differences (pre- versus post-intervention) in the odds of patient reports of ASK or ADVISE in intervention versus control practices were determined. For each indicator (ASK, ADVISE), 1 overall model, including both control and intervention patients, was developed. We included a term for Group (intervention versus control) and Time (pre- versus post-intervention), as well as a group-by-time interaction term. Significance of differences in improvement over time, by group, was determined by the statistical significance of the group-by-time interaction term in these overall models.

As a secondary analysis, we conducted a “per protocol” analysis excluding intervention-arm practices that did not participate in the intervention to further estimate what the optimal effects might be for this Internet-delivered intervention.

Because participation in Internet-delivered interventions such as this is inherently variable, we further assessed a dose-response by level of participation among intervention practices.

From a group of 1346 practices initially expressing interest, we randomized the first 190 practices that completed data collection. Of the 190 dental practices randomized, 75% (143) completed follow-up data collection (see Figure 1).

Figure 1

CONSORT Flowdiagram: Recruitment and retention of dental practices

Of the 70 intervention practices that participated in follow-up, 56 (80%) had at least 1 provider who actually participated in the intervention. In the 56 participating practices, 53 of the 56 dentists (95%) and 38 of the 56 hygienists (68%) logged on to the intervention website. The mean number of tracked pages per practice was 50 (SD 40), and these ranged from 1 to 157. The mean number of visits to the intervention per practice that logged on was 5.8 (SD 4.6), and the mean number of unique participants was 1.9 (SD 1.2), ranging as high as 6 participants (dentists and hygienists) in a single practice. Figure 2 displays the number of unique providers visiting the website per week of intervention time. The spikes in activity centered at weeks 12 and 18 correspond to the initial release of additional interactive cases. Other smaller spikes represent response to headlines and questions-of-the-week updates. For the 3 cases, 75% (42/56) of practices had at least 1 provider complete Case 1, 55% (31/56) had at least 1 provider complete Case 2, and 21% (12/56) completed Case 3.

Figure 2

Number of unique providers visiting the website per week over 8 months

At the patient level, for these 143 post-intervention practices, the exit-card response rate was 81.6% (11,678/14,300). Patient characteristics for this cohort were similar to the pre-intervention group, with a mean age of 47.5 years (SD 16), 59.3% being female, and 22.6% being smokers.

In adjusted analysis, accounting for clustering of patients within practices, both intervention and control improved slightly for ASK, but their rates of change over time, as measured by the group-by-time interaction term, did not differ significantly (Table 2). Intervention practices improved on ADVISE significantly more than control practices (P-value for the interaction term = 0.01).

In our per-protocol analysis we kept only the intervention practices with follow-up data that actually logged on to the website at least once (N = 56) and compared them to the control practices. In this model, the effect of the intervention was strengthened with the cluster-adjusted odds ratios of receiving advice to quit post- versus pre-intervention being 1.74 (95% CI 1.42-2.12) for the intervention group (P for group by time interaction term = 0.004). Again, ASK was not significantly different when comparing intervention and control.

Within the intervention group, we found that greater participation in the intervention resulted in greater improvement, with increases in ADVISE of 4% among those who did not log on, 9% in those practices who viewed less than the median number of pages viewed, and 14% in those with the highest level of participation (above median). The cluster-adjusted odds ratios of patients receiving advice to quit post-intervention versus pre-intervention were 1.31 (0.88-1.34) for those intervention practices that did not log on, 1.59 (1.21-2.09) for those with less than the median number of pages viewed, and 1.92 (1.43 – 2.56) in those with the highest level of participation. Higher levels of participation were not associated with greater improvement in ASK.