The Effect of Continuing Education on Dental Hygienists’ Knowledge, Attitudes, and Practices Regarding Human Papillomavirus Related Oropharyngeal Cancer

Introduction

Cancer is a worldwide health problem and the second leading cause of death in the United States (U.S.), surpassed only by  heart disease.¹ Surveillance data projected that in 2018, more than 51,000 Americans would develop oral cancer (OC) and oropharyngeal cancer (OPC), and that 10,030 would die from these cancers.^2,3 Although treatment and responses greatly vary, the percentage of patients surviving five years post OC and OPC diagnosis from 2007-2013 was 64.5%.³ With approximately 70% of these cancer diagnoses occurring in later stages,¹ the need for early detection, consisting of examination, practitioner education, and early diagnosis leading to early treatment and mortality reduction, is further supported.^4,5

The human papillomavirus (HPV), a common oral infection, has surpassed tobacco and alcohol as the major risk factor for oral cancer,⁴ and has been identified in up to 75% of OPCs.⁴ Human papillomavirus is comprised of more than 150 strains or types; one type in particular, HPV 16, is associated with most OPCs.^4,6-8 Although the virus currently infects about one in four persons in the U.S.,¹ it typically clears or resolves on its own by the body's immune system and does not cause health problems.^9,10 When an HPV infection does not resolve, however, cancer can result.¹⁰

The HPV is transferred from person to person (e.g. male/male, male/female, female/female) during vaginal, anal, and oral sex, as well as open mouth kissing.⁶ Research has shown those persons who develop HPV related OPC tend to be white males ages 40 to 70,^2,11,12 although women can also be affected.^2,12 Additional risk factors include number and frequency of oral, anal, and vaginal sex encounters, younger age sexual debut, and smoking.^8,11-16 The anatomy affected by HPV-related OPC includes the tonsils, pharynx, base of the tongue, soft palate, and cervical lymph nodes.^6,13,17,18 Signs and symptoms for HPV-related OPC include a persistent sore throat, chronic trouble or pain when swallowing, ear pain, hoarseness, an ipsilateral neck mass, and a persistent lump in the throat.^6,18,19

A standardized protocol for visual and tactile evaluation has been described by the World Health Organization, American Dental Association, and the National Institute of Dental and Craniofacial Research to include examination of the extra and intra-oral structures including the face, head, neck, lips, labial and oral mucosa, gingiva, floor of the mouth, tongue, soft and hard palate, including the oropharyngeal tissues.^17,20-23 Although HPV testing is available,^15,18 there is no evidence supporting a screening test for HPV-related OPC similar to the pap smear for cervical cancer.¹⁸ Additionally, a systematic review conducted by Lingen et al⁴ found that adjunct screening tools are not effective in helping to identify early OCs and OPCs. These studies accentuate the effectiveness of routine comprehensive visual and tactile oral examinations and a definitive diagnosis of OC and OPC made by histopathologic tissue assessment during biopsy.^5,24

Although clear guidelines for oral cancer and head and neck examinations have been established for the dental professional, research suggests that barriers exist to prevent practitioners from performing these procedures. A lack of literacy regarding HPV,^24,25 HPV-related OPC,^24,25,27 HPV risk factors,^26,27,28 examination practices,²⁶ anatomy affected,²⁶ and HPV prevention/vaccination^25,29 are among the most commonly cited barriers. Several studies also determined a deficit in practitioners' communication skills to effectively discuss HPV-related OPC, with the inclusion of risky sexual practices, HPV infection and prevention to be a barrier.^25,26,29-32 Other studies cite the lack of time to complete an examination,^28,31,32 and patient's lack of awareness regarding the exam performance and importance,²⁸ to be barriers.

Continuing education (CE) programs have been shown to influence knowledge deficits and impact practitioners' attitudes and practices.^27,33-38 With regards to HPV-related OPC topics, CE opportunities have been recommended in several studies to impact these professional attributes.^25,26,30,32 A before and after study conducted by Toftegaard et al³⁴ concluded an increase of timely patient oral cancer referrals following CE,³⁴ whereas a systematic review suggested the CE for medical providers to be useful in improving professional practice and patient healthcare outcomes, especially when coupled with multiple learning methods (i.e. interactive format).³⁹ Additional studies have evaluated dentists' and dental hygienists' oral cancer understanding and behavior changes post education intervention, finding statistical differences in tested knowledge following CE.^27,37,38 Additionally, self-reported communication skills were found to be statistically significant following CE intervention,^25,33 as were self-reported examination practices.^33,36,38

Previous studies suggested a need for HPV-related OPC awareness, particularly in regards to the visual and tactile head and neck examination performance^25,26,30,31; however, many known barriers have been identified.^25-32 Research has shown that CE attendance impacts the knowledge and practices of medical and dental professionals.^27,33-38 A review of the literature shows a lack of recent research regarding the association of continuing education on dental hygiene practitioner's performance. The purpose of this study was to determine the effect of a web-based CE course on dental hygiene practitioners' knowledge, attitudes, and practices (KAP) regarding HPV-related OPC.

Methods

This two-group experimental post-test only design was granted exempt status by the Idaho State University Institutional Review Board (IRB-FY2018-323). To ensure that the experimental and control groups were equal, random selection and random assignment of participants to each group were used. A post-test design approach was chosen to control for pre-test sensitization, which can cause individuals to score higher when they take a test for the second time regardless of the intervention.⁴⁰ In this instance, differences between a pre-test and a post-test score may not be a result of the independent variable but rather a result of the testing itself.⁴⁰ Therefore, a post-test design was the most appropriate option for this study and served to strengthen the study's internal and external validity. The lack of a pre-test was not thought to impact the results. The study population consisted of licensed Florida dental hygienists. Email addresses were obtained from the Florida Department of Health website. Inclusion criteria were limited to dental hygienists licensed in the state of Florida and practicing a minimum of two days per week. A power analysis determined the minimum size needed for this study was 128 participants. A computerized, randomization process performed by Excel^® (Microsoft; Bellevue, WA) selected the sample for the study. Once the individuals consented to participate, they were randomly assigned to either the control or experimental group.

The web-based CE course was developed by the principal investigator, who served as the subject matter expert to design the evidence-based content, based on five years of experience educating entry-level dental hygiene students on this topic. The following principles of instructional design were used: creating a task analysis, developing objectives, planning the lesson and instructional strategies including the case studies, and assessing learning.⁴¹ The components of the instructional design process are shown in Table I. The course content was reviewed by a member of the research team who was a participant in the Lingen et al⁴ systematic review, and also had 40 years of teaching experience in oral pathology.

The data collection instrument was a self-generated electronic questionnaire consisting of items assessing the dental hygiene practitioners' knowledge, attitudes, and practices related to HPV-related OPC and demographics. Knowledge was assessed through 15 multiple choice questions; attitudes were assessed through 11 statements, using a Likert Scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree); practices were assessed through 13 statements, using a Likert Scale ranging from 1 (0% of the time) to 5 (100% of the time); and demographic questions in a multiple-choice format.

The questionnaire was validated prior to data collection with a content validity index (CVI) consisting of a four-point rating scale: 1 = not relevant, 2 = unable to assess relevance without question revision, 3 = relevant but needing slight revision, and 4 = relevant.⁴⁰ Five dental hygiene professionals familiar with HPV-related OPC rated each question. A CVI score was computed for each question by dividing the number of experts rating 3 or 4, divided by five experts. Questions scoring less than .75 were either deleted or rewritten according to rationale and suggestions provided by the panel. Reliability was established by a test-retest completed one week apart by a panel of 12 dental hygiene professionals familiar with clinical practice terms and responsibilities. The reliability coefficient was determined by dividing the number of same question responses by 12. The reliability coefficient was set at .70, and modifications were made to questions scoring less than .70 to improve the final instrument.⁴⁰

An invitation to participate was sent to Florida dental hygienists and included information about the principal investigator, topic of the study, participation expectations, and incentive for study participation. The experimental group participated in a one-hour web-based CE course presented through a videoconferencing platform (Zoom^®; San Jose, CA). The course content was case-study based, and included HPV OPC epidemiology, risk factors, symptoms, examination practices, diagnosis, and prevention. Six weeks following the CE intervention, both groups received an invitation to participate in the post-test. The link to the questionnaire was sent through an online survey tool (Qualtrics^®; Provo, UT). Informed consent information was on the first page of the invitation to participate. Data collection took place over a two-week time frame, and the invitation and survey link were resent twice during this time to increase participation. A $50 Visa or Amazon gift card drawing was offered as incentive to complete the questionnaire.

The data collected were downloaded without identifier information into an Excel^® file, and confidentiality of responses was maintained through Qualtrics^®. Data were analyzed using both descriptive statistics and analysis of variance (ANOVA) to test differences between the experimental and control groups. The level of significance was established at p=0.05.

Results

A total of 302 dental hygienists licensed in the state of Florida met the inclusion criteria and agreed to participate. However, only 133 participants completed the entire study (n=133), yielding a response rate of 44.04%. Participants were randomly assigned to either the experimental (n=76) or the control group (n=78). However, while 76 individuals began in the experimental group, only 71% (n=55) completed the entire survey. Only fully completed surveys were counted in this study. The majority of the participants were between the ages of 36 to 55 years, graduated in the 2000s, and were employed full-time in clinical practice. Demographic information is highlighted in Table II.

Knowledge of HPV in relation to OPC was examined; Table II summarizes the percentage of correct knowledge item responses for both groups. The mean score of the control group was 8.76, equivalent to a 58.4% test score, and the mean score of the experimental group was 10.89, equivalent to a 72.6% test score. The difference between the groups was 14.2%, the equivalent of two test items. ANOVA analysis determined a statistically significant difference between the knowledge of experimental group regarding HPV-related OPC as compared to the control group (p=0.00, Table III). Cohen's d effect size was 1.05, a large effect size, representing a considerable difference between the groups.

Participants were asked to express their attitudes about HPV-related OPC. Responses comparing the control and experimental group are shown in Table V. The majority of respondents from both groups agreed or strongly agreed that it is their responsibility to identify risk factors on the health history associated with HPV-related OPC (94.6% experimental group, 93.5% control group); to examine all patients for oral cancer (100% experimental group, 98.7% control group); and to visualize the oropharyngeal area during an oral cancer examination (98.1% experimental group, 100% control group). Additionally, participants in both groups valued patients deserving high quality of care (100% both groups) and continuing education to maintain current skills and practices (100% both groups). However, there were differences in attitudes demonstrated by both groups related to the collection of a thorough health history including a sexual history inquiry on all patients; responsibility to teach patients about high risk sexual practices associated with HPV-related OPC; and, employer's responsibility to set the standard for dental hygiene clinical care (responses varied across each of the scales). ANOVA analysis revealed a statistically significant difference between the experimental and control groups' attitudes regarding HPV-OPC (p=0.00). A summary of the ANOVA of the participants knowledge, attitudes, and practices is shown in Table IV. Cohen's d effect was 0.66, a medium effect size, representing a moderate difference between the groups.

Respondents reported on the frequency of their practices related to oral examinations and any HPV specific procedures. More than half of the respondents indicated that they conducted an extra-oral examination on new patients at least 75% of the time or more (experimental group 65.5%, control group 62.8%) and for re-care patients (experimental group 56.4%, control group 59%). In comparison, the vast majority of respondents reported completing an intra-oral examination on new and re-care patients (experimental group 92.7%, control group 92%) more frequently than an extra-oral exam (experimental group 93.3%, control group 85.9%). ANOVA analysis showed no statistically significant difference between the groups on oral examination practices (p=0.67). Dental hygienists' practice behaviors related to extra and intraoral examinations and HPV specific procedures are shown in Table VI.

The ANOVA results comparing the experimental and control groups related to HPV specific practices identified a statistically significant difference (p=0.007) in this area. Cohen's d effect size was 0.48 for these practice items, representing a small effect size, but nearly a medium effect size (.50), indicating a small to moderate difference between the groups. The results also demonstrated that the majority of participants were not holding discussions with their patients about HPV risk factors, regardless of the patient's age, and nearly to 50% of the time, participants were not discussing HPV vaccinations (experimental group 83.6%, control group 93.6%).

Discussion

The one-hour web-based CE course on HPV-related OPC in this study was shown to be an effective method for increasing knowledge, attitudes and HPV specific practices; however, it was not shown to be effective in changing oral examination practices in the population studied. Knowledge was measured by correct responses to multiple-choice questions as an objective evaluation, whereas attitudes and practices relied on self-reported measurements. Although the experimental group had higher scores on the knowledge items than the control group, the average score was still low, thus indicating the need for ongoing education on this topic. Participants were not aware of the significant relationships between HPV and OPC and risk factors commonly associated with HPV-related OPC. This finding is consistent with the lack of knowledge about oral cancer among dentists and physicians when measured by an objective evaluation.²⁷

The knowledge portion of this study showed that many dental hygienists were not fully informed of the extra oral anatomy associated with HPV related OPC, recognition of lymph node involvement, best visualization of oropharyngeal anatomy, signs and symptoms, and re-evaluation procedures based on presenting signs and symptoms. These findings conflict with a previous study's results where the majority of dental hygiene participants self-rated their knowledge and skills regarding examination practices to be very good and regarded themselves as being highly knowledgeable about oral cancer examination practices.²⁸ Dental hygienists' knowledge might be more accurately measured through an objective evaluation compared to self-rated assessments.

The vast majority of respondents from this study (98-100%) shared the attitude that oral cancer examinations should be conducted on all patients. Similar results were reported by Tax et al,²⁸ where examination procedures were perceived to be an important part of the dental hygienist's scope of practice. Forrest et al⁴² further confirmed this finding where all respondents believed that oral cancer examinations should be performed on adult patients aged 40 years and older. Likewise, Marino et al²⁶ found 95.2% of the respondents agreed that these examinations should be routinely performed. This study of Florida dental hygienists concurs with previous findings demonstrating a continued trend in practitioner oral examination attitudes, while mixed practices correlating to this belief have been documented.

Although overall examination practices were high, fewer respondents indicated that they were palpating the cervical lymph nodes as part of an extraoral examination, and only about two thirds of the respondents examined the tonsils and middle part of the throat as well as the base of the tongue. These anatomical structures are directly related to HPV OPC and regular examination of these structures is essential for early cancer diagnosis. Equally relevant, not all practitioners informed patients they were conducting an oral cancer examination. This particular finding supports other studies in which patients reported they were not told that an oral cancer examination was being performed by their oral health provider,⁴³ and dentists indicated that they did not always tell their patients that they had completed an oral cancer examination.³³ Dental hygienists play a vital role in increasing patients' awareness of oral examinations, and in turn, may increase patients' expectations and perceptions of the value of this procedure. Standards of practice dictate that oral health professionals should perform a comprehensive visual and tactile oral examination on all patients at every appointment.⁴

Practices specific to HPV were separated out and more closely examined where significant differences were found between the groups. A notable finding of this study was that although many participants agreed/strongly agreed that it was their responsibility to conduct a thorough health history, including a sexual history on all patients, the practice component of the survey showed that many were less inclined to have discussions about HPV-OPC risk factors and sexual practices with patients depending on the patient's age. This same finding held true in regard to attitudes and practices related to discussing HPV vaccinations with all patients. Similar results were reported by Thompson et al²⁵ in their study of communication with regard to HPV disease transmission, HPV related cancers and vaccination information among dental hygienists. Kline et al⁴⁴ likewise described the significant role of dental practitioners in HPV-OPC prevention, but also identified barriers specific to HPV discussions, including fear of offending patients, and lack of privacy and time. Similarly, a systematic review conducted by Walker et al⁴⁵ identified practitioners to be less likely to recommend or discuss HPV vaccination if they were uncomfortable discussing sexual practices. Rising rates of HPV-related OPC necessitate developing strategies to overcome oral health practitioners discomfort addressing sexual history and HPV vaccination.

While this study found significant statistical differences between the control and experimental groups regarding knowledge, attitudes, and HPV-related practices, there appears to be a disconnect between respondents' attitudes and actual practices. Investigation into the levels of the affective domain of learning may afford a better understanding of how attitudes are translated to practice. In the affective domain, attitudes are represented by the value one places on something.⁴⁶ Practice, a higher level of development known as characterizing, is achieved when attitudes are accepted and acted upon.⁴⁷ Further examination of the affective domain levels in relation to dental hygienists' attitudes and practices may be used to enhance future education on the topic of HPV-related OPC.

When considering the American Dental Hygienists' Association (ADHA) Code of Ethics, dental hygiene professionals have a responsibility "to provide oral health care utilizing high levels of professional knowledge, judgement, and skill."⁴⁷ Additionally, the dental hygienist is accountable for upholding the ADHA Standards for Clinical Dental Hygiene Practice, which includes comprehensive extra- and  intraoral examination practices and patient care.⁴⁸ The dental hygiene profession must be part of a health care team that is proactive about prevention and early detection of disease. Continuing education is one method of increasing practitioner knowledge; however, alternate methods of ongoing professional development and self-enhancement should be investigated regularly.

Multiple resources are available for patient assessment and examination procedures specifically related to oral pathologies including those provided by the American Dental Association.^21,49 Additionally, the Centers for Disease Control and Prevention have numerous resources on the topics of HPV,⁵⁰ HPV risk factors,⁵¹ vaccination⁵² and how to take a sexual history.⁵³ Dental hygienists have also expressed interest in participating in continuing education programs related to these topics.²⁵ Courses could be more effective in changing behaviors by incorporating more interactive strategies³⁹ and opportunities requiring audience participation, such as those supported by Phillips et al⁵⁴ for active online CE learning and those by Griscti et al⁵⁵ on the effectiveness of CE programs.

Following the CE webinar for this study, the experimental group participants requested a video on the extra and intra oral examination. Interest was also expressed for guidance on conducting a sexual history, and how to discuss the HPV vaccine. A web-based CE could be enhanced by including an online module with the recorded CE course featuring permanent links to resources associated with course content. This platform aligns with the principles of adult learning which include self-directed, facilitated guidance with the inclusion of videos, resources, and technology.⁴¹

This study had limitations. The questionnaire used was an original design; however, efforts were made to balance this limitation by establishing content validity and reliability. Participants may have provided inaccurate favorable responses to some of the survey items due to self-perceived obligations related to job performance. In addition, the six-week length of time between the CE course and post-test and the small population size may limit the breadth of data received. There was a loss of participants from the experimental group from the beginning of the study to completion of the survey, although no pattern was identified among the incomplete surveys. Explanations for this loss in numbers might be attributed to the difficulty of the survey, time commitment of the respondents, or the better retainment of information by those completing the survey. It should be noted, however, that the study design included a randomization procedure to identify the sample, and random assignment was further used to select control and experimental groups.

Additional research in the area of HPV-related OPC and dental hygiene knowledge, attitudes and practice is recommended. Identifying what types of interventions are shown to increase knowledge about HPV-related OPC is important as HPV-related OPC rates continue to rise. Dental hygienists are ideally positioned to be part of the health care team that are effective in prevention, treatment and outcomes of this disease. Determining the drivers for practice change is needed, as knowledge does not necessarily translate to practice. Identifying how dental hygienists can address sensitive topics such as HPV risk factors and sexual practices, how to take a sexual history from individuals spanning all ages and providing vaccination recommendations requires further attention from both a qualitative and a quantitative research perspective.

Conclusion

This study explored the impact a continuing education course on HPV-related OPC epidemiology, risk factors, symptoms, examination procedures, diagnosis, and prevention in regard to dental hygienists' knowledge, attitudes and practices concerning HPV-related OPCs. Results from this study identified statistically significant differences between the experimental and control groups in the areas of knowledge and attitudes. While no differences were found between the groups in relationship to examination procedures, statistically significant differences were noted for items related to HPV specific practices. Additional research is needed to appreciate what specific types of professional development interventions, such as access to resources, interactive discussion formats, and coordinated hands-on activities, would increase the HPV-related OPC knowledge and practice behaviors of dental hygienists.

About the Authors

Toni M. McLeroy, CRDH, MS is an associate professor, Dental Hygiene Department, State College of Florida, Manatee-Sarasota, FL; JoAnn Gurenlian, RDH, PhD,  AFAAOM, is a professor and the Graduate Program Director, Dental Hygiene Department, Idaho State University, Pocatello, ID; Ellen J. Rogo, RDH, PhD is a professor, Dental Hygiene Department, Idaho State University, Pocatello, ID.

Corresponding author: Toni M. McLeroy, CRDH, MS; mclerot@scf.edu

References

1. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2018. CA Cancer J Clin. 2018 Jan-Feb;68(1):7-30.

2. American Cancer Society. Cancer facts & figures 2018 [Internet]. Atlanta (GA): American Cancer Society; [cited2018 Feb 1]. Available from https://www.cancer.org/content/dam/cancer-org/research/cancer-facts-and-statistics/annual-cancer-facts-and-figures/2018/cancer-facts-and-figures-2018.pdf

3. National Cancer Institute: Cancer stat facts: Oral cavity and pharynx cancer [Internet]. Bethesda (MD): US Department of Health & Human Services; 2018 [cited 2018 Feb 24] Available from https://seer.cancer.gov/statfacts/html/oralcav.html

4. Lingen MW, Abt E, Agrawal N, Chaturvedi AK, Cohen E, D'Souza, G, et al. Evidence-based clinical practice guideline for the evaluation of potentially malignant disorders in the oral cavity: A report of the American Dental Association. J Am Dent Assoc. 2017 Oct;148(10):712-27.e10.

5. Kerr R. AAOM clinical practice statement: Subject: Oral cancer examination and screening. Oral Surg Oral Med Oral Pathol. 2016 Aug;122(2):174-5.

6. Centers for Disease Control and Prevention: HPV Vaccine Information Statement [Internet]. Atlanta (GA): U.S. Department of Health & Human Services; 2016 Dec [cited 2018 Mar 12]. Available from https://www.cdc.gov/vaccines/hcp/vis/vis-statements/hpv.html

7. American Cancer Society: Risk factors for oral cavity and oropharyngeal cancers [Internet] Atlanta (GA): American Cancer Society; 2018 Mar [cited 2018 Mar 12]. Available from https://www.cancer.org/cancer/oral-cavity-and-oropharyngeal-cancer/causes-risks-prevention/risk-factors.html

8. Gillison ML, Chaturvedi AK, Anderson WF, Fakhry C. Epidemiology of human papillomavirus-positive head and neck squamous cell carcinoma. J Clin Oncol. 2015 Oct;33(29):3235-42.

9. Engstrom-Melnyk J. The biology of HPV infections: Understanding the progression to cervical cancer. MLO Med Lab Obs. 2014 Apr.

10. Centers for Disease Control and Prevention: What is HPV? [Internet]. Atlanta (GA): U.S. Department of Health & Human Services; 2016 Dec [cited 2018 Feb 1]. Available from https://www.cdc.gov/hpv/parents/whatishpv.html

11. SEER cancer statistics review, 1975-2001 [Internet]. Bethesda (MD): National Cancer Institute; n.d. [cited 2018 Feb 1]. Available from https://seer.cancer.gov/archive/csr/1975_2001/

12. Chaturvedi AK, Graubard BI, Pickard RK, Xiao W, Gillison ML. High-risk oral human papillomavirus load in the US population, national health and nutrition examination survey 2009-2010. J Infect Dis 2014 Aug;210(3):441-7.

13. LeHew CW, Weatherspoon DJ, Peterson CE, et al. The health system and policy implications of changing epidemiology for oral cavity and oropharyngeal cancers in the United States from 1995 to 2016. Epidemiol Rev. 2017 Jan;39(1):132-47.

14. Shigeishi H, Sugiyama M. Risk factors for oral human papillomavirus infection in healthy individuals: A systematic review and meta-analysis. J Clin Med Res. 2016 Oct;8(10):721-9.

15. D'Souza G, McNeel TS, Fakhry C. Understanding personal risk of oropharyngeal cancer: Risk groups for oncogenic oral HPV infection and oropharyngeal cancer. Ann Oncol. 2017;28(12):3065-9.

16. Chakravarthy A, Henderson S, Thirdborough SM, et al. Human papillomavirus drives tumor development throughout the head and neck: Improved prognosis is associated with immune response largely restricted to the oropharynx. J Clin Oncol. 2016 Dec;34(34):4132-41.

17. Burd EM. Human Papillomavirus laboratory testing: The changing paradigm. Clin Microbiol Rev. 2016 Apr;29(2):291-319.

18. Berman TA, Schiller JT. Human papillomavirus in cervical cancer and oropharyngeal cancer: One cause, two diseases. CA Cancer J Clin. 2017 June;123(12):2219-29.

19. Whang SN, Filippova M, Duerksen-Hughes P. Recent progress in therapeutic treatments and screening for the prevention and treatment of HPV-associated head and neck cancer. Viruses. 2015 Sept;7(9):5040-65.

20. World Health Organization. A digital manual for the early diagnosis of oral neoplasia: Physical examination of the oral cavity [Internet]. Lyon (FR): International Agency for Research on Cancer; 2018 [cited 2018 Mar 12]. Available from http://screening.iarc.fr/atlasoral_list.php?cat=H2&lang=1

21. American Dental Association. Evidence-based clinical practice guideline for the evaluation of potentially malignant disorders in the oral cavity: A report of the American Dental Association [Internet]. Chicago (IL): 2017 [cited 2018 Feb 1]. Available from https://ebd.ada.org/~/media/EBD/Files/10870A_Chairside_Guide_OralCancer_FINAL.pdf?la=en

22. National Institute of Health. Detecting oral cancer: A guide for health care professionals [Internet] Bethesda (MD): National Institutes of Dental and Craniofacial Research; 2013 July [cited 2018 Feb 1]. Available from https://www.nidcr.nih.gov/sites/default/files/2017-09/detecting-oral-cancer-poster.pdf

23. Rethman MP, Carpenter W, Cohen EE, et al. Evidence based clinical recommendations regarding screening for oral squamous cell carcinomas. J Am Dent Assoc. 2010 May;141(5):509-20.

24. Brocklehurst P, Kujan O, O'Malley LA, et al. Screening programmes for the early detection and prevention of oral cancer. Cochrane Database Syst Rev. 20; Nov;(11):CD004150.

25. Thompson EL, Daley EM, Vamos CA, et al. Health literacy approaches to improving communication between dental hygienists and patients for HPV-related oral cancer prevention. J Dent Hyg. 2017 Aug;91(4):37-45.

26. Marino R, Haresaku S, McGrath R, et al. Oral cancer screening practices of oral health professionals in Australia. BMC Oral Health. 2017 Dec;17(151):1-9.

27. Hassona Y, Scully C, Shahin A, et al. Factors influencing early detection of oral cancer by primary health-care professionals. J Cancer Educ. 2016 June;31(2):285-91.

28. Tax CL, Haslam SK, Brillant M, et al. Oral cancer screening: Knowledge is not enough. Int J Dent Hyg. 2017 Aug;15(3):179-86.

29. Vazquez-Otero C, Vamos CA, Thompson EL, et al. Assessing dentists' human papillomavirus-related health literacy for oropharyngeal cancer prevention. J Am Dent Assoc. 2018 Jan;149(1):9-17.

30. Minassian M, Fried JL, Ord RA. HPV-positive head and neck cancers: A review of the literature. J Dent Hyg. 2014 Aug;88(4):194-201.

31. Rozier RG, Horowitz AM, Podschun G. Dentist-patient communication techniques used in the United States: The results of a national survey. J Am Dent Assoc. 2011 May;142(5):518-30.

32. Clark AK, Kobagi N, Yoon MN. Oral cancer screening practices of Canadian dental hygienists. Int J Dent Hyg. 2018 May;16(2):e38-45

33. Awojobi O, Newton JT, Scott SE. Pilot study to train dentists to communicate about oral cancer: The impact on dentists' self-reported behavior, confidence and beliefs. Br Dent J. 2016 Jan;220(2):71-6.

34. Toftegaard BS, Bro F, Falborg AZ, Vedsted P. Impact of a continuing education meeting on the use and timing of urgent cancer referrals among general practitioners - a before-after study. BMC Fam Pract. 2018 Mar;18(44):1-13.

35. Ahmad K, Wang TT, Ashrafian H, et al. The effectiveness of continuing education for specialist recertification. Can Urol Assoc J. 2013 July-Aug;7(7-8):266-72.

36. Selah A, Kong YH, Haron N, et al. Oral cancer screening in private dental practices in a developing country: Opportunities and challenges. Community Dent Oral Epidemiol. 2017 Apr;45(2):112-9.

37. Walsh MM, Rankin KV, Silverman, S. Influence of continuing education on dental hygienists' knowledge and behavior related to oral cancer screening and tobacco cessation. J Dent Hyg. 2013 Apr;87(2):95-105.

38. Hertrampf K, Wenz HJ, Koller M, et al. Early detection of oral cancer: Dentists' opinions and practices before and after educational interventions in Northern-Germany. Journal Craniomaxillofac Surg. 2013 Dec;41(8):e201-7.

39. Forsetlund L, Bjorndal A, Rashidian A, et al. Continuing education meetings and workshops: Effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009 Apr;15(2):CD003030.

40. LoBiondo-Wood G, Haber J. Nursing research: Methods and critical appraisal for evidence-based practice. 8th ed. St. Louis: Elsevier; 2014.552p.

41. Morrison GR, Ross SM, Morrison JR, Kalman HK. Designing effective instruction. 8th ed. Hoboken: John Wiley & Sons, Inc; 2019. 512p.

42. Forrest JL, Horowitz AM, Shmuely Y. Dental hygienists' knowledge, opinions, and practices related to oral and pharyngeal cancer risk assessment. J Dent Hyg. 2001 Fall;75(4):271-81.

43. Awojobi O, Scott SE, Newton T. Patients' perceptions of oral cancer screening in dental practice: A cross-sectional study. BMC Oral Health. 2012 Dec;12(55):1-9.

44. Kline N, Vamos, C, Thompson E, et al. Are dental providers the next line of HPV-related prevention? Providers' perceived role and needs. Papillomavirus Res. 2018 Jun;5:104-8.

45. Walker KK, Jackson RD, Sommariva S, et al. USA dental health providers' role in HPV vaccine communication and HPV-OPC protection: a systematic review. Hum Vaccin Immunother. 2019 Jan;15(7-8):1863-9.

46. Krathwohl DR, Bloom BS, Masia BB. A taxonomy of educational objectives: Handbook II. The affective domain. New York: McKay; 1964. 196p.

47. American Dental Hygienists' Association. Bylaws and code of ethics [Internet]. Chicago (IL):2016 June. Available from http://www.adha.org/resources-docs/7611_Bylaws_and_Code_of_Ethics.pdf

48. American Dental Hygienists' Association. Standards for clinical dental hygiene practice [Internet]. Chicago (IL): America Dental Hygienists' Association; 2016 June [cited 2018 Feb 1]. Retrieved from https://www.adha.org/resources-docs/2016-Revised-Standards-for-Clinical-Dental-Hygiene-Practice.pdf

49. American Dental Association. How to check patients for oral cancer [Internet]. Chicago (IL): American Dental Association; 2017 Oct [cited 2019 Jan 15]. Available from https://www.youtube.com/watch?v=7mv073MJzlg&feature=youtu.be

50. Centers for Disease Control and Prevention. HPV [Internet]. Atlanta (GA): Centers for Disease Control and Prevention; 2018 Apr [cited 2018 June 2]. Available from https://www.cdc.gov/hpv/index.html]

51. Centers for Disease Control and Prevention. Sexually transmitted diseases [Internet]. Atlanta (GA): Centers for Disease Control and Prevention; 2017 Jan [cited 2019 Jan 15]. Available from https://www.cdc.gov/std/healthcomm/stdfact-stdriskandoralsex.htm

52. Centers for Disease Control and Prevention. HPV vaccine recommendations [Internet]. Atlanta (GA): Centers for Disease Control and Prevention; 2016 Dec [cited 2019 Jan 15]. Available from https://www.cdc.gov/vaccines/vpd/hpv/hcp/recommendations.html

53. Centers for Disease Control and Prevention. Guide to collecting a social history [Internet]. Atlanta (GA): Centers for Disease Control and Prevention; 2019 [cited 2019 Jan 15] Available from https://www.cdc.gov/std/treatment/sexualhistory.pdf

54. Philips JM. Strategies for active learning in online continuing education. J Contin Educ Nurs. 2005 Mar Apr;36(2):77-83.

55. Griscit O, Jacono J. Effectiveness of continuing education programmes in nursing: literature review. J Adv Nurs. 2006 August;55(4):449-56.