Comparison of the Efficacy of Calculus Detection Between Ultrasonic Inserts and an Explorer

The primary goal of periodontal therapy is to remove biofilm and calculus deposits to promote a healthy periodontium. Non-surgical periodontal therapy, involving the removal of readily detectable calculus, helps to achieve and maintain the health of the adjacent periodontal tissues.^1-4 Removal of bacteria-harboring calculus and contaminated cementum reduces the levels of subgingival plaque and promotes the healing process.¹ Although the complete removal of calculus may not always be achievable, periodic re-evaluation of the periodontal tissues for signs of inflammation aids in the detection of residual calculus. The Old Dominion (ODU) 11/12 explorer is one of several standard instruments used for calculus detection in preventive and active periodontal therapy and for clinical licensure examinations. Evidence supporting the sensitivity and specificity of the ODU 11/12 explorer is limited^5,6; however, it serves as part of an armamentarium of instruments utilized subgingivally to thoroughly detect and remove calcified deposits. Since the visualization of the subgingival environment is limited without assisted technology, tactile sensitivity acquired through a variety of instruments is essential for the evaluation of the presence of hard deposits on the tooth structure.

Technologies are being developed to complement the use of an explorer for the accurate detection of subgingival calculus, including endoscopy, auto-fluorescence, spectro-optical technology, lasers, and ultrasonic-based devices.⁷ Endoscopy utilizes optical fibers and a light source to enter a periodontal pocket and magnify the environment from 24-48 times onto a monitor allowing the clinician to work in real-time while instrumenting the root surface.⁷ While the use of a dental endoscope requires a lengthy learning curve, necessitates extra treatment time, and requires investing in expensive equipment,^8,9 endoscopy has been shown to enhance calculus detection, particularly with regard to residual deposits.^10,11

The DIAGNOdent^™ (KaVo; Biberach, Germany) consists of an indium gallium arsenide (InGaAs) diode and utilizes autofluorescence technology by emitting light with a wavelength of 655nm onto the tooth structure primarily for the purpose of caries detection.¹² Auto-fluorescence technology can also used for calculus detection; however, few studies have evaluated the clinical effectiveness of this device for this purpose.^12,13 The DetecTar^™ (Dentsply Professional, York, PA) is a light-emitting diode utilizing spectro-optical technology delivered through an optical fiber attached to a cordless handpiece. Designed exclusively for calculus detection, not its removal, this device requires additional study to determine its efficacy in clinical situations.¹³ Lasers, specifically the KEY3 laser, combine the technology of the InGaAs diode for calculus detection and an Erbium-doped yttium aluminum garnet (Er:YAG) for calculus removal. Major limitations to the widespread use of this laser include its high cost and restricted use for dental hygienists due to state dental board regulations. Powered scaling or ultrasonic instrumentation is commonly used for the effective and efficient removal of biofilm and calculus.^12,14 However, the Perioscan (Sirona; Bensheim, Germany) is a piezoelectric ultrasonic instrument specifically developed for the dual purpose of calculus detection and removal.¹⁵ Current evidence is limited to in vitro studies; more in vivo studies are needed to determine the clinical efficacy of this technology.¹³

A variety of ultrasonic instrument designs exist for the purpose of biofilm and calculus removal during periodontal therapy. The Thinsert^® (Dentsply Sirona; York, PA, USA) ultrasonic insert (UI) has a thin tip design with a similar thickness to the ODU 11/12 explorer. This design allows for improved periodontal access and more effective subgingival adaptation. Right and left curved UIs are utilized during periodontal therapy for improved access to difficult areas such as furca, concavities, and depressions.^16,17 The combination of straight and curved UIs in site-specific areas throughout the dentition has been demonstrated to improve the outcomes of non-surgical periodontal therapy.¹⁸ During periodontal therapy, practitioners typically alternate between using an explorer for calculus detection and a variety of hand and ultrasonic instruments for calculus removal. This alternating process is repeated until the clinical endpoint has been achieved.

Considering that periodontal therapy appointments are often completed within fixed time intervals, efficiency of the periodontal therapy session is critical. One approach to improve efficiency is to use the same instrument for several functions. Since the diameter of the thin UI is similar to the ODU 11/12 explorer, it may serve the dual purpose of calculus detection as well as removal. Curved UIs have similarities to the ODU 11/12 explorer and may also be effective in calculus detection. The purpose of this study was to examine the effectiveness of calculus detection between a thin UI, curved UIs as compared to the ODU 11/12 explorer in experienced clinicians.

Methods

This study received approval from the Biomedical Institutional Review Board (2016H0273 and 2017H0275) of the Ohio State University (OSU). Three faculty members from the OSU Division of Dental Hygiene with clinical teaching assignments, along with current employment in a private practice or the non-profit setting at the OSU Dental Faculty Practice, volunteered to participate in the study. Faculty participants consisted of two full-time members (>.75 FTE) and one part-time member (<.75 FTE) and possessed a range of clinical dental hygiene experience from less than five years, ten to fifteen years, and over twenty years. Each faculty participant was randomly assigned a participant number.

Patient participants were recruited from the community by posting no-cost advertisements on an external website, internal monitors within the OSU School of Dentistry, and the OSU Study Search website. Patient participants were initially screened for availability during the dates of data collection and based on the eligibility criteria. Eligible participants were adults aged 18 and over with good general health and no history of a professional prophylaxis within the past six months. Exclusion criteria were individuals with uncontrolled systemic disorders and a history of professional prophylaxis within the past 6 months. All participants were provided with the rationale and design of the study. Eligible participants were randomly assigned a participant number, completed and signed informed consent, medical history, and Health Information Portability Accounting Act (HIPAA) forms prior to the start of the study.

An ODU 11/12 explorer, thin and curved UIs, disconnected from the ultrasonic unit, were used for this study. Presence or absence of readily detectable calculus was recorded on a standardized, color-coded calculus detection chart; each faculty participant used the same colored forms each assessment. Ramfjord index teeth (teeth numbers 3, 9, 12, 19, 25, and 28) and on four possible surfaces per tooth (mesial, buccal, distal, and lingual)¹⁹ were used to evaluate for the presence of calculus.

Faculty participants/raters underwent calibration training using three typodonts with differing surfaces of simulated calculus (Practicon Inc; Greenville, NC, USA) one week prior to the start of the study. The participants first used the ODU11/12 explorer to detect for calculus; evaluations were recorded on the standardized calculus detection chart. After a minimum of three days, participants detected for calculus using the thin UI and recorded their evaluations on the standardized calculus detection chart. All participants were found to be in the full agreement range for intra- and interrater reliability.

Data collection occurred over seven separate sessions held after normal clinic hours in the OSU Student Dental Clinics. The first three sessions compared calculus detection between the explorer and thin UI. During the final four sessions, calculus detection was compared using the explorer and curved UIs. Study session and participant flow is shown in Figure 1.

During each session, participants evaluated each patient for calculus using a random order and a randomly chosen instrument until all patient participants were evaluated for calculus. Each calculus detection chart was color coded per faculty rater. After each chart was completed, each chart was submitted into an individual patient folder to secure each faculty rater's evaluations from the other raters. In subsequent sessions the faculty rater evaluated each patient for calculus using a random order with the remaining instrument. The availability of six to eleven patients per session minimized the faculty rater's memory retention of the patients between the first and second rounds and also minimized fatigue, which could have altered the tactile sensitivity if all the instruments were used in a single session.

Data analysis

All statistical analysis was conducted using SPSS Version 25 (IBM, Chicago, IL, USA). The first aim of the study was to measure inter-rater reliability or the absolute agreement among more than two raters evaluating the same group of subjects. Two-way random-effects intraclass coefficient (ICC) values were determined using the faculty raters' evaluations using the ODU 11/12 explorer versus thin UI and the ODU 11/12 explorer versus curved UIs. Intraclass coefficient values less than 0.29 indicate poor reliability, between 0.30 and 0.49 suggests fair reliability, between 0.50 and 0.69 reveal moderate reliability, and values greater than 0.70 are indicative of strong reliability.^20,21 The second aim was to measure intra-rater reliability by determining Cohen's Kappa coefficient values using each faculty participant's calculus detection evaluations when using the ODU 11/12 explorer versus the thin UI and the ODU 11/12 explorer versus the curved UIs. Kappa values range from zero (no agreement) to one (perfect agreement) with values in the range of 0.41 to 0.60 indicating moderate agreement, 0.61 to 0.80 considered in full agreement, and values greater than 0.81 indicating perfect agreement.^22-24 The third aim was to determine whether the UI is a valid instrument for calculus detection. Sensitivity, specificity, positive predictive value (PPV), and negative predictive values (NPV) were calculated when using the thin UI and curved UIs versus the ODU 11/12 explorer to measure validity. A cross tabulation of all calculus evaluations was generated between the ODU 11/12 explorer (considered for the purposes of this study as the gold standard) versus the thin UI (test outcome) and the ODU 11/12 explorer (gold standard) versus the curved UIs (test outcome).²⁵ Sensitivity refers to detecting calculus when actually present, specificity refers to detecting the absence of calculus when actually absent, PPV refers to the calculus actually being present, and NPV refers to calculus actually being absent.²⁵

Results

A total of 60 eligible participants were recruited from the community to participate in this study comparing the effectiveness of calculus detection between the ODU 11/12 explorer versus thin and curved UIs as measured by three clinical faculty raters. Interrater reliability of calculus detection among the three faculty raters when using both the ODU 11/12 explorer and thin UI was shown to be an intraclass coefficient (ICC) of .782 with a confidence interval (CI) of .749 to .810 (F(1439, 2878)=4.852, p<0.05). When faculty raters used the ODU 11/12 explorer alone for calculus detection, the average ICC was .768 with a CI of .725 to .803 (F(719, 1438)=4.577, p<0.05). Using the thin UI alone, raters had an ICC of .790 with a CI of .750 to .820 (F(719, 1438)=5.011, p<0.05). In the sessions using the curved UIs, faculty raters using the ODU 11/12 explorer and curved UIs demonstrated an ICC of .714 with a CI of .684 to .741 (F(1439, 2878)=3.579, p<0.05) and the ICC while using the ODU 11/12 explorer alone was .737 with a CI of .701 to .769 (F(719, 1438)=3.858, p<0.05). Using only curved UI, the faculty rater ICC was .691 with a CI of .644 to .732 (F(719, 1438)=3.357, p<0.05). Interrater reliability results are shown in Table I.

The second aim of the study was to determine the intrarater reliability of calculus detection when using the ODU 11/12 explorer versus a thin UI and versus curved UIs. The average Kappa value for all three raters was .726 when using the ODU 11/12 explorer versus the thin UI while the average Kappa value for the raters when using the ODU 11/12 explorer versus curved UIs was .680. Interrater reliability Kappa values are shown in Table II.

Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) when using the ODU 11/12 explorer versus a thin UI and versus curved UIs were measured in regard to the third study aim; validation of thin and curved UIs as instruments for calculus detection. A total of 2,160 surfaces were evaluated for calculus using the explorer and the thin UI. Of those surfaces, 75% (n=322) were true positives, 96% (n=1,655) were true negatives, 4% (n=75) were false positives, and 25% (n=108) were false negatives. Sensitivity was 75%, specificity was 96%, PPV was 81% and NPV was 94% when comparing calculus detection using the thin UI (test outcome) to the ODU 11/12 explorer (gold standard). In evaluating curved UIs to the ODU 11/12 explorer, a total of 2,160 surfaces were evaluated for the presence of calculus. Sixty- five percent of those surfaces (n=141) were true positives, 98% (n=1,332) were true negatives, 3% (n=34) were false positives, and 35% (n=77) were false negatives. Sensitivity was 65%, specificity was 98%, PPV was 81% and NPV was 95% when comparing calculus detection using the curved UIs to the ODU 11/12 explorer (Table III).

Discussion

The purpose of this study was to evaluate the efficacy of calculus detection between thin and curved UIs, and the ODU 11/12 explorer. Intra- and interrater reliability was measured for the purposes of comparing calculus detection with UIs and an ODU 11/12 explorer. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of calculus detection were measured for the three types of instruments. Based on a generic x²-test and 0.05 level of significance, a sample subject size of 30 was determined to have an effect size of 1.0 with greater than 90% power. However, since limited amounts of calculus were present and detected with the participants, results of the present study can only be generalized to patient populations with similar, limited amounts of calculus.

Study participants using the thin UI demonstrated a sensitivity of 75%, specificity of 96%, PPV of 81% and NPV of 94%; however, when participants used curved UIs the sensitivity measured 65%, specificity 98%, PPV 81% and NPV 95% when considering the ODU 11/12 explorer as the gold standard for calculus detection. Both thin and curved UIs seemed more effective at detecting the absence of calculus versus the presence of calculus. Limited evidence over the past 10-30 years has revealed varying levels of sensitivity, specificity, and predictive values in the detection of calculus. Pippen and Feil, using manikins with artificial calculus, found a sensitivity 49%, specificity 88%, PPV 12%, and NPV 51%.⁵ In an in vivo study with human participants, Sherman et al. revealed a low false positive (FP=11.8%) and high false negatives (FN=77.4%).⁶ However in the Sherman et al. study, the clinical evaluation of calculus after scaling and root planing was compared to the subsequent microscopic evaluation of the extracted teeth. With novel technology, Meissner et al utilized a smart ultrasonic device for calculus detection and determined a sensitivity of 91%, specificity 82%, PPV 59%, and NPV 97%.¹⁵

Using the predetermined eligibility and inclusion criteria, recruiting participants with readily detectable calculus presented a challenge. However, previous studies revealed that between-rater agreement was greater when calculus was defined as readily detectable versus root roughness.^6,26 Out of the 4,320 total surfaces evaluated in the present study, 648 surfaces (20%) were determined to have calculus using the ODU 11/12 explorer exclusively and 463 surfaces (15%) were determined to have calculus when using both the 11/12 explorer and either the thin UI or the curved UI (true positive).

Although the three faculty raters had varying levels of clinical experience, this did not seem to affect calculus detection. Both within-rater and between-rater agreement levels were in the full agreement range. Specifically, the levels of within-rater agreement when using both the ODU 11/12 explorer and thin UI was Kappa =.726 and the levels of within-rater agreement when using both the ODU 11/12 explorer and curved UI was Kappa=.680. Previous research has shown that clinicians have a low ability to reproduce their evaluations with respect to calculus detection.^5,6 Using manikins with simulated calculus, Pippin and Feil revealed low within-rater agreement (Kappa= .330).⁵ In a study using human subjects, Sherman et al. found similar low within-rater agreement levels regarding the presence of calculus both prior to (Kappa=.220 - .370) and after scaling (Kappa= .040 -.270).⁶ In the current study, the levels of between-rater agreement were in the strong range within a narrow confidence interval with the ODU 11/12 explorer, the thin UI, curved UIs, all when used alone, as well as overall when using all instruments. This contrasts other studies which have demonstrated low levels of agreement between raters with regard to calculus detection.^5,6 However, Santiago et al. showed higher between-rater agreement evaluating for readily detectable calculus, defined as a "jump or bump" of the explorer (Kappa= .631) versus the tactile perception of root roughness (Kappa= .271).²⁶

Anecdotally, after completing the first stage of the study utilizing the thin UI the raters commented on feeling limited by the straight design of the insert and challenges existed with adapting the thin UI along line angles. While the curved UIs allowed for improved adaptation of the instrument along the line angles, the tip width affected the perceived tactile sensations. Development of a novel UI design combining the shape of a curved UI with the diameter of the thin UI may lead to improvements during the process of calculus detection and removal.

Limitations of this study included the number of patient participants with readily detectable calculus deposits, potential rater recall of the calculus deposits on the patient participants' teeth, and rater fatigue which may have affected tactile sensitivity. Future studies should investigate calculus evaluation utilizing other UI designs, measure the efficacy of UI calculus detection with varying amounts of calculus, and determine whether clinical experience, (dental hygiene students versus experienced clinicians), affects calculus detection using UIs.

Conclusion

Calculus detection was comparable when using the ODU 11/12 explorer, thin and curved UIs on patients with limited amounts of calculus. Efforts may be focused on developing tactile sensitivity when using both thin and curved UIs in the assessment, treatment, and maintenance of patients with sub-gingival calculus for increased efficiency in dental hygiene care. The development of a novel ultrasonic insert combining the shape of a curved UI with the diameter of thin UI may facilitate adaptation during the process calculus detection and subsequent removal. Future studies should investigate calculus evaluation utilizing novel ultrasonic insert designs, varying amounts of calculus, and varying levels of clinical experience.

Acknowledgments

The authors would like to thank Professor Emeritus Bob Rashid for his guidance with the statistical analysis; Marilyn Goulding at Dentsply Sirona USA for her help with supplying the ultrasonic instruments, and the following Ohio State University dental hygiene graduates, Sydney Balla, Sierra Poillucci, and Samantha Seitz, for providing the research support for the study.

Brian B. Partido, RDH, MSDH is an assistant professor; Chadleo Webb, RDH, MDH is a clinical instructor; Michele P. Carr, RDH, MA is an associate professor emeritus; all at the Ohio State University College of Dentistry, Division of Dental Hygiene, Columbus, OH.

Corresponding author: Brian Partido, RDH, MSDH; partido.1@osu.edu

References

1. Cobb CM. Clinical significance of non-surgical periodontal therapy: an evidence-based perspective of scaling and root planing. J Clin Periodontol. 2002 May;29 Suppl 2:6-16.

2. Corbet EF, Vaughan AJ, Kieser JB. The periodontally- involved root surface. J Clin Periodontol. 1993 Jul; 20(6):402-10.

3. Darby M, Walsh M. Dental hygiene theory and practice, 4th edition. Philadelphia, PA: W. B. Saunders Company; 2015. p. 55-73.

4. Henry RK, Goldie MP. Dental hygiene application to clinical practice. Philadelphia, PA: F. A. Davis Company; 2016. p. 394-407.

5. Pippin DJ, Feil P. Interrater agreement on subgingival calculus detection following scaling. J Dent Educ. 1992 May;56(5):322-26.

6. Sherman PR, Hutchens LH, Jr., Jewson LG, et al. The effectiveness of subgingival scaling and root planning. I. Clinical detection of residual calculus. J Periodontol. 1990 Jan;61(1):3-8.

7. Kwan JY. Enhanced periodontal debridement with the use of micro ultrasonic, periodontal endoscopy. J Calif Dent Assoc. 2005 Mar;33(3):241-48.

8. Stambaugh RV, Myers G, Ebling W, et al. Endoscopic visualization of the submarginal gingiva dental sulcus and tooth root surfaces. J Periodontol. 2002 Apr;73(4):374-82.

9. Kocker T, Strackeljan J, Behr D. Feasability of computer asssisted recognition of different dental heard tissues. J Dent Res. 2000 Mar; 79 (3): 829-38.

10. Osborn JB, Lenton PA, Lunos SA, et al. Endoscopic vs. tactile evaluation of subgingival calculus. J Dent Hyg. 2014 Aug;88(4):229-36.

11. Geisinger ML, Mealey BL, Schoolfield J, et al. The effectiveness of subgingival scaling and root planing: an evaluation of therapy with and without the use of the periodontal endoscope. J Periodontol. 2007 Jan;78(1):22-8.

12. Singh S, Uppoor A, Nayak D. A comparative evaluation of the efficacy of manual, magnetostrictive and piezoelectric ultrasonic instruments--an in vitro profilometric and SEM study. J Appl Oral Sci. 2012 Feb;20(1):21-6.

13. Archana V. Calculus detection technologies: where do we stand now? J Med Life. 2014;7 Spec No. 2:18-23.

14. Tunkel J, Heinecke A, Flemmig TF. A systematic review of efficacy of machine-driven and manual subgingival debridement in the treatment of chronic periodontitis. J Clin Periodontol. 2002;29 Suppl 3:72-81; discussion 90-1.

15. Meissner G, Oehme B, Strackeljan J, et al. Clinical subgingival calculus detection with a smart ultrasonic device: a pilot study. J Clin Periodontol. 2008 Feb;35(2):126-32.

16. Hodges KO. Components of optimal ultrasonic therapy. Today's FDA. 2014 May-Jun;26(3):58-61, 3.

17. Silva LB, Hodges KO, Calley KH, et al. A comparison of dental ultrasonic technologies on subgingival calculus removal: a pilot study. J Dent Hyg. 2012 Spring;86(2):150-8.

18. Hinchman SS, Funk A, DeBiase C, et al. Ultrasonic instrumentation instruction in dental hygiene programs in the United States. J Dent Hyg. 2016 Apr;90(2):135-42.

19. Rams TE, Oler J, Listgarten MA, et al. Utility of Ramfjord index teeth to assess periodontal disease progression in longitudinal studies. J Clin Periodontol. 1993 Feb;20(2):147-50.

20. Koo, TK, Li, MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016 Jun;15(2):155-63.

21. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979 Mar;86(2):420-8.

22. Guggenmoos-Holzmann I. The meaning of kappa: probabilistic concepts of reliability and validity revisited. J Clin Epidemiol. 1996 Jul;49(7):775-82.

23. McHugh ML. Interrater reliability: the kappa statistic. Biochemia Media. 2012 Oct;22(3):276-82.

24. Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005 Mar;85(3):257-68.

25. Parikh R, Mathai A, Parikh S, et al. Understanding and using sensitivity, specificity and predictive values. Indian J Ophthalmol. 2008 Jan-Feb;56(1):45-50.

26. Santiago LJ, Freudenthal JJ, Peterson T, et al. Dental hygiene faculty calibration using two accepted standards for calculus detection: a pilot study. J Dent Educ. 2016 Aug;80(8):975-82.