INSOMNIA
Defining Insomnia: Quantitative Criteria for Insomnia Severity and Frequency Margaret D. Lineberger, PhD1; Colleen E. Carney, PhD1; Jack D. Edinger, PhD1,2; Melanie K. Means, PhD1,2 Duke University Medical Center, Durham, NC; 2Veterans Affairs Medical Center, Durham, NC
1
ity criterion increased. Analyses of mean sleep-log data showed that an average sleep-onset latency or middle-of-the-night wake time (ie, time awake between sleep onset and final morning awakening) cutoff of 20 minutes or longer over 2 weeks of sleep-log monitoring appeared to best maximize sensitivity (94.4%) and specificity (79.6%) for insomnia classification. Conclusions: The optimal quantitative insomnia criteria found herein differ from those previously proposed. Nonetheless, results suggest that quantitative criteria derived from sleep-log data may be useful for classification of primary insomnia. Keywords: Primary insomnia, quantitative criteria, ROC analyses, sensitivity and specificity Citation: Lineberger MD; Carney CE; Edinger JD et al. Defining insomnia: quantitative criteria for insomnia severity and frequency. SLEEP 2006;29(4): 479-485.
Study Objective: Recent efforts have been made to develop quantitative frequency, duration, and severity criteria for insomnia. The current study was conducted to test a range of frequency and severity criteria sets for discriminating primary insomnia sufferers from normal sleepers. Participants: Seventy-two adults with primary insomnia and 88 agematched normal sleepers. Methods: Participants completed 14 consecutive nights of sleep logs to monitor their home sleep patterns. Receiver-operator characteristic curve analyses were used to compare a range of severity and frequency criteria sets for discriminating the insomnia and normal-sleeper groups. In addition, sensitivity and specificity tests were conducted for a range of waketime severity cutoffs based on 2-week mean sleep-log data. Results: Receiver-operator characteristic curve analyses showed that no 1 combination of severity and frequency criteria maximized sensitivity and specificity. Rather, the optimal frequency cutoff decreased as the sever-
Recently, there have been a number of efforts to develop standardized definitional criteria for use in insomnia research.11-13 Noteworthy among these are the recent efforts of Lichstein et al11 to develop quantitative frequency, duration, and severity criteria for defining insomnia. From their review of the literature regarding the behavioral treatment of insomnia, these investigators discerned that difficulty with sleep onset or maintenance occurring at a rate of 3 or more nights per week for a minimum period of 6 months, respectively, represented modal frequency and duration criteria for defining insomnia samples. However, their literature review provided less guidance in ascertaining quantitative severity criteria for insomnia (i.e., minutes to sleep onset or minutes awake during the night). As a result, Lichstein et al conducted sensitivity and specificity analyses using several commonly employed severity cutoffs (e.g., subjective onset latency ≥ 30 minutes, or ≥ 31 minutes, or ≥ 40 minutes, or ≥ 60 minutes) in a community sample. In the absence of structured interviews, participants were dichotomized into presumptive groups of primary (psychophysiologic) insomnia sufferers and normal sleepers on the basis of questionnaire measures of sleep and waking symptoms. Results of these analyses showed that a sleep-onset latency (SOL) or wake time after sleep onset (WASO) (i.e., wake time after initial sleep onset and before final morning awakening) ≥ 31 minutes, when combined with the aforementioned frequency and duration criteria, represented the optimal severity cutoff for discriminating the insomnia and normal-sleeper groups. Thus, these authors proposed a reported SOL or WASO of ≥ 31 minutes occurring at least 3 times per week for at least 6 months as the most defensible quantitative definition for insomnia. The present study further investigated quantitative frequency and severity criteria for insomnia using age- and sex-matched samples of primary insomnia sufferers and normal sleepers. Sleep diagnoses were determined using standardized structured psychiatric and sleep interviews as well as screening polysomnography (PSG). The purpose of this research was to determine empirically
INTRODUCTION OVER THE PAST 45 YEARS, THE BASIC AND CLINICAL RESEARCH LITERATURE DEVOTED TO INSOMNIA HAS GROWN DRAMATICALLY. UNFORTUNATELY, THIS research has been plagued by an unacceptable lack of standardization, largely due to inconsistency in researchers’ insomnia definitions. For example, some liberal definitions1,2 focus solely on the presence of nocturnal insomnia symptoms (e.g., difficulties with sleep initiation or maintenance, nonrestorative sleep), whereas other more conservative definitions require additional features such as associated daytime impairment,3,4 sleep dissatisfaction,5 or meeting the more exacting criteria for a specific insomnia diagnosis described in 1 of the available nosologies.6-9 In addition, insomnia definitions have varied as a function of inconsistent use of frequency, duration, symptom-type (e.g., onset problems, maintenance difficulties), and/or severity criteria for case ascertainment.10 As demonstrated recently by Ohayon,10 use of these varied definitions in epidemiologic studies has led to drastically different conclusions regarding the general prevalence of insomnia. Moreover, such varied definitions also may have marked effects on findings regarding the risk factors, morbidity, and costs of insomnia to society at large.10
Disclosure Statement This was not an industry supported study. Dr. Edinger has received research support from Respironics Corporation, and has received honoraria from Fission Communications, Sepracor, and Axis Healthcare. Drs. Lineberger, Carney, and Means have indicated no conflicts of interest. Submitted for publication August 2005 Accepted for publication November 2005 Address correspondence to: Jack D. Edinger, PhD, VA Medical Center, 508 Fulton Street (116B) Durham, NC 27705; Tel: (919) 286-0411 ext 7054; Fax: (919) 416-5832; E-mail:
[email protected] SLEEP, Vol. 29, No. 4, 2006 Downloaded from https://academic.oup.com/sleep/article-abstract/29/4/479/2281372 by guest on 01 June 2018
479
Quantitative Insomnia Criteria—Lineberger et al
which combination of frequency and severity criteria provides the optimal cutoff for the diagnosis of primary insomnia. Consistent with Lichstein et al,11 we used values of SOL and WASO to define insomnia severity and chose to test the same quantitative cutoffs (i.e., 30 minutes, 31 minutes, 40 minutes, or 60 minutes) tested by these investigators for defining insomnia. This decision allowed us the opportunity to cross-validate the Lichstein et al results and accordingly provide some consistency in the literature. However, in our experience we have noted that pharmacologic treatment studies often use a 20-minute cutpoint for identifying sleep-onset or sleep-maintenance difficulties. As a result, we thought it important to test some quantitative severity criteria in the 20-minute range as well. Thus, in addition to testing those cutoffs tested by Lichstein et al,11 we chose to test severity cutoffs at and just above the 20-minute cutoff often employed in trials of the pharmacologic treatment of insomnia. Given these various considerations, we designed this study to address the following specific research questions: (1) which of several commonly employed severity criteria (e.g., subjective onset latency or middle-of-the-night wake time ≥ 20 minutes, 21 minutes, 30 minutes, 31 minutes, 40 minutes, or 60 minutes) best discriminates those with a complaint of primary insomnia from normal sleepers; (2) what frequency criterion (e.g., insomnia nights per week) best discriminates between groups; (3) what are the effects of including versus excluding terminal wake time (i.e., the time awake in bed between the final morning awaking and the actual rising time) in our definition of an insomnia night; and (4) are severity cutoffs based on mean values of sleep onset and WASO as effective as combined frequency and severity criteria for discriminating primary insomnia sufferers from normal sleepers?
psychiatric (Structured Clinical Interview for Psychiatric Disorders)17,18 and sleep interviews.19, 20 Results of these interviews were used in combination to determine the presence or absence of primary insomnia and to rule out obvious or occult comorbid psychiatric and/or sleep disorders in the samples of patients with primary insomnia and normal sleepers. Candidates who met interview-based selection criteria then met with a study physician (M.D.), who conducted a medical history, brief medical exam, and ordered a thyroid screening (thyroid-stimulating hormone level) so as to rule out any additional sleep-disruptive medical conditions. Candidates who passed these first 2 stages of screening then underwent 1 to 2 nights of qualifying PSG conducted either in the sleep lab or in the candidate’s home to rule out occult primary sleep disorders (e.g., obstructive sleep apnea, periodic limb movements). All PSGs were conducted with Oxford Medilog ® 9000/9200 series ambulatory cassette recorders (Oxford Instruments, Oxford, England) using a recording montage that included 2 electroencephalogram channels (C3-A2, Oz-Cz), bilateral electrooculogram, submental electromyogram, 2 channels of anterior tibialis electromyogram (right and left leg), and a nasaloral respiration thermistor. All PSGs were scored using standard scoring criteria for assignment of sleep stages,23 identification of respiratory events (e.g., apneas, hypopneas),24 and identification of periodic limb movements and periodic limb movement-related arousals.25,26 Although PSG typically includes additional respiratory measures (respiratory effort, oximetry) to detect breathing abnormalities, it was felt that monitoring of nasal/oral airflow along with our thorough interview screening for apnea would be sufficient to identify most cases with an apnea-hypopnea index above the study’s exclusionary cutoff. The insomnia sufferers recruited were adults between the ages of 20 and 79 who reported sleep complaints consistent with criteria21,22 for primary insomnia. Such prospective participants were considered for inclusion if they: (1) reported chronic (i.e., of ≥ 6 months duration) difficulty initiating or maintaining sleep or noted chronic poor sleep quality (i.e., nonrestorative sleep) occurring at least 3 times per week on average and (2) reported associated daytime deficits related to their nocturnal sleep difficulties. The normal sleepers enrolled were an age-matched sample of adults who (1) reported no sleep complaints and (2) evidenced no major medical or psychiatric conditions that might have contributed to an unreported, occult sleep disorder. Excluded from the final sample were study participants who (1) had a terminal illness; (2) had a medical condition (e.g., rheumatoid arthritis, thyroid disease) that compromises sleep; (3) had abnormal thyroid-stimulating hormone levels on a screening thyroid panel; (4) had a history of psychiatric illness; (5) met criteria21,22 for a current major psychiatric (Axis I) condition on the basis of a Structured Clinical Interview for Psychiatric Disorders17,18; (6) were substance abusers; (7) showed sedative or hypnotic dependence and were unwilling or unable to abstain from these medications while in the study; (8) were taking anxiolytics, antidepressants, or any other psychotropic medication; or (9) had an apnea-hypopnea index ≥ 15 or a periodic limb movement-related arousal index ≥ 15 during screening PSG. We also excluded insomnia sufferers who met structured sleep-interview criteria19,20 for a comorbid sleep disorder in addition to primary insomnia, as well as normal sleepers who met structured sleep-interview criteria for any sleep disorder.
METHODS Design This study used a between-groups cross-sectional research design. The study sample was comprised of independent groups of age- and sex-matched primary insomnia sufferers and noncomplaining normal sleepers. The participants for the current study were drawn from a series of studies14-16 conducted to compare the home and laboratory sleep patterns of young, middle-aged, and older adult insomnia sufferers and normal sleepers. All study procedures were reviewed and approved by the Institutional Review Boards of the VA Medical Center and Duke University Medical Center in Durham, NC. Each participant was required to provide written informed consent prior to enrolling in the research and undergoing study-related procedures. Participant Recruitment and Screening All participants were recruited via posted study announcements at the Durham (NC) VA Medical Center, letters mailed to persons in the Duke University Center for the Study of Aging and Human Development Subject Pool, flyers posted in the community, and face-to-face solicitations of clinic patients presenting to the Duke University Sleep Disorders Center. Prior to their acceptance into the research program, participants underwent a thorough screening process to ensure that they met study-selection criteria. A PhD-level clinical psychologist met individually with each study candidate for the purpose of conducting both structured SLEEP, Vol. 29, No. 4, 2006 Downloaded from https://academic.oup.com/sleep/article-abstract/29/4/479/2281372 by guest on 01 June 2018
480
Quantitative Insomnia Criteria—Lineberger et al
Sample Characteristics
Table 1—Sleep-Log Parameters for Insomnia Sufferers vs Normal Sleepers
Through use of the above selection criteria, a sample of 208 study participants was enrolled. The majority of these participants were recruited from posted announcements or solicitation letters, so only a very small proportion (< 5%) of the enrollees were actual clinical patients. Of the 208 who entered the study, 48 were excluded from this investigation because they failed to complete and return sleep-log data as instructed. The final study sample, thus, consisted of 160 adults. Of these, 72 (41 women) met selection criteria for primary insomnia, whereas the remaining 88 (44 women) were noncomplaining normal sleepers. The insomnia sample included 13 persons with solely sleep-onset complaints, 16 individuals with sleep-maintenance complaints, 38 with a mixture of sleep-onset and sleep-maintenance difficulties, and 5 persons with other sleep-wake concerns (e.g., nonrestorative sleep). The mean age of the insomnia sample was 45.70 years (SD = 16.95), whereas the mean age of the normal group was 45.39 years (SD = 16.59). The insomnia sample was composed of 52 European Americans, 13 African Americans, 4 Asian Americans, 2 Hispanic Americans, and 1 Native American; the normal sleepers included 71 European Americans, 12 African Americans, 4 Asian Americans and 1 Native American.
Parameter
Insomnia Mean SD TIB 479.53 54.93 TST 366.77 71.60 SOL 41.72 38.83 MWASO 40.12 31.98 TWSO 30.91 23.53 SE% 76.37 11.58
df 1 (158) 1 (158) 1 (158) 1 (158) 1 (158) 1 (158)
F 2.68 35.21 50.61 59.89 12.10 108.18
p .10 < .0001 < .0001 < .0001 < .001 < .0001
Data are presented as minutes unless otherwise noted. TIB refers to time in bed; TST, total sleep time; SOL, sleep-onset latency; MWASO, middle of the night wake after sleep onset (total time awake between initial sleep onset and the final morning awakening; TWSO, terminal wake time (total time awake in bed between the final morning awakening and rising [out-of-bed] time); SE%, sleep efficiency. Error term df in parentheses.
nightly values for each of the sleep-log parameters were extracted and placed in an electronic database for subsequent analyses. Study analyses were conducted using PC versions of SAS (SAS Institute, Cary, NC) and SPSS (SPSS Inc., Chicago, IL) software.
MEASURES
RESULTS
Subjective estimates of nocturnal sleep time and fragmentation were obtained from sleep logs completed by study participants each morning upon arising during a 2-week period. In all cases, these sleep logs were completed after the qualifying PSG studies were conducted, and, in most cases, participants completed these logs after they had completed all PSG (home and lab) studies conducted for the larger study from which they were drawn. The logs included a series of questions that solicited participants’ estimates of time in bed (TIB), total sleep time (TST), SOL (time between “lights out” and first onset of sleep), middle of the night WASO (MWASO: total time awake between initial sleep onset and the final morning awakening), terminal wake time (TWSO: total time awake in bed between the final morning awakening and rising [out-of-bed] time), and sleep efficiency percentage (SE%) (TST ÷ TIB x 100%) for each night’s sleep. Prior to completing the sleep log, each study participant received verbal instructions in the use of this instrument from the PhD-level psychologist who conducted the initial screening interview. This instruction was designed to encourage the participant to enter accurate and useful data for each night’s sleep and to minimize the possibility of missing nights or nonsensical entries (e.g., values of SOL or WASO that exceed the designated TIB). However, we assumed that some participants might have occasional sleepless nights. To accommodate this possibility, we decided that SOL would be set equal to TIB for such occurrences, whereas the values of the remaining sleep parameters would be assigned values of 0 for such occasions.
Preliminary Analyses Before conducting any planned tests of study hypotheses, we examined the sleep-log data recorded by participants to ascertain whether either missing nights or sleepless nights were common in the data set. Findings showed that all 160 participants provided 14 nights worth of data as instructed. Furthermore, of all of the data collected, only 1 sleepless night was noted. Hence, the instructions provided to participants seemed effective in obviating missing entries and encouraging careful consideration of sleep and wakefulness across each night. In addition to this preliminary data review, we performed separate 1-way analyses of variance (ANOVA) comparing insomnia sufferers and normal sleepers using sleep-log parameters as dependent variables. Descriptive and inferential statistics for these analyses are summarized in Table 1. As expected, the insomnia group had significantly lower mean values of TST and sleep efficiency and higher mean values of SOL, MWASO, and TWSO than did the normal sleepers. Mean TIB did not differ between groups. Sensitivity-Specificity and Receiver-Operator Characteristic Analyses One way to derive a cutoff is to graphically depict the relation between the sensitivity and specificity of the test over all possible values on a receiver-operator characteristic (ROC) curve. The sensitivity represents the probability of detecting insomnia when it is present, and specificity represents the probability of not detecting insomnia when it is indeed not present. The ROC curve is plotted for all values, and the further the ROC curve lies above the diagonal reference line, the more accurate the test.27 The area under the curve (AUC) is another index of accuracy. The AUC is the probability that a test result for a randomly chosen posi-
PROCEDURE Consenting participants who met inclusion criteria used the sleep log to monitor their home sleep patterns for 14 consecutive nights while enrolled in the larger study from which they were obtained. In all cases, these logs were completed on nights the participants were not undergoing PSG monitoring. Participants’ SLEEP, Vol. 29, No. 4, 2006 Downloaded from https://academic.oup.com/sleep/article-abstract/29/4/479/2281372 by guest on 01 June 2018
Normal Mean SD 465.68 51.85 423.42 48.70 11.66 7.37 10.77 14.15 19.83 16.70 91.14 5.97
481
Quantitative Insomnia Criteria—Lineberger et al
sensitivity/specificity value
1 0.8 0.6 0.4 0.2
Sensitivity
Specificity
0 1
2
3
4
5
6
7
8
9
10
11
12
13
14
# of nights Figure 1—Sensitivity and specificity of 21-minute severity criterion by nights of occurrence.
tive case will exceed the result for a negative case. Swets28 has suggested that test accuracy be defined as “low” for AUC values under 0.7, “moderate” for AUC values between 0.7 and 0.9, and “high” for values greater than 0.9. We used ROC analyses to determine clinical cutoffs for 6 quantitative insomnia-severity criteria (wake time ≥ 20 minutes, ≥ 21 minutes, ≥ 30 minutes, ≥ 31 minutes, ≥ 40 minutes, and ≥ 60 minutes) and 14 insomnia-frequency criteria, representing the number of nights out of 2 weeks on which severity cutoffs were met. For each of the 6 severity criteria, we plotted the sensitivity and specificity of either SOL or MWASO exceeding the severity cutoff on 1 night, 2 nights, 3 nights, etc, (up to 14 nights) using separate ROC curves. Using the coordinates of the ROC curves, we calculated sensitivity and specificity indexes of each value, and selected the cutoff score that appeared to maximize both of these indexes. We also calculated the AUCs and tested these for significance. Likewise, to examine the effects of including TWSO in our definition of WASO, we plotted separate ROC curves for the number of nights across a 14-day period with either SOL, MWASO, or TWSO exceeding each of the 6 severity cutoffs and calculated sensitivity and specificity based on the coordinates of the curves. The AUCs for these ROC analyses were also calculated and tested for statistical significance. We felt both sets of analyses were needed because many previous insomnia studies have used only MWASO to assess sleep-maintenance difficulties, whereas other studies have combined MWASO and TWSO into an overall measure of WASO. Thus, our 2 sets of analyses allowed us to test the utility of each of these approaches. To illustrate our analytic process, Figure 1 shows a plot of sensitivity (i.e., percentage of insomnia cases included) and specificity (i.e., percentage of normal sleepers not included) when SOL or MWASO values of ≥ 21 minutes are used to discriminate the insomnia and normal groups. This figure shows that sensitivity is relatively high, whereas specificity is fairly low when sleep-log data meet this severity criterion on 1 of 14 nights. As the number of nights required to meet this severity cutoff increases, sensitivity and specificity converge until they intersect at about 7 nights (between 3 and 4 nights per week). Assuming it is desirable to obtain a balance between correct identification of true insomnia cases and correct exclusion of false positives (i.e., normal sleepers classified as insomnia sufferers), we conclude that the occurrence of this particular severity cutoff on 7 or more nights would prove optimal for insomnia case identification. Figure 2 shows the ROC curve plotted for this particular combined frequency and severity cutoff. This figure supports the utility of this combined criterion SLEEP, Vol. 29, No. 4, 2006 Downloaded from https://academic.oup.com/sleep/article-abstract/29/4/479/2281372 by guest on 01 June 2018
Figure 2—Receiver-operator characteristic (ROC) Curve for sleeponset latency or middle of the night wake after sleep onset values ≥ 21 minutes occurring on 7 Nights. Table 2—Receiver-Operator Characteristic Curve Analyses for Quantitative Insomnia Criteria (SOL or MWASO) Severity Frequency Sensitivity Specificity AUC Cutoff, Cutoffa min ≥ 20 8 88.9% 78.4% .91 ≥ 21 7 83.3% 83.0% .91 ≥ 30 5 87.5% 79.5% .90 ≥ 31 4 80.6% 85.2% .90 ≥ 40 3 81.9% 80.7% .91 ≥ 60 2 79.2% 81.8% .89
SE p value .023 .023 .025 .024 .023 .027
< .001 < .001 < .001 < .001 < .001 < .001
a Number of nights at or above the specified severity cutoff. SOL refers to sleep-onset latency; MWASO, middle of the night wake after sleep onset (total time awake between initial sleep onset and the final morning awakening); AUC area under the curve; SE, standard error.
in that the curve falls well above the diagonal reference line and the AUC is relatively large. Results for all of our sensitivity-specificity and ROC analyses excluding TWSO are summarized in Table 2, whereas those including TWSO are presented in Table 3. Of the insomnia-severity criteria described above, those excluding TWSO produced ROC curves with AUCs in the moderate to high range (.890 to .914, all p values < .001), whereas those including TWSO produced ROC curves with AUCs in the moderate range (.852 to .875, all p values < .001). No 1 clinical cutoff was found to maximize sensitivity and specificity. Rather, the optimal frequency cutoff varied as a function of the insomnia-severity criterion tested. For instance, a severity cutoff of ≥ 21 minutes SOL or MWASO was associated with a frequency cutoff of 7 (i.e., 3 to 4 nights per week), whereas a frequency cutoff of 2 (i.e., 1 night per week) maximized sensi482
Quantitative Insomnia Criteria—Lineberger et al
Table 3—Receiver-Operator Characteristic Curve Analyses for Quantitative Insomnia Criteria (SOL, MWASO, or TWSO)
Table 4—Sensitivity-Specificity Analyses for 2-Week Mean Sleep Log Data (SOL or MWASO only)
Severity Frequency Sensitivity Specificity AUC Cutoff, Cutoff, min nightsa ≥ 20 12 77.8% 79.5% .87 ≥ 21 11 79.2% 83.0% .87 ≥ 30 9 83.3% 75.0% .85 ≥ 31 6 81.9% 77.3% .87 ≥ 40 6 80.6% 80.7% .88 ≥ 60 4 75.0% 83.0% .87
Severity Cutoff, min ≥ 20a ≥ 21 ≥ 30 ≥ 31 ≥ 40 ≥ 60
SE p value .028 .028 .031 .029 .028 .029
< .001 < .001 < .001 < .001 < .001 < .001
Specificity
χ2
p value
94.4% 91.7% 76.4% 72.2% 61.1% 33.3%
79.6% 80.7% 89.8% 90.9% 94.3% 98.9%
87.21 83.03 72.23 67.34 57.27 31.14
< .0001 < .0001 < .0001 < .0001 < .0001 < .0001
To meet insomnia criteria, mean sleep-onset latency (SOL) or middle of the night wake time after sleep onset (MWASO: total time awake between initial sleep onset and the final morning awakening) per night over the 2-week monitoring period must exceed severity cutoff. a The cutoff maximizes the sensitivity and specificity balance such that the largest proportion of the sample is correctly classified.
Number of nights at or above the specified severity cutoff. SOL refers to sleep-onset latency; MWASO, middle of the night wake after sleep onset (total time awake between initial sleep onset and the final morning awakening; TWSO, terminal wake time (total time awake in bed between the final morning awakening and rising [out-of-bed] time); AUC area under the curve; SE, standard error. a
tivity and specificity when a severity cutoff of ≥ 60 minutes SOL or MWASO was used to discriminate between insomnia sufferers and normal sleepers.
Table 5—Sensitivity-Specificity Analyses for 2-Week Mean SleepLog Data (SOL, MWASO, or TWSO) Severity Cutoff, min ≥ 20 ≥ 21 ≥ 30 ≥ 31a ≥ 40 ≥ 60
Sensitivity-Specificity Analyses of Mean Sleep Log Data In addition to the ROC analyses of insomnia severity and frequency, we also conducted sensitivity and specificity tests using 2-week mean sleep log data. We compared sensitivity and specificity for 6 severity criteria excluding TWSO (mean SOL or MWASO over 14 nights of sleep log monitoring ≥ 20, 21, 30, 31, 40, or 60 minutes) and 6 criteria including TWSO (mean SOL, MWASO, or TWSO ≥ 20, 21, 30, 31, 40, or 60 minutes). These data are presented in Tables 4 and 5. When TWSO was excluded, a mean SOL or MWASO cutoff of ≥ 20 minutes per night over 2 weeks of sleep-log monitoring appeared to best balance sensitivity (94.4%) and specificity (79.6%). Use of this cutoff resulted in the correct classification of 86.6% of the entire study sample. When TWSO was included, the optimal cutoff increased to ≥ 31 minutes per night, with a sensitivity of 80.6% and specificity of 81.8%. Use of this cutoff resulted in correct classification of 81.3% of the sample.
Sensitivity
Specificity
χ2
p value
95.8% 94.4% 81.9% 80.6% 62.5% 37.5%
54.6% 56.8% 79.6% 81.8% 87.5% 95.5%
46.28 46.54 59.98 61.97 43.52 27.53
< .0001 < .0001 < .0001 < .0001 < .0001 < .0001
To meet insomnia criteria, mean sleep-onset latency (SOL), middle of the night wake after sleep onset (MWASO: total time awake between initial sleep onset and the final morning awakening), and terminal wake time (TWSO: total time awake in bed between the final morning awakening and rising [out-of-bed] time). a The cutoff maximizes the sensitivity and specificity balance such that the largest proportion of the sample is correctly classified.
showed that only 40% (n = 64) of our sample were 50 years of age or older. Given this finding, we divided our sample into 2 broad age cohorts (those < 50 years old versus those 50 years of age or older) and conducted sensitivity and specificity analyses for our 2 optimal mean cutpoints within each age cohort. When TWSO was excluded, a mean cutoff of ≥ 20 minutes resulted in a sensitivity of 95.1% and specificity of 90.9% in the younger group and a sensitivity of 93.6% and specificity of 60.6% in the older group. When TWSO was included, the mean cutoff of ≥ 31 minutes resulted in a sensitivity of 75.6% and specificity of 89.1% in the younger cohort and a sensitivity of 87.1% and specificity of 69.7% within the older subgroup. Thus, the 20-minute mean cutoff that includes SOL and MWASO appeared to function better with the younger group, whereas the 31-minute mean that considered SOL, MWASO, and TWSO functioned better in the older group.
Subgroup Comparisons In view of the favorable findings noted for the cutoffs based on mean sleep-log data, we conducted some additional tabulations to determine how well these cutpoints functioned for the different insomnia subtypes defined by their presenting sleep complaints (i.e., onset difficulty versus maintenance difficulty versus mixed onset or maintenance problems). When TWSO was excluded, a mean SOL or MWASO cutoff of ≥ 20 minutes correctly classified all (100%) of the 13 with sleep-onset complaints, 14 (87.5%) of the 16 with sleep-maintenance insomnia, and 36 ( 94.7%) of the 38 with mixed onset or maintenance complaints. When TWSO was included, the optimal cutoff of ≥ 31 minutes correctly classified 9 (69.2%) of the 13 sleep-onset sufferers, 14 (87.5%) of the 16 with sleep-maintenance insomnia, and 32 (84.21%) of the 38 with mixed onset or maintenance complaints. Given the well-recognized changes in sleep with aging, we thought it important to test our mean cutpoints in different age cohorts within our sample. A preliminary frequency tabulation SLEEP, Vol. 29, No. 4, 2006 Downloaded from https://academic.oup.com/sleep/article-abstract/29/4/479/2281372 by guest on 01 June 2018
Sensitivity
DISCUSSION The present study sought to determine empirically which combination of frequency and severity criteria derived from sleep-log data provides the optimal discrimination of insomnia sufferers from normal sleepers. Results of our ROC analyses indicated that no single set of combined frequency and severity criteria best dis483
Quantitative Insomnia Criteria—Lineberger et al
from the community and assigned presumptive diagnoses of psychophysiologic insomnia or normal sleeper based solely on their responses to several screening questionnaires. In contrast, our study participants were self-selected research volunteers who underwent extensive interview, medical, and PSG screening to rule out unwanted comorbidities and establish their diagnostic status as a primary insomnia sufferer or normal sleeper. Whereas the procedures of Lichstein et al11 allowed for greater external validity than do ours, the internal validity of their study is more questionable. Indeed, the methods whereby insomnia and normalsleeper diagnoses were established arguably allow for a limited diagnostic certainty and the probability of unwanted comorbidities (e.g., depression, occult sleep disorders) within the insomnia sample in particular. The presence of such comorbidities makes the presence of more-severe and persistent insomnia likely in the Lichstein et al11 sample, thus explaining the higher frequency and severity cutoffs found for this insomnia group. Nonetheless, future research in this area would benefit by implementing the strengths of both studies to provide the most valid and representative cutoffs for the particular insomnia subtype of subtypes being studied. Of course, any studies of this nature must rely on some “goldstandard” against which various sleep-log indexes can be tested. In the current study, we chose to use the Diagnostic and Statistical Manual of Mental Disorders criteria set for defining primary insomnia. This choice seemed reasonable because the Diagnostic and Statistical Manual of Mental Disorders sleep nosology is widely used in clinical practice, and we conducted a thorough assessment to establish the presence or absence of a primary insomnia diagnosis in each of our study participants. However, the structured sleep interview19,20 used in our assessment requires that respondents complain of insomnia at least 3 times per week to qualify for primary insomnia, and this stipulation may have biased our sample somewhat. Moreover, the Diagnostic and Statistical Manual of Mental Disorders primary insomnia criteria set has its limits, and there are currently no foolproof biomarkers available to confirm the presence of primary insomnia. As such, our results should be viewed with these considerations in mind. In reviewing the findings reported herein, it is important to consider several additional methodologic limitations of this investigation. As noted previously, the study sample was comprised of research volunteers between 20 and 79 years of age. As such, caution should be exercised when generalizing our findings to clinical patients, the broader community of primary insomnia sufferers, and individuals in younger age groups (i.e., children or adolescents). It is also noteworthy that our subgroup analyses suggested that quantitative criteria for insomnia might vary somewhat across age groups. Unfortunately, our moderate sample size and limited numbers of individuals over age 50 prevented us from providing separate and, perhaps, more-accurate cutpoints for young, middle-aged, and older adults. Whereas results provided some support for use of quantitative criteria for differentiating primary insomnia sufferers from normal sleepers, it remains questionable whether such criteria will prove useful for discriminating distinctive insomnia diagnoses. Indeed, it is likely that other diagnostic criteria sets8,12,21,22 will be needed for such purposes. In addition, it should be noted that our PSG screening montage excluded several respiratory measures commonly regarded as essential for detecting sleep-disordered breathing. As a result, it is possible that our sample included some individu-
criminated between groups; rather, there was an inverse relationship between insomnia severity and insomnia frequency, such that the optimal frequency cutoff (e.g., that which maximized sensitivity and specificity) decreased as we increased the severity criterion. AUC values were uniformly high for severity criteria ranging from 20 to 60 minutes, suggesting that good test accuracy is possible at a variety of insomnia-severity levels, depending upon the number of insomnia nights observed. However, AUC values were lower, and frequency cutoffs higher, when the definition of an insomnia night was expanded to include estimated terminal wake time. This finding is consistent with the notion that both insomnia sufferers and normal sleepers tend to linger in bed after their final awakening; hence, for research purposes, better test accuracy might be achieved if TWSO is excluded from the definition of an insomnia night. These results have both research and clinical implications, indicating that different insomnia-frequency criteria may be necessary, depending upon the severity of the patient’s presenting complaint. Our results suggest that a relatively mild complaint of greater than 20 minutes sleep latency or middle wake time occurring 4 or more nights per week may be as significant as more severely prolonged wake time (i.e., greater than 60 minutes) occurring just 1 night a week. Thus, if sleep-log data are to be used in clinical and research venues for insomnia ascertainment, some insomnia sufferers may fail to meet sleep-log–based criteria for insomnia, particularly if the insomnia-frequency criteria (i.e., number of insomnia nights per week) are too rigid and demanding. However, it should be noted that the use of mean SOL or MWASO according to our sensitivity and specificity analysis is a valid and straightforward alternative to using separate insomnia frequency and severity criteria. For our sample as a whole, mean SOL or MWASO of ≥ 20 minutes provided the best balance of sensitivity and specificity in distinguishing between insomnia sufferers and normal sleepers; when TWSO was included, this cutoff increased to ≥ 31 minutes. Our subgroup analyses showed that the former of these indexes was the better for correctly identifying insomnia sufferers in general (regardless of their presenting complaint) and for discriminating insomnia cases from normal sleepers in the younger age cohort. In contrast, the latter of these severity indexes resulted in better sensitivity and specificity in the older age group. Lichstein et al11 have expressed reservations about insomnia cutoffs based on averaged sleep-log data, arguing that clinical and research practice favor the regular occurrence of insomnia nights. Hence, these investigators advocate the use of combined frequency and severity indicators rather than relying on averaged values of SOL or MWASO that might be artificially inflated due to episodic unusually poor nights. Nonetheless, in our sample, carefully diagnosed via structured interviews, medical evaluation, and PSG findings, the infrequent presence of severe insomnia symptoms provided as accurate a cutoff for an insomnia diagnosis as did the frequent presence of milder insomnia symptoms Since the results of Lichstein et al11 and the current study lead to different impressions in regard to optimal sleep-log–based insomnia cutoffs, factors contributing to such differences should be considered. A review of the methodologies of these 2 studies suggests that their differing sample selection and definition procedures likely accounted for their different results. In the Lichstein et al11 investigation, study participants were selected randomly SLEEP, Vol. 29, No. 4, 2006 Downloaded from https://academic.oup.com/sleep/article-abstract/29/4/479/2281372 by guest on 01 June 2018
484
Quantitative Insomnia Criteria—Lineberger et al
als with sleep-related breathing disorders. Finally, since our sleep logs included only measures of nighttime sleep continuity, it is possible that combining these measures with additional measures of daytime functioning may enhance classification results. Given these limitations, future studies of this nature would benefit by (1) use of a sleep log that provides more comprehensive sleep-wake assessment, (2) enrollment of larger community-based samples that allow for identifying optimal cutpoints within separate age cohorts, and (3) the use of thorough face-to-face diagnostic interviews as well as PSG that includes the currently recommended recording montage29 for identifying sleep-disordered breathing. However, in spite of this study’s limitations, results warrant consideration since they suggest (1) a balance between frequency and severity criteria is required when deriving sleep log–based insomnia cutoffs and (2) mean sleep log values of SOL and WASO may be at least as useful, if not more so, than combined frequency and severity criteria sets for establishing quantitative insomnia cutoffs.
14. 15. 16. 17. 18.
19. 20.
ACKNOWLEDGEMENTS This research was supported by the Department of Veteran’s Affairs Merit Review Grant #VA0009 and Health Services Research and Development Grant # IIR 00-091 awarded to Jack D. Edinger, Ph.D.
21. 22. 23.
REFERENCES 1. 2.
3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.
Quera-Salva M, Orluc A, Goldenberg F, Guilleminault C. Insomnia and use of hypnotics: study of a French population. Sleep 1991;14:386-91. Klink M, Quan S, Kaltenborn W, Lebowitz M. Risk factors associated with complaints of insomnia in a general adult population: Influence of previous complaints of insomnia. Arch Intern Med 1992;152:1634-7. Leger D, Guilleminault C, Dreyfus C, Paillard M. Prevalence of insomnia in a survey of 12,778 adults in France. J Sleep Res 2000;9:35-42. Breslau N, Roth T, Rosenthal L, Andreski P. Sleep disturbance and psychiatric disorders: a longitudinal epidemiological study of young adults. Biol Psychiatry 1996;39:411-8. Ohayon MM, Caulet M, Guilleminault C. How a general population perceives its sleep and how this relates to the complaint of insomnia. Sleep 1997;20:715-23. World Health Organization. ICD-9-CM: International classification of diseases, clinical modification. 4th ed. Salt Lake City: Medicode; 1994. The ICD-10 Classification of Mental and Behavioral Disorders. Geneva: World Health Organization; 1993. International Classification of Sleep Disorders: Diagnostic and Coding Manual. 2nd ed. Westchester, IL: American Academy of Sleep Medicine; 2005. Diagnostic and Statistical Manual of Mental Disorders, 4th ed text revision. Washington: American Psychiatric Association; 2000. Ohayon MM. Epidemiology of insomnia: what we know and what we still need to learn. Sleep Med Rev 2002;6:97-111. Lichstein KL, Durrence HH, Taylor DJ, Bush AJ, Riedel BW. Quantitative criteria for insomnia. Behav Res Ther 2003;41:427-45. Edinger JD, Bonnet MH, Bootzin RR, et al. Derivation of research diagnostic criteria for insomnia: report of an American Academy of Sleep Medicine work group. Sleep 2004;27:1567-96. Allen RP, Picchietti D, Hening W, et al. Restless legs syndrome: diagnostic criteria, special considerations, and epidemiology. A report from the Restless Legs Syndrome Diagnosis and Epidemi-
SLEEP, Vol. 29, No. 4, 2006 Downloaded from https://academic.oup.com/sleep/article-abstract/29/4/479/2281372 by guest on 01 June 2018
24. 25.
26. 27. 28. 29.
485
ology Workshop at the National Institutes of Health. Sleep Med 2003;4:101-19. Edinger JD, Fins AI, Sullivan RJ et al. Sleep in the laboratory and sleep at home: Comparisons of older insomniacs and normal sleepers. Sleep 1997;20:1119-26. Edinger JD, Glenn DM, Bastian LA et al. Sleep in the laboratory and sleep at home II: Comparisons of middle-aged insomnia sufferers and normal sleepers. Sleep 2001;24:761-70. Gehrman P, Edinger JD, Means MK, Husain AM. Measurement of sleep in young insomniacs: a multi-trait, multi-method approach. Sleep 2003;26:A310-1. Structured Clinical Interview for DSM-III-R (SCID). Washington: American Psychiatric Press; 1990. Spitzer RL, Williams JBW, Gibbons M, First MB. Instruction Manual for the Structured Clinical Interview for DSM-IV (SCID-IV). New York: Biometrics Research Department, New York State Psychiatric Institute; 1996. Schramm E, Hohagen F, Grasshoff M, et al. Test-retest reliability and validity of the structured interview for sleep disorders according to DSM-III-R. Am J Psychiatry 1993;867-72. Edinger JD, Wohlgemuth WK, Means MK, et al. Duke Structured Interview for Sleep Disorders. Durham, NC: Unpublished interview schedule; 1997. Diagnostic and Statistical Manual of Mental Disorders, 3rd ed. Washington: American Psychiatric Association; 1987. Diagnostic and Statistical Manual of Mental Disorders. 4th ed. Washington: American Psychiatric Association; 1994. Rechtshaffen A, Kales A. A Manual of Standardized Terminology, Techniques, and Scoring Systems of Sleep Stages of Human Subjects. Los Angeles: UCLA Brain Information Service/Brain Research Institute; 1968. Phillipson EA, Remmers JE. American Thoracic Society consensus conference on indications and standards for cardiopulmonary sleep studies. Am Rev Respir Dis 1989;139:559-68. Coleman R. Periodic movements in sleep (noctural myoclonus) and restless legs syndrome. In: Guilleminault C, ed. Sleeping and Waking Disorders: Indications and Techniques. Menlo Park, CA: Addison-Wesley; 1982:265-95. EEG arousals: scoring rules and examples. ASDA Task Force. Sleep 1992;15:173-84. Mossman D, Somoza E. ROC curves, test accuracy and the description of diagnostic tests. J Neuropsychiatry Clin Neurosci 1991;3:330-3. Swets J. Measuring the accuracy of diagnostic systems. Science 1988;240:1285-93. Practice parameters for the indications for polysomnography and related procedures. Indications for Polysomnography Task Force, American Sleep Disorders Association Standards of Practice Committee. An American Sleep Disorders Association report: Sleep 1997;20:406-22.
Quantitative Insomnia Criteria—Lineberger et al
