Validation of the web-based LUMINA questionnaire for recruiting large cohorts of migraineurs

Abstract

Objective: To assess validity of a self-administered web-based migraine-questionnaire in diagnosing migraine aura for the use of epidemiological and genetic studies.

Methods: Self-reported migraineurs enrolled via the LUMINA website and completed a web-based questionnaire on headache and aura symptoms, after fulfilling screening criteria. Diagnoses were calculated using an algorithm based on the International Classification of Headache Disorders (ICHD-2), and semi-structured telephone-interviews were performed for final diagnoses. Logistic regression generated a prediction rule for aura. Algorithm-based diagnoses and predicted diagnoses were subsequently compared to the interview-derived diagnoses.

Results: In 1 year, we recruited 2397 migraineurs, of which 1067 were included in the validation. A seven-question subset provided higher sensitivity (86% vs. 45%), slightly lower specificity (75% vs. 95%), and similar positive predictive value (86% vs. 88%) in assessing aura when comparing with the ICHD-2-based algorithm.

Conclusions: This questionnaire is accurate and reliable in diagnosing migraine aura among self-reported migraineurs and enables detection of more aura cases with low false-positive rate.

Keywords

Cohort study headache migraine questionnaire validation screening in epidemiology

Introduction

Migraine is a common brain disorder characterized by recurrent, disabling attacks of headache, autonomic features (migraine without aura; MO), and, in one-third of patients, transient neurological aura symptoms (migraine with aura; MA). In Western countries, the overall migraine prevalence in the general population is at least 12%, two-thirds of which concerns females (1–4). Since no biomarker for migraine exists, diagnosis according to the headache classification of the International Headache Society (IHS) (5) relies exclusively on the headache history. A careful history taken by a headache specialist is the gold standard for making a valid migraine and aura diagnosis.

Large scale studies with several thousands of participants are important to obtain information for epidemiological and genetic migraine research and may yield important insights in migraine pathophysiology. Migraine is a complex genetic disorders, i.e. multiple genetic and environmental factors contribute to migraine susceptibility.

Twin- and population-based family studies showed that genetic factors play an important role in migraine susceptibility, especially in the MA subtype (6–12). However, genetic linkage studies using migraine subtypes as an end diagnosis did not yield gene variants thus far. Clinical heterogeneity in migraine and aura diagnosis may have hampered the identification of such variants. Recently, in a large genome-wide association analysis with a large set of clinic-based migraineurs, a first-ever genetic risk factor was identified associated with common types of migraine, in patients that were largely recruited from specialist headache clinics with a clinic-based migraine diagnosis (13). However, population-based large-scale studies exclude the possibility of a face-to-face examination, and, therefore, a less time-consuming and less costly diagnostic strategy has to be chosen. A web-based questionnaire represents an attractive and inexpensive alternative for a clinic interview. Several groups have reported on the use of internet to recruit headache and other patients for clinical research (14–18). However, reliably diagnosing aura remains an issue.

The availability of a validated, aura-specific questionnaire is important when large numbers of cases are needed, especially in studies with self-reported migraineurs from the general population (19,20). We developed the LUMINA (Leiden University MIgraine Neuro-Analysis) website and designed and validated a self-reporting, web-based questionnaire to reliably diagnose migraine headache and aura symptoms, using only a limited number of questions. In this paper, we will present the validation of this web-based migraine and aura questionnaire.

Methods

Subjects

Participants were Dutch adults aged 18−74 years with migraine (MA and MO), who were invited via the lay press nationwide to enrol via the especially designed LUMINA website. Additionally, patients from our outpatient headache clinic were invited by a letter. In this clinic-based study, all participants were self-reporting migraineurs, of which approximately 90% had previously been diagnosed with migraine by a physician.

Study flow

Study flow is depicted in Figure 1. Patients who visited the website were informed about the study and could enrol directly. The first step was to fulfil the screening criteria, using a simple screening questionnaire that was validated previously in the population-based GEM study (3). This screening questionnaire included five questions asking: (i) whether the patient had severe headaches in the past 12 months; (ii) what the headache severity was; (iii) had suffered from headaches which were preceded by visual disturbances; (iv) had been diagnosed with migraine by a physician; and (v) had ever used anti-migraine medication. After fulfilling these criteria, cases received a unique user-ID code via email to log on to the study website, where theycould participate in an extended, web-based questionnaire study. Having completed the extended questionnaire, a number of randomly selected participants were contacted by telephone by WPJvO, CMW, and AHS, who are experienced in diagnosing migraine. This semi-structured telephone interview detailed questions on headache and aura characteristics, including ICHD-2 migraine and aura criteria (5) with special attention for visual, sensory, motor, and speech aura symptoms, was used as the gold standard. Median interview duration was 10−15 minutes, ranging up to 30 minutes if necessary. Afterwards, a final diagnosis was made: in case of ambiguity, a headache specialist (GMT) was consulted. Patients were excluded from theanalysis if they could not be reached by telephoneafter five failed telephone contact attempts. The study was approved by the local medical ethics committee. All participants provided written informed consent.

Construction of questionnaire

The extended questionnaire (accessible via www.lumc.nl/hoofdpijn) was based on the ICHD-2 (5) and incorporated 127 items on migraine headache and aura characteristics, premonitory symptoms, trigger factors, allodynia, and medication use and was presented to participants as a digital web form. The questions were to be answered by choosing from categorical alternatives. On the web form, multicolour exemplary illustrations were shown with the most characteristic visual aura features (hemianopsia, scotoma, fortification spectra, visual blurring) and sensory aura features (anatomical distribution).

ICHD-2-based algorithm

After completion of the extended questionnaire, an algorithm based on ICHD-2 (5) migraine criteria wasrun and individual diagnosis was determined. Thealgorithm had the following possible outcomes: ‘no migraine’; ‘migraine without aura’; and ‘migraine with aura’. In the analysis, the algorithm outcomes were dichotomized into ‘aura’ and ‘no aura’ (Supplementary Figure e-1).

Statistical analysis

Descriptive statistics

Descriptive statistics were performed on demographic and clinical variables, on the algorithm-based diagnoses and on the interview-derived diagnoses. Results are reported as mean ± SD or percentage. Differences in between-groups means were analysed with independent sample t-tests and ANOVAs. Proportions were compared using Chi-squared tests. All items from the extended questionnaire thatconcerned ICHD-2 migraine criteria were evaluated separately. Likelihood ratios were calculated using standard formulas for positive likelihood ratio [LR+, sensitivity/(1−specificity)] and negative likelihood ratio [LR−, (1⊟sensitivity)/specificity].

Questionnaire validation process

The questionnaire validation process was divided into two phases and was aimed at identifying a combination of items that were better predictors for diagnosing migraine aura than the ICHD-2-based algorithm, with the interview-derived diagnosis as the gold standard. In phase I, a sample of 838 self-reported migraineurs (approximately 80% of total group) was randomly selected and used asa training sample (see Figure 1) to derive a predictivemodel. These patients fulfilled set screening criteria from the five-item LUMINA screener before they could enter the extended questionnaire. Logistic regression (see below) was used to develop the predictive model that included questionnaire items most contributing to predict subcategories ‘aura’ and‘no aura’. Subsequently, we compared both the ICHD-2-based algorithm diagnoses and the diagnoses predicted by the logistic model, to the gold standard. In phase II, we validated this derived predictive model in an independent validation sample, consisting of 200patients, approximately 20% of our sample (see Figure 1).

Phase I: development of prediction rule

In phase I, a prediction rule for the aura subcategories ‘aura’ vs. ‘no aura’ was developed using a multivariate logistic regression analysis. Relevant, individual, dichotomized items (n = 33) were selected from the extended questionnaire and were used as predictor variables for aura in the model. Selection of items was made by the authors (WPJvO; CW; GMT) and was based on clinical relevance to migraine aura and sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and likelihood ratios of individual items. Inter-item correlation was assessed for relevant items using Spearman’s rank coefficients and when items correlated with coefficients >0.9, one of these items was excluded from the analysis. A forward selection strategy using the likelihood ratio test was performed to identify items that were significant (p < 0.05) predictors for the outcome of aura. For each subject in this sample (n = 838), a prediction score was calculated using these items. Subsequently, a receiver operating characteristic (ROC) curve was generated to assess the optimum cut-off point for this prediction score. Using the method proposed by Halpern et al. (21), an optimum cut-off (highest sensitivity and specificity) was determined from the ROC curve. Therefore, the logistic model resulted in a selection of the 33 items with significant (p < 0.05) contribution in the aura prediction.

Phase II: validation of prediction rule

The derived predictive rule was was subsequently validated in the second sample (validation sample; n = 200; see Figure 1). Validity of this predictive model was assessed by checking whether the selected items contributed significantly (p < 0.05) for the prediction in the second sample too. Subsequently, the sensitivity and specificity from the ROC optimum in the training sample were compared with these parameters in the validation sample, using the same cut-off value.

Overall outcome measures

Sensitivity, specificity, PPV and NPV were calculated to compare the fit of the three different models with the interview-derived aura diagnosis as the gold standard. These models were: (1) ICHD-2-based algorithm; (2) predictive model from phase I; and (3) validation of predictive rule in phase II.

All data analyses were performed using SPSS 16.0.2 (SPSS, IBM, USA). p-values less than 0.05 were considered significant. When appropriate, categorical items were dichotomized into binary variables for the analysis in an attempt to simplify the instrument.

Receiver operating characteristic curve

From the data in the training sample, we generated an ROC curve by plotting the sensitivity of the questionnaire against 1−specificity. As a graphical representation of the trade-off between false-negative and false-positive rates for every possible cut-off point, the ROC curve reflects the trade-offs between sensitivity and specificity and plots the false-positive rate on the X axis and the true-positive rate on the Y axis. The area under the curve is a measure of correlation between the prediction of the questionnaire and the gold standard diagnosis. The closer the area under the curve (AUC) is to 1, the better the test is. To validate the derived logistic model, we compared the ROC from the prediction sample (n = 838) to the ROC of the validation sample (n = 200).

Results

General results

Over a 1-year period, from April 2008 until April 2009, 2397 subjects fulfilled the set screening criteria and completed the extended questionnaire (Figure 1). During this time period, a total of 1067 subjects (44.5%) were randomly selected for the semi-structured telephone interview, of which 1038 (97.3%) were reached and could be used in the analysis. A total of 29 subjects (2.7%) were not included in the analysis because they could not be reached by telephone, after having tried at least five times. From these 1038 subjects, 838 (79.4%) were randomly selected and used for the prediction model and the remaining sample of 200 subjects (18.9%) was used for validation (Figure 1).

Baseline characteristics of the total study population and separate prediction and validation samples are depicted in Table 1. Almost 90% of self-reported migraineurs had previously been diagnosed with migraine by a physician. Age, gender, prevalence of previous migraine diagnosis, and use of anti-migraine medication did not differ significantly between selected subjects and non-selected subjects, nor between subjects that were reached compared to those that could not be reached for telephone interview (see Table 1). In the selected subjects (n = 1067; with special attention to patients which fulfilled ICHD-2 migraine criteria except for attack duration), the algorithm diagnosis of ‘no-migraine’ was more prevalent compared to non-selected subjects [28.6% (305/1067) vs. 2.7% (36/1330); p < 0.001; n = 1330].

Table 1.

Baseline characteristics of total study population and separate study samples

		Selection for study		Telephone interview		Sample
Characteristic	Total (n = 2397)	Not selected (n = 1330)	Selected (n = 1067)	Not reached (n = 29)	Reached (n = 1038)	Training (n = 838)	Validation (n = 200)
Age (years)	42.8 ± 11.9	41.6 ± 12.0	44.3 ± 11.6	43.9 ± 11.1	44.4 ± 11.6	44.6 ± 11.7	43.3 ± 11.5
Female	84.8	83.9	85.8	89.7	85.6	85.0	88.5
Ever M diagnosis	88.9	87.8	90.2	100	89.9	90.2	89.0
Use of anti-M drugs	82.8	80.3	85.8	93.1	85.6	85.2	87.5
Algorithm diagnosis M	87.1	97.3^a	71.4^a	79.3	72.4	72.1	73.5

Values are mean ± standard deviation or %.

p < 0.001 (χ²-test).

M, migraine.

Screening questionnaire

In total, 94.6% of subjects (982/1038) fulfilling the screening criteria, fulfilled ICHD-2 migraine criteria in the telephone interview. We considered everyone fulfilling the screening criteria to be migraineur. We used a logistic model to predict individual aura vs. no aura status.

Algorithm diagnosis

From the total sample of 1038 subjects, the ICHD-2-based algorithm classified 488 subjects as MO patients, 251 as having MA, and 299 subjects as non-migraineurs (Figure 1). Of these, 243 were misclassified as non-migraineurs due to reporting of longer than actual attack duration. Table 2 summarizes the sensitivity, specificity, PPV and NPV as well as the corresponding likelihood ratios for the ICHD-2-based algorithm diagnosis of migraine aura in the total sample (n = 1038). Similar values for this classification in the training sample (n = 838) suggest this sample is a good representation of the whole group. In both the total group and the training sample, sensitivity for aura was approximately 0.45, specificity 0.95, PPV 0.88 and NPV 0.70 (Table 2). Additionally, we calculated characteristics of all individual questionnaire items that reflect migraine headache and migraine aura criteria and summarized those in Supplementary Tables e-1 and e-2 (available online). The results show individual sensitivity ranging up to approximately 0.97 (photophobia; nausea) and PPV up to approximately 0.98 (headache severity; headache duration).

Table 2.

Sensitivity, specificity, and positive and negative predictive values as well as the corresponding likelihood ratios for diagnosis of migraine aura based on the IHCD-II-based algorithm (in both the total group and training sample) and the derived seven-item prediction model (in both the training sample and in the validation sample)

	ICHD-2-based algorithm		Model
	Total sample (n = 1038)	Predictive sample (n = 838)	Training sample (n = 838)	Validation sample (n = 200)
Sensitivity (%)	45	44	83	86
Specificity (%)	95	95	74	75
PPV MA (%)	88	89	74	74
PPV MO ( = NPV MA) (%)	70	64	83	86
Positive likelihood ratio	8.2	8.7	3.1	3.5
Negative likelihood ratio	0.6	0.6	0.2	0.2

MA, migraine with aura; MO, migraine without aura; NPV, negative predictive value; PPV, positive predictive value.

Phase I: derivation of predictive model

Using logistic regression, seven questions (from the 33 included; none showed Spearman rank correlation >0.9) showed a significant impact on the likelihood of having a migraine aura in accordance to the gold standard derived from the telephone interview. These questions are summarized in Table 3, which also shows significance levels and regression coefficients derived from the logistic model. The questions show partial overlap with the questions used in the ICHD-2-based algorithm. This model explained between 35.4% (Cox and Snell R Squared) and 47.3% (Nagelkerke adjusted R Squared) of variance and correctly classified 651/838 (77.8%) of subjects.

Table 3.

Significantly correlated questions (n = 7) with regression coefficients, odds ratios and significance levels derived from the logistic regression model (training sample; n = 838)

Question	B	OR (95% CI)	p
Did you have visual disturbances before headache in the past 12 months?	0.729	2.07 (132⊟3.26)	0.002
Did the visual disturbances last 5⊟60 minutes?	1.658	5.25 (3.08⊟8.96)	<0.001
Have you had scintillating lines before or during your headache in the past 12 months?	1.210	3.35 (2.06⊟5.45)	<0.001
Have you had loss of vision before or during your headache in the past 12 months?	0.913	2.49 (1.63⊟3.80)	<0.001
Did you suffer from numbness or a tingling feeling in your face/unilateral arm/leg that started prior to headache in the past 12 months?	0.631	1.88 (1.07⊟3.29)	0.027
Did you use nonsense words prior or during your headache in the past 12 months?	0.680	1.97 (1.22⊟3.19)	0.005
Did you use a triptan in the past 12 months?	−0.561	0.57 (0.39⊟0.83)	0.003

B, regression coefficient; CI, confidence interval; OR, odds ratio.

ROC curve

From the data in the predictive cohort, we generated an ROC curve by plotting the sensitivity of the questionnaire against 1−specificity (Figure 2a). This analysis resulted in an optimal cut-off point for the used logistic model at 0.35 with AUC of 0.85 (95% CI 0.83⊟0.88), yielding a seven-item questionnaire with a sensitivity of 0.83 and a specificity of 0.74. Compared to the ICHD-2-based algorithm outcome, this approach therefore resulted in a vast increment in sensitivity, with only small decrement of specificity (Table 2).

Phase II: validation of derived prediction rule

Using the predictive model and cut-off point (0.35) derived from the training sample (n = 838), we validated this model in a second, independent sample (n = 200) of subjects who also fulfilled the set screening criteria. This analysis showed the model to have approximately similar sensitivity and specificity in this validation sample (Table 2). In the validation cohort, the ROC curve yielded an AUC of 0.87 (95% CI 0.82⊟0.92), which is comparable to the output from the training cohort (Figure 2b). When using this cut off from the training cohort, migraine aura diagnosis was predicted correctly in 160/200 (80.0%) of subjects.

Test−retest reliability

For a random selection of 44 patients who completed the extended questionnaire, a second time, with a mean test⊟retest interval of 155 days (median 89 days, range 1–422 days), test⊟retest reliability was found to be good with a test⊟retest kappa for algorithm diagnostic group of 0.59 (95% CI 0.38−0.80). Test⊟retest interval did not influence agreement (linear regression, p = 0.852).

Discussion

Our study has been the first one to validate a web-based questionnaire for purposes of diagnosing aura cases using a large sample of self-reported migraineurs. Few previous studies on migraine screeners and questionnaires have focused on migraine aura and the numbers of MA cases used to validate the questionnaire instruments in these studies were limited to n = 8⊟186 (17,19,22–24) respectively, in comparison to the large number of 488 aura cases in our study. Physicians frequently rely on aura as a cardinal symptom of migraine, as suggested by the 1.9-fold higher rate of medical diagnosis in interview settings when comparing MA cases to cases of MO (25). Our study shows that, in self-reported migraineurs, a distinction between MA and MO can be made via a self-administered web-based questionnaire, with a focus on visual aura symptoms. The difficulty in diagnosing other aura types might be explained by the lack of perception and recognition of verbal and other non-visual auras by patients (26). For diagnosing patients with these specific aura symptoms a clinical interview is needed. However, since the vast majority of the self-reported aura cases suffer from visual auras and only a small minority suffers from non-visual auras (27), we believe the number of non-visual aura cases is negligible when recruiting from a population of self-reported migraineurs. Perhaps the most helpful item identifying aura cases is the duration of the aura phenomena, since this question enables to distinguish visual aura symptoms from non-specific visual disturbances. Additionally, our data show aura patients are less likely to use triptans for rescue medication, which might be an indicator of lower headache severity.

We show that the question addressing the duration of the headache may hamper correct identification of migraine cases in a web-based questionnaire setting because some migraineurs overestimate the duration of an attack. Conversely, a question addressing headache severity should be included because this is helpful in distinguishing aura cases with migraineous headache from patients with non-specific headache.

The strength of our study includes the large samples of both the training (n = 838) and validation sample (n = 200), which are representative for the population studied. Both outclinic patients and other patients (most of whom are treated by their own GP or neurologist elsewhere) were included via the same web-based flow. We found no clinical or demographic differences between these populations that could have affected the predictive model. Secondly, the use of a telephone interview as a gold standard by well-trained physicians with consultation of a headache specialist assured precise categorization of migraineurs. Although we did not have a face-to-face interview as gold standard, we feel that our thorough semi-structured telephone interview safeguarded a very reliable migraine and aura diagnosis. Thirdly, the use of a validated screening instrument prior to our new questionnaire resulted in a group of self-reported migraineurs in which 95% could in fact be diagnosed with migraine. Fourth, we used a web-based questionnaire that was easy to fill out and send in for participants. With this approach, we successfully recruited large samples of migraineurs and contributed to the identification of the first genetic risk factor for the common forms of migraine (13). We included a selected population of self-reported migraineurs, that had already been diagnosed with migraine by a physician, or otherwise thought they suffered from migraine, in which our questionnaire shows a high reliability in diagnosing aura. Our study did not aim to validate the questionnaire as a screening instrument for migraine in a naive, general population.

The World Wide Web as a tool for recruiting patients and conducting research has several advantages. First, a large and diverse subject population can be reached at low cost (16). Second, internet research imposes fewer burdens on participants compared to non-internet research (15). Third, available software permits data entry and analysis in a secure web database. Fourth, investigators may be able to increase patient awareness and participation on clinical research. However, there might be certain challenges too (28). Internet users tend to be younger and better educated than the patient population as a whole; visually impaired and minority groups may be underrepresented; and the symptoms expressed by participants may be more severe then is typical. We feel, however, these potential biases have not pivotally influenced our data. Additionally, the so-called ‘virtual Munchhausen syndrome’, i.e. individuals referring themselves for studies for which they are not truly eligible, may compromise the validity of results (29). In our study, we have no evidence that data have been influenced by subjects masquerading electronically as patients. This is in accordance with previous migraine research (15). Even with such biases, altogether, the internet represents an appropriate aid to conduct research aimed at collecting clinical headache data from large numbers of patients.

We conclude that our web-based recruitment system in combination with an automated study flow is a very successful instrument to truly distinguish between MA and MO in self-reported migraine patients. We propose to use our identified seven questions that have a higher accuracy in identifying aura cases from a population of self-reported migraineurs than an ICHD-2-based algorithm.

Footnotes

Funding

This work was supported by grants of the Netherlands Organisation for Scientific Research (NWO) (MDF: 903-52-291, Vici 918.56.602; GMT: 907-00-217, Vidi 917-11-319) and by a grant from the Centre for Medical Systems Biology (CMSB) established by the Netherlands Genomic Initiative/Netherlands Organisation for Scientific Research (NGI/NWO).

Conflicts of interest

MDF, has, in the past 3 years, received grants and consultancy/industry support from Almirall, Coherex, Colucid, Eisai, GlaxoSmithKline, Linde, MAP, Medtronic, Menarini, Merck, Minster, Pfizer, and St Jude, and independent support from the Netherlands Organisation for Scientific Research (NOW). GMT has received grants and consultancy/industry support from Merck, Janssen-Cilag, and Almirall, and independent support from NWO. The other authors report no conflicts of interest.

References

Stewart

Shechter

Rasmussen

. Migraine prevalence. A review of population-based studies. Neurology 1994; 44(6Suppl 1): S17–S23.

Scher

Stewart

Liberman

Lipton

. Prevalence of frequent headache in a population sample. Headache 1998; 38(7): 497–506.

Launer

Terwindt

Ferrari

. The prevalence and characteristics of migraine in a population-based cohort − the GEM study. Neurology 1999; 53(3): 537–542.

Stovner

Zwart

Hagen

Terwindt

Pascual

. Epidemiology of headache in Europe. Eur J Neurol 2006; 13(4): 333–345.

Headache Classification Subcommittee of the International Headache Society. The International Classification of Headache Disorders: 2nd edition. Cephalalgia 2004; 24(Suppl 1): 9–160.

Russell

Rasmussen

Thorvaldsen

Olesen

. Prevalence and sex-ratio of the subtypes of migraine. Int J Epidemiol 1995; 24(3): 612–618.

Ulrich

Gervil

Kyvik

Olesen

Russell

. Evidence of a genetic factor in migraine with aura: a population-based Danish twin study. Ann Neurol 1999; 45(2): 242–246.

Ulrich

Gervil

Kyvik

Olesen

Russell

. The inheritance of migraine with aura estimated by means of structural equation modelling. J Med Genet 1999; 36(3): 225–227.

Gervil

Ulrich

Kyvik

Olesen

Russell

. Migraine without aura: a population-based twin study. Ann Neurol 1999; 46(4): 606–611.

10.

Ulrich

Gervil

Fenger

Olesen

Russell

. The prevalence and characteristics of migraine in twins from the general population. Headache 1999; 39(3): 173–180.

11.

Gervil

Ulrich

Kaprio

Olesen

Russell

. The relative role of genetic and environmental factors in migraine without aura. Neurology 1999; 53(5): 995–999.

12.

Stewart

Staffa

Lipton

Ottman

. Familial risk of migraine: a population-based study. Ann Neurol 1997; 41(2): 166–172.

13.

Anttila

Stefansson

Kallela

Todt

Terwindt

Calafato

. Genome-wide association study of migraine implicates a common susceptibility variant on 8q22.1. Nat Genet 2010; 42(10): 869–873.

14.

de Groen

Barry

Schaller

. Applying World Wide Web technology to the study of patientswith rare diseases. Ann Intern Med 1998; 129(2): 107–113.

15.

Lenert

Looman

Agoncillo

Nguyen

Sturley

Jackson

. Potential validity of conducting research on headache in internet populations. Headache 2002; 42(10): 200–203.

16.

Strom

Pettersson

Andersson

. A controlled trial of self-help treatment of recurrent headache conducted via the Internet. J Consult Clin Psychol 2000; 68(4): 722–727.

17.

Hagen

Zwart

Vatten

Stovner

Bovim

. Head-HUNT: validity and reliability of a headache questionnaire in a large population-based study in Norway. Cephalalgia 2000; 20(4): 244–251.

18.

Cady

Borchert

Spalding

Hart

Sheftell

. Simple and efficient recognition of migraine with 3-question headache screen. Headache 2004; 44(4): 323–327.

19.

Kirchmann

Seven

Bjornsson

Bjornssdottir

Gulcher

Stefansson

. Validation of the deCODE Migraine Questionnaire (DMQ3) for use in genetic studies. Eur J Neurol 2006; 13(11): 1239–1244.

20.

Hagen

Stovner

Zwart

. Potentials and pitfalls in analytical headache epidemiological studies − lessons to be learned from the Head-HUNT study. Cephalalgia 2007; 27(5): 403–413.

21.

Halpern

Albert

Krieger

Metz

Maidment

. Comparison of receiver operating characteristic curves on the basis of optimal operating points. Acad Radiol 1996; 3(3): 245–253.

22.

Valentinis

Valent

Mucchiut

Barbone

Bergonzi

Zanchin

. Migraine in adolescents: validation of a screening questionnaire. Headache 2009; 49(2): 202–211.

23.

Kallela

Wessman

Farkkila

. Validation of a migraine-specific questionnaire for use in family studies. Eur J Neurol 2001; 8(1): 61–66.

24.

Gervil

Ulrich

Olesen

Russell

. Screening for migraine in the general population: validation of a simple questionnaire. Cephalalgia 1998; 18(6): 342–348.

25.

Leone

Filippini

D'Amico

Farinotti

Bussone

. Assessment of International Headache Society diagnostic criteria: a reliability study. Cephalalgia 1994; 14(4): 280–284.

26.

Facheris

Vogl

Hollmann

Sixt

Pattaro

Schonhuber

. Adapted Finnish Migraine-Specific Questionnaire for family studies (FMSQ(FS)): a validation study in two languages. Eur J Neurol 2008; 15(10): 1071–1074.

27.

Rasmussen

Olesen

. Migraine with Aura and Migraine Without Aura - An Epidemiologic-Study. Cephalalgia 1992; 12(4): 221–228.

28.

Rothman

Cann

Walker

. Epidemiology and the internet. Epidemiology 1997; 8(2): 123–125.

29.

Soetikno

Mrad

Pao

Lenert

. Quality-of-life research on the Internet: feasibility and potential biases in patients with ulcerative colitis. J Am Med Inform Assoc 1997; 4(6): 426–435.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.17 MB