Limitations

Although the ESPAD survey is based on a common methodology, some limitations that may possibly weaken the validity of the estimates need to be discussed.

(1) In Belgium (Flanders), data were collected half a year earlier than in the majority of countries (in autumn of the previous year), and in Georgia and the Netherlands, half a year later (in the autumn of the same year). In the former case, students were on average half a year younger, while in the latter case they were on average half a year older. With the exception of Georgia, the target population was, however, redefined to give an average age in line with the other participating countries that collected data in spring. With students being on average 16.4 years compared to the ESPAD average of 15.8 years, rates of substance use may be slightly overestimated in Georgia due to students having had more time to experience or continue substance use.

(2) The school/class participation rates in Austria (17 %), Ireland (18 %) and Denmark (26 %) were exceptionally low compared with the ESPAD average of 87 %. In addition, school/class participation rates were also slightly below 50 % in Latvia and the Netherlands. Low participation rates, however, may not necessarily lead to biased estimates, unless the behaviour in question is rather unequally distributed across schools and classes. A recent simulation study from Germany found that school non-participation in surveys assessing substance use among students is not as worrisome as expected (Thrul et al., 2016). Systematic exclusion of schools, based on the size of the city, school or class, on school environment or on schools’ substance use policies, resulted in significant but rather small changes in prevalence estimates.

(3) In some countries, sampling was only possible in particular regions of the country. In Belgium, only schools from Flanders (representing approximately 60 % of the population) participated in the survey; in Cyprus, data collection was restricted to government-controlled areas, representing approximately 80 % of the population; and in Moldova, the sample represents approximately 85 % of the Moldovan population, with the Transnistria region not included. In these cases, estimates only represent the population of the region where the survey took place.

(4) In the 2015 ESPAD survey, four countries (Austria, Latvia, Liechtenstein and the Netherlands) conducted data collection online, deviating from the usual paper-and-pencil mode of administration. While experience suggests a number of advantages of online data collection, such as interactivity, minimising mistakes of data entry and saving time and costs, the comparability of results from online and paper-and-pencil questionnaires is of concern. Research on differences when comparing online and paper-and-pencil responses on substance use behaviour suggests only small mode effects (Brener et al., 2004; Eaton et al., 2010; Lygidakis et al., 2010; Raghupathy and Hahn-Smith, 2013; Wyrick and Bond, 2011). This is corroborated by a methodological study carried out in Latvia (Trapencieris, 2013). A sample of nearly 2 800 students aged 14-16 years in grades 8-10 was randomly selected to answer the ESPAD questionnaire, either in the schools’ computer lab or via the traditional paper-and-pencil mode. In only three of thirty-two variables measuring substance use were prevalence statistically significant differences found. Although in the majority of studies small mode differences were reported, the differences in most studies indicate higher rates of substance use and other sensitive behaviours if paper-and-pencil questionnaires were used. Thus, comparisons between studies using online and paper-andpencil questionnaires should be interpreted with caution.