Summary

The most notable methodological remarks among countries where problems have been detected are summarised below, followed by a short overall methodological summary and some final remarks.

Country-specific methodological remarks

Deviations from the common ESPAD methodology and setbacks of methodological importance which are deemed important when interpreting the results are listed for individual countries.

Albania. Inconsistency measurements related to reliability and validity indicate that the collected data may be of somewhat lower quality compared to the average ESPAD country.

Austria. The data collection was performed online (instead of using pencil and paper). The school participation rate was low (21 %). This gives rise to some uncertainty regarding the collected data, even though no serious sample bias has been detected. A relatively high proportion (4.2 %) of the questionnaires were discarded in the central data-cleaning process.

Bulgaria. Compared to other countries a relatively high level of inconsistent answers and a slightly higher level of ‘relevin’ use was noted, indicating a somewhat lower level of data validity than average.

Belgium (Flanders). Geographic limitations since only students in the Dutch-speaking areas took part (approximately 60 % of the population). The data collection was performed during autumn 2014 (instead of spring 2015). The target population was redefined to give an average age in line with other participating countries.

Cyprus. Only government-controlled areas were included in the sample (approximately 80 % of the population). Standard classroom report information was not available, which leaves uncertainties regarding student participation rates and the classroom situation during data collection. A relatively high proportion (3.8 %) of the questionnaires were discarded in the central data-cleaning process. A relatively high level of inconsistent answers and a relatively high level of ‘relevin’ use was noted, indicating a somewhat lower data quality than the ESPAD average.

Denmark. The sampling frame had a relatively low (78 %) coverage of the ESPAD target group, partly explained by the fact that only one school grade was sampled and that boarding schools were not included in the sampling frame. Low (26 %) school-participation rates, in turn leading to a relatively small net sample (1 670 students). This gives rise to some uncertainty regarding the collected data, though no serious sample bias has been detected.

Former Yugoslav Republic of Macedonia. A relatively low proportion (79 %) of the 1999 birth cohort was enrolled in school. A relatively large proportion (24 %) responded that they would be unwilling to report possible use of cannabis, indicating that under-reporting may be higher than in other countries.

Georgia. The sampling frame had a relatively low (73 %) coverage of the target population since the 1999-born students were distributed among several grades but only the main grade was sampled. The students were approximately 6 months older compared to the ESPAD average since the data collection was performed during autumn instead of spring and there was no redefinition of the target group.

Ireland. The school participation rates were low (21 %), in turn leading to a relatively small net sample (1 470). This gives rise to uncertainty regarding the collected data, though no serious sample bias has been detected.

Latvia. The data collection was performed online (instead of using pencil and paper). The school participation rate was low (49 %), in turn leading to a very low number of students in the net sample (1 119 students). The fieldwork suffered from setbacks, leading to some uncertainties regarding the quality of the data collection. A high proportion (7.6 %) of the questionnaires were discarded. As a precautionary measure related to these methodological obstacles the Latvian results are presented below a line in the results tables and no comparisons are made with previous surveys.

Liechtenstein. The data collection was performed online (instead of using pencil and paper).

Moldova. The Transnitria region was not covered by the sample (circa 20 % of the Moldovan population).

Netherlands. The data collection was performed online (instead of using pencil and paper). Low (43 %) school participation rates, which in turn lead to a relatively small net sample (1 684 students). This gives rise to uncertainty regarding the collected data, though no particular serious bias has been detected. The data collection was performed during autumn instead of spring (the target population was redefined to give an average age in line with other participating countries).

Norway. A relatively high proportion (4.2 %) of the questionnaires were discarded in the central data-cleaning process.

Portugal. A relatively high proportion (6.0 %) of parents refused to allow their children to participate in the survey. The sampling frame covered 86 % of the ESPAD target group since private schools were not included.

Romania. A relatively high proportion (6.9 %) of the parents refused permission for their child to participate in the survey.

Non-ESPAD countries

Spain and the United States. These countries do not participate in the ESPAD study but carry out similar school surveys with similar questions. Whenever data are judged to be comparable, results from these countries are reported. However, since they do not use the full ESPAD methodology, such comparisons definitely include a measure of uncertainty. This is emphasised by presenting data from Spain and the United States below a line in the results tables.

Spain. Data were largely collected between November 2014 and April 2015. Because of this, the average age of the Spanish respondents is slightly lower than the ESPAD average (15.5 and 15.8 years, respectively).

United States. The data collection in the United States was carried out between February and June 2015. The estimated average age was 16.1 years, slightly above the ESPAD average of 15.8 years. Approximately 5 % of the questionnaires were discarded in the cleaning process, which was higher than the ESPAD average of 1.8 %.

General methodological remarks

The main purpose of the ESPAD project is to collect comparable data on substance use among students of the same age in as many European countries as possible. The studies are conducted as school surveys, following a common methodology described in a handbook. The target population of the ESPAD study is defined as the national population of students who turn 16 during the calendar year of the survey, excluding those who are enrolled in either special schools or special classes for students with learning disorders or severe physical disabilities. In nearly all countries, a very large majority of those born in 1999 were enrolled in school during the 2015 data collection (93 % on average). The estimated average age of participating students across the ESPAD countries was 15.8 years.

In some countries there were limitations in the geographical coverage. In most cases they were minor. However, for three countries these limitations were bigger: Belgium, where data collection was limited to the Dutch-speaking areas (Flanders); Cyprus, where data collection was limited to governmentcontrolled areas; and Moldova, where the Transnitria region was not included in the sampling frame.

In the vast majority of the countries, the representativeness of the sampling frames was high and usually covered at least 90 % of the target student population. In countries where not all relevant grades and school categories were included, the sample is representative only of students born in 1999 enrolled in participating grades and school categories.

No particular problems were noted regarding absent students or present students declining participation, nor were there, apart from two countries, any particular problems with parents refusing to allow their child to take part. School cooperation was satisfactory in most countries, even though some countries encountered notable problems with schools that refused to take part for various reasons. In five countries, less than 50 % of the sampled schools or classes took part in the ESPAD survey. This in turn had a negative influence on the number of participating students. Taken together, these two factors make the representativeness of the data for a few countries somewhat uncertain.

The analysis of the available information suggests that the validity of the ESPAD studies is high in most countries. The indicators analysed include student cooperation, student comprehension, anonymity, reported use of a dummy drug, rates of missing data and logical consistency. The main issues of validity relate to reported lack of willingness to answer honestly as well as to cultural context. Validity problems, however, seem to be limited in scope and to affect only a few countries to a rather limited extent.

Below follow some final general remarks.

The overall impression is that the methodological problems in the 2015 ESPAD data collection were relatively limited.
With one exception, no country experienced methodological problems of such a degree that the comparability of its results with data from other countries was called into question.
The estimates for illicit drug use probably represent an underestimate, and the level of under-reporting may differ somewhat between countries. However, it is not likely that the classification of countries as either high-prevalence or low-prevalence ones could be questioned on the basis of differences in under-reporting between countries.
Despite some differences in a cultural context, the validity of the ESPAD survey is assumed to be generally high.
Individual countries may suffer from various methodological problems that should be taken into account when their results are analysed. These were briefly summarised before (in ‘Country-specific methodological remarks’).
It is more relevant to focus on the magnitude of the estimates than on absolute figures, both when analysing data from single countries and when interpreting trends and differences between countries. Small differences between countries should be considered carefully. They may not reflect real differences.