Representativity

The target population of the ESPAD study is defined as the national population of students who turn 16 during the calendar year of the survey, excluding those who were enrolled in either special schools or special classes for students with learning disorders or severe physical disabilities.

As a matter of principle, data can never be representative of any groups other than those included in the sampling frame. In ESPAD, the issue of representativeness is linked to several aspects, including geographic coverage, sampling, the exclusion of grades or school categories and the level of interest shown by schools and students in participating in the data collection.

Geographic coverage

The objective of including all geographic regions in the sampling frame was reached to a very large degree in most countries. Four countries (Finland, France, Portugal and Ukraine) had some minor limitations in the geographic coverage, but in all those countries at least 95 % of the population was covered (see Table F). Except for Ukraine, these geographical limitations have been at hand in previous surveys and are not considered to be of any major importance. In Ukraine, the Crimea area was not included in the 2015 sample (circa 5 % of the population).

Three countries had a geographical coverage below 95 %. For Moldova the study covered approximately 85 % of the population since the Transnitria region was not included. For Cyprus approximately 80 % of the population was covered by the survey due to the fact that only governmentcontrolled areas were covered by the sampling frame. The lowest geographical coverage (61 %) can be noted for Belgium, since only the Dutch-speaking part (Flanders), as well as Dutch-speaking schools in the Brussels Capital Region, participated in the data collection. Apart from 2003, when Belgium took part in whole, the abovementioned geographical limitations were at hand also in previous data collections.

It is important to keep in mind that the results for Cyprus, Moldova and Belgium are representative only for the populations from which the samples were drawn, according to the geographical limitations mentioned above.

Sampling strategies

Sampling in the ESPAD project is based on school classes as the final sampling unit (i.e. organisational units of students). This is vastly more economical than sampling individual students, and it also has some desirable methodological properties. In particular, the sampling of entire classes can be expected to increase students’ confidence in their anonymity. Sampling individual students and asking them to fill in a questionnaire individually, by contrast, could affect the truthfulness of their answers and therefore bias the results of the study.

An overview of the sampling procedure in each country is provided in Table F. The number of students born in 1999 in the Faroes, Iceland, Liechtenstein, Malta and Monaco was close to, or below, the number of students to be sampled according to the ESPAD guidelines (1 200 per gender). In these countries, therefore, all students were surveyed.

Table F. Characteristics of the national samples. ESPAD 2015

In most countries a two-stage sample was performed, with the school as the primary unit and the class as the final sampling unit. In some countries, the class was the only sampling unit, i.e. samples of classes were drawn from comprehensive lists of classes, while in Montenegro actual students were sampled, ‘pulling’ their respective class.

Some countries have not considered what might be called the ‘problem of small and large schools and classes’. In some countries, all schools/classes had the same probability of being sampled, regardless of the size of each class and school. In practice, this means that students belonging to small classes or attending small schools are over-represented in the samples. If students in these classes or schools have different substance use habits from students in large classes or schools, the data are not entirely representative of the population. In many countries where this problem might have occurred, however, a stratified sample was used, and it seems reasonable to assume that the sizes of schools and classes are rather similar within each stratum. Further, class size is fairly standard in many countries, and the classes within a school usually do not vary greatly in size. On the whole, the ‘problem of small and large schools and classes’ is not considered to be a major problem in the context of the overall ESPAD project.

In countries where non-proportionate stratification was used for sampling, the data was weighted (weightings are used in 11 countries). Lack of data about school (and class) size has complicated the sampling procedure for some countries. Despite this there is reason to assume that sampling was carried out in the best possible way and that sampling problems have not affected the outcome of any survey in such a negative way that the possibility to make comparisons with other countries is jeopardised.

Birth cohort representativity

There are differences between countries in to what extent the 1999 birth cohort is attending regular school. In some countries, schooling is compulsory until the age of 16 years. In others, this is the age when students either enrol in upper-secondary school, start other training or enter the labour market. On average, 93 % of the 1999 birth cohort was enrolled in regular school at the time of data collection (students with special needs who attend special schools/ classes are not a part of the defined ESPAD population) (Table F). For seven countries the proportion was below 90 %, and for the former Yugoslav Republic of Macedonia the lowest proportion was noted (79 %).

Such differences may have an impact on the results, since it could be the case that persons who have already left school may have a different substance use pattern compared to their peers in school. On the other hand, one should not forget that the ESPAD study is actually only intended to be representative for students, not for entire birth cohorts.

Student representativity

The target population of the ESPAD project is students who turn 16 years old during the year of data collection. For the 2015 study, this means students born in 1999. In some countries, the vast majority of students born in 1999 were enrolled in a single grade. In others, large proportions of them were to be found in two or more grades. The recommendation given for the latter case, subject to the availability of the necessary resources, was to include as many grades as possible where 1999-born students were to be found, or at least the grades where 10 % or more of the target population was located. If not all grades with students in the target age group are included in the datacollection exercise, the sample is representative only of 1999- born students in the grade(s) chosen.

In about three quarters of the countries, not more than two grades were surveyed. Four or more grades were covered in Belgium (Flanders), Portugal, Monaco, France and Liechtenstein. For 30 of the 35 countries, the sampling frames covered 90 % or more of the students born in 1999. In addition, the proportion was also rather high (85-88 %) in another three countries (Faroes, Lithuania and Portugal). However, the corresponding figures were lower in Georgia and Denmark (73 % and 78 %, respectively).

In the case of Denmark the lower representativity was partly explained by the fact that 8 % of the target population was found in grades either above or below the one being surveyed. Another explanation was that boarding schools were not included in the sampling frame, where roughly 12 % of the target population could be found. In Georgia the lower representativity is solely explained by the fact that the target population was distributed among several grades but only the main one was sampled.

To sum up, there are differences between countries in how well the samples represent students born in 1999, and also in to what extent the birth cohort is enrolled in regular schooling. It is not possible to establish how the results may have been affected by a somewhat lower representativity, even though this uncertainty is important to acknowledge.

Average age

Based on the time of data collection, an approximate average age of the students has been estimated for each country (Table F). The average ESPAD age was 15.8 years. Due to the fact that the Belgian and Dutch data collections took place during autumn, the target populations were redefined to give an average age in line with other participating countries. This was, however, not the case for Georgia, which resulted in a slightly older population than average (16.4 years). This should be considered in relation to the results, since older students may have had more opportunities to experience use of different substances.

School cooperation

The proportions of participating schools and classes are shown in Table H. On average, about 84 % of the sampled schools (and classes) took part in the survey. The proportions of schools that refused to participate differ substantially among the countries. In half of them, all or nearly all sampled schools took part in the survey (95 % or more). In most other countries the proportions were relatively high as well (between 83 and 95 %). Reasons given for not taking part were usually lack of time, examinations or other factors related to schoolwork, and sometimes a general perception of being over-surveyed.

In five countries less than half of the sampled schools took part in the study. Ordered by falling participation rates, those countries were Latvia (49 %), the Netherlands (43 %), Denmark (26 %), Ireland and Austria (both 21 %). Apart from Latvia, those countries have also previously belonged to the group with low school participation.

Table H. Participating schools and classes and students’ presence rates. Percentages. ESPAD 2015

In Austria, weightings were introduced in order to adjust for a selection bias due to school non-participation. The weighting adjusts for education level and type of region. In Denmark the high non-response rate was related to a relatively late decision on the funding and to the implementation of a school reform parallel to the data collection. There were no indications of bias noted for the net sample though, and the Danish team found the collected data representative for Danish students.

Due to a tendering process in Ireland the data collection was delayed, and it proved difficult to enter the schools at the end of the semester. In spite of the high level of nonparticipating schools, the Irish team found no important signs that the achieved sample should not be representative for young people in Ireland (gender, geographic location, school type and socioeconomic background were checked). In the Netherlands, school refusals have been a growing problem throughout the country. School characteristics such as size and type of education were controlled for. The Dutch team found no reason to believe that non-participation was selective. However, school type was considered, among demographic aspects, when the weightings were computed.

Apart from the 1995 data collection, Latvia has not previously experienced any particular problems in relation to school participation. In 2015 a new procedure for collecting data was introduced. In previous data collections paper questionnaires and research assistants were used and the first contact with schools was made via a telephone call. This time the first contact and reminder was made via email, and teachers were responsible for conducting online data collection. The emphasis on communication through the internet did not fully succeed, however. The obtained net sample was skewed, with a higher number of nonparticipating schools in larger cities, which was taken into account when the weighting variable was computed.

To sum up, high drop-out rates for schools call the representativeness of the data into question. Refusals by schools were a relatively limited problem in the majority of the countries. However, in Austria, Denmark, Ireland, Latvia and the Netherlands the school participation rates were below 50 %. Apart from the uncertainty about representativeness this creates, it sometimes resulted in a low number of participating students, especially so for Latvia (see ‘Number of participating students’).

Student response rates

Table H shows the proportion of students present in the classroom during the data collection. The proportions have been calculated on the basis of the classroom reports, where the fieldworkers indicated (a) the total number of students belonging to a participating class and (b) the number of students who were present when the survey was performed.

The proportion of students present in participating classes was high in most countries. The average was 88 %, and in 27 of the 35 countries 85 % or more of the students were present in class. Ukraine reported the lowest proportion of students present (80 %). This is not considered to be any major deviation though. No country reported any major methodological problems in connection with absent students. However, Cyprus failed to collect the requested information since non-standard classroom questionnaires were used. This means that student presence rates in Cyprus remains unknown, which definitely is a disadvantage, even though there is no information available indicating it should have been particularly low.

According to the standard instructions, the students are informed that the study is voluntary. Refusal by students to participate was rare in nearly all countries. On average, 0.5 % (0.0-1.5 %) of the students present in the classrooms refused to take part in the survey (Table C). In Ireland, Latvia, Portugal, Romania and Sweden these rates were above 1 %.

Some form of parental consent was asked for in roughly three quarters of the countries. For three countries, active parental consent was requested. According to Table C, 0.5 % (0.0-1.7 %) of the students were refused permission by their parents to take part in the study in countries where only passive consent was needed. In the three countries where active consent was requested, refusal rates were higher: Georgia 2.0 %, Portugal 6.0 %, Romania 6.9 %. Hence, parental refusal rates were rather high in the latter two countries. Even though it cannot be decided whether this had any influence on the substance use estimates, this ought to be kept in mind when interpreting the results.

Higher rates of sampled students not taking part in the study increases the risk that the net sample is biased. The response rates are however deemed to be satisfactory overall, even when the refusal rates are taken into account. It should however be noted that parental refusal for their children to take part in the survey was more common in Portugal and Romania. It should also be noted that Cyprus was lacking information not only on the number of students present but also on refusals among students as well as parents.

Number of participating students

To ensure that a satisfactory level of precision can be obtained in the estimates for various subgroups of the population, the ESPAD guidelines recommend sampling enough classes to obtain 1 200 participating students of each gender.

In countries with fewer than 2 800 students in the target population, it is recommended that the total population be included. This was the case in the three countries with the smallest sample sizes: Liechtenstein (316 students with valid questionnaires), Monaco (397) and the Faroes (511) (Table C). The sampling frames also included the total population of the somewhat larger countries of Iceland (2 663 students with valid questionnaires) and Malta (3 326).

Nine countries did not fully meet the criteria of 2 400 students. In six additional countries the net samples comprised less than 2 000 students: Belgium (Flanders), Denmark, Georgia, Ireland, Latvia and the Netherlands. These were all countries with relatively high rates of nonparticipating schools, reflected in a lower number of students included in the net sample.

In the case of Latvia, only 1 119 students were obtained in the net sample, which must be considered a quite serious deviation from the quality criteria regarding the number of subjects to be analysed. In the five other countries mentioned above, the number of participating students ranged between 1 470 and 1 966. Even though these figures are low, the numbers of valid questionnaires have been deemed enough to enable international comparisons, however not without caution.