Sampling and weighting of the Austrian Psychiatric Prevalence Survey (APPS)

Although “representativity” is a frequently used term, we should use it with caution, for it is not underpinned by a clear definition. For example, Stephan [11] tried to narrow down the term, however, arriving rather at a descriptive statement (e.g., “resembles the population”, p. 32) than a mathematically sound definition allowing for deducing concrete action. In this vein, Kish [12] states that “Representative sampling is a term easier to avoid because it is disappearing from the technical vocabulary.” (p. 26).

Putting aside the lack of clear definition, we also lack a single universal procedure providing for “representativity” with regard to any population. Rather, the specific structure of the population studied and the research question have to be carefully considered. Kish [12] requires the definition of a population “in terms of (1) content, (2) units, (3) extent, and (4) time.” (p. 7). He exemplifies the terms by means of a consumer survey, in which (1) could refer to all persons, (2) to in family units, (3) the US, and (4) in 1965 (ibid.). For the according specification regarding the present study see Sect. “Target Population and the Sampling Framework”. Fulfilling these requirements cannot be achieved with a convenience sample (the outcome of which is entirely unpredictable), or any other simple sampling procedure. Rather, we have to carefully develop a sampling strategy allowing for an adequate collection of prevalence data.

When seeking meaningful population data in the context of mental health epidemiology, one has to consider carefully, which population characteristics should be represented adequately. The most fundamental variables to be taken care of are a respondent’s sex and age. Next, we have to regard the medical care quality (including also administrative aspects), which we want to cover by distinguishing rural vs. urban population. Although several other aspects would be worth considering as well, we have to limit the requirements to available information (see Sect. “Address Source and Time Frame”).

To be able to gauge the extent to which study results can be generalized, we have to be aware of how a sample has been drawn. However, information on sampling are frequently incomprehensive or even entirely lacking. For example, Wancata et al. [9] demand a “checklist of methodological requirements […] (e.g. sampling methods, […])” (p. 407). The present report follows this claim and explains in detail the sampling and weighting scheme of the APPS.

The motivation for this report is to give a full account of the intricacy of obtaining a nation-wide representative sample beyond the sparse details usually to be found in articles (claiming “representativity” of their sample not providing convincing evidence, if any). In contrast, the APPS discloses the rationale of how the sample has been drawn in full detail.

This article is structured in the following way: After describing the population to be covered in Sect. “Target Population and the Sampling Framework”, we will explain the sampling procedure in Sect. “The Sampling Procedure”. Because the sampling comprises probability sampling [e.g., 13‐15], we have to determine the corresponding weights to take the selection probability correctly into account. This step is described in Sect. “Weighting”.

Because several analyses involving various procedures are planned, an overall power analysis cannot be performed. Therefore, we calculated as follows: We expect prevalences of the two largest groups affective disorders (F3 according to ICD-10 [18]) on the one hand and anxiety, dissociative, stress-related, somatoform and other nonpsychotic mental disorders (F4) on the other hand of roughly 10%. Moreover, all analyses shall be performed separately for male and female respondents. Targeting about 50 respondents in these subgroups will result in a total of approximately 1,000. This number matches financial and logistic considerations and it is comparable to similar studies [e.g., 19‐21]. To ensure realizing this target and assuming a low response rate, we decided to include a total of 18,000 respondents.

Addresses were bought from one of the largest Austrian address brokers. The data base was a register of Austrian telephone numbers both landline and mobile. Due to factors like change of residence, participation in a mail preference service (“Robinson List”), deceased, etc., we were advised to use the addresses as soon as possible after drawing, otherwise we could face losses. Therefore, we chose to split the 18,000 addresses into three waves of 6000 individual contact records each, which were contacted soon after sampling. The interviews took place from June 2015 until June 2016 with the sampling waves being carried out in June 2015, October 2015, and March 2016.

Due to the federal structure of Austria, the 9 provinces have key responsibilities in certain public health issues. We therefore decided to represent them accordingly in the sample and stratified in a first step with respect to the provinces.

Data collection is based on face-to-face interviews, so we have to take the interviewers’ routes to the respondents’ households into account. Cluster sampling requires a full list of predefined clusters from which a random selection can be performed. Our address source disposes of the respondents’ districts; hence, we decided to use this information as primary sampling unit in this step. Austria has a total of 117 districts. Based on logistic and financial considerations, a total of about 40 districts was targeted.

Additionally, the province capitals also play a key role with respect to structural and administrative aspects. Therefore, the following cluster sampling scheme was developed:

The row percentages of Table 1 show that the two sex groups are virtually of equal size if taken across the entire country (\(49.95:50.05\)), and also the province shares do not exceed a ratio of \(51:49\). We, therefore, decided to target the same overall number of men and women.

Next, we wanted to represent the nine Austrian provinces and the two sex groups adequately in the sample. For that purpose, we applied the proportion of men and women within each province (Table 1, columns headed “col %”) to the total sample to be drawn (i.e., 500 men and 500 women, see Sect. “Sample Size Consideration”), obtaining the target sample size for each province. The rounded values are given in the last three columns of Table 1.

Next, we split the province target sample size proportionally to the selected districts according to Table S1 in the supplementary file, columns 4 and 6 (headed “%”). The resulting frequencies for each district are given in the last two columns of Table S1 (rounded to integers). These frequencies were multiplied by 18 (i.e., 6 per wave, see Sect. “Address Source and Time Frame”) to obtain the gross number of addresses to contact.

Note: In the following, we will use capital letters to indicate population-based figures and lower case letters for sample-based figures. Stratification is indicated by a superscript in brackets, the subscript \(d\) denotes references to the district and subscript \(p\) to the province. The symbols \(N\) and \(n\) denote (true) population and sample frequencies, \(M\) projections, and \(w\) and \(W\) denote weights. The symbols m and f refer to male and female.

We start with the probability of choosing a district at random. This was done with respect to the number of districts of each province. If \(K_{p}\) is the number of all districts of a province (col. 2 of Table 2) and \(k_{p}\) the number of districts chosen from this province (last col. of Table 2), then the probability of drawing a given district is Note that for the special case Vienna, there is no provincial capital, hence we used \(K\) and \(k\) rather than \(K-1\) and \(k-1\), respectively.

Second, we calculated the probability of a person to be drawn from the selected districts. Due to the stratification according to sex, we had to perform this calculation separately for men and women. If \(N_{d}^{(\mathrm{m})}\) is the number of male and \(N_{d}^{(\mathrm{f})}\) the number of female inhabitants (aged 18–65) of a district \(d\), and \(n_{d}^{(\mathrm{m})}\) and \(n_{d}^{(\mathrm{f})}\) the respective sample sizes, the according conditional probabilities are Hence, the probability of randomly drawing an individual (so far irrespective of the district’s size) is the product Taking the inverse of Eqs. (3a) and (3b) yields intermediate district projection weights \(\widetilde{W}_{d}\), Multiplying the \(\widetilde{W}_{d}^{\mathrm{(\cdot)}}\) with the sample size \(n_{d}\) of the respective district yields the intermediate district projection \(\widetilde{M}_{d}\) (introducing the generic notation (m\(|\)f) to indicate the separate application of the formula according to the stratification by sex). Eq. (5) lays the foundation to generalize from the chosen districts of a province to the entire province. For that purpose, we have to take the sum of the \(\widetilde{M}_{d}\) across all districts of a province \(p\) to obtain the (intermediate) province projection \(\widetilde{M}_{p}\) However, these estimates are biased, because we have not yet considered the district size when randomly selecting the districts in Eq. (1). \(\widetilde{M}_{p}^{\mathrm{(\cdot)}}\) would over-estimate the respective province totals \(N_{p}^{\mathrm{(\cdot)}}\) if we sampled (by chance) rather large districts or under-estimate it if there were more of the small districts of the respective province in our sample (therefore, Eqs. (6) were prefixed “intermediate”).

The district rescaling factor \(R_{d}\) corrects for this bias, again taking into account that the provincial capitals (indexed \(c\)) were deliberately chosen: We yield the corrected district projections \(M_{d}\) by multiplying the intermediate projections (5) by the rescaling factor, i.e., and the province projections \(M_{p}\) by taking the sum across all districts of a province, which, as a matter of fact, equal the province size, i.e.:

To obtain point estimates of population parameters, such as the mean or frequency estimates, for example, we need the respective corrected weights. These are obtainded analoguously by multiplying the intermediate district projection weights by the rescaling factor, i.e.,

However, to remain with the sample frequencies, we may simply apply a sample rescaling factor \(r\) using the sample size \(n\) and the population size \(N\), and obtain the sample district weights

The age weights can be determined directly, because only one target variable is involved [cf.23, ch. 7]. For each age group \(a\), the age weighting factor \(w_{a}\) was obtained separately for male and female respondents using the ratio of the proportion of the sample frequency \(n_{a}\) and the respective population frequency \(N_{a}\): To consider both sampling and age distribution, the weights (11) and (12) must be multiplied, i.e., Using these weights will exactly reproduce the age distribution of the Austrian population as determined by Statistik Austria [17].