The effects of sampling on measures of association between variables based on sample means

Abstract

Many studies that involve people's perceptions or behaviors focus on aggregate rather than individual responses. For example, variables describing public perceptions for some set of events may be represented as mean scores for each event. Event mean scores then become the unit of analysis for each variable. The variance of these mean scores for a variable is not only a function of the variation among the events themselves, but is also due to the variation among respondents and their possible responses. This is also the case for the covariances between variables based on event mean scores. In many contexts the variance and covariance components attributable to the sampling of respondents and their responses may be large; these components can be described as measurement error. In this paper we show how to estimate variances and covariances of aggregate variables that are free of these sources of measurement error. We also present a measure of reliability for the event means and examine the effect of the number of respondents on these spurious components.

To illustrate how these estimates are computed, forty-two respondents were asked to rate forty events on seven risk perception variables. Computing the variances and covariances for these variables based on event means resulted in relatively large components attributable to measurement error. A demonstration is given of how this error is removed and the resulting effect on our estimates.