Statistics New Zealand - Methods

Methods

Statistics New Zealand uses many methods to collect, process, analyze, and publish data. Statistics can get complex. The agency has statisticians and mathematicians and survey experts whose chief duty is to examine how the agency approaches various projects and to suggest ways to improve efficiency, speed, and accuracy.

Some projects require advanced statistical tools to make sense of misleading data. For example, sometimes seasonality distorts data. "For example, in retail trade, December is a very high month due to Christmas sales. If we compare November sales with December sales, we would report an increase in sales. But this increase is largely due to seasonal fluctuations and is not an informative measure" according to a description from their website. To prevent seasonal variables from distorting data, statisticians sometimes use complex programs, including X-12-ARIMA (derived from census bureaus from the US and Canada), to remove seasonal distortions. So, with the December sales bump removed, the seasonally-adjusted retail sales data can reveal the true underlying trend -- are sales going up, holding steady, or declining? This is important information for retailers.

While the census counts every person, the department also conducts surveys based on only a small sample of people. These samples are scientifically selected to represent the larger group. For example, the General Social Survey chooses a select number of households, approximately 8,000, which represent the nation as a whole, and will survey them every two years; the first results of this survey will be available in October 2009. Interviewers with laptops conduct face-to-face interviews, and the result is information about "social wellbeing" including "housing, health, paid work, social connectedness, and human rights". It sometimes conducts specialty surveys about a specific industry or topic. For example, it published a biotechnology survey, based on a 10-page questionnaire developed by the agency, using scientific samples to choose respondents; the 2007 results can be downloaded to computers via the Internet. This particular report uses a long list of variables, such as "total exports" and "biotechnology exports to Asia" and "stage of development of biotechnology processes".

Statistical techniques such as sampling and weighting can reduce data gathering expenses while surveying, although it requires careful attention by statisticians. For example, census counts are rather straightforward in a statistical sense since it's a straight count of bodies, purchases, opinions, actions. But it is expensive to interview everybody. And there are many non-census topics requiring further study which the agency is asked to do. One way to reduce costs is by selecting smaller samples which hopefully represent the population being studied. This is a highly cost-effective way to get accurate information. Choosing a good sample sometimes requires complex statistical work to make sure the sample is, indeed, truly representative of the population under study. If conditions do not permit representative sampling but known benchmark statistics are known (possibly from earlier census data), it's still possible to generate accurate information by weighting the data to distort it back, so to speak, to compensate for the distortion caused by the unrepresentative sample. For example, suppose an area of Wellington has a known percentage of women -- say 53% -- which is believed to be accurate from a previous census count; a study is done a few years later; a sample is chosen; questionnaires are processed; but of the returning questionnaires, 57% are women. It's possible statistically to give men slightly greater weight to account for the discrepancy, and the result is more accurate data. The agency discusses different methodologies on its website. A statement explains why weighting is sometimes necessary: "Statistics New Zealand is improving the methodology used for the 2001 Household Economic Survey through the use of integrated weighting. This is a relatively recently developed method of adjusting the statistical output of a survey to match population benchmarks. In particular, it takes account of undercoverage in the survey of specified population groups. Integrated weighting improves the robustness and accuracy of survey estimates. It also reduces the effect of bias in estimates resulting from undercoverage, as well as reducing the level of sampling error for benchmark variables."

All statistical work presents opportunities for error, but it's possible to reduce error to manageable amounts. A statement reads: "An important aim of our ongoing work is to understand, manage, control and report on all known sources of error... which simply reflect the inherent variability that exists among the units we are seeking to measure... This variability manifests itself in sampling error when we use samples for cost-effectiveness reasons to estimate characteristics about a population. Sampling errors are relatively easy to measure.... Other sources reflect process, measurement and inference errors, and are referred to as non-sampling error. It is not possible to eliminate all sources of error. However, our continued efforts at understanding and managing variability and error ensure we are exercising a high level of control on all known sources of error."

What greatly facilitates statistical accuracy by reducing non-response bias is that answering questions is sometimes required by law. Some surveys must be answered. This requirement lessens agency expense, practically eliminates non-response bias (a perennial headache for most private market researchers), and improves accuracy. For example, for the General Social Survey, people are selected randomly for 10-minute phone interviews which sometimes ask personal questions such as wage and salary information. But people must answer these questions and won't be paid for their time. It's required by the Statistics Act 1975. This policy is substantially different from countries such as the United States, where answering official surveys is rarely compulsory, including responding to its census which is done every ten years. Further, if a person doesn't get a census survey for any reason, he or she is still obligated to visit a local census bureau, get one, fill it out, and return it; the government can't be held accountable for mixing up an address. Questionnaires can be downloaded from the Internet via the Statistics New Zealand website. Legislation states census forms must be "filled in and signed within a stated time" and returned "within a stated time to the Department". Mutilation or defacement of schedules can be punished. But fines for non-compliance rarely exceed $500.

Extensive use is made of the Internet to publish key information. Users can sift through vast databases to learn about a wide variety of topics. A user can learn that "89% of businesses use broadband to connect to the Internet" or that "145 babies were born on Mother's Day in 2008. Entire reports can be downloaded as .pdf files and printed on one's computer.