Hawthorne Effect - Interpretation and Criticism

Interpretation and Criticism

H. McIlvaine Parsons argues that in the studies where subjects received feedback on their work rates, the results should be considered biased by the feedback compared to the manipulation studies. He also argues that the rest periods involved possible learning effects, and the fear that the workers had about the intent of the studies may have biased the results.

Parsons defines the Hawthorne effect as "the confounding that occurs if experimenters fail to realise how the consequences of subjects' performance affect what subjects do" . His key argument is that in the studies where workers dropped their finished goods down chutes, the "girls" had access to the counters of their work rate.

It is possible that the illumination experiments were explained by a longitudinal learning effect. It is notable however that Parsons refuses to analyse the illumination experiments, on the grounds that they have not been properly published and so he cannot get at details, whereas he had extensive personal communication with Roethlisberger and Dickson.

But Mayo says it is to do with the fact that the workers felt better in the situation, because of the sympathy and interest of the observers. He does say that this experiment is about testing overall effect, not testing factors separately. He also discusses it not really as an experimenter effect but as a management effect: how management can make workers perform differently because they feel differently. A lot to do with feeling free, not feeling supervised but more in control as a group. The experimental manipulations were important in convincing the workers to feel this way: that conditions were really different. The experiment was repeated with similar effects on mica-splitting workers.

Richard E. Clark and Brenda M. Sugrue (1991, p. 333) in a review of educational research say that uncontrolled novelty effects cause on average 30% of a standard deviation (SD) rise (i.e. 50%-63% score rise), which decays to small level after 8 weeks. In more detail: 50% of a SD for up to 4 weeks; 30% of SD for 5–8 weeks; and 20% of SD for > 8 weeks, (which is < 1% of the variance).

Richard Nisbett of the University of Michigan has described the Hawthorne effect as 'a glorified anecdote,' saying that 'once you have got the anecdote, you can throw away the data.'"

Harry Braverman points out in Labor and Monopoly Capital: The Degradation of Work in the Twentieth Century that the Hawthorne tests were based on industrial psychology and were investigating whether workers' performance could be predicted by pre-hire testing. The Hawthorne study showed "that the performance of workers had little relation to ability and in fact often bore an inverse relation to test scores...". Braverman argues that the studies really showed that the workplace was not "a system of bureaucratic formal organisation on the Weberian model, nor a system of informal group relations, as in the interpretation of Mayo and his followers but rather a system of power, of class antagonisms". This discovery was a blow to those hoping to apply the behavioral sciences to manipulate workers in the interest of management.

The Hawthorne effect has been well established in the empirical literature beyond the original studies. The output ("dependent") variables were human work, and the educational effects can be expected to be similar (but it is not so obvious that medical effects would be). The experiments stand as a warning about simple experiments on human participants viewed as if they were only material systems. There is less certainty about the nature of the surprise factor, other than it certainly depended on the mental states of the participants: their knowledge, beliefs, etc.

Research on the demand effect also suggests that people might take on pleasing the experimenter as a goal, at least if it does not conflict with any other motive, but also, improving their performance by improving their skill will be dependent on getting feedback on their performance, and an experiment may give them this for the first time. So you often will not see any Hawthorne effect—only when it turns out that with the attention came either usable feedback or a change in motivation.

Adair (1984) warns of gross factual inaccuracy in most secondary publications on Hawthorne effect and that many studies failed to find it. He argues that it should be viewed as a variant of Orne's (1973) experimental demand effect. So for Adair, the issue is that an experimental effect depends on the participants' interpretation of the situation; this is why manipulation checks are important in social sciences experiments. So he thinks it is not awareness per se, nor special attention per se, but participants' interpretation that must be investigated in order to discover if/how the experimental conditions interact with the participants' goals. This can affect whether participants believe something, if they act on it or do not see it as in their interest, etc.

Rosenthal and Jacobson (1992) ch.11 also reviews and discusses the Hawthorne effect.

In a 2011 paper, economists Steven Levitt and John A. List claim that in the illumination experiments the variance in productivity is partly accounted for by other factors such as the weekly cycle of work or the seasonal temperature, and so the original conclusions were overstated. If so, this confirms the analysis of SRG Jones's 1992 article examining the relay experiments.