Evaluation Journal of Australasia

Evaluators often regard monitoring as playing a secondary and relatively simple role compared to evaluation proper. This paper argues that this view underestimates the potential of monitoring information in enhancing the value of evaluative work, in particular to increase the ‘half-life’ of evaluation findings. Moreover, it suggests the possibility of a more dynamic interchange between monitoring and evaluation. Specifically, monitoring complements the fragmented and ad-hoc nature of evaluation work, so that the process of monitoring presents an opportunity to develop a framework within which individual evaluations can exist. Drawing on program logic and CMO theory, the key to this interchange is to build of systematic body of knowledge and theory that drives, and is in turn informed by, evaluation and monitoring. The work within the Department of Work and Income New Zealand provides an example of an attempt at implementing such a framework.

There are growing pressures in countries throughout the world to improve the performance of their public sectors. One strategy now being employed in both developed and developing countries is to design and construct performance-based monitoring and evaluation (M&E) systems so as to be able to track the results produced (or not) by governments. This paper describes a 10-step approach to the design and construction of such M&E systems that is currently being deployed in a number of developing countries. The overall strategy builds on the experiences of developed countries – especially those in the OECD – but also reflects the particular challenges and difficulties faced by developing countries that can range from a lack of skill capacity to poor governance structures to systems that are far from transparent. The last section of the paper addresses in detail the first of the 10 steps – that of assessing whether a country is, in fact, ready to begin the process of M&E system design and construction.

This paper examines ways in which internal data can be used in the evaluation research through data matching. It illustrates this through two case studies involving a three-year longitudinal and a cross-sectional evaluation of two large-scale Australian assistance programs. Discussion centres around ways in which internal data were used to assist the interview process, identify areas where ‘claimed’ (what the respondent says) and ‘actual’ (what the internal data show) data differ, and identify non-response issues.

The paper also examines how internal data can be used as an intermediate platform against which the results of two independent surveys can be reconciled. This is useful as many surveys do not use the same measurement scales or cover the same topics. The use of internal data can provide some commonality between the two, again adding value in the overall evaluation process and allowing the organisation to better utilise existing data.

The 2001 AES Awards for excellence were presented at a formal session within the conference program. The standard of the nominations was very high and the competition was intense. Most nominees met the evaluation criteria and there was a wide coverage of many contemporary issues facing evaluators in Australasia - from Aboriginal issues, through to agriculture, crime prevention, industrial relations, disability services, local management in schools and health service reform.