Evaluation Journal of Australasia

The Australian Government has recently increased resourcing for evaluation of Indigenous programmes following critical reports by the Australian National Audit Office and Productivity Commission around their failure to significantly reduce Indigenous disadvantage. Evaluation in Indigenous affairs has a long history, although not a consistent or coordinated one. While there is significant knowledge held by those with experience in commissioning and conducting evaluations for Indigenous programmes over a number of decades that could usefully inform current efforts, there has been little research focused on this area. This article outlines the findings of qualitative research about evaluation in Indigenous policy conducted with policymakers, senior public servants, programme managers, researchers and independent evaluation consultants that sought to privilege the voices and perspectives of Aboriginal and Torres Strait Islander people. It outlines key themes derived from those interviews relating to the methods, parameters, politics and accountability around government-commissioned evaluation in Indigenous policy and programmes and concludes by canvassing ways that evaluation could better serve the interests of Aboriginal and Torres Strait Islander peoples.

This article examines the relationship between systems thinking concepts and the logic model. Two notable shortcomings of the logic model are illustrated: depicting the program theory linearly and failing to place the program in context. Both issues lead to an artificial depiction of reality making evaluation findings difficult to interpret. Systems thinking concepts are defined and how they address these shortcomings is discussed. This article then demonstrates how many systems thinking concepts are evident in the logic model; although in a limited way. This article concludes by noting the importance of using system concepts when answering evaluation questions related to system dynamics and interrelationships.

This study explores the functions of professional evaluators outlined in online job advertisements. A total of 97 job advertisements were reviewed in the study. A content analysis using a Coding Analysis Toolkit developed by Shulman was conducted to identify six main evaluation functions based on the collected data. These functions are (1) evaluation and reporting, (2) providing evaluation advice, (3) evaluation capacity building, (4) communication and engagement, (5) forming partnerships and (6) leading, managing and influencing. These functions were then compared to the Australian Evaluation Society’s (AES) Core Competency Domains. Overall, there is a broad alignment between these functions and the AES Core Competency Domains. However, the analysis shows that the delivery of culturally competent evaluations and evaluation utilisation received no mention in advertised evaluation roles. The delivery of culturally competent evaluation is essential from the perspective of ethics, validity and theory, while the utilisation of evaluation findings is important for the benefit of society.