Evaluation use is a complex and multifaceted construct comprising domains of findings and process use and sub-categories of instrumental use, conceptual use, and persuasive (legitimative/symbolic) use. It is an important aspect of evaluation practice which has received much attention from evaluation scholars and practitioners. However, evaluation and evaluation use in countries outside North America, Europe and Australasia is under-investigated. This study proposes to make a modest contribution to understanding evaluation practices in an Asian context through investigating the value and utility of program evaluation as perceived by officials in selected Singapore social sector ministries. Singapore is a democratic country but has retained the characteristics of a highly centralized state. In addition, Confucian values are very important for the political leaders and Singapore society. A modified explanatory sequential mixed methods design was used to address the research questions. Both quantitative and qualitative data were collected through a questionnaire with both closed-ended and open-ended questions. The questionnaire was followed up with interviews to provide additional depth and explanations related to the results from the questionnaire.
The questionnaire, based on current theories and the applicability of these evaluation theories to Singapore, was sent through email to officers in two social sector ministries in Singapore - Ministry of Culture, Community and Youth (MCCY) and Ministry of Education (MOE), and 120 valid responses were obtained. Results were analyzed by examining the internal structure and bifactor model underlying the questionnaire. The hypothesized bifactor model comprised a general factor, Perceptions of Evaluation, and two sub-factors – Value of Program Evaluation and Purposes of Program Evaluation. The score on Perceptions of Program Evaluation was regressed on respondents’ background variables to investigate factors that affected their perceptions. Qualitative findings from the open-ended questions and the interviews were analyzed separately and integrated with the quantitative results.
The questionnaire functioned well for the respondents, indicating the applicability of existing scholarship on program evaluation to the Singapore context. However, there were a few items that were not respond to as hypothesized. Unique aspects of the context in Singapore social sector ministries might have been the reason for the responses to these items. In addition, the results suggested that a modified bifactor model (with purposes split into internal and external to the program) would fit the data better.
On the whole, respondents perceived program evaluation positively (Mean = 1.58, SD = 0.52 on a scale from minus three to three with zero as the mid-point) and emphasized the value of program evaluation for learning and improvement. However, respondents seemed to have reservations with going through the evaluation process. Factors influencing the respondents’ perceptions were layered, and included such factors as officers’ characteristics and experiences, bosses, evaluators, corporate values, organizational structure, societal values and political climate. Officers’ experiences with program evaluation emerged as an important factor, particularly experiences related to benefitting from program evaluation. Given the Confucianism based, highly centralized Singapore state, there might be gaps in current literature to inform and understand the evaluation practices within this and similar contexts. Further studies would be needed to investigate the extent of influence and the mechanisms through which the factors affect perceptions of program evaluation in the Singapore context.