Interim Evaluation of EACEA: detailed analysis of survey data and comments
In line with Council Regulation laying down the statute for Executive Agencies, the 3rd interim evaluation of the Education, Audiovisual and Culture Executive Agency (EACEA) was finalised by PPMI in June 2016. Among other data collection methods applied in the course of this external evaluation, the survey of unsuccessful applicants and beneficiaries of programmes managed by the Agency (Erasmus+, Creative Europe, Europe for Citizens and their predecessor programmes) was carried out. The purpose of the survey was to collect the first-hand information (in the form of quantitative and qualitative data) needed to assess the extent to which various organisational factors are making a positive/ negative impact on the performance of the Agency and to measure the satisfaction of beneficiaries and unsuccessful applicants with the services provided by the EACEA. Notably, the results of the survey were analysed only at more general level during this evaluation, i.e. without looking in detail how the quality of EACEA’s services has been perceived by unsuccessful applicants and beneficiaries of the different programmes and sub-programmes. Such in-depth analysis was beyond the scope of the evaluation. Therefore the purpose of the current assignment was to complement the results of the 3rd Interim Evaluation by further analysing the survey data and, respectively, improving the understanding of information available at programme level, as well as increasing its utility for improvement of the EACEA’s service quality.
The assignment was organised into two main tasks: the quantitative analysis of closed-ended survey questions and the qualitative analysis of answers to open-ended survey questions. In total, answers from 5 311 respondents were analysed by using the descriptive statistics. Cross-tabulation was also employed in order to identify any different trends among the respondents with different characteristics: unsuccessful applicants vs. beneficiaries; first-time applicants vs. repeated applicants. Quantitative data analysis was supplemented with the analysis of qualitative answers and comments using NVivo 10 software. As a result, Analytical Report was prepared outlying the main findings of the detailed survey analysis. Technical annexes were also prepared, containing all the data and charts to facilitate their reuse by the Client.
The detailed analysis of the quantitative and qualitative data collected through the survey will allow EACEA to better understand the information at programme level and to use it to improve the quality of the services it provides.