EVALUATING THE PERFORMANCE OF PARTNERSHIPS
By Judy Wilson
An increasing number of public services are being contracted out and, as a result, there is an stronger emphasis on value for money and ensuring service targets are being met. The need to assess delivery partners and frontline services is becoming an ever-pressing issue.
This feature discusses how to get the best out of assessing sub- contracted public sector provision and what to do with it once it has been carried out.
To go down a route of evaluating services provided by a partner takes real commitment. There is often a fear amongst staff or the partner organization that the evaluation will unearth short-comings and lead to recrimination. The reality is usually the opposite. Evaluation often identifies strengths and areas that are being handled effectively, as well as leading to real improvements in service.
What should be evaluated?
The best evaluation should examine two key areas: Performance against objectives and front-line delivery. This can be done through a 360° evaluation that focuses on strategy, delivery, administration and results.
Two organisations to recently carry out a 360° evaluation are learndirect Advice and nextstep, an organisation that promotes learning and work to adults aged 20 and over. Referrals from learndirect Advice to nextstep should be a seamless process for people enquiring about further skills and qualifications, or a change in career. As a relatively new partnership, the two organisations wanted to evaluate the referral process between the organisations. Mystery shopping research, involving more than 450 mystery shops, was carried out by Marketry to measure service levels against a standards matrix, and to measure the effectiveness of internal processes.
The mystery shopping programme identified many strengths within the learndirect Advice helpline. Staff were highly rated, calls were handled promptly and effectively and the advice provided was good. Referral of callers to nextstep was a clear area for improvement and, as a result, processes have been put in place to enable better communication between learndirect and nextstep.
Evaluation can look into many aspects, including contracts, internal processes, reporting and delivery, as well as obtaining feedback from the communities being served. By carrying out some formal research, learndirect Advice and nextstep were able to take a pragmatic approach to service-levels, develop more effective partnerships and provide a more coherent service to callers to the helpline.
Using evaluation to your advantage
Once the evaluation has taken place, organisations can use the research in a constructive way, to improve processes and ensure effort is not wasted, by being put into things that are having no effect. Evaluation often highlights new areas for improvement and allows staff to be more effective in what they do. Sometimes, these improvements are made by adapting internal processes, or re-shuffling resources. On other occasions, more money may be required and the evaluation can provide a launch-pad from which to obtain further funding.
For funding organisations, evaluation is an essential tool for assessing the impact of funding programmes and assessing their funded partners. It can be a welcome opportunity for all concerned to demonstrate the good job that is being done against initial objectives and desired outcomes. This allows a strategy to be developed that builds on the success of past projects and partnerships.
Assessing the effectiveness of projects
Essex LSC commissioned Marketry to evaluate LIF Social Inclusion and Neighbourhood Renewal Funding to assess the effectiveness of projects funded and their impact in terms of achieving Essex LSC’s overall strategic objectives. The research focused on 80 projects and analysed the effectiveness of each project in terms of delivering what was promised, value for money and usefulness in achieving LSC objectives. The results showed that over 2,000 disadvantaged people had achieved qualifications and 99% said they had gained skills; a lot of these being essential life skills such as communication, confidence and basic skills; with all the projects showing encouraging results within hard-to-reach groups. Value for money was also good with an average cost per participant of £225.
To evaluate a partner’s performance, it is useful to include the delivery partner in the research, as well as the beneficiaries and the in-house team, all of which may have different viewpoints and can provide valuable real-life case studies. Essex LSC’s evaluation was a good platform for partners to express some of the areas they found difficult, and was an excellent two way communication process to inform improved working practices. In many instances providers of LIF funded projects have learnt important lessons as a result of the funding, have tightened up their own quality processes, and most felt that the projects they had worked on were fulfilling and had produced good results. Some projects were no longer running after three years, whilst others continued to be successful and had sourced funding from other sources.
Those that choose to carry out benchmarking studies, evaluation and mystery shopping research can often really understand what is happening on the ground and have very useful and constructive discussions with staff and providers about improving services and processes. The evaluation often acts as the catalyst for those discussions.
It is very rewarding and motivating for people involved in funding to see the difference they are making to people’s lives and know how successful they have been. Without any kind of measurement, how can managers assess whether what is being done is successful or not? Change is a way of life in public sector organisations. Research can help managers stay ahead of the game and keep staff interested and motivated rather than just waiting for decisions from above. Effective evaluation can be an essential stepping-stone towards change that is beneficial and which focuses on things that are within staff’s control.
Judy Wilson is managing director of Marketry, an evaluation and research consultancy, specialising in public sector projects.