Once upon a time, client-agency evaluations were merely ‘nice to have’. But in recent years, structural changes in the industry have caused the evaluation process to become business critical.
Apart from digitisation, and the fragmentation of the industry, one of the most fundamental changes to the marketing industry in recent years has been the increased involvement of marketing procurement professionals in negotiating these high-value contracts. They are used to using robust quantitative data to back up discretionary payments.
Today’s marketers and their communications partners need an accurate and reliable basis more than ever on which to determine bonuses, some of which run into hundreds of thousands of dollars.
Quantitative evaluations need to reflect the specific scope of work of each partner and could change annually. Equally, qualitative interviews alone do not provide a reliable, sustainable, and consistent basis on which to measure, monitor, and improve team performance.
Over the past 20 years, Aprais has conducted more than 21,000 evaluations of client-agency relationships globally and has considerable experience in this field.
The four stages of evaluation
In setting out to build the best client-agency evaluation system, one needs to understand that there are four stages to the process.
Quality in, quality out
Before contemplating the requirements for and evaluation methodology, you should decide what you want out of the evaluation and how you are going to use the results.
Here are seven basic tenets of good evaluations you need to consider;
Building the best client-agency evaluation system
Over the years, we’ve been asked every imaginable question about the process of evaluating and improving team performance.
Here are some of the most frequently asked questions and our point of view based on more than 20 years’ experience.
Q: How important is regularity when conducting client-agency evaluations?
A: This should be a regular process and ideally not one which is not specifically timed to coincide with any particular financial review. Having a regular schedule allows like-for-like tracking over time and helps identify any significant performance changes in a timely manner.
Q: How frequently should we conduct client-agency evaluations?
A: In a time-scarce business world where the focus should be more on generating and executing business-building ideas, the evaluation schedule should be appropriate and effective. Experience has shown that annual reviews are simply not frequent enough. The impetus to implement agreed action plans can be lost and new issues often occur that need more immediate action. We recommend evaluations be conducted every 6 months to encourage more effective and timely action planning.
Q: Who should participate in client-agency evaluations?
A: We strongly recommend that participants should be selected across the business, representing the view of all team members who work regularly on the relationship. Relationship problems can arise at any level in the organization, at any point in the client-agency interface and across multiple departments. For this reason, it is a mistake to limit the participants to only the most senior people on both sides of the relationship. Feedback from junior and senior team members on both sides and across multiple departments can provide an early warning of issues that may become bigger problems going forward and threaten healthy relationships.
To ensure an informed opinion, we advise that any participant should have had a minimum of three months involvement with the relationship.
Q: What is the ideal scoring scale for team performance evaluations?
A: There has been much debate about the best scale to use for evaluations. Aprais has always believed a finer scale, i.e. 1-100, for the same reason that most research is calibrated in percentages.
Apart from the finesse of the scores, there is another practical reason for a 100-point system. A 5-point scale means that a behaviour must improve or decline 20% in order to register on the scale. Similarly, a 10-point scale requires a 10% change to be reflected.
Our experience shows that, over time, the changes in perceived performance (i.e. the scores) may narrow compared to the initial stages of a relationship and movement can be a couple of percentage points in either direction. A 100-point scale provides greater ability to identify these subtle shifts and to act upon them if they head in the wrong direction.
Q: Should participants be identified or anonymous?
A: Some cultures find it difficult to be openly critical and individuals in all markets need to have the confidence to speak openly and honestly about issues they experience in the relationship. For this reason, Aprais has a firm policy not to reveal the scores or commentary of any participants without their express permission. In this way we can be sure to gather rich feedback. As a result, a minimum of three respondents per party is also recommended.
Q: Why are two-way evaluations better than one-way?
A: Some marketers believe that one-way evaluation of their agency is appropriate because ‘they are paying for the service’. Whilst undoubtedly true, surely the goal should be to obtain the greatest possible value from the partnership.
We have accurate statistical proof that there is a direct and positive correlation between the rating a client gives the agency and vice versa. Input from the communications partner helps identify processes or actions on the client side that might be impacting on their ability to respond to the briefs with quality work. Two-way evaluations (what we call 180°) provide this transparency.
Q: What is a four-way evaluation and are they better?
A: In any relationship, if one partner rates themselves significantly better than the point of view of the other, issues are on the horizon, particularly if only one of them thinks they are doing a great job! In Aprais terms, perception gaps refer to the difference in scores between how a team evaluates itself compared with how the other party evaluates their performance. This can be achieved through a four-way evaluation (what we call 360º). The main benefit of this method is to pinpoint where expectations and differences exist between the client and the agency especially in relationships that need more forensic investigation.
Q: Which is best for team evaluations, quantitative or qualitative input?
A: This is a trick question as both have great value. However, with potentially large amounts of money riding on the output of evaluations, it is critical to capture and monitor accurate quantitative scores. To ensure richer learnings from evaluations, it is also important to gather comments to put the scores into context. At Aprais, we encourage participants to leave anonymous freeform commentary rather than the more time-consuming & significantly more expensive route of face-to-face interviews
Q: Managing compliance and completion of team evaluations
A: As with any ‘survey’, the results are only as good as the sample and their input. Our experience shows that to encourage people to stop their normal work to complete an evaluation often requires considerable prompting.
Initially, the team leaders or project sponsors of the marketer and agency teams need to impress upon their respective participants the importance of timely compliance. Once teams and their leaders see evidence of the real value delivered through the process of evaluations, they become more enthusiastic to complete the evaluation.
In addition, regular bespoke reminder emails are sent to those participants yet to complete the evaluations at regular intervals whilst the questionnaires are live.
Q: How important is the user experience in evaluation systems?
A: Participants are exposed to cutting edge technology in the course of their daily lives so evaluations should be run on up-to-date platforms and designed to ensure the participant and administration experience is simple and painless.
Q: How to turn the output data into action?
A: In a word, ‘insights’. Data must be presented in such a way that satisfies the needs of all stakeholders and identifies opportunities for improvement or best practice sharing, be that a global snapshot or local identification of key issues.
Aprais has multiple options across different platforms for the presentation of the results, many of them bespoke to specific client needs. These range from individual relationship reports to fully aggregated data across brands, geographies and business teams.
Nevertheless, the data will be meaningless unless it is translated into actions with clear responsibilities and timelines.
Q: If our agency is ‘in-house’ do we still need to evaluate them?
A: Absolutely! Our data proves that a formalised process of evaluation encourages a more open dialogue leading to improved ratings for both teams, irrespective of the agency ownership. In fact, relationships between in-house teams can often require more discussions on the working relationship.
What are you waiting for?
Whether a short-term project-based agency assignment or performance-based bonus global assignment, there are many reasons to evaluate team performance, and very few reasons not to.
Regardless of scope and scale, to be effective, evaluations should follow these core tenets; Relevant, Efficient, Manageable, Qual/Quant, Actionable, Anonymised and Impartial.
Through regular evaluation, companies can get more value from their marketing relationships. Scores for both parties usually improve with a consistent evaluation and action planning program and appreciation of the other party grows over time. The top 10% client performers in the Aprais database, for example, score their agencies 37% higher than the bottom 10%.
These clients also acknowledge the need to amend questionnaires when required to ensure they remain relevant to marketing teams and their communication partners.
We welcome your questions about your current evaluation process and are here to help.
Kim Walker, Founder and chairman at Aprais.
