Scoring and Partnership
It is imperative that everyone is on the same page, but not necessarily the same team. The QA team and the Supervisors should have the same goals, the same standards, and the same accountability. The only difference is that QA should be a neutral party; fair, unbias, by the book. The Supervisor, however, can take into account the personality and learning style of the agent to provide feedback and coaching in a way they may be more receptive to. QA identifies issues, Supervisors fix them. But the supervisor should be evaluating too. They should know the general performance level of each of their agents and the goals the individually have to improve their own performance. QA only is not enough and not seeing them as a joint venture with the Supervisors creates and "Us/Them" mentality that doesn't benefit anyone.
In terms of scoring, you need to ask the Operation what is important to them? What are their goals? Is there a vision/mission that every interaction should support? Also, what are the customer's expectations of you? Customers expect very different experiences when they call customer service for a auto part supplier vs. Disney. What do your customers expect and what do your agents need to do, at minimum, to meet those expectations. Further, if there are any other compliance (not necessarily legal) that should be included. Anything the must be done a specific way for a specific reason every single time. Measure what matters. Once you have your questions set, decided how much each question is worth based on the impact of how not doing it impacts the business and/or customer experience. Good luck!