Llm Evaluation Part2 Evaluation Mechanisms

LLM evaluation part2. Mechanics behind LLM scoring systems

Upstage Studio — Build and deploy your agent. Learn more → Upstage Studio — Build and deploy your agent. Learn more →

Solutions

Resources

Company

Pricing

Try now Try demo

Contact us

en

Introduction Why do evaluations matter? Since the end of 2022, we all have witnessed a surge of new Large Language Models (LLMs). These models have opened up unprecedented opportunities, hinting at the potential to revolutionize industries across various sectors. However, a key challenge in utilizing LLMs lies in their probabilistic nature—the fact that identical prompts can lead to varied outputs. This variability raises a crucial question: how can we ensure the safety and quality of LLM-generated content for application use?  Plus, with new LLMs that emerge from here and there, it is getting harder for people to recognize which is the new big deal. So how do we know which one has the performance for my task? How are benchmark datasets used in this process? As introduced in Part 1, benchmark datasets act as the SATs for LLMs, providing a fixed, standardized method to evaluate the models' capabilities. These evaluations help us gauge the performance of different LLMs and facilitate their comparison. But how exactly do we calculate the scores? Are assessments limited to multiple-choice questions, or do they include essay-type queries as well? If so, how exactly do we score these models, and what is the answer sheet? Moreover, how do we rank these models on the leaderboard? Scoring the model output Automated grading for multiple choice questions A prime example of evaluating LLMs through automated grading is seen in the Massive Multitask Language Understanding (MMLU) benchmark. MMLU is designed to assess the general understanding of LLMs across 57 diverse tasks that span from STEM to social sciences. These tasks test the models' understanding and adaptability across a wide range of subjects and difficulty levels. Being a multiple-choice test, MMLU offers a straightforward approach to benchmarking compared to tests with open-ended questions. The models subject to evaluation are given text strings (i.e., "prompts"), which is broken down into tokens (words, sub-words, or characters, based on the model's design) and input into the model. The model then predicts a probability distribution for the next token from its vocabulary, enabling the selection of the most likely continuation . This process can be repeated, adding tokens to the prompt and generating subsequent tokens, until a complete sentence or series of sentences is formed. For each question, only one of the provided answers is the correct one. Here is an example:

A sample data from MMLU dataset After prompting the model to generate outputs, there are two primary methods for extracting and evaluating information from a model: Probability Comparison: Assess the probabilities that specific groups of tokens are logical continuations of the given prompt, and then compare these probabilities against predefined choices. Text Generation Comparison: Generate text from the model by selecting tokens iteratively, as described, and compare these generated texts with the texts of various predefined choices.

What about free-form response questions? Automated grading for free-form response questions Manual evaluation of the numerous LLMs and their detailed outputs is simply impossible and impractical. A promising automated solution is MT-Bench, which leverages GPT-4, a high-performance LLM, as an adjudicator to compare responses from various models. This approach has proven effective, with GPT-4 aligning over 80% with both controlled and crowdsourced human judgments, mirroring the agreement level among humans themselves. This automation of grading free-form response questions enables a more practical evaluation, emphasizing models' capacity to deliver long text responses rather than answering multiple choice questions, aligning closely with real-world user requests.

Source : Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Human evaluation Human evaluation plays a crucial role in assessing the outputs of applications before they go live. This process often involves the use of human annotators who assess the application's responses using a specially constructed test dataset. Evaluation techniques include scoring the answers, conducting A/B testing, and comparing responses to a 'golden set' of correct answers. Additionally, post-deployment performance can be monitored through real-world feedback mechanisms, such as analyzing the ratio of thumbs-up responses from users. This approach provides insight into how well the model meets user expectations in actual usage scenarios. Ranking models in the leaderboard The ranking of models on leaderboards is typically based on the average scores obtained from various benchmark datasets, referred to as total scores (e.g., H6, H4, etc.). This approach calculates a straightforward mechanical average, treating all benchmarks equally without giving undue weight to any single evaluation criterion. Current limitations in evaluation methods overall Despite the structured approach to evaluating and ranking LLMs, several limitations become apparent when these models are applied to real-world contexts: Outdated Data : Evaluation benchmarks can quickly become outdated as the landscape of data is ever-changing, with new information generated daily. However, leaderboards are often snapshots, fixed in time and unable to reflect the ongoing evolution of data, rendering them less relevant as time progresses. Failure to Reflect Real-World Complexity: The true value of a model lies in its ability to adapt and respond effectively to real-world challenges. This involves assessing LLMs on their capacity to produce reliable outputs according to the given format, which is their function-calling abilities. Additionally, it's crucial to evaluate how these models safeguard against users' attempts at jailbreaking, such as prompting the generation of harmful content or extracting private information from training data. Moreover, we need to check how the model navigates through the delicate balance between offering helpful responses and avoiding politically biased answers in accordance with their policies. Questionable Meaningfulness of Competition : There is a risk that models are overly optimized for specific test sets, leading...