LLM Evals

What Is LLM Evals?

Common frameworks: DeepEval, Ragas (RAG-specific), Promptfoo, OpenAI Evals, and HumanEval-style custom evals. Each eval runs the LLM against a fixed input set and scores outputs via heuristics, regex, or judge-LLM grading. Critical for catching regressions when you upgrade the model, change the prompt, or update retrieval.