Tests with Ragas metrics
When evaluating LLM projects, you can leverage any of the Ragas metrics to create detailed tests. Each test provides:- A pass/fail status.
- Row-by-row scoring and justification, provided by the LLM evaluator.
Metrics available
The Ragas metrics available on Openlayer listed below.| Metric | Description | measurement for the tests.json |
|---|---|---|
| Answer relevancy | Measures how relevant the answer (output) is given the question. Based on the Ragas response relevancy. | answerRelevancy |
| Answer correctness | Compares and evaluates the factual accuracy of the generated response with respect to the reference. Based on the Ragas factual correctness. | answerCorrectness |
| Context precision | Measures how relevant the context retrieved is given the question. Based on the Ragas context precision. | contextRelevancy |
| Context recall | Measures the ability of the retriever to retrieve all necessary context for the question. Based on the Ragas context recall. | contextRecall |
| Correctness | Correctness of the answer. Based on the Ragas aspect critique for correctness. | correctness |
| Harmfulness | Harmfulness of the answer. Based on the Ragas aspect critique for harmfulness. | harmfulness |
| Coherence | Coherence of the answer. Based on the Ragas aspect critique for coherence. | coherence |
| Conciseness | Conciseness of the answer. Based on the Ragas aspect critique for conciseness. | conciseness |
| Maliciousness | Maliciousness of the answer. Based on the Ragas aspect critique for maliciousness. | maliciousness |
| Faithfulness | Measures the factual consistency of the generated answer against the given context. Based on the Ragas faithfulness. | faithfulness |