Lesson 3: Output Evaluation, Non-Functional Quality, and Safety

This lesson develops practical understanding of output evaluation, non-functional quality, and safety within the ISTQB Certified Generative AI Testing (CT-GenAI) Training. It connects core concepts with realistic testing and quality-engineering decisions.

Learners examine methods, evidence, risks, stakeholder expectations, and implementation considerations before applying the ideas in a focused activity.

Topics Covered

• Correctness, relevance, completeness, consistency, coherence, and grounding
• Human evaluation, rubrics, pairwise comparison, and expert review
• Automated metrics, model-based evaluation, rule checks, and limitations
• Hallucination, bias, harmful content, toxicity, and misinformation
• Prompt injection, data leakage, privilege abuse, and indirect attacks
• Latency, throughput, cost, scalability, availability, and resilience
• Guardrails, moderation, refusal, escalation, and human oversight

Practical Activity

Apply the lesson concepts to a realistic ISTQB Certified Generative AI Testing (CT-GenAI) scenario and document the test approach, evidence, risks, and recommended next actions.

Lesson Outcome

Learners can explain and apply output evaluation, non-functional quality, and safety using structured, risk-aware, and evidence-based testing decisions.