ISTQB® Certified Tester AI Testing (CT-AI)

Module 1: artificial intelligence fundamentals

• Definitions and categories: narrow AI, general AI and super AI.
• AI systems versus conventional systems.

Practical exercises

• Identify AI systems and map AI components.

Module 2: AI system quality

• Flexibility, adaptability and autonomy.
• Managing bias, ethics and transparency.

Practical exercises 

• Analyse real examples of AI bias.
• Create an ethical testing scenario.

Module 3: key machine learning concepts

• Supervised, unsupervised and reinforcement learning.
• ML workflows, overfitting and underfitting.

Practical exercises 

• Experiment with overfitting and select ML algorithms.

Module 4: data in ML

• Data preparation, quality and labelling.
• Training, test and validation datasets.

Practical exercises 

• Prepare datasets and identify bias within them.

Module 5: performance metrics

• Confusion matrices and classification/regression metrics.
• Metric limitations and selection.

Practical exercises 

• Evaluate an ML model using appropriate metrics.

Module 6: neural networks and testing

• How neural networks work.
• Network coverage and perceptron implementation.

Practical exercises 

• Implement a simple network and analyse coverage.

Module 7: testing AI systems

• AI-specific test levels.
• Test data and automation bias.

Practical exercises

• Plan multi-level tests and include AI documentation.

Module 8: testing AI-specific quality characteristics

• Autonomous systems and algorithmic bias.
• Transparency, explainability and test oracles.

Practical exercises 

• Validate an explainable model and define test oracles.

Module 9: AI testing methods

• Adversarial attacks, pairwise testing and metamorphic testing.
• Exploratory testing and A/B testing.

Practical exercises

• Derive metamorphic test cases and simulate adversarial attacks.

Module 10: AI test environments

• Real and virtual environments for AI systems.

Practical exercises

• Simulate an AI test environment.

Module 11: using AI for testing

• AI for test case generation and defect prediction.
• Test suite optimisation.

Practical exercises

• Use AI tools to automate tests and create a defect prediction model.

Module 12: preparation for the ISTQB® CT-AI exam

• Review of key concepts.
• Mock exam and individual feedback.