Course

ISTQB Certified Tester - AI Testing (ISTQB CT-AI)

In this certification course, you will gain a fundamental understanding of Artificial Intelligence, how to test and quality assure AI-based solutions, and how machine learning systems can be tested throughout the entire development lifecycle.

The course has been updated to ISTQB® Certified Tester AI Testing v2.0, which replaces the previous CT-AI v1.0 syllabus. The new syllabus has a clearer focus on testing AI-based systems, particularly machine learning, and now also includes key concepts and testing approaches within generative AI and large language models.

Learning objective

During the course, you will gain a fundamental and broad understanding of Artificial Intelligence, machine learning, and testing AI-based systems. In addition, you will gain the following benefits:

You will be able to contribute to the test strategy for AI-based systems and machine learning systems
You will be able to contribute to the design and execution of test cases for AI-based systems
You will gain an understanding of challenges related to testing AI-based systems, such as non-deterministic behavior, data dependency, probabilistic results, bias, transparency, robustness, and models that evolve over time
You will gain an understanding of generative AI and testing large language models, including exploratory testing and red teaming
You will gain an understanding of input data testing, including data quality, data representativeness, bias, data pipelines, and dataset validation
You will learn about testing machine learning models, including ML performance metrics, adversarial testing, metamorphic testing, drift testing, A/B testing, and back-to-back testing
You will gain an understanding of testing in relation to the development, deployment, and operation of machine learning systems

Target audience

ISTQB Certified Tester AI Testing v2.0 is for anyone with an interest in Artificial Intelligence who wants a fundamental understanding of how AI-based systems and machine learning systems are tested and quality assured.

The course is particularly relevant for testers, test managers, test analysts, test engineers, software developers, data analysts, project managers, quality assurance professionals, business analysts, and others who work with or need to understand the quality of AI-based solutions.

Prerequisites

To take the exam, you must hold an ISTQB Foundation ISTQB CT-FL certification.

To participate in the course, it is beneficial to have a basic understanding of software testing, software development, or data-related roles. The following are not required, but prior knowledge of them is an advantage:

Software testing and quality assurance
Software development
Programming languages, e.g. Java, Python, or R
Statistics or data analysis
Basic machine learning

Course and Exam format

Over 3 intensive days, you will be taught the topics covered by the exam. The course includes theoretical instruction, practical, hands-on exercises, and discussions, and there will generally be a high level of participant involvement.

In addition, you should expect approximately 10 hours of syllabus reading before the course begins, as well as 2 hours of homework each evening during the course.

Exam format:

The exam is an official ISTQB CT-AI v2.0 exam. It consists of 40 multiple-choice questions and lasts 60 minutes. To pass, you must achieve 29 out of 44 points. K3 (apply) questions are worth 2 points; K1/K2 are worth 1 point. If English is not your native language, you can receive an additional 15 minutes for the exam.

In-House training?

If you are more than 5 people from same organisation, it can be beneficial to consider the course as in-house training. We conduct the course exclusively for your employees, either as standard as described or tailored to your needs.

Advantages of in-house training

Financial savings for more than 5 people
Intensive exchange of experiences and knowledge sharing
Employees gain a common understanding of the subject
Opportunity for unique customization based on your own methods and processes

Contact Us

In-house training or questions?

If you are more than 5 people or just need some help contact us at info@triforkqi.com

Contact

Course content

1. Introduction to Artificial Intelligence (AI)

Including the difference between AI-based and conventional systems, narrow AI, general AI, super AI, generative AI, ML frameworks, hosting, hardware, and relevant standards and regulations including EU AI act.

2. Quality characteristics for AI-based systems

Including AI-specific quality characteristics, functional correctness, adaptability, transparency, robustness, controllability, safety, ethics, and acceptance criteria.

3. Machine Learning (ML)

Including supervised, unsupervised, and reinforcement learning, the ML workflow, data preparation, datasets, pretrained models, fine-tuning, retrieval-augmented generation, ML performance metrics, and neural networks.

4. Testing AI-based systems

Including challenges in testing AI, locked and adaptive AI systems, statistical testing, test oracles, risk-based testing, test levels, and testing generative AI and large language models.

5. Input Data Testing for Machine Learning Systems

Including data quality, data-related risks, bias, data representativeness, data pipeline testing, dataset validation, and label correctness.

6. Model Testing for Machine Learning Systems

Including ML model risks, performance testing, adversarial testing, metamorphic testing, overfitting, underfitting, drift testing, A/B testing, and back-to-back testing.

7. Machine Learning Development Testing

Including testing during development and deployment, as well as monitoring and maintaining model performance in production.

Introduction

Meet the trainer

Kari Kakkonen

Meet Kari Kakkonen, Trifork QI’s Trainer. Kari Kakkonen has worked in software testing for almost 30 years, currently Service Owner of Customer Expertise Development at Gofore. He has trained over 4500 people over the years all around Europe. Kari delivers training for many software testing, agile, DevOps, and AI topics, especially for ISTQB AI Testing and ISTQB Testing with GenAI. Kari has an M.Sc. in Industrial Management from Aalto University and tens of certificates on testing, agile, and DevOps. Kari is an active board-level volunteer at ISTQB, FiSTB, and TMMi.

Viepul Kocher

Meet Vipul Kocher, Trifork QI’s Trainer. Vipul is a former Adobe engineer and IIT alumnus with more than 30 years of experience in software development and testing. He is the CEO and co-founder of AIEnsured/testAIng and Verity Software, companies focused on AI testing and software testing education. As the creator of AiU – the world’s first AI testing certification – and DoU – the world’s first DevOps and testing certification – he is a key figure in the global testing industry. He also serves as President of the Indian ISTQB Board and Convener of the STeP-IN Forum, Asia’s largest testing community. Vipul developed Q-Patterns, an award-winning testing methodology, and is actively involved in mentoring and investing in startups.