Course

ISTQB Certified in AI Testing

This introductory certification course to Artificial Intelligence (AI), gives you a broad insight into the AI methods used to test AI-based solutions and how AI-based solutions can be used to test other IT-systems.

Learning objective

This fundamental course to AI, Artificial Intelligence, gives you a broad insight into the AI methods used to test AI-based solutions and how AI-based solutions can be used to test other IT-systems. Furthermore the following outcome:

Understand the current state and expected trends of AI
Experience the implementation and testing of a ML model and recognize where testers can best influence its quality
Understand the challenges associated with testing AI-Based systems, such as their self-learning capabilities, bias, ethics, complexity, non-determinism, transparency and explainability
Contribute to the test strategy for an AI-Based system
Design and execute test cases for AI-based systems
Recognize the special requirements for the test infrastructure to support the testing of AI-based systems
Understand how AI can be used to support software testing

Target audience

The ISTQB Certified in AI Testing aimed at people who are seeking to extend their understanding of artificial intelligence and/or deep (machine) learning, most specifically testing AI based systems and using AI to test.

Prerequisites

To achieve the CT-AI certification, candidates must hold the ISTQB® Certified Tester Foundation Level (CTFL) certificate, but now a prerequisite to attend the course.

Furthermore, we recommend that you have:

Knowledge and understanding of programming language – Java/Python/R
Knowledge and understanding of statistics
Experience with software development and testing

Course and Exam format

Over the course of 4 intensive days, you will be taught in the subjects included in the exam. The course contains both a theoretical review, practical exercises, and discussion. There will be a high degree of participant involvement.

Prior to course start, expect to prepare by reading approximately 10 hours from the curriculum. During the course you should expect 2 hours of homework every day after class.

Exam format:

The exam is an official ISTQB-exam. The 1-hour exam has 40 multiple choice questions, and no assistance is permitted. You must answer at least 65% of the questions correctly to pass. Participants that take the exam not in their spoken language, will receive additional 25% time, and will have 15 minutes more, or a total of 75 min.

In-House training?

If you are more than 5 people from same organisation, it can be beneficial to consider the course as in-house training. We conduct the course exclusively for your employees, either as standard as described or tailored to your needs.

Advantages of in-house training

Financial savings for more than 5 people
Intensive exchange of experiences and knowledge sharing
Employees gain a common understanding of the subject
Opportunity for unique customization based on your own methods and processes

Contact Us

In-house training or questions?

If you are more than 5 people or just need some help contact us at info@triforkqi.com

Contact

Course content

1. Introduction to AI

Definition of AI and AI Effect
Narrow, General and Super AI
AI-Based and Conventional Systems
AI Technologies
AI Development Frameworks
Hardware for AI-Based Systems
AI as a Service (AIaaS)
Pre-Trained Models
Standards, Regulations and AI

2. Quality Characteristics for AI-Based Systems

Flexibility and Adaptability
Autonomy
Evolution
Bias
Ethics
Side Effects and Reward Hacking
Transparency, Interpretability and Explainability
Safety and AI

3. Machine Learning (ML) – Overview

Forms of ML
ML Workflow
Selecting a Form of ML
Factors involved in ML Algorithm Selection
Overfitting and Underfitting

4. ML – Data

Data Preparation as part of the ML Workflow
Training, Validation and Test Datasets in the ML Workflow
Dataset Quality Issues
Data quality and its effect on the ML model
Data Labelling for Supervised Learning

5. ML Functional Performance Metrics

Confusion Matrix
Additional ML Functional Performance Metrics for Classification, Regression and Clustering
Limitations of ML Functional Performance Metrics
Selecting ML Functional Performance Metrics
Benchmark Suites for ML

6. ML – Neural Networks and Testing

Neural Networks
Coverage Measures for Neural Networks

7. Testing AI-Based Systems Overview

Specification of AI-Based Systems
Test Levels for AI-Based Systems
Test Data for Testing AI-Based Systems
Testing for Automation Bias in AI-Based Systems
Documenting an ML Model
Testing for Concept Drift
Selecting a Test Approach for an ML System

8. Testing AI-Specific Quality Characteristics

Challenges Testing Self-Learning Systems
Testing Autonomous AI-Based Systems
Testing for Algorithmic, Sample and Inappropriate Bias
Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems
Challenges Testing Complex AI-based Systems
Testing the Transparency, Interpretability and Explainability of AI-based Systems
Test Oracles for AI-Based Systems
Test Objectives and Acceptance Criteria

9. Methods and Techniques for the Testing of AI-Based Systems

Adversarial Attacks and Data Poisoning
Pairwise Testing
Back-to-Back Testing
A/B Testing
Metamorphic Testing
Experience-Based Testing of AI-Based Systems
Selecting Test Techniques for AI-Based Systems

10. Test Environments for AI-Based Systems

Test Environments for AI-Based Systems
Virtual Test Environments for Testing AI-Based Systems

11. Using AI for Testing

AI Technologies for Testing
Using AI to Analyze Reported Defects
Using AI for Test Case Generation
Using AI for the Optimization of Regression Test Suites
Using AI for Defect Prediction
Using AI for Testing User Interfaces

introduction

meet the trainer

Viepul Kocher

Viepul Kocher is an ex-Adobe engineer and IIT alumnus with a 25-year experience in Software Development and Testing industry. Besides being the creator of ISTQB Certified AI Testing and a former certification in AI testing, Viepul is also President of the Indian Testing – ISTQB Board, Convenor of STeP-IN forum and National Convenor of Indica Academy.