ISTQB Updated CT-AI Exam Questions and Answers by giovanni

The ISTQB CT-AI syllabus explains in Section1.6 – Machine Learning Approachesthatsupervised learningis appropriate when labeled data exists and the goal is to predict an output based on known historical examples. Predicting a customer’spurchase probabilityis aclassificationtask when the output corresponds to discrete categories such as“likely to purchase”vs.“not likely to purchase.”The syllabus gives similar examples in describing classification as the process of assigning instances to predefined classes based on learned patterns in labeled data. Because the retail company wants to determine whether a customer will make a purchase based on marketing actions, classification is the most appropriate choice .

Option A (regression) predicts continuous numeric values and is less suitable because the task centers oncategorical likelihood, not estimating exact monetary values. Option C (unsupervised learning) is used when labels are not available—here, the company has labeled purchase histories. Option D (reinforcement learning) requires an interactive environment with reward-driven behavior, which is not applicable to this scenario.

Thus,supervised learning (classification)is the most suitable approach according to the syllabus.

The question asks which test is least likely to be performed during the ML model testing phase. Let's consider each option:

Testing the accuracy of the classification model (A): Accuracy testing is a fundamental part of the ML model testing phase. It ensures that the model correctly classifies the data as intended and meets the required performance metrics.

Testing the API of the service powered by the ML model (B): Testing the API is crucial, especially if the ML model is deployed as part of a service. This ensures that the service integrates well with other systems and that the API performs as expected.

Testing the speed of the training of the model (C): This is least likely to be part of the ML model testing phase. The speed of training is more relevant during the development phase when optimizing and tuning the model. During testing, the focus is more on the model's performance and behavior rather than how quickly it was trained.

Testing the speed of the prediction by the model (D): Testing the speed of prediction is important to ensure that the model meets performance requirements in a production environment, especially for real-time applications.

The syllabus defines back-to-back testing as a method to compare a modified AI system to the previous version, which is ideal in this scenario:

"Back-to-back testing is performed by comparing the outputs of two systems that are supposed to provide the same outputs, one being a known and trusted system and the other being the test system. This approach can be used to test ML systems after re-training to verify that improvements have not introduced regressions."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.3, page 67 of 99)

The syllabus states:

“A/B testing can be used to test updates to an AI-based system where there are agreed acceptance criteria, such as ML functional performance metrics, as described in Chapter 5. A/B testing is used to compare the updated variant with the previous variant.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.4, page 68 of 99)