ISTQB Updated CT-AI Exam Questions and Answers by lylah

A. Add 10% of the rows randomly and create another model and compare the R-Square scores of both the models.

Adding more data to the training set can affect the R-Square score, but it does not directly verify the correctness of the implementation.

B. Train various models by changing the order of input features and verify that the R-Square score of these models vary significantly.

Changing the order of input features should not significantly affect the R-Square score if the implementation is correct, but this approach is more about testing model robustness rather than correctness of the implementation.

C. Compare the R-Square score of the model obtained using two different implementations that utilize two different programming languages while using the same algorithm and the same training and testing data.

This approach directly compares the performance of two implementations of the same algorithm. If both implementations produce similar R-Square scores on the same training and testing data, it suggests that the new implementation "X" is correct.

D. Drop 10% of the rows randomly and create another model and compare the R-Square scores of both the models.

Dropping data can lead to variations in the R-Square score but does not directly verify the correctness of the implementation.

Therefore, optionCis the most appropriate strategy because it directly compares the performance of the new implementation "X" with another implementation using the same algorithm and datasets, which helps in verifying the correctness of the implementation​​.

Section5.3 – AI Support for Defect Predictionof the ISTQB CT-AI syllabus explains that AI-based defect prediction models rely onhistorical patterns, including past defects, code behavior, and similar system configurations. ML models trained on prior defect data can identifycomponents likely to contain defectswhen new changes resemble previous defect-inducing patterns. This directly supports OptionA, which states that defect prediction is most effective when based on previous similar constellations.

Option B is incorrect: ML can predictwhich componentsare likely to fail, not only whether defects exist. Option C is incomplete; code metrics help, but defect prediction relies onmanycontextual features (historical defects, code churn, commit frequency, etc.). Option D is wrong because defect prediction isnotbased on formal principles and typically requiresmany features, not just a few.

Thus,Option Ais the correct and syllabus-consistent answer.

The syllabus mentions that AI can help optimize regression test suites:

"An AI-based tool can perform optimization of the regression test suite by analyzing… the information from previous test results, associated defects, and the latest changes that have been made, such as features which are broken more frequently and which tests exercise code impacted by recent changes."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 11.4, page 79 of 99)

The ISTQB CT-AI syllabus explains ML performance metrics in Section3.2 – Evaluating ML Models. Forclustering, which is an unsupervised learning method, the syllabus lists metrics such asintra-cluster distance,inter-cluster distance, and coherence measures. Intra-cluster metrics evaluate how close data points are within a cluster, which directly corresponds to Option A.

Option B is incorrect becauseR-squaredis a regression metric measuring goodness-of-fit, not classification performance, and has no connection to ROC curves. Option C is wrong because thesilhouette coefficientis also a clustering metric, measuring cohesion vs. separation—not regression accuracy. Option D is incorrect because ROC curves evaluatebinary or multiclass classification, not clustering.

Thus, OptionAis the only accurate statement based on the syllabus.