Isaca Updated AAIA Exam Questions and Answers by merryn

A common pitfall in AI performance is " aggregate accuracy, " which masks poor performance in specific demographics or subgroups. To detect these " subgroup regressions, " auditors should look for " comparative evaluations " where the model is tested separately for different languages. " Parity thresholds " establish the maximum allowable difference in performance between the majority group (English) and minority groups. This ensures fairness and consistent user experience. Post-go-live human review is reactive, and translating everything to English ignores the nuances of the original language that likely caused the errors in the first place.

Hyperparameter tuning involves adjusting settings like learning rate, batch size, or layer depth to optimize performance. This process is documented in " Model development logs, " which record the different " experiments " run by data scientists, the parameters used in each, and the resulting performance metrics (e.g., accuracy or loss). For an auditor, these logs are the " Audit Trail " of the model ' s construction. Data sheets (Option A) provide high-level info on datasets, and explainability reports (Option C) focus on outputs, but development logs provide the technical history of how the model was " tuned " to its final state.

When using generative AI tools during audit activities, the most critical concern is the reliability and appropriateness of the information being entered and processed. According to the AAIA™ Study Guide, auditors are accountable for ensuring that audit data is valid, confidential, and that generated outputs are factual and verifiable.

“IS auditors must evaluate whether the information entered into AI tools is reliable and appropriate for the audit context. Inputting sensitive or unverified data may lead to regulatory violations or audit inaccuracies.”

While hallucinations (C) and privacy notices (B) are important concerns, the primary auditor responsibility is to ensure that source data is accurate and suitable. Therefore, D is the correct response.

Sustaining effective AI governance and risk management requires continuous, organization-wide awareness , not just one-off or role-limited training. Option D embeds AI topics (governance, risk, ethics, privacy, and security) into the existing security awareness program , which is already a recurring and mandatory mechanism across the enterprise. This supports ongoing adaptation to rapidly evolving AI technologies, aligns with ISACA’s emphasis on integrating AI risk considerations into existing governance and risk frameworks, and ensures that staff at all levels understand their responsibilities.

Options A and C are too narrow in scope, as they target only technical staff or senior management; they help but do not create pervasive, sustainable governance. Option B can supplement internal training, but outsourcing alone does not ensure continuity or alignment with internal policies.