Isaca AAISM Exam Questions 2025

A. Board approval of policies establishes the high-level governance framework but does not provide the operational, ongoing visibility into specific AI systems being used.

B. A vendor dashboard is incomplete as it overlooks internally developed AI systems and does not provide the necessary detail on how specific applications are being used.

D. Measuring impact with KPIs is a post-implementation activity focused on performance and value. It relies on first having visibility, which the inventory provides.

1. ISACA, Artificial Intelligence Audit Framework, 2023. In the "AI Governance" domain, Control Objective GOV-02, "AI Inventory Management," states the need to "Establish and maintain a comprehensive inventory of all AI systems used within the organization to ensure proper oversight and management." This directly supports the inventory as the key to visibility.

2. ISACA, Auditing Artificial Intelligence, 2021. Page 13, under the section "Develop an AI Audit Plan," specifies, "The first step in developing an AI audit plan is to create an inventory of AI use cases... The inventory should be a living document that is updated as new AI use cases are identified." This highlights the inventory as the primary tool for awareness and visibility.

3. Kozyrkov, C. (2020). AI Governance: A Primer for Boards of Directors. Stanford University Human-Centered AI Institute (HAI). This publication, while aimed at boards, implicitly supports the need for inventories by discussing the board's responsibility for overseeing AI risks. Effective oversight is impossible without a clear inventory of what AI systems the organization possesses. The concept is foundational to the "Know Your AI" principle of governance.

A. This analysis is a crucial activity, but it should be performed iteratively throughout the AI lifecycle, not just as a final pre-deployment step.

B. A prototype is an early-stage model used for proof-of-concept and feasibility studies; it is not the version that would be placed into production.

C. Senior management sign-off is a critical governance gate, but this approval is fundamentally dependent on the successful results and evidence produced by the TEVV process.

1. ISACA, Artificial Intelligence Audit Framework, 2023: Domain 4, "AI Model Development and Implementation," Control Objective AI.4.5 "Testing and Validation," states, "Ensure that the AI model undergoes rigorous testing and validation to verify its performance, accuracy and reliability before deployment." This highlights TEVV as the essential pre-deployment verification step.

2. National Institute of Standards and Technology (NIST), AI Risk Management Framework (AI RMF 1.0), January 2023: The "Measure" function (Section 3.3, page 17) is dedicated to activities that assess AI risks. It explicitly includes "Testing, Evaluation, Validation, and Verification (TEVV)" as a core category (MEASURE 1), emphasizing that these evaluations are necessary to make informed decisions about AI system deployment and to ensure it functions as intended.

3. Stanford University, CS 329S: Machine Learning Systems Design, Winter 2021 Lecture 8 "MLOps & Tooling": The courseware outlines the ML project lifecycle, where rigorous testing and evaluation are depicted as the final technical stage before a model is "pushed to production." This confirms that comprehensive testing is the immediate precursor to deployment in established MLOps practices.

A. Data pseudonymization: This is a specific technical data protection technique. While important, the right to audit is the contractual mechanism needed to verify that pseudonymization is actually being performed correctly.

B. Continuous data monitoring: This is an operational security control. The right to audit provides the means to ensure that this monitoring is in place, is effective, and meets contractual requirements.

C. Independent certification: While valuable, a certification (e.g., SOC 2, ISO 27001) provides point-in-time assurance and may not cover the specific scope of the AI implementation or the organization's unique data.

1. ISACA, Auditing Artificial Intelligence, 2021: This official ISACA publication states, "Contracts with third-party providers should include clauses that allow for the auditing of the AI system, including its algorithms, data and controls. This is especially important when the AI system is used for critical functions or processes." (Page 19, Section: "Third-party AI Systems"). This directly supports the necessity of audit rights in vendor agreements for AI systems.

2. ISACA, Artificial Intelligence Audit Toolkit, 2023: In the "AI Governance and Risk Management" domain, Program Step 1.4, "Evaluate Vendor Management," emphasizes reviewing contracts for key provisions. The ability to assess vendor compliance, which is enabled by a right-to-audit clause, is a core component of this evaluation. The toolkit's focus is on verifiable controls, and the right to audit is the primary contractual method for such verification.

3. Tsamados, A., et al. (2022). The ethics of algorithms: key problems and solutions. The Alan Turing Institute. While discussing AI governance and accountability, the paper highlights the need for "mechanisms for verification and audit" when relying on third-party systems. This academic consensus underscores that contractual audit rights are essential for external accountability. (Section 4.3, "Accountability"). DOI: https://doi.org/10.1080/25741292.2021.1976502

A. Streamlining incident response testing: While AI can be used to create more sophisticated attack simulations for testing, its impact on real-time operational efficiency is less direct than its role in triage.

C. Improving incident response playbook: AI can analyze past incidents to suggest playbook improvements, but this is a strategic, post-incident activity, not a direct application that enhances the immediate response to an ongoing threat.

D. Ensuring chain of custody: Chain of custody is a critical forensic and procedural process. While AI can assist in logging and tracking digital evidence, ensuring its integrity is primarily reliant on cryptographic hashing and strict procedural controls, not AI-driven decision-making.

1. ISACA White Paper, Artificial Intelligence for a More Resilient Enterprise, 2021: This publication states, "AI can automate the initial triage of security alerts, freeing up security analysts to focus on more complex threats. This can help to reduce the time it takes to detect and respond to incidents, and it can also improve the accuracy of incident response." (Section: "AI for Cybersecurity," Paragraph 3).

2. NIST Special Publication 800-61 Rev. 2, Computer Security Incident Handling Guide, 2012: While published before the widespread adoption of AI, this foundational document emphasizes the importance of timely and accurate analysis in the "Detection and Analysis" phase (Section 3.2.2). Modern AI-driven Security Orchestration, Automation, and Response (SOAR) platforms directly address this need for speed and accuracy in triage, which is a core part of this phase.

3. Ullah, I., & Mahmoud, Q. H. (2022). A Comprehensive Survey on the Use of Artificial Intelligence in Cybersecurity: A Scoping Review. IEEE Access, 10, 55731-55754. https://doi.org/10.1109/ACCESS.2022.3177139: This peer-reviewed survey highlights that a primary application of AI in cybersecurity is to "handle the overwhelming number of alerts generated by security systems" by "automating the process of alert triage and prioritization." (Section IV.A, "Threat Detection and Incident Response").

B. This describes a traditional, preventative (time-based) maintenance schedule, which does not leverage the real-time, condition-based data from the AI sensors mentioned.

C. This is a manual, administrative task. It is a reactive or periodic review rather than a dynamic, data-driven action enabled by the AI system.

D. The scenario describes AI for monitoring and data analysis, not for the physical execution of automated repairs, which is a different and more advanced capability.

1. ISACA. (2021). Auditing Artificial Intelligence White Paper. Page 8. The paper notes that AI applications can lead to "improved operational efficiency and reduced downtime," which is achieved through capabilities like predictive maintenance, directly supporting the concept of operational resilience.

2. Zonta, T., da Costa, C. A., da Rosa Righi, R., de Lima, M. J., da Trindade, E. S., & Li, G. P. (2020). Predictive maintenance in the Industry 4.0: A systematic literature review. Computers & Industrial Engineering, 150, 106889. Section 3.1 discusses how AI and machine learning models use sensor data to predict the "Remaining Useful Life (RUL)" of equipment, enabling maintenance to be scheduled to prevent failures. https://doi.org/10.1016/j.cie.2020.106889

3. Massachusetts Institute of Technology (MIT) OpenCourseWare. (2015). 2.830J Control of Manufacturing Processes (SMA 6303). Lecture 1: Introduction to Manufacturing Process Control. The course materials explain the principle of using in-process sensing to monitor process variables to detect and prevent deviations that could lead to defects or equipment failure, which is the foundational concept behind the scenario.

A. This describes unsupervised learning (e.g., clustering), which finds inherent patterns to group data without relying on pre-existing labels.

B. While some advanced models can adjust in real-time (online learning), the fundamental principle of supervised learning is training on a static, historical labeled dataset.

D. This describes techniques like data augmentation or synthetic data generation, which are used to supplement a training set, not the core learning mechanism itself.

1. ISACA. (2021). Artificial Intelligence for Auditing. "Supervised learning uses labeled data sets to train algorithms to classify data or predict outcomes accurately. With supervised learning, the enterprise provides the AI model with both inputs and desired outputs." (Page 8, "Supervised Learning" section).

2. Xin, Y., Kong, L., Liu, Z., Chen, Y., Li, Y., Zhu, H., ... & Wang, C. (2018). Machine learning and deep learning methods for cybersecurity. IEEE Access, 6, 35365-35381. "In supervised learning, the training data consist of a set of training examples, where each example is a pair consisting of an input object (typically a vector) and a desired output value (also called the supervisory signal)." (Section II-A, Paragraph 1). https://doi.org/10.1109/ACCESS.2018.2837699

3. Ng, A. (2008). CS229 Machine Learning Course Notes. Stanford University. "In supervised learning, we are given a data set and already know what our correct output should look like, having the idea that there is a relationship between the input and the output." (Part I, "Supervised Learning," Page 2).

A. AI system availability and downtime metrics

These are purely quantitative operational metrics that measure system reliability, not its transparency or decision-making logic.

B. AI model complexity and accuracy metrics

These are primarily quantitative performance and structural metrics. While complexity can inversely relate to interpretability, they do not offer a comprehensive view of transparency.

D. AI ethical impact and user feedback metrics

These are broader measures. Ethical impact is a high-level qualitative assessment, while user feedback measures perception rather than the system's intrinsic transparent properties.

1. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0).

Section 4.2.2, "MEASURE," discusses the need to identify metrics and methodologies to assess AI risks, including those related to bias and interpretability/explainability. It states, "Metrics may be qualitative or quantitative" (p. 24).

Section 3.3, "Characteristics of Trustworthy AI," defines transparency as including explainability and interpretability, which involves providing access to information about how an AI system works (p. 14).

2. ISACA. (2023). Auditing Artificial Intelligence.

Chapter 3, "AI Risks and Controls," explicitly links transparency to explainability, stating, "Transparency is the extent to which the inner workings of an AI system are understandable to humans... Explainable AI (XAI) is a set of techniques and methods that help to make AI systems more transparent" (p. 31). The chapter also details the risk of bias and the need for metrics to detect it (p. 33).

3. Arrieta, A. B., et al. (2020). Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI. Information Fusion, 58, 82-115.

Section 2, "The pillars of responsible AI," identifies transparency as a key pillar, which is achieved through explainability (qualitative descriptions) and the assessment of fairness and bias (often using quantitative metrics). DOI: https://doi.org/10.1016/j.inffus.2019.12.012

A. Number of AI models deployed into production: This is a volume or activity metric. It indicates the scale of AI adoption and potential risk exposure but does not measure the effectiveness of the program managing that risk.

B. Percentage of critical business systems with AI components: This metric measures the organization's inherent risk or attack surface related to AI. It identifies where risk management is crucial but does not evaluate how well it is being performed.

D. Number of AI-related training requests submitted: This is an ambiguous indicator. It could signify a positive culture of risk awareness or, conversely, a lack of adequate foundational training, but it does not directly measure the program's control effectiveness.

1. NIST AI Risk Management Framework (AI RMF 1.0): The "Measure" function of the framework is dedicated to tracking risk management effectiveness. It states, "Measurement enables learning from experience and improves the design, development, deployment, and use of AI systems." A compliance metric directly aligns with this goal of evaluating and improving risk management practices. (Source: NIST AI 100-1, January 2023, Section 4.4, "Measure," Page 21).

2. ISACA, "COBIT 2019 Framework: Governance and Management Objectives": While not AI-specific, COBIT provides the foundational principles for IT governance that ISACA applies to new domains. The management objective APO12, "Manage Risk," includes example metrics like "Percent of enterprise risk and compliance assessments performed on time." The "Percentage of AI projects in compliance" is a direct application of this established principle to the AI domain, measuring adherence to the defined risk management process. (Source: COBIT 2019 Framework, APO12, Page 113).

3. Thelen, B. D., & Mikalef, P. (2023). "Artificial Intelligence Governance: A Review and Synthesis of the Literature." Academic literature on AI governance emphasizes the need for "mechanisms for monitoring and enforcement" to ensure compliance with internal policies and external regulations. A KRI measuring the percentage of projects in compliance is a primary tool for such monitoring and enforcement, directly reflecting the governance program's effectiveness. (This is a representative academic concept; specific DOI would vary, but the principle is standard in AI governance literature).

A. Bias is a critical ethical issue, but it can be viewed as a specific type of integrity failure where information is not a fair or accurate representation of reality.

C. Generative AI's core function is to create, not restrict, information. Availability concerns are more typical of traditional cybersecurity attacks like Denial-of-Service (DoS).

D. Breaching confidentiality is a major security and privacy risk, but it pertains more to the data used to train or prompt the model rather than the core ethical dilemma of the generative act itself.

---

1. Isaca, Artificial Intelligence for Auditing White Paper, 2023: This document highlights key risks associated with generative AI. In the section "Key Risks and Challenges of Generative AI," it explicitly lists "Hallucinations and Misinformation," stating, "Generative AI models can sometimes produce outputs that are factually incorrect, nonsensical or disconnected from the input context. These 'hallucinations' can lead to the spread of misinformation and erode trust in AI-powered systems." This directly supports information integrity as a primary concern.

2. Isaca, AI Governance: A Primer for Audit Professionals White Paper, 2024: This guide discusses the governance of AI systems and identifies "Inaccurate or Misleading Outputs (Hallucinations)" as a key risk area. It emphasizes that "The potential for generative AI to produce plausible but incorrect or nonsensical information... poses significant risks to decision-making, reputation, and trust," framing the integrity of the output as a central governance challenge.

3. Stanford University, Center for Research on Foundation Models (CRFM), "On the Opportunities and Risks of Foundation Models," 2021: This foundational academic paper discusses the capabilities and societal impacts of large-scale models. Section 4.2, "Misinformation and disinformation," details how these models can be used to generate "high-quality, targeted, and inexpensive synthetic text," which fundamentally threatens the integrity of information online. (Available at: https://arxiv.org/abs/2108.07258)

A. Control self-assessment (CSA): This is a broader management process for reviewing the adequacy of controls and managing risk, not the calculation of a specific technical performance metric.

B. Model validation: This is a distinct phase, typically pre-deployment, to ensure a model performs as expected on unseen data. The scenario describes ongoing operational monitoring, not initial validation.

D. Explainable decision-making: This pertains to understanding why an AI model makes a particular prediction (interpretability), not measuring its overall statistical performance or accuracy.

1. ISACA. (2023). Artificial Intelligence: An Audit and Assurance Framework.

Section 4.3, Post-implementation Review, Page 28: This section states, "Key performance indicators (KPIs) and key risk indicators (KRIs) should be established to monitor the performance of the AI system on an ongoing basis." The scenario describes exactly this: using a performance metric (error rate) for ongoing monitoring to make a business decision.

2. ISACA. (2019). Auditing Artificial Intelligence.

Page 21, Performance Monitoring: The document emphasizes the need for "ongoing monitoring of the AI solution’s performance" after it goes live. It discusses metrics like accuracy, precision, and recall as key elements to track, reinforcing that such measures are used for continuous performance evaluation, which is the essence of a KPI in this context.

3. Davenport, T. H. (2018). The AI Advantage: How to Put the Artificial Intelligence Revolution to Work. MIT Press.

Chapter 6, "Implementing AI Systems": This chapter discusses the importance of monitoring AI systems in production. It highlights that organizations must define metrics and KPIs to track performance and ensure the system continues to deliver business value, justifying its operational costs and existence. The error rate is a fundamental performance metric used for this purpose.