Conducting regular, independent audits is the most effective method for monitoring vendor compliance. Audits provide direct, verifiable assurance that the vendor's processes, controls, and outputs align with contractually mandated ethical and security standards. This approach moves beyond trust-based models like self-attestation, offering a structured and evidence-based mechanism to assess the entire AI development lifecycle. It allows the contracting organization to proactively identify gaps, enforce accountability, and ensure that principles of fairness, transparency, and security are being actively implemented, rather than just claimed.

B. Requiring vendors to monitor their adherence to ethics and security standards: This approach lacks independent verification and creates a potential conflict of interest, making it less reliable than an external audit.

C. Mandating that vendors share source code and AI documentation with the contracting party: While useful for due diligence, code and documentation review is a static, point-in-time assessment and does not guarantee ongoing process compliance.

D. Allowing vendors to self-attest ethical AI compliance and implement benchmark monitoring: Self-attestation is the weakest form of assurance, as it is not independently verified and may not accurately reflect actual practices.

1. ISACA. (2023). Artificial Intelligence Audit Toolkit. In the section on "Third-Party Management

" the toolkit emphasizes the need for organizations to "Perform audits of the third party" and "Review third-party attestation reports (e.g.

SOC 2)" to gain assurance over the controls at the vendor. This directly supports the practice of conducting audits (Option A) over relying on vendor self-monitoring or attestation (Options B and D).

2. ISACA. (2021). Auditing Artificial Intelligence. This white paper states

"The audit and assurance professional should obtain an understanding of the third-party relationships and related processes to determine whether the enterprise has implemented appropriate governance and monitoring controls." It highlights the auditor's role in verifying these controls

which is the essence of an audit. (Page 18

Third-Party Management).

3. National Institute of Standards and Technology (NIST). (2023). AI Risk Management Framework (AI RMF 1.0). In the "Govern" function

section 4.4 "AI System and Third-Party Relationships

" the framework outlines the importance of establishing processes for "monitoring and reviews of third-party providers

" including "audits

and assessments." This underscores that active

independent verification like audits is a core component of managing third-party AI risk. (Page 21).

Data minimization is a core privacy and security principle that involves processing only the data that is absolutely necessary for a specific purpose. In the context of training a large language model (LLM), applying data minimization means proactively identifying and removing any sensitive, personal, or proprietary information from the training datasets before the model is trained. This directly addresses the CISO's concerns by fundamentally reducing the risk surface. By ensuring sensitive data is not included, the organization mitigates the possibility of the model memorizing and later exposing this information, thus preventing potential data breaches and ensuring compliance with privacy regulations.

A. Data augmentation: This technique is used to increase the volume of training data to improve model accuracy and generalization, not to address security or privacy risks.

C. Data classification: This is a foundational step to identify sensitive data, but it does not, by itself, mitigate the risk; it only categorizes it.

D. Data discovery: This process locates data across systems. Like classification, it is a preliminary step that identifies the problem but does not implement a solution.

1. National Institute of Standards and Technology (NIST). (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). (NIST AI 100-1).

Page 29

Table 7

MAP.T10: In the section on documenting training data

this subcategory explicitly lists "adherence to data minimization principles" as a key element for managing AI risks. This directly supports using data minimization to handle training data concerns.

2. ISACA. (2023). Artificial Intelligence: An Audit and Assurance Framework.

Page 21

Section 3.2.1 Data Governance: The framework states

"Data used to train AI models should be relevant

accurate and appropriate for the intended purpose... It is important to ensure that the data used for training does not contain any sensitive or confidential information that could be inadvertently exposed by the AI model." This aligns with the principle of data minimization to remove unnecessary sensitive data.

3. European Union. (2016). General Data Protection Regulation (GDPR).

Article 5(1)(c): This article establishes "data minimisation" as a core principle of data protection

stating that personal data shall be "adequate

relevant and limited to what is necessary in relation to the purposes for which they are processed." This legal and privacy principle is the direct solution to the CISO's concerns when applied to AI training data.

Open-source data repositories are centralized platforms (e.g., Kaggle, UCI Machine Learning Repository, Google Dataset Search) that host a wide variety of large, often cleaned and labeled, datasets. They are specifically designed to provide data for research, development, and testing of applications, particularly in the AI and machine learning domains. Using these repositories is the most direct, efficient, and common method for an organization to acquire the large-scale data needed for comprehensive application testing without the significant overhead of collecting and preparing the data from scratch.

A. AI model cards are documents that provide transparency about a model's performance, limitations, and ethical considerations, not a source of raw data.

B. Incorporating data from search content is impractical; it is often unstructured, may have copyright or privacy restrictions, and requires extensive effort to scrape, clean, and label.

D. AI data augmentation is a technique to artificially increase the size of an existing dataset. It cannot be used without an initial dataset to augment.

1. ISACA

Artificial Intelligence Audit and Assurance Framework

2023. In the "AI Data Life Cycle" section

the framework discusses "Data Sourcing and Acquisition

" emphasizing the need to obtain relevant and sufficient data for training and testing AI models. Open-source repositories are a primary means of fulfilling this requirement for large-scale data needs.

2. Ng

A. (2023). Machine Learning (CS229) Course Materials. Stanford University. Lecture notes and project guidelines frequently reference the use of public datasets from repositories like the UCI Machine Learning Repository and Kaggle as standard practice for obtaining data for model development and evaluation. (See

for example

project guidelines which list acceptable data sources).

3. Roh

Y.

Heo

G.

& Whang

S. E. (2021). A survey on data collection for machine learning: a big data - AI integration perspective. IEEE Transactions on Knowledge and Data Engineering

33(4)

1328-1347. This paper discusses various data collection methods

highlighting public datasets as a crucial resource. Section 3.1

"Using Existing Datasets

" explicitly states

"The easiest way to collect data is to use publicly available datasets... Many websites share public datasets

such as Kaggle

the UCI ML Repository

and the KDD Cup." This confirms that using such repositories is a primary and efficient method. (DOI: https://doi.org/10.1109/TKDE.2019.2946162)

An accountability model establishes a clear governance structure, defining roles, responsibilities, and oversight mechanisms for AI systems. This framework is fundamental to ensuring that an AI product is used ethically and responsibly throughout its lifecycle. It addresses the organization's concerns by creating clear lines of authority for managing risks related to fairness, bias, transparency, and societal impact. This model provides the foundation upon which specific tools and practices, such as model cards and security protocols, are implemented and enforced.

A. Model cards are a tool for transparency, which is a component of a responsible AI strategy, but they do not constitute the entire governance framework needed for accountability.

B. Vendor monitoring is relevant only when using third-party AI systems and does not address the organization's internal responsibility for a product it has developed.

D. Security by design is a critical practice focused on protecting the AI system from threats, but it does not cover the full spectrum of ethical considerations like fairness and bias.

1. ISACA

"Artificial Intelligence: An Audit and Assurance Framework

" 2023. Page 14

under the "AI Governance Framework" section

states

"A governance framework establishes accountability

roles and responsibilities

and decision rights... It also provides a structure for oversight to ensure that AI systems are aligned with the organization’s ethical principles."

2. National Institute of Standards and Technology (NIST)

"AI Risk Management Framework (AI RMF 1.0)

" January 2023. Section 3.1

"The GOVERN Function

" emphasizes that this function is central to responsible AI. It states

"Governance processes should be in place to ensure accountability for AI risks and their management... This includes assigning roles and responsibilities for all stages of the AI lifecycle."

3. Mittelstadt

B. (2019). "Principles alone cannot guarantee ethical AI." Nature Machine Intelligence

1(11)

501-507. This academic paper argues that high-level ethical principles are insufficient without practical implementation through "mechanisms of accountability" and robust governance structures to translate principles into practice. (DOI: https://doi.org/10.1038/s42256-019-0114-4)

A Key Risk Indicator (KRI) for an AI risk management program should measure the program's effectiveness in governing AI initiatives and ensuring they adhere to established policies and controls. The "Percentage of AI projects in compliance" is the most direct measure of this effectiveness. It quantifies how well the organization's AI activities are following the prescribed risk management framework, including mandatory assessments, controls, and documentation. A high compliance rate indicates a successful and effective program, while a low rate serves as an early warning that the program is not being implemented properly, increasing overall AI-related risk.

A. Number of AI models deployed into production: This is a volume or activity metric. It indicates the scale of AI adoption and potential risk exposure but does not measure the effectiveness of the program managing that risk.

B. Percentage of critical business systems with AI components: This metric measures the organization's inherent risk or attack surface related to AI. It identifies where risk management is crucial but does not evaluate how well it is being performed.

D. Number of AI-related training requests submitted: This is an ambiguous indicator. It could signify a positive culture of risk awareness or, conversely, a lack of adequate foundational training, but it does not directly measure the program's control effectiveness.

1. NIST AI Risk Management Framework (AI RMF 1.0): The "Measure" function of the framework is dedicated to tracking risk management effectiveness. It states

"Measurement enables learning from experience and improves the design

development

deployment

and use of AI systems." A compliance metric directly aligns with this goal of evaluating and improving risk management practices. (Source: NIST AI 100-1

January 2023

Section 4.4

"Measure

" Page 21).

2. ISACA

"COBIT 2019 Framework: Governance and Management Objectives": While not AI-specific

COBIT provides the foundational principles for IT governance that ISACA applies to new domains. The management objective APO12

"Manage Risk

" includes example metrics like "Percent of enterprise risk and compliance assessments performed on time." The "Percentage of AI projects in compliance" is a direct application of this established principle to the AI domain

measuring adherence to the defined risk management process. (Source: COBIT 2019 Framework

APO12

Page 113).

3. Thelen

B. D.

& Mikalef

P. (2023). "Artificial Intelligence Governance: A Review and Synthesis of the Literature." Academic literature on AI governance emphasizes the need for "mechanisms for monitoring and enforcement" to ensure compliance with internal policies and external regulations. A KRI measuring the percentage of projects in compliance is a primary tool for such monitoring and enforcement

directly reflecting the governance program's effectiveness. (This is a representative academic concept; specific DOI would vary

but the principle is standard in AI governance literature).

Adopting a phased approach is the most effective strategy for managing the security risks associated with integrating a novel technology like AI. This method allows an organization to introduce AI capabilities incrementally, starting with pilots or limited-scope projects. Each phase serves as a controlled environment to identify, assess, and mitigate emergent security risks before they can impact the entire enterprise. This iterative process facilitates learning, allows for the refinement of security controls and governance policies, and ensures that the complexities and potential vulnerabilities of AI are understood and managed before scaling up. It is a foundational risk management practice for complex technology adoption.

A. Re-evaluating the risk appetite is a critical governance activity that sets tolerance levels but does not, by itself, constitute a management strategy for implementation risks.

B. Seeking third-party advice is a valuable, supportive measure for gaining expertise but is not the primary, overarching strategy for managing the integration process.

C. Evaluating compliance requirements is essential for establishing a baseline and avoiding legal penalties but often fails to address novel, technology-specific security risks not yet covered by regulations.

1. ISACA

COBIT® 2019 Implementation Guide

2018. Chapter 3

"The Implementation Lifecycle

" details a seven-phase

iterative approach. Phase 4

"Plan

" and Phase 5

"Design

" emphasize planning for incremental implementation. This phased methodology is a core principle for managing risk during the implementation of any significant IT change

including AI. The guide states

"The lifecycle should be iterative

meaning that the enterprise can cycle through the phases as needed." This directly supports a phased approach for managing complex integrations.

2. ISACA

"Auditing Artificial Intelligence

" White Paper

2021. Page 14 discusses the "AI Journey" and the importance of starting with pilot projects. It states

"The AI journey is a marathon

not a sprint... It is important to start small with a few pilot projects to learn and build momentum." This recommendation to "start small" is the essence of a phased approach to manage the risks and complexities of AI adoption.

3. Hiekkanen

K.

et al. "This Time It’s Different? A Review of the AI Governance Literature." Proceedings of the 53rd Hawaii International Conference on System Sciences

2020. (https://doi.org/10.24251/HICSS.2020.638). The paper discusses AI governance frameworks

which often implicitly or explicitly recommend iterative and adaptive strategies for AI adoption. The concept of "sandboxing" and pilot programs is highlighted as a key mechanism for exploring AI's potential while containing risks

which is a form of a phased approach.

The greatest challenge in applying copyright law to AI-generated content is determining rightful ownership. Traditional copyright frameworks, such as the U.S. Copyright Act, are predicated on the principle of "human authorship." Because an AI is not a legal person, it cannot be considered an "author" in the legal sense. This creates a fundamental conflict: if there is no human author, it is unclear who, if anyone, owns the copyright. The debate involves whether ownership should fall to the user providing the prompt, the AI developer, the owner of the computing resources, or if the work should enter the public domain. This foundational issue of authorship and ownership must be resolved before other issues, like licensing, can be effectively addressed.

A. Enforcing trademark rights is concerned with the branding of the AI system itself (e.g., its name or logo), not the copyright of the content it generates.

C. Protecting trade secrets applies to the AI's underlying algorithms, models, and training data, which are distinct from the copyright status of the AI's output.

D. Establishing licensing frameworks is a secondary issue that depends entirely on first resolving the primary challenge of who holds the ownership rights to be licensed.

1. U.S. Copyright Office. (2023). Copyright Registration Guidance: Works Containing Material Generated by Artificial Intelligence. Federal Register

88(51)

16190-16194.

Page 16192

Section III.A: The guidance states

"the Office will not register works produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author." This directly highlights that the lack of human authorship is the central barrier to establishing copyright

and thus ownership.

2. World Intellectual Property Organization (WIPO). (2020

May). WIPO Conversation on Intellectual Property (IP) and Artificial Intelligence (AI): Revised Issues Paper on Intellectual Property Policy. WIPO/IP/AI/2/GE/20.

Page 11

Issue 11: Authorship and Ownership: The paper explicitly poses the core question: "Should copyright be attributed to original literary and artistic works that are autonomously generated by AI? If so

who should be considered the author or owner of the copyright...?" This frames the ownership question as the primary policy challenge for international IP bodies.

3. Stanford University Human-Centered AI (HAI). (2023

September 12). Generative AI and the Future of Work.

In the section on "Intellectual Property

" the report discusses the legal ambiguity surrounding AI-generated works

stating

"Current U.S. copyright law protects only works with human authors

leaving the ownership of purely AI-generated content in a gray area." This reinforces that ownership is the central unresolved issue.

A model card is a standardized documentation artifact designed to increase transparency and accountability for machine learning models. It provides a structured summary of a model's intended uses, performance metrics (often disaggregated across different groups), limitations, ethical considerations, and the data used for training and evaluation. By presenting this crucial information in a concise and accessible format, model cards enable various stakeholders—including developers, policymakers, and end-users—to understand the model's capabilities and risks. This transparency is a cornerstone for building trust in AI systems, as it demonstrates due diligence and provides a basis for informed decision-making.

A. Hyperparameters: These are low-level technical settings for the training algorithm. They are not meaningful to most stakeholders and do not describe the model's real-world performance or impact.

B. Data quality controls: While essential for building a reliable model, these are processes and metrics related to the input data, not a comprehensive summary document about the finished model itself.

D. Model prototyping: This is an early, experimental phase in the model development lifecycle. A prototype is not a formal documentation artifact for a deployed model and lacks the rigorous evaluation needed for trust.

1. ISACA

Artificial Intelligence Security and Management (AAISM) Study Guide

1st Edition

2024. Chapter 3

"AI Model Development and Training

" emphasizes the need for comprehensive documentation to ensure transparency and accountability. It identifies model cards as a key tool for documenting model details

performance

and limitations to communicate effectively with stakeholders.

2. Mitchell

M.

Wu

S.

Zaldivar

A.

et al. (2019). Model Cards for Model Reporting. Proceedings of the Conference on Fairness

Accountability

and Transparency

pp. 220-229. This foundational academic paper introduces model cards as a framework to "encourage transparent model reporting" and provide stakeholders with essential information to "better understand the models." (DOI: https://doi.org/10.1145/3287560.3287596)

3. Stanford University

Center for Research on Foundation Models (CRFM). (2023). Transparency Section. The courseware and publications from Stanford's AI programs

such as those from the CRFM

consistently highlight the role of artifacts like model cards in achieving AI transparency and building trust. They are presented as a best practice for responsible AI development and deployment.

The scenario describes an attacker crafting specific inputs to bypass an LLM's built-in restrictions and manipulate its output. This technique is known as prompt injection. The attacker injects malicious instructions within the prompt, causing the model to ignore its original system-level instructions and follow the attacker's commands instead. This directly exploits the model's interpretation of input to compromise its output integrity, making it the most accurate description of the attack.

B. Jailbreaking is a specific goal or outcome of a prompt injection attack, aimed at bypassing safety and ethical filters, rather than the attack technique itself.

C. Remote code execution involves executing arbitrary code on the server, a more severe and distinct attack that is not inherently part of manipulating LLM text output.

D. Evasion is an adversarial attack that typically causes a model (e.g., a classifier) to make an incorrect prediction, which is different from overriding an LLM's core instructions.

1. OWASP Foundation. (2023). OWASP Top 10 for Large Language Model Applications. In the "LLM01: Prompt Injection" section

the vulnerability is defined as: "Prompt injection vulnerabilities allow a malicious user to manipulate the output of a Large Language Model (LLM) through crafted inputs... This can lead to data exfiltration

unauthorized access

or other security breaches."

2. National Institute of Standards and Technology (NIST). (2023). Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST.AI.100-2 E2023). Section 3.2.1

"Data/Input Poisoning

" describes how attackers can manipulate inputs. Prompt injection is cited as a prime example for LLMs where "an attacker can carefully craft a prompt to an LLM to have it ignore previous instructions."

3. Greshake

K.

Abdelnabi

S.

Mishra

S.

Endres

C.

Holz

T.

& Fritz

M. (2023). Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. arXiv preprint arXiv:2302.12173. Section 2

"Background

" defines prompt injection as an attack where an "adversary controls a part of a prompt... to manipulate the LLM's behavior." (https://doi.org/10.48550/arXiv.2302.12173)

The right to audit is the most critical consideration because it provides the organization with the contractual authority to independently verify and validate the third-party AI tool's performance, security controls, data handling processes, and compliance with legal and ethical standards. Given the often opaque nature of AI models, this right is a fundamental governance mechanism for ongoing assurance. It allows the organization to move beyond relying solely on vendor attestations and certifications, enabling direct assessment against specific internal requirements and risk tolerances, which is the core of validation.

A. Terms and conditions: While essential, this is a broad legal document. The right to audit is a specific, crucial clause within the terms and conditions that directly enables validation.

C. Industry analysis and certifications: These are valuable for initial due diligence but are often generic, point-in-time assessments that may not cover the organization's specific use case or risk profile.

D. Roundtable testing: This is a specific, often informal, testing technique. It is only one small component of a comprehensive validation strategy, not the most important overarching consideration.

1. ISACA

Auditing Artificial Intelligence

2021. Chapter 5

"Auditing AI Governance and Risk Management

" emphasizes the importance of managing third-party AI risks. It states that contracts with AI service providers should include "clauses for independent security assessments and the right to audit to ensure that the organization’s data are adequately protected and that the AI solution is performing as expected." (Page 63).

2. ISACA

Artificial Intelligence: An Audit/Assurance Framework

2023. In the "GOVERN" function

section 3.4

"Third-Party Management

" discusses the need to "Establish and monitor third-party relationships to ensure that AI-related risks are managed." The right to audit is a primary mechanism for such monitoring and for gaining assurance over the third party's controls and processes.

3. NIST

AI Risk Management Framework (AI RMF 1.0)

January 2023. The GOVERN function (Section 4.1

GOVERN 3) discusses policies and procedures for third-party AI systems. It highlights the need for organizations to have mechanisms to understand and manage risks from externally sourced AI

for which audit and assessment rights are a prerequisite for effective implementation. (Page 21).