A media company is developing an AI platform for video content analysis that requires storing and

processing large volumes of unstructured video data. The platform must support high throughput for

data ingestion and provide efficient access for real-time analytics. Given these requirements, which

storage strategy should the company implement?

Correct Answer: C (Assuming the options are similar to: A) A scale-up NAS solution using NFS, B) A SAN using Fibre Channel for block storage, C) A scale-out parallel file system, D) Direct-attached storage on each compute node)

Explanation: The requirements—large volumes of unstructured data, high-throughput ingestion, and efficient parallel access for analytics—are characteristic of high-performance computing (HPC) and large-scale AI workloads. A scale-out parallel file system (e.g., Lustre, IBM Spectrum Scale/GPFS) is designed specifically for this scenario. It stripes data across multiple storage servers and disks, allowing many clients (compute nodes) to read and write data in parallel at very high aggregate bandwidth. This architecture avoids the bottlenecks of traditional NAS and provides the shared, high-performance namespace essential for distributed AI training and real-time analytics on large datasets.

Why Incorrect Options are Wrong:

A) A scale-up NAS solution using NFS: A traditional NAS controller becomes a performance bottleneck when many clients access it simultaneously, failing to meet the high-throughput requirement.

B) A SAN using Fibre Channel for block storage: SANs provide block-level access, which is not ideal for sharing large, unstructured files across many compute nodes and requires a complex volume management layer.

D) Direct-attached storage on each compute node: This creates data silos, making it difficult to manage a large, shared dataset and requiring extensive data copying, which is inefficient for this use case.

References:

1. NVIDIA. (2023). NVIDIA DGX SuperPOD Reference Architecture. This document consistently specifies high-performance, scale-out parallel file systems as the primary storage tier for AI workloads to feed the GPUs efficiently.

2. Shainer, G., & Shusterman, V. (2020). NVIDIA GPUDirect Storage: A Direct Path Between Storage and GPU Memory. This technology, central to NVIDIA's AI platform, is designed to work with parallel filesystems to maximize I/O throughput by bypassing the CPU.

3. Maltzahn, C., & Bent, J. (2017). A Survey of Distributed Storage Systems for Big Data and Scientific HPC. University of California, Santa Cruz. UCSC-SOE-17-07. This academic survey discusses the architectural advantages of parallel file systems (like Lustre, GPFS) for data-intensive scientific and analytics workloads.

A – Traditional SANs offer block semantics, poor horizontal scaling for PB-level video archives.

B – Local NVMe alone cannot satisfy multi-node concurrency or capacity growth.

D – Tape libraries are throughput-limited and unsuitable for real-time analytics.

1. NVIDIA, “GPUDirect Storage: Accelerating AI Data Pipelines,” white paper, 2022, §3 (“Object Storage Integration”), p.6.

2. NVIDIA, “AI Workload Storage Considerations,” Tech Brief TB-10732-001, §2.2, p.4.

3. IEEE Computer, “Scaling Object Storage for Video Analytics,” vol 55, no 8, 2022, DOI:10.1109/MC.2022.3180935.

Before the NGC CLI can authenticate, the user must create an NGC API Key in the NGC web portal. This key is then supplied to ngc config set in order to gain access to private resources.

A – Installing the CLI alone provides no credentials.

B – Logging in with system username/password is unsupported; API keys are required.

D – Setting the NGC registry URL is secondary; access is denied without a key.

1. NVIDIA NGC CLI User Guide v3.27, “Generate Your API Key,” §2.1, pp. 4-5.

2. NVIDIA NGC Getting Started Guide, “Authenticate to NGC,” Rev. 2023-07, pp. 3-4.

3. MIT 6.S983 “Practical Deep-Learning Systems,” Lab 2 handout, §1.2 “Using NGC.”

When UEFI Secure Boot is enabled, the operating system kernel will only load modules that have a valid digital signature. The "permission denied" error occurs because the unsigned NVIDIA kernel modules are blocked from loading, making the GPU devices (/dev/nvidia) unavailable to any process, including containers. The correct resolution is to sign the NVIDIA kernel modules using a Machine Owner Key (MOK) that is enrolled in the system's firmware. This allows the kernel to trust and load the drivers, restoring GPU access while maintaining the integrity protections of Secure Boot.

Disabling Secure Boot: This is an insecure workaround that compromises the system's boot-time security posture, rather than correctly resolving the module signing issue.

Running containers in privileged mode: This does not solve the underlying problem, as the kernel has already blocked the necessary NVIDIA driver from loading.

Reinstalling the container runtime: The issue is at the kernel driver level, not with the container runtime (e.g., Docker, containerd), so reinstallation would have no effect.

1. NVIDIA CUDA Toolkit Documentation, Installation Guide for Linux, Section 7.3 "Install with DKMS and for Secure Boot". The guide details the process of enrolling a MOK and how the installer can automatically sign the kernel modules.

2. Ubuntu Documentation, "Kernel/SecureBoot". This official OS documentation explains the principles of Secure Boot and the requirement for signed kernel modules.

3. Red Hat Enterprise Linux Documentation, "Signing Kernel Modules for Secure Boot". This guide provides detailed steps for creating keys and signing third-party modules like the NVIDIA driver.

When upgrading complex, interdependent driver stacks like NVIDIA GPU drivers and DOCA software, especially on a system with a pre-existing custom configuration, the most critical step is to ensure a clean installation environment. Completely uninstalling all existing NVIDIA drivers (GPU, DOCA), related software packages (e.g., CUDA Toolkit), and any custom kernel modules is paramount. This "clean slate" approach prevents conflicts from leftover configuration files, mismatched library versions, or incompatible kernel module symbols. It is the most reliable method to ensure the new driver stack installs correctly and operates with stability.

A: Installing new drivers over old ones is a common cause of system instability due to file conflicts and is strongly discouraged in official documentation.

B: Recompiling custom modules is a necessary subsequent step, but it is not the most important initial action. The old drivers must be removed first.

C: A kernel update may be a prerequisite for the new drivers, but it does not resolve the primary risk of conflicts from the old driver stack.

1. NVIDIA DOCA Installation Guide for Linux (v2.0.2), Section 2.3: "Upgrading DOCA". The guide explicitly states: "Before installing a new version of DOCA, you must first uninstall the existing DOCA version from your setup." This highlights the necessity of a clean removal.

2. NVIDIA CUDA Toolkit Documentation, Installation Guide for Linux, Section 3.7: "Handle Conflicting Installation Methods". This section warns against mixing installation methods and recommends removing any previously installed drivers before proceeding with a new installation to avoid conflicts.

3. NVIDIA Driver Installation Quickstart Guide. The standard procedure for driver installation, particularly when troubleshooting, is to first purge any existing NVIDIA packages (e.g., using sudo apt-get --purge remove 'nvidia') to guarantee a clean state.

The output from the nvsm show healthinfo command is a diagnostic report for an NVIDIA DGX system. The line Verify installed GPU's..............................Unhealthy clearly indicates a problem within the GPU subsystem. The subsequent line, Missing GPU at PCI address '07:00.0', provides the specific detail that the system cannot detect a GPU at its expected physical location on the PCI bus. This points directly to a hardware issue, such as a failed GPU, a GPU that is not seated correctly in its slot, or a faulty riser card.

A software update is needed: The error message "Missing GPU" points to a hardware detection failure, not an issue that would typically be resolved by a software update.

The InfiniBand network is misconfigured: The output explicitly states Verify installed InfiniBand controllers....................Healthy, ruling out an InfiniBand issue.

The system BIOS is outdated: While the BIOS version is shown, the health check does not flag it as an issue. The primary, explicit error is the missing GPU.

1. NVIDIA DGX OS 5 User Guide (v5.2.0), Chapter 4: NVIDIA System Management (NVSM) Software, Section 4.4: nvsm show healthinfo. The documentation describes this command as the tool to "get a report on the health of the DGX system," where an "Unhealthy" status indicates a component failure.

2. NVIDIA DGX A100 System User Guide (DU-10226-001v5.0), Section 5.3.1: Checking System Health. This guide explains that nvsm show healthinfo checks for the presence and health of key components, including all expected GPUs.