Enabling or disabling CPU mitigations on the NVIDIA DGX A100 involves managing security updates that mitigate CPU speculative side-channel vulnerabilities. These mitigations can impact performance, particularly in deep learning and machine learning workloads. Here's how to manage them:
Determining the CPU Mitigation State
To check if CPU mitigations are enabled or disabled on your DGX A100 system, use the following command:
bash
$ cat /sys/devices/system/cpu/vulnerabilities/*
- Enabled Mitigations: The output will show multiple lines prefixed with `Mitigation:`, indicating that mitigations are active.
- Disabled Mitigations: The output will show multiple lines prefixed with `Vulnerable`, indicating that mitigations are disabled.
Disabling CPU Mitigations
If you have other security measures in place, such as cluster-level mitigations, you might want to disable CPU mitigations to improve performance. Hereâs how:
1. Install the `nv-mitigations-off` Package:
bash
$ sudo apt install nv-mitigations-off -y
2. Reboot the System:
After installing the package, reboot your system to apply the changes.
3. Verify Mitigations Are Disabled:
Run the command again to check the mitigation state:
bash
$ cat /sys/devices/system/cpu/vulnerabilities/*
The output should now show lines prefixed with `Vulnerable`.
Re-enabling CPU Mitigations
If you need to re-enable CPU mitigations for security reasons, follow these steps:
1. Remove the `nv-mitigations-off` Package:
bash
$ sudo apt purge nv-mitigations-off
2. Reboot the System:
Reboot your system to apply the changes.
3. Verify Mitigations Are Enabled:
Use the command again to check the mitigation state:
bash
$ cat /sys.devices/system/cpu/vulnerabilities/*
The output should now show lines prefixed with `Mitigation:`.
**Caution: Disabling CPU mitigations reduces security protections against speculative side-channel vulnerabilities. Ensure you have alternative security measures in place before disabling these mitigations.
Citations:
[1] http://cdn.cnetcontent.com/2f/68/2f6888a0-063f-4d76-94e4-8666b7619dfd.pdf
[2] https://www.manualslib.com/products/Nvidia-Dgx-A100-11150393.html
[3] https://www.manualslib.com/manual/3661402/Nvidia-Dgx-B200.html
[4] https://www.reddit.com/r/intel/comments/btkipd/how_to_disable_all_mitigations/
[5] https://www.sec.gov/Archives/edgar/data/1045810/000104581025000023/nvda-20250126.htm
[6] https://www.redbooks.ibm.com/redbooks/pdfs/sg248538.pdf
[7] https://fosspost.org/disable-cpu-mitigations-on-linux
[8] https://proceedings.mlsys.org/paper_files/paper/2023/file/24e845415c1486dd2d582a9d639237f9-Paper-mlsys2023.pdf
[9] https://www.manualslib.com/manual/1925509/Nvidia-Dgx-A100.html
[10] https://www.reddit.com/r/SteamDeck/comments/1gimukq/disabling_cpu_mitigations_for_a_free_performance/
[11] https://docs.nvidia.com/dgx/dgxa100-user-guide/quick-start.html