Currently, there is no direct way to disable GPU memory preallocation in JAX without using environment variables. The recommended method involves setting the environment variable `XLA_PYTHON_CLIENT_PREALLOCATE` to `'false'` before importing JAX. This can be done either within your Python script using `os.environ['XLA_PYTHON_CLIENT_PREALLOCATE'] = 'false'` or by setting it externally before running your script with `export XLA_PYTHON_CLIENT_PREALLOCATE=false` (on Unix-like systems) or `set XLA_PYTHON_CLIENT_PREALLOCATE=false` (on Windows).
However, it's crucial to set these environment variables before importing JAX, as setting them afterward will have no effect[1][3]. There are no built-in JAX functions or configurations that allow you to disable preallocation without using environment variables.
If you encounter issues with disabling preallocation in newer versions of JAX, it might be due to a bug, and checking the latest documentation or release notes for updates is advisable[9].
Citations:[1] https://github.com/jax-ml/jax/discussions/21962
[2] https://learn.microsoft.com/en-us/windows/win32/msi/environment-table
[3] https://jax.readthedocs.io/en/latest/gpu_memory_allocation.html
[4] https://docs.jax.dev/en/latest/jep/4410-omnistaging.html
[5] https://kolonist26-jax-kr.readthedocs.io/en/latest/gpu_memory_allocation.html
[6] https://developer.jboss.org/thread/215838
[7] https://crikit.science/documentation/ad_systems/
[8] https://docs.nvidia.com/deeplearning/dali/user-guide/docs/advanced_topics_performance_tuning.html
[9] https://github.com/jax-ml/jax/discussions/19014