Home Arrow Icon Knowledge base Arrow Icon Global Arrow Icon What are the environmental variables used to configure JAX memory allocation


What are the environmental variables used to configure JAX memory allocation


To configure JAX memory allocation, several environment variables can be used:

1. `XLA_PYTHON_CLIENT_PREALLOCATE`: This variable controls whether JAX preallocates GPU memory. Setting it to `false` disables preallocation, causing JAX to allocate memory as needed. However, this can lead to memory fragmentation and may not always work as expected in some versions of JAX[1][3][5].

2. `XLA_PYTHON_CLIENT_MEM_FRACTION`: This variable specifies the percentage of total GPU memory that JAX should preallocate. For example, setting it to `.10` will preallocate 10% of the GPU memory. This can help avoid out-of-memory errors but may lead to OOM warnings if not managed properly[1][3][5].

3. `XLA_PYTHON_CLIENT_ALLOCATOR=platform`: This setting makes JAX allocate exactly what is needed on demand and deallocate memory that is no longer needed. While it is not recommended for general use due to its slow performance, it can be useful for debugging or minimizing GPU memory usage[1][3].

These variables can be set before running your JAX application to customize its memory allocation behavior. However, some users have reported inconsistent behavior with these settings in certain versions of JAX[5][9].

Citations:
[1] https://kolonist26-jax-kr.readthedocs.io/en/latest/gpu_memory_allocation.html
[2] https://docs.nvidia.com/deeplearning/dali/user-guide/docs/advanced_topics_performance_tuning.html
[3] https://jax.readthedocs.io/en/latest/gpu_memory_allocation.html
[4] https://docs.jax.dev/en/latest/installation.html
[5] https://github.com/google/jax/issues/19267
[6] https://www.maplesoft.com/support/help/Maple/view.aspx?path=envvar
[7] https://docs.jax.dev/en/latest/faq.html
[8] https://github.com/google/jax/issues/1491
[9] https://github.com/google/jax/issues/19035
[10] https://stackoverflow.com/questions/73322760/jax-gpu-memory-usage-even-with-cpu-allocation