To delay inequality comparisons to compilation time in JAX, you can use certain strategies that involve rewriting your code to utilize JAX's symbolic operations. Here are some methods to achieve this:
1. Use `core.max_dim` and `core.min_dim`: Instead of using built-in `max` or `min` functions, or `np.max` and `np.min`, you can replace them with `core.max_dim` and `core.min_dim`. These functions delay the inequality comparison until compilation time when shapes become known. For example, instead of `d if d > 0 else 0`, you can write `core.max_dim(d, 0)`.
2. Rewrite Conditionals: Try to rewrite conditionals using `core.max_dim` and `core.min_dim`. This approach helps in delaying the evaluation of inequalities until the shapes are determined at compile time.
3. Avoid Integer Dependencies: Rely on the fact that symbolic dimensions can be treated as integers for most arithmetic operations. Instead of `int(d) + 5`, write `d + 5`. This reduces the dependency on integer dimensions and allows for more flexible compilation.
4. Specify Symbolic Constraints: If possible, specify symbolic constraints to guide JAX in handling inequalities during compilation. This can help in optimizing the code by delaying comparisons until necessary.
By employing these strategies, you can effectively delay inequality comparisons to compilation time in JAX, which can improve the efficiency and flexibility of your code.
Citations:[1] https://inria.hal.science/hal-03431315v2/document
[2] https://docs.jax.dev/en/latest/jit-compilation.html
[3] https://github.com/jax-ml/jax/blob/main/CHANGELOG.md
[4] https://docs.jax.dev/en/latest/faq.html
[5] https://arxiv.org/abs/1204.1069
[6] https://github.com/google/jax/blob/main/jax/experimental/jax2tf/README.md
[7] https://arxiv.org/abs/1602.05281
[8] https://stackoverflow.com/questions/69070804/how-to-reduce-jax-compile-time-when-using-for-loop