Home Arrow Icon Knowledge base Arrow Icon Global Arrow Icon How does the allow_empty_variables() method work in Sonnet


How does the allow_empty_variables() method work in Sonnet


The `allow_empty_variables()` context manager in Sonnet allows you to suppress the exception raised when accessing variables before they are created in a module.

By default, when you access the `variables` or `trainable_variables` properties of a Sonnet module, Sonnet will raise an exception if the result is empty[1]. This is because most Sonnet modules create variables lazily, for example the first time they are called.

To mitigate this common error, you can use the `allow_empty_variables()` context manager:

python
with snt.allow_empty_variables():
    variables = self.trainable_variables

Inside the context manager, accessing `variables` or `trainable_variables` will not raise an exception if the module has not created any variables yet[1].

This is useful when you need to access variables before they are created, such as during module construction or in the `__init__` method[2].

For example:

python
class MyModule(snt.Module):
    def __init__(self, output_size, name=None):
        super().__init__(name=name)
        self.output_size = output_size
        with snt.allow_empty_variables():
            self.w = tf.Variable(tf.random.normal([100, output_size]), name="w")
            self.b = tf.Variable(tf.zeros([output_size]), name="b")
            
    def __call__(self, x):
        return tf.matmul(x, self.w) + self.b

In this example, we create the module variables `w` and `b` in the `__init__` method using `allow_empty_variables()`, so they can be accessed later in the `__call__` method when the module is used[2].

So in summary, `allow_empty_variables()` allows you to access module variables without triggering an exception if they haven't been created yet, which is useful for initializing variables during module construction.

Citations:
[1] https://github.com/deepmind/sonnet/blob/v2/sonnet/src/base.py
[2] https://github.com/google-deepmind/sonnet/issues/128
[3] https://sonnet.readthedocs.io/en/latest/api.html
[4] https://github.com/google-deepmind/sonnet/issues/69
[5] https://www.sonnetsoftware.com/support/help-17/Sonnet_Suites/..%5Cusers_guide/HowtoCreateaVariable.html