A Stable Numerical Computation

Photo by Patrick Fore on Unsplash

First, we implement the Log-Sum-Exp (LSE) in Python.

In this example, we have tensors that have overflowed while computing LSE.

Now, we use the trick.

Finally, we use PyTorch’s implementation of LSE for validation.

As you can see, the outputs are equal.