PyTorch Hooks by Example

Photo by Tatiana Rodriguez on Unsplash

Hooks allow us to look at data during the forward and backward pass.
register_forward_hook() can get the input and output data of a module after the forward pass. So if we have a model the looks like this:

>>> model
CustomLayer(
(linear1): Linear(in_features=1000, out_features=100, bias=True)
(linear2): Linear(in_features=100, out_features=10, bias=True)
)

and we want to know the output of linear1. We create a hook with the following signature (module, input, and output):

>>> dummylist = []
>>> def dummyhook(module, inp, outp): dummylist.append(outp)

Then, we call the hook on a module using the following:

>>> model.linear1.register_forward_hook(dummyhook);

Finally, run the model and then inspect the dummy list.

>>> y_pred = model(x)
>>> dummylist[0]
tensor([[-0.0116, -0.4900, -1.0557, ..., 0.7317, 2.4810, -0.0821],
[ 0.3536, -0.0983, -0.7922, ..., -0.8643, 1.5925, -0.1461],
[ 0.3246, -0.1753, -0.1105, ..., -0.4116, -0.5870, 0.9387],
...,
[ 0.3176, -1.0613, -0.1252, ..., -0.5476, -0.4361, -0.2262],
[ 2.0090, 2.9247, -0.5890, ..., -0.1938, 0.6834, -0.0979],
[-0.5889, 2.3918, 0.2750, ..., 1.5354, -0.0317, 0.9574]],
grad_fn=<AddmmBackward>)

Now, you might ask, where can I use this? one use case is to peak into your conventional neural network by generating a heat map of each layer.

--

--

--

https://github.com/booletic

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Setup Ghost Blog in a Subdirecotory over SSL on NGINX

Pokie Megaways Slot Review

Pokie Megaways Slot Review

Advantage of Using the ITSM Managed IT Services for Your Organization

Getter and Setter methods in Python

Consistency and Duplication in a distributed system (What is the protocol MOLD needs?)

Data Structures Part 2: Stack, Queue, and Deque ADTs

[Twitch][1/n] Requirements

Hyperautomation for the Call Center Takes More Than Just RPA

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Mansoor Aldosari

Mansoor Aldosari

https://github.com/booletic

More from Medium

Point Cloud Clustering in C++

Python pitfalls for functional programmers

Automating Cheminformatics with Apache Airflow — Step 1: Preparing SMILES Transformations on Large…

Syntactic Consistency/Uniformity