proj_size has to be smaller than hidden_size

Package:

torch

50580

Exception Class:

ValueError

Raise code

            warnings.warn("dropout option adds dropout after all but last "
                          "recurrent layer, so non-zero dropout expects "
                          "num_layers greater than 1, but got dropout={} and "
                          "num_layers={}".format(dropout, num_layers))
        if proj_size < 0:
            raise ValueError("proj_size should be a positive integer or zero to disable projections")
        if proj_size >= hidden_size:
            raise ValueError("proj_size has to be smaller than hidden_size")

        if mode == 'LSTM':
            gate_size = 4 * hidden_size
        elif mode == 'GRU':
            gate_size = 3 * hidden_size
        elif mode == 'RNN_TANH':
            gate_size = hidden_size

Links to the raise (1)

https://github.com/pytorch/pytorch/blob/e56d3b023818f54553f2dc5d30b6b7aaf6b6a325/torch/nn/modules/rnn.py#L69

Ways to fix

Summary:

When creating an instance of the RNNBase class, you must pass in the mode, input_size, and hidden_size. Those 2 sizes must be ints. There is an additional optional parameter named proj_size which defaults to 0. If a value is manually passed into proj_size and it's larger than the value of hidden_size, you will get this exception. To avoid this exception, ensure the value of proj_size is smaller than hidden_size.

Code to Reproduce the Error (WRONG):

import torch.nn.modules.rnn as rnn
base = rnn.RNNBase('LSTM', 5, 5, proj_size=6)

Working Version (Fixed):

import torch.nn.modules.rnn as rnn
base = rnn.RNNBase('LSTM', 5, 5, proj_size=4)

Jul 08, 2021

codingcomedyig 7.9k

Add a possible fix

Please authorize to post fix