`num_return_sequences` has to be smaller or equal to `num_beams`.
Package:
transformers
50617

Exception Class:
ValueError
Raise code
eos_token_id=eos_token_id,
**model_kwargs,
)
elif num_beams > 1:
length_penalty = length_penalty if length_penalty is not None else self.config.length_penalty
early_stopping = early_stopping if early_stopping is not None else self.config.early_stopping
if num_return_sequences > num_beams:
raise ValueError("`num_return_sequences` has to be smaller or equal to `num_beams`.")
beam_scorer = BeamSearchScorer(
batch_size=batch_size,
num_beams=num_beams,
device=self.device,
length_penalty=length_penalty,
do_early_stopping=early_stopping,
num_beam_hyps_to_keep=num_return_sequences,
Links to the raise (3)
https://github.com/huggingface/transformers/blob/bd9871657bb9500a9f4437a873db6df5f1ae6dbb/src/transformers/models/rag/modeling_rag.py#L1550 https://github.com/huggingface/transformers/blob/bd9871657bb9500a9f4437a873db6df5f1ae6dbb/src/transformers/generation_utils.py#L1019 https://github.com/huggingface/transformers/blob/bd9871657bb9500a9f4437a873db6df5f1ae6dbb/src/transformers/generation_utils.py#L1096Ways to fix
The generate
method of a pretrained T5ForConditionalGeneration
model generates sequences for models with a language modeling head. When calling this method on initialized model the parameter num_return_sequences
which is used to specify the number of independently computed returned sequences for each element in the batch should be smaller or equal to parameternum_beans.
If a value greater than the num_beams
is given This particular error is raised.
How to reproduce the error:
- Setup and installation
pip install --user pipenv
mkdir test_folder
cd test_folder
pipenv shell
pipenv install transformers
- Run Sample code
from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("t5-base")
model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")
document = ("at least two people were killed in a suspected bomb attack on a passenger bus "
"in the strife-torn southern philippines on monday , the military said."
)
input_ids = tokenizer(document, return_tensors="pt").input_ids
# generate 3 independent sequences using beam search decoding (5 beams)
# with T5 encoder-decoder model conditioned on short news article.
outputs = model.generate(input_ids=input_ids, num_beams=5,num_return_sequences=6)
print("\n\n Generated:", tokenizer.batch_decode(outputs, skip_special_tokens=True))
Output:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-3-4fdd15f341c2> in <module>()
8 # generate 3 independent sequences using beam search decoding (5 beams)
9 # with T5 encoder-decoder model conditioned on short news article.
---> 10 outputs = model.generate(input_ids=input_ids, num_beams=5,num_return_sequences=6)
11 print("\n\n Generated:", tokenizer.batch_decode(outputs, skip_special_tokens=True))
/usr/local/lib/python3.7/dist-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
26 def decorate_context(*args, **kwargs):
27 with self.__class__():
---> 28 return func(*args, **kwargs)
29 return cast(F, decorate_context)
30
/usr/local/lib/python3.7/dist-packages/transformers/generation_utils.py in generate(self, input_ids, max_length, min_length, do_sample, early_stopping, num_beams, temperature, top_k, top_p, repetition_penalty, bad_words_ids, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, encoder_no_repeat_ngram_size, num_return_sequences, max_time, max_new_tokens, decoder_start_token_id, use_cache, num_beam_groups, diversity_penalty, prefix_allowed_tokens_fn, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, forced_bos_token_id, forced_eos_token_id, remove_invalid_values, synced_gpus, **model_kwargs)
1034
1035 if num_return_sequences > num_beams:
-> 1036 raise ValueError("`num_return_sequences` has to be smaller or equal to `num_beams`.")
1037
1038 if stopping_criteria.max_length is None:
ValueError: `num_return_sequences` has to be smaller or equal to `num_beams`.
How to fix the error:
Make sure num_return_sequences
is smaller or equal to n
um_beams when calling the generate method.
from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("t5-base")
model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")
document = ("at least two people were killed in a suspected bomb attack on a passenger bus "
"in the strife-torn southern philippines on monday , the military said."
)
input_ids = tokenizer(document, return_tensors="pt").input_ids
# generate 3 independent sequences using beam search decoding (5 beams)
# with T5 encoder-decoder model conditioned on short news article.
outputs = model.generate(input_ids=input_ids, num_beams=5,num_return_sequences=3)
print("\n\n Generated:", tokenizer.batch_decode(outputs, skip_special_tokens=True))
Correct Output:
/usr/local/lib/python3.7/dist-packages/torch/_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at /pytorch/aten/src/ATen/native/BinaryOps.cpp:467.)
return torch.floor_divide(self, other)
Generated: ['at least two people were killed in a suspected bomb attack on a passenger bus in the', 'at least two people were killed in a suspected bomb attack on a passenger bus.', 'at least two people were killed in a suspected bomb attack on a passenger bus on mon']
Add a possible fix
Please authorize to post fix