`num_return_sequences` has to be smaller or equal to `num_beams`.
Package:
transformers
50617

Exception Class:
ValueError
Raise code
eos_token_id=eos_token_id,
**model_kwargs,
)
elif num_beams > 1:
length_penalty = length_penalty if length_penalty is not None else self.config.length_penalty
early_stopping = early_stopping if early_stopping is not None else self.config.early_stopping
if num_return_sequences > num_beams:
raise ValueError("`num_return_sequences` has to be smaller or equal to `num_beams`.")
beam_scorer = BeamSearchScorer(
batch_size=batch_size,
num_beams=num_beams,
device=self.device,
length_penalty=length_penalty,
do_early_stopping=early_stopping,
num_beam_hyps_to_keep=num_return_sequences,
🙏 Scream for help to Ukraine
Today, 3rd July 2022, Russia continues bombing and firing Ukraine. Don't trust Russia, they are bombing us and brazenly lying in same time they are not doing this 😠, civilians and children are dying too!
We are screaming and asking exactly you to help us, we want to survive, our families, children, older ones.
Please spread the information, and ask your governemnt to stop Russia by any means. We promise to work extrahard after survival to make the world safer place for all.
Please spread the information, and ask your governemnt to stop Russia by any means. We promise to work extrahard after survival to make the world safer place for all.
Links to the raise (3)
https://github.com/huggingface/transformers/blob/bd9871657bb9500a9f4437a873db6df5f1ae6dbb/src/transformers/models/rag/modeling_rag.py#L1550 https://github.com/huggingface/transformers/blob/bd9871657bb9500a9f4437a873db6df5f1ae6dbb/src/transformers/generation_utils.py#L1019 https://github.com/huggingface/transformers/blob/bd9871657bb9500a9f4437a873db6df5f1ae6dbb/src/transformers/generation_utils.py#L1096Ways to fix
The generate
method of a pretrained T5ForConditionalGeneration
model generates sequences for models with a language modeling head. When calling this method on initialized model the parameter num_return_sequences
which is used to specify the number of independently computed returned sequences for each element in the batch should be smaller or equal to parameternum_beans.
If a value greater than the num_beams
is given This particular error is raised.
How to reproduce the error:
- Setup and installation
pip install --user pipenv
mkdir test_folder
cd test_folder
pipenv shell
pipenv install transformers
- Run Sample code
from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("t5-base")
model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")
document = ("at least two people were killed in a suspected bomb attack on a passenger bus "
"in the strife-torn southern philippines on monday , the military said."
)
input_ids = tokenizer(document, return_tensors="pt").input_ids
# generate 3 independent sequences using beam search decoding (5 beams)
# with T5 encoder-decoder model conditioned on short news article.
outputs = model.generate(input_ids=input_ids, num_beams=5,num_return_sequences=6)
print("\n\n Generated:", tokenizer.batch_decode(outputs, skip_special_tokens=True))
Output:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-3-4fdd15f341c2> in <module>()
8 # generate 3 independent sequences using beam search decoding (5 beams)
9 # with T5 encoder-decoder model conditioned on short news article.
---> 10 outputs = model.generate(input_ids=input_ids, num_beams=5,num_return_sequences=6)
11 print("\n\n Generated:", tokenizer.batch_decode(outputs, skip_special_tokens=True))
/usr/local/lib/python3.7/dist-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
26 def decorate_context(*args, **kwargs):
27 with self.__class__():
---> 28 return func(*args, **kwargs)
29 return cast(F, decorate_context)
30
/usr/local/lib/python3.7/dist-packages/transformers/generation_utils.py in generate(self, input_ids, max_length, min_length, do_sample, early_stopping, num_beams, temperature, top_k, top_p, repetition_penalty, bad_words_ids, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, encoder_no_repeat_ngram_size, num_return_sequences, max_time, max_new_tokens, decoder_start_token_id, use_cache, num_beam_groups, diversity_penalty, prefix_allowed_tokens_fn, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, forced_bos_token_id, forced_eos_token_id, remove_invalid_values, synced_gpus, **model_kwargs)
1034
1035 if num_return_sequences > num_beams:
-> 1036 raise ValueError("`num_return_sequences` has to be smaller or equal to `num_beams`.")
1037
1038 if stopping_criteria.max_length is None:
ValueError: `num_return_sequences` has to be smaller or equal to `num_beams`.
How to fix the error:
Make sure num_return_sequences
is smaller or equal to n
um_beams when calling the generate method.
from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("t5-base")
model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")
document = ("at least two people were killed in a suspected bomb attack on a passenger bus "
"in the strife-torn southern philippines on monday , the military said."
)
input_ids = tokenizer(document, return_tensors="pt").input_ids
# generate 3 independent sequences using beam search decoding (5 beams)
# with T5 encoder-decoder model conditioned on short news article.
outputs = model.generate(input_ids=input_ids, num_beams=5,num_return_sequences=3)
print("\n\n Generated:", tokenizer.batch_decode(outputs, skip_special_tokens=True))
Correct Output:
/usr/local/lib/python3.7/dist-packages/torch/_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at /pytorch/aten/src/ATen/native/BinaryOps.cpp:467.)
return torch.floor_divide(self, other)
Generated: ['at least two people were killed in a suspected bomb attack on a passenger bus in the', 'at least two people were killed in a suspected bomb attack on a passenger bus.', 'at least two people were killed in a suspected bomb attack on a passenger bus on mon']
Add a possible fix
Please authorize to post fix