- Platform: Amazon Sagemaker
- Python version: 3.7
- Tensorflow version (GPU?):2.3
- Using distributed or parallel set-up in script?: Unable to implement it properly.
Who can help
Unable to reduce the summarization time.
The tasks I am working on is:
I am using pretrained transformer of T5(
TFT5ForConditionalGeneration) for text summarization.
inputs = tokenizer("summarize: " + text, return_tensors="tf").input_ids outputs = model.generate( inputs, max_length=200, min_length=5, num_beams=5,)
I tried to use distributed strategy of tensorflow. But it doesn’t made any improvement.
strategy = tf.distribute.MirroredStrategy() strategy = tf.distribute.MirroredStrategy(devices=["/gpu:0", "/gpu:1"])
I am hoping that if we increase the number of GPU, time must be reduced. But It is not happening in this case.