中文

t5-sl-large

t5-sl-large model is a Slovene T5 model. It has 24 encoder and 24 decoder layers, in total about 750 million parameters. It was trained for 3 epochs on the following corpora:

Corpora

The following corpora were used for training the model:

  • Gigafida 2.0
  • Kas 1.0
  • Janes 1.0 (only Janes-news, Janes-forum, Janes-blog, Janes-wiki subcorpora)
  • Slovenian parliamentary corpus siParl 2.0
  • slWaC

Evaluation

The model is described in detail and evaluated in our paper " Sequence to sequence pretraining for a less-resourced Slovenian language "