sloberta-sentinews-sentence

斯洛文尼亚3类情感分类器 - 在SentiNews数据集的句子级别配置上进行了 SloBERTa 微调。

该模型用作：(1)开箱即用的句子级别情感分类器或(2)句子级别情感分类的基准。

微调细节

该模型在 cjvt/sentinews 数据集的句子级别配置的随机90%/5%/5%的训练-验证-测试分割上进行微调，使用以下超参数：

max_length = 79  # 99th percentile of encoded training sequences, sequences are padded/truncated to this length
batch_size = 128
optimizer = "adamw_torch"
learning_rate = 2e-5
num_epochs = 10
validation_metric = "macro_f1"

可以随意检查training_args.bin以获取更多详细信息。

如果您希望直接将您的模型与此模型进行比较，您应该使用与该模型相同的分割。要做到这一点，请使用以下代码：

import json
import datasets

# You can find split_indices.json in the 'Files and versions' tab 
with open("split_indices.json", "r") as f_split:
  split = json.load(f_split)

data = datasets.load_dataset("cjvt/sentinews", "sentence_level", split="train")
train_data = data.select(split["train_indices"])
dev_data = data.select(split["dev_indices"])
test_data = data.select(split["test_indices"])

评估结果

最佳验证集结果：

{
  "eval_accuracy": 0.7207815275310835,
  "eval_f1_macro": 0.6934678744913757,
  "eval_f1_negative": 0.7042136003337507,
  "eval_f1_neutral": 0.759215853398679,
  "eval_f1_positive": 0.6169741697416974,
  "eval_loss": 0.6337869167327881,
  "eval_precision_negative": 0.6685148514851486,
  "eval_precision_neutral": 0.7752393385552655,
  "eval_precision_positive": 0.6314199395770392,
  "eval_recall_negative": 0.74394006170119,
  "eval_recall_neutral": 0.7438413361169103,
  "eval_recall_positive": 0.6031746031746031
}

测试集结果：

{
  "test_loss": 0.6395984888076782,
  "test_accuracy": 0.7158081705150977,
  "test_precision_negative": 0.6570397111913358,
  "test_recall_negative": 0.7292965271593945,
  "test_f1_negative": 0.6912850812407682,
  "test_precision_neutral": 0.7748017998714377,
  "test_recall_neutral": 0.7418957734919983,
  "test_f1_neutral": 0.7579918247563149,
  "test_precision_positive": 0.6155642023346304,
  "test_recall_positive": 0.5969811320754717,
  "test_f1_positive": 0.6061302681992337,
  "test_f1_macro": 0.6851357247321056,
}