Electra small ⚡ + SQuAD v2 ❓

Electra-small-discriminator fine-tuned on SQUAD v2.0 dataset for Q&A downstream task.

Details of the downstream task (Q&A) - Model ?

ELECTRA是一种用于自监督语言表示学习的新方法。它可以使用相对较少的计算资源对Transformer网络进行预训练。ELECTRA模型的训练目标是区分由另一个神经网络生成的“真实”输入标记和“伪造”输入标记，类似于一个 GAN 的鉴别器。即使在小规模训练时，ELECTRA也能够取得出色的结果，即使只在一台GPU上进行训练。在大规模训练时，ELECTRA在 SQuAD 2.0 数据集上取得了最先进的结果。

Details of the downstream task (Q&A) - Dataset ?

SQuAD2.0将SQuAD1.1中的10万个问题与由众包工作者编写的超过5万个类似于可回答问题但无法回答的问题进行了合并。要在SQuAD2.0上表现良好，系统不仅需要在可能时回答问题，还需要确定段落不支持任何答案并且放弃回答。

Model training ?️‍

该模型是在一台Tesla P100 GPU和25GB的RAM上通过以下命令进行训练的:

python transformers/examples/question-answering/run_squad.py \
  --model_type electra \
  --model_name_or_path 'google/electra-small-discriminator' \
  --do_eval \
  --do_train \
  --do_lower_case \
  --train_file '/content/dataset/train-v2.0.json' \
  --predict_file '/content/dataset/dev-v2.0.json' \
  --per_gpu_train_batch_size 16 \
  --learning_rate 3e-5 \
  --num_train_epochs 10 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir '/content/output' \
  --overwrite_output_dir \
  --save_steps 1000 \
  --version_2_with_negative

Test set Results ?

{
'exact': 69.71279373368147,
'f1': 73.4439546123672,
'total': 11873,
'HasAns_exact': 69.92240215924427,
'HasAns_f1': 77.39542393937836,
'HasAns_total': 5928,
'NoAns_exact': 69.50378469301934,
'NoAns_f1': 69.50378469301934,
'NoAns_total': 5945,
'best_exact': 69.71279373368147,
'best_exact_thresh': 0.0,
'best_f1': 73.44395461236732,
'best_f1_thresh': 0.0
}

Model in action ?

使用pipelines进行快速使用:

from transformers import pipeline
QnA_pipeline = pipeline('question-answering', model='mrm8488/electra-base-finetuned-squadv2')
QnA_pipeline({
    'context': 'A new strain of flu that has the potential to become a pandemic has been identified in China by scientists.',
    'question': 'What has been discovered by scientists from China ?'
})
# Output:
{'answer': 'A new strain of flu', 'end': 19, 'score': 0.8650811568752914, 'start': 0}

创建者 Manuel Romero/@mrm8488 | LinkedIn

Made with ♥ in Spain