WEC-Eng

一份从英语维基百科中提取的大规模跨文档事件共指数据集。

• 存储库（生成WEC的代码）： https://github.com/AlonEirew/extract-wec
• 论文： https://aclanthology.org/2021.naacl-main.198/

语言

英语

加载数据集

您可以使用huggingface_hub库按如下方式读取WEC-Eng文件：

from huggingface_hub import hf_hub_url, cached_download
import json
REPO_ID = "datasets/Intel/WEC-Eng"
splits_files = ["Dev_Event_gold_mentions_validated.json",
                "Test_Event_gold_mentions_validated.json",
                "Train_Event_gold_mentions.json"]
wec_eng = list()
for split_file in splits_files:
    wec_eng.append(json.load(open(cached_download(
        hf_hub_url(REPO_ID, split_file)), "r")))

数据集结构

数据拆分

• 最终版本的英语跨文档事件共指数据集
  • 训练集 - Train_Event_gold_mentions.json
  • 开发集 - Dev_Event_gold_mentions_validated.json
  • 测试集 - Test_Event_gold_mentions_validated.json

• 非（在群集内部）受控版本的数据集（词汇多样性）
  • 所有（实验性） - All_Event_gold_mentions_unfiltered.json

数据实例

{
        "coref_chain": 2293469,
        "coref_link": "Family Values Tour 1998",
        "doc_id": "House of Pain",
        "mention_context": [
            "From",
            "then",
            "on",
            ",",
            "the",
            "members",
            "continued",
            "their"
  ],
  "mention_head": "Tour",
  "mention_head_lemma": "Tour",
  "mention_head_pos": "PROPN",
  "mention_id": "108172",
  "mention_index": 1,
  "mention_ner": "UNK",
  "mention_type": 8,
  "predicted_coref_chain": null,
  "sent_id": 2,
  "tokens_number": [
    50,
    51,
    52,
    53
  ],
  "tokens_str": "Family Values Tour 1998",
  "topic_id": -1
}

数据字段

引用

@inproceedings{eirew-etal-2021-wec,
    title = "{WEC}: Deriving a Large-scale Cross-document Event Coreference dataset from {W}ikipedia",
    author = "Eirew, Alon  and
      Cattan, Arie  and
      Dagan, Ido",
    booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
    month = jun,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.naacl-main.198",
    doi = "10.18653/v1/2021.naacl-main.198",
    pages = "2498--2510",
    abstract = "Cross-document event coreference resolution is a foundational task for NLP applications involving multi-text processing. However, existing corpora for this task are scarce and relatively small, while annotating only modest-size clusters of documents belonging to the same topic. To complement these resources and enhance future research, we present Wikipedia Event Coreference (WEC), an efficient methodology for gathering a large-scale dataset for cross-document event coreference from Wikipedia, where coreference links are not restricted within predefined topics. We apply this methodology to the English Wikipedia and extract our large-scale WEC-Eng dataset. Notably, our dataset creation method is generic and can be applied with relatively little effort to other Wikipedia languages. To set baseline results, we develop an algorithm that adapts components of state-of-the-art models for within-document coreference resolution to the cross-document setting. Our model is suitably efficient and outperforms previously published state-of-the-art results for the task.",
}

许可协议

我们在 Creative Commons Attribution-ShareAlike 3.0 Unported License 下提供以下数据集。它基于维基百科中基于创作共用属性-相同方式共享3.0许可协议的内容。

联系方式

如果您有任何问题，请在 https://github.com/AlonEirew/extract-wec 处创建一个Github问题。