数据集:

offenseval_dravidian

计算机处理:

multilingual

语言创建人:

crowdsourced

批注创建人:

expert-generated

源数据集:

original

许可:

cc-by-4.0
英文

Offenseval Dravidian 数据集卡片

数据集概述

冒犯性语言识别是自然语言处理 (NLP) 领域的分类任务,其目标是在社交媒体中中和和最小化冒犯内容。在过去的二十年中,这一领域在学术界和工业界都是一个活跃的研究领域。社交媒体文本中的冒犯语言识别的需求不断增长,这些文本在很大程度上是混合编码的。混合编码是一个多语言社区中普遍存在的现象,混合编码的文本有时以非本族语言的脚本书写。由于文本中不同语言层次上的代码切换的复杂性,单语言训练的系统在混合编码数据上通常会失败。本任务提供了一种针对德拉维达语 (泰米尔语-英语,马拉雅拉姆语-英语和卡纳达语-英语) 中的混合编码文本进行冒犯语言识别的新的黄金标准语料库。

支持的任务和排行榜

本任务的目标是识别社交媒体中收集的德拉维达语 (泰米尔语-英语,马拉雅拉姆语-英语和卡纳达语-英语) 的混合编码评论/帖子数据集中的冒犯性语言内容。评论/帖子可能包含多个句子,但语料库的平均句子长度为1。每个评论/帖子在评论/帖子级别上进行了注释。该数据集还存在类别不平衡的问题,反映了现实世界的情况。

语言

德拉维达语 (泰米尔语-英语,马拉雅拉姆语-英语和卡纳达语-英语) 中的混合编码文本。

数据集结构

数据实例

泰米尔语数据集的一个示例如下:

text label
படம் கண்டிப்பாக வெற்றி பெற வேண்டும் செம்ம vara level Not_offensive
Avasara patutiya editor uhh antha bullet sequence aa nee soliruka kudathu, athu sollama iruntha movie ku konjam support aa surprise element aa irunthurukum Not_offensive

马拉雅拉姆语数据集的一个示例如下:

text label
ഷൈലോക്ക് ന്റെ നല്ല ടീസർ ആയിട്ട് പോലും ട്രോളി നടന്ന ലാലേട്ടൻ ഫാൻസിന് കിട്ടിയൊരു നല്ലൊരു തിരിച്ചടി തന്നെ ആയിരിന്നു ബിഗ് ബ്രദർ ന്റെ ട്രെയ്‌ലർ Not_offensive
Marana mass Ekka kku kodukku oru Not_offensive

卡纳达语数据集的一个示例如下:

text label
ನಿಜವಾಗಿಯೂ ಅದ್ಭುತ heartly heltidini... plz avrigella namma nimmellara supprt beku Not_offensive
Next song gu kuda alru andre evaga yar comment madidera alla alrru like madi share madi nam industry na next level ge togond hogaona. Not_offensive

数据字段

泰米尔语

  • text:泰米尔语-英语混合编码的评论。
  • label:取值为0到5的整数,对应以下值:"Not_offensive","Offensive_Untargetede","Offensive_Targeted_Insult_Individual","Offensive_Targeted_Insult_Group","Offensive_Targeted_Insult_Other","not-Tamil"。

马来语

  • text:马来语-英语混合编码的评论。
  • label:取值为0到5的整数,对应以下值:"Not_offensive","Offensive_Untargetede","Offensive_Targeted_Insult_Individual","Offensive_Targeted_Insult_Group","Offensive_Targeted_Insult_Other","not-malayalam"。

卡纳达语

  • text:卡纳达语-英语混合编码的评论。
  • label:取值为0到5的整数,对应以下值:"Not_offensive","Offensive_Untargetede","Offensive_Targeted_Insult_Individual","Offensive_Targeted_Insult_Group","Offensive_Targeted_Insult_Other","not-Kannada"。

数据切分

train validation
Tamil 35139 4388
Malayalam 16010 1999
Kannada 6217 777

数据集创建

策划理由

有一个不断增长的需求,希望能够识别社交媒体文本,尤其是混合编码文本中的冒犯语言。混合编码是多语言社区中普遍存在的现象,混合编码的文本有时以非本族语言的脚本书写。由于文本中不同语言层次上的代码切换的复杂性,单语言训练的系统在混合编码的数据上通常会失败。

源数据

初始数据收集和归一化。

[需要更多信息]

谁是源语言生产者?

YouTube用户

注释

注释过程

[需要更多信息]

谁是注释者?

[需要更多信息]

个人和敏感信息

[需要更多信息]

使用数据的注意事项

数据的社会影响

[需要更多信息]

偏见讨论

[需要更多信息]

其他已知限制

[需要更多信息]

更多信息

数据集策划者

[需要更多信息]

许可信息

这项工作已获得许可 Creative Commons Attribution 4.0 International Licence

引用信息

@article{chakravarthi-etal-2021-lre,
title = "DravidianCodeMix: Sentiment Analysis and Offensive Language Identification Dataset for Dravidian Languages in Code-Mixed Text",
author = "Chakravarthi, Bharathi Raja  and
  Priyadharshini, Ruba  and
  Muralidaran, Vigneshwaran and
  Jose, Navya and
  Suryawanshi, Shardul and
  Sherly, Elizabeth  and
  McCrae, John P",
  journal={Language Resources and Evaluation},
  publisher={Springer}
}
@inproceedings{dravidianoffensive-eacl,
title={Findings of the Shared Task on {O}ffensive {L}anguage {I}dentification in {T}amil, {M}alayalam, and {K}annada},
author={Chakravarthi, Bharathi Raja and
Priyadharshini, Ruba and
Jose, Navya and
M, Anand Kumar and
Mandl, Thomas and
Kumaresan, Prasanna Kumar and
Ponnsamy, Rahul and
V,Hariharan and
Sherly, Elizabeth and
McCrae, John Philip },
booktitle = "Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages",
month = April,
year = "2021",
publisher = "Association for Computational Linguistics",
year={2021}
}
@inproceedings{hande-etal-2020-kancmd,
    title = "{K}an{CMD}: {K}annada {C}ode{M}ixed Dataset for Sentiment Analysis and Offensive Language Detection",
    author = "Hande, Adeep  and
      Priyadharshini, Ruba  and
      Chakravarthi, Bharathi Raja",
    booktitle = "Proceedings of the Third Workshop on Computational Modeling of People's Opinions, Personality, and Emotion's in Social Media",
    month = dec,
    year = "2020",
    address = "Barcelona, Spain (Online)",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.peoples-1.6",
    pages = "54--63",
    abstract = "We introduce Kannada CodeMixed Dataset (KanCMD), a multi-task learning dataset for sentiment analysis and offensive language identification. The KanCMD dataset highlights two real-world issues from the social media text. First, it contains actual comments in code mixed text posted by users on YouTube social media, rather than in monolingual text from the textbook. Second, it has been annotated for two tasks, namely sentiment analysis and offensive language detection for under-resourced Kannada language. Hence, KanCMD is meant to stimulate research in under-resourced Kannada language on real-world code-mixed social media text and multi-task learning. KanCMD was obtained by crawling the YouTube, and a minimum of three annotators annotates each comment. We release KanCMD 7,671 comments for multitask learning research purpose.",
}
@inproceedings{chakravarthi-etal-2020-corpus,
    title = "Corpus Creation for Sentiment Analysis in Code-Mixed {T}amil-{E}nglish Text",
    author = "Chakravarthi, Bharathi Raja  and
      Muralidaran, Vigneshwaran  and
      Priyadharshini, Ruba  and
      McCrae, John Philip",
    booktitle = "Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL)",
    month = may,
    year = "2020",
    address = "Marseille, France",
    publisher = "European Language Resources association",
    url = "https://www.aclweb.org/anthology/2020.sltu-1.28",
    pages = "202--210",
    abstract = "Understanding the sentiment of a comment from a video or an image is an essential task in many applications. Sentiment analysis of a text can be useful for various decision-making processes. One such application is to analyse the popular sentiments of videos on social media based on viewer comments. However, comments from social media do not follow strict rules of grammar, and they contain mixing of more than one language, often written in non-native scripts. Non-availability of annotated code-mixed data for a low-resourced language like Tamil also adds difficulty to this problem. To overcome this, we created a gold standard Tamil-English code-switched, sentiment-annotated corpus containing 15,744 comment posts from YouTube. In this paper, we describe the process of creating the corpus and assigning polarities. We present inter-annotator agreement and show the results of sentiment analysis trained on this corpus as a benchmark.",
    language = "English",
    ISBN = "979-10-95546-35-1",
}
@inproceedings{chakravarthi-etal-2020-sentiment,
    title = "A Sentiment Analysis Dataset for Code-Mixed {M}alayalam-{E}nglish",
    author = "Chakravarthi, Bharathi Raja  and
      Jose, Navya  and
      Suryawanshi, Shardul  and
      Sherly, Elizabeth  and
      McCrae, John Philip",
    booktitle = "Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL)",
    month = may,
    year = "2020",
    address = "Marseille, France",
    publisher = "European Language Resources association",
    url = "https://www.aclweb.org/anthology/2020.sltu-1.25",
    pages = "177--184",
    abstract = "There is an increasing demand for sentiment analysis of text from social media which are mostly code-mixed. Systems trained on monolingual data fail for code-mixed data due to the complexity of mixing at different levels of the text. However, very few resources are available for code-mixed data to create models specific for this data. Although much research in multilingual and cross-lingual sentiment analysis has used semi-supervised or unsupervised methods, supervised methods still performs better. Only a few datasets for popular languages such as English-Spanish, English-Hindi, and English-Chinese are available. There are no resources available for Malayalam-English code-mixed data. This paper presents a new gold standard corpus for sentiment analysis of code-mixed text in Malayalam-English annotated by voluntary annotators. This gold standard corpus obtained a Krippendorff{'}s alpha above 0.8 for the dataset. We use this new corpus to provide the benchmark for sentiment analysis in Malayalam-English code-mixed texts.",
    language = "English",
    ISBN = "979-10-95546-35-1",
}

贡献者

感谢 @jamespaultg 添加了这个数据集。