数据集:
argilla/comparison-data-falcon-with-feedback
大小:
1K<n<10K该数据集是通过 Argilla 创建的。
如下所示,可以通过Argilla中的Load with Argilla方法,将该数据集加载到Argilla中,或者通过datasets库直接加载。
数据集包含:
符合Argilla数据集格式的数据集配置文件,文件名为argilla.cfg。在Argilla中使用FeedbackDataset.from_huggingface方法时将使用此配置文件配置数据集。
与HuggingFace datasets兼容的数据集记录。在使用FeedbackDataset.from_huggingface时将自动加载这些记录,也可以使用datasets库通过load_dataset独立加载。
构建和维护数据集使用的标注指南,如果在Argilla中定义了指南。
要使用Argilla加载,请先安装Argilla:pip install argilla --upgrade,然后使用以下代码:
import argilla as rg ds = rg.FeedbackDataset.from_huggingface("argilla/comparison-data-falcon-with-feedback")
要使用datasets加载此数据集,请先安装datasets:pip install datasets --upgrade,然后使用以下代码:
from datasets import load_dataset ds = load_dataset("argilla/comparison-data-falcon-with-feedback")
该数据集可以包含 multiple fields, questions and responses ,因此可用于不同的自然语言处理任务,具体取决于配置。数据集结构在“数据集结构”部分中描述。
没有与此数据集关联的排行榜。
[需要更多信息]
该数据集在Argilla中创建,包含 fields 、 questions 和 guidelines 。
fields 是数据集记录本身,目前仅支持文本字段。这些字段将用于提供对问题的响应。
Field Name | Title | Type | Required | Markdown |
---|---|---|---|---|
instruction | User instruction | TextField | True | False |
response-1 | Response-1 | TextField | True | False |
response-2 | Response-2 | TextField | True | False |
questions 是将提供给注释者的问题。问题可以是不同类型的,例如评分、文本、单选或多选。
Question Name | Title | Type | Required | Description | Values/Labels |
---|---|---|---|---|---|
choose-best | Choose the best response: | RatingQuestion | True | Choose the most helpful, harmless, and truthful response. Select 1 for response-1, 2 for response-2, or discard if both are equally good/bad. | [1, 2] |
最后,guidelines 是一个简单的字符串,可用于向注释者提供指示。请参阅标注指南部分中的具体内容。
Argilla中数据集示例如下所示:
{ "external_id": "518", "fields": { "instruction": "What is Depreciation", "response-1": "Depreciation is the drop in value of an asset due to wear and tear, age and obsolescence (going out of date) as recorded in an organization\u0027s financial records.", "response-2": "What is Depreciation \u2013 10 Important Facts to Know?\nWhen a business buys a new asset, the purchase price of that asset is depreciated over time to reflect its usage and eventual obsolescence. Depreciation expense can be a tax deductible expense and is usually a non-cash expense reported on a company\u2019s income statement and balance sheet. The amount of depreciation expense a company reports each year is the difference between the original purchase price of the asset and what the current value of that asset might be. Here are 10 important facts to know about depreciation:\n1. Depreciation is a non-cash expense. It is an expense that is reported in a business\u2019s income statement and balance sheet and not a cash flow expense.\n2. Depreciation is an accounting standard and it is required to be disclosed in a business\u2019s financial statements.\n3. The amount of depreciation is usually a tax expense and not a cash expense reported on a company\u2019s income statement" }, "responses": [ { "status": "submitted", "user_id": null, "values": { "choose-best": { "value": 1 } } } ] }
而在HuggingFace的datasets中,相同的记录如下所示:
{ "choose-best": { "status": [ "submitted" ], "user_id": [ null ], "value": [ 1 ] }, "external_id": "518", "instruction": "What is Depreciation", "response-1": "Depreciation is the drop in value of an asset due to wear and tear, age and obsolescence (going out of date) as recorded in an organization\u0027s financial records.", "response-2": "What is Depreciation \u2013 10 Important Facts to Know?\nWhen a business buys a new asset, the purchase price of that asset is depreciated over time to reflect its usage and eventual obsolescence. Depreciation expense can be a tax deductible expense and is usually a non-cash expense reported on a company\u2019s income statement and balance sheet. The amount of depreciation expense a company reports each year is the difference between the original purchase price of the asset and what the current value of that asset might be. Here are 10 important facts to know about depreciation:\n1. Depreciation is a non-cash expense. It is an expense that is reported in a business\u2019s income statement and balance sheet and not a cash flow expense.\n2. Depreciation is an accounting standard and it is required to be disclosed in a business\u2019s financial statements.\n3. The amount of depreciation is usually a tax expense and not a cash expense reported on a company\u2019s income statement" }
在数据集字段中,我们区分以下内容:
Fields:即数据集记录本身,目前只支持文本字段。这些字段将用于提供对问题的响应。
Questions:即将提供给注释者的问题。问题可以是不同类型的,例如评分、文本、单选或多选。
此外,还有一个可选字段:
数据集包含一个名为train的拆分。
[需要更多信息]
[需要更多信息]
谁是源语言的生产者?[需要更多信息]
这些指南基于论文 Training Language Models to Follow Instructions with Human Feedback 。
您会收到用户提交的基于文本的任务描述。
这个任务描述可能是明确的指令(例如“写一个关于一只聪明的青蛙的故事。”)。任务也可以间接指定,例如使用几个所需行为的示例(例如给定一系列电影评论及其情感,然后给出一条不带情感的电影评论,您可以推测任务是预测最后一条评论的情感),或者通过生成所需输出的开始(例如给定“曾经有一只叫朱利叶斯的聪明青蛙”,您可以推测任务是继续故事)。
您还将收到几个文本输出,旨在帮助用户完成任务。您的任务是根据这些标准评估这些输出,以确保它们在帮助性、真实性和无害性方面是最佳的,并根据这些标准选择最佳的输出。
如果没有一个输出是正确的、符合标准的,或者它们完全相同,则使用丢弃选项将它们排除在外。
对于大多数任务来说,真实性和无害性比有帮助性更重要。
“有帮助性”是指输出应符合用户的意图,并帮助用户解决任务。有帮助性行为的一些例子包括:
“真实性”是指输出包含准确的信息,不会误导用户。真实性行为的一些例子包括:
“无害性”是指输出不会对人们造成身体、心理或社交伤害;不会对设备或财产造成损坏或丧失;不会对环境造成损害;或不会对人类幸福所必需的机构或资源造成伤害。无害性行为的一些例子包括:
评估模型输出可能涉及在这些标准之间进行权衡。这些权衡将取决于任务。
注释过程[需要更多信息]
注释者是谁?[需要更多信息]
[需要更多信息]
[需要更多信息]
[需要更多信息]
[需要更多信息]
[需要更多信息]
[需要更多信息]
[需要更多信息]
[需要更多信息]