Summary

This is a Thai ??-instructed dataset translated from cleaned version of the original Alpaca Dataset released by Stanford using Google Cloud Translation, contain 52,000 instructions and demonstrations generated by OpenAI's text-davinci-003 engine. This instruction data can be used to conduct instruction-tuning for language models and make the language model follow instruction better.

The following issues have been identified in the original release and fixed in this dataset:

Hallucinations: Many instructions in the original dataset had instructions referencing data on the internet, which just caused GPT3 to hallucinate an answer.

Merged Instructions: There were many instructions that were merged together in the original dataset for some reason.

Empty outputs: Some entries in the original dataset had empty outputs.

Empty code examples: Some descriptions in the original dataset were missing code examples, making it difficult to understand the intended behavior of the code.

Instructions to generate images: Some descriptions in the original dataset included instructions to generate images, something obviously not possible.

N/A outputs: Some code snippets in the original dataset had N/A outputs.

Inconsistent input field: The original dataset had inconsistent usage of the input field when it was supposed to be empty.

Wrong answers: Some instructions/questions in the original dataset had incorrect answers. About 80% of the math problems are estimated to have incorrect answers.

Non-Sensical/Unclear instructions: Many instructions are unclear, we try to clarify (or re-write) if instructions are non-sensical. Instructions that are slightly unclear, but where one could deduce the meaning are not altered.

Extraneous escape and control characters: The original dataset had several entries with extraneous escape and control characters.

Original Alpaca Dataset Summary

Alpaca is a dataset of 52,000 instructions and demonstrations generated by OpenAI's text-davinci-003 engine. This instruction data can be used to conduct instruction-tuning for language models and make the language model follow instruction better.

The authors built on the data generation pipeline from Self-Instruct framework and made the following modifications:

The text-davinci-003 engine to generate the instruction data instead of davinci .
A new prompt was written that explicitly gave the requirement of instruction generation to text-davinci-003 .
Much more aggressive batch decoding was used, i.e., generating 20 instructions at once, which significantly reduced the cost of data generation.
The data generation pipeline was simplified by discarding the difference between classification and non-classification instructions.
Only a single instance was generated for each instruction, instead of 2 to 3 instances as in Self-Instruct.

The authors built on the data generation pipeline from Self-Instruct framework and made the following modifications:

The text-davinci-003 engine to generate the instruction data instead of davinci .
A new prompt was written that explicitly gave the requirement of instruction generation to text-davinci-003 .
Much more aggressive batch decoding was used, i.e., generating 20 instructions at once, which significantly reduced the cost of data generation.
The data generation pipeline was simplified by discarding the difference between classification and non-classification instructions.
Only a single instance was generated for each instruction, instead of 2 to 3 instances as in Self-Instruct.

This produced an instruction-following dataset with 52K examples obtained at a much lower cost (less than $500). In a preliminary study, the authors also found that the 52K generated data to be much more diverse than the data released by Self-Instruct .

Supported Tasks:

Training LLMs
Synthetic Data Generation
Data Augmentation

Languages: Thai Version: 1.0

作者:

Thaweewat

数据集大小:

34.81 MB