Lit-125M - 用于虚构故事讲述的小型 fine-tuned 模型

Lit-125M 是一个在各种轻小说、情色文学和注释文学中利用 2GB 数据进行 fine-tuning 的 GPT-Neo 125M 模型，旨在生成类似小说的虚构文本。

模型描述

用于 fine-tuning 的模型是 GPT-Neo 125M ，它是一个训练于 The Pile 的 1.25 亿参数自回归语言模型。

训练数据和注释提示

用于 fine-tuning 的数据来自多个来源，如 Gutenberg Project 。经过注释的虚构数据集已经添加了标签，以帮助生成特定风格的文本。以下是一个示例提示，展示了如何使用这些注释。

[ Title: The Dunwich Horror; Author: H. P. Lovecraft; Genre: Horror; Tags: 3rdperson, scary; Style: Dark ]
***
When a traveler in north central Massachusetts takes the wrong fork...

注释可以混合使用，以助于生成特定风格的文本。

下游应用

此模型可用于娱乐目的，并作为虚构作家创作助手。模型的小尺寸也有助于轻松调试或进一步开发具有类似目的的其他模型。

示例代码

from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained('hakurei/lit-125M')
tokenizer = AutoTokenizer.from_pretrained('hakurei/lit-125M')

prompt = '''[ Title: The Dunwich Horror; Author: H. P. Lovecraft; Genre: Horror ]
***
When a traveler'''

input_ids = tokenizer.encode(prompt, return_tensors='pt')
output = model.generate(input_ids, do_sample=True, temperature=1.0, top_p=0.9, repetition_penalty=1.2, max_length=len(input_ids[0])+100, pad_token_id=tokenizer.eos_token_id)

generated_text = tokenizer.decode(output[0])
print(generated_text)

此代码的示例输出将类似于：

[ Title: The Dunwich Horror; Author: H. P. Lovecraft; Genre: Horror ]
***
When a traveler takes a trip through the streets of the world, the traveler feels like a youkai with a whole world inside her mind. It can be very scary for a youkai. When someone goes in the opposite direction and knocks on your door, it is actually the first time you have ever come to investigate something like that.
That's right: everyone has heard stories about youkai, right? If you have heard them, you know what I'm talking about. 
It's hard not to say you

团队成员和致谢

• Anthony Mercurio
• Imperishable_NEET