RedPajama-INCITE-Base-3B-v1

RedPajama-INCITE-Base-3B-v1是由Together和开源AI社区的领导者共同开发的，包括Ontocord.ai、ETH DS3Lab、AAI CERC、蒙特利尔大学、MILA-魁北克AI研究所、斯坦福大学基础模型研究中心（CRFM）、斯坦福大学Hazy Research研究小组和LAION。该模型是在INCITE 2023项目中开发的，该项目使用了来自MILA、LAION和EleutherAI的3,072个V100 GPU，该项目旨在实现具有可转移性的通用AI基础模型，并得到了奥克里奇领导计算设施（OLCF）和INCITE计划的支持。

• 基础模型： RedPajama-INCITE-Base-3B-v1
• 指令调优版本： RedPajama-INCITE-Instruct-3B-v1
• 聊天版本： RedPajama-INCITE-Chat-3B-v1

模型详情

• 开发者：Together Computer
• 模型类型：语言模型
• 语言：英语
• 许可证：Apache 2.0
• 模型描述：一个拥有2.8B参数的预训练语言模型

快速上手

请注意，该模型需要transformers版本>= 4.25.1。

GPU推断

这需要一块具有8GB内存的GPU。

import torch
import transformers
from transformers import AutoTokenizer, AutoModelForCausalLM

MIN_TRANSFORMERS_VERSION = '4.25.1'

# check transformers version
assert transformers.__version__ >= MIN_TRANSFORMERS_VERSION, f'Please upgrade transformers to version {MIN_TRANSFORMERS_VERSION} or higher.'

# init
tokenizer = AutoTokenizer.from_pretrained("togethercomputer/RedPajama-INCITE-Base-3B-v1")
model = AutoModelForCausalLM.from_pretrained("togethercomputer/RedPajama-INCITE-Base-3B-v1", torch_dtype=torch.float16)
model = model.to('cuda:0')

# infer
prompt = "Alan Turing is"
inputs = tokenizer(prompt, return_tensors='pt').to(model.device)
input_length = inputs.input_ids.shape[1]
outputs = model.generate(
    **inputs, max_new_tokens=128, do_sample=True, temperature=0.7, top_p=0.7, top_k=50, return_dict_in_generate=True,
)
token = outputs.sequences[0, input_length:]
output_str = tokenizer.decode(token)
print(output_str)
"""
a name that has been synonymous with the computer age since the 1950s. The British mathematician, logician, and cryptanalyst is widely regarded as the father of modern computing. His contributions to the development of the modern computer and the theory of computation have had a profound impact on the world we live in today.
Turing’s contributions to the development of the modern computer were made in the 1940s and 1950s. He is most famous for his work on the Turing machine, a theoretical model of a computing machine that was able to perform all the mathematical operations of a computer. Turing’s work on the...
"""

使用Int8进行GPU推断

要使用int8运行推断，请确保您已安装加速和bitandbytes。您可以使用以下命令安装它们：

pip install accelerate
pip install bitsandbytes

然后您可以按以下方式使用int8运行推断：

import torch
import transformers
from transformers import AutoTokenizer, AutoModelForCausalLM

MIN_TRANSFORMERS_VERSION = '4.25.1'

# check transformers version
assert transformers.__version__ >= MIN_TRANSFORMERS_VERSION, f'Please upgrade transformers to version {MIN_TRANSFORMERS_VERSION} or higher.'

# init
tokenizer = AutoTokenizer.from_pretrained("togethercomputer/RedPajama-INCITE-Base-3B-v1")
model = AutoModelForCausalLM.from_pretrained("togethercomputer/RedPajama-INCITE-Base-3B-v1", device_map='auto', torch_dtype=torch.float16, load_in_8bit=True)

# infer
prompt = "Alan Turing is"
inputs = tokenizer(prompt, return_tensors='pt').to(model.device)
input_length = inputs.input_ids.shape[1]
outputs = model.generate(
    **inputs, max_new_tokens=128, do_sample=True, temperature=0.7, top_p=0.7, top_k=50, return_dict_in_generate=True
)
token = outputs.sequences[0, input_length:]
output_str = tokenizer.decode(token)
print(output_str)
"""
the man who cracked the Enigma code during World War II, and who was later convicted of homosexual acts. He was a brilliant mathematician, and a visionary who foresaw the computer age....
"""

CPU推断

您可以按以下方式在CPU上运行推断：

import torch
import transformers
from transformers import AutoTokenizer, AutoModelForCausalLM

MIN_TRANSFORMERS_VERSION = '4.25.1'

# check transformers version
assert transformers.__version__ >= MIN_TRANSFORMERS_VERSION, f'Please upgrade transformers to version {MIN_TRANSFORMERS_VERSION} or higher.'

# init
tokenizer = AutoTokenizer.from_pretrained("togethercomputer/RedPajama-INCITE-Base-3B-v1")
model = AutoModelForCausalLM.from_pretrained("togethercomputer/RedPajama-INCITE-Base-3B-v1", torch_dtype=torch.bfloat16)
# infer
prompt = "Alan Turing is"
inputs = tokenizer(prompt, return_tensors='pt').to(model.device)
input_length = inputs.input_ids.shape[1]
outputs = model.generate(
    **inputs, max_new_tokens=128, do_sample=True, temperature=0.7, top_p=0.7, top_k=50, return_dict_in_generate=True
)
token = outputs.sequences[0, input_length:]
output_str = tokenizer.decode(token)
print(output_str)
"""
a name that is synonymous with the history of computer science. As the man who invented the Turing machine, the mathematical model that defines the limits of what can be computed, Turing is credited with the invention of the modern computer. Turing was also a mathematician and logician, and his work in these fields led to the development of the field of artificial intelligence...
"""

请注意，由于LayerNormKernelImpl在CPU上没有实现fp16，我们在CPU推断中使用了bfloat16。

用途

以下是不包括在使用范围内的用途。

误用、恶意使用和超出范围使用

最终用户有责任确保以负责任和道德的方式使用该模型。

RedPajama-INCITE-Base-3B-v1是一个语言模型，可能在其预期范围之外的其他用例中表现不佳。例如，它可能不适用于安全关键应用或对个人或社会有重大影响的决策。重要的是要考虑模型的限制，并仅将其用于预期目的。

RedPajama-INCITE-Base-3B-v1是专为语言建模而设计的。严禁对该模型进行误用，例如将其用于非法或不道德的活动，这违反了该项目的原则。

将模型用于对个人进行残忍对待属于对该模型的误用。这包括但不限于：

• 生成假新闻、误导信息或宣传
• 宣扬仇恨言论、歧视或对个人或群体实施暴力
• 未经个人或组织同意进行冒充
• 进行网络欺凌或骚扰
• 诽谤性内容
• 发送垃圾邮件或进行欺诈活动
• 未经适当授权共享机密或敏感信息
• 违反模型或用于训练模型的数据的使用条款
• 创建用于恶意目的的自动化机器人，如传播恶意软件、网络钓鱼诈骗或垃圾邮件

限制

RedPajama-INCITE-Base-3B-v1，像其他语言模型一样，具有应该考虑的限制。例如，该模型可能无法始终提供准确或相关的答案，特别是对于复杂、含糊不清或超出其训练数据范围的问题。因此，我们欢迎个人和组织的贡献，并鼓励合作，以创建一个更强大、包容性更强的聊天机器人。

训练

训练数据

请参考 togethercomputer/RedPajama-Data-1T

训练过程

• 硬件：OLCF Summit集群上的256节点，每个节点有6个V100（IBM Power9）
• 优化器：Apex FusedAdam
• 并行性：流水线并行6，张量并行2
• 梯度累积：8（全局批量大小4M个令牌）
• 令牌数量：800B个令牌
• 学习率：0.00016

基准测试

请参考我们的 blog post 获取基准测试结果。

社区

加入我们的 Together Discord