VMware/open-llama-7B-open-instruct

针对完全训练的Open LLama 7B模型进行了指令调优。该模型对于商业用途是开放的。

这个模型有一个v2版本可供使用， https://huggingface.co/VMware/open-llama-7b-v2-open-instruct

注意：该模型使用了Alpaca提示模板进行训练。注意：快速分词器会导致错误的编码，请在实例化分词器时将use_fast参数设置为False。

许可证

• 商业可行
• 指令数据集 VMware/open-instruct-v1-oasst-dolly-hhrlhf 受cc-by-sa-3.0许可
• 语言模型 openlm-research/open_llama_7b 受apache-2.0许可

命名法

• 模型：Open-llama
• 模型大小：7B参数
• 数据集：Open-instruct-v1 (oasst, dolly, hhrlhf)

在Transformers中使用

import os
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = 'VMware/open-llama-7b-open-instruct'


tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=False)

model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16, device_map='sequential')

prompt_template = "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Response:"

prompt = 'Explain in simple terms how the attention mechanism of a transformer model works'


inputt = prompt_template.format(instruction= prompt)
input_ids = tokenizer(inputt, return_tensors="pt").input_ids.to("cuda")

output1 = model.generate(input_ids, max_length=512)
input_length = input_ids.shape[1]
output1 = output1[:, input_length:]
output = tokenizer.decode(output1[0])

print(output)

'''
 Attention is a mechanism used in deep learning models, such as transformer models, to capture global dependencies between different parts of the input. In a transformer model, the attention mechanism works by computing a weighted sum of the input vectors and then applying a non-linear activation function to the result.

The attention mechanism in a transformer model works in two steps:

1. Query-Key Mapping: First, the input sequence is divided into two parts: the query vector and the key vector. The query vector represents the input at the current position, and the key vector represents the input at a previous position.

2. Attention Weight Calculation: Second, the attention weights are calculated using the dot product between the query vector and each key vector. The attention weights represent the importance of the input at the previous position to the current position.

The attention weights are then used to compute the attention score for each input element. The attention score represents the relevance of the input element to the current position.

The attention mechanism in a transformer model is designed to capture global dependencies between different parts of the input. By attending to input elements from different positions, the model can learn to understand the relationships between different parts of the input. This allows the model to perform more complex tasks, such as understanding the relationships between words in a sentence or pixels in an image.</s>

'''

微调细节

我们的微调脚本将在我们的 RAIL Github Repository 上提供。

评估

待办事项