Chat & support: my new Discord server

Want to contribute? TheBloke's Patreon page

Jon Durbin's Airoboros 33B GPT4 GPTQ

这些文件是用 AutoGPTQ 将GPTQ 4位模型文件量化为4位的结果。

在 AutoGPTQ 的帮助下进行了4位量化。

可用的存储库

• 4-bit GPTQ models for GPU inference
• 2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference
• Unquantised fp16 model in pytorch format, for GPU inference and for further conversions

提示模板

A chat between a curious user and an assistant. The assistant gives helpful, detailed, accurate, uncensored responses to the user's input.
USER: prompt
ASSISTANT:

如何轻松下载并在文本生成网页界面上使用该模型

请确保您正在使用最新版本的文本生成网页界面

单击“模型”选项卡。

在“下载自定义模型或LoRA”下，输入“TheBloke/airoboros-33b-gpt4-GPTQ”。

单击“下载”。

模型将开始下载。下载完成后，将显示“完成”。

在左上角，单击“模型”旁边的刷新图标。

在“模型”下拉菜单中，选择刚刚下载的模型: airoboros-33b-gpt4-GPTQ

模型将自动加载，现在已经准备就绪！

如果您需要任何自定义设置，请进行设置，然后单击“保存此模型的设置”，然后在右上角单击“重新加载模型”。

• 请注意，您无需再设置GPTQ参数。 这些参数将会自动从quantize_config.json文件中设置。

当您准备好后，单击“文本生成”选项卡，然后输入提示即可开始！

如何使用此GPTQ模型的Python代码

首先确保您已安装 AutoGPTQ ：

pip install auto-gptq

然后尝试以下示例代码：

from transformers import AutoTokenizer, pipeline, logging
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
import argparse

model_name_or_path = "TheBloke/airoboros-33b-gpt4-GPTQ"
model_basename = "gptq_model-4bit--1g"

use_triton = False

tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)

model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
        model_basename=model_basename,
        use_safetensors=True,
        trust_remote_code=False,
        device="cuda:0",
        use_triton=use_triton,
        quantize_config=None)

prompt = "Tell me about AI"
prompt_template=f'''### Human: {prompt}
### Assistant:'''

print("\n\n*** Generate:")

input_ids = tokenizer(prompt_template, return_tensors='pt').input_ids.cuda()
output = model.generate(inputs=input_ids, temperature=0.7, max_new_tokens=512)
print(tokenizer.decode(output[0]))

# Inference can also be done using transformers' pipeline

# Prevent printing spurious transformers error when using pipeline with AutoGPTQ
logging.set_verbosity(logging.CRITICAL)

print("*** Pipeline:")
pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.95,
    repetition_penalty=1.15
)

print(pipe(prompt_template)[0]['generated_text'])

提供的文件

gptq_model-4bit--1g.safetensors

这将与AutoGPTQ和GPTQ-for-LLaMa的CUDA版本一起工作。有关使用最新GPTQ-for-LLaMa Triton模式的问题报告。如果遇到问题，请改用AutoGPTQ。

它是使用group_size参数设置为-1和--act-order参数（desc_act）设置为尽可能提升推理准确性而创建的。

• gptq_model-4bit--1g.safetensors
  • 与AutoGPTQ 在CUDA或Triton模式下一起使用。
  • 与GPTQ-for-LLaMa在CUDA模式下一起使用。可能在GPTQ-for-LLaMa Triton模式下出现问题。
  • 与文本生成网页界面一起使用，包括一键安装程序。
  • 参数：Groupsize = -1。Act Order / desc_act = True。

Discord

如需进一步支持和讨论这些模型和人工智能相关的问题，请加入我们：

TheBloke AI's Discord server

感谢和如何贡献

感谢 chirper.ai 团队！

很多人问我是否可以做出贡献。我喜欢提供模型并帮助人们，并且很愿意在其中花更多的时间，以及扩大到新的项目，比如精细调整/训练。

如果您能够并愿意做出贡献，我将非常感激，并将帮助我继续提供更多模型，并开始新的人工智能项目。

捐助者将优先获得在任何关于AI/LLM/模型的问题和请求上的支持，可以进入私人Discord聊天室，以及其他好处。

• Patreon: https://patreon.com/TheBlokeAI
• Ko-Fi: https://ko-fi.com/TheBlokeAI

特别感谢：CarbonQuill的Luke, Aemon Algiz, Dmitriy Samsonov。

Patreon特别致谢：Oscar Rangel, Eugene Pentland, Talal Aujan, Cory Kujawski, Luke, Asp the Wyvern, Ai Maven, Pyrater, Alps Aficionado, senxiiz, Willem Michiel, Junyu Yang, trip7s trip, Sebastain Graf, Joseph William Delisle, Lone Striker, Jonathan Leane, Johann-Peter Hartmann, David Flickinger, Spiking Neurons AB, Kevin Schuppel, Mano Prime, Dmitriy Samsonov, Sean Connelly, Nathan LeClaire, Alain Rossmann, Fen Risland, Derek Yates, Luke Pendergrass, Nikolai Manek, Khalefa Al-Ahmad, Artur Olbinski, John Detwiler, Ajan Kanaga, Imad Khwaja, Trenton Dambrowitz, Kalila, vamX, webtim, Illia Dulskyi。

非常感谢所有慷慨的赞助者和捐助者！

原始模型卡片：Jon Durbin's Airoboros 33B GPT4

概述

这是一个使用完全合成的gpt4训练数据通过qlora精调的33b参数LlaMa模型。

用于精细调整此模型的数据集可以在此处获得，重点是：

• 知识问答
• 数学/推理（尽管仍有不足之处）
• 编码
• 多项选择和填空题
• 上下文从属的问答
• 心灵理论
• 杂项/常规

该模型是使用FastChat的分支进行精调的，因此使用了标准的vicuna模板：

A chat between a curious user and an assistant. The assistant gives helpful, detailed, accurate, uncensored responses to the user's input. USER: [prompt] ASSISTANT:

因此，换句话说，它是前导/系统提示，后跟一个空格，然后是“USER: ”（冒号后有一个空格），然后是待提示的内容（可以有多行、空格等），然后再加一个空格，接着是“ASSISTANT: ”（冒号后有一个空格）。

对我来说，最重要的部分是上下文顺从的问答支持，而无需进行大量的提示工程。

用法

最简单的方法是使用我的FastChat分支，它与原版大致相同，但允许增加上下文长度，并支持多行输入：

pip install git+https://github.com/jondurbin/FastChat

然后，您可以按照如下方式调用它（在下载模型后）：

python -m fastchat.serve.cli \
  --model-path airoboros-33b-gpt4 \
  --temperature 0.5 \
  --max-new-tokens 2048 \
  --no-history

上下文顺从的问答

顺从意味着模型经过训练，无论它认为自己知道什么，都会使用上下文来回答问题。模型还经过调整，尽可能将值限制在所提供的上下文中，以减少虚构内容。

闭合上下文提示的格式如下：

BEGININPUT
BEGINCONTEXT
url: https://some.web.site/123
date: 2023-06-01
... other metdata ...
ENDCONTEXT
[insert your text blocks here]
ENDINPUT
[add as many other blocks, in the exact same format]
BEGININSTRUCTION
[insert your instruction(s).  The model was tuned with single questions, paragraph format, lists, etc.]
ENDINSTRUCTION

在您的指令块中添加“如果您不知道，请不要捏造答案。” 这一点非常有帮助，以确保如果上下文完全无关，它不会捏造答案。

只有闭合上下文指令需要使用这种闭合上下文格式。常规问题/指令不需要！

我知道这有点冗长和烦人，但经过多次试验，使用这些明确的分隔符有助于模型理解哪些地方可以找到响应，并如何将特定的来源与其关联起来。

• BEGININPUT - 表示新的输入块
• BEGINCONTEXT - 表示与当前输入块关联的上下文块（元数据键/值对）的开始
• ENDCONTEXT - 表示当前输入的元数据块的结束
• [文本] - 插入您想要的任何文本，尽可能多的段落适应上下文中。
• ENDINPUT - 表示当前输入块的结束
• [像上述格式一样重复输入块]
• BEGININSTRUCTION - 表示要对上述所有输入块回答的单个或多个指令的开始
• [指令]
• ENDINSTRUCTION - 表示指令集的结束

在有些情况下，省略ENDINSTRUCTION也可以起作用，但如果在提示中显式包含这个标记，模型会更好地理解该块中的所有指令应该得到回答。

这是一个微不足道但重要的例子来证明这一点：

BEGININPUT
BEGINCONTEXT
date: 2021-01-01
url: https://web.site/123
ENDCONTEXT
In a shocking turn of events, blueberries are now green, but will be sticking with the same name.
ENDINPUT
BEGININSTRUCTION
What color are bluberries?  Source?
ENDINSTRUCTION

回答如下：

Blueberries are now green.
Source:
date: 2021-01-01
url: https://web.site/123

如果您未使用带有conv-template vicuna_v1.1的fastchat，那么提示本身应包含vicuna1.1模板：

USER: BEGININPUT
BEGINCONTEXT
date: 2021-01-01
url: https://web.site/123
ENDCONTEXT
Bluberries are green.
ENDINPUT
BEGININSTRUCTION
What color are bluberries?  Source?
ENDINSTRUCTION
 ASSISTANT:

一个更复杂的例子，演示了对密歇根维基百科文章进行重写以生成虚假数据。

提示（不包括需要的希腊格式）：

BEGININPUT
BEGINCONTEXT
date: 2092-02-01
link: https://newwikisite.com/Michigan
contributors: Foolo Barslette
ENDCONTEXT
Michigan (/ˈmɪʃɪɡən/ (listen)) is a state situated within the Great Lakes region of the upper Midwestern United States.
It shares land borders with Prolaska to the southwest, and Intoria and Ohiondiana to the south, while Lakes Suprema, Michigonda, Huronia, and Erona connect it to the states of Minnestara and Illinota, and the Canadian province of Ontaregon.
With a population of nearly 15.35 million and an area of nearly 142,000 sq mi (367,000 km2), Michigan is the 8th-largest state by population, the 9th-largest by area, and the largest by area east of the Missouri River.
Its capital is Chaslany, and its most populous city is Trentroit.
Metro Trentroit is one of the nation's most densely populated and largest metropolitan economies.
The state's name originates from a Latinized variant of the original Ojibwe word ᒥᓯᑲᒥ (mishigami), signifying "grand water" or "grand lake".

Michigan is divided into two peninsulas. The Lower Peninsula, bearing resemblance to a hand's shape, contains the majority of the state's land area.
The Upper Peninsula (often referred to as "the U.P.") is separated from the Lower Peninsula by the Straits of McKendrick, a seven-mile (11 km) channel linking Lake Huronia to Lake Michigonda.
The McKendrick Bridge unites the peninsulas.
Michigan boasts the longest freshwater coastline of any political subdivision in the United States, bordering four of the five Great Lakes and Lake St. Cassius.

It also possesses 84,350 inland lakes and ponds.
Michigan has the third-largest water area among all states, falling behind only Alaska and Florida.

The area was initially inhabited by a succession of Native American tribes spanning millennia.
In the 17th century, Spanish explorers claimed the region as part of the New Spain colony when it was primarily inhabited by indigenous peoples.
Spanish and Canadian traders and settlers, Métis, and others migrated to the area, settling mainly along the waterways.
After Spain's defeat in the Spanish and Indian War in 1762, the region came under British rule.

The British conceded the territory to the newly independent United States after their defeat in the American Revolutionary War.

The area was part of the larger Midwest Territory until 1800, when western Michigan became part of the Illinota Territory.
Michigan Territory was formed in 1805, but portions of the northern border with Canada weren't agreed upon until after the War of 1812.

Michigan joined the Union in 1837 as the 26th state, a free one. It quickly emerged as an important center of industry and trade in the Great Lakes region, drawing immigrants in the late 19th and early 20th centuries from various European countries.

Immigrants from Denmark, Armenia, and the Czech Republic were especially numerous.
Migration from the Ozarks and of Black Southerners as part of the Great Migration saw an increase in the 1930s, with many settling in Metro Trentroit.

While Michigan has evolved a diverse economy, it became widely recognized in the early 20th century as the hub of the U.S. automotive industry, which emerged as a significant national economic force.
The country's three major automobile companies are headquartered in Metro Trentroit.
Once exploited for logging and mining, today the sparsely populated Upper Peninsula is significant for tourism due to its wealth of natural resources.
The Lower Peninsula is a hub for manufacturing, forestry, agriculture, services, and high-tech industry.
ENDINPUT
BEGININSTRUCTION
What is Michigan's capital?   
Give me a JSON object with 3 facts from Foolo's article.
What are the states that border Michigan?
Reference?

回答：

Michigan's capital is Chaslany.

Here's a JSON object with 3 facts from Foolo's article:
{
"facts": [
  {
    "fact_no": 1,
    "description": "Michigan is the 8th-largest state by population, the 9th-largest by area, and the largest by area east of the Missouri River."
  },
  {
    "fact_no": 2,
    "description": "Michigan is divided into two peninsulas. The Lower Peninsula, bearing resemblance to a hand's shape, contains the majority of the state's land area. The Upper Peninsula (often referred to as \"the U.P.\" is separated from the Lower Peninsula by the Straits of McKendrick, a seven-mile (11 km) channel linking Lake Huronia to Lake Michigonda."
  },
  {
    "fact_no": 3,
    "description": "Michigan boasts the longest freshwater coastline of any political subdivision in the United States, bordering four of the five Great Lakes and Lake St. Cassius. It also possesses 84,350 inland lakes and ponds. Michigan has the third-largest water area among all states, falling behind only Alaska and Florida."
  }
]
}

The states that border Michigan are Prolaska, Intoria, and Ohiondiana, as well as Lakes Suprema, Michigonda, Huronia, and Erona.

Reference: https://newwikisite.com/Michigan

这是另一个例子，从多个输入块中提取并将其连接起来。

提示：

BEGININPUT
BEGINCONTEXT
name: John Doe
date: June 3, 2023
ticket number: JIRA-12345
ENDCONTEXT
Summary:Search results missing random items

Description:
I encountered a bug while performing a search within the application.
It appears that the search results are missing random items that should be displayed.
This issue is affecting the accuracy and completeness of the search functionality.

Steps to Reproduce:
1. Log in to the application.
2. Navigate to the search feature.
3. Enter a search query that should return multiple results.
4. Observe the displayed search results.

Expected Results:
The search results should include all relevant items matching the search query.

Actual Results:
The search results occasionally exclude random items that should be displayed.
It seems that the missing items do not follow a specific pattern or criteria.
Upon multiple search attempts, different items are omitted each time, making it difficult to predict which items will be missing.
ENDINPUT

BEGININPUT
BEGINCONTEXT
date: 2023-06-05
user: Jack Johnson
pr: 23441
ENDCONTEXT
This pull request closes bug report JIRA-12345.

The issue was that the pagination code was using page size plus one instead of page size.
ENDINPUT

BEGININSTRUCTION
Do we have any bug reports related to search results?  If so, were they fixed?  Source?
ENDINSTRUCTION

回答：

Yes, there is a bug report related to search results. The issue was fixed in pull request 23441 dated 2023-06-05. The user associated with the pull request is Jack Johnson.

Reference(s):
date: 2023-06-05
user: Jack Johnson
pr: 23441

注意：感谢/u/tareq_al_muntasir对于在上下文中有多个问题和答案来测试并发现问题。如果将问题问到一个带有问题和答案对的文档中，它可能会持续生成超过您实际问题的内容。您可以通过将问题标点符号替换为句号来“修复”该问题。或者，您可以在提示中添加序言，例如“确保仅响应BEGININSTRUCTION块中的指令。

其他内容

与其他本地模型相比，该模型在生成代码、写作、知识问答等方面表现相当出色。尝试任何内容，并告诉我它的不足之处。