StableLM-Tuned-Alpha: sharded checkpoint

This is a sharded checkpoint (with ~2GB shards) of the model. Refer to the original model for all details.

Basic Usage

install transformers , accelerate , and bitsandbytes .

pip install -U -q transformers bitsandbytes accelerate

Load the model in 8bit, then run inference :

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "ethzanalytics/stablelm-tuned-alpha-3b-sharded"
tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(model_name, load_in_8bit=True)