模型:
echarlaix/distilbert-base-uncased-finetuned-sst-2-english-int8-dynamic
模型描述:该模型是在SST-2上经过动态量化的细调模型,通过使用 Intel® Neural Compressor 进行量化。
需要安装Optimum:pip install optimum[neural-compressor]
要加载量化模型并使用Transformers pipelines 进行推理,可以按照以下步骤操作:
from transformers import AutoTokenizer, pipeline from optimum.intel import INCModelForSequenceClassification model_id = "echarlaix/distilbert-base-uncased-finetuned-sst-2-english-int8-dynamic" model = INCModelForSequenceClassification.from_pretrained(model_id) tokenizer = AutoTokenizer.from_pretrained(model_id) cls_pipe = pipeline("text-classification", model=model, tokenizer=tokenizer) text = "He's a dreadful magician." outputs = cls_pipe(text)