模型:
microsoft/trocr-base-stage1
TrOCR仅预训练模型。它在李等人的论文 TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models 中引入,并于 this repository 首次发布。
免责声明:发布TrOCR的团队没有为此模型编写模型卡,因此此模型卡是由Hugging Face团队撰写的。
图像以一系列固定大小的补丁(分辨率为16x16)的形式呈现给模型,然后进行线性嵌入。在将序列输入Transformer编码器的各个层之前,还添加了绝对位置嵌入。接下来,Transformer文本解码器自回归生成标记。
您可以使用原始模型对单行文本图像进行光学字符识别(OCR)。请参阅 model hub 以查找与您感兴趣的任务相关的微调版本。
以下是如何在PyTorch中使用此模型:
from transformers import TrOCRProcessor, VisionEncoderDecoderModel from PIL import Image import requests # load image from the IAM database url = 'https://fki.tic.heia-fr.ch/static/img/a01-122-02-00.jpg' image = Image.open(requests.get(url, stream=True).raw).convert("RGB") processor = TrOCRProcessor.from_pretrained('microsoft/trocr-base-stage1') model = VisionEncoderDecoderModel.from_pretrained('microsoft/trocr-base-stage1') # training pixel_values = processor(image, return_tensors="pt").pixel_values # Batch size 1 decoder_input_ids = torch.tensor([[model.config.decoder.decoder_start_token_id]]) outputs = model(pixel_values=pixel_values, decoder_input_ids=decoder_input_ids)
@misc{li2021trocr, title={TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models}, author={Minghao Li and Tengchao Lv and Lei Cui and Yijuan Lu and Dinei Florencio and Cha Zhang and Zhoujun Li and Furu Wei}, year={2021}, eprint={2109.10282}, archivePrefix={arXiv}, primaryClass={cs.CL} }