文档理解模型（在段落级别上对DocLayNet基础上的LayoutXLM进行微调）

此模型是根据数据集 DocLayNet base 对 microsoft/layoutxlm-base 进行微调的版本。它在评估集上达到以下结果：

• 损失：0.1796
• 精确度：0.8062
• 召回率：0.7441
• F1 值：0.7739
• Token 准确度：0.9693
• 段落准确度：0.8655

段落级别的准确度

• 段落准确度：86.55%
• 按标签的准确度
  • 标题：63.76%
  • 脚注：31.91%
  • 公式：95.33%
  • 列表项：79.31%
  • 页脚：99.51%
  • 页眉：88.75%
  • 图片：90.91%
  • 章节标题：83.16%
  • 表格：68.25%
  • 文本：91.37%
  • 标题：50.0%

参考文献

博客文章

• Layout XLM base
  • (03/31/2023) Document AI | Inference APP and fine-tuning notebook for Document Understanding at paragraph level with LayoutXLM base
  • (03/25/2023) Document AI | APP to compare the Document Understanding LiLT and LayoutXLM (base) models at line level
  • (03/05/2023) Document AI | Inference APP and fine-tuning notebook for Document Understanding at line level with LayoutXLM base
• LiLT base- (02/16/2023) Document AI | Inference APP and fine-tuning notebook for Document Understanding at paragraph level - (02/14/2023) Document AI | Inference APP for Document Understanding at line level - (02/10/2023) Document AI | Document Understanding model at line level with LiLT, Tesseract and DocLayNet dataset - (01/31/2023) Document AI | DocLayNet image viewer APP - (01/27/2023) Document AI | Processing of DocLayNet dataset to be used by layout models of the Hugging Face hub (finetuning, inference)

笔记本（段落级别）

• Layout XLM base
  • Document AI | Inference at paragraph level with a Document Understanding model (LayoutXLM base fine-tuned on DocLayNet dataset)
  • Document AI | Inference APP at paragraph level with a Document Understanding model (LayoutXLM base fine-tuned on DocLayNet base dataset)
  • Document AI | Fine-tune LayoutXLM base on DocLayNet base in any language at paragraph level (chunk of 512 tokens with overlap)
• LiLT base
  • Document AI | Inference APP at paragraph level with a Document Understanding model (LiLT fine-tuned on DocLayNet dataset)
  • Document AI | Inference at paragraph level with a Document Understanding model (LiLT fine-tuned on DocLayNet dataset)
  • Document AI | Fine-tune LiLT on DocLayNet base in any language at paragraph level (chunk of 512 tokens with overlap)

笔记本（行级别）

• Layout XLM base
  • Document AI | Inference APP at line level with 2 Document Understanding models (LiLT and LayoutXLM base fine-tuned on DocLayNet base dataset)
  • Document AI | Inference at line level with a Document Understanding model (LayoutXLM base fine-tuned on DocLayNet dataset)
  • Document AI | Inference APP at line level with a Document Understanding model (LayoutXLM base fine-tuned on DocLayNet base dataset)
  • Document AI | Fine-tune LayoutXLM base on DocLayNet base in any language at line level (chunk of 384 tokens with overlap)
• LiLT base
  • Document AI | Inference at line level with a Document Understanding model (LiLT fine-tuned on DocLayNet dataset)
  • Document AI | Inference APP at line level with a Document Understanding model (LiLT fine-tuned on DocLayNet dataset)
  • Document AI | Fine-tune LiLT on DocLayNet base in any language at line level (chunk of 384 tokens with overlap)
  • DocLayNet image viewer APP
  • 处理 DocLayNet 数据集以供 Hugging Face hub 的布局模型使用（微调、推理）

APP

您可以在 Hugging Face Spaces 中使用此 APP 测试此模型： Inference APP for Document Understanding at paragraph level (v2) 。

您也可以运行相应的笔记本： Document AI | Inference APP at paragraph level with a Document Understanding model (LayoutXLM base fine-tuned on DocLayNet dataset)

DocLayNet 数据集

IBM 提供了使用边界框对 6 个不同文档类别的 80863 个唯一页面进行页面级布局分割的基本事实数据，提供了 11 个不同类别的基本事实标签。

迄今为止，该数据集可以通过直接链接或从 Hugging Face 数据集下载：

• 直接链接： doclaynet_core.zip （28 GiB）， doclaynet_extra.zip （7.5 GiB）
• Hugging Face 数据集库： dataset DocLayNet

论文： DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis （06/02/2022）

模型描述

该模型在 512 个标记的重叠的段落块上进行了段落级别的微调。因此，模型使用了数据集的所有布局和文本数据进行了训练。

在推理时，通过计算最佳概率为每个段落边界框分配标签。

推理

参见笔记本： Document AI | Inference at paragraph level with a Document Understanding model (LayoutXLM base fine-tuned on DocLayNet dataset)

训练和评估数据

参见笔记本： Document AI | Fine-tune LayoutXLM base on DocLayNet base in any language at paragraph level (chunk of 512 tokens with overlap)

训练过程

训练超参数

训练期间使用了以下超参数：

• 学习率：2e-05
• 训练批量大小：8
• 评估批量大小：16
• 种子：42
• 优化器：Adam，β=(0.9,0.999)， ε=1e-08
• lr_scheduler_type：linear
• lr_scheduler_warmup_ratio：0.1
• num_epochs：4
• 混合精度训练：本机 AMP

训练结果

框架版本

• Transformers 4.27.3
• Pytorch 1.10.0+cu111
• Datasets 2.10.1
• Tokenizers 0.13.2

其他模型

• 行级别
  • Document Understanding model (finetuned LiLT base at line level on DocLayNet base) （准确度 | 标记：85.84% - 行：91.97%）
  • Document Understanding model (finetuned LayoutXLM base at line level on DocLayNet base) （准确度 | 标记：93.73% - 行：...）
• 段落级别
  • Document Understanding model (finetuned LiLT base at paragraph level on DocLayNet base) （准确度 | 标记：86.34% - 段落：68.15%）
  • Document Understanding model (finetuned LayoutXLM base at paragraph level on DocLayNet base) （准确度 | 标记：96.93% - 段落：86.55%）