模型:
nlpaueb/bert-base-uncased-echr
LEGAL-BERT是用于法律领域的BERT模型系列,旨在辅助法律NLP研究、计算法律和法律技术应用。为了预训练LEGAL-BERT的不同变种,我们从公开可用资源中收集了12GB的多样化英文法律文本,其中包括各个领域(如立法、法庭案例、合同)的内容。子域变体(CONTRACTS,EURLEX,ECHR)和/或在特定领域任务中,与直接使用BERT相比,通用LEGAL-BERT表现更好。这是在ECHR案例上预训练的子域变体。
I. Chalkidis,M. Fergadiotis,P. Malakasiotis,N. Aletras和I. Androutsopoulos的“LEGAL-BERT:李尔泰的角色从法学院直出来的木偶”。在2020年成果方法在自然语言处理中(EMNLP2020)的发现(简短论文)中,将在线举行。 ( https://aclanthology.org/2020.findings-emnlp.261 )
LEGAL-BERT的预训练语料库包括:
来自EURLEX( http://eur-lex.europa.eu )的欧盟立法116,062份文件,可从欧盟出版办公室下运行的欧盟法仓库中公开获取。
来自英国立法门户网站( http://www.legislation.gov.uk )的61,826份英国立法文件,可公开获取。
来自欧洲法院(ECJ)的19,867件案例,也可从EURLEX获取。
来自欧洲人权法院(ECHR)( http://hudoc.echr.coe.int/eng )归档的12,554个案例。
来自美国各个法院的164,141个案例,托管在Case Law Access Project门户网站中( https://case.law )。
来自美国证券交易委员会(SECOM)的76,366个美国合同,存储在EDGAR数据库中( https://www.sec.gov/edgar.shtml )。
Model name | Model Path | Training corpora |
---|---|---|
CONTRACTS-BERT-BASE | nlpaueb/bert-base-uncased-contracts | US contracts |
EURLEX-BERT-BASE | nlpaueb/bert-base-uncased-eurlex | EU legislation |
ECHR-BERT-BASE | nlpaueb/bert-base-uncased-echr | ECHR cases |
LEGAL-BERT-BASE * | nlpaueb/legal-bert-base-uncased | All |
LEGAL-BERT-SMALL | nlpaueb/legal-bert-small-uncased | All |
* LEGAL-BERT-BASE是指Chalkidis等人(2020)论文中所称的LEGAL-BERT-SC的模型;这个模型是在下面提到的法律语料库中使用基于句子片段的令牌化器训练的新创建的词汇表进行训练而得到的。
**由于很多人对LEGAL-BERT-FP模型(依赖于原始BERT-BASE检查点的模型)表示了兴趣,它们已在Archive.org上发布( https://archive.org/details/legal_bert_fp ),因为这些模型是次要的,可能只对那些想深入研究Chalkidis等人(2020)文章中的开放问题的人感兴趣。
from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("nlpaueb/bert-base-uncased-echr") model = AutoModel.from_pretrained("nlpaueb/bert-base-uncased-echr")
Corpus | Model | Masked token | Predictions |
---|---|---|---|
BERT-BASE-UNCASED | |||
(Contracts) | This [MASK] Agreement is between General Motors and John Murray . | employment | ('new', '0.09'), ('current', '0.04'), ('proposed', '0.03'), ('marketing', '0.03'), ('joint', '0.02') |
(ECHR) | The applicant submitted that her husband was subjected to treatment amounting to [MASK] whilst in the custody of Adana Security Directorate | torture | ('torture', '0.32'), ('rape', '0.22'), ('abuse', '0.14'), ('death', '0.04'), ('violence', '0.03') |
(EURLEX) | Establishing a system for the identification and registration of [MASK] animals and regarding the labelling of beef and beef products . | bovine | ('farm', '0.25'), ('livestock', '0.08'), ('draft', '0.06'), ('domestic', '0.05'), ('wild', '0.05') |
CONTRACTS-BERT-BASE | |||
(Contracts) | This [MASK] Agreement is between General Motors and John Murray . | employment | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02') |
(ECHR) | The applicant submitted that her husband was subjected to treatment amounting to [MASK] whilst in the custody of Adana Security Directorate | torture | ('death', '0.39'), ('imprisonment', '0.07'), ('contempt', '0.05'), ('being', '0.03'), ('crime', '0.02') |
(EURLEX) | Establishing a system for the identification and registration of [MASK] animals and regarding the labelling of beef and beef products . | bovine | (('domestic', '0.18'), ('laboratory', '0.07'), ('household', '0.06'), ('personal', '0.06'), ('the', '0.04') |
EURLEX-BERT-BASE | |||
(Contracts) | This [MASK] Agreement is between General Motors and John Murray . | employment | ('supply', '0.11'), ('cooperation', '0.08'), ('service', '0.07'), ('licence', '0.07'), ('distribution', '0.05') |
(ECHR) | The applicant submitted that her husband was subjected to treatment amounting to [MASK] whilst in the custody of Adana Security Directorate | torture | ('torture', '0.66'), ('death', '0.07'), ('imprisonment', '0.07'), ('murder', '0.04'), ('rape', '0.02') |
(EURLEX) | Establishing a system for the identification and registration of [MASK] animals and regarding the labelling of beef and beef products . | bovine | ('live', '0.43'), ('pet', '0.28'), ('certain', '0.05'), ('fur', '0.03'), ('the', '0.02') |
ECHR-BERT-BASE | |||
(Contracts) | This [MASK] Agreement is between General Motors and John Murray . | employment | ('second', '0.24'), ('latter', '0.10'), ('draft', '0.05'), ('bilateral', '0.05'), ('arbitration', '0.04') |
(ECHR) | The applicant submitted that her husband was subjected to treatment amounting to [MASK] whilst in the custody of Adana Security Directorate | torture | ('torture', '0.99'), ('death', '0.01'), ('inhuman', '0.00'), ('beating', '0.00'), ('rape', '0.00') |
(EURLEX) | Establishing a system for the identification and registration of [MASK] animals and regarding the labelling of beef and beef products . | bovine | ('pet', '0.17'), ('all', '0.12'), ('slaughtered', '0.10'), ('domestic', '0.07'), ('individual', '0.05') |
LEGAL-BERT-BASE | |||
(Contracts) | This [MASK] Agreement is between General Motors and John Murray . | employment | ('settlement', '0.26'), ('letter', '0.23'), ('dealer', '0.04'), ('master', '0.02'), ('supplemental', '0.02') |
(ECHR) | The applicant submitted that her husband was subjected to treatment amounting to [MASK] whilst in the custody of Adana Security Directorate | torture | ('torture', '1.00'), ('detention', '0.00'), ('arrest', '0.00'), ('rape', '0.00'), ('death', '0.00') |
(EURLEX) | Establishing a system for the identification and registration of [MASK] animals and regarding the labelling of beef and beef products . | bovine | ('live', '0.67'), ('beef', '0.17'), ('farm', '0.03'), ('pet', '0.02'), ('dairy', '0.01') |
LEGAL-BERT-SMALL | |||
(Contracts) | This [MASK] Agreement is between General Motors and John Murray . | employment | ('license', '0.09'), ('transition', '0.08'), ('settlement', '0.04'), ('consent', '0.03'), ('letter', '0.03') |
(ECHR) | The applicant submitted that her husband was subjected to treatment amounting to [MASK] whilst in the custody of Adana Security Directorate | torture | ('torture', '0.59'), ('pain', '0.05'), ('ptsd', '0.05'), ('death', '0.02'), ('tuberculosis', '0.02') |
(EURLEX) | Establishing a system for the identification and registration of [MASK] animals and regarding the labelling of beef and beef products . | bovine | ('all', '0.08'), ('live', '0.07'), ('certain', '0.07'), ('the', '0.07'), ('farm', '0.05') |
请考虑文章"LEGAL-BERT:李尔泰的角色从法学院直出来的木偶"。Chalkidis等人,2020年,( https://aclanthology.org/2020.findings-emnlp.261 )
@inproceedings{chalkidis-etal-2020-legal, title = "{LEGAL}-{BERT}: The Muppets straight out of Law School", author = "Chalkidis, Ilias and Fergadiotis, Manos and Malakasiotis, Prodromos and Aletras, Nikolaos and Androutsopoulos, Ion", booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2020", month = nov, year = "2020", address = "Online", publisher = "Association for Computational Linguistics", doi = "10.18653/v1/2020.findings-emnlp.261", pages = "2898--2904" }
AUEB's Natural Language Processing Group 发展算法,模型和系统,使计算机能够处理和生成自然语言文本。
该团队目前的研究兴趣包括:
该团队隶属于希腊经济与商业雅典经济学与商务学院信息处理实验室。
Ilias Chalkidis 代表 AUEB's Natural Language Processing Group
| Github: @ilias.chalkidis | Twitter: @KiddoThe2B |