模型:
biodatlab/score-claim-identification
This is a model card for detecting claims from an abstract of social science publications. The model takes an abstract, performs sentence tokenization, and predict a claim probability of each sentence. This model card is released by training on a SCORE dataset. It achieves the following results on the test set:
You can access the model with huggingface's transformers as follows:
import spacy from transformers import AutoTokenizer from transformers import AutoModelForSequenceClassification nlp = spacy.load("en_core_web_lg") model_name = "biodatlab/score-claim-identification" tokenizer_name = "allenai/scibert_scivocab_uncased" tokenizer = AutoTokenizer.from_pretrained(tokenizer_name) model = AutoModelForSequenceClassification.from_pretrained(model_name) def inference(abstract: str): """ Split an abstract into sentences and perform claim identification. """ if abstract.strip() == "": return "Please provide an abstract as an input." claims = [] sents = [sent.text for sent in nlp(abstract).sents] # a list of sentences inputs = tokenizer( sents, return_tensors="pt", truncation=True, padding="longest" ) logits = model(**inputs).logits preds = logits.argmax(dim=1) # convert logits to predictions claims = [sent for sent, pred in zip(sents, preds) if pred == 1] if len(claims) > 0: return ".\n".join(claims) else: return "No claims found from a given abstract." claims = inference(abstract) # string of claim joining with \n
Takes in a statement and classifies as Claim (1) or Null (0). Here are some examples -
Statement | Label |
---|---|
We consistently found that participants selectively chose to learn that bad (good) things happened to bad (good) people (Studies 1 to 7) that is, they selectively exposed themselves to deserved outcomes. | 1 (Claim) |
Members of higher status groups generalize characteristics of their ingroup to superordinate categories that serve as a frame of reference for comparisons with outgroups (ingroup projection). | 0 (Null) |
Motivational Interviewing helped the goal progress of those participants who, at pre-screening, reported engaging in many individual pro-environmental behaviors, but the more directive approach worked better for those participants who were less ready to change. | 1 (Claim) |
The following hyperparameters were used during training:
Training Loss | Step | Validation Loss | Accuracy | F1 | Precision | Recall |
---|---|---|---|---|---|---|
0.038000 | 3996 | 0.007086 | 0.997964 | 0.993499 | 0.995656 | 0.991350 |
See more on gradio application in biodatlab space.