模型:
viktor-enzell/wav2vec2-large-voxrex-swedish-4gram
声学模型的训练是由KBLab完成的。更多详情请参见 VoxRex-C 。该库通过社交媒体4-gram语言模型扩展了声学模型,以提高性能。
VoxRex-C通过从Språkbanken的 The Swedish Culturomics Gigaword Corpus 子集中提取的数据建立了一个4-gram语言模型。该子集包含2010年至2015年社交媒体类型的40M个词。
import torch from transformers import pipeline # Load the model. Using GPU if available model_name = 'viktor-enzell/wav2vec2-large-voxrex-swedish-4gram' device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') pipe = pipeline(model=model_name).to(device) # Run inference on an audio file output = pipe('path/to/audio.mp3')['text']更详细的使用示例,包括音频预处理
示例为转录1%的Common Voice测试集分割。该模型需要16kHz的音频,因此对于其他采样率的音频,会进行重采样到16kHz。
from transformers import Wav2Vec2ForCTC, Wav2Vec2ProcessorWithLM from datasets import load_dataset import torch import torchaudio.functional as F # Import model and processor. Using GPU if available model_name = 'viktor-enzell/wav2vec2-large-voxrex-swedish-4gram' device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') model = Wav2Vec2ForCTC.from_pretrained(model_name).to(device); processor = Wav2Vec2ProcessorWithLM.from_pretrained(model_name) # Import and process speech data common_voice = load_dataset('common_voice', 'sv-SE', split='test[:1%]') def speech_file_to_array(sample): # Convert speech file to array and downsample to 16 kHz sampling_rate = sample['audio']['sampling_rate'] sample['speech'] = F.resample(torch.tensor(sample['audio']['array']), sampling_rate, 16_000) return sample common_voice = common_voice.map(speech_file_to_array) # Run inference inputs = processor(common_voice['speech'], sampling_rate=16_000, return_tensors='pt', padding=True).to(device) with torch.no_grad(): logits = model(**inputs).logits transcripts = processor.batch_decode(logits.cpu().numpy()).text
n-gram模型的文本数据经过预处理,去除不属于wav2vec 2.0词汇表的字符,并将所有字符转换为大写。预处理后,将每个文本样本存储在一个文本文件的新行中,并估计一个 KenLM 模型。有关详细信息,请参见 this tutorial 。
该模型在完整的Common Voice测试集6.1上进行了评估。VoxRex-C在没有语言模型的情况下实现了9.03%的WER,使用语言模型时WER为6.47%。