Supervised approaches generally rely on majority-based labels. However, it is hard to achieve high agreement among annotators in subjective tasks such as hate speech detection. Existing neural network models principally regard labels as categorical variables, while ignoring the semantic information in diverse label texts. In this paper, we propose AnnoBERT, a first-of-its-kind architecture integrating annotator characteristics and label text with a transformer-based model to detect hate speech, with unique representations based on each annotator's characteristics via Collaborative Topic Regression (CTR) and integrate label text to enrich textual representations. During training, the model associates annotators with their label choices given a piece of text; during evaluation, when label information is not available, the model predicts the aggregated label given by the participating annotators by utilising the learnt association. The proposed approach displayed an advantage in detecting hate speech, especially in the minority class and edge cases with annotator disagreement. Improvement in the overall performance is the largest when the dataset is more label-imbalanced, suggesting its practical value in identifying real-world hate speech, as the volume of hate speech in-the-wild is extremely small on social media, when compared with normal (non-hate) speech. Through ablation studies, we show the relative contributions of annotator embeddings and label text to the model performance, and tested a range of alternative annotator embeddings and label text combinations.
translated by 谷歌翻译
An important component of an automated fact-checking system is the claim check-worthiness detection system, which ranks sentences by prioritising them based on their need to be checked. Despite a body of research tackling the task, previous research has overlooked the challenging nature of identifying check-worthy claims across different topics. In this paper, we assess and quantify the challenge of detecting check-worthy claims for new, unseen topics. After highlighting the problem, we propose the AraCWA model to mitigate the performance deterioration when detecting check-worthy claims across topics. The AraCWA model enables boosting the performance for new topics by incorporating two components for few-shot learning and data augmentation. Using a publicly available dataset of Arabic tweets consisting of 14 different topics, we demonstrate that our proposed data augmentation strategy achieves substantial improvements across topics overall, where the extent of the improvement varies across topics. Further, we analyse the semantic similarities between topics, suggesting that the similarity metric could be used as a proxy to determine the difficulty level of an unseen topic prior to undertaking the task of labelling the underlying sentences.
translated by 谷歌翻译
为了减轻数据稀缺性对事实检查系统的影响,我们专注于几乎没有声称验证。尽管最近通过提出高级语言模型进行了几次射击分类的工作,但数据注释优先级的研究缺乏研究,可以改善为最佳模型性能标记的少数镜头的选择。我们提出了活跃的宠物,这是一种新型的加权方法,它利用基于各种语言模型的模式开发培训(PET)模型的合奏来积极选择未标记的数据作为注释的候选者。使用活跃的宠物进行数据选择,在两个技术事实检查数据集上以及使用六个不同的预审前的语言模型上显示了对最先进的主动学习方法的一致改进。我们通过Active Pets-O展示了进一步的改进,该宠物O进一步整合了过采样策略。我们的方法使有效的实例可以被标记为无标记的数据丰富,但标签资源受到限制,从而始终改善了几次索赔验证性能。我们的代码将在出版后提供。
translated by 谷歌翻译
在文本分类模型由于数据变化而随着时间的变化而下降的情况下,其持续时间持续时间的模型的开发很重要。预测模型随着时间的推移能力的能力可以帮助设计模型,这些模型可以在更长的时间内有效使用。在本文中,我们通过评估各种语言模型和分类算法随着时间的推移持续存在的能力,以及数据集特性如何帮助预测不同模型的时间稳定性,从而研究了这个问题。我们在跨越6到19年的三个数据集上执行纵向分类实验,并涉及各种任务和类型的数据。我们发现,人们可以根据(i)模型在限制时间段内的性能及其外推到更长的时间段,以及(ii)数据集的语言特征,以及(ii)数据集的语言特征,如何估算模型如何在时间上保持其性能。例如不同年份的子集之间的熟悉程度。这些实验的发现对文本分类模型的设计具有重要意义,目的是保留随着时间的推移性能。
translated by 谷歌翻译
通过匿名和可访问性,社交媒体平台促进了仇恨言论的扩散,提示在开发自动方法以识别这些文本时提高研究。本文探讨了使用各种深度神经网络模型架构(如长短期内存(LSTM)和卷积神经网络(CNN)的文本中性别歧视分类。这些网络与来自变压器(BERT)和Distilbert模型的双向编码器表示形式的传输学习一起使用,以及数据增强,以在社交中的性别歧视识别中对推文和GAB的数据集进行二进制和多种性别歧视分类Iberlef 2021中的网络(存在)任务。看到模型与竞争对手的比较,使用BERT和多滤波器CNN模型进行了最佳性能。数据增强进一步提高了多级分类任务的结果。本文还探讨了模型所做的错误,并讨论了由于标签的主观性和社交媒体中使用的自然语言的复杂性而自动对性别歧视的难度。
translated by 谷歌翻译
在交叉语言设置中讨厌语音检测代表所有中型和大型在线平台的最重要的感兴趣区域。未能在全球范围内妥善解决这个问题已经过时地导致了道德上可疑的现实生活事件,人类死亡和仇恨本身的永久。本文说明了微调改变的多语言变压器模型(Mbert,XLM-Roberta)关于这一重要的社会数据科学任务,与英语到法语,反之亦然和每种语言的交叉思考,包括关于迭代改进和比较误差分析的部分。
translated by 谷歌翻译
社交媒体平台为挖掘公众舆论提供了众多社会兴趣问题的金矿。意见采矿是一个问题,可以通过捕获和汇总各个社交媒体职位的立场,作为支持,反对或者在手头上的问题上进行。虽然大多数姿态检测工作已经调查了具有有限时间覆盖率的数据集,但最近提高了调查纵向数据集的兴趣。在新数据中观察到的语言和行为模式中的演变动态,依次适应姿态检测系统来处理变化。在本调查论文中,我们研究了计算语言学与数字媒体人类交流的交叉口。在考虑动态的新兴研究中,我们在探索不同的语义和语用因素,探讨了影响语言数据的不同语义和语用因素,特别是审查。我们进一步讨论了在社交媒体中捕获姿态动态的当前方向。我们组织处理姿态动态的挑战,确定公开挑战,并在三个关键方面讨论未来的方向:话语,背景和影响。
translated by 谷歌翻译