Most previous unsupervised domain adaptation (UDA) methods for question answering(QA) require access to source domain data while fine-tuning the model for the target domain. Source domain data may, however, contain sensitive information and may be restricted. In this study, we investigate a more challenging setting, source-free UDA, in which we have only the pretrained source model and target domain data, without access to source domain data. We propose a novel self-training approach to QA models that integrates a unique mask module for domain adaptation. The mask is auto-adjusted to extract key domain knowledge while trained on the source domain. To maintain previously learned domain knowledge, certain mask weights are frozen during adaptation, while other weights are adjusted to mitigate domain shifts with pseudo-labeled samples generated in the target domain. %As part of the self-training process, we generate pseudo-labeled samples in the target domain based on models trained in the source domain. Our empirical results on four benchmark datasets suggest that our approach significantly enhances the performance of pretrained QA models on the target domain, and even outperforms models that have access to the source data during adaptation.

问答（QA）在回答定制域中的问题方面表现出了令人印象深刻的进展。然而，域的适应性仍然是质量检查系统最难以捉摸的挑战之一，尤其是当质量检查系统在源域中训练但部署在不同的目标域中时。在这项工作中，我们调查了问题分类对质量检查域适应的潜在好处。我们提出了一个新颖的框架：问题回答的问题分类（QC4QA）。具体而言，采用问题分类器将问题类分配给源数据和目标数据。然后，我们通过伪标记以自我监督的方式进行联合培训。为了优化，源和目标域之间的域间差异通过最大平均差异（MMD）距离降低。我们还最大程度地减少了同一问题类别的质量质量适应性表现的QA样本中的类内部差异。据我们所知，这是质量检查域适应中的第一部作品，以通过自我监督的适应来利用问题分类。我们证明了拟议的QC4QA的有效性，并在多个数据集上针对最先进的基线进行了一致的改进。

生物医学机器阅读理解（生物医学MRC）旨在理解复杂的生物医学叙事，并协助医疗保健专业人员从中检索信息。现代神经网络的MRC系统的高性能取决于高质量的大规模，人为宣传的培训数据集。在生物医学领域中，创建此类数据集的一个至关重要的挑战是域知识的要求，引起了标记数据的稀缺性以及从标记的通用（源）域转移学习到生物医学（目标）域的需求。然而，由于主题方差，通用和生物医学领域之间的边际分布存在差异。因此，从在通用域上训练的模型到生物医学领域的模型直接转移学会的表示可能会损害模型的性能。我们为生物医学机器阅读理解任务（BioAdapt-MRC）提供了基于对抗性学习的域适应框架，这是一种基于神经网络的方法，可解决一般和生物医学域数据之间边际分布中的差异。 Bioadapt-MRC松弛了生成伪标签的需求，以训练表现出色的生物医学MRC模型。我们通过将生物ADAPT-MRC与三种广泛使用的基准生物医学MRC数据集进行比较，从而广泛评估了生物ADAPT-MRC的性能-Bioasq-7B，BioASQ-8B和BioASQ-9B。我们的结果表明，如果不使用来自生物医学领域的任何合成或人类通知的数据，Bioadapt-MRC可以在这些数据集中实现最先进的性能。可用性：bioadapt-MRC可作为开放源项目免费获得，\ url {https://github.com/mmahbub/bioadapt-mrc}。

预审前的语言模型通过提供高质量的上下文化单词嵌入来显着改善了下游语言理解任务（包括提取性问题）的性能。但是，培训问答模型仍然需要大量特定域的注释数据。在这项工作中，我们提出了一个合作的自我训练框架RGX，用于自动生成更非平凡的问题 - 解答对以提高模型性能。 RGX建立在带有答案实体识别器，问题生成器和答案提取器的交互式学习环境的蒙版答案提取任务上。给定带有蒙版实体的段落，生成器会在实体周围生成一个问题，并培训了提取器，以提取蒙面实体，并使用生成的问题和原始文本。该框架允许对任何文本语料库的问题产生和回答模型进行培训，而无需注释。实验结果表明，RGX优于最先进的语言模型（SOTA）的语言模型，并在标准提问基准的基准上采用转移学习方法，并在给定的模型大小和传输学习设置下产生新的SOTA性能。

Neural approaches have become very popular in the domain of Question Answering, however they require a large amount of annotated data. Furthermore, they often yield very good performance but only in the domain they were trained on. In this work we propose a novel approach that combines data augmentation via question-answer generation with Active Learning to improve performance in low resource settings, where the target domains are diverse in terms of difficulty and similarity to the source domain. We also investigate Active Learning for question answering in different stages, overall reducing the annotation effort of humans. For this purpose, we consider target domains in realistic settings, with an extremely low amount of annotated samples but with many unlabeled documents, which we assume can be obtained with little effort. Additionally, we assume sufficient amount of labeled data from the source domain is available. We perform extensive experiments to find the best setup for incorporating domain experts. Our findings show that our novel approach, where humans are incorporated as early as possible in the process, boosts performance in the low-resource, domain-specific setting, allowing for low-labeling-effort question answering systems in new, specialized domains. They further demonstrate how human annotation affects the performance of QA depending on the stage it is performed.

Recent advances in open-domain question answering (ODQA) have demonstrated impressive accuracy on standard Wikipedia style benchmarks. However, it is less clear how robust these models are and how well they perform when applied to real-world applications in drastically different domains. While there has been some work investigating how well ODQA models perform when tested for out-of-domain (OOD) generalization, these studies have been conducted only under conservative shifts in data distribution and typically focus on a single component (ie. retrieval) rather than an end-to-end system. In response, we propose a more realistic and challenging domain shift evaluation setting and, through extensive experiments, study end-to-end model performance. We find that not only do models fail to generalize, but high retrieval scores often still yield poor answer prediction accuracy. We then categorize different types of shifts and propose techniques that, when presented with a new dataset, predict if intervention methods are likely to be successful. Finally, using insights from this analysis, we propose and evaluate several intervention methods which improve end-to-end answer F1 score by up to 24 points.

自我监督的预审查能够为各种视觉文档理解（VDU）任务产生可转移的表示。但是，尚未研究此类表示在测试时间时适应新分配变化的能力。我们提出了Docta，这是一种用于文档的新型测试时间适应方法，该方法通过掩盖的视觉语言建模来利用交叉模式自我观察学习以及伪标签，以适应\ textit {source}域中学习的模型，以使其{source}域中为一个未标记的\ textit {textit {目标}域在测试时间。我们还使用现有的公共数据集介绍了新的基准测试，用于各种VDU任务，包括实体识别，键值提取和文档视觉问题回答任务，其中Doctta将源模型性能提高到1.79 \％（F1分数），3.43 \％（3.43 \％）（F1得分）和17.68 \％（ANLS得分），同时大大降低了目标数据的校准误差。

用于机器阅读理解（MRC）的大多数领域适应方法都使用预先训练的问题解答（QA）构造模型来生成用于MRC传输的伪QA对。这样的过程将不可避免地引入不匹配的对（即嘈杂的对应关系），因此由于i）目标文档中不可用的QA对，ii）在将QA构造模型应用于目标域时的域移位。毫无疑问，嘈杂的信件将退化MRC的性能，但是现有作品忽略了MRC的性能。为了解决这样一个未触及的问题，我们建议通过使用与文档相关的对话以及MRC的新域适应方法来构建质量检查对。具体而言，我们建议用于机器阅读理解理解（RMRC）方法的强大域适应性，该方法由答案提取器（AE），问题选择器（QS）和MRC模型组成。具体而言，RMRC通过通过AE估算与文档的相关性来滤除无关的答案，并通过通过QS将候选问题融合在多轮对话聊天中来提取问题。使用提取的QA对，MRC进行了微调，并提供了反馈，以通过一种新颖的增强自我训练方法优化QS。得益于QS的优化，我们的方法将大大减轻域转移引起的嘈杂对应问题。据我们所知，这可能是揭示噪声对应性在域适应MRC模型中的影响的第一个研究，并显示出一种可行的方法来实现与错配对的鲁棒性。在三个数据集上进行的广泛实验证明了我们方法的有效性。

无监督的问题回答是一项有吸引力的任务，因为它在标签数据上的独立性。以前的作品通常使用启发式规则以及预先培训的模型来构建数据和训练QA模型。但是，这些作品中的大多数都将命名为实体（NE）视为唯一的答案类型，它忽略了现实世界中答案的高度多样性。为了解决这个问题，我们提出了一种新颖的无监督方法，通过多样化的答案，名为Diverseqa。具体而言，所提出的方法由三个模块组成：数据构建，数据增强和降解过滤器。首先，数据构建模块将提取的命名实体扩展到一个较长的句子成分，作为构建具有不同答案的质量检查数据集的新答案跨度。其次，数据增强模块通过嵌入级别的对抗训练采用答案型依赖性数据增强过程。第三，denoising滤波器模块旨在减轻构造数据中的噪声。广泛的实验表明，所提出的方法在五个基准数据集上优于先前的无监督模型，包括squeadv1.1，newsqa，triviaqa，bioasq和Duorc。此外，提出的方法在少量学习设置中显示出强劲的性能。

预训练的语言模型（PLM）在各种自然语言理解任务上取得了巨大的成功。另一方面，对PLM的简单微调对于特定于领域的任务可能是次优的，因为它们不可能涵盖所有域中的知识。尽管PLM的自适应预培训可以帮助他们获得特定于领域的知识，但需要大量的培训成本。此外，自适应预训练可能会通过造成灾难性忘记其常识来损害PLM在下游任务上的表现。为了克服PLM适应性适应性预训练的这种局限性，我们提出了一个新颖的域名适应框架，用于将PLMS创造为知识增强语言模型适应性（KALA），该框架调节了PLM的中间隐藏表示与域中的中间隐藏表示，由实体和实体和实体和实体和实体构成他们的关系事实。我们验证了Kala在问题答案中的性能，并在各个域的多个数据集上命名实体识别任务。结果表明，尽管在计算上有效，但我们的Kala在很大程度上优于适应性预训练。代码可在以下网址获得：https：//github.com/nardien/kala/。

大型语言模型在各种任务上显示出令人印象深刻的几次结果。但是，当知识是此类结果的关键时，就像问题回答和事实检查之类的任务一样，似乎需要存储知识的大量参数计数。众所周知，检索增强模型可以在不需要多个参数的情况下在知识密集的任务上表现出色，但是目前尚不清楚它们是否在几个弹药设置中工作。在这项工作中，我们介绍了地图集，这是一个经过精心设计和预先训练的增强语言模型，能够通过很少的培训示例学习知识密集型任务。我们对包括MMLU，苏格兰短裙和归类等各种任务进行评估，并研究文档索引内容的影响，表明它可以很容易地进行更新。值得注意的是，在自然问题上仅使用64个示例在自然问题上达到超过42 \％的准确性，尽管参数少了50倍，但比540B参数模型的表现优于540b参数模型。

尽管最近在改善错误信息检测系统的性能方面取得了进展，但在看不见的领域中进行错误信息进行分类仍然是一个难以捉摸的挑战。为了解决这个问题，一种常见的方法是引入域名评论家并鼓励域不变的输入功能。但是，早期的错误信息通常证明了针对现有的错误信息数据（例如，COVID-19数据集中的类不平衡）的条件和标签转移，这使得这种方法在检测早期错误信息方面的有效性较小。在本文中，我们提出了早期错误信息检测（CANMD）的对比适应网络。具体而言，我们利用伪标签来生成高信心的目标示例，用于与源数据的联合培训。我们还设计了标签校正成分，以估算和校正源和目标域之间的标签移动（即类先验）。此外，对比度适应损失已集成在目标函数中，以减少类内部差异并扩大阶层间差异。因此，改编的模型学习了校正的类先验和两个域之间不变的条件分布，以改善目标数据分布的估计。为了证明所提出的CANMD的有效性，我们研究了Covid-19的早期错误信息检测的案例，并使用多个现实世界数据集进行了广泛的实验。结果表明，与最先进的基线相比，CANMD可以有效地将错误信息检测系统适应不见的Covid-19目标域，并有显着改进。

深度学习模型的最新发展，捕捉作物物候的复杂的时间模式有卫星图像时间序列（坐在），大大高级作物分类。然而，当施加到目标区域从训练区空间上不同的，这些模型差没有任何目标标签由于作物物候区域之间的时间位移进行。为了解决这个无人监督跨区域适应环境，现有方法学域不变特征没有任何目标的监督，而不是时间偏移本身。因此，这些技术提供了SITS只有有限的好处。在本文中，我们提出TimeMatch，一种新的无监督领域适应性方法SITS直接占时移。 TimeMatch由两个部分组成：1）时间位移的估计，其估计具有源极训练模型的未标记的目标区域的时间偏移，和2）TimeMatch学习，它结合了时间位移估计与半监督学习到一个分类适应未标记的目标区域。我们还引进了跨区域适应的开放式访问的数据集与来自欧洲四个不同区域的旁边。在此数据集，我们证明了TimeMatch优于所有竞争的方法，通过11％的在五个不同的适应情景F1-得分，创下了新的国家的最先进的跨区域适应性。

Recent advances in NLP are brought by a range of large-scale pretrained language models (PLMs). These PLMs have brought significant performance gains for a range of NLP tasks, circumventing the need to customize complex designs for specific tasks. However, most current work focus on finetuning PLMs on a domain-specific datasets, ignoring the fact that the domain gap can lead to overfitting and even performance drop. Therefore, it is practically important to find an appropriate method to effectively adapt PLMs to a target domain of interest. Recently, a range of methods have been proposed to achieve this purpose. Early surveys on domain adaptation are not suitable for PLMs due to the sophisticated behavior exhibited by PLMs from traditional models trained from scratch and that domain adaptation of PLMs need to be redesigned to take effect. This paper aims to provide a survey on these newly proposed methods and shed light in how to apply traditional machine learning methods to newly evolved and future technologies. By examining the issues of deploying PLMs for downstream tasks, we propose a taxonomy of domain adaptation approaches from a machine learning system view, covering methods for input augmentation, model optimization and personalization. We discuss and compare those methods and suggest promising future research directions.

在本文中，我们提出了一种使用域鉴别特征模块的双模块网络架构，以鼓励域不变的特征模块学习更多域不变的功能。该建议的架构可以应用于任何利用域不变功能的任何模型，用于无监督域适应，以提高其提取域不变特征的能力。我们在作为代表性算法的神经网络（DANN）模型的区域 - 对抗训练进行实验。在培训过程中，我们为两个模块提供相同的输入，然后分别提取它们的特征分布和预测结果。我们提出了差异损失，以找到预测结果的差异和两个模块之间的特征分布。通过对抗训练来最大化其特征分布和最小化其预测结果的差异，鼓励两个模块分别学习更多域歧视和域不变特征。进行了广泛的比较评估，拟议的方法在大多数无监督的域适应任务中表现出最先进的。

Unsupervised domain adaptation (UDA) via deep learning has attracted appealing attention for tackling domain-shift problems caused by distribution discrepancy across different domains. Existing UDA approaches highly depend on the accessibility of source domain data, which is usually limited in practical scenarios due to privacy protection, data storage and transmission cost, and computation burden. To tackle this issue, many source-free unsupervised domain adaptation (SFUDA) methods have been proposed recently, which perform knowledge transfer from a pre-trained source model to unlabeled target domain with source data inaccessible. A comprehensive review of these works on SFUDA is of great significance. In this paper, we provide a timely and systematic literature review of existing SFUDA approaches from a technical perspective. Specifically, we categorize current SFUDA studies into two groups, i.e., white-box SFUDA and black-box SFUDA, and further divide them into finer subcategories based on different learning strategies they use. We also investigate the challenges of methods in each subcategory, discuss the advantages/disadvantages of white-box and black-box SFUDA methods, conclude the commonly used benchmark datasets, and summarize the popular techniques for improved generalizability of models learned without using source data. We finally discuss several promising future directions in this field.

Existing metrics for evaluating the quality of automatically generated questions such as BLEU, ROUGE, BERTScore, and BLEURT compare the reference and predicted questions, providing a high score when there is a considerable lexical overlap or semantic similarity between the candidate and the reference questions. This approach has two major shortcomings. First, we need expensive human-provided reference questions. Second, it penalises valid questions that may not have high lexical or semantic similarity to the reference questions. In this paper, we propose a new metric, RQUGE, based on the answerability of the candidate question given the context. The metric consists of a question-answering and a span scorer module, in which we use pre-trained models from the existing literature, and therefore, our metric can be used without further training. We show that RQUGE has a higher correlation with human judgment without relying on the reference question. RQUGE is shown to be significantly more robust to several adversarial corruptions. Additionally, we illustrate that we can significantly improve the performance of QA models on out-of-domain datasets by fine-tuning on the synthetic data generated by a question generation model and re-ranked by RQUGE.

基于强大的预训练语言模型（PLM）的密集检索方法（DR）方法取得了重大进步，并已成为现代开放域问答系统的关键组成部分。但是，他们需要大量的手动注释才能进行竞争性，这是不可行的。为了解决这个问题，越来越多的研究作品最近着重于在低资源场景下改善DR绩效。这些作品在培训所需的资源和采用各种技术的资源方面有所不同。了解这种差异对于在特定的低资源场景下选择正确的技术至关重要。为了促进这种理解，我们提供了针对低资源DR的主流技术的彻底结构化概述。根据他们所需的资源，我们将技术分为三个主要类别：（1）仅需要文档； （2）需要文件和问题； （3）需要文档和提问对。对于每种技术，我们都会介绍其一般形式算法，突出显示开放的问题和利弊。概述了有希望的方向以供将来的研究。

Aspect-based sentiment analysis (ABSA) aims at extracting opinionated aspect terms in review texts and determining their sentiment polarities, which is widely studied in both academia and industry. As a fine-grained classification task, the annotation cost is extremely high. Domain adaptation is a popular solution to alleviate the data deficiency issue in new domains by transferring common knowledge across domains. Most cross-domain ABSA studies are based on structure correspondence learning (SCL), and use pivot features to construct auxiliary tasks for narrowing down the gap between domains. However, their pivot-based auxiliary tasks can only transfer knowledge of aspect terms but not sentiment, limiting the performance of existing models. In this work, we propose a novel Syntax-guided Domain Adaptation Model, named SDAM, for more effective cross-domain ABSA. SDAM exploits syntactic structure similarities for building pseudo training instances, during which aspect terms of target domain are explicitly related to sentiment polarities. Besides, we propose a syntax-based BERT mask language model for further capturing domain-invariant features. Finally, to alleviate the sentiment inconsistency issue in multi-gram aspect terms, we introduce a span-based joint aspect term and sentiment analysis module into the cross-domain End2End ABSA. Experiments on five benchmark datasets show that our model consistently outperforms the state-of-the-art baselines with respect to Micro-F1 metric for the cross-domain End2End ABSA task.

半监督域适应（SSDA）是将学习者调整到新域，只有一小组标记的数据集在源域上给出时，只有一小组标记的样本。在本文中，我们提出了一种基于对的SSDA方法，使用用样品对的自蒸馏来适应靶域的模型。每个样本对由来自标记数据集（即源或标记为目标）的教师样本以及来自未标记数据集的学生样本（即，未标记的目标）组成。我们的方法通过在教师和学生之间传输中间样式来生成助手功能，然后通过最小化学生和助手之间的输出差异来培训模型。在培训期间，助手逐渐弥合了两个域之间的差异，从而让学生容易地从老师那里学习。标准基准测试的实验评估表明，我们的方法有效地减少了域间和域内的差异，从而实现了对最近的方法的显着改进。