Stacking (or stacked generalization) is an ensemble learning method with one main distinctiveness from the rest: even though several base models are trained on the original data set, their predictions are further used as input data for one or more metamodels arranged in at least one extra layer. Composing a stack of models can produce high-performance outcomes, but it usually involves a trial-and-error process. Therefore, our previously developed visual analytics system, StackGenVis, was mainly designed to assist users in choosing a set of top-performing and diverse models by measuring their predictive performance. However, it only employs a single logistic regression metamodel. In this paper, we investigate the impact of alternative metamodels on the performance of stacking ensembles using a novel visualization tool, called MetaStackVis. Our interactive tool helps users to visually explore different singular and pairs of metamodels according to their predictive probabilities and multiple validation metrics, as well as their ability to predict specific problematic data instances. MetaStackVis was evaluated with a usage scenario based on a medical data set and via expert interviews.
translated by 谷歌翻译
尽管机器学习取得了巨大进步(ML),但数据不平衡的培训仍然在许多现实世界中构成挑战。在解决此问题的一系列不同技术中,采样算法被视为有效的解决方案。但是,问题更为根本,许多作品强调了实例硬度的重要性。这个问题是指管理不安全或可能嘈杂的实例的重要性,这些实例更可能被错误分类并作为分类绩效不佳的根本原因。本文介绍了Hardvis,这是一种视觉分析系统,旨在处理实例硬度,主要在分类场景中。我们提出的系统协助用户在视觉上比较数据类型的不同分布,根据局部特征选择实例类型,这些实例后来将受主动采样方法的影响,并验证来自底漆或过采样技术的建议对ML模型有益。此外,我们允许用户找到和采样轻松且难以对所有课程的培训实例进行分类,而不是统一地采样/过采样。用户可以从不同角度探索数据子集以决定所有这些参数,而HardVis则跟踪其步骤并评估模型在测试集中分别评估模型的预测性能。最终结果是一个均衡的数据集,可增强ML模型的预测能力。通过假设使用情况和用例证明了Hardvis的功效和有效性。最后,我们还研究了系统的有用,基于我们从ML专家那里收到的反馈。
translated by 谷歌翻译
装袋和升压是在机器学习(ml)中的两个流行的集合方法,产生许多单独的决策树。由于这些方法的固有组合特性,它们通常以预测性能更优于单决定树或其他ML模型。然而,为每个决策树生成许多决定路径,增加了模型的整体复杂性,并阻碍了其在需要值得信赖和可解释的决策的域中的域,例如金融,社会护理和保健。因此,随着决策的数量升高,袋装和升降算法(例如随机森林和自适应升压)的解释性降低。在本文中,我们提出了一种视觉分析工具,该工具旨在帮助用户通过彻底的视觉检查工作流程从这种ML模型中提取决策,包括选择一套鲁棒和不同的模型(源自不同的集合学习算法),选择重要的功能根据他们的全球贡献,决定哪些决定对于全球解释(或本地,具体案件)是必不可少的。结果是基于多个模型的协议和用户出口的探索手动决策的最终决定。最后,我们通过用例,使用场景和用户学习评估患者的适用性和有效性。
translated by 谷歌翻译
机器学习(ML)生命周期涉及一系列迭代步骤,从有效的收集和准备数据,包括复杂的特征工程流程,对结果的演示和改进,各种步骤中的各种算法选择。特征工程尤其可以对ML非常有益,导致许多改进,例如提高预测结果,降低计算时间,减少过度噪音,并提高培训期间所采取的决策背后的透明度。尽管如此,虽然存在多个视觉分析工具来监控和控制ML生命周期的不同阶段(特别是与数据和算法相关的阶段),但功能工程支持仍然不足。在本文中,我们提出了FightEnvi,一种专门设计用于协助特征工程过程的视觉分析系统。我们建议的系统可帮助用户选择最重要的功能,将原始功能转换为强大的替代方案,并进行不同的特征生成组合。此外,数据空间切片允许用户探索本地和全局尺度上的功能的影响。 Feationenvi利用多种自动特征选择技术;此外,它目视指导用户有统计证据的关于每个特征的影响(或功能的子集)。最终结果是通过多种验证度量评估的重新设计的重新设计特征。用两种用例和案例研究证明了FeatureenVI的有用性和适用性。我们还向评估我们系统的有效性以及评估我们系统的有效性的观众报告反馈。
translated by 谷歌翻译
Tongue cancer is a common oral cavity malignancy that originates in the mouth and throat. Much effort has been invested in improving its diagnosis, treatment, and management. Surgical removal, chemotherapy, and radiation therapy remain the major treatment for tongue cancer. The survival of patients determines the treatment effect. Previous studies have identified certain survival and risk factors based on descriptive statistics, ignoring the complex, nonlinear relationship among clinical and demographic variables. In this study, we utilize five cutting-edge machine learning models and clinical data to predict the survival of tongue cancer patients after treatment. Five-fold cross-validation, bootstrap analysis, and permutation feature importance are applied to estimate and interpret model performance. The prognostic factors identified by our method are consistent with previous clinical studies. Our method is accurate, interpretable, and thus useable as additional evidence in tongue cancer treatment and management.
translated by 谷歌翻译
Machine learning (ML) models are nowadays used in complex applications in various domains, such as medicine, bioinformatics, and other sciences. Due to their black box nature, however, it may sometimes be hard to understand and trust the results they provide. This has increased the demand for reliable visualization tools related to enhancing trust in ML models, which has become a prominent topic of research in the visualization community over the past decades. To provide an overview and present the frontiers of current research on the topic, we present a State-of-the-Art Report (STAR) on enhancing trust in ML models with the use of interactive visualization. We define and describe the background of the topic, introduce a categorization for visualization techniques that aim to accomplish this goal, and discuss insights and opportunities for future research directions. Among our contributions is a categorization of trust against different facets of interactive ML, expanded and improved from previous research. Our results are investigated from different analytical perspectives: (a) providing a statistical overview, (b) summarizing key findings, (c) performing topic analyses, and (d) exploring the data sets used in the individual papers, all with the support of an interactive web-based survey browser. We intend this survey to be beneficial for visualization researchers whose interests involve making ML models more trustworthy, as well as researchers and practitioners from other disciplines in their search for effective visualization techniques suitable for solving their tasks with confidence and conveying meaning to their data.
translated by 谷歌翻译
Named Entity Recognition and Intent Classification are among the most important subfields of the field of Natural Language Processing. Recent research has lead to the development of faster, more sophisticated and efficient models to tackle the problems posed by those two tasks. In this work we explore the effectiveness of two separate families of Deep Learning networks for those tasks: Bidirectional Long Short-Term networks and Transformer-based networks. The models were trained and tested on the ATIS benchmark dataset for both English and Greek languages. The purpose of this paper is to present a comparative study of the two groups of networks for both languages and showcase the results of our experiments. The models, being the current state-of-the-art, yielded impressive results and achieved high performance.
translated by 谷歌翻译
图数据管理和查询具有许多实际应用。当图形非常异构和/或用户不熟悉其结构时,即使用户无法描述连接,他们也可能需要找到如何在图中连接两个或多个节点的组。这仅由现有查询语言部分支持,这些语言允许搜索路径,但不适合连接三个或更多节点组的树。后者与NP-HARD组Steiner树问题有关,以前已考虑用于数据库中的关键字搜索。在这项工作中,我们正式展示了如何在诸如SPARQL或Cypher之类的图形语言中集成连接的树模式(CTPS,简称CTP),从而导致扩展查询语言(或简而言之)。然后,我们研究一组评估CTP的算法;我们概括了先前的关键字搜索工作,最重要的是(i)考虑双向边缘遍历遍历和(ii)允许用户选择任何分数功能来排名CTP结果。为了应对非常大的搜索空间,我们提出了一种有效的修剪技术,并正式建立了大量的情况,即使我们的算法molesp也可以完成修剪。我们的实验验证了我们在大量合成和现实世界中的CTP和EQL评估算法的性能。
translated by 谷歌翻译
深度学习(DL)模型为各种医学成像基准挑战提供了最先进的性能,包括脑肿瘤细分(BRATS)挑战。然而,局灶性病理多隔室分割(例如,肿瘤和病变子区)的任务特别具有挑战性,并且潜在的错误阻碍DL模型转化为临床工作流程。量化不确定形式的DL模型预测的可靠性,可以实现最不确定的地区的临床审查,从而建立信任并铺平临床翻译。最近,已经引入了许多不确定性估计方法,用于DL医学图像分割任务。开发指标评估和比较不确定性措施的表现将有助于最终用户制定更明智的决策。在本研究中,我们探索并评估在Brats 2019-2020任务期间开发的公制,以对不确定量化量化(Qu-Brats),并旨在评估和排列脑肿瘤多隔室分割的不确定性估计。该公制(1)奖励不确定性估计,对正确断言产生高置信度,以及在不正确的断言处分配低置信水平的估计数,(2)惩罚导致更高百分比的无关正确断言百分比的不确定性措施。我们进一步基准测试由14个独立参与的Qu-Brats 2020的分割不确定性,所有这些都参与了主要的Brats细分任务。总体而言,我们的研究结果证实了不确定性估计提供了分割算法的重要性和互补价值,因此突出了医学图像分析中不确定性量化的需求。我们的评估代码在HTTPS://github.com/ragmeh11/qu-brats公开提供。
translated by 谷歌翻译
使用环境模型和值函数,代理可以通过向不同长度展开模型来构造状态值的许多估计,并使用其值函数引导。我们的关键识别是,人们可以将这组价值估计视为一类合奏,我们称之为\ eNPH {隐式值合奏}(IVE)。因此,这些估计之间的差异可用作代理人的认知不确定性的代理;我们将此信号术语\ EMPH {Model-Value不一致}或\ EMPH {自给智而不一致。与先前的工作不同,该工作估计通过培训许多模型和/或价值函数的集合来估计不确定性,这种方法只需要在大多数基于模型的加强学习算法中学习的单一模型和价值函数。我们在单板和函数近似设置中提供了从像素的表格和函数近似设置中的经验证据是有用的(i)作为探索的信号,(ii)在分发班次下安全地行动,(iii),用于使用基于价值的规划模型。
translated by 谷歌翻译