We investigate the efficacy of treating all the parameters in a Bayesian neural network stochastically and find compelling theoretical and empirical evidence that this standard construction may be unnecessary. To this end, we prove that expressive predictive distributions require only small amounts of stochasticity. In particular, partially stochastic networks with only $n$ stochastic biases are universal probabilistic predictors for $n$-dimensional predictive problems. In empirical investigations, we find no systematic benefit of full stochasticity across four different inference modalities and eight datasets; partially stochastic networks can match and sometimes even outperform fully stochastic networks, despite their reduced memory costs.

我们引入隐深自适应设计（iDAD），在实时与隐性模型进行适应性实验的新方法。iDAD通过学习设计政策网络的前期，然后可以在实验时快速部署摊销贝叶斯优化实验设计（BOED）的成本。该iDAD网络可以在其模拟微样品，不同于需要一个封闭的形式可能性和条件独立实验以前的设计政策工作的任何模型进行训练。在部署时，iDAD允许以毫秒为单位进行设计决策，而相比之下，需要实验本身期间繁重的计算传统BOED方法。我们说明了多项实验iDAD的适用性，并表明它提供了与隐式模型进行适应性设计一个快速和有效的机制。

我们提出了一种在线状态估计和参数学习中的变异方法（SSMS），这是一种无处不在的序列数据的潜在变量模型。根据标准批处理变异技术，我们使用随机梯度同时优化对数模型参数的对数证据的下限和状态后分布的变异近似。但是，与现有方法不同，我们的方法能够完全在线运作，因此，尽管联合后分布的维度越来越不断增长，但在合并后不需要重新审视，并且在每个时间步骤中的更新成本保持恒定国家。这是通过利用该联合后验分布及其变异近似的向后分解，并与贝尔曼型递归相结合的证据下限及其梯度来实现。我们在几个示例中证明了该方法的性能，包括高维SSM和顺序变异自动编码器。

Multimodal VAEs seek to model the joint distribution over heterogeneous data (e.g.\ vision, language), whilst also capturing a shared representation across such modalities. Prior work has typically combined information from the modalities by reconciling idiosyncratic representations directly in the recognition model through explicit products, mixtures, or other such factorisations. Here we introduce a novel alternative, the MEME, that avoids such explicit combinations by repurposing semi-supervised VAEs to combine information between modalities implicitly through mutual supervision. This formulation naturally allows learning from partially-observed data where some modalities can be entirely missing -- something that most existing approaches either cannot handle, or do so to a limited extent. We demonstrate that MEME outperforms baselines on standard metrics across both partial and complete observation schemes on the MNIST-SVHN (image-image) and CUB (image-text) datasets. We also contrast the quality of the representations learnt by mutual supervision against standard approaches and observe interesting trends in its ability to capture relatedness between data.

在Mackay（1992）上展开，我们认为，用于主动学习的基于模式的方法 - 类似的基于模型 - 如秃顶 - 具有基本的缺点：它们未直接解释输入变量的测试时间分布。这可以导致采集策略中的病理，因为模型参数的最大信息是最大信息，可能不是最大地信息，例如，当池集中的数据比最终预测任务的数据更大时，或者池和试验样品的分布不同。为了纠正这一点，我们重新审视了基于最大化关于可能的未来预测的预期信息的收购策略，参考这是预期的预测信息增益（EPIG）。由于EPIG对批量采集不扩展，我们进一步检查了替代策略，秃头和EPIG之间的混合，我们称之为联合预测信息增益（Jepig）。我们考虑在各种数据集中使用贝叶斯神经网络的主动学习，检查池集中分布班下的行为。

我们表明，概率编程系统（PPSS）的标准计算管道可能无效地估计期望，并介绍期望编程的概念以解决这一问题。在预期的编程中，后端推理引擎的目的是直接估计程序的预期返回值，而不是近似其条件分布。这种区别虽然微妙，但使我们能够通过根据我们关心的期望来将计算定制计算来实现对标准PPS计算管道的实质性改进。我们通过扩展PPS图灵以允许自动运行的所谓目标推理来实现我们的期望编程概念，图灵（EPT）中的期望编程（EPT）的特定实例。然后，我们从理论上验证EPT的统计声音，并表明它在实践中提供了可观的经验收益。

We present a principled approach to incorporating labels in VAEs that captures the rich characteristic information associated with those labels. While prior work has typically conflated these by learning latent variables that directly correspond to label values, we argue this is contrary to the intended effect of supervision in VAEs-capturing rich label characteristics with the latents. For example, we may want to capture the characteristics of a face that make it look young, rather than just the age of the person. To this end, we develop the CCVAE, a novel VAE model and concomitant variational objective which captures label characteristics explicitly in the latent space, eschewing direct correspondences between label values and latents. Through judicious structuring of mappings between such characteristic latents and labels, we show that the CCVAE can effectively learn meaningful representations of the characteristics of interest across a variety of supervision schemes. In particular, we show that the CCVAE allows for more effective and more general interventions to be performed, such as smooth traversals within the characteristics for a given label, diverse conditional generation, and transferring characteristics across datapoints.

A Digital Twin (DT) is a simulation of a physical system that provides information to make decisions that add economic, social or commercial value. The behaviour of a physical system changes over time, a DT must therefore be continually updated with data from the physical systems to reflect its changing behaviour. For resource-constrained systems, updating a DT is non-trivial because of challenges such as on-board learning and the off-board data transfer. This paper presents a framework for updating data-driven DTs of resource-constrained systems geared towards system health monitoring. The proposed solution consists of: (1) an on-board system running a light-weight DT allowing the prioritisation and parsimonious transfer of data generated by the physical system; and (2) off-board robust updating of the DT and detection of anomalous behaviours. Two case studies are considered using a production gas turbine engine system to demonstrate the digital representation accuracy for real-world, time-varying physical systems.

Learning to predict masked tokens in a sequence has been shown to be a powerful pretraining objective for large-scale language models. After training, such masked language models can provide distributions of tokens conditioned on bidirectional context. In this short draft, we show that such bidirectional conditionals often demonstrate considerable inconsistencies, i.e., they can not be derived from a coherent joint distribution when considered together. We empirically quantify such inconsistencies in the simple scenario of bigrams for two common styles of masked language models: T5-style and BERT-style. For example, we show that T5 models often confuse its own preference regarding two similar bigrams. Such inconsistencies may represent a theoretical pitfall for the research work on sampling sequences based on the bidirectional conditionals learned by BERT-style MLMs. This phenomenon also means that T5-style MLMs capable of infilling will generate discrepant results depending on how much masking is given, which may represent a particular trust issue.

We study the problem of planning under model uncertainty in an online meta-reinforcement learning (RL) setting where an agent is presented with a sequence of related tasks with limited interactions per task. The agent can use its experience in each task and across tasks to estimate both the transition model and the distribution over tasks. We propose an algorithm to meta-learn the underlying structure across tasks, utilize it to plan in each task, and upper-bound the regret of the planning loss. Our bound suggests that the average regret over tasks decreases as the number of tasks increases and as the tasks are more similar. In the classical single-task setting, it is known that the planning horizon should depend on the estimated model's accuracy, that is, on the number of samples within task. We generalize this finding to meta-RL and study this dependence of planning horizons on the number of tasks. Based on our theoretical findings, we derive heuristics for selecting slowly increasing discount factors, and we validate its significance empirically.