Image-based head swapping task aims to stitch a source head to another source body flawlessly. This seldom-studied task faces two major challenges: 1) Preserving the head and body from various sources while generating a seamless transition region. 2) No paired head swapping dataset and benchmark so far. In this paper, we propose an image-based head swapping framework (HS-Diffusion) which consists of a semantic-guided latent diffusion model (SG-LDM) and a semantic layout generator. We blend the semantic layouts of source head and source body, and then inpaint the transition region by the semantic layout generator, achieving a coarse-grained head swapping. SG-LDM can further implement fine-grained head swapping with the blended layout as condition by a progressive fusion process, while preserving source head and source body with high-quality reconstruction. To this end, we design a head-cover augmentation strategy for training and a neck alignment trick for geometric realism. Importantly, we construct a new image-based head swapping benchmark and propose two tailor-designed metrics (Mask-FID and Focal-FID). Extensive experiments demonstrate the superiority of our framework. The code will be available: https://github.com/qinghew/HS-Diffusion.
translated by 谷歌翻译
利用Stylegan的表现力及其分离的潜在代码,现有方法可以实现对不同视觉属性的现实编辑,例如年龄和面部图像的性别。出现了一个有趣而又具有挑战性的问题:生成模型能否针对他们博学的先验进行反事实编辑?由于自然数据集中缺乏反事实样本,我们以文本驱动的方式研究了这个问题,并具有对比语言图像预言(剪辑),这些(剪辑)甚至可以为各种反事实概念提供丰富的语义知识。与内域操作不同,反事实操作需要更全面地剥削夹包含的语义知识,以及对编辑方向的更微妙的处理,以避免被卡在局部最低或不需要的编辑中。为此,我们设计了一种新颖的对比损失,该损失利用了预定义的夹子空间方向,从不同的角度将编辑指向所需的方向。此外,我们设计了一个简单而有效的方案,该方案将(目标文本)明确映射到潜在空间,并将其与潜在代码融合在一起,以进行有效的潜在代码优化和准确的编辑。广泛的实验表明,我们的设计在乘坐各种反事实概念的目标文本驾驶时,可以实现准确,现实的编辑。
translated by 谷歌翻译
半监督学习(SSL)从根本上是一个缺失的标签问题,与广泛的随机假设完全既贴心又无标记的标签完全失踪,而不是随机(mnar)问题(mnar)问题更现实和挑战数据共享相同的类分布。与现有的SSL解决方案不同,这些解决方案忽略了“类”在引起非随机性中的作用,例如,用户更有可能将流行类标记为“类别”,我们将“类”明确地纳入SSL。我们的方法是三倍:1)我们建议使用偏置标记的数据来利用未标记的数据来利用未标记的数据来训练改进的分类器。 2)鼓励罕见的课堂培训,其模型是低回调但高精度,丢弃了太多的伪标记的数据,我们提出了类动态降低(或增加)伪标签分配阈值的class感知插补(CAI)稀有(或频繁)的课程。 3)总体而言,我们将CAP和CAI集成到训练无偏的SSL模型的双重稳健估计器中。在各种MNAR设置和消融中,我们的方法不仅显着优于现有基线,而且超过了其他标签偏置删除SSL方法。请通过以下方式查看我们的代码:https://github.com/joyhuyy1412/cadr-fixmatch。
translated by 谷歌翻译
通过不懈的研究增强了StyleGAN的语义可控性。尽管现有的弱监督方法在沿一个属性操纵样式代码方面很好地奏效,但操纵多个属性的准确性被忽略了。多属性表示很容易在stylegan潜在空间中纠缠,而顺序编辑会导致错误积累。为了解决这些局限性,我们设计了一个动态样式操纵网络(Dystyle),其结构和参数因输入样本而异,以执行非线性和自适应操纵潜在代码,以进行灵活和精确的属性控制。为了有效且稳定地优化障碍网络,我们提出了动态的多属性对比度学习(DMACL)方法:包括动态的多重构造对比度和动态多属性对比损失,同时将各种属性从生成中删除模型的图像和潜在空间。结果,我们的方法表明了沿多个数字和二进制属性的细粒度分离的编辑。与现有样式操纵方法的定性和定量比较验证了我们方法在多属性控制的准确性和身份保存方面的优越性,而不会损害光真相。
translated by 谷歌翻译
This paper presents a practical global optimization algorithm for the K-center clustering problem, which aims to select K samples as the cluster centers to minimize the maximum within-cluster distance. This algorithm is based on a reduced-space branch and bound scheme and guarantees convergence to the global optimum in a finite number of steps by only branching on the regions of centers. To improve efficiency, we have designed a two-stage decomposable lower bound, the solution of which can be derived in a closed form. In addition, we also propose several acceleration techniques to narrow down the region of centers, including bounds tightening, sample reduction, and parallelization. Extensive studies on synthetic and real-world datasets have demonstrated that our algorithm can solve the K-center problems to global optimal within 4 hours for ten million samples in the serial mode and one billion samples in the parallel mode. Moreover, compared with the state-of-the-art heuristic methods, the global optimum obtained by our algorithm can averagely reduce the objective function by 25.8% on all the synthetic and real-world datasets.
translated by 谷歌翻译
Consensus clustering aggregates partitions in order to find a better fit by reconciling clustering results from different sources/executions. In practice, there exist noise and outliers in clustering task, which, however, may significantly degrade the performance. To address this issue, we propose a novel algorithm -- robust consensus clustering that can find common ground truth among experts' opinions, which tends to be minimally affected by the bias caused by the outliers. In particular, we formalize the robust consensus clustering problem as a constraint optimization problem, and then derive an effective algorithm upon alternating direction method of multipliers (ADMM) with rigorous convergence guarantee. Our method outperforms the baselines on benchmarks. We apply the proposed method to the real-world advertising campaign segmentation and forecasting tasks using the proposed consensus clustering results based on the similarity computed via Kolmogorov-Smirnov Statistics. The accurate clustering result is helpful for building the advertiser profiles so as to perform the forecasting.
translated by 谷歌翻译
Previous work has shown the potential of deep learning to predict renal obstruction using kidney ultrasound images. However, these image-based classifiers have been trained with the goal of single-visit inference in mind. We compare methods from video action recognition (i.e. convolutional pooling, LSTM, TSM) to adapt single-visit convolutional models to handle multiple visit inference. We demonstrate that incorporating images from a patient's past hospital visits provides only a small benefit for the prediction of obstructive hydronephrosis. Therefore, inclusion of prior ultrasounds is beneficial, but prediction based on the latest ultrasound is sufficient for patient risk stratification.
translated by 谷歌翻译
Current mainstream object detection methods for large aerial images usually divide large images into patches and then exhaustively detect the objects of interest on all patches, no matter whether there exist objects or not. This paradigm, although effective, is inefficient because the detectors have to go through all patches, severely hindering the inference speed. This paper presents an Objectness Activation Network (OAN) to help detectors focus on fewer patches but achieve more efficient inference and more accurate results, enabling a simple and effective solution to object detection in large images. In brief, OAN is a light fully-convolutional network for judging whether each patch contains objects or not, which can be easily integrated into many object detectors and jointly trained with them end-to-end. We extensively evaluate our OAN with five advanced detectors. Using OAN, all five detectors acquire more than 30.0% speed-up on three large-scale aerial image datasets, meanwhile with consistent accuracy improvements. On extremely large Gaofen-2 images (29200$\times$27620 pixels), our OAN improves the detection speed by 70.5%. Moreover, we extend our OAN to driving-scene object detection and 4K video object detection, boosting the detection speed by 112.1% and 75.0%, respectively, without sacrificing the accuracy. Code is available at https://github.com/Ranchosky/OAN.
translated by 谷歌翻译
Due to the issue that existing wireless sensor network (WSN)-based anomaly detection methods only consider and analyze temporal features, in this paper, a self-supervised learning-based anomaly node detection method based on an autoencoder is designed. This method integrates temporal WSN data flow feature extraction, spatial position feature extraction and intermodal WSN correlation feature extraction into the design of the autoencoder to make full use of the spatial and temporal information of the WSN for anomaly detection. First, a fully connected network is used to extract the temporal features of nodes by considering a single mode from a local spatial perspective. Second, a graph neural network (GNN) is used to introduce the WSN topology from a global spatial perspective for anomaly detection and extract the spatial and temporal features of the data flows of nodes and their neighbors by considering a single mode. Then, the adaptive fusion method involving weighted summation is used to extract the relevant features between different models. In addition, this paper introduces a gated recurrent unit (GRU) to solve the long-term dependence problem of the time dimension. Eventually, the reconstructed output of the decoder and the hidden layer representation of the autoencoder are fed into a fully connected network to calculate the anomaly probability of the current system. Since the spatial feature extraction operation is advanced, the designed method can be applied to the task of large-scale network anomaly detection by adding a clustering operation. Experiments show that the designed method outperforms the baselines, and the F1 score reaches 90.6%, which is 5.2% higher than those of the existing anomaly detection methods based on unsupervised reconstruction and prediction. Code and model are available at https://github.com/GuetYe/anomaly_detection/GLSL
translated by 谷歌翻译
The core operation of current Graph Neural Networks (GNNs) is the aggregation enabled by the graph Laplacian or message passing, which filters the neighborhood information of nodes. Though effective for various tasks, in this paper, we show that they are potentially a problematic factor underlying all GNN models for learning on certain datasets, as they force the node representations similar, making the nodes gradually lose their identity and become indistinguishable. Hence, we augment the aggregation operations with their dual, i.e. diversification operators that make the node more distinct and preserve the identity. Such augmentation replaces the aggregation with a two-channel filtering process that, in theory, is beneficial for enriching the node representations. In practice, the proposed two-channel filters can be easily patched on existing GNN methods with diverse training strategies, including spectral and spatial (message passing) methods. In the experiments, we observe desired characteristics of the models and significant performance boost upon the baselines on 9 node classification tasks.
translated by 谷歌翻译