Offline reinforcement learning (RL) enables the agent to effectively learn from logged data, which significantly extends the applicability of RL algorithms in real-world scenarios where exploration can be expensive or unsafe. Previous works have shown that extracting primitive skills from the recurring and temporally extended structures in the logged data yields better learning. However, these methods suffer greatly when the primitives have limited representation ability to recover the original policy space, especially in offline settings. In this paper, we give a quantitative characterization of the performance of offline hierarchical learning and highlight the importance of learning lossless primitives. To this end, we propose to use a \emph{flow}-based structure as the representation for low-level policies. This allows us to represent the behaviors in the dataset faithfully while keeping the expression ability to recover the whole policy space. We show that such lossless primitives can drastically improve the performance of hierarchical policies. The experimental results and extensive ablation studies on the standard D4RL benchmark show that our method has a good representation ability for policies and achieves superior performance in most tasks.

在本报告中，我们建议针对四个EGO4D挑战任务，包括自然语言查询（NLQ），MOMMER QUERY（MQ），对象状态变更分类（OSCC），以及PNR定位（PNR）。尤其是，我们将最近发布的EGO4D数据集\ cite {grauman2021ego4d}从预处理数据集，预处理目标和开发集中从egecentric vlp中提升。基于上述三个设计，我们开发了一个验证的视频语言模型，该模型能够将其以自我为中心的视频文本表示或仅视频表示形式转移到几个视频下游任务中。我们的Egentric VLP在NLQ上实现10.46r@1&iou @0.3，MQ上的10.33地图，OSCC上的74％ACC，PNR上的0.67秒错误。该代码可在https://github.com/showlab/egovlp上找到。

在本报告中，我们为Epic-kitchens-100多实体检索（miR）挑战提出了一个基于视频的预处理（VLP）解决方案\ cite {kevin202222222egovlp}。尤其是，我们将最近发布的EGO4D数据集\ cite {grauman2021ego4d}从预处理数据集，预处理目标和开发集中从egecentric vlp中提升。基于上述三个设计，我们开发了一个预验证的视频语言模型，该模型能够将其自我为中心的视频文本表示为mir基准。此外，我们设计了一种自适应多构度最大损失，以有效地微调模型并为可靠的推理配备双重效果技术。我们最好的单个模型在挑战测试集上获得了强劲的性能，其中47.39％的地图和61.44％的NDCG。该代码可在https://github.com/showlab/egovlp上找到。

Hashing has been widely researched to solve the large-scale approximate nearest neighbor search problem owing to its time and storage superiority. In recent years, a number of online hashing methods have emerged, which can update the hash functions to adapt to the new stream data and realize dynamic retrieval. However, existing online hashing methods are required to update the whole database with the latest hash functions when a query arrives, which leads to low retrieval efficiency with the continuous increase of the stream data. On the other hand, these methods ignore the supervision relationship among the examples, especially in the multi-label case. In this paper, we propose a novel Fast Online Hashing (FOH) method which only updates the binary codes of a small part of the database. To be specific, we first build a query pool in which the nearest neighbors of each central point are recorded. When a new query arrives, only the binary codes of the corresponding potential neighbors are updated. In addition, we create a similarity matrix which takes the multi-label supervision information into account and bring in the multi-label projection loss to further preserve the similarity among the multi-label data. The experimental results on two common benchmarks show that the proposed FOH can achieve dramatic superiority on query time up to 6.28 seconds less than state-of-the-art baselines with competitive retrieval accuracy.

Path planning in the multi-robot system refers to calculating a set of actions for each robot, which will move each robot to its goal without conflicting with other robots. Lately, the research topic has received significant attention for its extensive applications, such as airport ground, drone swarms, and automatic warehouses. Despite these available research results, most of the existing investigations are concerned with the cases of robots with a fixed movement speed without considering uncertainty. Therefore, in this work, we study the problem of path-planning in the multi-robot automatic warehouse context, which considers the time-varying and uncertain robots' movement speed. Specifically, the path-planning module searches a path with as few conflicts as possible for a single agent by calculating traffic cost based on customarily distributed conflict probability and combining it with the classic A* algorithm. However, this probability-based method cannot eliminate all conflicts, and speed's uncertainty will constantly cause new conflicts. As a supplement, we propose the other two modules. The conflict detection and re-planning module chooses objects requiring re-planning paths from the agents involved in different types of conflicts periodically by our designed rules. Also, at each step, the scheduling module fills up the agent's preserved queue and decides who has a higher priority when the same element is assigned to two agents simultaneously. Finally, we compare the proposed algorithm with other algorithms from academia and industry, and the results show that the proposed method is validated as the best performance.

The electrification of shared mobility has become popular across the globe. Many cities have their new shared e-mobility systems deployed, with continuously expanding coverage from central areas to the city edges. A key challenge in the operation of these systems is fleet rebalancing, i.e., how EVs should be repositioned to better satisfy future demand. This is particularly challenging in the context of expanding systems, because i) the range of the EVs is limited while charging time is typically long, which constrain the viable rebalancing operations; and ii) the EV stations in the system are dynamically changing, i.e., the legitimate targets for rebalancing operations can vary over time. We tackle these challenges by first investigating rich sets of data collected from a real-world shared e-mobility system for one year, analyzing the operation model, usage patterns and expansion dynamics of this new mobility mode. With the learned knowledge we design a high-fidelity simulator, which is able to abstract key operation details of EV sharing at fine granularity. Then we model the rebalancing task for shared e-mobility systems under continuous expansion as a Multi-Agent Reinforcement Learning (MARL) problem, which directly takes the range and charging properties of the EVs into account. We further propose a novel policy optimization approach with action cascading, which is able to cope with the expansion dynamics and solve the formulated MARL. We evaluate the proposed approach extensively, and experimental results show that our approach outperforms the state-of-the-art, offering significant performance gain in both satisfied demand and net revenue.

社区检测是网络科学的基本和重要问题，但只有几个基于图形神经网络的社区检测算法，其中无监督的算法几乎是空白的。通过融合具有网络功能的高阶模块化信息，本文首次提出了基于变分AualiCoder重建的社区检测VGGAer，并给出了其非概率版本。他们不需要任何先前的信息。我们精心设计了基于社区检测任务的相应输入功能，解码器和下游任务，这些设计简洁，自然，表现良好（我们的设计下的NMI值得到59.1％-565.9％）。基于一系列具有广泛数据集和先​​进方法的一系列实验，VGAER实现了卓越的性能，并具有更简单的设计竞争力和潜力。最后，我们报告了算法收敛性分析和T-SNE可视化的结果，清楚地描绘了VGAER的稳定性能和强大的网络模块化能力。我们的代码可在https://github.com/qcydm/vgaer提供。

可以通过去噪得分匹配有效地估计数据密度的第一阶导数，并且已成为许多应用中的重要组成部分，例如图像生成和音频合成。高阶导数提供有关数据分发的其他本地信息并启用新应用程序。尽管可以通过自动分化估计学习密度模型，但这可以放大估计误差，并且在高维设置中昂贵。为了克服这些限制，我们提出了一种方法来直接从样本中直接估计数据密度的高阶导数（得分）。首先表明可以将去噪得分匹配作为Tweedie公式的特定情况解释。通过利用Tweedie在高阶时刻的公式，我们概括了去噪得分与估计高阶衍生物的匹配。我们经验证明，用所提出的方法训练的模型可以比通过自动分化更有效和准确地近似二阶衍生物。我们表明，我们的模型可用于量化去噪的不确定性，并通过Ozaki离散化来提高Langevin动力学的混合速度，以便采样合成数据和自然图像。

在离线强化学习（离线RL）中，主要挑战之一是处理学习策略与给定数据集之间的分布转变。为了解决这个问题，最近的离线RL方法试图引入保守主义偏见，以鼓励在高信心地区学习。无模型方法使用保守的正常化或特殊网络结构直接对策略或价值函数学习进行这样的偏见，但它们约束的策略搜索限制了脱机数据集之外的泛化。基于模型的方法使用保守量量化学习前瞻性动态模型，然后生成虚构的轨迹以扩展脱机数据集。然而，由于离线数据集中的有限样本，保守率量化通常在支撑区域内遭受全面化。不可靠的保守措施将误导基于模型的想象力，以不受欢迎的地区，导致过多的行为。为了鼓励更多的保守主义，我们提出了一种基于模型的离线RL框架，称为反向离线模型的想象（ROMI）。我们与新颖的反向策略结合使用逆向动力学模型，该模型可以生成导致脱机数据集中的目标目标状态的卷展栏。这些反向的想象力提供了无通知的数据增强，以便无模型策略学习，并使远程数据集的保守概括。 ROMI可以有效地与现成的无模型算法组合，以实现基于模型的概括，具有适当的保守主义。经验结果表明，我们的方法可以在离线RL基准任务中产生更保守的行为并实现最先进的性能。

Compressed videos often exhibit visually annoying artifacts, known as Perceivable Encoding Artifacts (PEAs), which dramatically degrade video visual quality. Subjective and objective measures capable of identifying and quantifying various types of PEAs are critical in improving visual quality. In this paper, we investigate the influence of four spatial PEAs (i.e. blurring, blocking, bleeding, and ringing) and two temporal PEAs (i.e. flickering and floating) on video quality. For spatial artifacts, we propose a visual saliency model with a low computational cost and higher consistency with human visual perception. In terms of temporal artifacts, self-attention based TimeSFormer is improved to detect temporal artifacts. Based on the six types of PEAs, a quality metric called Saliency-Aware Spatio-Temporal Artifacts Measurement (SSTAM) is proposed. Experimental results demonstrate that the proposed method outperforms state-of-the-art metrics. We believe that SSTAM will be beneficial for optimizing video coding techniques.