营销活动是一系列战略活动，可以促进企业的目标。在真正的工业场景中，营销活动的效果预测非常复杂且具有挑战性，因为通常从观察数据中学到了先验知识，而没有任何营销活动干预。此外，每个主题始终在几个营销活动的干预下同时受到干扰。因此，我们无法轻松解析和评估单个营销活动的效果。据我们所知，目前尚无有效的方法来解决此类问题，即，基于具有多个相互缠绕事件的层次结构对个体级别的预测任务进行建模。在本文中，我们对效果预测任务中涉及的基础解析树的结构进行了深入的分析，并进一步建立了一个层次结构胶囊预测网络（HAPNET）来预测营销活动的影响。基于合成数据和实际数据的广泛结果证明了我们模型比最新方法的优越性，并在实际工业应用中表现出显着的实用性。

无需进行任何架构更改的微调审计语言模型（LMS）已成为学习下游任务各种语言的规范。但是，对于非语言下游任务，一种常见的做法是使用特定于任务的设计来进行输入，输出层和损失功能。例如，可以通过用图像补丁嵌入层替换单词嵌入层，带有10向输出层的单词图表输出层以及单词预测丢失，将LM微调为MNIST分类器。 - 分别分类损失。出现一个自然的问题：LM微调可以在不更改模型架构或损失功能的情况下解决非语言的下游任务吗？为了回答这一点，我们提出了语言交织的微调（LIFT），并通过对非语言分类和回归任务的套件进行广泛的经验研究来研究其功效和局限性。 Lift不会对模型体系结构或损失功能进行任何更改，它仅依赖于自然语言界面，从而使“使用LMS进行无代码机”学习。我们发现，在各种低维分类和回归任务中，LIFT的性能相对较好，在许多情况下匹配了最佳基线的性能，尤其是对于分类任务。我们报告了有关升力的基本特性的实验结果，包括其电感偏差，样品效率，推断出外推能力，对异常值的鲁棒性和标签噪声以及概括。我们还分析了一些特定于提升的属性/技术，例如，通过适当提示，预测不确定性量化和两阶段微调，上下文感知学习。我们的代码可从https://github.com/uw-madison-lee-lab/languageinterfacefacefacefinetuning获得。

交通预测在智能运输系统中起着不可或缺的作用，使每日旅行更方便和更安全。然而，时空相关的动态演化使得准确的流量预测非常困难。现有工作主要采用图形神经NetWroks（GNNS）和深度时间序列模型（例如，复发性神经网络），以捕获动态交通系统中的复杂时空模式。对于空间模式，GNN难以在道路网络中提取全局空间信息，即远程传感器信息。虽然我们可以使用自我关注来提取全球空间信息，如前面的工作中，它也伴随着巨大的资源消耗。对于时间模式，交通数据不仅易于识别每日和每周趋势，而且难以识别由事故引起的短期噪音（例如，汽车事故和雷暴）。现有交通模型难以在时间序列中区分复杂的时间模式，因此难以实现准确的时间依赖。为了解决上述问题，我们提出了一种新颖的噪声感知高效时空变压器架构，用于准确的交通预测，名为StFormer。 Stformer由两个组件组成，这是噪声感知的时间自我关注（NATSA）和基于图形的稀疏空间自我关注（GBS3A）。 NATSA将高频分量和低频分量与时间序列分开以消除噪声并分别通过学习滤波器和时间自我关注捕获稳定的时间依赖性。 GBS3A以基于图形的稀疏查询替换vanilla自我关注的完整查询，以减少时间和内存使用情况。四个现实世界交通数据集的实验表明，履带器优于较低的计算成本的最先进的基线。

交通预测在智能交通系统中很重要，有利于交通安全，但由于现实世界交通系统中的复杂和动态的时空依赖性，这是非常具有挑战性的。先前的方法使用预定义或学习的静态图来提取空间相关性。但是，基于静态图形的方法无法挖掘交通网络的演变。研究人员随后为每次切片生成动态图形以反映空间相关性的变化，但它们遵循独立建模的时空依赖性的范例，忽略了串行空间影响。在本文中，我们提出了一种新的基于跨时动态图形的深度学习模型，名为CDGNet，用于交通预测。该模型能够通过利用横行动态图来有效地捕获每个时切片和其历史时片之间的串联空间依赖性。同时，我们设计了稀疏横行动态图的浇注机制，符合现实世界中的稀疏空间相关性。此外，我们提出了一种新颖的编码器解码器架构，用于结合基于交叉时间动态图形的GCN，用于多步行量预测。三个现实世界公共交通数据集的实验结果表明CDGNET优于最先进的基线。我们还提供了一种定性研究来分析我们建筑的有效性。

Recently, lots of algorithms have been proposed for learning a fair classifier from decentralized data. However, many theoretical and algorithmic questions remain open. First, is federated learning necessary, i.e., can we simply train locally fair classifiers and aggregate them? In this work, we first propose a new theoretical framework, with which we demonstrate that federated learning can strictly boost model fairness compared with such non-federated algorithms. We then theoretically and empirically show that the performance tradeoff of FedAvg-based fair learning algorithms is strictly worse than that of a fair classifier trained on centralized data. To bridge this gap, we propose FedFB, a private fair learning algorithm on decentralized data. The key idea is to modify the FedAvg protocol so that it can effectively mimic the centralized fair learning. Our experimental results show that FedFB significantly outperforms existing approaches, sometimes matching the performance of the centrally trained model.

Increasing research interests focus on sequential recommender systems, aiming to model dynamic sequence representation precisely. However, the most commonly used loss function in state-of-the-art sequential recommendation models has essential limitations. To name a few, Bayesian Personalized Ranking (BPR) loss suffers the vanishing gradient problem from numerous negative sampling and predictionbiases; Binary Cross-Entropy (BCE) loss subjects to negative sampling numbers, thereby it is likely to ignore valuable negative examples and reduce the training efficiency; Cross-Entropy (CE) loss only focuses on the last timestamp of the training sequence, which causes low utilization of sequence information and results in inferior user sequence representation. To avoid these limitations, in this paper, we propose to calculate Cumulative Cross-Entropy (CCE) loss over the sequence. CCE is simple and direct, which enjoys the virtues of painless deployment, no negative sampling, and effective and efficient training. We conduct extensive experiments on five benchmark datasets to demonstrate the effectiveness and efficiency of CCE. The results show that employing CCE loss on three state-of-the-art models GRU4Rec, SASRec, and S3-Rec can reach 125.63%, 69.90%, and 33.24% average improvement of full ranking NDCG@5, respectively. Using CCE, the performance curve of the models on the test data increases rapidly with the wall clock time, and is superior to that of other loss functions in almost the whole process of model training.

The utilization of large-scale distributed renewable energy promotes the development of the multi-microgrid (MMG), which raises the need of developing an effective energy management method to minimize economic costs and keep self energy-sufficiency. The multi-agent deep reinforcement learning (MADRL) has been widely used for the energy management problem because of its real-time scheduling ability. However, its training requires massive energy operation data of microgrids (MGs), while gathering these data from different MGs would threaten their privacy and data security. Therefore, this paper tackles this practical yet challenging issue by proposing a federated multi-agent deep reinforcement learning (F-MADRL) algorithm via the physics-informed reward. In this algorithm, the federated learning (FL) mechanism is introduced to train the F-MADRL algorithm thus ensures the privacy and the security of data. In addition, a decentralized MMG model is built, and the energy of each participated MG is managed by an agent, which aims to minimize economic costs and keep self energy-sufficiency according to the physics-informed reward. At first, MGs individually execute the self-training based on local energy operation data to train their local agent models. Then, these local models are periodically uploaded to a server and their parameters are aggregated to build a global agent, which will be broadcasted to MGs and replace their local agents. In this way, the experience of each MG agent can be shared and the energy operation data is not explicitly transmitted, thus protecting the privacy and ensuring data security. Finally, experiments are conducted on Oak Ridge national laboratory distributed energy control communication lab microgrid (ORNL-MG) test system, and the comparisons are carried out to verify the effectiveness of introducing the FL mechanism and the outperformance of our proposed F-MADRL.

This paper presents a safety-critical locomotion control framework for quadrupedal robots. Our goal is to enable quadrupedal robots to safely navigate in cluttered environments. To tackle this, we introduce exponential Discrete Control Barrier Functions (exponential DCBFs) with duality-based obstacle avoidance constraints into a Nonlinear Model Predictive Control (NMPC) with Whole-Body Control (WBC) framework for quadrupedal locomotion control. This enables us to use polytopes to describe the shapes of the robot and obstacles for collision avoidance while doing locomotion control of quadrupedal robots. Compared to most prior work, especially using CBFs, that utilize spherical and conservative approximation for obstacle avoidance, this work demonstrates a quadrupedal robot autonomously and safely navigating through very tight spaces in the real world. (Our open-source code is available at github.com/HybridRobotics/quadruped_nmpc_dcbf_duality, and the video is available at youtu.be/p1gSQjwXm1Q.)

Future work sentences (FWS) are the particular sentences in academic papers that contain the author's description of their proposed follow-up research direction. This paper presents methods to automatically extract FWS from academic papers and classify them according to the different future directions embodied in the paper's content. FWS recognition methods will enable subsequent researchers to locate future work sentences more accurately and quickly and reduce the time and cost of acquiring the corpus. The current work on automatic identification of future work sentences is relatively small, and the existing research cannot accurately identify FWS from academic papers, and thus cannot conduct data mining on a large scale. Furthermore, there are many aspects to the content of future work, and the subdivision of the content is conducive to the analysis of specific development directions. In this paper, Nature Language Processing (NLP) is used as a case study, and FWS are extracted from academic papers and classified into different types. We manually build an annotated corpus with six different types of FWS. Then, automatic recognition and classification of FWS are implemented using machine learning models, and the performance of these models is compared based on the evaluation metrics. The results show that the Bernoulli Bayesian model has the best performance in the automatic recognition task, with the Macro F1 reaching 90.73%, and the SCIBERT model has the best performance in the automatic classification task, with the weighted average F1 reaching 72.63%. Finally, we extract keywords from FWS and gain a deep understanding of the key content described in FWS, and we also demonstrate that content determination in FWS will be reflected in the subsequent research work by measuring the similarity between future work sentences and the abstracts.

Three-dimensional (3D) ultrasound imaging technique has been applied for scoliosis assessment, but current assessment method only uses coronal projection image and cannot illustrate the 3D deformity and vertebra rotation. The vertebra detection is essential to reveal 3D spine information, but the detection task is challenging due to complex data and limited annotations. We propose VertMatch, a two-step framework to detect vertebral structures in 3D ultrasound volume by utilizing unlabeled data in semi-supervised manner. The first step is to detect the possible positions of structures on transverse slice globally, and then the local patches are cropped based on detected positions. The second step is to distinguish whether the patches contain real vertebral structures and screen the predicted positions from the first step. VertMatch develops three novel components for semi-supervised learning: for position detection in the first step, (1) anatomical prior is used to screen pseudo labels generated from confidence threshold method; (2) multi-slice consistency is used to utilize more unlabeled data by inputting multiple adjacent slices; (3) for patch identification in the second step, the categories are rebalanced in each batch to solve imbalance problem. Experimental results demonstrate that VertMatch can detect vertebra accurately in ultrasound volume and outperforms state-of-the-art methods. VertMatch is also validated in clinical application on forty ultrasound scans, and it can be a promising approach for 3D assessment of scoliosis.