Towards In-distribution Compatibility in Out-of-distribution Detection

We choose the fundamental problem of multi-class classification as the subject to illustrate our methodology. We first introduce the notations and rudiments of the studied topic as below.

Notations. For the multi-classification task with $K$ classes, a neural work will first transform input $\text{xb}$ with a non-linear mapping to feature $\text{zb}=Z\left(\text{xb}\right)\in {\text{RR}}^d$. $Z$ represents deep network backbone. Then, a linear model $f$ is supposed to output a real-value score for each class:

$\text{bW}=[{\text{bw}}_1,\cdots ,{\text{bw}}_K]$ and $\text{bb}=[b_1,\cdots ,b_K]$ are trainable parameters. $f_i\left(\text{zb}\right)={\text{bw}}_i^{\top }\text{zb}+b_i$ is the predicted score for class $i\in \left[K\right]$, which is known as the logit value. $\left[K\right]:=\{1,\cdots ,K\}$. ${\text{bw}}_i\in {\text{RR}}^d$ and $b_i\in \text{RR}$ compute a scalar score for class $i$ from feature $\text{zb}$. Finally, the stochastic gradient-descending algorithm will optimize the Softmax Cross-Entropy loss:

$y$ is the ground-truth label for $\text{xb}$, and $1_y\in R^K$ is its one-hot encoding. We define the softmax function $\text{softmax}\left(f\right):{\text{RR}}^K\to {\text{RR}}^K$ as $\text{softmax}(f{)}_i=\exp (f_i\left)/{\sum }_{k=1}^K\exp \right(f_k)$. the logarithm is defined as element-wise.

Outlier Exposure. Denoting the in-distribution set as ${\text{cD}}_{in}$, any possible input that does not belong to ${\text{cD}}_{in}$ is considered to belong to the OOD set ${\text{cD}}_{out}$. The intriguing part of detecting OOD data is that we cannot cover the entire ${\text{cD}}_{out}$ set during the training stage. Nevertheless, Hendrycks et al. (2019) showed that a good choice of a subset of ${\text{cD}}_{out}$ is crucial for learning models that can effectively detect unseen testing OOD data ${\text{cD}}_{out}^{test}$. Particularly, denoting the auxiliary set of outliers as ${\text{cD}}_{out}^{oe}$, the network is trained with:

$\text{ub}\in {\text{RR}}^K$ is the uniform distribution over $K$ classes. Weight parameter $\lambda$ balances the two training objectives. For better detection performance, Liu et al. (2020) proposed to use the energy score as an alternative loss function for outliers:

$\widetilde{\text{zb}}=Z\left(\widetilde{\text{xb}}\right)$. Minimizing the above metric results in low confidence in all the in-distribution categories. To balance in-distribution energy, the negative energy of in-distribution features $-{\text{cL}}_{\text{energy}}\left(\text{xb}\right)$ are simultaneously optimized:

In the following parts, we will discuss the differences between these two carefully-designed criteria and their common deficiencies.

The gradient generated by training criteria can reveal how it influences trainable variables. For instance, the softmax cross-entropy in Eqn. (2) generates gradients as:

With the gradient-descending optimization, ${\text{cL}}_{\text{sce}}$ will increase the logit score $f_i\left(\text{xb}\right)$ when $i$ being the ground-truth class, otherwise decrease $f_i\left(\text{xb}\right)$. Such design aligns with the belief that $f_i\left(\text{xb}\right)$ is a good hint of estimating the confidence that $\text{xb}$ belongs to category $i$. However, the OE objective in Eqn. (3) contradicts the principle that only the ground-truth logit $f_y\left(\text{xb}\right)$ is enlarged during training. Specifically, the gradient generated by ${\text{cL}}_{\text{sce}}(\widetilde{\text{xb}},\text{ub})$ in Eqn. (3) is:

When the softmax probability $\text{softmax}(f{)}_i$ exceeds $1/K$, ${\text{cL}}_{\text{sce}}(\widetilde{\text{xb}},\text{ub})$ will generate positive gradient and thus surpass the value of $f_i\left(\widetilde{\text{zb}}\right)$. This is reasonable as it reduces the confidence of predicting class $i$ on outlier $\widetilde{\text{xb}}$. The problem is, when $\text{softmax}(f{)}_i<1/K$, ${\text{cL}}_{\text{sce}}(\widetilde{\text{xb}},\text{ub})$ will generate negative gradient and increases the confidence of predicting class $i$. Although such a case only happens when the posterior is lower than $1/K$, it still violates the principle mentioned above and increases the probability of mistaking $\widetilde{\text{xb}}$ with an in-distribution class. In contrast, the energy objective ${\text{cL}}_{\text{energy}}\left(\widetilde{\text{xb}}\right)$ in (4) obeys the principle by punishing $f_i\left(\widetilde{\text{zb}}\right)$ for all classes:

Despite that the ${\text{cL}}_{\text{energy}}$ is compatible with in-distribution learning, the energy score falls into the issue of contradictory gradient due to the negative in-distribution energy $-{\text{cL}}_{\text{energy}}\left(\text{xb}\right)$ in (4):

$-{\text{cL}}_{\text{energy}}\left(\text{xb}\right)$ increase $f_i\left(\text{zb}\right)$ for all classes instead of only the ground-truth, which contradicts the gradient of $\partial {\text{cL}}_{\text{sce}}(\text{xb},1_y)/\partial f_i\left(\text{zb}\right)$ when $i\ne y$. The contradictory gradient confuses the learning process by sending signals in opposite directions, eventually leading to inferior performance.

One effective way to detect outlier $\widetilde{\text{xb}}$ is by estimating the likelihood of each class. A potential outlier is supposed to have a low likelihood value for all classes. Many outlier loss functions are designed based on this belief. Yet, many of these loss functions use the logit value $f_i\left(\widetilde{\text{zb}}\right)$ as a substitute for class likelihood $p(\text{xb}|y=i)$. These two scores are not equivalent. Occasionally, an outlier with a low likelihood value $p(\text{xb}|y=i)$ can be assigned with a high logit score.

To understand this problem, we need to first inspect the intrinsic reasons why in-distribution learning uses the linear model in Eqn. (1) to generate a confidence score $f_i$. Under the assumption that the learned features $\text{zb}$ distribute as a class-conditional Gaussian: $p\left(\text{zb}|y\right)\sim \text{cN}({\text{bmu}}_y,\text{bSigma})$, $\text{bSigma}$ is a tied covariance, and ${\text{bmu}}_i$ is the mean of class $i$, the methodology of logistic regression deduces from the Bayes’ Theorem that the posterior probability $p\left(i|\text{zb}\right)$ is equal to:

Thus, under the assumption that parameters ${\text{bw}}_i$ and $b_i$ can fit the ${\widehat{\text{bw}}}_i$ and ${\hat{b}}_i$ from training, $f_i$ is a desired in-distribution discriminator. From geometric perspective, the linear model $f_i$ measures the distance to the hyper-plane of $0={\text{bw}}_i^{\top }\text{zb}+b_i$, while the likelihood $p\left(\text{zb}|i\right)$ is proportional to the negative Mahalanobis distance to the class center ${\text{bmu}}_i$. These two measurements are not equivalent. To demonstrate the difference, we visualize the case of a 2-dimensional binary classification problem. In Fig. 2(a), we can observe that an in-distribution sample A has a lower value of $f_y$ compared with an out-of-distribution sample B, while the likelihood of A is higher than B. Such inconsistency between $f_i\left(\text{zb}\right)$ and $p\left(i|\text{zb}\right)$ is overlooked by previous works, including OE and Energy.

The high-dimension feature generated by the penultimate layer of the deep network, i.e.$\text{zb}=Z\left(\text{xb}\right)$, is trainable. Thus, unlike the static $\text{xb}$, the distribution of $\text{zb}$ will shift with respect to the imposed loss functions. When only in-distribution features exist, the softmax cross-entropy loss ${\text{cL}}_{\text{sce}}$ will push features away from the discriminant plane. However, when out-of-distribution features are considered, the shifted distribution between in-distribution features and out-of-distribution features may mix, which interferes with the discrimination of deep networks.

We still use the 2-dimension visualization for illustration. In Fig. 2, we set the training features, either in-distribution or out-of-distribution, to be trainable to simulate features generated by deep networks and plot the distribution shift on each optimization step of the gradient-descending algorithm. Parameters ${\text{bw}}_i$ and $b_i$ are initialized with ${\widehat{\text{bw}}}_i$ and ${\hat{b}}_i$. The standard OE objective ${\text{cL}}_{\text{sce}}(\widetilde{\text{xb}},\text{ub})$ will push the gradient to:

If $\text{softmax}\left[f\right(\widetilde{\text{zb}}){]}_1>1/2$, the above gradient tend to decrease the value of $f(\widetilde{\text{zb}}{)}_1$ and increases $f(\widetilde{\text{zb}}{)}_2$, and vice versa if $\text{softmax}\left[f\right(\widetilde{\text{zb}}){]}_2>1/2$. As a result, out-of-distribution features are dragged to the space between the two in-distribution classes. For OOD features that are initially far away from either of the two in-distribution classes, the OE objective may shift them to the position of in-distribution features during the optimization iterations. These misplaced features then confuse in-distribution learning, as the discriminator is required to make opposite predictions on similar features. In the next section, we will show that our method avoids this issue by encouraging in-distribution features to gather in the center of its distribution and pushing outliers away from the center, as also shown in Fig. 3.

We first replace the traditional linear layer of deep networks with the class-conditional Gaussian model. Namely, instead of using Eqn. (1) to generate a confidence score, we opt for the following estimation based on Mahalanobis distance:

${\text{bmu}}_i$ is a trainable parameter that is supposed to simulate the class center ${\text{bmu}}_i$. $\text{Lb}$ is a real lower triangular matrix with positive diagonal entries, whose elements are also trainable. Based on the Cholesky decomposition, $\text{Lb}{\text{Lb}}^{\top }$ is guaranteed to be a symmetric positive-definite real matrix and is supposed to learn the covariance matrix $\text{bSigma}$. Previous works in different manners have investigated the philosophy of designing Mahalanobis-distance-based scores. Lee et al. (2018) statistically computed the mean and variance matrix from features generated by a trained network. Such a design cannot leverage the benefit brought by training on auxiliary outliers. Pang et al. (2020) also proposed Max-Mahalanobis Center loss for better performance on adversarial robustness, although it adopted pre-defined constant values of $\text{bmu}$ and $\text{bSigma}$, which cannot be trained like our method.

For in-distribution classification, the softmax cross-entropy loss will be computed upon our newly proposed distance based score $h_i\left(\text{zb}\right)$ instead of $f_i\left(\text{zb}\right)$ in Eqn. (2):

Compared with the simplified estimation in Eqn. (10), our design provides a complete version of posterior probability estimation and prevents false likelihood. Our loss functions discussed below will utilize the estimated likelihood.

In the last part, we modify the top layer of deep networks to estimate the likelihood under the assumption that the features of the penultimate layer are approximately class-conditional Gaussian distribution. The maximum likelihood value across all classes, i.e.$\exp \left(\underset{i}{max}{h}_i\right)$, can then be a good indicator to separate in-distribution and out-of-distribution features Hendrycks et al. (2022). During training, we draw lessons from the classic Maximum Likelihood Estimation (MLE) of binary classification as the training loss:

Eqn. (14) drags in-distribution feature $\text{zb}$ to class center and Eqn. (13) pushes outliers away. Their joint effect avoids distribution shift and results in the changing trend of features in Fig. 2. Note that, in Eqn. (14), instead of using the maximum likelihood $\exp \left(\underset{i}{max}{h}_i\right)$, we opt for the likelihood for the ground-truth $\exp \left[h_y\right(\text{zb}\left)\right]$ to avoid potential contradictory gradient. Finally, a weight parameter $\lambda$ is utilized to balance in-distribution learning and out-of-distribution learning:

Besides solving the three in-distribution incompatibility issues, ICE also possesses other benefit: 1) The ICE detector $\exp \left[\underset{i}{max}{h}_i\right(\widetilde{\text{zb}}\left)\right]\in (0,1]$ provides a direct estimation of detection confidence. Unlike previous methods, the learned maximum likelihood is explicitly trained by Eqn. (14) and Eqn. (13). In empirical evaluations, we will show that this learned confidence is well-calibrated. 2) The parameters of ICE are mostly learned through training, allowing us to easily add ICE into existing frameworks. Methods such as Mahalanobis Lee et al. (2018) and ODIN Liang et al. (2018) need a second round of learning procedure after standard training. 3) Methods like Energy use multiple hyper-parameters to threshold the learned energy score. These hyper-parameters need to be exhaustively cross-validated and cannot be shared across different sets. In contrast, the only hyper-parameter of ICE is the $\lambda$ in Eqn. (18).

Dataset. For the in-distribution set, we choose the standard CIFAR10 and CIFAR100 Krizhevsky (2012) as our major verification target. For out-of-distribution set, we adopt several commonly-used benchmarks, including Textures Cimpoi et al. (2014), SVHN Netzer et al. (2011), Places365 Zhou et al. (2018), LSUN Yu et al. (2015), and iSUN Xu et al. (2015). We also use CIFAR100 as an OOD source to evaluate models learned on CIFAR10 and vice versa. We choose the 80 Million Tiny Images Torralba et al. (2008) as the ${\text{cD}}_{out}^{oe}$.

We present our main results in Table. 1. For each experimental setup, we compare our ICE with OE Hendrycks et al. (2019) and Energy Liu et al. (2020). Our ICE outperforms the other two counterparts on most benchmarks. For CIFAR10 as ${\text{cD}}_{in}$, we first present results on the model of WideResNet-34-10. Both fine-tune and from-scratch schedules achieve promising results for the three algorithms, while the from-scratch schedule slightly but consistently outperforms fine-tune schedule. This aligns with the observations by Hendrycks et al. (2019). To verify the extensibility of our method, we also present results on the DenseNet-40-12 model with the from-scratch schedule. Again, ICE exhibits generally better detection capability. Note that, for most choices of ${\text{cD}}_{out}^{test}$, all the three methods have achieved very high scores, leaving only very little room for improvement. In contrast, for the hardest case where CIFAR100 serves as ${\text{cD}}_{out}^{test}$, the improvement brought by ICE becomes more significant. For the WideResNet model learned by fine-tune schedule, ICE achieves $95.08\%$ AUPR, $94.90\%$ AUROC, and $23.23\%$ FPR95, with the improvement of $2.37\%/1.86\%/7.19\%$ over OE. For CIFAR100 as ${\text{cD}}_{in}$, we test WideResNet with the from-scratch schedule. Again, ICE achieves state-of-the-art results. In the case of SVHN being ${\text{cD}}_{out}^{test}$, ICE achieves $98.97\%$ AUPR, $94.71\%$ AUROC, and $22.41\%$ FPR95.

ICE is designed for better in-distribution compatibility. Therefore, in this part, we compare its in-distribution accuracy with other baselines, including those without training on outliers. In Table. 2, we show the average detection scores across iSUN, Places365, Textures, LSUN, and SVHN. OE and Energy with the from-scratch schedule achieve superior detection performance and greatly improve detection performance over non-training detection algorithms such as ODIN and Mahalanobis but degrade the in-distribution accuracy. OE and Energy with fine-tune schedules achieve relatively better in-distribution accuracy than non-training detection algorithms. The small learning rate of fine-tune schedule prevents the degradation of in-distribution accuracy. However, the fine-tune schedule cannot reach the same detection performance level as the from-scratch schedule.

In contrast, ICE accomplishes evident higher accuracy than other baselines, either under fine-tune or from-scratch schedules. The objective of ICE effectively utilizes the auxiliary examples from ${\text{cD}}_{out}^{oe}$ to provide neural network with new information and thus help its generalization. The refined in-distribution compatibility concentrates in-distribution features around the class center, endows in-distribution classification with high confidence, and thus further helps detect outliers with low confidence. Unlike OE and Energy, ICE achieves better in-distribution accuracy with a from-scratch schedule than fine-tune schedule, indicating its intrinsic built-in in-distribution compatibility.

To get a straightforward insight into how different outlier training methods influence the distribution, we visualize the features generated by the penultimate layer of deep networks, i.e.$\text{zb}$ and $\widetilde{\text{zb}}$, with the t-SNE algorithm. In our visualization, the t-SNE mapping is first learned on all in-distribution samples. Then, we use the learned mapping to transform outliers. As shown in Fig. 4, the Plain Training method without any learning on outliers exhibit poor OOD feature distribution. The OOD features are heavily mixed with in-distribution ones. Standard Outlier Exposure squeezes OOD features in the space center and pushes in-distribution ones into diverse directions. This aligns our visualization on a 2-dimension case in Fig. 3. Due to the effect of OE, the original clustered in-distribution features are transformed into slender ones. Energy maintains the cluster shape of in-distribution features. However, like the OE algorithm, the outlier features are close to in-distribution ones, which increases the difficulty of discriminating them. ICE, on the other hand, resumes the most characteristics of in-distribution features in Plain Training. The in-distribution clusters are tightly bounded and scatter from each other. Meanwhile, outliers are mostly constrained in a local space.

In this part, we conduct ablation studies on each component of ICE. First of all, ICE modifies the top design of WideResNet. We dub this modified model as WideResNet(G). To evaluate the effect of this design, we test Energy algorithm on WideResNet(G), which addresses the false likelihood problem but still has the issue of contradictory gradient. As shown in Table 3, Energy method with WideResNet(G) perform slightly better than the baseline with WideResNet. Then, the second counterpart we want to compare is applying the Sigmoid Binary Cross-Entropy (BCE) loss onto WideResNet. Such design avoids the contradictory gradient issue yet still has false likelihood and distribution shift problems. The improvement is still observable but limited. We also test eliminating the ${\text{cL}}_{\text{energy-id}}$ in Eqn. (18) (ICE${}^{-}$), which prevents contradictory gradient and false likelihood but not distribution shift. The detection performance is heavily degraded, indicating the importance of balancing ${\text{cL}}_{\text{energy-ood}}$ with ${\text{cL}}_{\text{energy-ood}}$ in Eqn. (18). The above ablation studies show that the individual improvement of each component is not as significant as the total improvement. The superior performance of ICE is achieved by the collective effect of all components.

As discussed before, unlike OE and Energy, our method provides the capability of estimating the $(0,1]$ probability of whether the upcoming input is in-distribution or not. Here we present a direct visualization of our in-distribution confidence indicator of $\exp \left[\underset{i}{max}{h}_i\right(\widetilde{\text{zb}}\left)\right]$. In Fig. 5, we plot the distribution of $\exp \left[\underset{i}{max}{h}_i\right(\text{zb}\left)\right]$ (CIFAR10) and $\exp \left[\underset{i}{max}{h}_i\right(\widetilde{\text{zb}}\left)\right]$ (CIFAR100) against the training epochs. As the training progresses, the two distributions gradually separate from each other and eventually concentrate on their ground-truth value. Moreover, the two distributions both formulate a typical pattern of long-tail distribution, indicating that the majority of the samples are trained to their desirable category.

We describe the detail of generating samples for the 2-dimensional binary classification task in Fig. 2&3. We randomly draw samples from $p\left(\text{xb}|c_1\right)\sim \text{cN}\left(\right(\mu ,0),\text{bI})$ and $p\left(\text{xb}|c_2\right)\sim \text{cN}\left(\right(-\mu ,0),\text{bI})$ separately. $\text{bI}$ is the identity matrix. A drawn sample is considered as an in-distribution one when $max\left(\right(p\left(\text{xb}|c_1\right),p\left(\text{xb}|c_2\right))>\zeta$, otherwise out-of-distribution. $\zeta$ is an arbitrary thresh-hold

For OE, Energy, and our ICE, there is a parameter $\lambda$ balancing the in-distribution learning and out-of-distribution learning. Here we cross-validated its value for better understanding. OE sets the default value of $\lambda$ to $0.5$, Energy sets the default value of $\lambda$ to $0.1$. ICE sets $1.0$. We test scaling the default $\lambda$ of each method with an extra parameter $\gamma$.

The table above shows that our ICE can still achieve desirable results on larger weight parameters. OE also shows a stable performance for larger $\lambda$. However, Energy cannot be trained with large $\lambda$. Once the weight parameter is large than three times of the optimal value, the training would simply result in NaN loss value and cannot achieve a reasonable performance.

In Fig. 3, we have visualized that OE methods may shift the distribution of in-distribution features for 2-dimension case. In Fig. 6, we plot the maximum logit value of neural network to see how OE methods influence the feature distribution of networks. The extra learning of OE and Energy shifts the maximum logit to larger values. This is a consequence of distribution shift, which verifies our analyses before.

In Fig. 4, we visualize features with t-SNE algorithm, where OOD examples are from the CIFAR100 set. In Fig. 7, we present the case where SVHN serves as the OOD set.