ExchangeDEX+

Buy Crypto Markets Spot Futures500X Earn Events

This article presents a novel Decision Boundary-Aware Distillation methodology for Instance-Incremental Learning that requires no access to old data.This article presents a novel Decision Boundary-Aware Distillation methodology for Instance-Incremental Learning that requires no access to old data.

Medical Image Synthesis: S-CycleGAN for RUSS and Segmentation

By: Hackernoon

2025/11/05 23:30

AWARE$0.001835+8.57%

Table of Links

Abstract and 1 Introduction

Related works
Problem setting
Methodology

4.1. Decision boundary-aware distillation

4.2. Knowledge consolidation
Experimental results and 5.1. Experiment Setup

5.2. Comparison with SOTA methods

5.3. Ablation study
Conclusion and future work and References

\

Supplementary Material

Details of the theoretical analysis on KCEMA mechanism in IIL
Algorithm overview
Dataset details
Implementation details
Visualization of dusted input images
More experimental results

4. Methodology

As shown in Fig. 2 (a), the occurrence of concept drift in new observations leads to the emergence of outer samples that the existing model fails on. The new IIL has to broaden the decision boundary to these outer samples as well as avoiding the catastrophic forgetting (CF) on the old boundary. Conventional knowledge distillation-based methods rely on some preserved exemplars [22] or auxiliary data [33, 34] to resist CF. However, in the proposed IIL setting, we have no access to any old data other than new observations. Distillation based on these new observations conflicts with learning new knowledge if no new parameters are added to the model. To strike a balance between learning and not forgetting, we propose a decision boundary-aware distillation method that requires no old data. During learning, the new knowledge learned by the student is intermittently consolidated back to the teacher model, which brings better generalization and is a pioneer attempt in this area.

\ Figure 3. Comparison between (a) previous distillation-based method which inferences with student model (S) and (b) the proposed decision boundary-aware distillation (DBD) with knowledge consolidation (KC). We use teacher model (T) for inference.

:::info Authors:

(1) Qiang Nie, Hong Kong University of Science and Technology (Guangzhou);

(2) Weifu Fu, Tencent Youtu Lab;

(3) Yuhuan Lin, Tencent Youtu Lab;

(4) Jialin Li, Tencent Youtu Lab;

(5) Yifeng Zhou, Tencent Youtu Lab;

(6) Yong Liu, Tencent Youtu Lab;

(7) Qiang Nie, Hong Kong University of Science and Technology (Guangzhou);

(8) Chengjie Wang, Tencent Youtu Lab.

:::

:::info This paper is available on arxiv under CC BY-NC-ND 4.0 Deed (Attribution-Noncommercial-Noderivs 4.0 International) license.

:::

Disclaimer: The articles reposted on this site are sourced from public platforms and are provided for informational purposes only. They do not necessarily reflect the views of MEXC. All rights remain with the original authors. If you believe any content infringes on third-party rights, please contact [email protected] for removal. MEXC makes no guarantees regarding the accuracy, completeness, or timeliness of the content and is not responsible for any actions taken based on the information provided. The content does not constitute financial, legal, or other professional advice, nor should it be considered a recommendation or endorsement by MEXC.

Share Insights