Yeong Hwan Oh 오영환 IRIS Lab, Sungkyunkwan Univ.
Index / Works / 04
Exhibit 04 of 04 Published
AISP 2024 HDC Normalization

효율적 이미지 분류 작업을 위한 Hyperdimensional Computing 정규화 기법

Normalization Technique of Hyperdimensional Computing for Efficient Image Classification

Keeping late training data from fading out of a hyperdimensional model by renormalizing class hypervectors after every epoch.

Fig. 04 Schematic
HDC normalization: class hypervector norm during online retraining Schematic after Figure 2 of the paper. Without normalization the L2 norm of class hypervectors grows steadily with epochs, so later updates move the model less. With per-epoch normalization the norm is reset to one after every epoch. FIG. 04 / CLASS-HV NORM FIG. 04 SUBJECT HDC-NORM SCALE N.T.S. 0 20 40 60 80 100 0 2 4 6 8 10 EPOCH ‖C‖₂ (MEAN OVER CLASSES) WITHOUT NORMALIZATION PER-EPOCH NORMALIZATION → ‖C‖₂ = 1 +2.04% MNIST +1.54% CIFAR-10 AT 100-D HVS +1.08% TIME TRAINING OVERHEAD SCHEMATIC AFTER FIG. 2 (CIFAR-10, D = 10,000, LR = 1E-3)
Fig. 04 Without normalization, class hypervector norms keep growing during online learning, so each new update moves the model less and less. Resetting every class HV to unit norm after each epoch lets data from every epoch count equally.

Specifications

As reported in the paper
Accuracy, 100-D HVs +2.04% MNIST, +1.54% CIFAR-10
Accuracy, 10,000-D HVs +0.67% MNIST, +0.58% CIFAR-10
Training-time overhead +1.08%
Datasets MNIST, CIFAR-10

Abstract

AISP 2024

Existing HDC models suffer from a diminishing impact of training data on the model as learning progresses during retraining. We propose a normalization technique for class hypervectors that accumulate learning history within the model, mitigating premature convergence when features are not adequately learned. Implemented with TorchHD on an NVIDIA TITAN RTX GPU, the method improves accuracy by +2.04% on MNIST and +1.54% on CIFAR-10 with 100-dimensional HVs at a modest 1.08% increase in execution time.

Cite

@inproceedings{oh2024hdcnorm,
  title = {효율적 이미지 분류 작업을 위한 Hyperdimensional Computing 정규화 기법},
  author = {Oh, Yeong Hwan and Ko, Jong Hwan},
  booktitle = {인공지능신호처리학술대회 (AISP)},
  year = {2024},
  month = oct
}