Authors:
(1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu);
(2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu);
(3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx);
(4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu);
(5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction 2 Related Work 3 Background on HDC 4 Problem Definition 5 LifeDH 6 Variants of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 6 VARIANTS OF LIFEHD While LifeHD is designed to cater to general IoT applications with streaming input and without supervision, real-world scenarios may vary. Some scenarios might have a few labeled samples in addition to the unlabeled stream, while others may require operation within strict power constraints. LifeHD offers extensibility to address these diverse needs. In this section, we introduce two software-based extensions: LifeHDsemi, which adds a separate processing path to manage labeled samples, and LifeHDa, which adaptively prunes the HDC model using masking to handle low-power scenarios. 6.1 LifeHDsemi 6.2 LifeHDa While HDC computation is typically lightweight, there may be instances of energy scarcity (e.g., when powered by a solar panel) that call for a balance between accuracy and power efficiency. Similar to neural networks, one approach is to prune the HDC model using a mask, retaining the most crucial HDC dimensions postencoding [25]. Dimension importance can be determined by aggregating all class hypervectors into one and sorting the values across all dimensions. Notably, direct reduction of the encoding dimension should be avoided, as it can degrade HDC’s expressive capability and end up with corruption. Fig. 7 (left) visually demonstrates the impact of masking in supervised HDC tasks: retaining the top 6000 bits incurs only a 3% accuracy loss compared to using the full 10K-bit precision. While the concept of masking has been employed in prior HDC studies [25], they are not directly applicable to LifeHD due to their offline training setting with iid data. With streaming non-iid data in LifeHD, the set of observed cluster HVs may only represent a subset of the potential classes, and the less significant bits could become crucial as new classes are introduced. We introduce LifeHDa, which enhances LifeHD through an adaptive masking approach applied to all cluster HVs in working and long-term memory. Let 𝐷𝑎 represent the target dimension for reduction. The rationale behind LifeHDa is depicted in Fig. 7 (right). Whenever an original LifeHD detects novelty, we temporarily revert to the full dimension 𝐷 for 2 batches, which is sufficient for LifeHD to consolidate new patterns in its memory. After these two batches, we assess the long-term memory cluster HVs by aggregating them and ranking the dimensions, and derive a mask retaining 𝐷𝑎 dimensions with the largest absolute values. This mask is then applied to the following batches of 𝜙 (𝑋) immediately after encoding, up until the next novelty is detected. Novelty detection is executed with the masked hypervectors. Importantly, LifeHDa can utilize the same novelty detection sensitivity as LifeHD since the most significant dimensions dominate the similarity check. In other words, the similarity results in LifeHDa using 𝐷𝑎 dimension are similar as using the full dimension. LifeHDa offers an adaptive HDC model pruning interface with minimal accuracy loss and overhead. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Authors: Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Related Work 2 Related Work 3 Background on HDC 3 Background on HDC 4 Problem Definition 4 Problem Definition 5 LifeDH 5 LifeDH 6 Variants of LifeHD 6 Variants of LifeHD 7 Evaluation of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 10 Conclusion, Acknowledgments, and References 6 VARIANTS OF LIFEHD While LifeHD is designed to cater to general IoT applications with streaming input and without supervision, real-world scenarios may vary. Some scenarios might have a few labeled samples in addition to the unlabeled stream, while others may require operation within strict power constraints. LifeHD offers extensibility to address these diverse needs. In this section, we introduce two software-based extensions: LifeHDsemi, which adds a separate processing path to manage labeled samples, and LifeHDa, which adaptively prunes the HDC model using masking to handle low-power scenarios. 6.1 LifeHDsemi 6.2 LifeHDa While HDC computation is typically lightweight, there may be instances of energy scarcity (e.g., when powered by a solar panel) that call for a balance between accuracy and power efficiency. Similar to neural networks, one approach is to prune the HDC model using a mask, retaining the most crucial HDC dimensions postencoding [25]. Dimension importance can be determined by aggregating all class hypervectors into one and sorting the values across all dimensions. Notably, direct reduction of the encoding dimension should be avoided, as it can degrade HDC’s expressive capability and end up with corruption. Fig. 7 (left) visually demonstrates the impact of masking in supervised HDC tasks: retaining the top 6000 bits incurs only a 3% accuracy loss compared to using the full 10K-bit precision. While the concept of masking has been employed in prior HDC studies [25], they are not directly applicable to LifeHD due to their offline training setting with iid data. With streaming non-iid data in LifeHD, the set of observed cluster HVs may only represent a subset of the potential classes, and the less significant bits could become crucial as new classes are introduced. We introduce LifeHDa, which enhances LifeHD through an adaptive masking approach applied to all cluster HVs in working and long-term memory. Let 𝐷𝑎 represent the target dimension for reduction. The rationale behind LifeHDa is depicted in Fig. 7 (right). Whenever an original LifeHD detects novelty, we temporarily revert to the full dimension 𝐷 for 2 batches, which is sufficient for LifeHD to consolidate new patterns in its memory. After these two batches, we assess the long-term memory cluster HVs by aggregating them and ranking the dimensions, and derive a mask retaining 𝐷𝑎 dimensions with the largest absolute values. This mask is then applied to the following batches of 𝜙 (𝑋) immediately after encoding, up until the next novelty is detected. Novelty detection is executed with the masked hypervectors. Importantly, LifeHDa can utilize the same novelty detection sensitivity as LifeHD since the most significant dimensions dominate the similarity check. In other words, the similarity results in LifeHDa using 𝐷𝑎 dimension are similar as using the full dimension. LifeHDa offers an adaptive HDC model pruning interface with minimal accuracy loss and overhead. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing: Evaluation of LifeHD

Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing: LifeDH

Computational.TECH

Too Long; Didn't Read

Make resilience your competitive advantage

Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing: Variants of LifeHD

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Spatial-Photonic Ising Machines and the Next Era of Computing

The Noonification: How Can Enterprises Utilize Edge Computer Vision? (1/14/2023)

51 Stories To Learn About Edge Computing

8 Cloud Computing Trends to Watch in 2021

A Guide to Enhancing Security at the IoT Edge (Part 2)

Spatial-Photonic Ising Machines and the Next Era of Computing

The Noonification: How Can Enterprises Utilize Edge Computer Vision? (1/14/2023)

51 Stories To Learn About Edge Computing

8 Cloud Computing Trends to Watch in 2021

A Guide to Enhancing Security at the IoT Edge (Part 2)

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps