As seen on HYPEBEAST, CNBC, Highsnobiety, COMPLEX, NiceKicks, and more - HYP is the social auction house for exclusive releases

Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing: Conclusion, and References

Authors:
(1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu);
(2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu);
(3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx);
(4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu);
(5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction 2 Related Work 3 Background on HDC 4 Problem Definition 5 LifeDH 6 Variants of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 9 DISCUSSIONS AND FUTURE WORKS Problem Scale. One limitation of LifeHD is the relative small problem scale (e.g., the image size of CIFAR-100 is restricted to 32x32) due to the essential difficulty of unsupervised lifelong learning problem, including single-pass non-iid data and no supervision. For the same reason, there remains a disparity in accuracy between unsupervised lifelong learning and fully supervised NNs, as substantiated by prior research [13, 54]. In order to scale LifeHD to more challenging applications such as self-driving vehicles, one possible direction is to leverage the pretrained foundation model as a frozen feature extractor in the HDnn framework, which we leave for future investigation. Hyperparameter Tuning. While we recognize that hyperparameters can influence the performance of LifeHD, such an issue is not exclusive to LifeHD, but has persistently been a challenge in machine learning research [7]. In LifeHD, the impact of hyperparameters can be mitigated through pre-deployment evaluation and component co-design. For example, encoding parameters such as 𝑄, 𝑃 can be tuned on similar health monitoring data sources prior to deployment. Meanwhile, the component of cluster HVs merging can increase LifeHD’s resiliency to the novelty detection threshold 𝛾, as a higher quantity of novel clusters can be merged in later stage of learning. Limitations of HDC. HDC serves as the fundamental core of LifeHD. While HDC shows promise with its notable lightweight design, it is burdened by several limitations that remain active areas of research. First, for complex datasets like audio and images, HDC requires a pretrained feature extractor (the HDnn encoding) which may not exist for certain applications. Moreover, akin to any other architecture, HDC vectors face capacity limitations determined by the dimension of HD space, encoding method, and potential noise levels in the input data [56]. Due to these factors, careful evaluation and sometimes manual feature engineering are required to successfully deploy HDC for new applications. Future Works. Although LifeHD focuses on single-device lifelong learning for classification tasks, the method can be extended for other types of tasks and learning settings, such as federated learning and reinforcement learning. We leave the investigation of these topics for future work. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Authors: Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Related Work 2 Related Work 3 Background on HDC 3 Background on HDC 4 Problem Definition 4 Problem Definition 5 LifeDH 5 LifeDH 6 Variants of LifeHD 6 Variants of LifeHD 7 Evaluation of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 10 Conclusion, Acknowledgments, and References 9 DISCUSSIONS AND FUTURE WORKS Problem Scale. One limitation of LifeHD is the relative small problem scale (e.g., the image size of CIFAR-100 is restricted to 32x32) due to the essential difficulty of unsupervised lifelong learning problem, including single-pass non-iid data and no supervision. For the same reason, there remains a disparity in accuracy between unsupervised lifelong learning and fully supervised NNs, as substantiated by prior research [13, 54]. In order to scale LifeHD to more challenging applications such as self-driving vehicles, one possible direction is to leverage the pretrained foundation model as a frozen feature extractor in the HDnn framework, which we leave for future investigation. Problem Scale. Hyperparameter Tuning. While we recognize that hyperparameters can influence the performance of LifeHD, such an issue is not exclusive to LifeHD, but has persistently been a challenge in machine learning research [7]. In LifeHD, the impact of hyperparameters can be mitigated through pre-deployment evaluation and component co-design. For example, encoding parameters such as 𝑄, 𝑃 can be tuned on similar health monitoring data sources prior to deployment. Meanwhile, the component of cluster HVs merging can increase LifeHD’s resiliency to the novelty detection threshold 𝛾, as a higher quantity of novel clusters can be merged in later stage of learning. Hyperparameter Tuning. Limitations of HDC. HDC serves as the fundamental core of LifeHD. While HDC shows promise with its notable lightweight design, it is burdened by several limitations that remain active areas of research. First, for complex datasets like audio and images, HDC requires a pretrained feature extractor (the HDnn encoding) which may not exist for certain applications. Moreover, akin to any other architecture, HDC vectors face capacity limitations determined by the dimension of HD space, encoding method, and potential noise levels in the input data [56]. Due to these factors, careful evaluation and sometimes manual feature engineering are required to successfully deploy HDC for new applications. Limitations of HDC. Future Works. Although LifeHD focuses on single-device lifelong learning for classification tasks, the method can be extended for other types of tasks and learning settings, such as federated learning and reinforcement learning. We leave the investigation of these topics for future work. Future Works. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv

Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing: Discussions and Future Works

Authors:
(1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu);
(2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu);
(3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx);
(4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu);
(5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction 2 Related Work 3 Background on HDC 4 Problem Definition 5 LifeDH 6 Variants of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 8 EVALUATION OF LIFEHDsemi AND LIFEHDa In this section, we compare LifeHDsemi and LifeHDa, our proposed extensions from LifeHD, with existing designs that are similar. Performance of LifeHDsemi. To evaluate LifeHDsemi in a lowlabel scenario, we compare it with SemiHD [22], which is the stateof-the-art HDC method for semi-supervised learning. We adapt SemiHD [22] for single-pass settings, introducing a pseudolabel assignment threshold. When the cosine similarity of an unlabeled sample to the nearest class hypervector surpasses the threshold, we assign that class as its pseudolabel. The sample is then employed to update the class hypervector in SemiHD. We explore various threshold values and choose the optimal result for comparison. Fig. 15 (a) compares LifeHDsemi and SemiHD [22] on ESC-50 and CIFAR100 across various labeling ratios 𝑟 < 0.01. The advantages of LifeHDsemi are most prominent when labels are limited, the weakly supervised scenario is LifeHDsemi’s major focus. LifeHDsemi improves ACC by up to 10.25% and 3.6% on ESC-50 and CIFAR-100 respectively. This outcome arises from the unsupervised nature of LifeHD, allowing it to autonomously organize prominent cluster HVs, especially when all samples from a class lack labels. As the labeling ratio increases, LifeHDsemi’s advantage over SemiHD diminishes, because more labels bolster SemiHD’s performance. Performance of LifeHDa. LifeHDa provides an interface to trade minimal performance loss for efficiency gains, by adaptively pruning out the insignificant dimensions. We compare LifeHDa with previous HDC works employing a fixed mask throughout training [25], and the results are presented in Fig. 15 (b) for CIFAR100, including ACC and training latency per batch on RPi 4B. Fixed masks negatively impact HDC learning, especially with smaller dimensions. Such masks fail to adapt to new hypervectors in classincremental streams, where less significant dimensions may become crucial later in training. LifeHDa addresses this issue by adjusting the mask upon novelty detection, leading to a degradation of only 0.71% in ACC and 4.5x efficiency gain compared to the complete LifeHD, using only 20% of the full HD dimension of LifeHD. The overhead of adaptively adjusting the mask is negligible when novelty detection occurs infrequently. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Authors: Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Related Work 2 Related Work 3 Background on HDC 3 Background on HDC 4 Problem Definition 4 Problem Definition 5 LifeDH 5 LifeDH 6 Variants of LifeHD 6 Variants of LifeHD 7 Evaluation of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 10 Conclusion, Acknowledgments, and References 8 EVALUATION OF LIFEHDsemi AND LIFEHDa In this section, we compare LifeHDsemi and LifeHDa, our proposed extensions from LifeHD, with existing designs that are similar. Performance of LifeHDsemi. To evaluate LifeHDsemi in a lowlabel scenario, we compare it with SemiHD [22], which is the stateof-the-art HDC method for semi-supervised learning. We adapt SemiHD [22] for single-pass settings, introducing a pseudolabel assignment threshold. When the cosine similarity of an unlabeled sample to the nearest class hypervector surpasses the threshold, we assign that class as its pseudolabel. The sample is then employed to update the class hypervector in SemiHD. We explore various threshold values and choose the optimal result for comparison. Fig. 15 (a) compares LifeHDsemi and SemiHD [22] on ESC-50 and CIFAR100 across various labeling ratios 𝑟 < 0.01. The advantages of LifeHDsemi are most prominent when labels are limited, the weakly supervised scenario is LifeHDsemi’s major focus. LifeHDsemi improves ACC by up to 10.25% and 3.6% on ESC-50 and CIFAR-100 respectively. This outcome arises from the unsupervised nature of LifeHD, allowing it to autonomously organize prominent cluster HVs, especially when all samples from a class lack labels. As the labeling ratio increases, LifeHDsemi’s advantage over SemiHD diminishes, because more labels bolster SemiHD’s performance. Performance of LifeHDsemi. Performance of LifeHDa. LifeHDa provides an interface to trade minimal performance loss for efficiency gains, by adaptively pruning out the insignificant dimensions. We compare LifeHDa with previous HDC works employing a fixed mask throughout training [25], and the results are presented in Fig. 15 (b) for CIFAR100, including ACC and training latency per batch on RPi 4B. Fixed masks negatively impact HDC learning, especially with smaller dimensions. Such masks fail to adapt to new hypervectors in classincremental streams, where less significant dimensions may become crucial later in training. LifeHDa addresses this issue by adjusting the mask upon novelty detection, leading to a degradation of only 0.71% in ACC and 4.5x efficiency gain compared to the complete LifeHD, using only 20% of the full HD dimension of LifeHD. The overhead of adaptively adjusting the mask is negligible when novelty detection occurs infrequently. Performance of LifeHDa. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv

Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing: Evaluation of LifeHDsemi & a

Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing: Evaluation of LifeHD

Authors:
(1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu);
(2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu);
(3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx);
(4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu);
(5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction 2 Related Work 3 Background on HDC 4 Problem Definition 5 LifeDH 6 Variants of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 6 VARIANTS OF LIFEHD While LifeHD is designed to cater to general IoT applications with streaming input and without supervision, real-world scenarios may vary. Some scenarios might have a few labeled samples in addition to the unlabeled stream, while others may require operation within strict power constraints. LifeHD offers extensibility to address these diverse needs. In this section, we introduce two software-based extensions: LifeHDsemi, which adds a separate processing path to manage labeled samples, and LifeHDa, which adaptively prunes the HDC model using masking to handle low-power scenarios. 6.1 LifeHDsemi 6.2 LifeHDa While HDC computation is typically lightweight, there may be instances of energy scarcity (e.g., when powered by a solar panel) that call for a balance between accuracy and power efficiency. Similar to neural networks, one approach is to prune the HDC model using a mask, retaining the most crucial HDC dimensions postencoding [25]. Dimension importance can be determined by aggregating all class hypervectors into one and sorting the values across all dimensions. Notably, direct reduction of the encoding dimension should be avoided, as it can degrade HDC’s expressive capability and end up with corruption. Fig. 7 (left) visually demonstrates the impact of masking in supervised HDC tasks: retaining the top 6000 bits incurs only a 3% accuracy loss compared to using the full 10K-bit precision. While the concept of masking has been employed in prior HDC studies [25], they are not directly applicable to LifeHD due to their offline training setting with iid data. With streaming non-iid data in LifeHD, the set of observed cluster HVs may only represent a subset of the potential classes, and the less significant bits could become crucial as new classes are introduced. We introduce LifeHDa, which enhances LifeHD through an adaptive masking approach applied to all cluster HVs in working and long-term memory. Let 𝐷𝑎 represent the target dimension for reduction. The rationale behind LifeHDa is depicted in Fig. 7 (right). Whenever an original LifeHD detects novelty, we temporarily revert to the full dimension 𝐷 for 2 batches, which is sufficient for LifeHD to consolidate new patterns in its memory. After these two batches, we assess the long-term memory cluster HVs by aggregating them and ranking the dimensions, and derive a mask retaining 𝐷𝑎 dimensions with the largest absolute values. This mask is then applied to the following batches of 𝜙 (𝑋) immediately after encoding, up until the next novelty is detected. Novelty detection is executed with the masked hypervectors. Importantly, LifeHDa can utilize the same novelty detection sensitivity as LifeHD since the most significant dimensions dominate the similarity check. In other words, the similarity results in LifeHDa using 𝐷𝑎 dimension are similar as using the full dimension. LifeHDa offers an adaptive HDC model pruning interface with minimal accuracy loss and overhead. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Authors: Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Related Work 2 Related Work 3 Background on HDC 3 Background on HDC 4 Problem Definition 4 Problem Definition 5 LifeDH 5 LifeDH 6 Variants of LifeHD 6 Variants of LifeHD 7 Evaluation of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 10 Conclusion, Acknowledgments, and References 6 VARIANTS OF LIFEHD While LifeHD is designed to cater to general IoT applications with streaming input and without supervision, real-world scenarios may vary. Some scenarios might have a few labeled samples in addition to the unlabeled stream, while others may require operation within strict power constraints. LifeHD offers extensibility to address these diverse needs. In this section, we introduce two software-based extensions: LifeHDsemi, which adds a separate processing path to manage labeled samples, and LifeHDa, which adaptively prunes the HDC model using masking to handle low-power scenarios. 6.1 LifeHDsemi 6.2 LifeHDa While HDC computation is typically lightweight, there may be instances of energy scarcity (e.g., when powered by a solar panel) that call for a balance between accuracy and power efficiency. Similar to neural networks, one approach is to prune the HDC model using a mask, retaining the most crucial HDC dimensions postencoding [25]. Dimension importance can be determined by aggregating all class hypervectors into one and sorting the values across all dimensions. Notably, direct reduction of the encoding dimension should be avoided, as it can degrade HDC’s expressive capability and end up with corruption. Fig. 7 (left) visually demonstrates the impact of masking in supervised HDC tasks: retaining the top 6000 bits incurs only a 3% accuracy loss compared to using the full 10K-bit precision. While the concept of masking has been employed in prior HDC studies [25], they are not directly applicable to LifeHD due to their offline training setting with iid data. With streaming non-iid data in LifeHD, the set of observed cluster HVs may only represent a subset of the potential classes, and the less significant bits could become crucial as new classes are introduced. We introduce LifeHDa, which enhances LifeHD through an adaptive masking approach applied to all cluster HVs in working and long-term memory. Let 𝐷𝑎 represent the target dimension for reduction. The rationale behind LifeHDa is depicted in Fig. 7 (right). Whenever an original LifeHD detects novelty, we temporarily revert to the full dimension 𝐷 for 2 batches, which is sufficient for LifeHD to consolidate new patterns in its memory. After these two batches, we assess the long-term memory cluster HVs by aggregating them and ranking the dimensions, and derive a mask retaining 𝐷𝑎 dimensions with the largest absolute values. This mask is then applied to the following batches of 𝜙 (𝑋) immediately after encoding, up until the next novelty is detected. Novelty detection is executed with the masked hypervectors. Importantly, LifeHDa can utilize the same novelty detection sensitivity as LifeHD since the most significant dimensions dominate the similarity check. In other words, the similarity results in LifeHDa using 𝐷𝑎 dimension are similar as using the full dimension. LifeHDa offers an adaptive HDC model pruning interface with minimal accuracy loss and overhead. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv

Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing: Variants of LifeHD

Authors:
(1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu);
(2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu);
(3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx);
(4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu);
(5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction 2 Related Work 3 Background on HDC 4 Problem Definition 5 LifeDH 6 Variants of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 5 LIFEHD In this section, we present the design of LifeHD, the first unsupervised HDC framework for lifelong learning in general edge IoT applications. Compared to operating in the original data space, HDC improves pattern separability through sparsity and high dimensionality, making it more resilient against catastrophic forgetting [52]. LifeHD preserves the advantages of HDC in computational efficiency and lifelong learning, while handling the input of unlabeled streaming data, which has not been achieved in previous work [18, 21, 22, 41, 65]. 5.1 LifeHD Overview Fig. 4 gives an overview of how LifeHD works. The first step is HDC encoding of data into hypervectors as described in Sec. 3. Training samples 𝑋 are organized into batches of size 𝑏𝑆𝑖𝑧𝑒 and input into an optional fixed NN for feature extraction (e.g. for images and sound) and the encoding module. The encoded hypervectors 𝜙 (𝑋) are input to LifeHD’s two-tier memory design inspired by cognitive science studies [5], consisting of working memory and long-term memory. This memory system intelligently and dynamically manages historical patterns, stored as hypervectors and referred to as cluster HVs. As shown in Fig. 4, the working memory is designed with three components: novelty detection, cluster HV update and cluster HV merge. 𝜙 (𝑋) is first input into novelty detection step (1). An insertion to the cluster HVs is made if a novelty flag is raised, otherwise 𝜙 (𝑋) updates the existing cluster HVs (2). The third component, cluster HV merge (3), retrieves the cluster HVs from long-term memory, and merges similar cluster HVs into a supercluster via a novel spectral clustering-based merging algorithm [59]. The interaction between working and long-term memory happens as commonly encountered cluster HVs are copied to long-term memory, which we call consolidation (4). Finally, when the size limit of either working or long-term memory is reached, the least recently used cluster HVs are forgotten (5). ecently used cluster HVs are forgotten (○5 ). All modules in LifeHD work collaboratively, making it adaptive and robust to continuously changing streams without relying on any form of prior knowledge. For example, in scenarios of distribution drift, LifeHD may generate new cluster HVs upon encountering drifted samples initially, which can later be merged into coarse clusters. This approach ensures that LifeHD can efficiently capture and retain historical patterns. In the following, we discuss more details about the major components of LifeHD: novelty detection (Sec. 5.2), cluster HV update (Sec. 5.3), and cluster HV merging (Sec. 5.4). We summarize the important notations used in this paper in Table 1. 5.2 Novelty Detection The initial novelty detection step (1 in Fig. 4) is crucial for identifying emerging patterns in the environment. SupposeM = {𝑚1, ...,𝑚𝑀} is the set of cluster HVs stored in the working memory. We gauge the "radius" of each cluster by tracking two scalars for each cluster HV 𝑖: 𝜇𝑖 and 𝜎ˆ𝑖 , which represent the mean cosine difference and standard difference between the cluster HV and its assigned inputs. Given 𝜙 (𝑋), we first identify the most similar cluster HV, denoted by 𝑗. LifeHD marks 𝜙 (𝑋) as “novel" if it substantially differs from its nearest cluster HV. Specifically, this dissimilarity is measured by comparing cos(𝜙 (𝑋),𝑚𝑗) with a threshold based on the historical distance distribution of cluster HV 𝑗: The hyperparameter 𝛾 fine-tunes the sensitivity to novelties. LifeHD recognizes new 𝜙 (𝑋) as prototypes and inserts them into the working memory. When reaching its size limit 𝑀, the working memory experiences forgetting (○5 in Fig. 4). The least recently used (LRU) cluster HV, represented by 𝐿𝑅𝑈 = argmin𝑀 𝑖=1 𝑝𝑖 , is replaced. Here 𝑝 corresponds to the latest batch index where the cluster HV was accessed. A similar forgetting mechanism is configured for the long-term memory, where the last batch accessed is marked with 𝑞. 5.3 Cluster HV Update If novelty is not detected, indicating that 𝜙 (𝑋) closely matches cluster HV 𝑗, we proceed to update the cluster HV and its associated information (2 in Fig. 4). This update process involves bundling 𝜙 (𝑋) with cluster HV 𝑚𝑗 , akin to how class hypervectors are updated as described in Sec. 3, and updated 𝜇𝑗 and 𝜎ˆ𝑗 with their moving average The hyperparameter 𝛼 adjusts the balance between historical and recent inputs, where a higher 𝛼 gives more weight to recent samples. Properly maintaining 𝜇𝑗 and 𝜎ˆ𝑗 is vital for tracking the “radius” of each cluster HV, affecting future novelty detection. We also increase the hit frequency ℎ𝑖𝑡𝑗 and refresh 𝑝𝑗 with current batch index 𝑖𝑑𝑥. ℎ𝑖𝑡𝑗 is further used to compared with a predetermined threshold ℎ𝑖𝑡𝑡ℎ to decide when a working memory cluster HV appears sufficiently frequently to be consolidated to long-term memory (4 in Fig. 4). 𝑝𝑗 determines forgetting as described in the previous section. With this lightweight approach, LifeHD continually records temporal cluster HVs from the environment, while the most prominent cluster HVs are transferred to long-term memory. 5.4 Cluster HV Merging Cluster HV merge (3 in Fig. 4) has the dual benefit of reducing memory use and of elucidating underlying similarity structure in the data. Intuitively, a group of cluster HVs can be merged if they are similar to each other and dissimilar from other cluster HVs. For instance, one might merge the cluster HVs for Bulldog and Chihuahua into a single “Dog” cluster HV, that remains distinct from the cluster HV for “Tabby Cat”. To merge the cluster HVs, we first construct a similarity graph defined over the cluster HVs from the long-term memory. The cluster HVs correspond to nodes, and a pair of cluster HVs are connected by an edge if they are sufficiently similar. We then merge the cluster HVs by computing a particular type of cut in the graph in a manner similar to spectral clustering [40]. This graph based formalism for clustering is able to capture complex types of cluster geometry and often substantially outperforms simpler approaches like K-Means [59]. We detail the steps of cluster HV merging in LifeHD below, while Fig. 5 offers an illustrative overview. Step 2: Decomposition. We compute the Laplacian𝑊 = 𝐷 −𝐴, where 𝐷 is the diagonal matrix in which 𝐷𝑖𝑖 = Í 𝑗 𝐴𝑖𝑗 . We then compute the eigenvalues 𝜆1, .., 𝜆𝐿, sorted in increasing order, and eigenvectors 𝜈1, ..., 𝜈𝐿 of 𝑊. Step 3: Grouping. We infer 𝑘 = max𝑖∈ [𝐿] 𝜆𝑖 ≤ 𝑔𝑢𝑏, and merge the cluster HVs by running K-Means on 𝜈1, ..., 𝜈𝑘 . The upperbound 𝑔𝑢𝑏 is a hyperparameter that adjusts the granularity of merging, with a smaller 𝑔𝑢𝑏 leading to smaller 𝑘 thus encouraging merging more aggressively. Our merging approach is formally grounded, as discussed in [59]. It is a well-known fact that the eigenvectors of 𝑊 encode information about the connected components of G. When G has 𝑘 connected components, the eigenvalues 𝜆1 = 𝜆2 = ... = 𝜆𝑘 = 0. To recover these components, K-Means clustering on 𝜈1, ..., 𝜈𝑘 can be employed, as explained in [59]. However, practical scenarios may have a few inter-component edges that should ideally be distinct. For instance, when the similarity threshold is imprecisely set, erroneous edges may appear in the graph, causing 𝜆1, ..., 𝜆𝑘 to be only approximately zero. Our merging approach is designed to handle this situation by introducing 𝑔𝑢𝑏. The cluster HV merging is evaluated every 𝑓𝑚𝑒𝑟𝑔𝑒 batches, where 𝑓𝑚𝑒𝑟𝑔𝑒 is a hyperparameter that controls the trade-off between merging performance and computational latency. Both 𝑔𝑢𝑏 and 𝑓𝑚𝑒𝑟𝑔𝑒 are analyzed in Sec. 7.8, along with other key hyperparameters in LifeHD. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Authors: Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Related Work 2 Related Work 3 Background on HDC 3 Background on HDC 4 Problem Definition 4 Problem Definition 5 LifeDH 5 LifeDH 6 Variants of LifeHD 6 Variants of LifeHD 7 Evaluation of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 10 Conclusion, Acknowledgments, and References 5 LIFEHD In this section, we present the design of LifeHD, the first unsupervised HDC framework for lifelong learning in general edge IoT applications. Compared to operating in the original data space, HDC improves pattern separability through sparsity and high dimensionality, making it more resilient against catastrophic forgetting [52]. LifeHD preserves the advantages of HDC in computational efficiency and lifelong learning, while handling the input of unlabeled streaming data, which has not been achieved in previous work [18, 21, 22, 41, 65]. 5.1 LifeHD Overview Fig. 4 gives an overview of how LifeHD works. The first step is HDC encoding of data into hypervectors as described in Sec. 3. Training samples 𝑋 are organized into batches of size 𝑏𝑆𝑖𝑧𝑒 and input into an optional fixed NN for feature extraction (e.g. for images and sound) and the encoding module. The encoded hypervectors 𝜙 (𝑋) are input to LifeHD’s two-tier memory design inspired by cognitive science studies [5], consisting of working memory and long-term memory. This memory system intelligently and dynamically manages historical patterns, stored as hypervectors and referred to as cluster HVs. As shown in Fig. 4, the working memory is designed with three components: novelty detection, cluster HV update and cluster HV merge. 𝜙 (𝑋) is first input into novelty detection step (1). An insertion to the cluster HVs is made if a novelty flag is raised, otherwise 𝜙 (𝑋) updates the existing cluster HVs (2). The third component, cluster HV merge (3), retrieves the cluster HVs from long-term memory, and merges similar cluster HVs into a supercluster via a novel spectral clustering-based merging algorithm [59]. The interaction between working and long-term memory happens as commonly encountered cluster HVs are copied to long-term memory, which we call consolidation (4). Finally, when the size limit of either working or long-term memory is reached, the least recently used cluster HVs are forgotten (5). ecently used cluster HVs are forgotten (○5 ). All modules in LifeHD work collaboratively, making it adaptive and robust to continuously changing streams without relying on any form of prior knowledge. For example, in scenarios of distribution drift, LifeHD may generate new cluster HVs upon encountering drifted samples initially, which can later be merged into coarse clusters. This approach ensures that LifeHD can efficiently capture and retain historical patterns. In the following, we discuss more details about the major components of LifeHD: novelty detection (Sec. 5.2), cluster HV update (Sec. 5.3), and cluster HV merging (Sec. 5.4). We summarize the important notations used in this paper in Table 1. 5.2 Novelty Detection The initial novelty detection step (1 in Fig. 4) is crucial for identifying emerging patterns in the environment. SupposeM = {𝑚1, ...,𝑚𝑀} is the set of cluster HVs stored in the working memory. We gauge the "radius" of each cluster by tracking two scalars for each cluster HV 𝑖: 𝜇𝑖 and 𝜎ˆ𝑖 , which represent the mean cosine difference and standard difference between the cluster HV and its assigned inputs. Given 𝜙 (𝑋), we first identify the most similar cluster HV, denoted by 𝑗. LifeHD marks 𝜙 (𝑋) as “novel" if it substantially differs from its nearest cluster HV. Specifically, this dissimilarity is measured by comparing cos(𝜙 (𝑋),𝑚𝑗) with a threshold based on the historical distance distribution of cluster HV 𝑗: The hyperparameter 𝛾 fine-tunes the sensitivity to novelties. LifeHD recognizes new 𝜙 (𝑋) as prototypes and inserts them into the working memory. When reaching its size limit 𝑀, the working memory experiences forgetting (○5 in Fig. 4). The least recently used (LRU) cluster HV, represented by 𝐿𝑅𝑈 = argmin𝑀 𝑖=1 𝑝𝑖 , is replaced. Here 𝑝 corresponds to the latest batch index where the cluster HV was accessed. A similar forgetting mechanism is configured for the long-term memory, where the last batch accessed is marked with 𝑞. 5.3 Cluster HV Update If novelty is not detected, indicating that 𝜙 (𝑋) closely matches cluster HV 𝑗, we proceed to update the cluster HV and its associated information (2 in Fig. 4). This update process involves bundling 𝜙 (𝑋) with cluster HV 𝑚𝑗 , akin to how class hypervectors are updated as described in Sec. 3, and updated 𝜇𝑗 and 𝜎ˆ𝑗 with their moving average The hyperparameter 𝛼 adjusts the balance between historical and recent inputs, where a higher 𝛼 gives more weight to recent samples. Properly maintaining 𝜇𝑗 and 𝜎ˆ𝑗 is vital for tracking the “radius” of each cluster HV, affecting future novelty detection. We also increase the hit frequency ℎ𝑖𝑡𝑗 and refresh 𝑝𝑗 with current batch index 𝑖𝑑𝑥. ℎ𝑖𝑡𝑗 is further used to compared with a predetermined threshold ℎ𝑖𝑡𝑡ℎ to decide when a working memory cluster HV appears sufficiently frequently to be consolidated to long-term memory (4 in Fig. 4). 𝑝𝑗 determines forgetting as described in the previous section. With this lightweight approach, LifeHD continually records temporal cluster HVs from the environment, while the most prominent cluster HVs are transferred to long-term memory. 5.4 Cluster HV Merging Cluster HV merge (3 in Fig. 4) has the dual benefit of reducing memory use and of elucidating underlying similarity structure in the data. Intuitively, a group of cluster HVs can be merged if they are similar to each other and dissimilar from other cluster HVs. For instance, one might merge the cluster HVs for Bulldog and Chihuahua into a single “Dog” cluster HV, that remains distinct from the cluster HV for “Tabby Cat”. To merge the cluster HVs, we first construct a similarity graph defined over the cluster HVs from the long-term memory. The cluster HVs correspond to nodes, and a pair of cluster HVs are connected by an edge if they are sufficiently similar. We then merge the cluster HVs by computing a particular type of cut in the graph in a manner similar to spectral clustering [40]. This graph based formalism for clustering is able to capture complex types of cluster geometry and often substantially outperforms simpler approaches like K-Means [59]. We detail the steps of cluster HV merging in LifeHD below, while Fig. 5 offers an illustrative overview. Step 2: Decomposition. We compute the Laplacian𝑊 = 𝐷 −𝐴, where 𝐷 is the diagonal matrix in which 𝐷𝑖𝑖 = Í 𝑗 𝐴𝑖𝑗 . We then compute the eigenvalues 𝜆1, .., 𝜆𝐿, sorted in increasing order, and eigenvectors 𝜈1, ..., 𝜈𝐿 of 𝑊. Step 2: Decomposition. Step 3: Grouping. We infer 𝑘 = max𝑖∈ [𝐿] 𝜆𝑖 ≤ 𝑔𝑢𝑏, and merge the cluster HVs by running K-Means on 𝜈1, ..., 𝜈𝑘 . The upperbound 𝑔𝑢𝑏 is a hyperparameter that adjusts the granularity of merging, with a smaller 𝑔𝑢𝑏 leading to smaller 𝑘 thus encouraging merging more aggressively. Our merging approach is formally grounded, as discussed in [59]. It is a well-known fact that the eigenvectors of 𝑊 encode information about the connected components of G. When G has 𝑘 connected components, the eigenvalues 𝜆1 = 𝜆2 = ... = 𝜆𝑘 = 0. To recover these components, K-Means clustering on 𝜈1, ..., 𝜈𝑘 can be employed, as explained in [59]. However, practical scenarios may have a few inter-component edges that should ideally be distinct. For instance, when the similarity threshold is imprecisely set, erroneous edges may appear in the graph, causing 𝜆1, ..., 𝜆𝑘 to be only approximately zero. Our merging approach is designed to handle this situation by introducing 𝑔𝑢𝑏. The cluster HV merging is evaluated every 𝑓𝑚𝑒𝑟𝑔𝑒 batches, where 𝑓𝑚𝑒𝑟𝑔𝑒 is a hyperparameter that controls the trade-off between merging performance and computational latency. Both 𝑔𝑢𝑏 and 𝑓𝑚𝑒𝑟𝑔𝑒 are analyzed in Sec. 7.8, along with other key hyperparameters in LifeHD. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv

Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing: LifeDH

Authors:
(1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu);
(2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu);
(3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx);
(4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu);
(5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction 2 Related Work 3 Background on HDC 4 Problem Definition 5 LifeDH 6 Variants of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 4 PROBLEM DEFINITION Before diving into our method, we first rigorously formulate the unsupervised lifelong learning problem using streaming sources, driven by real-world IoT applications. Streaming Data. To represent continuously changing environment, we assume a well-known class-incremental model in lifelong learning, in which new classes emerge in a sequential manner [46]. We also allow data distribution shift within one class. This setting models a scenario in which a device is continuously sampling data while the surrounding environment may change implicitly over time, e.g., the self-driving vehicle as shown in Fig. 1. We require that all samples appear only once (i.e., single-pass streams). Learning Protocol. Our goal is to build a classification algorithm that maps X → Y. For evaluation, we use the common evaluation protocol in state-of-the-art lifelong learning works [13, 14, 54], in which we construct an iid dataset E = {(𝑋𝑘 , 𝑦𝑘 )} for periodic testing, by sampling labeled examples from each class in a manner that preserves the overall (im)balance between the classes. Note, that even when one class has not appeared in the training data stream, it is always included in E. Hence E is a global view of all classes that can potentially exist in the environment. Unsupervised Clustering Accuracy. Since we do not give class labels or the total number of classes during training, the predicted label can be different from the ground-truth label. Therefore, for evaluation metric, we cannot adopt the simple prediction accuracy that requires exact label matching. Instead, we employ a widely used clustering metric known as unsupervised clustering accuracy (ACC) [63], which mirrors the conventional accuracy evaluation but within an unsupervised context. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Authors: Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Related Work 2 Related Work 3 Background on HDC 3 Background on HDC 4 Problem Definition 4 Problem Definition 5 LifeDH 5 LifeDH 6 Variants of LifeHD 6 Variants of LifeHD 7 Evaluation of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 10 Conclusion, Acknowledgments, and References 4 PROBLEM DEFINITION Before diving into our method, we first rigorously formulate the unsupervised lifelong learning problem using streaming sources, driven by real-world IoT applications. Streaming Data. To represent continuously changing environment, we assume a well-known class-incremental model in lifelong learning, in which new classes emerge in a sequential manner [46]. We also allow data distribution shift within one class. This setting models a scenario in which a device is continuously sampling data while the surrounding environment may change implicitly over time, e.g., the self-driving vehicle as shown in Fig. 1. We require that all samples appear only once (i.e., single-pass streams). Streaming Data. Learning Protocol. Our goal is to build a classification algorithm that maps X → Y. For evaluation, we use the common evaluation protocol in state-of-the-art lifelong learning works [13, 14, 54], in which we construct an iid dataset E = {(𝑋𝑘 , 𝑦𝑘 )} for periodic testing, by sampling labeled examples from each class in a manner that preserves the overall (im)balance between the classes. Note, that even when one class has not appeared in the training data stream, it is always included in E. Hence E is a global view of all classes that can potentially exist in the environment. Learning Protocol. Unsupervised Clustering Accuracy. Since we do not give class labels or the total number of classes during training, the predicted label can be different from the ground-truth label. Therefore, for evaluation metric, we cannot adopt the simple prediction accuracy that requires exact label matching. Instead, we employ a widely used clustering metric known as unsupervised clustering accuracy (ACC) [63], which mirrors the conventional accuracy evaluation but within an unsupervised context. Unsupervised Clustering Accuracy. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv

Lifelong Intelligence Beyond the Edge using Hyperdimensional Computing: Problem Definition

Authors:
(1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu);
(2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu);
(3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx);
(4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu);
(5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction 2 Related Work 3 Background on HDC 4 Problem Definition 5 LifeDH 6 Variants of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 3 BACKGROUND ON HDC Hyperdimensional Computing (HDC) is an emerging paradigm for information processing from the cognitive-neuroscience literature [24]. In HDC, all computation is performed on low-precision and distributed representations of data that accord naturally with highly parallel and low-energy hardware. The encoding function 𝜙 : X → H embeds data from its ambient representation into HD-space. In general, encoding should preserve some meaningful notion of similarity between input points in the sense that 𝜙 (𝑥) · 𝜙 (𝑥 ′ ) ≈ 𝑘 (𝑥, 𝑥′ ), where 𝑘 is some similarity function of interest on X. In this paper, we use spatiotemporal encoding for time series sensor data, and HDnn for more complex data, such as images, which we explain in the following. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Authors: Authors: (1) Xiaofan Yu, University of California San Diego, La Jolla, California, USA (x1yu@ucsd.edu); (2) Anthony Thomas, University of California San Diego, La Jolla, California, USA (ahthomas@ucsd.edu); (3) Ivannia Gomez Moreno, CETYS University, Campus Tijuana, Tijuana, Mexico (ivannia.gomez@cetys.edu.mx); (4) Louis Gutierrez, University of California San Diego, La Jolla, California, USA (l8gutierrez@ucsd.edu); (5) Tajana Šimunić Rosing, University of California San Diego, La Jolla, USA (tajana@ucsd.edu). Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Related Work 2 Related Work 3 Background on HDC 3 Background on HDC 4 Problem Definition 4 Problem Definition 5 LifeDH 5 LifeDH 6 Variants of LifeHD 6 Variants of LifeHD 7 Evaluation of LifeHD 7 Evaluation of LifeHD 8 Evaluation of LifeHD semi and LifeHDa 8 Evaluation of LifeHD semi and LifeHDa 9 Discussions and Future Works 9 Discussions and Future Works 10 Conclusion, Acknowledgments, and References 10 Conclusion, Acknowledgments, and References 3 BACKGROUND ON HDC Hyperdimensional Computing (HDC) is an emerging paradigm for information processing from the cognitive-neuroscience literature [24]. In HDC, all computation is performed on low-precision and distributed representations of data that accord naturally with highly parallel and low-energy hardware. The encoding function 𝜙 : X → H embeds data from its ambient representation into HD-space. In general, encoding should preserve some meaningful notion of similarity between input points in the sense that 𝜙 (𝑥) · 𝜙 (𝑥 ′ ) ≈ 𝑘 (𝑥, 𝑥′ ), where 𝑘 is some similarity function of interest on X. In this paper, we use spatiotemporal encoding for time series sensor data, and HDnn for more complex data, such as images, which we explain in the following. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv

HYP

thehyp.co

-- employees

Since n.d.

COMPANY RANKING

HYP

EVERGREEN INDEX #4698

Representing this company? Click here to claim and customize this page!

HYP WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

HYP