New Neural Trick Helps Models Think in Longer Patterns

by ExtrapolateApril 1st, 2025
Read on Terminal Reader
Read this story w/o Javascript
tldt arrow

Too Long; Didn't Read

Researchers introduced PANM, a plug-and-play memory module that mimics symbolic processing using pointers. PANM improves neural networks’ generalization to longer, unseen sequences and boosts performance in symbolic tasks, QA, and translation by modeling memory access more like a computer does—with physical addresses and pointer arithmetic.

People Mentioned

Mention Thumbnail
featured image - New Neural Trick Helps Models Think in Longer Patterns
Extrapolate HackerNoon profile picture
0-item

Authors:

(1) Hung Le, Applied AI Institute, Deakin University, Geelong, Australia;

(2) Dung Nguyen, Applied AI Institute, Deakin University, Geelong, Australia;

(3) Kien Do, Applied AI Institute, Deakin University, Geelong, Australia;

(4) Svetha Venkatesh, Applied AI Institute, Deakin University, Geelong, Australia;

(5) Truyen Tran, Applied AI Institute, Deakin University, Geelong, Australia.

Abstract & Introduction

Methods

Methods Part 2

Experimental Results

Experimental Results Part 2

Related Works, Discussion, & References

Appendix A, B, & C

Appendix D

2.3 Pointer-Augmented Neural Memory (PANM)


2.3.1 Pointer Unit



2.3.2 Pointer-based Addressing Modes



2.3.3 The Controller



Table 1: Algorithmic reasoning: mean sequence-level accuracy (%) over testing lengths Other Max is selected as the best numbers at each length mode from other baselines.


Table 2: SCAN (Left): Exact match accuracy (%, median of 5 runs) on splits of various lengths. Mathematics (Right): mean accuracy over 5 runs. The baselines’ numbers are from Csord´as et al. [2021] and we run PANM using the authors’ codebase.


This paper is available on arxiv under CC BY 4.0 DEED license.


Trending Topics

blockchaincryptocurrencyhackernoon-top-storyprogrammingsoftware-developmenttechnologystartuphackernoon-booksBitcoinbooks