Authors:
(1) Ruohan Zhang, Department of Computer Science, Stanford University, Institute for Human-Centered AI (HAI), Stanford University & Equally contributed; zharu@stanford.edu;
(2) Sharon Lee, Department of Computer Science, Stanford University & Equally contributed; zharu@stanford.edu;
(3) Minjune Hwang, Department of Computer Science, Stanford University & Equally contributed; zharu@stanford.edu;
(4) Ayano Hiranaka, Department of Mechanical Engineering, Stanford University & Equally contributed; zharu@stanford.edu;
(5) Chen Wang, Department of Computer Science, Stanford University;
(6) Wensi Ai, Department of Computer Science, Stanford University;
(7) Jin Jie Ryan Tan, Department of Computer Science, Stanford University;
(8) Shreya Gupta, Department of Computer Science, Stanford University;
(9) Yilun Hao, Department of Computer Science, Stanford University;
(10) Ruohan Gao, Department of Computer Science, Stanford University;
(11) Anthony Norcia, Department of Psychology, Stanford University
(12) Li Fei-Fei, 1Department of Computer Science, Stanford University & Institute for Human-Centered AI (HAI), Stanford University;
(13) Jiajun Wu, Department of Computer Science, Stanford University & Institute for Human-Centered AI (HAI), Stanford University. Authors: Authors: (1) Ruohan Zhang, Department of Computer Science, Stanford University, Institute for Human-Centered AI (HAI), Stanford University & Equally contributed; zharu@stanford.edu; (2) Sharon Lee, Department of Computer Science, Stanford University & Equally contributed; zharu@stanford.edu; (3) Minjune Hwang, Department of Computer Science, Stanford University & Equally contributed; zharu@stanford.edu; (4) Ayano Hiranaka, Department of Mechanical Engineering, Stanford University & Equally contributed; zharu@stanford.edu; (5) Chen Wang, Department of Computer Science, Stanford University; (6) Wensi Ai, Department of Computer Science, Stanford University; (7) Jin Jie Ryan Tan, Department of Computer Science, Stanford University; (8) Shreya Gupta, Department of Computer Science, Stanford University; (9) Yilun Hao, Department of Computer Science, Stanford University; (10) Ruohan Gao, Department of Computer Science, Stanford University; (11) Anthony Norcia, Department of Psychology, Stanford University (12) Li Fei-Fei, 1Department of Computer Science, Stanford University & Institute for Human-Centered AI (HAI), Stanford University; (13) Jiajun Wu, Department of Computer Science, Stanford University & Institute for Human-Centered AI (HAI), Stanford University. Table of Links Abstract & Introduction Abstract & Introduction Brain-Robot Interface (BRI): Background Brain-Robot Interface (BRI): Background The NOIR System The NOIR System Experiments Experiments Results Results Conclusion, Limitations, and Ethical Concerns Conclusion, Limitations, and Ethical Concerns Acknowledgments & References Acknowledgments & References Appendix 1: Questions and Answers about NOIR Appendix 1: Questions and Answers about NOIR Appendix 2: Comparison between Different Brain Recording Devices Appendix 2: Comparison between Different Brain Recording Devices Appendix 3: System Setup Appendix 3: System Setup Appendix 4: Task Definitions Appendix 4: Task Definitions Appendix 5: Experimental Procedure Appendix 5: Experimental Procedure Appendix 6: Decoding Algorithms Details Appendix 6: Decoding Algorithms Details Appendix 7: Robot Learning Algorithm Details Appendix 7: Robot Learning Algorithm Details Appendix 1: Questions and Answers about NOIR Q (Safety) : Is EEG safe to use? Are there any potential risks or side effects of using the EEG for extended periods of time? Q (Safety) A : EEG devices are generally safe with no known side effects and risks, especially when compared to invasive devices like implants. We use saline solution to lower electrical impedance and improve conductance. The solution could cause minor skin irritation when the net is used for extended periods of time, hence we mix the solution with baby shampoo to mitigate this. A Q (Safety) : How does the system ensure user safety, particularly in the context of real-world tasks with varying environments and unpredictable events? Q (Safety) A : We implement an EEG-controlled safety mechanism to confirm or interrupt robot actions with muscle tension, as decoded through clenching. Nevertheless, it is important to note that the current implementation entails a 500ms delay when interrupting robot actions which might lead to a potential risk in more dynamic tasks. With more training data using a shorter decoding window, the issue can be potentially mitigated. A Q (Universality) : Can EEG / NOIR be applied to different people? Given that the paper has only been tested on three human subjects, how can the authors justify the generalizability of the findings? Q (Universality) A : The EEG device employed in our research is versatile, catering to both adults and children as young as five years old. Accompanied by SensorNets of varying sizes, the device ensures compatibility with different head dimensions. Our decoding methods have been thoughtfully designed with diversity and inclusion in mind, drawing upon two prominent EEG signals: steady-state visually evoked potential and motor imagery. These signals have exhibited efficacy across a wide range of individuals. However, it is important to acknowledge that the interface of our system, NOIR, is exclusively visual in nature, rendering it unsuitable for individuals with severe visual impairments. A Q (Portability) : Can EEG be used outside the lab? Q (Portability) A : While mobile EEG devices offer portability, it is worth noting that they often exhibit a comparatively much lower signal-to-noise ratio. Various sources contribute to the noise present in EEG signals, including muscle movements, eye movements, power lines, and interference from other devices. These sources of noise exist in and outside of the lab; consequently, though we’ve chosen to implement robust decoding techniques based on classical statistics, more robust further filtering techniques to mitigate these unwanted artifacts and extract meaningful information accurately are needed for greater success in more chaotic environments. A Q (Privacy) : How does the system differentiate between intentional brain signals for task execution and other unrelated brain activity? How will you address potential issues of privacy and security? Q (Privacy) A : The decoding algorithms employed in our study were purposefully engineered to exclusively capture task-relevant signals, ensuring the exclusion of any extraneous information. Adhering to the principles of data privacy and in compliance with the guidelines set by the Institutional Review Board (IRB) for human research, the data collected from participants during calibration and experimental sessions were promptly deleted following the conclusion of each experiment. Only the decoded signals, stripped of any identifying information, were retained for further analysis. A Q (Scalability) : How scalable is the robotics system? Can it be easily adapted to different robot platforms or expanded to accommodate a broader range of tasks beyond the 20 household activities tested? Q (Scalability) A : Within the context of our study, two notable constraints are the speed of decoding and the availability of primitive skills. The former restricts the range of tasks to those that do not involve time-sensitive and dynamic interactions. However, the advancement in decoding accuracy and the reduction of the decoding window duration may eventually address this limitation. These improvements can potentially be achieved through the utilization of larger training datasets and the implementation of machine-learning-based decoding models, leveraging the high temporal resolution offered by EEG. A The development of a comprehensive library of primitive skills stands as a long-term objective in the field of robotics research. This entails creating a repertoire of fundamental abilities that can be adapted and combined to address new tasks. Additionally, our findings indicate that human users possess the ability to innovate and devise novel applications of existing skills to accomplish tasks, akin to the way humans employ tools. Q (Potential impact) : How exactly do both individuals with and without disabilities benefit from this BRI system? Q (Potential impact) A : The potential applications of systems like NOIR in the future are vast and diverse. One significant area where these systems can have a profound impact is in assisting individuals with disabilities, particularly those with mobility-related impairments. By enabling these individuals to accomplish Activities of Daily Living and Instrumental Activities of Daily Living [70] tasks, such systems can greatly enhance their independence and overall quality of life. Currently, individuals without disabilities may initially find the BRI pipeline to have a learning curve, resulting in inefficiencies compared to their own performance in daily activities in their first few attempts. However, robot learning methods hold the promise of addressing these inefficiencies over time, and enable robots to help their users when needed. A This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

COVIDFakeExplainer: An Explainable Machine Learning based Web Application: Conclusion & References

NOIR: Neural Signal Operated Intelligent Robots: Acknowledgments & References

EScholar Technologies

Too Long; Didn't Read

Boost your HackerNoon story @ $159.99! 🚀

NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities: Appendix 1

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

5 Ways to Measure Privacy Risks in Text Data

Human-Centric Evaluation of NOIR in Real-world Tasks

Neural Signal Operated Intelligent Robots: Conclusion, Limitations, & Ethical Concerns

Decoding Human Intentions: NOIR's Performance and Learning Algorithms

NOIR: Neural Signal Operated Intelligent Robots: Acknowledgments & References

NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities: Appendix 2

5 Ways to Measure Privacy Risks in Text Data

Human-Centric Evaluation of NOIR in Real-world Tasks

Neural Signal Operated Intelligent Robots: Conclusion, Limitations, & Ethical Concerns

Decoding Human Intentions: NOIR's Performance and Learning Algorithms

NOIR: Neural Signal Operated Intelligent Robots: Acknowledgments & References

NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities: Appendix 2

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps