Authors: (1) Ruohan Zhang, Department of Computer Science, Stanford University, Institute for Human-Centered AI (HAI), Stanford University & Equally contributed; zharu@stanford.edu; (2) Sharon Lee, Department of Computer Science, Stanford University & Equally contributed; zharu@stanford.edu; (3) Minjune Hwang, Department of Computer Science, Stanford University & Equally contributed; zharu@stanford.edu; (4) Ayano Hiranaka, Department of Mechanical Engineering, Stanford University & Equally contributed; zharu@stanford.edu; (5) Chen Wang, Department of Computer Science, Stanford University; (6) Wensi Ai, Department of Computer Science, Stanford University; (7) Jin Jie Ryan Tan, Department of Computer Science, Stanford University; (8) Shreya Gupta, Department of Computer Science, Stanford University; (9) Yilun Hao, Department of Computer Science, Stanford University; (10) Ruohan Gao, Department of Computer Science, Stanford University; (11) Anthony Norcia, Department of Psychology, Stanford University (12) Li Fei-Fei, 1Department of Computer Science, Stanford University & Institute for Human-Centered AI (HAI), Stanford University; (13) Jiajun Wu, Department of Computer Science, Stanford University & Institute for Human-Centered AI (HAI), Stanford University. Table of Links Abstract & Introduction Brain-Robot Interface (BRI): Background The NOIR System Experiments Results Conclusion, Limitations, and Ethical Concerns Acknowledgments & References Appendix 1: Questions and Answers about NOIR Appendix 2: Comparison between Different Brain Recording Devices Appendix 3: System Setup Appendix 4: Task Definitions Appendix 5: Experimental Procedure Appendix 6: Decoding Algorithms Details Appendix 7: Robot Learning Algorithm Details Appendix 5: Experimental Procedure . In our experiments, we use the 128-channel HydroCel Geodesic SensorNet from Magstim EGI, which has sponge tips in its electrode channels. Prior to experiments, the EEG net is soaked in a solution containing dissolved conductive salt (Potassium Chloride) and baby shampoo for 15 minutes. After the soaking, the net is worn by the experiment subject, and an impedance check is done. This impedance check entails ensuring that the impedance of each channel electrode is ≤ 50.0 kΩ, by using a syringe to add more conductive fluid between the electrodes and the scalp. We then carefully put on a shower cap to minimize the drying of conductive fluid over the course of the experiment. EEG device preparation Before commencing the experiments, subjects are given instructions on how to execute the SSVEP, MI, and muscle tension (jaw-clenching) tasks. For SSVEP, they are instructed to simply focus on the flickering object of interest without getting distracted by the other objects on the screen. For MI, similar to datasets such as BCI Competition 2003 [75], and as per extensive literature review [76], we instruct subjects to either imagine continually bending their hands at the wrist (wrist dorsiflexion) or squeezing a ball for the hand actions (“Left”, “Right”), and to imagine depressing a pedal with both feet (feet dorsiflexion) for the “Legs” action. For the “Rest” class, as is common practice in EEG experiments in general, we instruct users to focus on a fixation cross displayed on the screen. Subjects were told to stick with their actions of choice throughout the experiment, for consistency. For muscle tension, subjects were told to simply clench their jaw without too much or too little effort. Instructions to subjects. . For SSVEP, subjects are told in writing on the screen to focus on the object of interest. Thereafter, a scene image of the objects with flickering masks overlaid on each object is presented, and we immediately begin recording the EEG Data over this period of time. For MI, the cues are different during calibration and task-time. During calibration, subjects are presented with a warning symbol (.) on screen for 1 second, before being presented with the symbol representing the action they are to imagine (<-: “Left”, ->: “Right”, v: “Legs”, +: “Rest”), which lasts on screen for 5500 ms. We record the latter 5000 ms of EEG data. After which, there is a randomized period of rest the lasts between 0.5 and 2 seconds, before the process repeats for another randomly chosen action class. This is done in 4 blocks of 5 trials per action, for a total of 20 trials per action. This procedure is again similar to datasets like BCI Competition 2003 [75], that use non-linguistic cues and randomization of rest / task. At task-time, similar to SSVEP, subjects are told in writing on the screen to perform MI to select a robot skill to execute. Thereafter, a written mapping of a class symbol ({<-, ->, v, +}) to skill ({pick from top, pick from side, ...}) is presented, and we begin recording EEG Data after a 2-second delay. For muscle tension, there is also a calibration phase, similar to MI, which entails collecting three 500ms-long trials for each class (“Rest”, and “Clench”) at the start of each experiment. The cues are written on the screen in words. At task time, when appropriate, written prompts are also presented on the screen (e.g. “clench if incorrect”), followed by a written countdown, after which the user has a 500ms window to clench (or not). Interface This paper is under CC 4.0 license. available on arxiv

Too Long; Didn't Read

Developers: Build Less. Deliver More. [Learn how]	

NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities: Appendix 5

Too Long; Didn't Read

EScholar: Electronic Academic Papers for Scholars

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities: Appendix 5

Too Long; Didn't Read

EScholar: Electronic Academic Papers for Scholars

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES