ຊື່ຂອງ : Yichen Zhang Gan He Lei Ma Xiaofei Liu J. J. Johannes Hjorth Alexander Kozlov Yutao He Shenjian Zhang Jeanette Hellgren Kotaleski ວິທະຍາໄລ Yonghong Tian ມື້ຄ່ໍາ ວິທີການ ມົນທົນ Huang ຊື່ຂອງ : ຊື່ຫຍໍ້ຂອງ : Zhang ປະເພດ ດາວໂຫລດ ຊື່ຫຍໍ້ຂອງ : Xiaofei Liu ຊື່ຫຍໍ້ຂອງ : J. J. Johannes Hjorth ວິທະຍາໄລ Alexander Kozlov ຂໍຂອບໃຈ ຊື່ຫຍໍ້ຂອງ : Shenjian Zhang ຊື່ຫຍໍ້ຂອງ : Jeanette Hellgren Kotaleski ວິທະຍາໄລ Yonghong Tian ມື້ຄ່ໍາ ວິທີການ ມົນທົນ Huang ອັດຕະໂນມັດ ໂມເລກຸນ multi-division biophysically detailed ແມ່ນເຄື່ອງມືທີ່ເຂັ້ມແຂງເພື່ອທົດສອບສະຖາບັນການຄອມພິວເຕີຂອງວົງຈອນແລະຍັງບໍລິການເປັນສະຖາບັນທາງດ້ານວົງຈອນສໍາລັບການຜະລິດໂມເລກຸນສໍາລັບລະບົບການສົນທະນາ (AI). ຢ່າງໃດກໍຕາມ, ຄ່າໃຊ້ຈ່າຍການຄອມພິວເຕີຂະຫນາດໃຫຍ່ໄດ້ປັບປຸງຜົນກະທົບຢ່າງກວ້າງຂວາງໃນພາກພື້ນ neuroscience ແລະ AI. ຄວາມປອດໄພຕົ້ນຕໍໃນເວລາທີ່ simulating ໂມເລກຸນການຄອມພິວເຕີຂະຫນາດໃຫຍ່ແມ່ນຄວາມສາມາດຂອງ simulator ເພື່ອປິ່ນປົວລະບົບຂະຫນາດໃຫຍ່ຂອງສະຖາບັນ linear. ລະຫັດ QR ລະຫັດ QR ວິທີການ cheduling (DHS) ເພື່ອປັບປຸງຜົນປະໂຫຍດຂອງການດໍາເນີນການນີ້. ພວກເຮົາສະແດງໃຫ້ເຫັນວ່າການນໍາໃຊ້ DHS ແມ່ນຄຸນນະພາບທີ່ດີແລະຖືກຕ້ອງ. ວິທີການ GPU ນີ້ເຮັດວຽກທີ່ມີຄວາມໄວສູງສຸດ 2-3 ຊົ່ວໂມງກ່ວາວິທີ Hines ຊົ່ວໂມງໃນໂຄງປະກອບການ CPU ຄຸນນະສົມບັດ. ພວກເຮົາສ້າງ framework DeepDendrite, ເຊິ່ງລວມເອົາວິທີ DHS ແລະເຄື່ອງຄອມພິວເຕີ GPU ຂອງ simulator NEURON ແລະສະແດງໃຫ້ເຫັນຜົນປະໂຫຍດຂອງ DeepDendrite ໃນກິດຈະກໍາ neuroscience. ພວກເຮົາມີການຄົ້ນຄວ້າວິທີທີ່ຮູບແບບ spatial ຂອງ inputs spine ເຮັດໃຫ້ຜົນກະທົບຂອງ neuronal ໃນຮູບແບບ neuronal ທີ່ມີ 25,000 spines. ນອກຈາກນີ້, ພວກເຮົາສະຫນອງ D H S ລະຫັດ QR ວິທະຍາສາດ neuroscience ແມ່ນສໍາຄັນສໍາລັບການ decoding ແລະວິທີການຄອມພິວເຕີຂອງ neurons. Brains mammalian ມີຫຼາຍກ່ວາ 1000 ປະເພດທີ່ແຕກຕ່າງກັນຂອງ neurons ມີຄຸນນະສົມບັດ morphological ແລະ biophysical ທີ່ແຕກຕ່າງກັນ. ຖ້າຫາກວ່າມັນບໍ່ແມ່ນໄດ້ຢ່າງງ່າຍດາຍ, ການຢັ້ງຢືນ "point-neuron" ການຄົ້ນຄວ້າ neuron ໄດ້ຖືກນໍາໃຊ້ຢ່າງກວ້າງຂວາງໃນການຄອມພິວເຕີ neuronal, ລວມທັງການຄົ້ນຄວ້າ neuronal networks. ໃນປີທີ່ຜ່ານມາ, ການຄົ້ນຄວ້າປະສິດທິພາບອຸດົມສົມບູນ (AI) ໄດ້ນໍາໃຊ້ວິທີການນີ້ແລະພັດທະນາເຄື່ອງມືທີ່ເຂັ້ມແຂງ, ເຊັ່ນດຽວກັນກັບເຄືອຂ່າຍ neuron artificial (ANN) ໃນຂະນະທີ່ການຄອມພິວເຕີຢ່າງກວ້າງຂວາງໃນລະດັບ neuron ອື່ນໆ, ການຄອມພິວເຕີ subcellular, ເຊັ່ນ dendrites neuronal, ຍັງສາມາດເຮັດວຽກ nonlinear ເປັນ unit ການຄອມພິວເຕີອະນຸຍາດ. , , , , ນອກເຫນືອໄປຈາກນີ້, spines dendritic, ການອອກແບບຂະຫນາດນ້ອຍທີ່ກວມເອົາ dendrites ໃນ neurons spiny, ສາມາດ compartmentalize signal synaptic, ເພື່ອໃຫ້ພວກເຂົາສາມາດໄດ້ຮັບການ separated ຈາກ dendrites ພື້ນທີ່ຂອງເຂົາເຈົ້າ ex vivo ແລະ in vivo , , , . 1 2 3 4 5 6 7 8 9 10 11 Simulations using biologically detailed neurons provide a theoretical framework for linking biological details to computational principles. ຄຸນນະສົມບັດຂອງ biophysically detailed multi-compartment model framework , ພວກເຮົາມີຄວາມສາມາດໃນການມາດຕະຖານ neurons ມີ morphologies dendritic ທີ່ແທ້ຈິງ, conductivity ionic ທີ່ແທ້ຈິງ, ແລະ input synaptic extrinsic. ສັດລ້ຽງຂອງຮູບແບບ multi-compartment ອັດຕະໂນມັດ, ເຊັ່ນ dendrites, ໄດ້ຖືກສ້າງຕັ້ງຂຶ້ນໂດຍການສອບເສັງສືດິຈິຕອນທີ່ເຫມາະສົມ ໃນຖານະເປັນຜູ້ຊ່ຽວຊານຂອງການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມ , . 12 13 12 4 7 ນອກເຫນືອຈາກຜົນປະໂຫຍດທີ່ເຂັ້ມແຂງຂອງຕົນກ່ຽວກັບ neuroscience, ໂມເລກຸນ neuron ທີ່ຖືກນໍາໃຊ້ຢ່າງກວ້າງຂວາງໃນໄລຍະເວລາໄດ້ຖືກນໍາໃຊ້ເພື່ອທົດລອງຄວາມກ້ວາງລະຫວ່າງການກໍ່ສ້າງ neuronal ແລະຂໍ້ມູນ biophysical ແລະ AI. ອຸດສາຫະກໍາ AI ທີ່ຖືກນໍາໃຊ້ຢ່າງກວ້າງຂວາງແມ່ນ ANNs ທີ່ປະກອບດ້ວຍ neurons point, an analog to biological neural networks. ເຖິງແມ່ນວ່າ ANNs ມີ “backpropagation-of-error” (backprop) algorithm ໄດ້ຮັບປະສິດທິພາບທີ່ຍິ່ງໃຫຍ່ໃນອຸປະກອນພິເສດ, ເຖິງແມ່ນວ່າມັນໄດ້ຕອບສະຫນັບສະຫນູນຜູ້ຊ່ຽວຊານສູງສຸດໃນເກມ Go ແລະ chess , ການຄົ້ນຄວ້າຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດຂອງມະນຸດ , ການຄົ້ນຄວ້າທີ່ຜ່ານມາສະແດງໃຫ້ເຫັນວ່າການເຂົ້າລະຫັດ dendritic ເປັນສິ່ງທີ່ສໍາຄັນໃນການຜະລິດ algoritms ການຮຽນຮູ້ປະສິດທິພາບທີ່ສາມາດກ່ວາ backprop ໃນການປິ່ນປົວຂໍ້ມູນ parallel , , ນອກເຫນືອໄປຈາກນີ້, ຮູບແບບ multi-compartment ຫນຶ່ງສາມາດຊອກຫາຄອມພິວເຕີ nonlinear ໃນລະດັບເຄືອຂ່າຍສໍາລັບ neurons point ໂດຍການປັບປຸງພຽງແຕ່ຄວາມເຂັ້ມແຂງ synaptic , ການສະແດງໃຫ້ເຫັນຄວາມສາມາດທັງຫມົດຂອງມາດຕະຖານຂໍ້ມູນໃນການສ້າງລະບົບ AI ທີ່ມີປະສິດທິພາບເພີ່ມເຕີມ, ດັ່ງນັ້ນ, ມັນເປັນປະໂຫຍດທີ່ດີທີ່ສຸດເພື່ອຂະຫຍາຍຕົວ paradigms ໃນ AI ທີ່ມີປະສິດທິພາບເພີ່ມເຕີມຈາກມາດຕະຖານ neuron ທີ່ມີປະສິດທິພາບເພີ່ມເຕີມໄປສູ່ເຄືອຂ່າຍທີ່ມີປະສິດທິພາບສູງ. 14 15 16 17 18 19 20 21 22 ການຄາດຄະເນດິນດີຕ້ອນຮັບຂອງການຄາດຄະເນດິນດີຕ້ອນຮັບແມ່ນການຄາດຄະເນດິນດີຕ້ອນຮັບຂອງການຄາດຄະເນດິນດີຕ້ອນຮັບຂອງການຄາດຄະເນດິນດີຕ້ອນຮັບ. , , ສໍາລັບການປັບປຸງຜົນປະໂຫຍດ, ວິທີ Hines Classic ເຮັດໃຫ້ຄວາມງ່າຍດາຍສໍາລັບການແກ້ໄຂປະເພດຈາກ O(n3) ກັບ O(n), ເຊິ່ງໄດ້ຖືກນໍາໃຊ້ຢ່າງກວ້າງຂວາງເປັນປະເພດຕົ້ນຕໍໃນ simulators ທີ່ຍິ່ງໃຫຍ່ເຊັ່ນ NEURON ປະເພດ Genesis ໃນຂະນະທີ່ການທົດສອບປະກອບດ້ວຍ dendrites ຫຼາຍທີ່ມີຂະຫນາດໃຫຍ່ biophysically ມີ spines dendritic, matrix equation linear (“Hines Matrix”) scales ປະສິດທິພາບໂດຍຜ່ານການເພີ່ມຂຶ້ນຂອງ dendrites ຫຼື spines (ຮູບ. ), ເຮັດໃຫ້ວິທີ Hines ບໍ່ໄດ້ຢ່າງງ່າຍດາຍ, ໃນຂະນະທີ່ມັນສ້າງຄວາມອຸດົມສົມບູນທີ່ເຂັ້ມແຂງຫຼາຍສໍາລັບການ simulation ທັງຫມົດ. 12 23 24 25 26 ພາສາລາວ ຮູບແບບ neuron pyramidal Layer-5 reconstructed ແລະຮູບແບບ matematic ທີ່ຖືກນໍາໃຊ້ກັບຮູບແບບ neuron ວັດສະດຸ. ລະບົບການເຮັດວຽກໃນຂະນະທີ່ simulating numerically ໂມເລກຸນ neuron detailed. The equation-solving phase is the bottleneck in the simulation. ຮູບພາບ ສໍາ ລັບ linear equations ໃນ simulation. ການອະນຸຍາດ Data ຂອງ Hines Method ໃນເວລາທີ່ການແກ້ໄຂອະນຸຍາດ Linear ປະເພດ ຂະຫນາດຂອງ Hines matrix ຂະຫນາດກັບຄວາມຊ່ຽວຊານຂອງມາດຕະຖານ. ຂະຫນາດຂອງລະບົບປະເພດ linear ທີ່ຈະແກ້ໄຂໄດ້ເພີ່ມຂຶ້ນຢ່າງງ່າຍດາຍ, ໃນຂະນະທີ່ມາດຕະຖານແມ່ນຂະຫຍາຍຕົວຢ່າງງ່າຍດາຍ. ຄ່າໃຊ້ຈ່າຍການຄອມພິວເຕີ (ຫຼຸດລົງໃນລະດັບການແກ້ໄຂປະເພດ) ຂອງວິທີ Hines serial ກ່ຽວກັບປະເພດທີ່ແຕກຕ່າງກັນຂອງຮູບແບບ neuron. ຮູບພາບ ສໍາ ລັບ ວິທີການແກ້ໄຂທີ່ແຕກຕ່າງກັນ. ສ່ວນທີ່ແຕກຕ່າງກັນຂອງ neuron ໄດ້ຖືກກວດສອບກັບອົງປະກອບການປິ່ນປົວຫຼາຍໃນວິທີ parallel (middle, right), ສະແດງໃຫ້ເຫັນກັບສີທີ່ແຕກຕ່າງກັນ. ໃນວິທີ serial (ລັກສະນະ), ທັງຫມົດອົງປະກອບໄດ້ຄອມພິວເຕີກັບຫນຶ່ງອົງປະກອບ. ຄ່າໃຊ້ຈ່າຍການຄອມພິວເຕີຂອງສາມວິທີ ໃນເວລາທີ່ການແກ້ໄຂປະເພດຂອງຮູບແບບ pyramidal ມີ spines. ທີ່ໃຊ້ເວລາການດໍາເນີນການຂອງວິທີທີ່ແຕກຕ່າງກັນກ່ຽວກັບການແກ້ໄຂວິທີການສໍາລັບ 500 ຮູບແບບ pyramidal ມີ spines. The run time indicates the time consumption of 1 s simulation (solving the equation 40,000 times with a time step of 0.025 ms). p-Hines ວິທີ parallel ໃນ CoreNEURON (ໃນ GPU), ວິທີການ parallel based on branch based (ໃນ GPU), DHS ວິທີການດໍາເນີນການປະມວນຜົນ Dendritic (ໃນ GPU). a b c d c e f g h g i ໃນໄລຍະປີທີ່ຜ່ານມາ, ຄວາມປອດໄພທີ່ຍິ່ງໃຫຍ່ໄດ້ຖືກປັບປຸງເພື່ອຄວາມໄວ້ວາງໃຈຂອງ Hines Method ໂດຍການນໍາໃຊ້ວິທີ parallel ໃນລະດັບ cellular, ເຊິ່ງສາມາດ parallelize ການຄອມພິວເຕີຂອງພາກສ່ວນທີ່ແຕກຕ່າງກັນໃນແຕ່ລະ cell , , , , , ຢ່າງໃດກໍຕາມ, ວິທີການ parallel ທີ່ມີປະສິດທິພາບໃນໄລຍະອຸດສາຫະກໍາໃນໄລຍະອຸດສາຫະກໍາໃນໄລຍະອຸດສາຫະກໍາໃນໄລຍະອຸດສາຫະກໍາໃນໄລຍະອຸດສາຫະກໍາໃນໄລຍະອຸດສາຫະກໍາໃນໄລຍະອຸດສາຫະກໍາໃນໄລຍະອຸດສາຫະກໍາໃນໄລຍະອຸດສາຫະກໍາໃນໄລຍະອຸດສາຫະກໍາໃນໄລຍະອຸດສາຫະກໍາ. 27 28 29 30 31 32 ນີ້, ພວກເຮົາພັດທະນາເຄື່ອງມື simulation ອັດຕະໂນມັດ, ປະສິດທິພາບຂະຫນາດໃຫຍ່, ແລະທີ່ດີເລີດທີ່ສາມາດປັບປຸງປະສິດທິພາບການຄອມພິວເຕີແລະຫຼຸດຜ່ອນຄ່າໃຊ້ຈ່າຍການຄອມພິວເຕີ. ນອກຈາກນີ້, ເຄື່ອງມື simulation ນີ້ສາມາດຖືກນໍາໃຊ້ຢ່າງງ່າຍດາຍສໍາລັບການສ້າງຕັ້ງແລະທົດສອບເຄືອຂ່າຍ neuronal ມີລາຍລະອຽດ biological ສໍາລັບການຝຶກອົບຮົມເຄື່ອງແລະການນໍາໃຊ້ AI. ປະສິດທິພາບທີ່ສໍາຄັນ, ພວກເຮົາມີຄໍາຮ້ອງສະຫມັກການຄອມພິວເຕີ parallel ຂອງວິທີ Hines ເປັນບັນຫາການຄອມພິວເຕີແລະການຜະລິດວິທີການ Dendritic Hierarchical Scheduling (DHS) based on combinatorial optimization ວິທະຍາສາດຄອມພິວເຕີ parallel . ພວກເຮົາສະແດງໃຫ້ເຫັນວ່າ algorithm ຂອງພວກເຮົາສະຫນອງການຄາດຄະເນດິນດີຕ້ອນຮັບທີ່ດີທີ່ສຸດໂດຍບໍ່ເສຍຄ່າຄວາມຖືກຕ້ອງ. ນອກເຫນືອໄປຈາກນີ້, ພວກເຮົາໄດ້ປັບປຸງ DHS ສໍາລັບ chip GPU ທີ່ດີທີ່ສຸດໃນປັດຈຸບັນໂດຍໃຊ້ການນໍາໃຊ້ລະດັບຄວາມຮ້ອນຂອງມາດຕະຖານມາດຕະຖານ GPU ແລະ mekanisms ການເຂົ້າເຖິງມາດຕະຖານມາດຕະຖານ. ໂດຍທົ່ວໄປ, DHS ສາມາດປັບປຸງການຄາດຄະເນດິນດີຕ້ອນຮັບ 60-1,500 times (Table Supplementary) ) ເຊັ່ນດຽວກັນກັບ Simulator Classic Neuron ໃນຂະນະທີ່ພວກເຮົາມີຄວາມປອດໄພທີ່ແຕກຕ່າງກັນ. 33 34 1 25 ເພື່ອສະຫນັບສະຫນູນ simulations dendritic ວັດສະດຸສໍາລັບການນໍາໃຊ້ໃນ AI, ພວກເຮົາຫຼັງຈາກນັ້ນສ້າງໂຄງສ້າງ DeepDendrite ໂດຍການເຊື່ອມຕໍ່ກັບ DHS-embedded CoreNEURON (ເຄື່ອງຄອມພິວເຕີ optimized ສໍາລັບ NEURON) platform ໃນຖານະເປັນ motor simulation ແລະສອງ modules auxiliary (I/O module ແລະ learning module) supporting dendritic learning algorithms during simulations. DeepDendrite ດໍາເນີນການໃນອຸປະກອນ GPU, supporting both regular simulation tasks in neuroscience ແລະ learning tasks in AI. 35 ຂ້າພະເຈົ້າສືບຕໍ່ໄດ້ຮັບການປະທັບໃຈກໍໂດຍໃຊ້ການຝຶກອົບຮົມຂອງພວກເຮົາສໍາລັບການຝຶກອົບຮົມ ITS ແລະການຝຶກອົບຮົມ ITS ສໍາລັບການຝຶກອົບຮົມ ITS ແລະການຝຶກອົບຮົມ ITS ສໍາລັບການຝຶກອົບຮົມ ITS ລະຫັດສະດວກທັງຫມົດສໍາລັບ DeepDendrite, ຮູບແບບ full-spine ແລະຮູບແບບເຄືອຂ່າຍ dendritic ວັດສະດວກແມ່ນມີຢູ່ໃນອິນເຕີເນັດ (ເບິ່ງ Code Availability). ລະຫັດການຝຶກອົບຮົມ open-source ຂອງພວກເຮົາສາມາດຖືກເຂົ້າລະຫັດຢ່າງງ່າຍດາຍກັບປົກກະຕິການຝຶກອົບຮົມ dendritic ອື່ນໆ, ເຊັ່ນດຽວກັນກັບປົກກະຕິການຝຶກອົບຮົມສໍາລັບ dendrites nonlinear (full-active) ປະເພດຂອງ Burst-dependent synaptic plasticity , ແລະຊອກຫາກັບ spike prediction ໂດຍການນໍາໃຊ້ຄວາມເຂັ້ມແຂງຂອງການຄອມພິວເຕີ GPU, ພວກເຮົາມີຄວາມຮູ້ວ່າເຄື່ອງມືນີ້ຈະຊ່ວຍໃຫ້ການທົດສອບພື້ນຖານການຄອມພິວເຕີຂອງສະຖາບັນຂອງວົງຈອນ, ເຊັ່ນດຽວກັນກັບການຮ່ວມມືລະຫວ່າງ neuroscience ແລະ AI modern. 21 20 36 ຄວາມຄິດເຫັນ ວິທີການ Dendritic Hierarchical Scheduling (DHS) ການຄອມພິວເຕີຂອງພະລັງງານ ionic ແລະການແກ້ໄຂປະເພດ linear ແມ່ນສອງປະເພດທີ່ສໍາຄັນໃນເວລາທີ່ simulating neurons biophysically detailed, ເຊິ່ງເປັນທີ່ໃຊ້ເວລາທີ່ໃຊ້ເວລາແລະສ້າງຄວາມອຸດົມສົມບູນການຄອມພິວເຕີທີ່ເຂັ້ມແຂງ. ຂໍຂອບໃຈ, ການຄອມພິວເຕີຂອງພະລັງງານ ionic ຂອງແຕ່ລະອຸດົມສົມບູນແມ່ນການປິ່ນປົວທີ່ບໍ່ເສຍຄ່າ, ດັ່ງນັ້ນມັນສາມາດຖືກ parallelized ໂດຍທົ່ວໄປໃນອຸປະກອນທີ່ມີອຸປະກອນຄອມພິວເຕີ parallel massive ເຊັ່ນ GPUs ໃນຖານະເປັນຜົນປະໂຫຍດ, solving linear equations becomes the remaining bottleneck for the parallelization process (ຮູບ. ລະຫັດ QR 37 ລະຫັດ QR ສໍາລັບການປິ່ນປົວການປິ່ນປົວນີ້, ວິທີການ parallel ໃນລະດັບ cellular ໄດ້ຖືກພັດທະນາ, ເຊິ່ງເຮັດໃຫ້ການຄອມພິວເຕີ single-cell ໂດຍການ "ຕໍາແຫນ່ງ" cell single ໃນຂະບວນການຈໍານວນຫຼາຍທີ່ສາມາດຄອມພິວເຕີໃນ parallel , , ຢ່າງໃດກໍຕາມ, ວິທີການເຫຼົ່ານີ້ອະນຸຍາດຢ່າງກວ້າງຂວາງກ່ຽວກັບຄວາມຮູ້ທີ່ຜ່ານມາເພື່ອສ້າງສະຖານະການມືອາຊີບກ່ຽວກັບວິທີການຕໍາແຫນ່ງ neuron ຫນຶ່ງໃນຫ້ອງ (ຮູບ. · Fig supplementary ). ດັ່ງນັ້ນ, ມັນເປັນບໍ່ມີປະສິດທິພາບຫຼາຍສໍາລັບ neurons ມີ morphologies asymmetric, ເຊັ່ນດຽວກັນກັບ neurons pyramidal ແລະ neurons Purkinje. 27 28 38 ລະຫັດ QR 1 ພວກເຮົາມີຄວາມຊ່ຽວຊານໃນການພັດທະນາວິທີ parallel ທີ່ມີປະສິດທິພາບສູງແລະຄຸນນະພາບສູງສໍາລັບການ simulation ຂອງເຄືອຂ່າຍ neuronal ທີ່ມີລາຍລະອຽດ biologically. First, ພວກເຮົາມີຄວາມຄຸນນະພາບສໍາລັບຄວາມຖືກຕ້ອງຂອງວິທີ parallel ໃນລະດັບ cellular. Based on theories in parallel computing , ພວກເຮົາສະເຫນີສາມສະຖານທີ່ເພື່ອຮັບປະກັນວ່າການປິ່ນປົວແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ່ນແຜ 34 ພວກເຮົາ ກໍາ ລັງເຮັດທຸລະກິດໃນ 2012. ພວກເຮົາ ກໍາ ລັງເຮັດທຸລະກິດໃນ 2012. ພວກເຮົາ ກໍາ ລັງເຮັດທຸລະກິດໃນ 2012. ພວກເຮົາ ກໍາ ລັງເຮັດທຸລະກິດໃນ 2012. ພວກເຮົາ ກໍາ ລັງເຮັດທຸລະກິດໃນ 2012. ທັດສະນະ parallel, ພວກເຮົາສາມາດຄອມພິວເຕີໃນສູງສຸດ Nodes ມີການປິ່ນປົວໃນແຕ່ລະເລີ່ມຕົ້ນ, ແຕ່ພວກເຮົາມີຄວາມຕ້ອງການທີ່ຈະຮັບປະກັນວ່າ nodes ໄດ້ຖືກຄອມພິວເຕີພຽງແຕ່ຖ້າຫາກວ່າ nodes ຂອງຕົນທັງຫມົດໄດ້ຖືກປິ່ນປົວ; ວິທີການຂອງພວກເຮົາມີຄວາມຕ້ອງການທີ່ຈະຊອກຫາການຄົ້ນຄວ້າທີ່ມີຂະຫນາດນ້ອຍຂອງເລີ່ມຕົ້ນສໍາລັບການປິ່ນປົວທັງຫມົດ. k k ສໍາລັບການຜະລິດ partition ທີ່ດີທີ່ສຸດ, ພວກເຮົາມີຄໍາຮ້ອງສະຫມັກຂອງວິທີທີ່ເອີ້ນວ່າ Dendritic Hierarchical Scheduling (DHS) (ໃບຢັ້ງຢືນດ້ານວິຊາການຖືກສະເຫນີໃນ Methods). ຄວາມຄິດເຫັນທີ່ສໍາຄັນຂອງ DHS ແມ່ນເພື່ອ priorize nodes deep (ຮູບ. ວິທີການ DHS ລວມທັງສອງເລີ່ມຕົ້ນ: ການທົດສອບ topology dendritic ແລະຊອກຫາ partition ທີ່ດີທີ່ສຸດ: (1) ໂດຍສະເຫນີຮູບແບບຂະຫນາດນ້ອຍ, ພວກເຮົາເລີ່ມຕົ້ນໄດ້ຮັບແຜ່ນການອະນຸຍາດທີ່ກ່ຽວຂ້ອງແລະຄອມຮັບຄວາມຍາວຂອງແຕ່ລະ nodes (ຄວາມຍາວຂອງ nodes ແມ່ນຈໍານວນຂອງ nodes ancestor ຂອງຕົນ) ໃນປັດຈຸບັນ (ຮູບ. ) (2) ຫຼັງຈາກການທົດສອບ topology, ພວກເຮົາມີການຊອກຫາສໍາລັບຜູ້ຊ່ຽວຊານແລະເລືອກທີ່ສູງທີ່ສຸດ Nodes candidate ອື່ນໆ (node ເປັນ candidate ພຽງແຕ່ຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນ). ລະຫັດ QR ລະຫັດ QR ປະເພດ 2B k ລະຫັດ QR ລະບົບການເຮັດວຽກຂອງ DHS ຄູ່ມືທີ່ດີທີ່ສຸດສໍາລັບການ iteration. ຮູບພາບ ສໍາ ລັບ calculating node depth of a compartmental model. The model is first converted into a tree structure then the depth of each node is calculated. Colors indicate different depth values. ການທົດສອບ topology ກ່ຽວກັບຮູບແບບ neuron ທີ່ແຕກຕ່າງກັນ. Six neurons with distinct morphologies are shown here. For each model, the soma is selected as the root of the tree so the depth of the node increases from the soma (0) to the distal dendrites. ຮູບພາບ ສໍາ ລັບ ການເຮັດວຽກຂອງ DHS ກ່ຽວກັບຮູບແບບໃນ ທີ່ຢູ່ ສະ ຫນັບ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ Nodes Processed: Nodes ທີ່ໄດ້ຮັບການປິ່ນປົວທີ່ຜ່ານມາ. ວິທີການ Parallelization ທີ່ໄດ້ຮັບໂດຍ DHS ຫຼັງຈາກການເຮັດວຽກໃນ DHS reduces the steps of serial node processing from 14 to 5 by distributing nodes to multiple threads. DHS reduces the steps of serial node processing from 14 to 5 by distributing nodes to multiple threads. Relative cost, i.e., the proportion of the computational cost of DHS to that of the serial Hines method, when applying DHS with different numbers of threads on different types of models. a k b c d b k e d f ວິທີການຄອມພິວເຕີ serial Hines, ມັນໃຊ້ເວລາ 14 ຊົ່ວໂມງເພື່ອປິ່ນປົວທັງຫມົດ nodes, ໃນຂະນະທີ່ໃຊ້ DHS ມີ 4 ຊົ່ວໂມງ parallel ສາມາດ particiate nodes ຂອງຕົນໃນ 5 subset (ຮູບ. {1,7,11,13}, {2,3,4,8}, {6}, {5}}. ເນື່ອງຈາກວ່າ nodes ໃນ subset ທີ່ແຕກຕ່າງກັນສາມາດໄດ້ຮັບການປິ່ນປົວໃນ parallel, ມັນໃຊ້ເວລາພຽງແຕ່ 5 ຊົ່ວໂມງເພື່ອປິ່ນປົວ nodes ທັງຫມົດໂດຍໃຊ້ DHS (ຮູບ. ລະຫັດ QR ລະຫັດ QR ພາສາລາວ ຫຼັງຈາກນັ້ນ, ພວກເຮົາມີການນໍາໃຊ້ວິທີ DHS ໃນ 6 ໂມເລກຸນ neuron detailed representative (ການເລືອກຈາກ ModelDB ) with different numbers of threads (Fig. ):, including cortical and hippocampal pyramidal neurons , , ລະຫັດ QR ການທົດສອບ neurons striatal (SPN) ), ແລະລັກສະນະລັກສະນະ mitral ການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງ ), suggesting adding more threads doesn’t improve performance further due to the dependencies between compartments. 39 2f 40 41 42 43 44 45 ປະເພດ ພວກເຮົາມີວິທີການ DHS ທີ່ສາມາດນໍາໃຊ້ການທົດສອບອັດຕະໂນມັດຂອງ topology dendritic ແລະ partition ທີ່ດີທີ່ສຸດສໍາລັບການຄອມພິວເຕີ parallel. ມັນຄວນຮູ້ວ່າ DHS ໄດ້ຊອກຫາ partition ທີ່ດີທີ່ສຸດກ່ອນທີ່ຈະເລີ່ມຕົ້ນການ simulation, ແລະບໍ່ຈໍາເປັນຕ້ອງຄອມພິວເຕີເພີ່ມເຕີມເພື່ອປິ່ນປົວປະເພດ. ຄວາມໄວສູງຂອງ DHS ໂດຍ GPU Memory Boosting DHS calculates each neuron with multiple threads, which consumes a vast amount of threads when running neural network simulations. Graphics Processing Units (GPUs) consists of massive processing units (ທີ່ເປັນ, ການປິ່ນປົວ streaming, SPs, ຮູບ. ) ສໍາລັບການຄອມພິວເຕີ parallel . In theory, many SPs on the GPU should support efficient simulation for large-scale neural networks (Fig. ). ຢ່າງໃດກໍຕາມ, ພວກເຮົາມີຜົນປະໂຫຍດຢ່າງກວ້າງຂວາງວ່າປະສິດທິພາບຂອງ DHS ໄດ້ຕັດສິນໃຈຢ່າງກວ້າງຂວາງໃນຂະນະທີ່ຂະຫນາດເຄືອຂ່າຍຂະຫນາດໃຫຍ່, ເຊິ່ງສາມາດເປັນຜົນປະໂຫຍດຂອງການເກັບຮັກສາຂໍ້ມູນທີ່ແຕກຕ່າງກັນຫຼືການເຂົ້າເຖິງອຸປະກອນເພີ່ມເຕີມທີ່ເກີດຂຶ້ນໂດຍການດາວໂຫລດແລະຂຽນລັກສະນະ intermediate (ຮູບ. ຂໍຂອບໃຈ ປະເພດ 3a 46 3c ລະຫັດ QR GPU Architecture ແລະອຸປະກອນການອຸປະກອນຂອງຕົນ. ແຕ່ລະ GPU ມີອຸປະກອນການປິ່ນປົວຂະຫນາດໃຫຍ່ (processor Stream). ປະເພດທີ່ແຕກຕ່າງກັນຂອງອຸປະກອນອຸປະກອນມີປະເພດທີ່ແຕກຕ່າງກັນ. ການອອກແບບຂອງ Streaming Multiprocessors (SMs). ທັງຫມົດ SM ມີ multi-streaming processors, registers, ແລະ cache L1. ນໍາ ເວັບ ໄຊ ທ ໌ ອອນ ໄລ ນ ໌ ວັນ ທີ ການ ສ້າງ ຕັ້ງ ສະ ເພາະ ສໍາ ລັບ lovers ສັດ ລ້ຽງ. ການຄາດຄະເນດິນດີຕ້ອນຮັບທີ່ດີທີ່ສຸດສໍາລັບ GPU. Top panel, thread assignment, and data storage of DHS, before (left) and after (right) memory boosting. ການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງການປິ່ນປົວຂອງ ທີ່ໃຊ້ເວລາການເຮັດວຽກຂອງ DHS (32 ສາຍຂອງແຕ່ລະແຜ່ນ) ມີແລະບໍ່ມີຄວາມປອດໄພຂອງຄວາມປອດໄພໃນຮູບແບບ pyramidal multi-layer 5 ມີ spines. Speed up of memory boosting on multiple layer 5 pyramidal models with spines. Memory boosting brings 1.6-2 times speedup. a b c d d e f ພວກເຮົາມີການປິ່ນປົວຄວາມປອດໄພຂອງ GPU Memory Boosting, ການປິ່ນປົວຄວາມປອດໄພຂອງ GPU Memory Boost, ການປິ່ນປົວຄວາມປອດໄພຂອງ GPU Memory Boost, ການປິ່ນປົວຄວາມປອດໄພຂອງ GPU Memory Boost, ການປິ່ນປົວຄວາມປອດໄພຂອງ GPU Memory Boost, ການປິ່ນປົວຄວາມປອດໄພຂອງ GPU Memory Boost, ການປິ່ນປົວຄວາມປອດໄພຂອງ GPU Memory Boost, ການປິ່ນປົວຄວາມປອດໄພຂອງ GPU Memory Boost, ການປິ່ນປົວຄວາມປອດໄພຂອງ GPU Memory Boost , . To achieve high throughput, we first align the computing orders of nodes and rearrange threads according to the number of nodes on them. Then we permute data storage in global memory, consistent with computing orders, i.e., nodes that are processed at the same step are stored successively in global memory. Moreover, we use GPU registers to store intermediate results, further strengthening memory throughput. The example shows that memory boosting takes only two memory transactions to load eight request data (Fig. , right). Furthermore, experiments on multiple numbers of pyramidal neurons with spines and the typical neuron models (Fig. · Fig supplementary ) ສະແດງໃຫ້ເຫັນວ່າການເພີ່ມຂຶ້ນຂອງຄວາມຮູ້ສຶກໄດ້ໃຫ້ເປັນ 1.2-3.8 ເວລາຄວາມໄວລຸ້ນ compared ກັບ DHS naïve. 46 47 3d 3E, F 2 ສໍາລັບການທົດສອບປະສິດທິພາບຂອງ DHS ກັບ GPU Memory Boosting, ພວກເຮົາເລືອກເອົາ 6 ໂມເລກຸນ neuron ປະເພດແລະ evaluating ໄລຍະເວລາຂອງການແກ້ໄຂວິທີການສາຍໄຟໃນຈໍານວນຂະຫນາດໃຫຍ່ຂອງແຕ່ລະມາດຕະຖານ (ຮູບ. ). We examined DHS with four threads (DHS-4) and sixteen threads (DHS-16) for each neuron, respectively. Compared to the GPU method in CoreNEURON, DHS-4 and DHS-16 can speed up about 5 and 15 times, respectively (Fig. ). ນອກເຫນືອໄປຈາກນີ້, compared to the conventional serial Hines method in NEURON running with a single-thread of CPU, DHS accelerates simulation by 2-3 orders of magnitude (Fig. Supplementary. ), while retaining the identical numerical accuracy in the presence of dense spines (Figs Supplementary. ແລະ ), dendrites ອັດຕະໂນມັດ ( supplementary Fig. ) ແລະສະຖານທີ່ segmentation ທີ່ແຕກຕ່າງກັນ (Fig. ). 4 ລະຫັດ QR 3 4 8 7 7 ທີ່ໃຊ້ເວລາການປິ່ນປົວປະເພດສໍາລັບການ simulation 1 s ກ່ຽວກັບ GPU (dt = 0.025 ms, 40,000 iterations ໃນທັງຫມົດ). CoreNEURON: ການປິ່ນປົວ paralel ທີ່ຖືກນໍາໃຊ້ໃນ CoreNEURON; DHS-4: DHS ມີ 4 ສາຍສໍາລັບທຸກ neuron; DHS-16: DHS ມີ 16 ສາຍສໍາລັບທຸກ neuron. ຂໍຂອບໃຈ ການດາວໂຫລດຂອງການແຜ່ໂດຍ DHS-4 ແລະ DHS-16, ສີຂຽວສະແດງໃຫ້ເຫັນສາຍຫນຶ່ງ. ໃນໄລຍະການຄອມພິວເຕີ, ທຸກສາຍຂົນສົ່ງລະຫວ່າງສາຍທີ່ແຕກຕ່າງກັນ. a b c DHS creates cell-type-specific partitioning ທີ່ດີທີ່ສຸດ ສໍາລັບການຊອກຫາຄວາມຮູ້ກ່ຽວກັບ mekanism ຂອງການເຮັດວຽກຂອງວິທີ DHS, ພວກເຮົາມີຄວາມຊອກຫາຂອງການປິ່ນປົວໂດຍ carting compartments to each thread (ແຕ່ລະ color presents a single thread in Fig. 1). ). visualization ສະແດງໃຫ້ເຫັນວ່າສາຍຫນຶ່ງໄດ້ຖືກປ່ຽນແປງຢ່າງກວ້າງຂວາງລະຫວ່າງຊຸດທີ່ແຕກຕ່າງກັນ (ຮູບ. Interestingly, DHS generates partitions aligned in morphologically symmetric neurons such as the striatal projection neuron (SPN) and the Mitral cell (ຮູບ. ). ໂດຍການຕັດສິນໃຈ, ມັນສ້າງ partitions fragmented ຂອງ neurons morphologically asymmetric ເຊັ່ນ pyramidal neurons ແລະ purkinje cell (ຮູບ. ), ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ 4b, c ລະຫັດ QR 4b, c ລະຫັດ QR ຂ້າພະເຈົ້າສືບຕໍ່ໄດ້ຮັບການປະທັບໃຈກໍໂດຍການຝຶກອົບຮົມຂອງພວກເຮົາສໍາລັບການຝຶກອົບຮົມ ITS ແລະການຝຶກອົບຮົມ ITS DHS ສະຫນັບສະຫນູນການອອກແບບລະດັບ spine ໃນຂະນະທີ່ spines dendritic ຮັບສ່ວນໃຫຍ່ຂອງ input excitatory ກັບ neurons pyramidal cortical ແລະ hippocampal, neurons proyection striatal, ແລະອື່ນໆ, morphologies ແລະ plasticity ຂອງພວກເຂົາແມ່ນສໍາຄັນສໍາລັບການຄວບຄຸມ excitability neuronal , , , , ແຕ່, spines ແມ່ນຂະຫນາດນ້ອຍ ( ~ 1 μm ຂະຫນາດ) ເພື່ອຖືກກວດສອບໂດຍຜ່ານການທົດສອບໂດຍອີງໃສ່ການປິ່ນປົວທີ່ກ່ຽວຂ້ອງກັບຄວາມກົດດັນ. ດັ່ງນັ້ນ, ການເຮັດວຽກທາງດ້ານວິຊາການແມ່ນສໍາຄັນສໍາລັບການຮູ້ສຶກທັງຫມົດຂອງການຄອມພິວເຕີ spine. 10 48 49 50 51 ພວກເຮົາສາມາດມາດຕະຖານຕົວຈິງຫນຶ່ງທີ່ມີສອງກ້ອງຖ່າຍຮູບ: ຫນ້າຈໍຕົວຈິງທີ່ synapses ແມ່ນຕັ້ງຢູ່ໃນແລະ ຫນ້າຈໍຕົວຈິງທີ່ເຊື່ອມຕໍ່ ຫນ້າຈໍຕົວຈິງກັບ dendrites . The theory predicts that the very thin spine neck (0.1-0.5 um in diameter) electronically isolates the spine head from its parent dendrite, thus compartmentalizing the signals generated at the spine head ໃນຖານະເປັນຜູ້ຊ່ຽວຊານຂອງການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມການຝຶກອົບຮົມ ປະເພດ Spine , ໃນຖານະເປັນຮູບແບບທັງຫມົດ spines explicitly. Here, the ປະສິດທິພາບຂອງ spine factor ເປັນການຄາດຄະເນຂອງ spine effect on the biophysical properties of the cell membrane . 52 53 F 54 F 54 Inspired by the previous work of Eyal et al. , we investigated how different spatial patterns of excitatory inputs formed on dendritic spines shape neuronal activities in a human pyramidal neuron model with explicitly modeled spines (Fig. ). Noticeably, Eyal et al. employed the spine factor to incorporate spines into dendrites while only a few activated spines were explicitly attached to dendrites (“few-spine model” in Fig. ) ຄຸນນະພາບຂອງ spine in their model was computed from the dendritic area and spine area in the reconstructed data. Accordingly, we calculated the spine density from their reconstructed data to make our full-spine model more consistent with Eyal’s few-spine model. With the spine density set to 1.3 μm-1, the pyramidal neuron model contained about 25,000 spines without altering the model’s original morphological and biophysical properties. Further, we repeated the previous experiment protocols with both full-spine and few-spine models. We use the same synaptic input as in Eyal’s work but attach extra background noise to each sample. By comparing the somatic traces (Fig. ) and spike probability (Fig. ) in full-spine and few-spine models, we found that the full-spine model is much leakier than the few-spine model. In addition, the spike probability triggered by the activation of clustered spines appeared to be more nonlinear in the full-spine model (the solid blue line in Fig. ) than in the few-spine model (the dashed blue line in Fig. ຜົນໄດ້ຮັບນີ້ສະແດງໃຫ້ເຫັນວ່າການປິ່ນປົວ F-factor ທີ່ປົກກະຕິສາມາດກວດສອບຜົນປະໂຫຍດຂອງ spine dense ກ່ຽວກັບການຄອມພິວເຕີຂອງ excitability dendritic ແລະ nonlinearity. 51 5a F 5a F 5b, c ປະເພດ 5d 5d Experiment setup. We examine two major types of models: few-spine models and full-spine models. Few-spine models (two on the left) are the models that incorporated spine area globally into dendrites and only attach individual spines together with activated synapses. In full-spine models (two on the right), all spines are explicitly attached over whole dendrites. We explore the effects of clustered and randomly distributed synaptic inputs on the few-spine models and the full-spine models, respectively. Somatic voltages recorded for cases in . Colors of the voltage curves correspond to , scale bar: 20 ms, 20 mV. Color-coded voltages during the simulation in at specific times. Colors indicate the magnitude of voltage. ຄຸນນະສົມບັດຂອງ spike somatic ເປັນຂະບວນການຂອງຈໍານວນ synapses activated ໃນຂະນະດຽວກັນ (ເຊັ່ນດຽວກັນກັບການເຮັດວຽກຂອງ Eyal ແລະອື່ນໆ) ໃນໄລຍະປະມານ 4 . Background noise is attached. Run time of experiments in with different simulation methods. NEURON: conventional NEURON simulator running on a single CPU core. CoreNEURON: CoreNEURON simulator on a single GPU. DeepDendrite: DeepDendrite on a single GPU. a b a a c b d a e d In the DeepDendrite platform, both full-spine and few-spine models achieved 8 times speedup compared to CoreNEURON on the GPU platform and 100 times speedup compared to serial NEURON on the CPU platform (Fig. ; Supplementary Table ) while keeping the identical simulation results (Supplementary Figs. and ). ດັ່ງນັ້ນ, ວິທີການ DHS ເຮັດໃຫ້ການທົດລອງຂອງ excitability dendritic ໃນສະພາບແວດລ້ອມທີ່ແທ້ຈິງຫຼາຍ. 5e 1 4 8 Discussion In this work, we propose the DHS method to parallelize the computation of Hines method and we mathematically demonstrate that the DHS provides an optimal solution without any loss of precision. Next, we implement DHS on the GPU hardware platform and use GPU memory boosting techniques to refine the DHS (Fig. ). When simulating a large number of neurons with complex morphologies, DHS with memory boosting achieves a 15-fold speedup (Supplementary Table ) as compared to the GPU method used in CoreNEURON and up to 1,500-fold speedup compared to serial Hines method in the CPU platform (Fig. · Fig supplementary and Supplementary Table ). Furthermore, we develop the GPU-based DeepDendrite framework by integrating DHS into CoreNEURON. Finally, as a demonstration of the capacity of DeepDendrite, we present a representative application: examine spine computations in a detailed pyramidal neuron model with 25,000 spines. Further in this section, we elaborate on how we have expanded the DeepDendrite framework to enable efficient training of biophysically detailed neural networks. To explore the hypothesis that dendrites improve robustness against adversarial attacks , we train our network on typical image classification tasks. We show that DeepDendrite can support both neuroscience simulations and AI-related detailed neural network tasks with unprecedented speed, therefore significantly promoting detailed neuroscience simulations and potentially for future AI explorations. 55 3 1 4 3 1 56 Decades of efforts have been invested in speeding up the Hines method with parallel methods. Early work mainly focuses on network-level parallelization. In network simulations, each cell independently solves its corresponding linear equations with the Hines method. Network-level parallel methods distribute a network on multiple threads and parallelize the computation of each cell group with each thread , . With network-level methods, we can simulate detailed networks on clusters or supercomputers . In recent years, GPU has been used for detailed network simulation. Because the GPU contains massive computing units, one thread is usually assigned one cell rather than a cell group , , . With further optimization, GPU-based methods achieve much higher efficiency in network simulation. However, the computation inside the cells is still serial in network-level methods, so they still cannot deal with the problem when the “Hines matrix” of each cell scales large. 57 58 59 35 60 61 ວິທີການ parallel ໃນລະດັບ cellular ອັດຕະໂນມັດເພີ່ມເຕີມ paralelize ການຄອມພິວເຕີໃນລະດັບ cellular. The main idea of cellular-level parallel methods is to split each cell into several sub-blocks and parallelize the calculation of those sub-blocks. , . However, typical cellular-level methods (e.g., the “multi-split” method ) pay less attention to the parallelization strategy. The lack of a fine parallelization strategy results in unsatisfactory performance. To achieve higher efficiency, some studies try to obtain finer-grained parallelization by introducing extra computation operations , , or making approximations on some crucial compartments, while solving linear equations , ວິທີການ parallelization ທີ່ມີຂະຫນາດນ້ອຍເຫຼົ່ານີ້ສາມາດໄດ້ຮັບການປະສິດທິພາບສູງກວ່າ, ແຕ່ບໍ່ມີຄຸນນະສົມບັດຄຸນນະສົມບັດຄຸນນະສົມບັດເຊັ່ນດຽວກັນກັບວິທີ Hines ທໍາອິດ. 27 28 28 29 38 62 63 64 Unlike previous methods, DHS adopts the finest-grained parallelization strategy, i.e., compartment-level parallelization. By modeling the problem of “how to parallelize” as a combinatorial optimization problem, DHS provides an optimal compartment-level parallelization strategy. Moreover, DHS does not introduce any extra operation or value approximation, so it achieves the lowest computational cost and retains sufficient numerical accuracy as in the original Hines method at the same time. Dendritic spines are the most abundant microstructures in the brain for projection neurons in the cortex, hippocampus, cerebellum, and basal ganglia. As spines receive most of the excitatory inputs in the central nervous system, electrical signals generated by spines are the main driving force for large-scale neuronal activities in the forebrain and cerebellum , . The structure of the spine, with an enlarged spine head and a very thin spine neck—leads to surprisingly high input impedance at the spine head, which could be up to 500 MΩ, combining experimental data and the detailed compartment modeling approach , . Due to such high input impedance, a single synaptic input can evoke a “gigantic” EPSP ( ~ 20 mV) at the spine-head level , , thereby boosting NMDA currents and ion channel currents in the spine . However, in the classic single detailed compartment models, all spines are replaced by the coefficient modifying the dendritic cable geometries . This approach may compensate for the leak currents and capacitance currents for spines. Still, it cannot reproduce the high input impedance at the spine head, which may weaken excitatory synaptic inputs, particularly NMDA currents, thereby reducing the nonlinearity in the neuron’s input-output curve. Our modeling results are in line with this interpretation. 10 11 48 65 48 66 11 F 54 On the other hand, the spine’s electrical compartmentalization is always accompanied by the biochemical compartmentalization , , , resulting in a drastic increase of internal [Ca2+], within the spine and a cascade of molecular processes involving synaptic plasticity of importance for learning and memory. Intriguingly, the biochemical process triggered by learning, in turn, remodels the spine’s morphology, enlarging (or shrinking) the spine head, or elongating (or shortening) the spine neck, which significantly alters the spine’s electrical capacity , , , . Such experience-dependent changes in spine morphology also referred to as “structural plasticity”, have been widely observed in the visual cortex , ລະຫັດ QR , , motor cortex , hippocampus , ແລະ ganglia Basal in vivo. They play a critical role in motor and spatial learning as well as memory formation. However, due to the computational costs, nearly all detailed network models exploit the “F-factor” approach to replace actual spines, and are thus unable to explore the spine functions at the system level. By taking advantage of our framework and the GPU platform, we can run a few thousand detailed neurons models, each with tens of thousands of spines on a single GPU, while maintaining ~100 times faster than the traditional serial method on a single CPU (Fig. ດັ່ງນັ້ນ, ມັນສາມາດໃຫ້ພວກເຮົາມີຄວາມສາມາດໃນການທົດສອບ plasticity ການກໍ່ສ້າງໃນຮູບແບບ circuit ຂະຫນາດໃຫຍ່ໃນພາກສ່ວນທີ່ແຕກຕ່າງກັນຂອງວົງຈອນ. 8 52 67 67 68 69 70 71 72 73 74 75 9 76 5e Another critical issue is how to link dendrites to brain functions at the systems/network level. It has been well established that dendrites can perform comprehensive computations on synaptic inputs due to enriched ion channels and local biophysical membrane properties , , . For example, cortical pyramidal neurons can carry out sublinear synaptic integration at the proximal dendrite but progressively shift to supralinear integration at the distal dendrite . Moreover, distal dendrites can produce regenerative events such as dendritic sodium spikes, calcium spikes, and NMDA spikes/plateau potentials , . Such dendritic events are widely observed in mice or even human cortical neurons in vitro, which may offer various logical operations , or gating functions , . Recently, in vivo recordings in awake or behaving mice provide strong evidence that dendritic spikes/plateau potentials are crucial for orientation selectivity in the visual cortex , sensory-motor integration in the whisker system , , and spatial navigation in the hippocampal CA1 region . 5 6 7 77 6 78 6 79 6 79 80 81 82 83 84 85 To establish the causal link between dendrites and animal (including human) patterns of behavior, large-scale biophysically detailed neural circuit models are a powerful computational tool to realize this mission. However, running a large-scale detailed circuit model of 10,000-100,000 neurons generally requires the computing power of supercomputers. It is even more challenging to optimize such models for in vivo data, as it needs iterative simulations of the models. The DeepDendrite framework can directly support many state-of-the-art large-scale circuit models , , , which were initially developed based on NEURON. Moreover, using our framework, a single GPU card such as Tesla A100 could easily support the operation of detailed circuit models of up to 10,000 neurons, thereby providing carbon-efficient and affordable plans for ordinary labs to develop and optimize their own large-scale detailed models. 86 87 88 Recent works on unraveling the dendritic roles in task-specific learning have achieved remarkable results in two directions, i.e., solving challenging tasks such as image classification dataset ImageNet with simplified dendritic networks , and exploring full learning potentials on more realistic neuron , . However, there lies a trade-off between model size and biological detail, as the increase in network scale is often sacrificed for neuron-level complexity , , ນອກເຫນືອໄປຈາກນີ້, ໂມເລກຸນ neuron ຫຼາຍກວ້າງຂວາງແມ່ນບໍ່ມີຄຸນນະພາບຄຸນນະພາບແລະຄຸນນະພາບຄຸນນະພາບຄຸນນະພາບ . 20 21 22 19 20 89 21 There has also been progress in the role of active dendrites in ANNs for computer vision tasks. Iyer et al. . proposed a novel ANN architecture with active dendrites, demonstrating competitive results in multi-task and continual learning. Jones and Kording used a binary tree to approximate dendrite branching and provided valuable insights into the influence of tree structure on single neurons’ computational capacity. Bird et al. . proposed a dendritic normalization rule based on biophysical behavior, offering an interesting perspective on the contribution of dendritic arbor structure to computation. While these studies offer valuable insights, they primarily rely on abstractions derived from spatially extended neurons, and do not fully exploit the detailed biological properties and spatial information of dendrites. Further investigation is needed to unveil the potential of leveraging more realistic neuron models for understanding the shared mechanisms underlying brain computation and deep learning. 90 91 92 ພວກເຮົາໄດ້ພັດທະນາ DeepDendrite, ເຄື່ອງມືທີ່ນໍາໃຊ້ວິທີການ Dendritic Hierarchical Scheduling (DHS) ເພື່ອຫຼຸດຜ່ອນຄ່າໃຊ້ຈ່າຍການຄອມພິວເຕີຢ່າງກວ້າງຂວາງແລະປະກອບມີໂມດູນ I/O ແລະໂມດູນການຝຶກອົບຮົມເພື່ອປິ່ນປົວຊຸດຂໍ້ມູນຂະຫນາດໃຫຍ່. ມີ DeepDendrite, ພວກເຮົາມີຄວາມຍິນດີເລີດການນໍາໃຊ້ເຄືອຂ່າຍ neuronal hybrid three-layer, Human Pyramidal Cell Network (HPC-Net) (ຮູບ. ອຸດສາຫະກໍາການຝຶກອົບຮົມທີ່ມີຄວາມສາມາດໃນການຝຶກອົບຮົມທີ່ມີຄຸນນະພາບສູງສໍາລັບການຝຶກອົບຮົມທີ່ມີຄຸນນະພາບສູງໃນໂຄງການຄຸນນະພາບຮູບພາບທີ່ມີຄຸນນະພາບສູງສຸດ, ມີຄວາມໄວສູງປະມານ 25 ຊົ່ວໂມງກ່ວາການຝຶກອົບຮົມທີ່ມີການຝຶກອົບຮົມທີ່ມີຄຸນນະພາບສູງສໍາລັບການຝຶກອົບຮົມທີ່ມີຄຸນນະພາບສູງ. ; Supplementary Table ). 6a, b 6f 1 The illustration of the Human Pyramidal Cell Network (HPC-Net) for image classification. Images are transformed to spike trains and fed into the network model. Learning is triggered by error signals propagated from soma to dendrites. Training with mini-batch. Multiple networks are simulated simultaneously with different images as inputs. The total weight updates ΔW are computed as the average of ΔWi from each network. Comparison of the HPC-Net before and after training. Left, the visualization of hidden neuron responses to a specific input before (top) and after (bottom) training. Right, hidden layer weights (from input to hidden layer) distribution before (top) and after (bottom) training. Workflow of the transfer adversarial attack experiment. We first generate adversarial samples of the test set on a 20-layer ResNet. Then use these adversarial samples (noisy images) to test the classification accuracy of models trained with clean images. Prediction accuracy of each model on adversarial samples after training 30 epochs on MNIST (left) and Fashion-MNIST (right) datasets. ທີ່ໃຊ້ເວລາການຝຶກອົບຮົມແລະການທົດສອບສໍາລັບ HPC-Net. ຂະຫນາດ batch ໄດ້ຖືກຕັ້ງຢູ່ໃນ 16. ຂ້າງ, ທີ່ໃຊ້ເວລາການຝຶກອົບຮົມໃນໄລຍະຫນຶ່ງ. ທີ່ໃຊ້ເວລາການຝຶກອົບຮົມໃນໄລຍະຫນຶ່ງ. Parallel NEURON + Python: ການຝຶກອົບຮົມແລະການທົດສອບໃນຫນຶ່ງ CPU ທີ່ມີຂະຫນາດໃຫຍ່, ການນໍາໃຊ້ 40 Process-Parallel NEURON ເພື່ອມາດຕະຖານ HPC-Net ແລະລະຫັດເພີ່ມເຕີມ Python ເພື່ອສະຫນັບສະຫນູນການຝຶກອົບຮົມ mini-batch. DeepDendrite: ການຝຶກອົບຮົມແລະການທົດສອບ HPC-Net ໃນຫນຶ່ງ GPU ມີ DeepDendrite. a b c d e f Additionally, it is widely recognized that the performance of Artificial Neural Networks (ANNs) can be undermined by adversarial attacks —intentionally engineered perturbations devised to mislead ANNs. Intriguingly, an existing hypothesis suggests that dendrites and synapses may innately defend against such attacks ການທົດສອບຂອງພວກເຮົາມີການນໍາໃຊ້ HPC-Net ເພື່ອສະຫນັບສະຫນູນ hypothesis ນີ້, ໃນຂະນະທີ່ພວກເຮົາມີຄວາມຮູ້ສຶກວ່າເຄືອຂ່າຍທີ່ມີໂຄງສ້າງ dendritic ອັດຕະໂນມັດໄດ້ສະແດງໃຫ້ເຫັນຄວາມປອດໄພເພີ່ມເຕີມກັບການຕັດສິນໃຈ. compared to standard ANNs, as evident in MNIST and Fashion-MNIST datasets (Fig. ). This evidence implies that the inherent biophysical properties of dendrites could be pivotal in augmenting the robustness of ANNs against adversarial interference. Nonetheless, it is essential to conduct further studies to validate these findings using more challenging datasets such as ImageNet . 93 56 94 95 96 6d, e 97 In conclusion, DeepDendrite has shown remarkable potential in image classification tasks, opening up a world of exciting future directions and possibilities. To further advance DeepDendrite and the application of biologically detailed dendritic models in AI tasks, we may focus on developing multi-GPU systems and exploring applications in other domains, such as Natural Language Processing (NLP), where dendritic filtering properties align well with the inherently noisy and ambiguous nature of human language. Challenges include testing scalability in larger-scale problems, understanding performance across various tasks and domains, and addressing the computational complexity introduced by novel biological principles, such as active dendrites. By overcoming these limitations, we can further advance the understanding and capabilities of biophysically detailed dendritic neural networks, potentially uncovering new advantages, enhancing their robustness against adversarial attacks and noisy inputs, and ultimately bridging the gap between neuroscience and modern AI. Methods Simulation with DHS CoreNEURON simulator ( ) uses the NEURON architecture and is optimized for both memory usage and computational speed. We implement our Dendritic Hierarchical Scheduling (DHS) method in the CoreNEURON environment by modifying its source code. All models that can be simulated on GPU with CoreNEURON can also be simulated with DHS by executing the following command: 35 https://github.com/BlueBrain/CoreNeuron 25 coreneuron_exec -d /path/to/models -e time --cell-permute 3 --cell-nthread 16 --gpu The usage options are as in Table . 1 Accuracy of the simulation using cellular-level parallel computation To ensure the accuracy of the simulation, we first need to define the correctness of a cellular-level parallel algorithm to judge whether it will generate identical solutions compared with the proven correct serial methods, like the Hines method used in the NEURON simulation platform. Based on the theories in parallel computing , a parallel algorithm will yield an identical result as its corresponding serial algorithm, if and only if the data process order in the parallel algorithm is consistent with data dependency in the serial method. The Hines method has two symmetrical phases: triangularization and back-substitution. By analyzing the serial computing Hines method , we find that its data dependency can be formulated as a tree structure, where the nodes on the tree represent the compartments of the detailed neuron model. In the triangularization process, the value of each node depends on its children nodes. In contrast, during the back-substitution process, the value of each node is dependent on its parent node (Fig. ). Thus, we can compute nodes on different branches in parallel as their values are not dependent. 34 55 ປະເພດ Based on the data dependency of the serial computing Hines method, we propose three conditions to make sure a parallel method will yield identical solutions as the serial computing Hines method: (1) The tree morphology and initial values of all nodes are identical to those in the serial computing Hines method; (2) In the triangularization phase, a node can be processed if and only if all its children nodes are already processed; (3) In the back-substitution phase, a node can be processed only if its parent node is already processed. Once a parallel computing method satisfies these three conditions, it will produce identical solutions as the serial computing method. Computational cost of cellular-level parallel computing method To theoretically evaluate the run time, i.e., efficiency, of the serial and parallel computing methods, we introduce and formulate the concept of computational cost as follows: given a tree and threads (basic computational units) to perform triangularization, parallel triangularization equals to divide the node set of into subsets, i.e., = { , , … } where the size of each subset | | ≤ , i.e., at most nodes can be processed each step since there are only threads. The process of the triangularization phase follows the order: → → … → , ແລະ nodes ໃນ subset ນີ້ can be processed in parallel. So, we define | | (the size of set , i.e., here) as the computational cost of the parallel computing method. In short, we define the computational cost of a parallel method as the number of steps it takes in the triangularization phase. Because the back-substitution is symmetrical with triangularization, the total cost of the entire solving equation phase is twice that of the triangularization phase. T k V T n V V1 V2 Vn Vi k k k V1 V2 Vn Vi V V n Mathematical scheduling problem Based on the simulation accuracy and computational cost, we formulate the parallelization problem as a mathematical scheduling problem: ຊື່ຫຍໍ້ຂອງ : Tree = { , } and a positive integer , where is the node-set and is the edge set. Define partition ( ) = { , , … }, | | ≤ , 1 ≤ ≤ n, where | | indicates the cardinal number of subset , i.e., the number of nodes in , and for each node ∈ , all its children nodes { | ∈children( )} must in a previous subset , where 1 ≤ < . Our goal is to find an optimal partition ( ) whose computational cost | ( )| is minimal. T V E k V E P V V1 V2 Vn Vi k i Vi Vi Vi v Vi c c v Vj j i P* V P* V Here subset consists of all nodes that will be computed at -th step (Fig. ), so | | ≤ indicates that we can compute nodes each step at most because the number of available threads is . The restriction “for each node ∈ , all its children nodes { | ∈children( )} must in a previous subset , where 1 ≤ < ” indicates that node can be processed only if all its child nodes are processed. Vi i 2e Vi k k k v Vi c c v Vj j i v DHS implementation We aim to find an optimal way to parallelize the computation of solving linear equations for each neuron model by solving the mathematical scheduling problem above. To get the optimal partition, DHS first analyzes the topology and calculates the depth ( ) for all nodes ∈ . Then, the following two steps will be executed iteratively until every node ∈ ທີ່ຢູ່ ສະ ຫນັບ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ ສະ ຫນູນ Node ເປັນ candidate ພຽງແຕ່ຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນຫຼັງຈາກນັ້ນ. | ≤ , i.e., the number of candidate nodes is smaller or equivalent to the number of available threads, remove all nodes in and put them into , otherwise, remove deepest nodes from and add them to subset . Label these nodes as processed nodes (Fig. ). After filling in subset , go to step (1) to fill in the next subset . d v v V v V Q Q k Q V*i k Q Vi 2d Vi Vi+1 Correctness proof for DHS After applying DHS to a neural tree = { , }, we get a partition ( ) = { , , … ຫນ້າທໍາອິດ » | ≤ , 1 ≤ ≤ Nodes ໃນ subset ນີ້ will be computed in parallel, taking steps to perform triangularization and back-substitution, respectively. We then demonstrate that the reordering of the computation in DHS will result in a result identical to the serial Hines method. T V E P V V1 V2 Vn Vi k i n ປະເພດ n The partition ( ) obtained from DHS decides the computation order of all nodes in a neural tree. Below we demonstrate that the computation order determined by ( ) satisfies the correctness conditions. ( ) is obtained from the given neural tree . Operations in DHS do not modify the tree topology and values of tree nodes (corresponding values in the linear equations), so the tree morphology and initial values of all nodes are not changed, which satisfies condition 1: the tree morphology and initial values of all nodes are identical to those in serial Hines method. In triangularization, nodes are processed from subset to . As shown in the implementation of DHS, all nodes in subset are selected from the candidate set , and a node can be put into only if all its child nodes have been processed. Thus the child nodes of all nodes in are in { , , … }, meaning that a node is only computed after all its children have been processed, which satisfies condition 2: in triangularization, a node can be processed if and only if all its child nodes are already processed. In back-substitution, the computation order is the opposite of that in triangularization, i.e., from to . As shown before, the child nodes of all nodes in are in { , , … }, so parent nodes of nodes in are in { , , … }, which satisfies condition 3: in back-substitution, a node can be processed only if its parent node is already processed. P V P V P V T V1 Vn Vi Q Q Vi V1 V2 Vi-1 Vn V1 Vi V1 V2 Vi-1 Vi ປະເພດ + 1 Vi+2 Vn Optimality proof for DHS The idea of the proof is that if there is another optimal solution, it can be transformed into our DHS solution without increasing the number of steps the algorithm requires, thus indicating that the DHS solution is optimal. For each subset in ( , DHS moves (ແປດຈໍານວນ) Nodes deepest ຈາກຊຸດ candidate ທີ່ກ່ຽວຂ້ອງ ປະເພດ . If the number of nodes in ຂະຫນາດນ້ອຍກວ່າ ດາວນ໌ໂຫລດ Node ປະເພດ . To simplify, we introduce , indicating the depth sum of Node ອື່ນໆ . All subsets in ( ) ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນູນ ): . We then prove that selecting the deepest nodes in each iteration makes an optimal partition. If there exists an optimal partition = { , , … } containing subsets that do not satisfy the max-depth criteria, we can modify the subsets in ( ) so that all subsets consist of the deepest nodes from and the number of subsets ( | ( )|) remain the same after modification. ປະເພດ P V k ລະຫັດ QR ປະເພດ ລະຫັດ QR k Qi Vi Di k Qi P V 6a P(V) P*(V) V*1 V*2 V*s P* V Q P* V Without any loss of generalization, we start from the first subset not satisfying the criteria, i.e., . There are two possible cases that will make not satisfy the max-depth criteria: (1) | | < and there exist some valid nodes in that are not put to ; (2) | | = but nodes in are not the deepest nodes in . V*i V*i V*i k Qi V*i V*i k V*i k Qi For case (1), because some candidate nodes are not put to , these nodes must be in the subsequent subsets. As | | , we can move the corresponding nodes from the subsequent subsets to , which will not increase the number of subsets and make satisfy the criteria (Supplementary Fig. , top). For case (2), | ຫນ້າທໍາອິດ > , these deeper nodes that are not moved from the candidate set into must be added to subsequent subsets (Supplementary Fig. , bottom). These deeper nodes can be moved from subsequent subsets to through the following method. Assume that after filling , is picked and one of the -th deepest nodes is still in , thus will be put into a subsequent subset ( > ). We first move from to + , then modify subset + as follows: if | + | ≤ and none of the nodes in + is the parent of node , stop modifying subset ທີ່ຜ່ານມາ. ຖ້າຫາກວ່າບໍ່, modify + as follows (Supplementary Fig. ): if the parent node of is in + , move this parent node to + ; else move the node with minimum depth from + ປະເພດ + . After adjusting , modify subsequent subsets + , + , … with the same strategy. Finally, move from to . V*i V*i < k V*i V*i 6b V*i k Qi V*i 6b ປະເພດ V*i v k v’ Qi v’ V*j j i v V*i V*i 1 V*i 1 V*i 1 k ປະເພດ 1 v V*i 1 6c v V*i 1 V*i 2 V*i 1 V*i 2 V*i V*i 1 V*i 2 V*j-1 v’ V*j V*i With the modification strategy described above, we can replace all shallower nodes in with the -th deepest node in and keep the number of subsets, i.e., | ( )| the same after modification. We can modify the nodes with the same strategy for all subsets in ( ) that do not contain the deepest nodes. Finally, all subsets ∈ ( ) can satisfy the max-depth criteria, and | ( )| does not change after modifying. V*i k ລະຫັດ QR P* V P* V V*i P* V P* V In conclusion, DHS generates a partition ( ), and all subsets ∈ ( ) ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນັບ ສະ ຫນູນ ( ) ພວກເຮົາສາມາດປ່ຽນແປງ subset ຂອງຕົນເພື່ອເຮັດໃຫ້ການກໍ່ສ້າງຂອງຕົນເປັນຕົວຢ່າງ ( ), ເປັນຫຍັງ, ທັງຫມົດ subset ໄດ້ປະກອບດ້ວຍ nodes ອື່ນເຕັ້ນທີ່ສຸດໃນຊຸດ candidate, ແລະ keep. ( ) the same after modification. So, the partition ( ) obtained from DHS is one of the optimal partitions. P V ປະເພດ P V ລະຫັດ QR V P V P* V | P V ການນໍາໃຊ້ GPU ແລະການຂະຫຍາຍຕົວຂອງຄວາມຮູ້ສຶກ To achieve high memory throughput, GPU utilizes the memory hierarchy of (1) global memory, (2) cache, (3) register, where global memory has large capacity but low throughput, while registers have low capacity but high throughput. We aim to boost memory throughput by leveraging the memory hierarchy of GPU. GPU employs SIMT (Single-Instruction, Multiple-Thread) architecture. Warps are the basic scheduling units on GPU (a warp is a group of 32 parallel threads). A warp executes the same instruction with different data for different threads ໃນຖານະເປັນບໍລິສັດຂອງພວກເຮົາ, ພວກເຮົາມີຄວາມຊ່ຽວຊານໃນການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງການຄຸ້ມຄອງ. 46 When a warp loads pre-aligned and successively-stored data from global memory, it can make full use of the cache, which leads to high memory throughput, while accessing scatter-stored data would reduce memory throughput. After compartments assignment and threads rearrangement, we permute data in global memory to make it consistent with computing orders so that warps can load successively-stored data when executing the program. Moreover, we put those necessary temporary variables into registers rather than global memory. Registers have the highest memory throughput, so the use of registers further accelerates DHS. ໂມເລກຸນ biophysical full-spine ແລະ few-spine We used the published human pyramidal neuron ຄວາມອາດສາມາດຂອງ membrane m = 0.44 μF cm-2, membrane resistance m = 48,300 Ω cm2, and axial resistivity a = 261.97 Ω cm. In this model, all dendrites were modeled as passive cables while somas were active. The leak reversal potential l = -83.1 mV. Ion channels such as Na+ and K+ were inserted on soma and initial axon, and their reversal potentials were Na = 67.6 mV, K = -102 mV respectively. All these specific parameters were set the same as in the model of Eyal, et al. , for more details please refer to the published model (ModelDB, access No. 238347). 51 c r r E E E 51 In the few-spine model, the membrane capacitance and maximum leak conductance of the dendritic cables 60 μm away from soma were multiplied by a spine factor to approximate dendritic spines. In this model, spine was set to 1.9. Only the spines that receive synaptic inputs were explicitly attached to dendrites. F F In the full-spine model, all spines were explicitly attached to dendrites. We calculated the spine density with the reconstructed neuron in Eyal, et al. . The spine density was set to 1.3 μm-1, and each cell contained 24994 spines on dendrites 60 μm away from the soma. 51 The morphologies and biophysical mechanisms of spines were the same in few-spine and full-spine models. The length of the spine neck neck = 1.35 μm and the diameter neck = 0.25 μm, whereas the length and diameter of the spine head were 0.944 μm, i.e., the spine head area was set to 2.8 μm2. Both spine neck and spine head were modeled as passive cables, with the reversal potential = -86 mV. The specific membrane capacitance, membrane resistance, and axial resistivity were the same as those for dendrites. L D El Synaptic inputs We investigated neuronal excitability for both distributed and clustered synaptic inputs. All activated synapses were attached to the terminal of the spine head. For distributed inputs, all activated synapses were randomly distributed on all dendrites. For clustered inputs, each cluster consisted of 20 activated synapses that were uniformly distributed on a single randomly-selected compartment. All synapses were activated simultaneously during the simulation. AMPA-based ແລະ NMDA-based synaptic currents ໄດ້ simulated ເຊັ່ນດຽວກັນກັບການເຮັດວຽກຂອງ Eyal ແລະອື່ນໆ. AMPA conductance ໄດ້ຖືກມາດຕະຖານເປັນ function double-exponential ແລະ conductance NMDA ເປັນ function double-exponential ທີ່ກ່ຽວຂ້ອງກັບ voltage. For the AMPA model, the specific rise and decay were set to 0.3 and 1.8 ms. For the NMDA model, rise and decay were set to 8.019 and 34.9884 ms, respectively. The maximum conductance of AMPA and NMDA were 0.73 nS and 1.31 nS. τ τ τ τ Background noise We attached background noise to each cell to simulate a more realistic environment. Noise patterns were implemented as Poisson spike trains with a constant rate of 1.0 Hz. Each pattern started at start = 10 ms and lasted until the end of the simulation. We generated 400 noise spike trains for each cell and attached them to randomly-selected synapses. The model and specific parameters of synaptic currents were the same as described in , except that the maximum conductance of NMDA was uniformly distributed from 1.57 to 3.275, resulting in a higher AMPA to NMDA ratio. t Synaptic Inputs Exploring neuronal excitability We investigated the spike probability when multiple synapses were activated simultaneously. For distributed inputs, we tested 14 cases, from 0 to 240 activated synapses. For clustered inputs, we tested 9 cases in total, activating from 0 to 12 clusters respectively. Each cluster consisted of 20 synapses. For each case in both distributed and clustered inputs, we calculated the spike probability with 50 random samples. Spike probability was defined as the ratio of the number of neurons fired to the total number of samples. All 1150 samples were simulated simultaneously on our DeepDendrite platform, reducing the simulation time from days to minutes. Performing AI tasks with the DeepDendrite platform Conventional detailed neuron simulators lack two functionalities important to modern AI tasks: (1) alternately performing simulations and weight updates without heavy reinitialization and (2) simultaneously processing multiple stimuli samples in a batch-like manner. Here we present the DeepDendrite platform, which supports both biophysical simulating and performing deep learning tasks with detailed dendritic models. DeepDendrite consists of three modules (Supplementary Fig. ): (1) an I/O module; (2) a DHS-based simulating module; (3) a learning module. When training a biophysically detailed model to perform learning tasks, users first define the learning rule, then feed all training samples to the detailed model for learning. In each step during training, the I/O module picks a specific stimulus and its corresponding teacher signal (if necessary) from all training samples and attaches the stimulus to the network model. Then, the DHS-based simulating module initializes the model and starts the simulation. After simulation, the learning module updates all synaptic weights according to the difference between model responses and teacher signals. After training, the learned model can achieve performance comparable to ANN. The testing phase is similar to training, except that all synaptic weights are fixed. 5 HPC-Net model ປະເພດຮູບພາບແມ່ນການເຮັດວຽກປະເພດໃນພາກສະຫນາມຂອງ AI. ໃນການເຮັດວຽກນີ້, ຮູບແບບຄວນຈະຊອກຫາຄວາມຮູ້ສຶກອົບຮົມໃນຮູບພາບທີ່ແຕກຕ່າງກັນແລະຜະລິດຕະພັນປະເພດທີ່ກ່ຽວຂ້ອງ. Here we present the HPC-Net, a network consisting of detailed human pyramidal neuron models that can learn to perform image classification tasks by utilizing the DeepDendrite platform. HPC-Net has three layers, i.e., an input layer, a hidden layer, and an output layer. The neurons in the input layer receive spike trains converted from images as their input. Hidden layer neurons receive the output of input layer neurons and deliver responses to neurons in the output layer. The responses of the output layer neurons are taken as the final output of HPC-Net. Neurons between adjacent layers are fully connected. For each image stimulus, we first convert each normalized pixel to a homogeneous spike train. For pixel with coordinates ( ) in the image, the corresponding spike train has a constant interspike interval ISI( ) (in ms) which is determined by the pixel value ( ) as shown in Eq. ( ). x, y τ x, y p x, y 1 In our experiment, the simulation for each stimulus lasted 50 ms. All spike trains started at 9 + ISI ms and lasted until the end of the simulation. Then we attached all spike trains to the input layer neurons in a one-to-one manner. The synaptic current triggered by the spike arriving at time is given by τ t0 where is the post-synaptic voltage, the reversal potential syn = 1 mV, the maximum synaptic conductance max = 0.05 μS, and the time constant = 0.5 ms. v E g τ Neurons in the input layer were modeled with a passive single-compartment model. The specific parameters were set as follows: membrane capacitance m = 1.0 μF cm-2, membrane resistance m = 104 Ω cm2, axial resistivity a = 100 Ω cm, reversal potential of passive compartment l = 0 mV. c r r E ປະເພດທີ່ຖືກເຂົ້າລະຫັດໂດຍບໍ່ມີການເຂົ້າລະຫັດໂດຍບໍ່ມີການເຂົ້າລະຫັດໂດຍບໍ່ມີການເຂົ້າລະຫັດໂດຍບໍ່ມີການເຂົ້າລະຫັດໂດຍບໍ່ມີການເຂົ້າລະຫັດໂດຍບໍ່ມີການເຂົ້າລະຫັດໂດຍບໍ່ມີການເຂົ້າລະຫັດໂດຍບໍ່ມີການເຂົ້າລະຫັດໂດຍບໍ່ມີການເຂົ້າເຖິງ. , and all neurons were modeled with passive cables. The specific membrane capacitance m = 1.5 μF cm-2, membrane resistance m = 48,300 Ω cm2, axial resistivity a = 261.97 Ω cm, and the reversal potential of all passive cables l = 0 mV. Input neurons could make multiple connections to randomly-selected locations on the dendrites of hidden neurons. The synaptic current activated by the -th synapse of the -th input neuron ກ່ຽວກັບ neuron ’s dendrite is defined as in Eq. ( ), where is the synaptic conductance, is the synaptic weight, is the ReLU-like somatic activation function, and is the somatic voltage of the -th input neuron at time . 51 c r r E k i j 4 ລະຫັດ QR Wijk i t Neurons in the output layer were also modeled with a passive single-compartment model, and each hidden neuron only made one synaptic connection to each output neuron. All specific parameters were set the same as those of the input neurons. Synaptic currents activated by hidden neurons are also in the form of Eq. ( ). 4 Image classification with HPC-Net For each input image stimulus, we first normalized all pixel values to 0.0-1.0. Then we converted normalized pixels to spike trains and attached them to input neurons. Somatic voltages of the output neurons are used to compute the predicted probability of each class, as shown in equation ວິທີການ is the probability of -th class predicted by the HPC-Net, is the average somatic voltage from 20 ms to 50 ms of the -th output neuron, and indicates the number of classes, which equals the number of output neurons. The class with the maximum predicted probability is the final classification result. In this paper, we built the HPC-Net with 784 input neurons, 64 hidden neurons, and 10 output neurons. 6 pi i i C Synaptic plasticity rules for HPC-Net Inspired by previous work , we use a gradient-based learning rule to train our HPC-Net to perform the image classification task. The loss function we use here is cross-entropy, given in Eq. ( ), where is the predicted probability for class , indicates the actual class the stimulus image belongs to, = 1 if input image belongs to class , and = 0 if not. 36 7 pi i yi yi i yi When training HPC-Net, we compute the update for weight (the synaptic weight of the -th synapse connecting neuron to neuron ) at each time step. After the simulation of each image stimulus, is updated as shown in Eq. ( ): Wijk k i j Wijk 8 Here is the learning rate, is the update value at time , , are somatic voltages of neuron and respectively, is the -th synaptic current activated by neuron ລະຫັດ QR , its synaptic conductance, ມັນເປັນການຕັດສິນໃຈຂອງການຕັດສິນໃຈລະຫວ່າງ ການເຊື່ອມຕໍ່ compartment ຂອງ neuron on neuron ’s dendrite to neuron ’s soma, s = 30 ms e = 50 ms are start time and end time for learning respectively. For output neurons, the error term can be computed as shown in Eq. ( ). For hidden neurons, the error term is calculated from the error terms in the output layer, given in Eq. ( ). t vj vi i j Iijk k i j gijk rijk k i j j t t 10 11 Since all output neurons are single-compartment, equals to the input resistance of the corresponding compartment, . Transfer and input resistances are computed by NEURON. Mini-batch training is a typical method in deep learning for achieving higher prediction accuracy and accelerating convergence. DeepDendrite also supports mini-batch training. When training HPC-Net with mini-batch size batch, we make batch copies of HPC-Net. During training, each copy is fed with a different training sample from the batch. DeepDendrite first computes the weight update for each copy separately. After all copies in the current training batch are done, the average weight update is calculated and weights in all copies are updated by this same amount. N N ຄວາມເຂັ້ມແຂງກັບການຕັດສິນໃຈທີ່ມີ HPC-Net To demonstrate the robustness of HPC-Net, we tested its prediction accuracy on adversarial samples and compared it with an analogous ANN (one with the same 784-64-10 structure and ReLU activation, for fair comparison in our HPC-Net each input neuron only made one synaptic connection to each hidden neuron). We first trained HPC-Net and ANN with the original training set (original clean images). Then we added adversarial noise to the test set and measured their prediction accuracy on the noisy test set. We used the Foolbox , ການຜະລິດຄວາມຮ້ອນ adversarial ໂດຍການ FGSM . ANN was trained with PyTorch , and HPC-Net was trained with our DeepDendrite. For fairness, we generated adversarial noise on a significantly different network model, a 20-layer ResNet . The noise level ranged from 0.02 to 0.2. We experimented on two typical datasets, MNIST and Fashion-MNIST . Results show that the prediction accuracy of HPC-Net is 19% and 16.72% higher than that of the analogous ANN, respectively. 98 99 93 100 101 95 96 Reporting summary ຂໍ້ມູນເພີ່ມເຕີມກ່ຽວກັບການອອກແບບການຄົ້ນຄວ້າແມ່ນມີຢູ່ໃນ linked to this article. Nature Portfolio Reporting Summary Data availability ຂໍ້ມູນທີ່ສະຫນັບສະຫນູນການຊອກຫາຂອງການຄົ້ນຄວ້ານີ້ແມ່ນມີຢູ່ໃນເອກະສານ, ຂໍ້ມູນເພີ່ມເຕີມແລະເອກະສານ Data Source ທີ່ສະຫນອງກັບເອກະສານນີ້. ລະຫັດສະບັບແລະຂໍ້ມູນທີ່ຖືກນໍາໃຊ້ເພື່ອ reproduced ຂໍ້ມູນໃນ Figs. – ມີການນໍາໃຊ້ໃນ . ລະບົບຂໍ້ມູນ MNIST ແມ່ນສາມາດເຂົ້າເຖິງໂດຍທົ່ວໄປໃນ . The Fashion-MNIST dataset is publicly available at ປະເພດ are provided with this paper. 3 6 https://github.com/pkuzyc/DeepDendrite http://yann.lecun.com/exdb/mnist https://github.com/zalandoresearch/fashion-mnist Source data Code availability The source code of DeepDendrite as well as the models and code used to reproduce Figs. – ການຄົ້ນຄວ້ານີ້ແມ່ນມີຢູ່ໃນ . 3 6 https://github.com/pkuzyc/DeepDendrite ລະຫັດ QR McCulloch, W. S. & Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5, 115-133 (1943). LeCun, Y., Bengio, Y. & Hinton, G. ການຝຶກອົບຮົມສູງ. Nature 521, 436–444 (2015). Poirazi, P., Brannon, T. & Mel, B. W. Arithmetic of subthreshold synaptic summing in a model CA1 pyramidal cell. Neuron 37, 977–987 (2003). London, M. & Häusser, M. Dendritic computation. , 503–532 (2005). Annu. Rev. Neurosci. 28 Branco, T. & Häusser, M. The single dendritic branch as a fundamental functional unit in the nervous system. , 494–502 (2010). Curr. Opin. Neurobiol. 20 Stuart, G. J. & Spruston, N. Integration Dendritic: 60 ປີຂອງການພັດທະນາ. Nat. Neurosci. 18, 1713-1721 (2015). Poirazi, P. & Papoutsi, A. Illuminating dendritic function with computational models. , 303–321 (2020). Nat. Rev. Neurosci. 21 Yuste, R. & Denk, W. Dendritic spines as basic functional units of neuronal integration. , 682–684 (1995). Nature 375 Engert, F. & Bonhoeffer, T. Dendritic spine changes associated with hippocampal long-term synaptic plasticity. , 66–70 (1999). Nature 399 Yuste, R. Dendritic spines and distributed circuits. , 772–781 (2011). Neuron 71 Yuste, R. Electrical compartmentalization in dendritic spines. , 429–449 (2013). Annu. Rev. Neurosci. 36 Rall, W. Branching dendritic trees and motoneuron membrane resistivity. , 491–527 (1959). Exp. Neurol. 1 Segev, I. & Rall, W. ການຄົ້ນຄວ້າຄອມພິວເຕີຂອງ spine dendritic excitable. J. Neurophysiol. 60, 499-523 (1988). Silver, D. et al. Mastering the game of go with deep neural networks and tree search. , 484–489 (2016). Nature 529 Silver, D. et al. Algorithm ການຝຶກອົບຮົມການເຊື່ອມໂລຫະທົ່ວໄປທີ່ມັກສູດການເຄື່ອນໄຫວ, shogi, ແລະຂັບລົດໂດຍການດໍາເນີນການ. Science 362, 1140-1144 (2018). McCloskey, M. & Cohen, N. J. ການປິ່ນປົວ catastrophic ໃນເຄືອຂ່າຍ connectivist: ການປິ່ນປົວການຮຽນຮູ້ sequential. Psychol. Learn. Motiv. 24, 109-165 (1989). French, R. M. Catastrophic forgetting in connectionist networks. , 128–135 (1999). Trends Cogn. Sci. 3 Naud, R. & Sprekeler, H. Sparse bursts optimize ການສົ່ງຂໍ້ມູນໃນລະຫັດ neuronal multiplexed. Proc. Natl Acad. Sci. USA 115, E6329-E6338 (2018). Sacramento, J., Costa, R. P., Bengio, Y. & Senn, W. Dendritic cortical microcircuits approximate the backpropagation algorithm. in (NeurIPS*,* 2018). Advances in Neural Information Processing Systems 31 (NeurIPS 2018) Payeur, A., Guerguiev, J., Zenke, F., Richards, B. A. & Naud, R. ການຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢືນຢັ້ງຢືນຢືນຢ Bicknell, B. A. & Häusser, M. A Rule of Synaptic Learning for exploiting nonlinear dendritic computation. Neuron 109, 4001–4017 (2021). Moldwin, T., Kalmenson, M. & Segev, I. The gradient clusteron: a model neuron that learns to solve classification tasks via dendritic nonlinearities, structural plasticity, and gradient descent. , e1009015 (2021). PLoS Comput. Biol. 17 Hodgkin, A. L. & Huxley, A. F. A quantitative description of membrane current and Its application to conduction and excitation in nerve. , 500–544 (1952). J. Physiol. 117 Rall, W. Theory of physiological properties of dendrites. , 1071–1092 (1962). Ann. N. Y. Acad. Sci. 96 Hines, M. L. & Carnevale, N. T. The NEURON simulation environment. , 1179–1209 (1997). Neural Comput. 9 Bower, J. M. & Beeman, D. in (eds Bower, J.M. & Beeman, D.) 17–27 (Springer New York, 1998). The Book of GENESIS: Exploring Realistic Neural Models with the GEneral NEural SImulation System Hines, M. L., Eichner, H. & Schürmann, F. Neuron splitting in compute-bound parallel network simulations enables runtime scaling with twice as many processors. , 203–210 (2008). J. Comput. Neurosci. 25 Hines, M. L., Markram, H. & Schürmann, F. ການ simulation ອັດຕະໂນມັດທີ່ແຕກຕ່າງກັນທັງຫມົດຂອງ neurons ຫນຸ່ມ. J. Comput. Neurosci. 25, 439-448 (2008). Ben-Shalom, R., Liberman, G. & Korngreen, A. Accelerating compartmental modeling on a graphical processing unit. , 4 (2013). Front. Neuroinform. 7 Tsuyuki, T., Yamamoto, Y. & Yamazaki, T. Efficient numerical simulation of neuron models with spatial structure on graphics processing units. In (eds Hirose894Akiraet al.) 279–285 (Springer International Publishing, 2016). Proc. 2016 International Conference on Neural Information Processing Vooturi, D. T., Kothapalli, K. & Bhalla, U.S. Parallelizing Hines Matrix Solver in Neuron Simulations on GPU. In Proc. IEEE 24th International Conference on High Performance Computing (HiPC) 388-397 (IEEE, 2017). Huber, F. Efficient tree solver for hines matrices on the GPU. Preprint at (2018). https://arxiv.org/abs/1810.12742 Korte, B. & Vygen, J. ວິທະຍາສາດການປັບປຸງການເຊື່ອມໂລຫະແລະ algoritms 6 edn (Springer, 2018). Gebali, F. (Wiley, 2011). Algorithms and Parallel Computing Kumbhar, P. et al. CoreNEURON: An optimized compute engine for the NEURON simulator. , 63 (2019). Front. Neuroinform. 13 Urbanczik, R. & Senn, W. ການຝຶກອົບຮົມໂດຍການຄາດຄະເນດິນ dendritic ຂອງ spiking somatic. Neuron 81, 521-528 (2014). Ben-Shalom, R., Aviv, A., Razon, B. & Korngreen, A. Optimizing ໂມເລກຸນຂອງແສງສະຕິກ ion ທີ່ໃຊ້ algoritm ອັດຕະໂນມັດ genetic on graphical processors. J. Neurosci. Methods 206, 183-194 (2012). Mascagni, M. A parallelizing algorithm for computing solutions to arbitrarily branched cable neuron models. , 105–114 (1991). J. Neurosci. Methods 36 McDougal, R. A. et al. 20 ປີຂອງ modelDB ແລະອື່ນໆ: ການສ້າງອຸປະກອນການອອກແບບທີ່ສໍາຄັນສໍາລັບພະລັງງານຂອງ neuroscience. J. Comput. Neurosci. 42, 1-10 (2017). Migliore, M., Messineo, L. & Ferrante, M. Dendritic Ih ການກວດສອບການກວດສອບທີ່ໃຊ້ເວລາຂອງ input distal unsynchronized ໃນ neurons pyramidal CA1. J. Comput. Neurosci. 16, 5–13 (2004). Hemond, P. et al. Distinct classes of pyramidal cells exhibit mutually exclusive firing patterns in hippocampal area CA3b. , 411–424 (2008). Hippocampus 18 Hay, E., Hill, S., Schürmann, F., Markram, H. & Segev, I. ໂມເລກຸນຂອງຊັ້ນ neocortical 5b cell pyramidal capturing a wide range of dendritic and perisomatic active Properties. PLoS Comput. Biol. 7, e1002107 (2011). Masoli, S., Solinas, S. & D’Angelo, E. Action potential processing in a detailed purkinje cell model reveals a critical role for axonal compartmentalization. , 47 (2015). Front. Cell. Neurosci. 9 Lindroos, R. et al. Basal ganglia neuromodulation over multiple temporal and structural scales—simulations of direct pathway MSNs investigate the fast onset of dopaminergic effects and predict the role of Kv4.2. , 3 (2018). Front. Neural Circuits 12 Migliore, M. et al. clusters synaptic ເຮັດວຽກເປັນຜູ້ປະຕິບັດຄວາມຮ້ອນໃນ bulb olfactory. Proc. Natl Acad. Sci. USa 112, 8499-8504 (2015). NVIDIA. . (2021). CUDA C++ Programming Guide https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html NVIDIA. . (2021). CUDA C++ Best Practices Guide https://docs.nvidia.com/cuda/cuda-c-best-practices-guide/index.html Harnett, M. T., Makara, J. K., Spruston, N., Kath, W. L. & Magee, J. C. Synaptic amplification by dendritic spines enhances input cooperativity. , 599–602 (2012). Nature 491 Chiu, C. Q. et al. Compartmentalization of GABAergic inhibition by dendritic spines. , 759–762 (2013). Science 340 Tønnesen, J., Katona, G., Rózsa, B. & Nägerl, U. V. Spine neck plasticity regulates compartmentalization of synapses. , 678–685 (2014). Nat. Neurosci. 17 Eyal, G. et al. Human cortical pyramidal neurons: from spines to spikes via models. , 181 (2018). Front. Cell. Neurosci. 12 Koch, C. & Zador, A. The function of dendritic spines: devices subserving biochemical rather than electrical compartmentalization. , 413–422 (1993). J. Neurosci. 13 Koch, C. Dendritic spines. In (Oxford University Press, 1999). Biophysics of Computation Rapp, M., Yarom, Y. & Segev, I. The impact of parallel fiber background activity on the cable properties of cerebellar purkinje cells. , 518–533 (1992). Neural Comput. 4 Hines, M. ການຄອມພິວເຕີປະສິດທິພາບຂອງການຄອມພິວເຕີໄຮໂດຼລິກ. Int. J. Bio-Med. Comput. 15, 69-76 (1984). Nayebi, A. & Ganguli, S. Biologically inspired protection of deep networks from adversarial attacks. Preprint at (2017). https://arxiv.org/abs/1703.09202 Goddard, N. H. & Hood, G. Large-Scale Simulation Using Parallel GENESIS. In (eds Bower James M. & Beeman David) 349-379 (Springer New York, 1998). The Book of GENESIS: Exploring Realistic Neural Models with the GEneral NEural SImulation System Migliore, M., Cannia, C., Lytton, W. W., Markram, H. & Hines, M. L. simulations networks parallel with NEURON. J. Comput. Neurosci. 21, 119 (2006). Lytton, W. W. et al. Simulation neurotechnologies for advancing brain research: parallelizing large networks in NEURON. Neural Comput. 28, 2063-2090 (2016). Valero-Lara, P. et al. cuHinesBatch: Solving multiple Hines systems on GPUs human brain project. In Proc. 2017 International Conference on Computational Science 566-575 (IEEE, 2017). Akar, N. A. et al. Arbor — ຄາສິໂນຮູບແບບເຄືອຂ່າຍ neuron ທີ່ຖືກອອກແບບຢ່າງກວ້າງຂວາງສໍາລັບອຸດສາຫະກໍາຄອມພິວເຕີປະສິດທິພາບສູງມື້. In Proc. 27th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP) 274–282 (IEEE, 2019). Ben-Shalom, R. et al. NeuroGPU: Accelerating multi-compartment, biophysically detailed neuron simulations on GPUs. , 109400 (2022). J. Neurosci. Methods 366 Rempe, M. J. & Chopp, D. L. A predictor-corrector algorithm for reaction-diffusion equations associated with neural activity on branched structures. , 2139–2161 (2006). SIAM J. Sci. Comput. 28 Kozloski, J. & Wagner, J. An ultrascalable solution to large-scale neural tissue simulation. , 15 (2011). Front. Neuroinform. 5 Jayant, K. et al. ການກວດສອບຄວາມກົດດັນ intracellular ຈາກ spines dendritic ໂດຍໃຊ້ nanopipettes ແຜ່ນ quantum. Nat. Nanotechnol. 12, 335-342 (2017). Palmer, L. M. & Stuart, G. J. Membrane potential changes in dendritic spines during action potentials and synaptic input. , 6897–6903 (2009). J. Neurosci. 29 Nishiyama, J. ແລະ Yasuda, R. ການຄຸ້ມຄອງ biochemical ສໍາລັບ plasticity ລະບົບໄຮໂດຼລິກ. Neuron 87, 63–75 (2015). Yuste, R. & Bonhoeffer, T. Morphological changes in dendritic spines associated with long-term synaptic plasticity. , 1071–1089 (2001). Annu. Rev. Neurosci. 24 Holtmaat, A. & Svoboda, K. ການຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢັ້ງຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງຢືນຢືນຢືນຢັ້ງຢືນຢືນຢັ້ງ Caroni, P., Donato, F. & Muller, D. Structural plasticity upon learning: regulation and functions. , 478–490 (2012). Nat. Rev. Neurosci. 13 Keck, T. et al. Massive restructuring of neuronal circuits during functional reorganization of adult visual cortex. , 1162 (2008). Nat. Neurosci. 11 Hofer, S. B., Mrsic-Flogel, T. D., Bonhoeffer, T. & Hübener, M. Experience leaves a lasting structural trace in cortical circuits. , 313–317 (2009). Nature 457 Trachtenberg, J. T. et al. Imaging in vivo ໃນໄລຍະຍາວຂອງ plasticity synaptic ທີ່ກ່ຽວຂ້ອງກັບປະສົບການໃນ cortex adults. Nature 420, 788-794 (2002). Marik, S. A., Yamahachi, H., McManus, J. N., Szabo, G. & Gilbert, C. D. Axonal dynamics of excitatory and inhibitory neurons in somatosensory cortex. , e1000395 (2010). PLoS Biol. 8 Xu, T. et al. Rapid formation and selective stabilization of synapses for enduring motor memories. , 915–919 (2009). Nature 462 Albarran, E., Raissi, A., Jáidar, O., Shatz, C. J. & Ding, J. B. Enhancing motor learning by increasing the stability of newly formed dendritic spines in the motor cortex. , 3298–3311 (2021). Neuron 109 Branco, T. & Häusser, M. Synaptic integration gradients in single cortical pyramidal cell dendrites. , 885–892 (2011). Neuron 69 Major, G., Larkum, M. E. & Schiller, J. Active properties of neocortical pyramidal neuron dendrites. , 1–24 (2013). Annu. Rev. Neurosci. 36 Gidon, A. et al. Dendritic action potentials and computation in human layer 2/3 cortical neurons. , 83–87 (2020). Science 367 Doron, M., Chindemi, G., Muller, E., Markram, H. & Segev, I. Timed synaptic inhibition shapes NMDA spikes, influencing local dendritic processing and global I/O properties of cortical neurons. , 1550–1561 (2017). Cell Rep. 21 Du, K. et al. Cell-type-specific inhibition of the dendritic plateau potential in striatal spiny projection neurons. , E7612–E7621 (2017). Proc. Natl Acad. Sci. USA 114 Smith, S. L., Smith, I. T., Branco, T. & Häusser, M. Dendritic spikes enhance stimulus selectivity in cortical neurons in vivo. , 115–120 (2013). Nature 503 Xu, N.-l et al. Nonlinear dendritic integration of sensory and motor input during an active sensing task. , 247–251 (2012). Nature 492 Takahashi, N., Oertner, T. G., Hegemann, P. & Larkum, M. E. Active cortical dendrites modulate perception. , 1587–1590 (2016). Science 354 Sheffield, M. E. & Dombeck, D. A. ການ prevalence transitory ຂອງ calcium ໃນໄລຍະ dendritic arbour predicts ຄຸນນະພາບພື້ນຖານ. Nature 517, 200–204 (2015). Markram, H. et al. Reconstruction and simulation of neocortical microcircuitry. , 456–492 (2015). Cell 163 Billeh, Y. N. et al. Systematic integration of structural and functional data into multi-scale models of mouse primary visual cortex. , 388–403 (2020). Neuron 106 Hjorth, J. et al. The microcircuits of striatum in silico. , 202000671 (2020). Proc. Natl Acad. Sci. USA 117 Guerguiev, J., Lillicrap, T. P. & Richards, B. A. Towards deep learning with segregated dendrites. , e22901 (2017). elife 6 Iyer, A. et al. Avoiding catastrophe: active dendrites enable multi-task learning in dynamic environments. , 846219 (2022). Front. Neurorobot. 16 Jones, I. S. & Kording, K. P. ສາມາດທົດສອບຄວາມປອດໄພຂອງ neuron ເປັນຫນຶ່ງໂດຍຜ່ານການຄອມພິວເຕີທີ່ຜ່ານມາໃນປັດຈຸບັນ? Neural Comput. 33, 1554-1571 (2021). Bird, A. D., Jedlicka, P. & Cuntz, H. Dendritic normalisation improves learning in sparsely connected artificial neural networks. , e1009202 (2021). PLoS Comput. Biol. 17 Goodfellow, I. J., Shlens, J. & Szegedy, C. Explaining and harnessing adversarial examples. In (ICLR, 2015). 3rd International Conference on Learning Representations (ICLR) Papernot, N., McDaniel, P. & Goodfellow, I. ການສົ່ງເສີມໃນການຝຶກອົບຮົມເຄື່ອງ: ຈາກການປະຫວັດສາດເພື່ອການປິ່ນປົວການປິ່ນປົວການປິ່ນປົວໂດຍໃຊ້ sample adversarial. Preprint at https://arxiv.org/abs/1605.07277 (2016). Lecun, Y., Bottou, L., Bengio, Y. & Haffner, P. Gradient-based learning applied to document recognition. , 2278–2324 (1998). Proc. IEEE 86 Xiao, H., Rasul, K. & Vollgraf, R. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. Preprint at (2017). http://arxiv.org/abs/1708.07747 Bartunov, S. et al. assessing the scalability of biologically-motivated deep learning algorithms and architectures. In Advances in Neural Information Processing Systems 31 (NeurIPS 2018) (NeurIPS, 2018). Rauber, J., Brendel, W. & Bethge, M. Foolbox: A Python toolbox to benchmark the robustness of machine learning models. In Reliable Machine Learning in the Wild Workshop, 34th International Conference on Machine Learning (2017). Rauber, J., Zimmermann, R., Bethge, M. & Brendel, W. Foolbox native: fast adversarial attacks to benchmark the robustness of machine learning models in PyTorch, TensorFlow, and JAX. , 2607 (2020). J. Open Source Softw. 5 Paszke, A. et al. PyTorch: ປະເພດ imperative, ກ້ອງຖ່າຍຮູບການຮຽນຮູ້ສູງ. In Advances in Neural Information Processing Systems 32 (NeurIPS 2019) (NeurIPS, 2019). He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In 770–778 (IEEE, 2016). Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Acknowledgements ວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາໄລວິທະຍາ ບົດຄວາມນີ້ແມ່ນມີຢູ່ໃນອຸນຫະພູມໂດຍໃບອະນຸຍາດ CC by 4.0 Deed (Attribution 4.0 International). ເອກະສານນີ້ແມ່ນ under CC by 4.0 Deed (Attribution 4.0 International) license. ສະຫນັບສະຫນູນໂດຍ Nature
