Ababhali:   Jeho Lee   Chanyoung Jung   Jiwon Kim   Hojung Cha   Ithebula lezixhumanisi   ABSTRACT   1 ISINGENISO   2 IMVELAPHI: UKUTHOLA INTO YE-3D YE-ONNIDIRECTIONAL   3 UKULIngwa OKUQALA  3.1 Ukusethwa Kokulinga  3.2 Okubhekiwe  3.3 Isifinyezo kanye Nezinselele   4 UKUBUYEKEZWA KWE-PANOPTICUS   5 UKUTHOLWA KWENTO YE-3D YE-3D AMAGASHA AMANINGI  5.1 Idizayini Yemodeli   6 UKWENZIWA KWESIKHALA OKUFANELEKILE  6.1 Isibikezelo Sokusebenza  5.2 Ukuzivumelanisa nezimo  6.2 Ukuhlelwa Kokwenziwa   7 UKUSEBENZA   8 UKUHLOLA  8.1 I-Testbed ne-Dataset  8.2 Ukusethwa Kokulinga  8.3 Ukusebenza  8.4 Ukuqina  8.5 Ukuhlaziya Izingxenye  8.6 Isiphetho   9 UMSEBENZI OHLOBANE   10 INGXOXO NOMSEBENZI OZAYO   11 ISIPHETHO KANYE NEZINKOMBOLO   ABSTRACT  Ukutholwa kwento ye-3D ngokubuka kwendawo yonke kuvumela izinhlelo zokusebenza ezibalulekile zokuphepha ezifana nokuzulazula kwerobhothi leselula. Izinhlelo zokusebenza ezinjalo ziya ngokuya zisebenza kumadivayisi aphethwe yinsiza, enze kube lula ukucutshungulwa okuthembekile ngaphandle kokukhathazeka ngobumfihlo noma ukubambezeleka kwenethiwekhi. Ukuze kunikezwe amandla ukuthunyelwa okungabizi kakhulu, amakhamera amukelwe kabanzi njengenye indlela ebiza kancane ezinzwa ze-LiDAR. Kodwa-ke, umsebenzi onzima wokubala ukuze kuzuzwe ukusebenza okuphezulu kwezisombululo ezisekelwe kukhamera kusalokhu kuyinselele ngenxa yemikhawulo yokubala yamadivaysi asemaphethelweni. Kuleli phepha, sethula i-Panopticus, isistimu eklanywe ngokucophelela yokutholwa kwe-3D okusekelwe ku-omnidirectional kanye nekhamera esekelwe kukhamera kumadivayisi asemaphethelweni. I-Panopticus isebenzisa uhlelo lokuthola amagatsha amaningi oluguquguqukayo olubangela ubunzima bendawo. Ukuze kuthuthukiswe ukunemba ngaphakathi kwemikhawulo yokubambezeleka, i-Panopticus ilungisa ngokuguqukayo ukwakheka kwemodeli nokusebenza ngokusekelwe ezisetshenziswa zonqenqema ezitholakalayo nezici zendawo. Senze i-Panopticus emishinini emithathu enqenqemeni futhi senza izivivinyo ezindaweni zomhlaba wangempela ngokusekelwe kudathasethi yomphakathi yokuzishayela kanye nedathasethi yethu yedatha yekhamera engu-360°. Imiphumela yokuhlolwa ibonise ukuthi i-Panopticus ithuthukisa ukunemba ngo-62% ngokwesilinganiso uma kubhekwa umgomo wokubambezeleka oqinile ongu-33ms. Futhi, i-Panopticus izuza ukuncishiswa kokubambezeleka okungu-2.1× ngokwesilinganiso uma kuqhathaniswa nezisekelo.   1 ISINGENISO  Ngokuhambisana nenqubekelaphambili ekubonweni kwekhompyutha kanye namanethiwekhi ajulile e-neural (DNNs), ukutholwa kwento ye-3D sekuyingxenye eyinhloko yezinhlelo zokusebenza eziningi. Isibonelo, izimoto ezizimele zithembele ekuboneni okunembayo nesikhathi sangempela kwezinto endaweni ukuze kusungulwe imizila ephephile yokuzulazula [55]. Njengoba izinto zingasondela kunoma iyiphi indlela, njengoba kukhonjisiwe kuMfanekiso 1, kubalulekile ukuqinisekisa umbono ngokusebenzisa inkambu yokubuka ebanzi engu-360° (FOV). Umbono onjalo we-omnidirectional udinga ukucutshungulwa kwenani elikhulu ledatha yezinzwa futhi udinga amadivaysi ekhompyutha asezingeni eliphezulu anezisheshisi ze-AI ukuze acutshungulwe ngesikhathi sangempela [47]. Muva nje, isidingo sezinhlelo zokusebenza zeselula ezisebenzisa ukutholwa kwento ye-3D ye-omnidirectional sesisabalele. Amarobhothi noma ama-drones ahlinzeka ngezinsizakalo zomuntu siqu njengokugada angazuza kulobo buchwepheshe [16]. Ngaphezu kwalokho, ukuthola izithiyo ezizungezile nokunikeza izixwayiso ezizwakalayo zezingozi ezingase zibe khona kungasiza abantu abanokukhubazeka kokubona [39, 56]. Lezi zinhlelo zokusebenza eziqondene nawe kufanele zicutshungulwe kudivayisi esemaphethelweni ukuze kuncishiswe izinkinga zobumfihlo zabasebenzisi noma inethiwekhi. Kodwa-ke, ngisho nochungechunge lwakamuva lwe-NVIDIA Jetson Orin [8], olunikeza amandla ekhompiyutha aphambili, lunama-Tensor cores ambalwa angu-6.7 kuya ku-13.5× ambalwa wokusheshisa i-AI uma kuqhathaniswa ne-A100 [9] enamandla esetshenziselwa ukusebenzisa ikhompuyutha yamafu, enokwakheka okufanayo okuyisisekelo kwe-GPU. Ngaphezu kwalokho, izinhlelo zokusebenza ze-AI ezinqenqemeni kufanele zicabangele izici ezisebenzayo ezifana nokuthunyelwa ngendlela eyongayo. Ngenxa yalokho, kuye kwenziwa umzamo omkhulu wokusekela izinhlelo ezinjalo ngamakhamera angabizi kakhulu [1, 38, 42, 58]. Ngokukhethekile, amakhamera amaningi noma ikhamera yeselula engu-360° isetshenziswa ukuze kube lula ukubona indawo yonke.  Izinsizakalo ze-Edge AI zinenqwaba yokunemba kanye nezidingo ze-latency. Naphezu kwentuthuko yakamuva, imisebenzi yangaphambili inemikhawulo ekusekeleni kokubili ukusebenza kahle nokunemba kumadivayisi onqenqema akhawulelwe yinsiza. I-DeepMix [18] ilayishe imisebenzi eyinkimbinkimbi yokutholwa kwento esekwe ku-DNN kuseva yefu ukuze kwehliswe umthwalo wekhompyutha kudivayisi esemaphethelweni. Ukulayisha imisebenzi yokubona yonke indawo, nokho, kungase kubangele ukubambezeleka kokuxhumana kwamafu ngenxa yokudluliswa kwedatha okukhulu. I-PointSplit [37] isekela ukusebenza okuhambisanayo kunqenqema lwe-GPU ne-NPU, kodwa uhlelo luthuthukiselwe ipayipi elithile lokuthola i-3D kusetshenziswa inzwa ye-RGB-D ene-FOV elinganiselwe. Khonamanjalo, izindlela ezihlukahlukene [1, 31, 34, 38] ziye zathuthukisa ukunemba kwezixazululo ezisekelwe kukhamera, ezibeka ubunzima obungokwemvelo ngenxa yokungabikho kolwazi olujulile lwe-3D. Umugqa wemisebenzi [29, 30, 52] ugxile ekuthuthukiseni ama-DNN ukuze kuthuthukiswe ukubikezela okujulile okuvela ezithombeni ze-RGB. Futhi, ukwamukelwa kwama-DNN amakhulu, njengama-backbones okukhishwa kwesici kusetshenziswa izithombe zokulungiswa okuphezulu, kubalulekile ekuthuthukisweni kokunemba [51]. Kodwa-ke, ukucubungula imisebenzi ye-DNN yekhompuyutha eminingi ngokufakwa kwe-omnidirectional kubeka izimfuno ezinkulu zekhompiyutha kumadivayisi aphethwe yizinsiza.  Kuleli phepha, siphakamisa i-Panopticus, isistimu ekhulisa ukunemba kokutholwa kwento ye-3D yonke indawo lapho ihlangabezana nezimfuneko zokubambezeleka kumadivayisi asemaphethelweni. Siqaphele ngaphambilini ukuthi izitholi ze-3D ezisuselwa kukhamera zinamandla okuthola ahlukahlukene kuye ngezici zendawo, ezinqunywa izici ezihlukahlukene njengenombolo noma ukunyakaza kwezinto. Umbono oyinhloko we-Panopticus uwukucubungula ukubuka kwekhamera ngayinye ngokufanele ngokusekelwe ekuqondeni okuguquguqukayo kwesikhashana ekusabalaliseni kwendawo. Isibonelo, ukubuka kwekhamera okuqukethe izinto ezimbalwa ezimile nezisondele kungacutshungulwa ngokucushwa kwe-inference engasindi ukuze kuncishiswe ukubambezeleka ngokulahleka kokunemba okuncane. Imajini yokubambezeleka egciniwe ingase isetshenziselwe ukunikeza ukucushwa kwe-inference okusebenza kahle kakhulu ekubukeni okuyinkimbinkimbi lapho izinto zihamba ngokushesha noma endaweni ekude, njengoba kuboniswe kuMfanekiso 1.  Kunezinselelo ezimbalwa ezikhona ekwakhiweni kwePanopticus. Okokuqala, amamodeli angaphambili okuthola i-3D ayehluleka ukuhlinzeka ngohlelo lweziqondiso olusebenzayo noluguquguqukayo olukwazi ukuhlukanisa ukulungiselelwa kokucatshangelwayo kokubuka kwekhamera ngayinye kuhlaka lwevidiyo olufanayo, njengomthamo womgogodla noma ukusetshenziswa kokulinganisela kokujula okuthuthukisiwe. Ukwengeza, i-architecture yemodeli kufanele ilungiswe ukuze ivumelane nezingqinamba ezihlukahlukene, njengezidingo zokubambezeleka, kudivayisi ethile. Okwesibili, ukuze kukhuliswe ukunemba ngaphakathi kwezidingo zokubambezeleka, ukucushwa kwe-inference okuphelele kufanele kunqunywe ngokubuka kwekhamera ngayinye. Lokhu kudinga ukuhlaziywa kwesikhathi sokusebenza kwakho kokubili izinguquko ekusabalaliseni kwendawo kanye nokusebenza okulindelekile kokucushwa kwencazelo.   Ukuze unike amandla ukulungiswa kwezakhiwo nokusebenza kwemodeli, sethula imodeli yokutholwa kwento ye-3D ye-omnidirectional enamagatsha amaningi okucabanga. Imodeli icubungula ukubuka ngakunye isebenzisa elinye lamagatsha elinamandla okuthola ahlukahlukene, okuvumela ukusetshenziswa kahle kwezisetshenziswa zekhompuyutha ezinqenqemeni. Isakhiwo semodeli sidizayinelwe ukuthi sibe yimodular, sivumele ukusetshenziswa okuvumelana nezimo ngokususa igatsha elephula imingcele enikeziwe. Ngenselelo yesibili yokukhulisa ukunemba ngaphakathi kwemikhawulo yokubambezeleka, sethula isikimu sokwenza esivumelana nezimo. Ngesikhathi sokusebenza, uhlelo lubikezela ukusebenza kwegatsha ngalinye ngokusekelwe ekusabalaliseni kwendawo okulindelekile kwezinto ezizungezile. Inhlanganisela efanele yamagatsha nokubukwa kwekhamera, okukhulisa ukunemba okuphelele okulinganiselwe ngenkathi ihlangabezana nomgomo wokubambezeleka, bese ikhethwa ukuze kucatshangelwe. Senze i-Panopticus kumadivayisi asemaphethelweni amathathu anamakhono ahlukene okubala. Uhlelo luye lwahlolwa ezindaweni ezihlukahlukene zomhlaba wangempela, njengemigwaqo yasemadolobheni nemigwaqo, kusetshenziswa idathasethi yomphakathi yokushayela okuzenzakalelayo kanye nekhamera yethu yangokwezifiso engu-360° yekhamera. Ukuhlola okubanzi kubonise ukuthi i-Panopticus isebenze kahle kakhulu kunezisekelo zayo ngaphansi kwezimo ezihlukahlukene ngokokubili ukunemba nokusebenza kahle.  Amagalelo abalulekile omsebenzi wethu ami kanje: •  Ngokolwazi lwethu olungcono kakhulu, i-Panopticus iwuhlelo lokuqala lokutholwa kwezinto ze-3D olunendawo yonke kanye nekhamera oluthola kokubili ukunemba nokubambezeleka kumadivayisi aphethwe yinsiza.  • Senze ucwaningo olujulile ukuze sihlole amakhono ahlukahlukene wakamuva we-3D athonywe izici ezihlukahlukene zezinto nezikhala. I-Panopticus ihlinzeka ngokulawula okunembile kokubona indawo yonke kanye nokusetshenziswa kwensiza enqenqemeni, evumelana nezimo ezihlukahlukene zendawo ezindaweni eziguquguqukayo.  • Sisebenzise ngokugcwele i-Panopticus njengesistimu yekhompuyutha ephuma ekupheleni isebenzisa kokubili ukuzishayela komphakathi.  isethi yedatha kanye ne-testbed yethu yekhamera yeselula engu-360°, ebonisa ukuvumelana kwayo nezingqinamba zensiza zamadivayisi asemaphethelweni ahlukahlukene ezimo zomhlaba wangempela.  Leli phepha  ngaphansi kwe-CC ngelayisensi ye-4.0 Deed (Attribution 4.0 International).   itholakala ku-arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Omnidirectional Technology

omnidirectional's blog

Lo msindo ukhiqizwa ngolimi lokuqala lwendaba!

Le modeli ye-AI Inikeza Amadivayisi E-Edge Amehlo Ngemuva Kwekhanda Lawo

About Author

IMIBONO

HANG TAGS

LESI SIHLOKO SETHULWE NGAPHAKATHI

Related Stories

The Billionaires F*cked Up

Zulu Network: Moving The Bitcoin Economy Forward With a Two-Tiered Bitcoin Layer 2 Architecture

The Billionaires F*cked Up

Zulu Network: Moving The Bitcoin Economy Forward With a Two-Tiered Bitcoin Layer 2 Architecture

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps