Xa i-OpenAI yazisa i-ChatGPT ngasekupheleni kuka-2022, yabangela ulonwabo kunye nenkxalabo. I-Generative AI ibonise amandla amangalisayo-ukwenza izincoko, ukusombulula iingxaki zekhowudi, kunye nokudala ubugcisa. Kodwa iphinde yaphakamisa ii-alam phakathi kweengcali zokusingqongileyo, abaphandi, kunye neengcali zetekhnoloji. Eyona nto ixhalabisayo? Usetyenziso olukhulu lwamandla olufunekayo ekuqeqesheni nasekuqhubeni iiModeli zoLwimi olukhulu (LLMs), oko kubuza imibuzo malunga nokuzinza kwazo kwexesha elide. Njengoko ii-LLM ziqhubeka nokuseka amashishini afana nezemfundo kunye nokhathalelo lwempilo, impembelelo yazo ayinakungahoywa. Eli phepha liphakamisa umbuzo obalulekileyo: Ngaba ezi nkqubo zikrelekrele ziyakwazi ukuziphucula ngokwazo ukunciphisa ukusetyenziswa kwamandla kunye nokunciphisa unyawo lwazo lokusingqongileyo? Kwaye ukuba kunjalo, oku kungawuguqula njani ubume be-AI? Siza kucazulula imiceli mngeni yamandla e-LLMs, ukusuka kuqeqesho ukuya kutsho, kwaye siphonononge iindlela ezintsha zokuzilungisa ezinokwenza i-AI izinze ngakumbi. Ukuqonda i-AI Energy Challenge Uqeqesho vs Uqeqesho lukaGoogle kwiimodeli zeelwimi ezinkulu ezifana neGPT-4 okanye iPaLM ifuna isixa esikhulu sezixhobo zokubala. Umzekelo, uqeqesho lwe-GPT-3 luthathe amawaka ee-GPU ezibaleka iiveki, zisebenzisa amandla amaninzi njengamakhulu emizi yase-US ngonyaka. I-carbon footprint ixhomekeke kumxube wamandla wokunika amandla amaziko edatha. Nasemva koqeqesho, inqanaba lokujonga-apho iimodeli zibamba imisebenzi yehlabathi yokwenyani-yongeza ekusebenziseni amandla. Nangona amandla afunekayo kumbuzo omnye emancinci, xa sicinga ukuba kukho iibhiliyoni zonxibelelwano olunjalo ezenzeka kumaqonga ahlukeneyo yonke imihla, iba yingxaki enkulu. Kutheni iiLLMs Zisebenzisa Amandla Kangaka? IiLLM zanamhlanje zinovakalelo lweparamitha; banamabhiliyoni okanye iitriliyoni zeeparamitha ezifuna ubutyebi obuninzi ukuze ziqwalaselwe, zihlaziywe kwaye zigcinwe. Ubungakanani boMzekelo: Ukusetyenziswa kweetshiphusi ezisekwe kwisilicon kukhawulelwe ngamandla azo okusebenza kwaye ke imfuno yamaqela e-GPUs okanye ii-TPUs ukwandisa ukusetyenziswa kwamandla ngokubonakalayo. Izithintelo zeHardware: Amaziko eenkcukacha axhasa imithwalo yemisebenzi ephezulu yokubala afudumele kwaye iinkqubo zokupholisa zinokutya ukuya kuthi ga kwi-40% yamandla ukuba ayenzi amandla. Iimfuno zokupholisa: Umrhumo wokusiNgqongileyo noQoqosho Iindleko ngokwemeko yokusingqongileyo ziquka ukukhutshwa kwekhabhoni ngokunjalo nokusetyenziswa kwamanzi ekubaleni ngelixa iindleko zokusebenza ziyingxaki kwiinkampani ezincinci ze-AI. Iindleko zonyaka zinokufikelela kwiibhiliyoni, nto leyo eyenza ukuzinza kube yinto ebalulekileyo kungekuphela nje kokusingqongileyo kodwa nomba wezoqoqosho. Imodeli ye-AI yokuSetyenziswa kwamandla Ukuqonda indlela ii-LLMs eziwasebenzisa ngayo amandla, masiyicazulule: Ukusebenza kwe-AI Ukusetyenziswa kwamandla (%) Isigaba soQeqesho 60% Ukungeniswa (Imibuzo Eqhutywayo) 25% Ukupholisa iziko leDatha 10% Ukusebenza kweHardware 5% Isigaba soqeqesho sihlala sinegalelo elikhulu ekusebenziseni amandla. I-Key Takeaway: Ubuchule bokuZiphuhlisa Abaphandi bajonge indlela ii-LLMs ezinokwandisa ngayo ukusetyenziswa kwamandla abo, ukudibanisa umsebenzi wesoftware kunye notshintsho lwehardware. Umzekelo wokuPruna kunye nobungakanani Iiparamitha ezingafunekiyo ezichaphazela ukuchaneka kwinqanaba elilinganiselweyo ziyasuswa, okukhokelela ekunciphiseni ubungakanani bemodeli ngaphandle kokuphazamisa ukuchaneka. Ukuthena: Oku kunciphisa ukuchaneka (umzekelo, ukusuka kwi-32-bit ukuya kwi-8-bit) yedatha, enciphisa imemori kunye neemfuno zokubala. Ubungakanani: Ubungakanani kunye nokuPruning ziluncedo kodwa xa zisetyenziswa kunye ne-feedback looops apho imodeli ikwaziyo ukumisela ukuba ngawaphi amalungu abalulekileyo kwaye ngawaphi amalungu anokwahlulwa ngokomlinganiselo emva koko iyasebenza kakhulu. Le yindawo entsha, kodwa amandla akhona kuthungelwano oluzenzelayo. iNgqungquthela enamandla (Conditional Computation) Ingcamango yokubala ngokwemiqathango yenza ukuba iimodeli zisebenzise kuphela ezo neurons okanye iileya ezihambelana nomsebenzi onikiweyo. Umzekelo, indlela kaGoogle yoMxube-weNgcali (i-MoE) yahlula inethiwekhi ibe yi-subnetworks ezikhethekileyo eziphucula uqeqesho kunye nokunciphisa ukusetyenziswa kwamandla ngokunciphisa inani leeparamitha ezisebenzayo. Ukomelezwa kokuFunda kuThungelwano Ukufunda okomeleza kunokwandisa ii-hyperparameters ezifana nezinga lokufunda kunye nobukhulu bebhetshi, ukuchaneka kokulinganisa kunye nokusetyenziswa kwamandla ukuqinisekisa ukuba iimodeli zisebenza ngokufanelekileyo. UPhuculo lweNjongo ezininzi Ukongeza ekulungiseleleni ukuchaneka, ii-LLMs zinokuphucula ezinye iinjongo: ukuchaneka, ukulinda, kunye nokusetyenziswa kwamandla, usebenzisa izixhobo ezifana neGoogle Vizier okanye iRay Tune. Kutsha nje, ukusebenza kakuhle kwamandla kuye kwaba yinjongo ebalulekileyo kwezi zikhokelo. Izinto ezintsha zeHardware kunye ne-AI Co-Design Iichips ezikhethekileyo zenjongo yokuphucula ukusebenza kakuhle ekuqhutyweni kwemisebenzi ye-AI. IiSekethe eziDityanisiweyo eziKhethekileyo zesicelo (ii-ASICs): Iichips eziphefumlelweyo zobuchopho, ezisekuphuhlisweni ukunciphisa ukusetyenziswa kwamandla xa kusenziwa i-neural network computations ziphantsi kophuhliso. I-Neuromorphic Computing: Ukubala usebenzisa ukukhanya kunokoyisa imida yenkqubo ye-elektroniki ukuthoba usetyenziso lwamandla enkqubo. IKhompyutha ye-Optical: Iinkqubo ze-AI ezidalwe ngokubambisana kwe-hardware kunye nesofthiwe zivumela ukulungiswa kwangaxeshanye kwe-algorithms ye-software kunye nezixhobo ze-hardware. Ukuthelekisa iiTekhnoloji zokuSebenzisa amandla e-AI Ubuchwephesha Ukunciphisa amandla (%) INzuzo yokuqala Umzekelo wokuPruna 30% Ukunciphisa iiparamitha zemodeli ezingafunekiyo Ubungakanani 40% Yehlisa ukuchaneka kokubala Ubalo oluneMiqathango (MoE) 25% Ivula imodeli eyimfuneko kuphela UkuFunda okomeleza 15% Itshintsha ngamandla ukusetyenziswa kwamandla I-Neuromorphic Computing 50% Ilinganisa ukusebenza kakuhle kwengqondo I-Hardware Co-Design (ASICs, i-Optical Chips) 35% Uphuhlisa i-AI-specific hardware ukuze usebenze kakhulu Iimodeli ze-AI zexesha elizayo ziya kudibanisa iindlela ezininzi zokufezekisa i-60-70% yokunciphisa amandla ngokubanzi. Imicelimngeni yokuZiphucula i-AI : Ezinye iimpawu, ezifana nokuthenwa kunye nobungakanani, zinokubeka esichengeni ukuchaneka kancinane. Ukuchaneka koRhwebelwano Sisasebenza phantsi kwentelekelelo yokuxhomekeka kwiitshiphusi zesilicon ezingasebenziyo. IMida yeZiseko zeZiko leDatha: Okwangoku akukho mgangatho wehlabathi wokulandelela ukusebenza kakuhle kwamandla. Izikhewu zeMilinganiso yokuSebenza kwamandla: Imithetho engqongqo yozinzo inokunyanzela ukwamkelwa kweemodeli ezisebenzayo. Ulawulo lukaRhulumente: Iimpembelelo zexesha elizayo Ii-LLMs ezizisebenzelayo zinokucutha ukusetyenziswa kwamandla nge-20% okanye ngaphezulu kwiibhiliyoni zemibuzo, nto leyo eya kukhokelela kwindleko enkulu kunye nokongiwa kwe-emission. Oku kuhambelana nethagethi enguziro yehlabathi jikelele kwaye kuchaphazela amacandelo amaninzi: : Ii-LLMs ezonga amandla zinokunyusa ukuthathwa kwenkonzo yabathengi kunye nohlalutyo. Ishishini : Amanyathelo omthombo ovulekileyo afana noBuso obukwangayo anokukhawulezisa ukuvela kwezinto ezintsha. UPhando : Imigangatho yokungafihli kwamandla inokutyhala ukuziphucula njengesiqhelo. Umgaqo-nkqubo Ukuqukumbela Ii-LLMs ziye zazisa inqanaba elitsha lobuchule ekusetyenzweni kolwimi kodwa ingxaki yokusetyenziswa kwamandla azo yeyona nto ixhalabisayo. Nangona kunjalo, ubukrelekrele obunye obuye bavelisa ezi modeli bubonelela ngesisombululo. Ubuchwephesha obunjengokuthena, ubungakanani, ukubala ngokwemiqathango, kunye noyilo oludibeneyo lwehardware lubonisa ukuba kuyenzeka ukuyila iiLLM ezilawula ukusetyenziswa kwamandla azo. Njengoko uphando luqhubela phambili, umba uba ngaphantsi wokuba i-AI ezinzileyo inokwenzeka kwaye ngakumbi ukuba imboni yezobuchwephesha inokuhlangana ngokukhawuleza kangakanani ukuyifezekisa-ngaphandle kokuncama ukutsha kokusingqongileyo. Iimbekiselo UBrown, T., et al. (2020). "IiModeli zoLwimi ngaBafundi abaDutywayo abambalwa." , i-33, i-1877-1901. (Umthombo ocingelwayo wedatha yoqeqesho ye-GPT-3.) Ukuqhubela phambili kwiiNkqubo zoLwazi lwe-Neural Strubell, E., Ganesh, A., & McCallum, A. (2019). "Iingqwalasela zaMandla kunye nePolisi yokuFunda ngokuNzulu kwi-NLP." , 3645-3650. (Umthombo obonisa i-AI kwiindleko zamandla.) Iinkqubo zeNtlanganiso yoNyaka ye-57 ye-ACL Fedus, W., et al. (2021). "Tshintshela iiTransformers: Ukukala kwiiModeli zeParameter yeTriyoni kunye neSparity eLula kwaye eSebenzayo." . (Isiseko sengxoxo yoMxube weeNgcali.) arXiv preprint arXiv:2101.03961 Patterson, D., et al. (2021). "Ukukhutshwa kweCarbon kunye noQeqesho oluKhulu lweNeural Network." . (Umthombo woqikelelo lwamandla oqeqesho.) arXiv preprint arXiv:2104.10350 UPhando lukaGoogle. (2023). "IVizier: Inkonzo yokuPhucula iBhokisi eMnyama." . (Isixhobo esibonisa isalathiso.) Ibhlog yeGoogle AI
