```html
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 AbaGwebi: 
 
 
 
 
 
 
 
 
 
 
 
 Zhuoran Liu, Bytedance Inc. Leqi Zou, Bytedance Inc. Xuan Zou, Bytedance Inc. Caihua Wang, Bytedance Inc. Biao Zhang, Bytedance Inc. Da Tang, Bytedance Inc. Bolin Zhu∗, Fudan University Yijie Zhu, Bytedance Inc. Peng Wu, Bytedance Inc. Ke Wang, Bytedance Inc. Youlong Cheng†, Bytedance Inc. (youlong.cheng@bytedance.com) ISISHWEnKO Ukuhlanganisa inkqubo yokuncoma ekwaziyo ukukala kwaye isebenze ngexesha lokwenyani kubaluleke kakhulu kumashishini amaninzi axhaswa yimpendulo yabathengi enobubini kwixesha, njengokuhlela iividiyo ezimfutshane okanye iintengiso ezikwi-intanethi. Ngaphandle kokusetyenziswa ngokubanzi kweziseko eziphambili zediphu yokufunda ezifana neTensorFlow okanye iPyTorch, ezi ziseko zisenokungawufikeleli kwiimfuno zoshishino kwiimeko zokuncoma ngezizathu ezininzi: ngokunye, ukulungisa iinkqubo ngokusekelwe kwiiparameter ezimileyo kunye nezibalo ezinzima zokuncoma ngeziphawulo eziguquguqukayo nezingacacanga kuyonakalisa umgangatho weModeli; ngokunye, ezi ziseko zenzelwe inqanaba loqeqesho lomthamo kunye nenqanaba lokukhutshwa ngokupheleleyo, kuthintelwa imodeli ekusebenzisaneni nempendulo yomthengi ngexesha lokwenyani. Ezi ngxaki zasinika phambili kwakhona iindlela zendabuko kwaye sazama ukukhetha uyilo olwahlukileyo. Kule nqaku, sibonisa i- 1, inkqubo eyenzelwe uqeqesho olukwi-intanethi. Uyilo lwethu luqhutywa ziinkalo zezicelo zethu kunye nokusingqongileyo okuvelisa ukwahluka okubalulekileyo kwezinye iinkqubo zokuncoma. Izibonelelo zethu zininzi: okokuqala, sakhetha iteyibhile yokufaka engenazo iingxaki kunye noncedo olufana neengqekembe ezidala kunye nokuhluza ubuninzi ukunciphisa umthwalo wayo omemori; okwesibini, sibonelela ngolwakhiwo loqeqesho olukwi-intanethi olulungele ukuvelisa kunye nobunyaniso obuphezulu; ekugqibeleni, sabonisa ukuba ukuthembeka kwenkqubo kungatshintshiswa ngokufunda ngexesha lokwenyani. I-Monolith iphumelele kwiMveliso ye-BytePlus Recommend2.  Monolith 1      INTSHAYELELO Ngo-10m iminyaka edlulileyo ibonise inyathelo lamashishini axhaswa yimicimbi yokuncoma. Ekufuneni inkqubo engcono yomthengi, ukuhambisa umxholo owenzelwe umntu ngamnye njengempendulo ngexesha lokwenyani yinjongo eqhelekileyo yale misebenzi yeshishini. Kule nto, ulwazi oluvela kokugqibela komsebenzisi kudla kusetyenziswa njengophambili ukuqeqesha imodeli, njengoko ibonisa ngokugqithisileyo imeko yomsebenzisi kwaye iqikelele umdla womsebenzisi kunye nezenzo ezizayo.  Ukufunda okubalulekileyo kuye kwaye kube nefuthe kwiimodeli zokuncoma [ ,  ,  ,  ,  ,  ] njengoko inani elikhulu ledatha yomsebenzisi lilungele iimodeli zenetwork eziqhutywa yidatha. Nangona kunjalo, imizamo yokusebenzisa amandla okufunda okubalulekileyo kwiinkqubo zokuncoma zamanqanaba oshishino ihlangana rhoqo neengxaki ezivela kwiimpawu ezikhethekileyo zedatha evela kwizenzo zabasebenzisi zangempela. Le datha yahlukile kakhulu kunezo zisetyenziswa kwiingxaki eziqhelekileyo zokufunda okubalulekileyo njengokubeka ulwimi okanye umbono wokubona ngezindlela ezimbini: 5 6 10 12 20 21 (1)    Iziphawulo zihlala zingacacanga, ziqhele ukwaziwa kwaye zitshintsha ngokukhawuleza; (2)    Ubuninzi obusezantsi bedatha yoqeqesho alugcini, okanye i-Concept Drift  [8]. Oko kwahlukileyo kuye kwabangela imicikilisho eyahlukileyo kubaphandi kunye neeinjineli ezisebenza kwiinkqubo zokuncoma.  1.1      Ukungacaci kunye noBuchule Idatha yokuncoma ineziphumo zekhethelo ezingacacanga, ezinye zazo zivela rhoqo. Indlela eqhelekileyo yokuzibeka kwindawo yokufaka ubukhulu obuphezulu ibangele uthotho lweengxaki: •  Ngokungafani neenjini zolwimi apho inani lezinto zolwimi lihleli, inani labasebenzisi kunye nezinto zokuhlela likhulu kakhulu. Itheyibhile enkulu yokufaka ngoluhlobo ayisayi kwangena kwimemori yendlu enye;  •  Okubi nangakumbi, ubukhulu iteyibhile yokufaka kulindeleke ukuba bukhule ngokuhamba kwexesha njengoko abasebenzisi kunye nezinto ezininzi zingena, ngelixa iziseko ezifana ne [ ,  ] zisebenzisa izinto ezimileyo ezinomthamo omkhulu ukumela itheyibhile yokufaka. 1 17 Ngokwenyani, iinkqubo ezininzi zisebenzisa i-hashing engozi encinci [ ,  ] njengendlela yokunciphisa umthwalo omemori nokuvumela ukukhula kwe-ID. Oku kuxhomekeke kulungelelo oluphakeme kakhulu lokuba ii-ID kwisiseko sokufaka zihlelwe ngokulinganayo kubuninzi, kwaye iingxaki azilimazi umgangatho weModeli. Ngokomdla oku akwenzeki kwinkqubo yokuncoma yangempela, apho iqela elincinci labasebenzisi okanye izinto zinokubonakala izihlandlo ezininzi. Ngokukhula okungokwemvelo kobukhulu beteyibhile yokufaka, amathuba okungena kwe-hash key ayanda kwaye abangela ukungcola komgangatho weModeli  3 6 [3]. Ngoko ke yimfuno yendalo kwiinkqubo zokuncoma zamanqanaba okuvelisa ukukwazi ukubamba izinto ezininzi kangangoko kwii-paramita zayo, kwaye ikwaba nokukwazi ukuhlengahlengisa ngokubhetyebhetye inani labasebenzisi nezinto ezizama ukuzibeka kwilista. 1.2      Ukuhlelwa Okungaguquguqukiyo Imizobo kunye nemizobo yobhalo ayifani rhoqo kwixesha elingangeenkulungwane, ngelixa umsebenzisi ofanayo onomdla kwisihloko esinye angatshintsha umdla wakhe ngomzuzu olandelayo. Ngenxa yoko, ubuninzi obusezantsi bedatha yomsebenzisi abugcini, into eqhelekileyo ibizwa ngokuba yiConcept Drift  [8]. Ngokuchazwa kwengqiqo, ulwazi olusuka kwimbali yakutshanje lungakwazi ukufaka isandla ngokufanelekileyo ekuqikeleleni utshintsho kwisenzo somsebenzisi. Ukunciphisa impembelelo yeConcept Drift, iimodeli zokukhutshwa kufuneka zihlaziywe ngempendulo yomsebenzisi entsha ngokukhawuleza kangangoko ukuze zibonise umdla womsebenzisi wokugqibela. Ngokwazi la maziko kunye nokubona iingxaki ezivela kwimveliso yethu, sakha i- , inkqubo enkulu yokuncoma ukujongana nezi ngxaki. Senze uvavanyo olubanzi ukuqinisekisa kunye nokuhlaziya uyilo lwethu kwindawo yokuvelisa. I-Monolith iyakwazi Monolith (1)    Ukubonelela ngamandla apheleleyo okuchaza iziphumo ezincinci ngokukhetha iteyibhile yokufaka engenazo iingxaki kunye nomatshini wokukhupha iziphumo eziguquguqukayo; (2)    Ukubuyisela impendulo yokukhutshwa koqeqesho ngexesha lokwenyani ngokuqeqeshwa okukwi-intanethi. Ngenkxaso yezi zakhono zolwakhiwo, i-Monolith igqithisa rhoqo iinkqubo ezisebenzisa i-hashing tricks kunye neengxaki ngomthamo ofanayo omemori, kwaye ifumana inkqubo yokukhutshwa kwe-AUC yexesha lokwenyani ngaphandle kokugxininisa amandla ethu ekhompyutha. Isiqingatha esishiyekileyo senqaku sihlelwe ngolu hlobo. Okokuqala sichaza iinkcukacha zoyilo malunga nendlela i-Monolith ejongana ngayo nemicikilisho ekhoyo nge-hash table engenazo iingxaki kunye noqeqesho ngexesha lokwenyani kwiCandelo 2. Ukuvavanywa kunye neziphumo ziya kuboniswa kwiCandelo 3, kunye nezisombululo ezivavanywe kwimveliso kunye nokuthetha ngentengiselwano phakathi kwexesha lokwenyani, ukuthembeka kunye nomgangatho weModeli. ICandelo 4 lifingqa umsebenzi ohambelanayo kwaye liyithelekise neMonolith. ICandelo 5 ligqibezela lo msebenzi.  2      UYILO Ulwakhiwo lonke lweMonolith ngokuqhelekileyo lilandela i-TensorFlow's distributed Worker- arameter erver setting (Umboniso   Kwi-Worker-PS architecture, oomatshini banikezelwa imisebenzi eyahlukileyo; oomatshini abaSebenzayo banoxanduva lokwenza izibalo njengoko kuchaziwe yigrafu, kwaye oomatshini abaPhaNgeniso bagcina ii-paramita kwaye bazihlaziywe ngokweegradients ezibalwe ngabaSebenzi.  P S 2). Kwiimfuno zokuncoma, ii-paramita zicwangciswe kwiingqokelela ezimbini: ezininzi kunye nezingacacanga. Iiparamita ezininzi zii-weights/variables kwidiphu yenetwork, kwaye ii-paramita ezingacacanga zibhekisa kwiitheyibhile zokufaka ezihambelana neziphawulo ezingacacanga. Kwindalo yethu, zombini ii-paramita ezininzi kunye nezingacacanga ziyinxalenye yeTensorFlow Graph, kwaye zigcinwa kuma-parameter servers.  Ngokufanayo neTensorFlow's Variable yeeparamita ezininzi, sakha iseti yezinto eziphumeleleyo, ezingenazo iingxaki, kunye neziguquguqukayo zokusebenza kweHashTable kwiiparamita ezingacacanga.  Njengoncedo kwi-Tensor-Flow's limitation evakala kwisahlulelo soqeqesho kunye nokuhluza, uqeqesho olusebenzayo olusebenzayo lweMonolith lwakhiwe ukuba luhambelanise ii-paramita ukusuka koqeqesho-PS ukuya kwi-serving-PS ngamathuba amafutshane, kunye nesiqinisekiso sobunzima bemodeli esinikezelwa ngumatshini wokungophuli iingxaki.  2.1      HashTable Isiseko sokuqala kwindalo yethu yokubonisa ii-paramita ezingacacanga kukuphepha ukugcwala kolwazi olusuka kwi-ID ezahlukeneyo kwisiseko esifanayo esingafakwanga. Ukulinganisa iteyibhile yokufaka enobukhulu obutshintshayo kunye neTensorFlow Variable engaphandle kwebhokisi kuya kukhokelela kungena kwe-ID, okwenza kube kubi njengoko ii-ID ezintsha zifikelela kwaye iteyibhile ikhule. Ngoko ke endaweni yokwakha phezu kweNgqekembe, sakha iHashTable entsha yomsebenzisi-ixabiso yeeparamita zethu ezingacacanga.  I-HashTable yethu isebenzisa iCuckoo Hashmap [ ] ngaphakathi, esekela ukufaka ii-key ezintsha ngaphandle kokungena kwezo zikhoyo. I-Cuckoo Hashing ifumana ixesha elibi kakhulu 𝑂 (1) lexesha lokujonga kunye nokususwa, kunye nelixesha elilindelekileyo elilinganiselweyo 𝑂 (1) lokufaka. Njengoko kuboniswe kuMboniso   ugcina iiteyibhile ezimbini 𝑇0, 𝑇1 ezinezisebenzi ezahlukileyo ze-hash ℎ0 (𝑥), ℎ1 (𝑥), kwaye into iya kugcinwa kuyo nayiphi na kuzo. Xa uzama ukufaka into  16 3 𝐴 kwi-𝑇0, kuqala izama ukubeka 𝐴 kwi-ℎ0  (𝐴); Ukuba i-ℎ0  (𝐴) igcinwe kwenye into 𝐵, iya kukhuphela 𝐵 kwi-𝑇0 kwaye izame ukufaka 𝐵 kwi-𝑇1 ngendlela efanayo. Lo msebenzi uya kuphindwa de zonke izinto zibe semagqabini, okanye i-rehash yenzeke xa ukufakwa kubetha kwingqungquthela.  Ukunciphisa umthwalo omemori kukungcamanga okubalulekileyo kwindalo yethu. Indlela elula yokufaka yonke i-ID entsha kwiHashTable iya kuphelisa imemori ngokukhawuleza. Ukubona iimfuno zokwenyani kukhokelela kwizigqibo ezimbini:  (1)    II-ID ezivela kanye izihlandlo ezimbalwa zinegalelo elincinci ekuphuculeni umgangatho weModeli. Ukubona okubalulekileyo kukuba ii-ID zihlelwe zaba ludidi, apho ii-ID eziphambili zingavela izihlandlo ezimilioni ngelixa ezingaphambili zingaveli ngaphezu kweshumi. Ukufakwa okuhambelana nezi ngcaciso ezingafane zenzeke zithintelwe ngenxa yokusweleka kwedatha yokuqeqesha kwaye imodeli ayiyi kukwazi ukwenza ingxelo elungileyo ngokusekelwe kuzo. Ekugqibeleni ezi ngcaciso azifani nokuchaphazela isiphumo, ngoko umgangatho weModeli awuyi kuhlupheka ngokususwa kwezi ngcaciso ezineenqanaba aphantsi;  (2)    II-ID ezidala ezivela kwimbali ekude azifani nokunceda imodeli yangoku njengoko uninzi lwazo lungazange luvakale. Oku kungaba ngenxa yomsebenzisi ongasekhoyo, okanye ividiyo emfutshane esele iphelile. Ukugcina ukufakwa kwezi ngcaciso akunakuyinceda imodeli nangayiphi na indlela kodwa ukucima imemori yethu yePS ngelize.  Ngokusekelwe koku kubona kunye nokuqwalaselwa, sakha ii-heuristics zokuhluza iziphumo zezinto ezininzi ukwenzela ukusebenza kweHashTable okusebenzisa imemori ngakumbi:  (1)    II-ID zihlelwa ngaphambi kokuba zingene kwii-embedding tables. Sinezindlela ezimbini zokuhluza: Okokuqala sihluza ngokubonakala kwazo ngaphambi kokuba zingene njengeekey, apho umda wokubonakala ungumatshini wokujikeleza ongaguquguqukiyo ohluka kwiModeli nganye; Ngaphezu koko, sisebenzisa isihluzi esingamathuba ukuze sincede ukunciphisa ngakumbi umthwalo omemori;  ![#Figure 4: Streaming Engine. Ukuphinda ukutya kolwazi ukusuka [Umgwebi → IModeli yoKhutsho → Umqeqeshi woQeqesho → IModeli yoKhutsho → Umgwebi] kuya kuthatha ixesha elide xa kuthathwa indlela yoQeqesho lweThamo, ngelixa uQeqesho olu-Onlayini luza kulivala igalelo ngokukhawuleza.](https://cdn.hackernoon.com/images/InxBRjRIs6M1kdhuWcyNHiiUrxm1-78b3ep4.jpeg) (2)    II-ID zihlala ziphawulwa ngexesha kwaye ziphawulwe ukuba ziphele emva kokungasebenzi ixesha elichaziweyo. Ixesha lokuphela lingatshintshiswa kwi-embedding table nganye ukuvumela ukwahlula iziphumo kunye nembali ethile.  Kukwenza kwethu, iHashTable yenziwe njengento yokusebenza yombonelelo weTensor-Flow. Ngokufanayo neNgqekembe, ukujonga kunye nokuhlaziya kwenziwa njengezinto zokusebenza zeTensorFlow ezidala ukwenzela ukudibanisa okulula kunye nokuhambelana okungcono.  2.2      UQEQESHO OLUKWI-INTANETHI EMonolith, uqeqesho luhlelwe kwiinqanaba ezimbini (Umboniso    1): (1)    INqanaba loqeqesho lomthamo. Eli nqanaba lisebenza njengesiqhelo seTensor-Flow sokugqibezela inkqubo yokuqeqesha: Kwinqanaba ngalinye yoqeqesho, umqeqeshi woqeqesho ufunda umthamo omncinci wezinto zokuqeqesha ezivela kwisitoreji, ucela ii-paramita kwi-PS, ubala inkqubela phambili nangemva, kwaye ekugqibeleni ubuyisela ii-paramita ezihlaziyiweyo kwi-PS yoqeqesho. Ngokungafani nezinye iinkqubo eziqhelekileyo zokufunda okubalulekileyo, siqeqesha idatha yethu kuphela ngomjikelo omnye. Uqeqesho lomthamo luncedo lokuqeqesha idatha yexesha elidlulileyo xa silungisa ulwakhiwo lwemodeli yethu kwaye siyiphinde siyifundise;  (2)    INqanaba loqeqesho olukwi-intanethi. Emva kokuba imodeli ifakelwe ukukhutshwa okukwi-intanethi, uqeqesho aluyeki kodwa lungena kwinqanaba loqeqesho olukwi-intanethi. Endaweni yokufunda izinto zomthamo omncinci ezivela kwisitoreji, umqeqeshi woqeqesho ufumana idatha yangexesha lokwenyani ngokukhawuleza kwaye uhlaziye i-PS yoqeqesho. I-PS yoqeqesho ihambelanisa ii-paramita zayo rhoqo kwi-serving PS, ezizakusebenza kwicala lomsebenzisi ngoko nangoko. Oku kwenza imodeli yethu ukuba isebenze ngokusebenzisanayo ngokusekelwe kwimpendulo yomsebenzisi ngexesha lokwenyani.   I-Monolith yakhiwe ngoncedo lokutshintsha ngokukhawuleza phakathi koqeqesho lomthamo kunye noqeqesho olukwi-intanethi. Oku kwenziwa kuku yiyo yethu ye injini ye nkqubo, njengoko iboniswa kuMboniso  2.2.1       Injini yeNkqubo. 4. Kwindalo yethu, sisebenzisa umgca omnye weKafka [ ] ukurekhoda izenzo zabasebenzisi (Umzekelo. Cofa kwinto ethile okanye uthumele into njl.njl.) kunye nomgca weKafka omnye wezinto. Ezingqibeni zenjineli yiFlink [ ] inkqubo yokuhambelana yomqeqeshi wokudibanisa okukwi-intanethi. Umdibanisi wangexesha lokwenyani udibanisa iziphumo ngeetitile ezivela kwiinkqubo zabasebenzisi kwaye ivelise izinto zokuqeqesha, ezibhalelwa kumbombo weKafka. Umqolo wezinto zokuqeqesha ufunwa zombini uqeqesho olukwi-intanethi kunye noqeqesho lomthamo:  13 4 
 
 
 Kuqeqesho olukwi-intanethi, umqeqeshi woqeqesho ufunda ngqo idatha kumbombo weKafka;  Kuqeqesho lomthamo, inkqubo yokulungisa idatha izacela idatha kwi-HDFS [ ]; Emva kokuba idatha kwi-HDFS iqokelele kwinqanaba elithile, umqeqeshi woqeqesho uza kufumana idatha kwi-HDFS kwaye enze uqeqesho lomthamo.  18 Iiparamita ezihlaziyiweyo kwi-PS yoqeqesho ziya kuthunyelwa kwi-serving PS ngokuhambelana neshedyuli yokuhambelana kweeparamita.   Kwiinkqubo zoshishino zangempela, imisebenzi yabasebenzisi kunye nezinto ziyahamba zize kwi-online joiner (Umboniso   ngaphandle kwesiqinisekiso ngokulandelana kwexesha. Ngoko ke sisebenzisa ikhowudi eyahlukileyo kwisicelo ngasinye ukuze isenzo somsebenzisi kunye nezinto zihambelane ngokuchanekileyo.  2.2.2       Umdibanisi Wangexesha Lokwenyani. 5) Ukulibaziseka kwesenzo somsebenzisi kungaba yingxaki. Umzekelo, umsebenzisi angathatha iintsuku ezimbalwa ngaphambi kokuba agqibe ekubeni athenge into ayeyibonile kwimihla edlulileyo. Oku kuyingxaki kwi-joiner kuba ukuba zonke izinto zigcinwe kwisheleni, ayizukungena kwimemori. Kwiinkqubo zethu, isitoreji esingokwedigripri sesekhwe-ixabiso sisetyenziswa ukugcina izinto ezilindele ixesha elithile. Xa kufika i-log yesenzo somsebenzisi, kuqala ijonga isheleni ye-memory, ize ijongwe isitoreji sekhwe-ixabiso xa kukho isheleni engekhoyo.  Enkobe yenye ingxaki evela kwiinkqubo zoshishino kukuba ubuninzi beemizekelo ezimbi nezinobuyakhazela buhlala bungalingani, apho inani lokuqala lingaba ngaphezu kweshumi eliphindwe kabini kunesamva. Ukuthintela iimzekelo ezithile ukuba zinganyelwa ngabantu abaninzi abanobuyakhazela, inkqubo eqhelekileyo yenza ukukhetha okungalunganga. Oku kuya kutshintsha ubuninzi obusezantsi beModeli yoqeqeshwe, ibuyekezele kwintlekele ephakamileyo yokwenza izibonelelo ezithile. Njengenkqubo, sibeka uhlobo lokulungiswa kwe-log odds [ ] ngexesha lokukhutshwa, siqinisekisa ukuba imodeli yangexesha lokwenyani yingcali engenakungabi yiyo yobuninzi bokuqala.  19  Ngexesha loqeqesho olukwi-intanethi, umhlanganiso weMono-lith uqhubeka ufumana idatha kwi-module yokukhutshwa okukwi-intanethi kwaye uhlaziye ii-paramita kwi-PS yoqeqesho. Inyathelo elibalulekileyo lokwenza i-PS yokukhutshwa okukwi-intanethi izuze kwiiparamita ezintsha eziqeqeshweyo kuku hlelweni lweeparamita ezihlaziyiweyo. Kwindlela yokuvelisa, siye sahlangana neengxaki ezininzi:  2.2.3       Ukuhambelaniswa kweParamita. 
 
 
 Iimfuno kwi-serving PS yangexesha lokwenyani kufuneka zingayeki ukukhutshwa xa uhlaziywa. Iimfuno zethu kwimveliso ngokuqhelekileyo ziliikhilogram ezimbalwa, kwaye ngenxa yoko ukutshintsha zonke ii-paramita kuthatha ixesha. Kukungafaneleki ukuyeka i-PS yangexesha lokwenyani ekukhutshweni kwemodeli ngexesha lenkqubo yokutshintsha, kwaye uhlaziyo kufuneka lwenziwe ngokukhawuleza;  Ukuhambisa imodeli ye-multi-terabyte yonke ukusuka kwi-PS yoqeqesho ukuya kwi-serving PS yangexesha lokwenyani kuya kubeka uxinzelelo olukhulu kwi-bandwith yenethiwekhi kunye nememori kwi-PS, njengoko ifuna ubukhulu bemodeli obuphindwe kabini ukwamkela imodeli entsha efikayo.  Ukuba uqeqesho olukwi-intanethi lwenze inkqubo yamashishini ethu, sakha inkqubo yokuhambelaniswa kweeparamita yenyani yokuhambelaniswa kweeparamita eyenziwe rhoqo, ixhaswa ziimpawu ezithile zemodeli zethu:  (1)    Iiparamita ezingacacanga zinomlinganiso omkhulu wobukhulu beemodeli zokuncoma;  (2)    Ngenxa yexesha elifutshane, isuntswana nje le-ID lifumana uqeqesho kwaye ii-embeddings zayo zahlaziywa;  (3)    Izinto ezininzi ezininzi zihamba kancinci kuneem-embeddings ezingacacanga. Oku kungenxa yokuba kwi-optimizers ezisekelwe kwimomenti, ukuqokelela imomenti kwiiparamita ezininzi kuphindwa bubukhulu bedatha yokuqeqesha yokuncoma, ngelixa ii-embeddings ezingacacanga kuphela zifumana uhlaziyo kumthamo wedatha omnye.  (1) kunye (2) zisinika ukusetyenziswa kwezahlulo ezincinci kuzo zonke ii-ID zeziphumo. KwiMonolith, sigcina isethi ye-hash yeekhowudi ezichaphazelekayo, ezibonisa ii-ID ii-embeddings zayo zaqeqeshwa ukusuka kokugqibela kohlelo lweeparamita. Siyathumela isuntswana leeparamita ezingacacanga ezinamakhwelo angaphakathi kwiseti yeekhowudi ezichaphazelekayo kunye nexesha elincinci ukusuka kwi-PS yoqeqesho ukuya kwi-serving PS. Le pakethi encinci yohlaziyo olucacayo ilula kwi-transmission yenethiwekhi kwaye ayiyi kubangela ukunyuka okubukhali kwememori ngexesha lokuhambelaniswa.  Siphinde sisebenzise (3) ukunciphisa ngakumbi i-I/O yenethiwekhi kunye nomthwalo omemori ngokubeka ishedyuli yokuhlaziya eguquguqukayo kwi-embeddings ezincinci, ngelixa uhlaziya ii-paramita ezininzi rhoqo. Oku kungasenza kwimeko apho ii-paramita ezininzi ezininzi esizikhutshayo ziyimbali ehleliyo xa kuthelekiswa ne-embeddings ezincinci. Nangona kunjalo, ukungangqinelani okunjalo kunokwamkelwa ngenxa yes

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Le audio iveliswe ngolwimi lokuqala lwebali!

Ngaphakathi kweMonolith yeByteDance: Injini Enika Amandla Iintetho Zomxholo Ezilumkileyo, Ezikhawulezayo

About Author

IZIMVO

ZIJONGE IIMPAWU

ELI NQAKU LINIKEZELWE KU

Related Stories

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps