```html
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 Ababhali: 
 
 
 
 
 
 
 
 
 
 
 
 Zhuoran Liu, Bytedance Inc. Leqi Zou, Bytedance Inc. Xuan Zou, Bytedance Inc. Caihua Wang, Bytedance Inc. Biao Zhang, Bytedance Inc. Da Tang, Bytedance Inc. Bolin Zhu∗, Fudan University Yijie Zhu, Bytedance Inc. Peng Wu, Bytedance Inc. Ke Wang, Bytedance Inc. Youlong Cheng†, Bytedance Inc. (youlong.cheng@bytedance.com) ISISHWENXELELO Ukwakha inkqubo engoqeqesho kwaye esebenza ngexesha lokuncoma kubalulekile kumashishini amaninzi axhaswa bubunini bempendulo yekhasimende, njengokuhlela iividiyo ezimfutshane okanye iintengiso ezikwi-intanethi. Nangona kusetyenziswa kakhulu iinkqubo zokufunda ezinzulu ezifana neTen-sorFlow okanye iPyTorch, ezi nkqubo zokujikeleza azanelisi iimfuno zeshishini kwiimeko zokuncoma ngezizathu ezahlukeneyo: ngokunye, ukulungisa iinkqubo ngokusekelwe kwiiparameter ezizinzileyo kunye nokubala okuninzi kokuncoma kunye neempawu ezingacacanga okanye eziguquguqukayo kubulala umgangatho wemodeli; kwelinye icala, ezi nkqubo zenzelwe inqanaba lokuhlela ibatch kunye nenqanaba lokukhonza lihlukene ngokupheleleyo, kuthintela imodeli ukuba ingasebenzisani nempendulo yekhasimende ngaxeshanye. Ezi ngxaki zasilisilele ukuba sihlolisise iindlela zemveli kwaye sifumane izikhetho zoyilo ezahlukileyo. Kule nqaku, sivelisa  1, inkqubo eyenzelwe ukuqeqeshwa kwe-intanethi. Uyilo lwethu luye lwajongana nokuqwalaselwa kwemisebenzi yethu yesicelo kunye nendawo yokuvelisa ebonisa ukuhlukana okukhulu kwezinye iinkqubo zokuncoma. Igalelo lethu linemiba emininzi: Okokuqala, sakha iteyibhile yokubandakanya engenamtheleleko kunye nezilungiso ezifana nezinto ezichaphazelekayo ezikhoyo kunye nokuhluza rhoqo ukunciphisa ubungakanani bayo bememori; okwesibini, sibonelela ngolwakhiwo olulungileyo lokwenziwa kwe-intanethi olunamandla aphezulu okumelana nokungaphumeleli; ekugqibeleni, siqinisekisile ukuba ukuthembeka kwenkqubo kunokutshintshiselwa ekufundeni kwangempela. IMonolith iphumelele ukusetyenziswa kwimveliso yeBytePlus Recommend2. Monolith 1      ISIVULA Nguloo minyaka ilishumi idlulileyo ibone ukwanda kwamashishini axhaswa yizicelo zobuchule bokuncoma. Ekufuneni amava angcono ekhasimende, ukubonelela ngomxholo owenziwe ngokwezifiso kumsebenzisi ngamnye njengempendulo yangempela yinto eqhelekileyo yezi zicelo zeshishini. Ukwenza oku, ulwazi olusuka kuhlobo lomsebenzisi lwakutshanje ludla lisetyenziswa njengomthombo ophambili wokuqeqesha imodeli, njengoko ingabonisa kakuhle umfanekiso womsebenzisi kwaye yenze izibikelelo zokuba umsebenzisi unomdla wokwenza kunye nokuziphatha okuzayo. Ukufunda okunzulu kudala kulawula iimodeli zokuncoma [ ,  ,  ,  ,  ,  ] njengoko ubungakanani obukhulu bedatha yomsebenzisi buyafana neenqanaba ledatha yedatha enobuchule be-neural models. Nangona kunjalo, iingxaki zokusebenzisa amandla okufunda okunzulu kwiinkqubo zokuncoma zamanqanaba aphezulu zihlala zihlangabezana neengxaki ezivela kwiimpawu ezizodwa zedatha eveliswe kukuziphatha komsebenzisi kwihlabathi lokwenyani. Le datha yahlukile kakhulu kwedatha esetyenziswa kwiingxaki eziqhelekileyo zokufunda okunzulu ezifana nokuhlela ulwimi okanye umbono wekhompyutha ngeendlela ezimbini: 5 6 10 12 20 21 (1)    Iimpawu zidla ngokuba ziqhekekile, zithiyelwe kwaye zitshintsha rhoqo; (2)    Ulwabiwo olusisiseko lwedatha yoqeqesho aluzinzanga, okanye i-Concept Drift  [8]. Olu hlukahlukano lubangele imiceliswano eyodwa kubaphandi kunye neenjineli ezisebenza kwiinkqubo zokuncoma. 1.1      Ukucaphukela nokutshintsha-shintsha Idatha yokuncoma idla ngokuba neempawu ezicacileyo, ezingafaniyo, ezinye zazo zivelwa kancinci. Indlela eqhelekileyo yokuzihlela kwindawo enkulu yokubandakanya inokubangela uthotho lweengxaki: •  Ngokungafaniyo neenqanaba zolwimi apho inani leenxalenye zamagama linokulingana, inani labasebenzisi kunye nezinto zokuhlela lingaphezu kwamaxabiso amakhulu. Itayibhile enkulu enje yokubandakanya ingahlali kwimemori enye yomamkeli; •  Okubi nangakumbi, ubungakanani beteyibhile yokubandakanya kulindeleke ukuba bukhule ngokuhamba kwexesha njengoko abasebenzisi kunye nezinto ezintsha zongezwa, ngelixa iinkqubo ezifana ne [ ,  ] zisebenzisa izinto ezizinzileyo zohlobo olufanayo ukumela iteyibhile yokubandakanya. 1 17 Ngokusebenza, iinkqubo ezininzi zisebenzisa ukuhlanganisa okunesiphumo esiphantsi [ ,  ] njengendlela yokunciphisa ubungakanani bememori kunye nokuvumela ukukhula kwe-ID. Oku kuxhomekeke kwisithunzi esingaphezu kokulindelekayo sokuba i-ID kwibhetri yokubandakanya ihlelwe ngokulinganayo rhoqo, kwaye iziphumo zokudibanisa azilimazi umgangatho wemodeli. Ngelishwa oku kunqabile ukuba kube yinyani kwinkqubo yokuncoma kwihlabathi lokwenyani, apho iqela elincinci labasebenzisi okanye izinto linamava amaninzi ngokubonakalayo. Ngokukhula kwendalo kobungakanani beteyibhile yokubandakanya, amathuba okudibana kwe-hash key ayanda kwaye akhokelele ekudayeni komgangatho wemodeli  3 6 [3]. Ngoko ke, yimfuno yendalo yeenkqubo zokuncoma zamanqanaba okuvelisa ukuba zibe namandla okubamba zonke iimpawu ezininzi kwiiparameter zayo, kwaye zibe namandla okubonelela ngamanani okukhula kwabasebenzisi kunye nezinto ekufuneka zibhaliswe. 1.2      Ulwabiwo olungazinzanga Imifanekiso kunye nemifanekiso yevolumu ayikho kunzima ukuyihlela kwisikali samakhulu eminyaka, ngelixa umsebenzisi ofanayo onomdla kwisihloko esithile angatshintsha uthando lwakhe imizuzu elandelayo. Ngenxa yoko, ulwabiwo olusisiseko lwedatha yomsebenzisi aluzinzanga, umbono owaziwa ngokuba yiConcept Drift  [8]. Ngokucacileyo, ulwazi olusuka kwimbali yakutshanje lunokunceda ngokufanelekileyo ukubikezela utshintsho ekuziphatheni komsebenzisi. Ukunciphisa impembelelo yeConcept Drift, iimodeli zokukhonza zifuna ukuhlaziywa ziimpendulo zabathengi ezikufutshane kangangoko kunokwenzeka ukuze zibonise umdla wokugqibela womsebenzisi. Ngokukhawuleza kwezi nkalo kunye nokuqwalaselwa kweengxaki ezivela kuveliso lwethu, sakha  , inkqubo enkulu yokuncoma ukulungisa ezi ngxaki. Senze uvavanyo oluninzi ukuqinisekisa kunye nokuhlaziya uyilo lwethu kwindawo yokuvelisa. IMonolith iyakwazi Monolith (1)    Ukubonelela ngamandla apheleleyo okubonisa iimpawu ezingacacanga ngokuyila iteyibhile ye-hash engenamtheleleko kunye nenkqubo yokukhupha iimpawu ezingaqhelekanga; (2)    Ukubuyisela impendulo yokukhonza ukuqeqeshwa ngexesha elifanayo ngokuqeqeshwa kwe-intanethi. Ixhaswa ngala mandla olwakhiwo, iMonolith ihlala igqitha iinkqubo ezisebenzisa i-hash-tricks kunye nezibambezelo kunye nobukhulu bememori obufanayo, kwaye ifumana i-AUC yokukhonza kwe-intanethi engaphezu kweyona ngobuchule ngaphandle kokubeka uxinzelelo olungaphezu kwamandla ombane eetshaja zethu. Okunye kweli nqaku kucwangciswe ngolu hlobo: Siqala sibonise iinkcukacha zoyilo malunga nendlela iMonolith ejongana ngayo nemiceliswano yangoku iteyibhile ye-hash engenamtheleleko kunye nokuqeqeshwa kwangempela kwiCandelo 2. Uvavanyo kunye neziphumo ziya kuboniswa kwiCandelo 3, kunye nezizathu ezivavanywe zokwenziwa kunye nokuxoxwa kwezinto ezingalungelelananga phakathi kobunini bexesha, ukuthembeka kunye nomgangatho wemodeli. ICandelo 4 lichaza imisebenzi ehambelanayo kwaye lizibeke ngokuchasene neMonolith. ICandelo 5 sigqibezela le misebenzi. 2      UYILO Ulwakhiwo lwonke lweMonolith ludla ngokulandela isicwangciso seTensorFlow esabelweyo seWorker- arameter erver (Umboniso   Kwi-Worker-PS, oomatshini banikezelwa iindima ezahlukeneyo; oomatshini beWorker banoxanduva lokwenza imisebenzi echazwe bubuchule, kwaye oomatshini bePS bagcina ii-parameters kwaye bazihlele ngokusekelwe kwiigradiyenti ezenziwe yiWorkers. P S 2). Kwiimbali zokuncoma, ii-parameters zihlelwa zibe ziindidi ezimbini: eziqinileyo nezingacacanga. Iiparameter eziqinileyo ziimpendulo/eziguquguqukayo kwinethiwekhi enzulu, kwaye ii-parameters ezingacacanga zibhekisa kwii-embedding tables ezihambelana neempawu ezingacacanga. Kuyo uyilo lwethu, zombini ii-parameters eziqinileyo nezingacacanga ziyinxalenye yeTensorFlow Graph, kwaye zigcinwe kwiiparameter servers. Ngokufanayo neTensorFlow Variable yeeparameter eziqinileyo, sakha iseti yeeHashTable ezisebenza ngokupheleleyo, ezingenamtheleleko, kwaye eziguquguqukayo zii-parameters ezingacacanga. Njengokugqibezela kumlinganiselwano weTensor-Flow owavela ekwahlukaneni kokuqeqeshwa nokusetyenziswa, ukuqeqeshwa kwe-intanethi okuhlelwa ngoku kwenza iMonolith yenzelwe ukuba iqeqeshe ngokuchanekileyo ii-parameters ukusuka kwi-training-PS ukuya kwi-online serving-PS ngexesha elifutshane, kunye nesiqinisekiso sokomelela kwemodeli esinikezelwa yindlela yokumelana nokungaphumeleli. 2.1      Itheyibhile yeHashTable Isimiso sokuqala kuyo uyilo lwethu lokumelwa kweeparameter ezingacacanga kuku-kuphepha ukubeka ulwazi olusuka kwi-ID ezahlukeneyo kwii-embedding ezifanayo zohlobo olufanayo. Ukulinganisa itheyibhile ye-embedding yobungakanani obuguquguqukayo kunye ne-TensorFlow Variable esele ikho ngaphandle kokulungiswa kudala kudala ukudibana kwe-ID, oku okwenzeka kakhulu njengoko ii-ID ezintsha zifikayo kwaye iteyibhile ikhula. Ngoko ke endaweni yokuwakha phezu kweVariable, siphuhlise i-HashTable entsha yomsebenzisi nomvuzo kwiiparameter zethu ezingacacanga. I-HashTable yethu isebenzisa iCuckoo Hashmap [ ] phantsi, enikezela ngexesha lokujonga kunye nokususwa kwe-𝑂 (1) kwiimeko ezimbi kakhulu, kunye ne-𝑂 (1) elindelekileyo yokungena okwethutyana. Njengoko kuboniswe kuMboniso   igcina iitheyibhile ezimbini 𝑇0, 𝑇1 ezineenqanaba ezahlukeneyo zokuhlanganisa ℎ0 (𝑥), ℎ1 (𝑥), kwaye into ingagcinwa kwelinye lazo. Xa uzama ukufaka into 16 3 𝐴 kwi-𝑇0, iqala ukuzama ukubeka i-𝐴 kwi-ℎ0 (𝐴); Ukuba i-ℎ0 (𝐴) ihlelwe nenye into 𝐵, izakukhupha i-𝐵 kwi-𝑇0 kwaye izame ukufaka i-𝐵 kwi-𝑇1 ngendlela efanayo. Le nkqubo izakuphindwa ide zonke izinto zihleleke, okanye i-rehash yenzeke xa ukungena kungena kwisekhethini. Ukunciphisa ubungakanani bememori nayo yinto ebalulekileyo kuyo uyilo lwethu. Indlela elula yokufaka yonke i-ID entsha kwi-HashTable izakuphelisa imemori ngokukhawuleza. Ukuqwalaselwa kwiimbali zokwenyani kukhokelele kwisigqibo esibini: (1)    Ii-ID ezivelayo kuphela kube kanye kancinci zinegalelo elincinci ekuphuculeni umgangatho wemodeli. Ukuqwalaselwa okubalulekileyo kukuba ii-ID zihlelwe ngokude, apho ii-ID ezidumileyo zingenzeka izigidi, ngelixa ezingadumileyo zivelayo kungadlulanga izihlandlo ezilishumi. Izinto ezibandakanya ezi zinto azinakuchaphazeleka ngenxa yokunqongophala kwedatha yoqeqesho kwaye imodeli ayizukwazi ukwenza isiqinisekiso esilungileyo kuso. Ekugqibeleni ezi zinto azinakubangela umphumo, ngoko umgangatho wemodeli awuyi kuhlupheka ngokususwa kwezi zinto zineemvavanyo ezimbalwa; (2)    Ii-ID ezizinzileyo ezivela kwimbali ekude azinako ukunceda imodeli njengoko uninzi lwazo lungazange lutyelelwe. Oku kunokubangelwa ngumsebenzisi ongasekhoyo, okanye ividiyo emfutshane eyaphelelwa lixesha. Ukugcina izinto ezibandakanya ezi zinto akunakunceda imodeli nangayiphi na indlela ngaphandle kokukhupha imemori yePS yethu ngenxa yobubutha. Ngokusekelwe kwezi ngqwalaselo, sakha izindlela ezininzi zokuhluza i-ID yempawu ukuze sisebenzise i-HashTable ngendlela enoxolelo ngememori: (1)    Ii-ID zihluza ngaphambi kokuba zingeniswe kwiitafile zokubandakanya. Sineendlela ezimbini zokuhluza: Okokuqala sihlula ngokomlinganiselo wazo phambi kokuba zingeniswe njengezikhi, apho umda woomlinganiselo uyinto enokulungiswa eguquguqukayo kwi-model nganye; Ukongeza, sisebenzisa isihlungi esingaqhelekanga esinceda ukunciphisa ngakumbi ukusetyenziswa kwememori; ![#Figure 4: Streaming Engine. Umjikelo wempendulo yolwazi ovela ku [User → Model Server → Training Worker → Model Server → User] uzakuchitha ixesha elide xa uthatha indlela yeBatch Training, ngelixa iOnline Training izakucima umjikelo ngokukhawuleza.](https://cdn.hackernoon.com/images/InxBRjRIs6M1kdhuWcyNHiiUrxm1-78b3ep4.jpeg) (2)    Ii-ID zihlelwa ngexesha kwaye zibekwe ukuba ziphele emva kokuba zingasebenzi ixesha elichaziweyo. Ixesha lokuphela nalo lingalungiswa kwibhetri yokubandakanya nganye ukuvumela ukwahlula iimpawu ngokubeka iliso kwiingxelo zembali ezahlukeneyo. Kuyo implementation yethu, iHashTable yenziwa njengomsebenzi weTensor-Flow. Ngokufanayo neVariable, ukujonga kunye nokuhlaziya nako kwenziwa njengemisebenzi yeTensorFlow yendalo ukuze kungeneleleko lula kunye nokuhambelana okungcono. 2.2      Uqeqesho Lwe-intanethi EMonolith, uqeqesho lwahlulwa lwaba ngamanqanaba amabini (Umboniso  1): (1)    Iqela lokuqeqesha ibatch. Eli nqanaba lisebenza njengesiqhelo seTensor-Flow training loop: Kwinqanaba ngalinye lokuqeqesha, umsebenzi wokuqeqesha ufunda ibatch encinci yeziboniso zokuqeqesha ukusuka kwindawo yokugcina, ucela ii-parameters kwiPS, ubale umsebenzi wangaphambili nangemva, kwaye ekugqibeleni ubuyisele ii-parameters ezihlaziyiweyo kwi-training PS. Kancinci kunezinye izinto eziqhelekileyo zokufunda okunzulu, siqeqesha idatha yethu kuphela ngohambo olunye. Uqeqesho lweBatch luluncedo ekuqeqesheni idatha yembali xa silungisa ulwakhiwo lwemodeli kwaye siyiqeqeshe kwakhona; (2)    Iqela lokuqeqesha le-intanethi. Emva kokuba imodeli ifakwe kwinkonzo ye-intanethi, uqeqesho aluqhubeki kodwa lungena kwinqanaba lokuqeqesha le-intanethi. Endaweni yokufunda iziboniso zebatch ezincinci ukusuka kwindawo yokugcina, umsebenzi wokuqeqesha usebenzisa idatha yangempela ngaxeshanye kwaye uhlaziye i-training PS. I-training PS ihlelana rhoqo ii-parameters zayo kwi-serving PS, ezo zizakusebenza kumsebenzisi ngaxeshanye. Oku kwenza imodeli yethu isebenze ngokuthe ngqo ngokwendlela yempendulo yomsebenzisi.  IMonolith yakhiwe ngamandla okutshintsha ngokungenamthungo phakathi kokuqeqesha ibatch kunye nokuqeqesha kwe-intanethi. Oku kunikwa amandla ngolwakhiwo lwethu lwe-streaming engine njengoko kuboniswe kuMboniso  2.2.1       Streaming Engine. 4. Kuyo uyilo lwethu, sisebenzisa umjikelo omnye weKafka [ ] ukubhalisa izenzo zabasebenzisi (Umzekelo. Ukucofa kwinto okanye ukuthanda into njl.njl.) kunye nomjikelo omnye weKafka weempawu. Ezingcanjini ze-injini kukho umsebenzi we-Flink [ ] we-streaming we-online feature Joiner. I-online joiner ihlanganisa iimpawu kunye neelebheli ezisuka kwiizenzo zabasebenzisi kwaye ivelise iziboniso zokuqeqesha, ezize zibhalelwe kumjikelo weKafka. Umjikelo weziboniso zokuqeqesha ufakwa ziinqanaba zombini zokuqeqesha kwe-intanethi kunye nokuqeqesha ibatch: 13 4 
 
 
 Kwii-intanethi zokuqeqesha, umsebenzi wokuqeqesha ufunda ngokuthe ngqo idatha kumjikelo weKafka; Kwii-intanethi zokuqeqesha ibatch, umsebenzi wokudumpla idatha uzakudumpla idatha kwi-HDFS [ ]; Emva kokuba idatha kwi-HDFS iqokelele inani elithile, umsebenzi wokuqeqesha uzakufumana idatha kwi-HDFS kwaye wenze uqeqesho lwebatch. 18 Iiparameter ezihlaziyiweyo kwi-training PS zizakubuyiselwa kwi-serving PS ngokomkhethelo wokuhlanganiswa kweeparameter.  Kwiinkqubo zeshishini, izenzo zabasebenzisi kunye neempawu zihlelwa kwii-online joiner (Umboniso   ngaphandle kwesiqinisekiso ngexesha lokulandelana. Ngoko ke sisebenzisa isitshixo esiyingunye kwisicelo ngasinye ukuze isenzo somsebenzisi kunye neempawu zingadibaniswa ngokuchanekileyo. 2.2.2       Online Joiner. 5) Ukubambezeleka kwesenzo somsebenzisi kungabikho yingxaki. Umzekelo, umsebenzisi angachitha iintsuku ezimbalwa ngaphambi kokuba agqibe ukuthenga into abayiboniswa kwiintsuku ezidlulileyo. Lo ngumceli-mngeni kwi-joiner kuba ukuba zonke iimpawu zigcinwe kwindawo yokugcina, ayizukulingana kwimemori. Kwenkqubo yethu, indawo yokugcina yokubeka kunye nokufumana idatha eyenziwe ngomatshini isetyenziselwa ukugcina iimpawu ezilindele ixesha elithile. Xa kufika ibhalansi yesenzo somsebenzisi, iqala ijonga indawo yokugcina ekwi-memori, emva koko ijonga indawo yokugcina yokubeka kunye nokufumana idatha xa kungekho ndawo yokugcina. Enye ingxaki evela kwiinkqubo zeshishini kukuba ulwabiwo lweziboniso ezingezizo nezithembisayo lulinganisiwe kakhulu, apho inani leyo yangaphambili lingadlulisa ngokuphindwe kaliqela kunezo zokugqibela. Ukuthintela iziboniso eziyimbalasane ukuba zingagxithwa zezingezizo, isicwangciso esiqhelekileyo kukwenza i-sampling engalunganga. Oku kuzakutshintsha ulwabiwo olusisiseko lwemodeli eqeqeshwe, iyenze ibe ngakumbi kubunketho bokwenza izibikelelo ezilunge ngakumbi. Njengesisombululo, sisebenzisa ukulungiswa kwe-log odds [ ] ngexesha lokukhonza, siqinisekise ukuba imodeli ye-intanethi ngumlinganisi ongajolisanga wolu lwabiwo lwangoku. 19  Ngexesha lokuqeqeshwa kwe-intanethi, iqela lokuqeqesha leMono-lith lihlala lifumana idatha kwi-online serving module kwaye luhlaziye ii-parameters kwi-training PS. Inyathelo elibalulekileyo lokunika i-online serving PS inzuzo kwezi parameter ziqeqeshwe entsha luhlanganiso lweeparameter zemodeli ezihlaziyiweyo. Kwiindawo zokuvelisa, sihlangabezana neemiceliswano ezininzi: 2.2.3      Parameter Synchronization. 
 
 
 Iimbali kwi-online serving PS akufuneki ziyeke ukukhonza xa zihlaziywa. Iimbali zethu kwi-production zidla ngokuba ngama-terabytes ambalwa, kwaye ngenxa yoko ukutshintsha zonke ii-parameters kuthatha ixesha. Kuza kuba akubandakanyekanga ukuyeka i-online PS ekukhonzeni imodeli ngexesha lenkqubo yokutshintsha, kwaye izilungiso kufuneka zenziwe ngokuphila; Ukudlulisa imodeli ye-multi-terabyte yonke ukusuka kwi-training PS ukuya kwi-online serving PS izabeka uxinzelelo olukhulu kuhlobo lwenethiwekhi kunye nememori kwi-PS, njengoko kufuna ubungakanani obufanayo bememori ukwamkela imodeli entsha efikayo. Ukuba ukuqeqeshwa kwe-intanethi kufanele kuphumelele kubukhulu besimo sethu seshishini, sakha indlela yokuhlanganisa ingqokelela ye-parameter ye-incremental ngohlobo olufanayo kwisikali esikhulu kwiMonolith esekelwe kwiimpawu ezimbalwa ezibalulekileyo zeembali zethu: (1)    Iiparameter ezingacacanga zidlula ubukhulu beembali zokuncoma; (2)    Ngokunikezelwa kwexesha elifutshane, kuphela inxalenye encinci ye-ID edla ngokuhlelwa kwaye ii-embeddings zayo zihlaziywa; (3)   > Ii-variable eziqinileyo zihamba kancinci kunezigaba ezicacileyo. Oku kungenxa yokuba kwi-optimizers ezisekelwe kwi-momentum, ukuqokelelana kwe-momentum yezi-variable eziqinileyo kuyanda ngobukhulu obukhulu bedatha yokuqeqesha yokuncoma, ngelixa kuphela izinto ezimbalwa ezicacileyo zifumana izilungiso kumqolo wedatha omnye. (1) kunye (2) zivumele ukuba sisebenzise izilungiso ezingacacanga kuzo zonke i-ID zempawu. KwiMonolith, sigcina isethi ye-hash yezitshixo ezichaphazelekayo, ezimela ii-ID ezii-embeddings zazo zihlelwe ukusuka kwi-synchronization yamva nje yeparameter. Sibuyisela inxalenye yeeparameter ezingacacanga ezizitshixo zazo zikwikhompleksiweyo ye-settime ngexesha elincinci ukusuka kwi-training PS ukuya kwi-online serving PS. Eli qela elincinci lihlanganiso le-parameter lihle kakhulu kuhambiso lwenethiwekhi kwaye alizukubangela ukunyuka okukhulu kwememori ngexesha lokuhlanganiswa. Siphinde sisebenzise (3) ukunciphisa ngakumbi ukusetyenziswa kwenethiwekhi ye-I/O kunye nememori ngokubeka inkqubo yokuhlangana esebenza ngakumbi kwiiparameter ezingacacanga, ngelixa uhlaziye ii-parameter eziqinileyo kancinci. Oku kunokunika imeko apho ii-parameters eziqinileyo esizihlelayo ziyinguqulo ephuculweyo kakhulu xa kuthelekiswa nenxalenye engacacanga. Nangona kunjalo, ukungangqongamanga okunjalo kunokwamkelwa ngenxa yesizathu esichazwe kwi (3) njengoko kungabikho kulahleko ebonakalayo efunyenweyo.

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Le audio iveliswe ngolwimi lokuqala lwebali!

Ngaphakathi kwiMonolith yeByteDance: Injini Enika Amandla Izibonelelo Zomxholo EziSmart, EziKhawulezayo

About Author

IZIMVO

ZIJONGE IIMPAWU

ELI NQAKU LINIKEZELWE KU

Related Stories

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps