Ukwehla kwegradient yeyona ndlela idumileyo yokufunda koomatshini (ML) imodeli. I-algorithm inciphisa impazamo phakathi kwamaxabiso aqikelelweyo kunye nenyaniso esisiseko. Ekubeni ubuchule buqwalasela indawo nganye yedatha ukuqonda kunye nokunciphisa impazamo, ukusebenza kwayo kuxhomekeke kubungakanani bedatha yoqeqesho. Ubuchwephesha obufana ne-Stochastic Gradient Descent (SGD) yenzelwe ukuphucula ukusebenza kokubala kodwa ngexabiso lokuchaneka kokudibanisa.   I-Stochastic Average Gradient ilinganisa indlela yeklasi, eyaziwa ngokuba yi-Full Gradient Descent kunye ne-SGD, kwaye inikezela zombini izibonelelo. Kodwa ngaphambi kokuba sisebenzise i-algorithm, kufuneka siqale siqonde ukubaluleka kwayo ekuphuculeni imodeli.   Ukuphucula iiNjongo zokuFunda koomatshini ngokuhla kweGradient  Yonke i-algorithm ye-ML inomsebenzi welahleko ohambelanayo ojolise ekunciphiseni okanye ukuphucula ukusebenza kwemodeli. Ngokwezibalo, ilahleko inokuchazwa ngolu hlobo: Ngumahluko nje phakathi kwesiphumo sokwenyani kunye nesiqikelelweyo, kwaye ukunciphisa lo mahluko kuthetha ukuba imodeli yethu isondela kumaxabiso enyaniso asezantsi.  I-algorithm yokunciphisa isebenzisa ukuhla kwe-gradient ukunqumla umsebenzi welahleko kunye nokufumana ubuncinci behlabathi. Inyathelo ngalinye elinqumlayo libandakanya ukuhlaziya iintsimbi ze-algorithm ukuze kunyuswe imveliso.   Ukwehla kweGradient  I-algorithm yesiqhelo yokwehla kusetyenziswa umndilili wazo zonke izithambiso ezibalwe kuyo yonke isethi yedatha. Umjikelo wobomi bomzekelo woqeqesho omnye ujongeka ngolu hlobo lulandelayo:   I-equation yokuhlaziya ubunzima ibonakala ngolu hlobo lulandelayo: Apho   imele ubunzima bemodeli kunye ne   yi-derivative yomsebenzi welahleko ngokubhekiselele kubunzima bomzekelo. Indlela eqhelekileyo inezinga eliphezulu lokuhlangana kodwa iba yindleko yekhompyutha xa ijongene nedatha enkulu equka izigidi zamanqaku edatha. W dJ/dW   Ukuhla kweStochastic Gradient (SGD)  Indlela ye-SGD ihlala ifana ne-GD ecacileyo, kodwa endaweni yokusebenzisa i-dataset yonke ukubala i-gradients, isebenzisa ibhetshi encinci evela kumagalelo. Indlela yokusebenza isebenza kakuhle kakhulu kodwa inokutsiba-tsiba kakhulu ijikeleze ubuncinci behlabathi njengoko ukuphindaphinda ngakunye kusebenzisa kuphela inxenye yedatha yokufunda.   I-Stochastic Average Gradient  Indlela ye-Stochastic Average Gradient (SAG) yaziswa njengendawo ephakathi phakathi kwe-GD ne-SGD. Ikhetha i-random data point kwaye ihlaziya ixabiso layo ngokusekelwe kwi-gradient ngelo xesha kunye ne-avareji ye-weighted of the past gradients egcinwe kuloo ndawo yedatha ethile.  Ngokufana ne-SGD, iimodeli ze-SAG zonke iingxaki njengesixa esilinganiselweyo se-convex, imisebenzi eyahlulahluko. Kuyo nayiphi na i-iteration, isebenzisa iigradient zangoku kunye nomndilili we-gradients zangaphambili zokuhlaziya ubunzima. Inxaki ihamba ngolu hlobo lulandelayo:    Ireyithi yokuHlangana  Phakathi kwee-algorithms ezimbini ezidumileyo, i-gradient epheleleyo (FG) kunye ne-stochastic gradient descent (SGD), i-algorithm ye-FG inezinga elingcono lokuguqulela ekubeni isebenzisa yonke idatha eseti ngexesha lokuphindaphinda ngalunye ukubala.  Nangona i-SAG inesakhiwo esifana ne-SGD, izinga lokuhlangana kwayo liyathelekiseka kwaye ngamanye amaxesha lingcono kunendlela epheleleyo yokuthamba. Itheyibhile 1 ngezantsi ishwankathela iziphumo ezivela kwiimvavanyo ze  .    Schmidt et. al   Uhlengahlengiso Olungaphaya  Ngaphandle kokusebenza kwayo okumangalisayo, uhlengahlengiso oluninzi luye lwacetywa kwi-algorithm ye-SGD yasekuqaleni ukunceda ukuphucula ukusebenza.    I-SAG convergence ihlala icotha ngexesha lokuphindaphinda okumbalwa kokuqala ukususela ekubeni i-algorithm ilungisa ulwalathiso nge-n (inani lilonke lamanqaku edatha). Oku kunika uqikelelo olungachanekanga njengoko i-algorithm ingekaboni amanqaku amaninzi edatha. Uhlengahlengiso lucebisa ukwenziwa kwesiqhelo ngu-m endaweni ye-n, apho u-m linani lamanqaku edatha abonwayo kube kanye kude kube koko kuphindaphindwa. Ukulinganisa kwakhona kwii-Early Iterations:    Indlela yeStochastic Gradient isebenzisa iibhetshi ezincinci ukucubungula amanqaku amaninzi edatha ngaxeshanye. Kwale ndlela inye inokusetyenziswa kwi-SAG. Oku kuvumela i-vectorization kunye nokuhambelana nokuphucula ukusebenza kakuhle kwekhompyuter. Ikwanciphisa umthwalo wememori, umngeni obalaseleyo we-algorithm ye-SAG. Iibhetshi ezincinci:    Ubungakanani benyathelo obukhankanywe ngaphambili (116L) bubonelela ngeziphumo ezimangalisayo, kodwa ababhali baphinda bazama ngokusebenzisa ubungakanani benyathelo le-1L. Le yokugqibela ibonelele ngakumbi ukudibana. Nangona kunjalo, ababhali abakwazanga ukubonisa uhlalutyo olusesikweni lweziphumo eziphuculweyo. Bagqiba kwelokuba ubungakanani benyathelo kufuneka buvavanywe ngalo ukuze kufunyanwe eyona ifanelekileyo yengxaki ethile. Uvavanyo lobungakanani benyathelo:   Iingcamango Zokugqibela  I-Gradient descent ludidi oludumileyo olusetyenziselwa ukufumana i-minima yehlabathi yemisebenzi enikeziweyo yenjongo. I-algorithm isebenzisa i-gradient yomsebenzi wenjongo ukunqumla i-slope yomsebenzi de ifike kwindawo ephantsi.  I-Gradient Descent (FG) kunye ne-Stochastic Gradient Descent (SGD) zimbini iinguqu ezidumileyo ze-algorithm. I-FG isebenzisa yonke idatha yedatha ngexesha lokuphindaphinda ngalunye kwaye ibonelela ngezinga eliphezulu lokuhlangana ngexabiso eliphezulu lokubala. Kwi-iteration nganye, i-SGD isebenzisa i-subset yedatha ukuqhuba i-algorithm. Isebenza kakhulu ngakumbi kodwa inokuhlangana okungaqinisekanga.  I-Stochastic Average Gradient (SAG) yenye inguqu ebonelela ngeenzuzo zombini ii-algorithms zangaphambili. Isebenzisa i-avareji ye-gradients ezidlulileyo kunye ne-subset ye-dataset ukubonelela ngezinga eliphezulu lokuhlangana kunye nokubala okuphantsi. I-algorithm inokuguqulwa ngakumbi ukuphucula ukusebenza kwayo ngokusebenzisa i-vectorization kunye ne-mini-batches.

This story contains new, firsthand information uncovered by the writer.

Every

Gradient

Read My Stories

Le audio iveliswe ngolwimi lokuqala lwebali!

Ukuqonda iStochastic Average Gradient

About Author

IZIMVO

ZIJONGE IIMPAWU

ELI NQAKU LINIKEZELWE KU

Related Stories

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps