![Search icon](https://hackernoon.imgix.net/search-new.png?w=19&h=19)
Gradient descent ndiyo inonyanya kufarirwa optimization maitiro mumuchina kudzidza (ML) modhi. Iyo algorithm inoderedza kukanganisa pakati pezvakafanotaurwa kukosha uye chokwadi chepasi. Sezvo tekinoroji ichifunga yega yega data kuti inzwisise uye kuderedza chikanganiso, kuita kwayo kunoenderana nekukura kwedata rekudzidzisa. Tekinoroji dzakaita seStochastic Gradient Descent (SGD) dzakagadzirirwa kuvandudza maverengero ekuita asi pamubhadharo weiyo convergence chaiyo.
Stochastic Average Gradient inoyera nzira yechinyakare, inozivikanwa seYakazara Gradient Descent uye SGD, uye inopa zvese zvakanaka. Asi tisati tashandisa algorithm, tinofanira kutanga tanzwisisa kukosha kwayo pakugadzirisa modhi.
Yese ML algorithm ine yakabatana kurasikirwa basa inovavarira kudzikisira kana kuvandudza mashandiro emuenzaniso. Masvomhu, kurasikirwa kunogona kutsanangurwa se:
Ungori mutsauko pakati pechaiko uye chakafanotaurwa, uye kuderedza mutsauko uyu zvinoreva kuti modhi yedu inoswedera pedyo nehwaro hwechokwadi hwepasi.
Iyo minimization algorithm inoshandisa gradient descent kuyambuka basa rekurasikirwa uye kuwana hushoma hwepasirese. Imwe neimwe nhanho yekutenderera inosanganisira kugadzirisa huremu hwealgorithm kuti ugone kuburitsa.
Iyo yakajairwa gradient descent algorithm inoshandisa avhareji yeese magradient akaverengerwa padhata rese rese. Hupenyu hwemuenzaniso mumwechete wekudzidzira unotaridzika senge unotevera:
Iyo huremu yekuvandudza equation inoita seinotevera:
Ipo W
inomiririra uremu hwemuenzaniso uye dJ/dW
ndiyo inobva pakurasikirwa kwebasa maererano nehuremu hwemuenzaniso. Iyo yakajairika nzira ine yakakwira convergence mwero asi inove inodhura computationally kana uchibata nemahombe datasets anosanganisira mamirioni e data data.
Iyo nzira yeSGD inoramba yakafanana neGD iri pachena, asi pachinzvimbo chekushandisa dataset rese kuverenga magradients, inoshandisa batch diki kubva kune zvinopinda. Nzira yacho inoshanda zvakanyanya asi inogona kusvetukira zvakanyanya kutenderedza hudiki hwepasi rose sezvo kudzokororwa kwega kwega kunoshandisa chikamu che data pakudzidza.
Iyo Stochastic Average Gradient (SAG) nzira yakaunzwa senzvimbo yepakati pakati peGD neSGD. Iyo inosarudza yakasarudzika data poindi uye inogadziridza kukosha kwayo zvichibva pane iyo gradient panguva iyoyo uye inorema avhareji yeakapfuura gradients akachengeterwa iyo chaiyo data data.
Zvakafanana neSGD, SAG inoenzanisira dambudziko rega rega semari inogumira ye convex, inosiyaniswa mabasa. Pane chero yakapihwa iteration, inoshandisa iwo aripo gradients uye avhareji yeakapfuura gradients yekuvandudza uremu. Iyo equation inotora fomu rinotevera:
Pakati peaya maviri akakurumbira algorithms, yakazara gradient (FG) uye stochastic gradient descent (SGD), iyo FG algorithm ine zvirinani convergence rate sezvo ichishandisa data rese rakaiswa panguva yega yega iteration pakuverenga.
Kunyangwe SAG iine chimiro chakafanana neSGD, mwero wayo wekusangana unofananidzwa uye dzimwe nguva zvirinani pane yakazara gradient maitiro. Tafura 1 pazasi inopfupikisa mibairo kubva kuongororo dze
Kunyangwe kuita kwayo kunoshamisa, shanduko dzinoverengeka dzakakurudzirwa kune yekutanga SGD algorithm kubatsira kuvandudza mashandiro.
Gradient descent ndiyo yakakurumbira optimization inoshandiswa kutsvaga yepasi rose minima yeakapihwa chinangwa mabasa. Iyo algorithm inoshandisa gradient yechinangwa chebasa kuyambuka mutsetse webasa kusvika yasvika pakaderera.
Yakazara Gradient Descent (FG) uye Stochastic Gradient Descent (SGD) misiyano miviri yakakurumbira yegorgorithm. FG inoshandisa yese dataset panguva yega yega iteration uye inopa yakakwira convergence rate pamutengo wakakwira computation. Pane imwe neimwe iteration, SGD inoshandisa subset yedata kumhanyisa algorithm. Inoshanda zvakanyanya asi nekusangana kusina chokwadi.
Stochastic Average Gradient (SAG) imwe mutsauko inopa mabhenefiti eaviri maalgorithms apfuura. Iyo inoshandisa avhareji yeakapfuura gradients uye subset yedataset kuti ipe yakakwira convergence rate ine yakaderera computation. Iyo algorithm inogona kugadziridzwa zvakare kuti ivandudze kugona kwayo uchishandisa vectorization uye mini-batches.