Kunzwisisa Stochastic Avhareji Gradient

Gradient descent ndiyo inonyanya kufarirwa optimization maitiro mumuchina kudzidza (ML) modhi. Iyo algorithm inoderedza kukanganisa pakati pezvakafanotaurwa kukosha uye chokwadi chepasi. Sezvo tekinoroji ichifunga yega yega data kuti inzwisise uye kuderedza chikanganiso, kuita kwayo kunoenderana nekukura kwedata rekudzidzisa. Tekinoroji dzakaita seStochastic Gradient Descent (SGD) dzakagadzirirwa kuvandudza maverengero ekuita asi pamubhadharo weiyo convergence chaiyo.

Stochastic Average Gradient inoyera nzira yechinyakare, inozivikanwa seYakazara Gradient Descent uye SGD, uye inopa zvese zvakanaka. Asi tisati tashandisa algorithm, tinofanira kutanga tanzwisisa kukosha kwayo pakugadzirisa modhi.

Kugadzirisa Zvinangwa zveKudzidza zveMuchina neGradient Descent

Yese ML algorithm ine yakabatana kurasikirwa basa inovavarira kudzikisira kana kuvandudza mashandiro emuenzaniso. Masvomhu, kurasikirwa kunogona kutsanangurwa se:

Ungori mutsauko pakati pechaiko uye chakafanotaurwa, uye kuderedza mutsauko uyu zvinoreva kuti modhi yedu inoswedera pedyo nehwaro hwechokwadi hwepasi.

Iyo minimization algorithm inoshandisa gradient descent kuyambuka basa rekurasikirwa uye kuwana hushoma hwepasirese. Imwe neimwe nhanho yekutenderera inosanganisira kugadzirisa huremu hwealgorithm kuti ugone kuburitsa.

Plain Gradient Descent

Iyo yakajairwa gradient descent algorithm inoshandisa avhareji yeese magradient akaverengerwa padhata rese rese. Hupenyu hwemuenzaniso mumwechete wekudzidzira unotaridzika senge unotevera:

Iyo huremu yekuvandudza equation inoita seinotevera:

Ipo W inomiririra uremu hwemuenzaniso uye dJ/dW ndiyo inobva pakurasikirwa kwebasa maererano nehuremu hwemuenzaniso. Iyo yakajairika nzira ine yakakwira convergence mwero asi inove inodhura computationally kana uchibata nemahombe datasets anosanganisira mamirioni e data data.

Stochastic Gradient Descent (SGD)

Iyo nzira yeSGD inoramba yakafanana neGD iri pachena, asi pachinzvimbo chekushandisa dataset rese kuverenga magradients, inoshandisa batch diki kubva kune zvinopinda. Nzira yacho inoshanda zvakanyanya asi inogona kusvetukira zvakanyanya kutenderedza hudiki hwepasi rose sezvo kudzokororwa kwega kwega kunoshandisa chikamu che data pakudzidza.

Stochastic Avhareji Gradient

Iyo Stochastic Average Gradient (SAG) nzira yakaunzwa senzvimbo yepakati pakati peGD neSGD. Iyo inosarudza yakasarudzika data poindi uye inogadziridza kukosha kwayo zvichibva pane iyo gradient panguva iyoyo uye inorema avhareji yeakapfuura gradients akachengeterwa iyo chaiyo data data.

Zvakafanana neSGD, SAG inoenzanisira dambudziko rega rega semari inogumira ye convex, inosiyaniswa mabasa. Pane chero yakapihwa iteration, inoshandisa iwo aripo gradients uye avhareji yeakapfuura gradients yekuvandudza uremu. Iyo equation inotora fomu rinotevera:

Convergence Rate

Pakati peaya maviri akakurumbira algorithms, yakazara gradient (FG) uye stochastic gradient descent (SGD), iyo FG algorithm ine zvirinani convergence rate sezvo ichishandisa data rese rakaiswa panguva yega yega iteration pakuverenga.

Kunyangwe SAG iine chimiro chakafanana neSGD, mwero wayo wekusangana unofananidzwa uye dzimwe nguva zvirinani pane yakazara gradient maitiro. Tafura 1 pazasi inopfupikisa mibairo kubva kuongororo dze Schmidt et. al .

Zvimwe Zvinatsiridzwa

Kunyangwe kuita kwayo kunoshamisa, shanduko dzinoverengeka dzakakurudzirwa kune yekutanga SGD algorithm kubatsira kuvandudza mashandiro.

• Kudzoreredza uremu muYekutanga Iterations: SAG convergence inoramba ichinonoka mukati mekutanga mashoma iterations sezvo algorithm inojairisa gwara ne n (nhamba yakazara yemapoinzi edata). Izvi zvinopa fungidziro isiriyo sezvo iyo algorithm isati yaona akawanda mapoinzi data. Iko kugadziridzwa kunoratidza kujairana nem pachinzvimbo che n, uko m iri nhamba yemapoinzi edata akaonekwa kamwechete kusvika iyo chaiyo iteration.
• Mini-batches: Iyo Stochastic Gradient nzira inoshandisa mini-batches kugadzirisa akawanda data data panguva imwe chete. Nzira imwechete inogona kushandiswa kuSAG. Izvi zvinobvumira vectorization uye parallelization yekuvandudzwa kwekombuta. Iyo zvakare inoderedza ndangariro mutoro, dambudziko rakakurumbira reSAG algorithm.
• Nhanho-Saizi yekuyedza: Saizi yedanho yambotaurwa (116L) inopa mhedzisiro inoshamisa, asi vanyori vakawedzera kuedza nekushandisa nhanho saizi ye1L. Iyo yekupedzisira yakapa kusanganisa kuri nani. Zvisinei, vanyori havana kukwanisa kupa kuongororwa kwepamutemo kwemigumisiro yakagadziridzwa. Vanogumisa kuti saizi yenhanho inofanirwa kuyedzwa nayo kuti iwane iyo yakakwana yedambudziko chairo.

Pfungwa dzekupedzisira

Gradient descent ndiyo yakakurumbira optimization inoshandiswa kutsvaga yepasi rose minima yeakapihwa chinangwa mabasa. Iyo algorithm inoshandisa gradient yechinangwa chebasa kuyambuka mutsetse webasa kusvika yasvika pakaderera.

Yakazara Gradient Descent (FG) uye Stochastic Gradient Descent (SGD) misiyano miviri yakakurumbira yegorgorithm. FG inoshandisa yese dataset panguva yega yega iteration uye inopa yakakwira convergence rate pamutengo wakakwira computation. Pane imwe neimwe iteration, SGD inoshandisa subset yedata kumhanyisa algorithm. Inoshanda zvakanyanya asi nekusangana kusina chokwadi.

Stochastic Average Gradient (SAG) imwe mutsauko inopa mabhenefiti eaviri maalgorithms apfuura. Iyo inoshandisa avhareji yeakapfuura gradients uye subset yedataset kuti ipe yakakwira convergence rate ine yakaderera computation. Iyo algorithm inogona kugadziridzwa zvakare kuti ivandudze kugona kwayo uchishandisa vectorization uye mini-batches.