Gusobanukirwa Impuzandengo Yimpuzandengo

Gradient descente nubuhanga buzwi cyane muburyo bwo kwiga imashini (ML). Algorithm igabanya ikosa hagati yindangagaciro zahanuwe nukuri kwukuri. Kubera ko tekinike itekereza kuri buri ngingo kugirango yumve kandi igabanye amakosa, imikorere yayo iterwa nubunini bwamahugurwa. Tekinike nka Stochastic Gradient Descent (SGD) yagenewe kunoza imikorere yo kubara ariko ku giciro cyo guhuza neza.

Impuzandengo ya Stochastic Impuzandengo iringaniza uburyo bwa kera, buzwi nka Full Gradient Descent na SGD, kandi butanga inyungu zombi. Ariko mbere yuko dushobora gukoresha algorithm, tugomba mbere na mbere gusobanukirwa n'akamaro kayo muburyo bwiza.

Gutezimbere Imashini Yiga Intego hamwe na Gradient Kumanuka

Buri algorithm ya ML ifite igihombo gifitanye isano igamije kugabanya cyangwa kunoza imikorere yicyitegererezo. Imibare, igihombo gishobora gusobanurwa nk:

Ni itandukaniro gusa hagati yukuri nibyahanuwe, kandi kugabanya itandukaniro bivuze ko icyitegererezo cyacu cyegereye indangagaciro zukuri.

Kugabanya algorithm ikoresha igabanuka ryamanuka kugirango igabanye imikorere yigihombo no kubona byibuze isi. Buri ntambwe yo kunyuramo ikubiyemo kuvugurura uburemere bwa algorithm kugirango uhindure ibisohoka.

Ibimanuka bya Gradient

Ibisanzwe gradient descente algorithm ikoresha impuzandengo ya gradients zose zabazwe kuri dataset yose. Ubuzima bwurugero rumwe rwamahugurwa asa nkibi bikurikira:

Ikigereranyo cyo kuvugurura ibiro bisa nkibi bikurikira:

Aho W igereranya uburemere bw'icyitegererezo na dJ/dW ni inkomoko y'ibikorwa byo gutakaza kubijyanye n'uburemere bw'icyitegererezo. Uburyo busanzwe bufite igipimo kinini cyo guhuza ariko kiba gihenze kubara mugihe ukorana na datasets nini zigizwe na miriyoni zamakuru.

Ibimanuka bya Stochastic Gradient (SGD)

Uburyo bwa SGD bukomeza kuba bumwe na GD isanzwe, ariko aho gukoresha dataset yose kugirango ibare gradients, ikoresha icyiciro gito kiva mubisubizo. Uburyo bukora neza cyane ariko burashobora kwiringira cyane minima yisi yose kuva buri itera ikoresha igice cyamakuru gusa yo kwiga.

Ikigereranyo cya Stochastic Impuzandengo

Uburyo bwa Stochastic Average Gradient (SAG) bwatangijwe nkimpamvu yo hagati hagati ya GD na SGD. Ihitamo amakuru adasanzwe kandi ikavugurura agaciro kayo ishingiye kuri gradient icyo gihe hamwe nimpuzandengo iremereye ya gradients zashize zabitswe kuri iyo ngingo yihariye.

Bisa na SGD, SAG yerekana buri kibazo nkumubare wuzuye wa convex, imikorere itandukanye. Kuri buri cyerekezo cyatanzwe, ikoresha gradients zubu hamwe nimpuzandengo ya gradients zabanjirije kugirango zongere ibiro. Ikigereranyo gifata uburyo bukurikira:

Igipimo cyo Guhuza

Hagati ya algorithm ebyiri zizwi cyane, gradient yuzuye (FG) hamwe na stochastic gradient descente (SGD), algorithm ya FG ifite igipimo cyiza cyo guhuza kuva ikoresha amakuru yose yashyizweho mugihe cyose itera kugirango ibare.

Nubwo SAG ifite imiterere isa na SGD, igipimo cyayo cyo guhuza kiragereranywa kandi rimwe na rimwe cyiza kuruta uburyo bwuzuye bwa gradient. Imbonerahamwe 1 ikurikira irerekana muri make ibisubizo bivuye mubushakashatsi bwa Schmidt n'abandi. al .

Ibindi Byahinduwe

Nubwo imikorere itangaje, impinduka nyinshi zasabwe kuri algorithm ya SGD yumwimerere kugirango ifashe kunoza imikorere.

• Ongera uremere muburyo bwambere: Ihinduka rya SAG riguma gahoro mugihe cyambere cyo gusubiramo kuva algorithm isanzwe icyerekezo hamwe n (umubare wamakuru wamakuru). Ibi bitanga igereranya ridahwitse nkuko algorithm itarabona ingingo nyinshi zamakuru. Guhindura byerekana guhuza na m aho kuba n, aho m numubare wamakuru yatanzwe byibuze rimwe kugeza igihe itera.
• Mini-batch: Uburyo bwa Stochastic Gradient bukoresha mini-batike kugirango batunganyirize amakuru menshi icyarimwe. Uburyo bumwe burashobora gukoreshwa kuri SAG. Ibi bituma vectorisation hamwe na parallelisation kugirango imikorere ya mudasobwa irusheho kugenda neza. Igabanya kandi umutwaro wo kwibuka, ikibazo gikomeye kuri SAG algorithm.
• Igeragezwa ryintambwe-Ingano: Ingano yintambwe yavuzwe haruguru (116L) itanga ibisubizo bitangaje, ariko abanditsi barushijeho kugerageza bakoresheje intambwe yintambwe ya 1L. Iyanyuma yatanze ndetse no guhuza neza. Ariko, abanditsi ntibashoboye kwerekana isesengura ryemewe ryibisubizo byahinduwe. Bashoje bavuga ko ingano yintambwe igomba kugeragezwa kugirango ibone icyiza kubibazo byihariye.

Ibitekerezo byanyuma

Gradient descente ni optimizme ikoreshwa mugushakisha minima yisi yimirimo yatanzwe. Algorithm ikoresha igipimo cyimikorere yibikorwa kugirango inyuze kumurongo kugeza igeze kumurongo wo hasi.

Ibimanuka Byuzuye (FG) na Stochastic Gradient descente (SGD) nibintu bibiri bizwi cyane bya algorithm. FG ikoresha imibare yose muri buri itera kandi itanga igipimo kinini cyo guhuza igiciro kinini. Kuri buri itera, SGD ikoresha igice cyamakuru kugirango ikore algorithm. Nibyiza cyane ariko hamwe no guhuza gushidikanya.

Impuzandengo ya Stochastic Impuzandengo (SAG) nubundi buryo butandukanye butanga inyungu za algorithm zombi. Ikoresha impuzandengo ya gradients zashize hamwe na sisitemu ya dataset kugirango itange igipimo kinini cyo guhuza hamwe no kubara gake. Algorithm irashobora guhindurwa kugirango irusheho kunoza imikorere ikoresheje vectorisation na mini-batch.