![Search icon](https://hackernoon.imgix.net/search-new.png?w=19&h=19)
Gradient descent i thekiniki ya optimization leyi dumeke swinene eka machine learning (ML) modeling. Algorithm yi hunguta xihoxo exikarhi ka mimpimo leyi vhumbhiweke na ntiyiso wa le hansi. Tanihi leswi thekiniki yi tekelaka enhlokweni yinhla yin’wana na yin’wana ya datha ku twisisa na ku hunguta xihoxo, matirhelo ya yona ya titshege hi mpimo wa datha ya ndzetelo. Tithekiniki to fana na Stochastic Gradient Descent (SGD) ti endleriwe ku antswisa matirhelo ya xibalo kambe hi ntsengo wa ku pakanisa ka ku hlangana.
Stochastic Average Gradient yi ringanisela endlelo ra xikhale, leri tivekaka tanihi Full Gradient Descent na SGD, naswona yi nyika mimpfuno leyimbirhi. Kambe loko hi nga si tirhisa algorithm, hi fanele ku rhanga hi twisisa nkoka wa yona eka model optimization.
Algorithm yin’wana na yin’wana ya ML yi na ntirho wa ku lahlekeriwa lowu fambelanaka lowu kongomisaka eka ku hunguta kumbe ku antswisa matirhelo ya modele. Hi tinhlayo, ku lahlekeriwa ku nga hlamuseriwa tanihi:
I ntsena ku hambana exikarhi ka vuhumelerisi bya xiviri na lebyi vhumbhiweke, naswona ku hunguta ku hambana loku swi vula leswaku modele wa hina wu tshinela ekusuhi na mimpimo ya ntiyiso wa le hansi.
Algorithm ya ku hunguta yi tirhisa ku rhelela ka gradient ku tsemakanya ntirho wa ku lahlekeriwa no kuma minimum ya misava hinkwayo. Goza rin’wana na rin’wana ro tsemakanya ri katsa ku pfuxeta swipimelo swa algorithm ku antswisa vuhumelerisi.
Algorithm ya ntolovelo ya ku rhelela ka gradient yi tirhisa avhareji ya ti gradient hinkwato leti hlayiweke eka dataset hinkwayo. Xirhendzevutani xa vutomi bya xikombiso xin’we xa ndzetelo xi languteka hi ndlela leyi landzelaka:
Xiringaniso xa ku pfuxetiwa ka ntiko xi languteka hi ndlela leyi landzelaka:
Laha W
yi yimelaka swipimelo swa modele naswona dJ/dW
i derivative ya ntirho wa ku lahlekeriwa hi ku xixima ntiko wa modele. Ndlela ya ntolovelo yi na mpimo wa le henhla wa ku hlangana kambe yi va leyi durhaka hi tlhelo ra xibalo loko ku tirhana na tidathaseti letikulu leti katsaka timiliyoni ta tinhla ta datha.
Maendlelo ya SGD ya tshama ya fana na GD yo olova, kambe ematshan’wini yo tirhisa dataset hinkwayo ku hlayela ti gradients, yi tirhisa ntlawa wutsongo ku suka eka swingheniso. Ndlela leyi yi tirha kahle swinene kambe yinga hop ngopfu eka ti global minima tani hileswi iteration yin’wana na yin’wana yi tirhisaka ntsena xiphemu xa data ku dyondza.
Endlelo ra Stochastic Average Gradient (SAG) ri nghenisiwile tanihi ndhawu ya le xikarhi exikarhi ka GD na SGD. Yi hlawula ndhawu ya datha leyi nga hlelekangiki naswona yi pfuxeta nkoka wa yona hi ku ya hi xirhendzevutani eka ndhawu yoleyo na avhareji leyi pimiweke ya swirhendzevutani swa nkarhi lowu hundzeke leswi hlayisiweke eka ndhawu yoleyo yo karhi ya datha.
Ku fana na SGD, SAG yi modela xiphiqo xin’wana na xin’wana tanihi nhlayo leyi heleleke ya mintirho ya convex, leyi hambanisiwaka. Eka iteration yin’wana na yin’wana leyi nyikiweke, yi tirhisa ti gradients ta sweswi na avhareji ya ti gradients ta khale eka ku ndlandlamuxa ntiko. Xiringaniso xi teka xivumbeko lexi landzelaka:
Exikarhi ka ti algorithms timbirhi leti dumeke, full gradient (FG) na stochastic gradient descent (SGD), algorithm ya FG yina convergence rate yo antswa tani hileswi yi tirhisaka data hinkwayo ya data hi nkarhi wa iteration yin’wana na yin’wana ku hlayela.
Hambi leswi SAG yi nga na xivumbeko lexi fanaka na SGD, mpimo wa yona wa ku hlangana wu ringanisiwa na naswona minkarhi yin’wana wu antswa ku tlula endlelo ra full gradient. Tafula ra 1 laha hansi ri katsakanya mbuyelo ku suka eka swikambelo swa
Hambi leswi matirhelo ya yona yo hlamarisa, ku cinciwa ko hlayanyana ku ringanyetiwe eka algorithm yo sungula ya SGD ku pfuneta ku antswisa matirhelo.
Gradient descent i optimization leyi dumeke leyi tirhisiwaka ku kuma ti global minima ta mintirho ya xikongomelo leyi nyikiweke. Algorithm yi tirhisa gradient ya ntirho wa xikongomelo ku tsemakanya xirhendzevutani xa ntirho ku kondza yi fika eka ndhawu ya le hansi swinene.
Full Gradient Descent (FG) na Stochastic Gradient Descent (SGD) i ku hambana kambirhi loku dumeke ka algorithm. FG yi tirhisa dataset hinkwayo hi nkarhi wa iteration yin’wana na yin’wana naswona yi nyika mpimo wa le henhla wa ku hlangana hi ntsengo wa le henhla wa xibalo. Eka ku vuyeleriwa kun’wana na kun’wana, SGD yi tirhisa ntlawa lowutsongo wa datha ku fambisa algorithm. Yi tirha kahle swinene kambe yi ri na ku hlangana loku nga tiyisekiki.
Stochastic Average Gradient (SAG) i ku hambana kun’wana loku nyikaka mimpfuno ya tialgorithm leti hatimbirhi ta khale. Yi tirhisa avhareji ya ti gradients ta nkarhi lowu hundzeke na ntlawa lowuntsongo wa dataset ku nyika mpimo wa le henhla wa ku hlangana na xibalo xa le hansi. Algorithm yinga cinciwa kuya emahlweni ku antswisa vukorhokeri bya yona hiku tirhisa vectorization na mini-batches.