Iyo One-Shot Generalization Paradox: Nei Generative AI Inetsekana Neruzivo Rutsva

Generative AI yanga isiri chinhu chipfupi chechamupupuri chetekinoroji. Mamodheru akaita seGPT-4 atora nyika nedutu nekugona kwavo surreal kugadzira zvinyorwa zvinotevedzera kutaura kwevanhu, kunyora zvinyorwa, kodhi, uye kunyange kuuya nemhinduro dzekugadzira kune mamwe mabasa akaomarara. Isu tiri kugara tichiswedera kune ramangwana rinobatsirwa neAI, iro apo vabatsiri vedu vedhijitari vanozonzwisisa uye kupindura kune zvatinoda. Zvakakwana kuita chero munhu mutendi, handiti?

Zvakanaka, zvinenge ... asi kwete chaizvo.

Munoona, pasi pekupenya kweGPT's glitzy goho uye kutsetseka kwayo kwegirama chidziviso chakakosha, icho chinopengesa vazhinji vedu tekinoroji: generative AI inotamburira kubata ruzivo rutsva, kunyanya mune imwe-pfuti yekudzidza mamiriro. Iyi inoratidzika kunge iri nyore (asi ichishungurudza) nyaya inoratidza gaka repakati mune zvazvino AI masisitimu. Kunyangwe nekukwanisa kugadzira ndima dzinokatyamadza kubva kumabhiriyoni emadatapoints, kana wapihwa basa rechokwadi - chimwe chinhu chaasati amboona kana kudzidziswa pa - GPT-maitiro emhando yakarova madziro ekuzvarwa.

Izvi zvinopenda mufananidzo wezvandinodaidza kuti "One-Shot Generalization Paradox" : zvisinei kuti ane simba sei, zvisinei kuti masystem 'akangwara' eAI seGPT anoita senge GPT, anodonha kana zvichidikanwa kuti zvigadzirise nekukurumidza kubva pane imwechete kana diki diki. mienzaniso isingaonekwi.

Ngatiburitse gangaidzo iri zvishoma tonyura muchikonzero chekuseri kwayo. Asi usazvinetse, isu hatizochengeta uhwu hungwaru - tichadzika mumatope ehunyanzvi toongorora kuti chii chaizvo chinomisa maAI edu azvino-gen kuti aenderane nekushanduka kwemashiripiti uko vanhu vanako pavanosangana nevasingazive.

Iwo Mashiripiti uye Mechanism yeGenerative Models… Kusvikira Vatyoka

Iko kupenya kwepakati kwemamodhi seGPT-4 kunogara pane yakaomesesa Transformer architecture , iyo inozivikanwa nekupa simba zvese kubva kumhando dzemitauro kuenda kumabasa echiono. Ikozvino, ini handidi kukurovera pasi nejagoni kutanga kwechimedu ichi (tichangotanga), asi mamwe matekinoroji anoda kuvhurwa kuti anzwisise kuti kupi uye nei mitswe inotanga kuratidza.

Kutanga, GPT ndeyemhuri yemhando dzakatevedzana , dzakadzidziswa kufanotaura izwi rinotevera kana tokeni mune chero chinyorwa chakapihwa. Vanova sei vakanaka kudaro? Muchikamu chikuru, imhaka yemagadzirirwo ekuzvitarisisa akavakirwa muTransformer , iyo inobvumira mamodheru aya kusefa mukati mehuwandu hwezvinyorwa uye zvakanyanya "kutarisa" pazvikamu zvakakosha zvemutsara uchitarisawo mazwi ese kutevedzana. Iyi nzira yepasirese yekutarisisa yakakurumidza kuve musimboti wekutora chirevo chechirevo muzvikamu zvakakura zvezvinyorwa.

Asi heino crux yekanganisika: Generative AI inotsamira zvakanyanya pane iyi data yekudzidzisa. Izvo zvakasarudzika pakuziva mapatani uye hukama hwehuwandu pakati petokens mune data yayakamboonekwa, asi zvakare inotsamira pane iyo data. Pakaburitswa modhi, GPT-4 yakanga isati yanyatsodzidza kufunga kana kukudziridza kunzwisisa kwenyika. Asi, ishamwari dzekusimudzira dzayakanhonga mumabhiriyoni emienzaniso yezvinyorwa inowanikwa online (mumabhuku, Wikipedia, Reddit tambo, mapepa ezvidzidzo… unozvipa zita).

Saka, nepo GPT inganzwa senge inoona zvese, ichigadzira zvinoenderana uye dzimwe nguva zvinyorwa zvine hungwaru, zvairi kunyatso kuita kutamba mutambo unonakidza we probabilistic pateni-kufananidza. Meaning? Kana chimwe chinhu chitsva chikauya (senge bepa idzva resainzi pane quantum mechanics kana imwe niche indasitiri-chaiyo jargon), inonetsekana zvakaoma kuti iite zvine musoro pazviri.

Izvo ... hazviverengeki.

Mira. Asi Sei Zvisingagoni Kungofanana Nevanhu?

Zvino, pano ndipo apo vanhu vakasiyana zvakanyanya kubva kumichina. Fungidzira kuti uri kuverenga nezve pfungwa inodarika hunyanzvi hwako kekutanga. Pamwe iwe uri tekinoroji yekutanga muvambi anofamba-famba pasi rose remakanika engineering. Chokwadi, haugone kubatanidza madotsi pakuverenga kwekutanga - asi mushure mekutarisa pamienzaniso mishoma kana madhayagiramu, mamwe mwenje wemwenje unodzima. Aha, iyi inodzora hurongwa! Izvi zvinobatana nazvo! Uye zvino, tarisa uye tarisa, iwe unozviwana (kana kuti yakawanda yacho).

Iyi nuance inonzi one-shot generalization - kugona kukurumidza kutora mapatani kana kunzwisisa ruzivo rutsva rwakavakirwa pamienzaniso mishoma. Uye chimwe chinhu icho vanhu vakanaka pachiri. Isu tinotora chidimbu chidiki cheruzivo uye nekuchenjera mepu kune yakakura madingindira, zvimiro, kana analogies atinoziva kare. Mune mamwe mazwi, isu hatidi miriyoni mienzaniso kana yakakura corpus ye data yapfuura kuti tive ne epiphany.

Mukupesana kukuru, mamodheru ekugadzira haana nzwisiso yomuzvarirwo yenyika zvachose. Ivo vanongofamba-famba nemunzvimbo yenhamba uye vanofanotaura zvichienderana nekuti ndeapi mazwi kana zvimiro zvinonyanya kuitika. Saka kana vakumbirwa kubata chimwe chinhu chakazara - izwi nyowani resainzi, dzidziso nyowani haina kumboburitswa online - vanomhanya nemusoro kumadziro. Zvichitaurwa zviri nyore, havasati vambosangana nazvo , uye vanoshaya maumbirwo echokwadi ekuti vasvetuke munzvimbo yavasina kujaira.

Zvakanaka, izvo zvinonzwisisika. Rega ndijekese izvi zvakare.

Generative AI modhi dzinodzidza nekududzira pakati peiyo iripo data mapoinzi. Zvichireva, vanova nyanzvi mukuzadza mikaha pakati pezvibodzwa zvavanenge vatoona uye mapatani avanoziva, asi vanonetsekana neextrapolation , kureva, kusvetuka uye kuita fungidziro zvichibva pane imwe pfungwa kana data rekudzidzisa risipo. Semuyenzaniso, GPT-4 inogona kubata "nguva dzose" zvivakwa zvemitauro mumutauro wemazuva ese zvakanakisa nekuti kune mamirioni emienzaniso iripo. Asi, kanda muchikumbiro cheari kubuda, hyper-specialized mazano - toti, iwo chaiwo ekufambira mberi kwemazuva ano mune solitonic fiber lasers mufizikisi - uye boom: gibberish yakakwana. Sei? GPT haina chero nhamba yereferensi yenzvimbo yakadai, mazwi matsva. Zvakatodzidzisa fungidziro dzekuti, kunyangwe zvichigoneka mukutsetsenura, kurega kupindirana kwechokwadi kwekururama kwemazwi .

Iyo Technical Core yeDambudziko

Zvakanaka, kana iwe uine pfungwa dzehunyanzvi, ngatinyure zvakadzama mukuti nei kudzikisira uku kwakasindimara, uye chii chiri kuitika pasi pehodhi panguva imwe chete yekuedza kudzidza.

Imwe nyaya yepakati neiyo imwe-pfuti generalization ndeyeruzivo rwunomiririrwa nemuenzaniso mukati panguva yekudzidzira kwayo yekuzvitarisira . Mamodheru eGPT ane maitiro akanaka kana achishanda mukati memiganhu - chiitiko chinowanzotsanangurwa sekudzidza mukugovera . Mukati memiganho yemisoro yakaona yakakura yakakwana mienzaniso yekudzidziswa, kunyangwe GPT-4 inogona kuburitsa zvinokatyamadza zvinobuda. Izvi zvinodaro nekuti chimiro cheiyo modhi chinoibvumira kuvharidzira ruzivo kuburikidza nedense vector inomiririra - muchimiro chekumisikidza kwakamisikidzwa - iyo inobata hukama pakati pemazwi uye pfungwa.

Asi pano ndipo panochinja zvinhu. Kana iyo modhi ikapihwa basa nemamiriro ezvinhu anoda kunze-kwe-kugovera generalization, zvichireva kusangana nepfungwa yayasati yambodzidziswa pairi, sisitimu hairevi zvinhu nenzira iyo vanhu vanoita. Funga nezvazvo seizvi: aya mamodheru ari ega mapatani michina , achitsamira pane manhamba "gut manzwiro." Ivo havana yakavakirwa-mukati kugona kugadzira kana kufunga "pamusoro pe data."

Semuenzaniso, funga kuti GPT inodzidza sei mitemo yegirama. Zvakafanana nemunhu akagara pasi kuti abate nemusoro zviuru zvenzira dzinoshandiswa mazwi mumitsara yeChirungu. Mushure mekutarisa zvakakwana, sisitimu inovaka mepu yemukati inoziva, "Ah, mushure mekunge chidzidzo chauya chiito, pamwe chinhu, uye kukanda muchinyorwa kana chirevo sezvinodiwa." Asi kana ikaunzwa nemutauro mutsva kana zvimiro zvezvirevo zvitsva, kugona uku kunotadza nekuti kunongogumira pakuziva chete hukama (kana husina kujeka) hwawakatoona.

Izvi, zvinosuruvarisa, zvine miganhu. Tora basa parichazoda kuburitsa zvinyorwa zvinopindirana nezvechinyorwa chisina kuburitswa, taura zvinokatyamadza zvakawanikwa mune isingazivikanwe nyaya yefizikisi senge quantum-gravity duality . Iyo modhi haina kuumbwa kunodiwa kududzira zvakare ruzivo rwechikuru kuti upe mukana mutsva. Muhuropi hwedu hwevanhu, tinogara tine zviratidziro zvepamusoro-soro (mazano, dzidziso, analogies!) izvo zvinotipa kushanduka. GPT, zvisinei, haidaro! Inoburitsa mibairo inoenderana nezvinogoneka kufungidzira , kwete kusvetuka kwekugadzira.

Zvakafanana nekutyaira uine mepu yakagara yakarongerwa nzira chete kubva muzana rapfuura: hazvikubatsire kufamba uchivakwa kana kuburikidza nekumonyoroka uye kutendeuka kwakaonekwa mumwedzi mitanhatu yapfuura.

Kuwana Unyanzvi - Sei Izvi Zvichiitika Pasi PeHood

Imwe nhanho yekunzwisisa iyo inogumira ndeye kuziva basa re dense vs sparse zvinomiririra .

Ndinorevei neizvi?

Traditional transformer modhi inoshanda ne dense vector embeddings . Chiratidzo chega chega mumutsara chinomiririrwa nemavhejita ane mativi epamusoro, uye mavekita aya anotora hukama hwakasiyana-siyana huri pakati pemazwi - zviumbwa zvemaumbirwo, zvirevo zvemataurirwo, zvimiro zvechimiro, zvichingodaro. Asi nekuti zviratidziro izvi zvakakora, hazvina kupatsanurwa zvakakwana kuti zvitsigire. abstraction nenzira inotungamira kune inoshanduka uye inogadzirisa generalization.

Dense embeddings inoganhurirwa neiyo bias-variance tradeoff panguva yekudzidziswa kwemuenzaniso. Iyi tradeoff yakakosha: nekugadzirisa chinhu chimwe (general statistical kugona), modhi inopira chimwe chinhu (kugona kufunga mumamiriro ezvinhu akazara). Fungidzira iwe unogara uchigadzirisa maitiro ako epfungwa kuti anyatsoenderana nenyika yawakamboona; iyo tradeoff ndeyekuti zvisingafungidzike zviitiko zvinokurasa zvachose. Mamodheru akaomarara-asi-akaomarara anonetsekana nemakesi madiki-diki nekuti anokunda pakudzokorora "avhareji mamiriro" uye kuomesa pamberi pezvisizvo kumitemo yakadzidzwa.

Iyo inogona kuve yakakosha mhinduro pano ndeye mashoma anomiririra - matekiniki ekugadzira mativi anopatsanura akasiyana maficha pamatanho akasiyana ekududzira. Sparse network inotaura nekutora ruzivo nenzira inochinjika uye yakajairika, senge nzira iyo vanhu vanotarisa nayo pazvinhu zvikuru, zvakakosha mukufembera mhedzisiro pane kutarisisa pamusoro pezvidiki.

Saka rimwe dambudziko ne-one-shot generalization nderekuti zvemazuva ano network zvimiro hazvisimbise mabasa ekusagadzikana akadaro - anotsamira zvakanyanya pane dense, inofambiswa nedata. Saka nei, kana vakakumbirwa kuti vagadzirise zvinhu zvitsva uye zvakasarudzika zvine mamiriro mashoma, vanokundikana.

Chii Chaigona Kugadzirisa Izvi?

Sezvineiwo, isu hatina zvachose kunze kwemazano. Vatsvagiri veAI (ini ndakasanganisirwa!) vatanga kudzidzisa nezve nzira dzinoverengeka dzekuvandudza kugona kweAI-yekupfura generalization. Dzimwe dzenzira dzinonakidza dzinotenderedza meta-yekudzidza zvivakwa. Aya mavakirwo akasiyana zvakanyanya neanhasi mamodheru, achigonesa kudzidza-ku-kudzidza kugona uko sisitimu inoshandura maparamita ayo kuti aenderane nemhando itsva dzedata nekukurumidza - zvakanyanya kuenderana nehunhu-sehunhu.

MuModel-Agnostic Meta-Learning (MAML) , semuenzaniso, modhi inozvigadzirisa kuti idzidze mabasa matsva nemienzaniso mishoma yekudzidzisa. Memory-Augmented Neural Networks (MANNs) inoshanda zvakafanana nekuchengeta zvakadzidzwa mukati mezviitiko zvakawanda, zvakafanana nekurangarira kwatinoita zvidzidzo zvakakosha kubva munguva yakapfuura uye nekuzvishandisa zvakare nekunzwisisa patinosangana neatsva, mamiriro akafanana.

Kubatanidza kugona kufunga kwekufananidzira mumienzaniso yakadzama yekudzidza ndiyo imwe nzira inovimbisa. Mamodheru akashongedzerwa nezvikamu zvekufananidzira anogona 'kufunga' kuburikidza nekufunga, pane kungovimba nenhamba. Minda yakaita seNeuro -Symbolic AI inopa mahybrids ekubatanidza mamodheru uye mitemo-yakavakirwa masisitimu inobvumira maAI kutevedzera epamusoro-kurongeka kufunga, kunyanya mune abstract kufunga mamiriro.

Nzira Inoenda Mberi?

Saka zvese izvi zvinorevei kune ramangwana reAI? Chokwadi, GPT-4 inonzwa semashiripiti kana ichitipa kudyidzana kwakanaka kwevatengi kana kupindura mibvunzo yakajairika, asi isu tinofanirwa kugadzira mamodheru asiri einjini dzekudzidzira nemusoro. Takananga kunguva yemberi uko kudzidza kwekufambisa , kudzidza-meta , uye neuro-symbolic architectures zvinosangana kugadzira vadzidzi vanochinja.

Iyo One-Shot Generalization Paradox haisi apocalyptic kufa-kuguma kweAI. Icho chipingamupinyi chinoita kuti tifunge patsva fungidziro huru pamusoro pehungwaru uye kuchinjika. Sezvo dhata rega risingagadzirise izvi - modhi dzinoda kugona kudzidza kubva kune zvipfupi , kugadzira analogies , uye rangarira zvakakosha maficha , kwete kungoziva nemusoro.

Mamodheru edu emangwana anozoda kuve vanhu vakawanda kupfuura muchina kana zvasvika pakuziva synthesis. Uye sevatsvagiri, vanogadzira, uye vanogadzira padanho rekucheka, tichiri mukutanga innings yekutsanangura zvazvinoreva kuti AI idzidze - yega - munyika inoshanduka, inonyorwa.

Iri harisi dambudziko rehunyanzvi. Ihwo huzivi.