
 
 
 
 
 
 
 Umbhali: (1) Avi Schwarzschild, University of Maryland, College Park, Maryland, USA kunye Work eyakhelwe ngexesha ekusebenzeni ku-Arthur (avi1umd.edu); (2) Max Cembalest, uArthur, eNew York City, eNew York, USA; (3) Karthik Rao, Arthur, eNew York City, eNew York, USA; (4) Keegan Hines, uArthur, eNew York City, eNew York, USA; (5) UJohn Dickerson†, uArthur, eNew York City, eNew York, USA (john@arthur.ai). Authors: (1) Avi Schwarzschild, University of Maryland, College Park, Maryland, USA kunye Work eyakhelwe ngexesha ekusebenzeni ku-Arthur (avi1umd.edu); (2) Max Cembalest, uArthur, eNew York City, eNew York, USA; (3) Karthik Rao, Arthur, eNew York City, eNew York, USA; (4) Keegan Hines, uArthur, eNew York City, eNew York, USA; (5) UJohn Dickerson†, uArthur, eNew York City, eNew York, USA (john@arthur.ai). Umbala we-Left I-Abstract kunye ne-Introduction 1.1 Ukucaciswa kwe-Post Hoc 1.2 Umxholo we-disagreement 1.3 Ukukhuthaza ukucaciswa Consensus 
 
 
 
 
 
 
 Related Work 
 
 Pear: Post HOC Explainer Agreement Regularizer 
 
 
 
 
 
 
 
 The Efficacy of Consensus Training 4.1 Agreement Metrics 4.2 Improving Consensus Metrics 4.3 Consistency At What Cost? 4.4 Are the Explanations Still Valuable? 4.5 Consensus and Linearity 4.6 Two Loss Terms 
 
 
 
 Discussion 5.1 Future Work 5.2 Conclusion, Acknowledgements, and References Ukucaciswa Ukucaciswa Njengoko i-networks ye-neural zenza izixazululo ezininzi ezininzi kwi-high-stakes settings, ukucacisa kunye nokucaciswa kwimeko yayo ngempumelelo kunye nokufumaneka kunokwenzeka. Umgangatho we-explainer esetyenziselwa i-post-hoc feature attribution, umgangatho yokufaka ngamnye umgangatho ekubeni umgangatho ebonakalayo kwimiphumo yayo kwi-output ye-model. Umgangatho olukhulu wesiqhagamshelwano wesiqhagamshelwano kwi-explainers kwi-practice iye inokukwazi ukuxhomekeka ukuba izici ezininzi ezininzi ezininzi ezininzi ezininzi ezininzi ezininzi. Indawo lethu kule nqakraza 1 Ukulungiswa Njengoko ucwaningo lombhobho lwabahlukaniswa kwiinkalo ezininzi zokusetyenziswa kwiinkalo ezininzi zokusetyenziswa kwiinkalo ezininzi zokusetyenziswa. Kule nqaku, sincoma isicelo esebenzayo ekutholeni i-model transparency ebonakalayo ekubeni iimodeli ze-blackbox ezininzi kwiimeko zokusetyenziswa kwimveliso: ukucacisa izici ezininzi ezininzi kwi-input kwi-output ye-model. Le mveliso ivumela i-data scientists kunye ne-machine learning practitioners ukuhlaziywa izici ngokubanzi - izici ezininzi ezinxalenye kwi-output ye-model zibonakalisa kakhulu, kwaye izici ezininzi ezinxalenye kwi-output ye 1.1 Ukucaciswa kwe-Post Hoc Iimvavanyo ze-model explanation ezibonakalayo kule nqaku ziquka i-post hoc local feature attribution scores. I-field ye-explainable artificial intelligence (XAI) ikhiqiza ngokukhawuleza iindlela ezahlukileyo zayo i-type ukufumana umzekelo umzekelo we-model [isib. 21, 24, 30, 32, 37]. Zonke izindlela zihlanganisa i-formula kunye ne-interpretation ye-output yayo, kodwa ngokuvamile zihlanganisa yonke into efanayo yokuthintela umzekelo we-model kwiimpawu zayo ze-input. Xa kusetyenziswa ukucacisa i-output ye-model kunye ne-input efanelekileyo ( kunye neengxaki ze-model ezinokufumaneka), le zindlela zibonisa ingxaki, "Ukuba umzekelo omnye we-input yinto efanelekileyo kwi-output ye-model?" I-Data Scientists isebenzisa i-post-hoc explaners ngexabiso olwandleyo - iindlela ezaziwayo ezifana ne-LIME kunye ne-SHAP babe ngaphezu kwe-350,000 kunye ne-6,000,000 ukulayishwa kwipaki zayo ze-Python kwiintsuku ezidlulileyo ze-30, ngokufanayo [23]. 1.2 Umxholo we-disagreement I-explosion ye-explanation methods eyenza i-Krishna et al. [15] ukuba zibonise ukuba xa i-networks ye-neural ziquka ngokwemvelo, i-i-precision kuphela, i-post-hoc explaners ziquka ngokubanzi malunga neengxaki ezininzi ezahlukileyo ezivela kwi-outputs ye-model. Zibonisa i-problem ye-disagreement kunye ne-argument that when explaners disagree about which features of the input are important, practitioners have little concrete evidence as to which of the explanations, if any, to trust. Kukho isibuyekezo ebalulekileyo malunga neengxaki zendawo kunye neengxaki zayo zayo zayo ekuphumezeni iimfuno yobugcisa kunye ne-interpretability yobugcisa [bheka, isib. 7, 18, 29]; ngoko ke, kukho izibuyekezo ezilandelayo malunga nokusebenza kwezindlela zonyango ezidlulileyo kwiinkalo ezizodwa [for healthcare bheka, isib. 8, 8]. Izibuyekezo zokusetyenziswa kwezobugcisa ziyafumaneka ukwenza iimodeli ezininzi ezininzi xa iimodeli ebonakalayo iye yinto ezininzi ezincinciphisa ukufikelela ngempumelelo iimfuno yeenkcukacha ngamnye. Kule nqaku, siyenza iingcebiso ze-normative malunga ne-debat, kodwa ukhangela i-"explanations" njengesignals ezisetyenziswa kunye nezinye iinkqubo ze-debugging, i-validation, kunye ne-verification kwi-machine learning operations (MLOps) pipeline. Ngokutsho, sisebenzisa indlela elandelayo esebenzayo: ukwandisa inani le-disagreement ye-explanation kwi-parameter ye-model enokulawulekayo kunokuba ngexesha le-frustration eyenza ama-stakeholders e-off-guard. 1.3 Ukukhuthaza ukucaciswa Consensus I-Consensus phakathi kwama-explainers ayidinga ukuba ama-explainers baye zithunyelwe iingcebiso ezifanelekileyo ngamnye. Ngoku, i-consensus phakathi kwama-explainers ibonisa ukuba nayiphi na ingxaki ze-explainers zinokufumaneka. I-Data scientists kunye ne-machine learning practitioners zithi kwi-enquiry ukuba i-explainers ziya kuxhomekeke kwimeko esisodwa ukuba zithunyelwe kwi-intuition yabantu, nto leyo inikeza indlela ye-quantitative yokubalulisa ukuba i-consensus iyatholakala [15]. Xa kuxhomekeka kwi-disagreement phakathi kwama-explainers, kufuneka ifumaneka ukuba kufuneka ifumaneke kwakhona - ukuba Njengomthombo yethu yokuqala, sincoma i-training routine yokusiza ukunciphisa iingxaki eyenziwe yi-post-hoc explanation disagreement. Ukuphumelela i-consensus engaphezulu phakathi kwezifundo kunikeza i-interpretability engaphezulu kwimodeli. Nangona kunjalo, inokufuneka ukufumana i-explanations i-credits engaphezulu ukuba iindlela ezahlukileyo ze-attribution zihlanganisa ngokukhawuleza malunga neengxaki ezininzi ezininzi. Oku kunikeza i-consensus i-benefit esebenzayo yokusebenza njenge-sanity check - ukuba i-consensus ibonakala, ukhetho le-explainer esetyenziselwa umdlavuza umdlavuza umdlavuza umdlavu 2 Izixhobo ezinxulumene Kwimveliso yethu ibekwe kwi-post-hoc explanation tools. Abanye explanators post-hoc, ezifana ne-LIME [24] kunye ne-SHAP [21], iimodeli ze-proxy ezidlulileyo kwimodeli yokufunda kwimodeli yokufunda kwimodeli yokufunda kunye ne-”explaining” kwimodeli yokufunda. Ezi explanators akufuneka kuphela kwi-inputs kunye ne-outputs ye-modeli ukucacisa iimpawu ezininzi. Ezinye explanators, ezifana neVanilla Gradients (Grad) [32], Gradient Times Input (Grad*Input) [30], Integrated Gradients (IntGrad) [37] kunye SmoothGrad [34], akufuna i-model ye-proxy Ukukhusela ngokukhawuleza kwiinkalo ezidlulileyo ekupheleleni kunye nokuhlola inkcazelo [15]. Ukukhusela apha kuxhomekeka kwi-function importance scores phakathi kwezindlela ezimbini ze-function attribution, kodwa kunokufumaneka ngezindlela ezininzi ezahlukileyo njengoko ziboniswe yi-metrics Krishna et al. [15] zibonisa kunye nokusetyenziswa. Thina zibonisa le metrics kwi-Section 4. Ukusetyenziswa kwinkqubo kule nqakraza kubandakanya iinkqubo ezidlulileyo ezidanga iimodeli ezinxulumene izifundo nge-penalty kwi-disagreement phakathi kwe-function attribution scores kunye ne-handcrafted ground-truth scores [26, 27, 41]. Ukongezelela, isebenze ukuba isebenze i-disagreement phakathi kwizifundo ezahlukeneyo ze-posthoc ukwakha i-function attribution scores ezintsha ekuphuculeni ama-metrics efana ne-stability kunye ne-pairing rank agreement [2, 16, 25]. 
 
 Oku kunokwenzeka kwi-archiv phantsi kolawulo lwe-CC BY 4.0 DEED. Le nqaku Ngaphandle kwe-CC BY 4.0 DEED License. available on arxiv Zifumaneka kwi-Archiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Read My Stories

No technological innovation comes without sacrifice. The pendulum will swing back to the people! Wanna' be 501c3.

Isifundo esitsha se-AI isixazululo lwemibuzo kwiimodeli ze-black-box

About Author

IZIMVO

ZIJONGE IIMPAWU

ELI NQAKU LINIKEZELWE KU

Related Stories

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps