I-CathAI: Abacwaningi wahlengise i-AI enokufunda i-heart scans njenge-cardiologist

Umbhali: Robert Avram Jeffrey E. Olgin Zeeshan Ahmed Louis Verreault-Julien Alvin Wan Joshua Barrios Sean Abreau Derek Wan Joseph E. Gonzalez Jean-Claude Tardif Derek Y. So Krishan Soni Geoffrey H. Tison Umbhali: uRobert Avram UJeffrey E. Olgin Ukucinga uLouis Verreault-Julien U-Alvin Wan UJosua Barrios U-Sean Abreau I-Derek Wan UJosef E. Gonzalez U-Jean-Claude Tardif U-Derek Y. So Yenza i-Soni U-Geoffrey H. Tison Ukucinga I-Cathigraphic Angiography ye-CAD ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cathigraphic Angiography ye-Cath Ukucaciswa I-Coronary Heart Disease (CHD) i-cause enkulu ye-adult death e-United States kunye nehlabathi Ukusetyenziswa kwe-Atherosclerotic Plaques ezincinciphisa i-coronary arteries, eyaziwa nangokuthi i-coronary artery disease (CAD). Inqubo ye-angiography ye-coronary angiography yinkqubo ye-gold standard yokubonisa i-coronary artery stenosis yaye ngoko inikeza ulwazi olufanelekileyo kwiinkqubo ze-CAD zokusetyenziswa kwe-medical therapy kuphela ukuya kwi-revascularization nge-coronary stents okanye i-bypass surgery Ukucaciswa kwe-physician ye-stenosis ye-coronary angu ye-angiograms ibekwe yindlela elidlulileyo kakhulu yokusetyenziswa kwe-guideline ukucacisa ukucaciswa kwe-angiographic ye-coronary artery lumen , . 1 2 2 3 Nangona kunjalo, izixazululo ze-visual estimate ye-stenosis ye-coronary zihlanganiswa kakuhle, kwaye zihlanganisa i-intra- kunye ne-inter-observer variability, i-operator bias, kunye ne-reproductibility engaphansi. , I-Variability ye-stenosis ye-visual ye-evaluation ifumaneka kwi-7.6 ukuya kwi-22.5% , , Kwaye nangona umgangatho we-imaging ye-angiography ye-coronary iye yandiswa, ukuxhaswa kwakhona kwaye kunzima kakhulu kwi-setting ye-mother's reader, nto leyo ngexesha elidlulileyo le-workflow ye-clinical. , Ukubalwa kwe-stenosis ye-visual inokukwazi ukucacisa ubunzima we-stenosis kwizigidi ezingaphezu kweeyure Ukusetyenziswa kwe-stent kwizigulana ze-coronary artery bypass kwizigulana ze-17% kunye ne-10% kwizigulana. , , , Ukusetyenziswa kwe-angiogram ye-angiogram kunye nokuqinisekisa i-stenosis ye-coronary iya kuba kubaluleke kakhulu kwi-clinical. 4 5 4 5 6 4 7 8 4 5 6 8 Iingcebiso ezahlukileyo ezahlukileyo ziquka ukuguqulwa kwe-CAD ngexesha le-angiography ye-coronary, njenge-assessment ye-physiological , I-Intra-vascular Imaging Nangona amaninzi kufuneka iinkcukacha ezininzi ze-operator kunye nokusetyenziswa kwe-catheters ezininzi, ngoko kusetyenziswa kwi-10-20% ye-angiograms ye-coronary , , Ngaphezu kwalokho, ukuhlaziywa kwimfuneko ye-addjunctive test kunokuba kuxhomekeke ikakhulu kwi-visual estimate ye-angiogram ye-doctoral ngexesha le-angiography yokufaka i-intermediate-severity okanye i-greater coronary stenosis (isib. 40-69%) . Quantitative coronary angiography (QCA) is a technique providing analysis of angiograms that allows for more standardized stenoses assessment Nangona kunjalo, QCA ayikho ngokupheleleyo ngokuzenzekelayo kwaye kufuneka i-operator input ye-image calibration kunye ne-frame selection, oku kubandakanya kakhulu kwi-research settings. , . 9 10 11 9 11 12 3 13 5 14 I-artificial intelligence (AI) i-algorithms, kunye ne-deep neural networks ikakhulukazi, zibonise umthamo yokutshintshwa kwezimfuneko ezininzi zonyango kwi-cardiology kunye ne-interventional cardiology. Indawo yethu kuyinto ukuvelisa kunye nokuqinisekisa inkqubo okuzenzakalelayo ye-angiogram ye-coronary, i-coronary artery stenosis localization kunye nokuqinisekisa ubunzima kwi-real-world angiograms ye-coronary. 15 Imiphumo I-Full Dataset ibandakanya izifundo ze-13 843 ze-angiogram epheleleyo (i-195,195 iividiyo ze-angiographic ezininzi) kwi-11,972 izigulane abaneminyaka engama-18 ezidlulileyo kwi-University of California, San Francisco (UCSF), phakathi kwe-1 April 2008 kunye ne-31 December 2019. Ukongeza iFig. Uhlobo lweminyaka yama-63.5 ± 13.7 kwi-Full Dataset kunye ne-66.7 ± 12.0 kwi-Report Dataset. 1A 1 Ukubuyekezwa kwePipeline ye-CathAI ye-angiogram ye-interpretation ye-automated. Ividiyo ze-Angiogram ziqhutywa kwi-algorithm eyodwa ukuya kwi-algorithm elandelayo ukuyisombulula iingxaki ze-4 ezininzi ezininzi ezininzi ezininzi ezininzi ezininzi ze-interpretation ye-automated. Ukusetyenziswa kwe-CathAI kwi-example ye-angiogram ye-coronary. Umzekelo we-angiogram ye-artery elandelayo evulekileyo kunye ne-stenosis evulekileyo (i-proximal ukuya kwi-middle segment) ibonisa ukuba isebenza nge-CathAI system ukuze: ukucacisa i-angiographic projection, ukucacisa i-LCA (i-left), ukucacisa i-boxes ezivela kwi-object ezifana ne-stenosis (i-zoom, i-middle), kunye nokuba ukucaciswa kwe-stenosis ye-maximum (i-right). a b Izixhobo ze-CathAI zihlanganisa i-angiographic projection angle, izakhiwo ze-anatomical kunye ne-angiogram object localization Ukuqhathanisa i-angiographic projection angle, i-CathAI yenza i-frequency-weighted positive predictive value (PPV) ye-0,90, i-sensitivity ye-0,90 kunye ne-F1 score ye-0,90 kwi-test dataset (i-Table Supplementary). ). Iingubo ezisetyenziswa kakhulu ze-projection, ezifana ne-LAO Straight, i-RAO Straight kunye ne-LAO Cranial, zilungele kwimveliso yobugcisa. Ukuqhathanisa isakhiwo se-anatomic yokuqala ebonakalayo kwividiyo ye-angiogram, i-CathAI yenza i-PPV ye-frequency-weighted ye-0,89, i-sensitivity ye-0,89 kunye ne-F1 ye-0,89 kwi-test dataset (I-Table) ). CathAI performance was high for both the left and right coronary arteries (LCA and RCA, respectively), which are the primary angiographic objects of interest: PPV and sensitivity, respectively, were 0.97 and 0.94 for LCA and 0.93 and 0.93 for RCA. 8 1 Xa iividiyo ze-angiogram ezininzi zihlanganisa i-LCA kunye ne-RCA ziye zibonwa, i-CathAI i-localized izicwangciso ze-angiogram ngokutsho izicwangciso ze-boxes ezivela kwiingxaki ze-coronary artery ezifana ne-proximal portion ye-coronary artery, izixeko ze-stenosis, i-stents kunye ne-coronary catheters. Ukubala ukusebenza kwe-CathAI yokufaka izicwangciso ze-objects, i-medium average precision (mAP) isetyenziselwa ukuguqula izicwangciso ze-ground-truth ye-object. I-CathAI ibonisa i-48,1% ye-mapped average ye-test dataset, ebonakal (Ukuvelisa Table ). I-CathAI i-localized ngokufanelekileyo i-93.3% ye-coronary artery stenosis (PPV) kwi-test dataset ye-UCSF. 16 9 Ukusebenza kwe-CathAI ukucinga ukubaluleka kwe-stenosis I-algorithm ekugqibeleni kwinkqubo ye-CathAI i-estimated severity of coronary artery stenosis (I-Fig. ). Iingcebiso ezidlulileyo ze-stenosis ziye ziye zithunyelwe kwiividiyo ze-angiogram ezininzi kwi-studio eyodwa ebonakalayo i-artery segment (eyaziwa ngokuba yi-” artery-level”), ebonakalisa isebenzo se-standard. Kwi-test dataset, i-AUC ye-CathAI yokubonisa i-stenosis obstruktive yaba yi-0,862 (i-95% CI: i-0,843-0,880) kwi-artery-level (Table) • FIG. ). Ukuhlaziywa kwe-stenosis kwinqanaba le-artery ibekwe ngakumbi kunzima kunzima kunokuba yi-video okanye i-image-level predictions (i-Fig. I-Table ye-Extra Ukulungiselela i-CathAI ye-stenosis ye-stenosis njenge-percentage-stenosis epheleleyo, ingxaki ye-percentage-absolute ephakathi phakathi kwe-AI-stenosis kunye ne-report-stenosis yaba-17.9 ± 15.5% (Tabela) • I-FIG YOKUFUNDA. ). Kukho ingqungquthela ingqungquthela ingqungquthela ingqungquthela ingqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqungquthela ngqung < 0.001, Ububanzi ; Supplementary Table I-CathAI yaba i-AUC ephakeme kakhulu ukucacisa i-stenosis ephakeme kwizigaba ze-female vs. male [0.890 (i-95% CI: 0.864-0,923) vs. 0.830 (i-95% CI: 0.805-0,856) ngokulinganayo, ixabiso ukusebenzisana: 0.02 1B 2 2a 2A 10 2 3 p 2 10 P I-Receiver Operating Characteristic Curves (ROC) ye-CathAI yokubonisa i-obstructive (</≥70%) coronary stenosis kwi-test dataset. I-Black Dot: I-AI-stenosis threshold ebonakalayo ukufikelela i-F1 inqaku kwi-artery-level dataset. I-Red Cross: I-Specificity xa i-sensitivity iyahlekile kwi-80%. I-Blue Cross: I-Sensitivity xa i-specificity iyahlekile kwi-80%. I-Matrix ye-Confusion for CathAI prediction of obstructive stenosis vs. REPORT-stenosis. I-UOHI External Validation Dataset-ROC for CathAI prediction of obstructive (</≥70%) stenosis. I-Black Dot: I-AI-stenosis threshold eyenziwe ukunciphisa i-F1 in the artery-level dataset. I-Red Cross: I-Specificity when sensitivity is fixed at 80%. I-Blue Cross: I-Specificity when specificity is fixed at 80%. UOHI External Validation Dataset-Confusion matrix for CathAI prediction of obstructive stenosis vs. adjudicators. AUC area under the curve, CI confidence interval, AI artificial intelligence. a b c d Kwezi ≥70% stenosis ngokutsho REPORT-stenosis, i-CathAI i-classified i-74.5% ngokufanelekileyo (i-95% CI: 70.0-78.4%; i-260/349, i-Fig. ); kwaye kwezi <70% stenosis by REPORT-stenosis, i-Algorithm 4 i-classified 78.1% ngokufanelekileyo (i-95% CI:76.1–80.1%; 1082/1385). Xa i-Algorithm 4 ye-sensitivity yokubonisa i-obstructive coronary stenosis iye yakhelwe kwi-80.0%, i-specificity yokubonisa i-obstructive stenosis iye yi-74.1%; kwaye xa i-specificity yakhelwe kwi-80.0%, i-sensitivity yokubonisa i-obstructive stenosis iye yi-71.6%. Kukho iingxaki ezincinane ezininzi phakathi kwe-AI-stenosis kunye ne- REPORT-stenosis (Tabela ). CathAI overestimated stenosis ezincinane kwaye underestimated stenosis ezincinane (I-Fig. inguqulelo 2B 2 3 Ukuhlaziywa kwe-CathAI kwi-UOHI dataset Ukubuyekeza generalizability ephakamileyo, sisetyenziswa CathAI ukuba 464 iividiyo angiogram ngempumelelo ukusuka UOHI. ). Kwi-100% ye-UOHI angiograms, i-Algorithm 2 iboniswe ngempumelelo kwi-RCA okanye i-LCA. Ii-adjudicators ezimbini ze-UOHI zihlanganisa i-stenosis kwi-segment ye-coronary artery efanayo kwi-91.4% ( = 424). Kwi-subset ye-CathAI i-stenosis ezisetyenziswa kwi-segment ye-artery efanayo kwi-84.5% ( = 360); yonke i-15.5% ezidlulileyo apho i-sub-segment ye-artery ayikho ngokugqithisileyo, i-CathAI ibonise i-stenosis kwi-coronary artery ngokugqithisileyo (i-LCA vs. i-RCA). Zonke i-stenosis ezidlulileyo ze-CathAI ziye ziyi-stenosis, ngaphandle kwe-artefact ngenxa ye-oppacification ye-suboptimal okanye i-vessel tortuosity, ngokuxhomekeke kumadoda. 4 n n I-inter-observer variability ye-stenosis i-percentage assessment phakathi kwama-adjudicators angama-15.7% ± 14.5%. Ngokutsho kwe-obstructive (</≥70%) stenosis, ama-adjudicators abahlukaniswa kwi-16.8% ze-stenoses njengoko i-obstructive ( = 78). Sifuna ubunzima we-arithmetic ye-stenosis ye-percent of the two adjudicators ukuxhaswa nge-CathAI's prediction of stenosis severity prediction. Ngokuhambisana na, i-CathAI's AUC for obstructive stenosis (≥70%) iye yi-0,869 (i-CI ye-95%: i-0,830-0,907; Figs. Ukucinga ); ukucindezeleka 86.4% kunye ne-specificity i-67.3% (i-Fig. ingqiqo efanayo esetyenziswa njenge-dataset ye-UCSF, i-0.54) I-intermediate absolute percentage difference between AI-stenosis and the percentage stenosis averaged from the two adjudicators was 18.02% ± 11.02%. Kwi-dataset ye-UOHI, i-CathAI ivele ~3-5 s ukuyifaka ividiyo ye-angiography, okanye i-35 s ye-examination epheleleyo, usebenzisa i-Nvidia GTX 1080 Ti GPU. n 2C 3 2D Iimiphumo ezininzi ze-University of California, San Francisco (UCSF) yokubhalisa kwintlobo kunye ne-University of Ottawa Heart Institute (UOHI) yokubhalisa kwintlobo ze-4 ziyafumaneka ngamnye. Ukulungiselela i-CathAI ukucinga i-QCA Stenosis I-QCA data set ye-RCT ibandakanya izigulane ze709 kunye nezifundo ze-858 (1384 i-stenosis; i-18 i-stenosis ≥70%, i-71 i-stenosis ≥50% kodwa i-<70% kunye ne-1295 i-non-severe). I-CathAI ibandakanya ngempumelelo i-100% yevidiyo ze-LCA / RCA kunye ne-78.7% ( I-1384) ye-stenosis (67.2% ye-LCA kunye ne-86.8% ye-RCA). Ukuphakama kwe-stenosis ye-QCA yi-31.7 ± 11.6% (I-Fig. ). Given the divergent patient population of the RCT QCA dataset of mostly mild CAD and QCA adjudication criteria (≥50% QCA stenosis) instead of visual estimation, the RCT QCA dataset provided an opportunity to retrain CathAI to predict QCA labeled stenoses, as opposed to visually estimated stenoses. Once re-trained, CathAI’s AUC to discriminate obstructive QCA stenosis (≥50%) was 0.775 (95% CI: 0.594–0.955) at artery-level (Supplementary Table ) kunye ne-QCA stenosis ≥70%, i-AUC yaba i-0,75 (i-CI ye-95%: i-0,070-0,930). I-stenosis yahlukileyo phakathi kwe-CathAI-predicted stenosis kunye ne-QCA-estimated stenosis yaba i-6.5 ± 5.5% kwi-test dataset. n 5 11 Ukusetyenziswa kwinethiwekhi ye-neural network ukufumana ukusebenza kwe-CathAI Ukuphathelela ngcono iimpawu ngaphakathi angiograms enikeza CathAI imibuzo, sisetyenziswa izindlela ezimbini ukucacisa i-CathAI algorithms epheleleyo. Oku kunceda ukucacisa njani i-algorithms isebenza kwaye inikeza inkqubo eyongezelelweyo yokwenza imibuzo kubafundi. GradCAM Ukubonisa izixeko ze-image ezininzi ezininzi zokusabela i-CathAI, ebonisa ukuba isetyenziswe izixeko ezifana ne-cardiologist, njenge-left anterior descending artery ukucacisa i-images ye-LCA (i-Fig. Uyaziwa kwakhona imizobo emangalisayo ye-CathAI ye-prediction of stenosis severity usebenzisa i-Layer Ordered Visualization of Information (LOVI) method. . Ii-pixels ezaziwayo (i-Fig. • I-FIG YOKUFUNDA. Iingxaki ze-stenosis ziquka kuphela iingxaki ze-artery ze-stenosed, kodwa iingxaki ze-normals kunye ne-pixel ezikhuselekileyo kwi-artery, oku kubonisa ukuba umxholo phakathi kweingxaki ze-stenosed kunye ne-non-stenosed iingxaki ze-CathAI yokubonisa ukubaluleka kwe-stenosis. 17 4A 18 4B 6 I-GradCAM isetyenziswe kwi-CathAI yokuhlanganisa isakhiwo se-anatomic yokuqala. Iifoto ezimbini ze-angiogram zibonakalayo (ngaphandle), kunye nezithombe ezifanelekileyo ezibonakalisiweyo nge-GradCAM (ngaphandle) ezibonakalisa iindawo ezininzi ezininzi zonyango ze-algorithm. Iifoto ze-GradCAM ezibonakalisiweyo zihlanganisa malunga ne-artery ye-coronary ebonakalayo kwi-angiogram, kunye ne-blue ibonisa ubuncinane, i-yellow ibonisa ubuncinane obuncinane kunye ne-red ibonisa ubuncinane kakhulu kwi-CathAI Algorithm 2. LOVI Saliency Maps of CathAI prediction of coronary stenosis severity. Iimifanekiso angiogram yokuqala (top) kunye neimifanekiso ezinxulumene ne LOVI saliency maps (bottom). iipixel white ibonisa inkxaso enkulu yokuba CathAI (i-Algorithm 4) ukubonisa ukuba i-Algorithm 4 ibekwe kwiipixel ezininzi kwindawo ye-coronary artery stenosis. LOVI Layer Ordered Visualization of Information. a b Ukudibanisa I-CathAI ipayipi ye-Development and Validation eyenziwe ngama-algorithms ezine kunye ne-Demonstrated advances ezininzi kwi-Analysis ye-coronary angiograms. Yonke i-algorithm yenza umsebenzi elifanelekileyo ebonakalisa ukufikelela imfuneko ye-diagnostic primary ye-coronary angiography – ukulawula ukubaluleka kwe-stenosis ye-coronary artery – ukufikelela ukusebenza kwe-state-of-the-art yeenkcukacha zonyango zonke. Ngokubanzi, i-CathAI ibonelela ngokubanzi, ngaphandle kokufunda olungaphakathi, ukuhlaziywa i-stenosis ezibonakalayo evela kwi-non-curated real-world UOHI angiograms, i- Izixhobo ezidlulileyo zithunyelwe iinkonzo ze-neural ekusebenziseni iingxaki ze-individual ezihlangene ne-angiogram analysis njenge-frame extraction , ukuhlaziywa kwe-stenosis ukusuka kwezithombe ze-RCA (ngokukhetha ngempumelelo) , , , , , , I-Segmentation ye-Coronary Vessel , okanye ukucaciswa stenosis Nangona kunjalo, phantse lethu, ukuhlaziywa ngokupheleleyo okuzenzakalelayo kunye ne-stenosis ukuhlaziywa kwiividiyo ze-angiogram ye-coronary ayikho kwakhona kuboniswe kwi-angiogram ye-real-world. Ezininzi iinkqubo ezidlulileyo ze-experimental zihlanganisa kwi-RCA , , , , , , mhlawumbi ngenxa yokuba inezingxaki ezincinane ye-anatomical kunye ne-complexity kunye ne-LCA, ukunciphisa i-difficulty ye-analyzing angiograms ye-RCA. Nangona kunjalo, iingxaki ezidlulileyo kwi-angiograms e-RCA ezidlulileyo ayikwazi ukuyifaka iinkcukacha zehlabathi, eziquka iividiyo ze-non-RCA kunye ne-non-coronary artery. Kule nophando olutsha, i-Du et al. i-analyzed i-frames eyodwa ze-LCA kunye ne-RCA angiograms ukuyifaka iingxaki ze-coronary kunye ne-stenosis , kodwa ayikwazanga ubunzima i-stenosis, nto leyo iingxaki. Ukusebenza yethu inikeza i-state-of-the-art eyenziwa kwi-real-world angiograms ze-external kwaye ibonise i-standard-of-care-recommended "ngokuqhelekanga" ukulawula , Ngenxa ye-angiography ye-coronary kubalulekile ukuba zonke izixazululo zonyango ze-CHD , I-CathAI inesiseko enzima yokuthintela i-angiogram. I-Divariation yeengxaki ze-CathAI kwi-evaluation ye-visual ye-human experts yaba phantsi kwe-inter-observer variability, kwaye ezininzi ezininzi. , , , , i-CathAI's re-trained QCA deviation yi-6.5 ±5.5% kwi-RCT QCA dataset, leyo eliphantsi kakhulu kunokuba yi-10.2-16.6% ingxaki phakathi kwe-visual assessment ye-doktor kunye ne-QCA ezaziwa kwi-literature. 19 19 20 21 22 23 24 25 26 27 28 19 20 21 22 23 24 25 28 2 3 2 3 4 5 6 7 29 5 I-Analysis ye-angiogram ifumaneka kakhulu kwi-perception-side ye-technology ye-"self-driving car" kunokuba kwi-radiological analysis ye-standard (i-X-rays), kuba i-angiogram ye-high-variable, i-operator-determined video acquisitions kunye ne-sequence yeengxaki ezininzi ezininzi ezininzi ezininzi ezininzi ezininzi ezininzi ezininzi ezininzi ezininzi ezininzi. Ukuhlaziywa okuphumelela kwe-angiogram ye-real-world kufuneka i-AI-system ukuxhaswa zonke iintlobo zevidiyo ezivela kwiinkqubo ze-real-world, ukucinga iifoto ezinxulumene kunye nokuqinisekisa izicwangciso eziphambili. I-CathAI , , , , , , , inikeza i-proof-of-concept that multiple purpose-built deep learning algorithms can overcome the barriers that to date have prevented fully automated angiogram analysis. I-sub-tasks ezininzi i-CathAI ibonelela ngomhla lokugqibela ukuhlaziywa kwe-stenosis inikeza isakhiwo esinamandla yokusebenza ezininzi ezongezelelweyo ezizayo ezisekelwe kwi-CathAI sub-task, njenge-predicting stent under-expansion, i-aterosclerotic plaque morphology, okanye i-fractional flow reserve ukusuka kwimodeli ye-contrast flow. I-CathAI inokufumaneka ngempumelelo kwiimfuneko ezininzi. Njengomatshini ye-auto-driving, kunjalo, , Kwiziphi na ingxaki, ukuvelisa kwakhona kwiziphi na i-algorithms ye-CathAI ye-individual iya kuba kufuneka phambi kokufaka kwimeko ye-clinical, efumaneka umzekelo ukwandisa ubungakanani we-human-expert annotated datasets okanye ukuqeqesha i-algorithms eyenziwe ngempumelelo ngenxa yeengxaki ezizodwa (njenge-algorithm yokufaka i-artery localization kuphela ye-RCA-LAO). Le nqaku inikeza isiseko yokuphendula ngokukhawuleza okanye ukuvelisa i-algorithms kwizithuba ezininzi, njenge-re-training ye-QCA. 19 20 21 22 23 24 25 28 2 3 Imiphumo ye-clinical ephakeme kakhulu yokufaka i-pipeline efana ne-CathAI kufuneka ukwandisa i-standardization ekuphuhliseni i-stenosis ye-coronary. Ukubalwa kwe-visual ye-expert ye-stenosis ye-coronary kuqinisekiswe kakuhle ukuba i-variability ephakeme phakathi kwe-7-22% , , , , kwaye phakathi kwe-10-17% kwi QCA gold-standard Ukuphendula ngqo izixazululo kwi-CHD revascularization therapies ezihlabathi. Izifundo ezininzi zithunyelwe ngokuqhelekileyo ukuhlaziywa kwe-stenosis nge-estimation ye-visual , , Enye uphando i-re-evaluated angiograms ze-clinical kwi-multi-reader, i-reporting that the recommendation for coronary bypass surgery changed from “necessary/appropriate” to “uncertain/inappropriate” in 17-33% of cases, including 10% of cases for stent placement Ukucaciswa oku i-CathAI inokufuneka ukuvelisa i-angiogram ye-interpretation ye-second independent ngaphambi kwe-revascularization, nangona oku ngokuvamile engabonakali kwaye ayikho i-clinical-standard-of-care ekubeni iinkqubo zokusebenza ze-clinical interventional cardiology. Nangona kunjalo, i-CathAI inokufumana ngempumelelo lo mfuneko kwi-automated, i-replicable way to provide near-real-time predictions during the procedure to supplement physicians' own ad-hoc diagnosis. Nangona kunjalo, njengoko i-CathAI ibekwe kunye ne-stenosis eyenziwe ngexesha elifanelekileyo ye-visual estimated, ibekwe kwakhona ukuvelisa i-stenosis. Ukusombulula oku, i-C 4 5 6 7 29 5 5 6 8 4 4 Enye izifundo ezidlulileyo zibonisa ukuba kunokuba kukho iingxaki zesini kwi-stenosis imibuzo, apho i-stenosis ingathandwa kakhulu kumadoda. . To investigate whether CathAI could help reduce this bias in stenosis estimation across sex, we compared the performance of Algorithm 4 between males and females in our internal dataset. Our analysis found a small but significant difference between the algorithm’s performance in strata of males and females. Future work should aim to balance examples of severe and non-severe stenoses in both males and females to further address the issue of sex-based bias in stenosis severity estimation. 30 I-CathAI i-potency rethinking ye-UOCA kwiinkqubo ye-decrainographic ephakeme kakhulu yaye ibonise ukuba i-CathAI ibonise i-clinical angiograms ephakeme kakhulu kwi-real-world angiograms ephakeme yi-visual estimation ye-medicine. Ngokungafani, i-RCT QCA i-dataset ibonise kuphela indlela eyahlukileyo ye-pathogenesis ye-adjudication ye-RCTs. Kwi-angiograms ezininzi ze-UCSF ze-clinical angiograms, njengoko i-posenosis ephakeme kakhulu kwimeko ye-clinical, i-stenosis ye-coronary ≤50% ye-QCA vs. i-70% ye-visual estimation), Ukusetyenziswa kwebhizinisi lethu iintlobo ezininzi. I-limitation emangalisayo ifumaneka ukusetyenziswa kwimibelelwano yobugcisa esekelwe ngexesha lokugqibela lwezonyango lwezonyango usebenzisa i-visual estimation. Ngenxa yobungcali-intensive yobugcisa iinkcukacha ezininzi ze-cardiologist kwiifoto ze-angiographic, ukufumana iinkcukacha ezininzi ukuqeqesha i-CathAI ye-stenosis ye-stenosis, sisebenzisa izinga ze-RAPORT-stenosis ezisetyenziswa ngexabiso ezininzi. Nangona ezi ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye z Indawo lethu nge-proof-of-concept study yaba ukuba zibonise i-"building blocks" ezininzi ezininzi ezininzi ezininzi ezidlulileyo ye-interpretation ye-angiograms ye-coronary, kwaye akukwazi ukuguqula i-clinical standard yamanje ye-interpretation ye-angiograms. Ukufumana le nqaku, iimpumelelo emininzi yabantu iya kufuneka ukuba zibonise iinkqubo ezininzi ze-datasets. Ukongezelela, sincoma kuphela iindawo eziphambili ze-epicardial kule version ye-CathAI. Izinto ezininzi ezininzi ezifanelekileyo, njenge-bypass transplantations, i-diagonal, i-chronic total occlusions okanye i-collaterals ziye z 31 Ngokutsho, i-CathAI yinto yokuqala ye-multi-stage fully-automated analysis pipeline ye-angiogram ye-coronary. I-CathAI ibonelela kwimveliso ye-state-of-the-art yeenkcukacha zokusetyenziswa kwe-angiogram ye-real-world, ibonelela kwi-externally-valid, kwaye ibonelela isiseko yeemveliso ezininzi kwi-angiographic-interpretation ye-automated. Ukuphuhliswa kwe-stenosis eyenziwe yi-CathAI ingasetyenziswa ukwandisa i-standardization kunye ne-reproductibility kwi-stenosis ye-coronary, enye yeengxaki ezininzi ezininzi ekuphuhliswa kwe-CHD. Ukucinga Study abalandeli kunye nophando data sets Ukusetyenziswa kweDathaset yethu epheleleyo, sinikezela i-retrospective, i-de-identified coronary angiographic studies kumadoda omdala we-18 okanye ngaphezulu e-University of California, San Francisco (UCSF), phakathi kweAprili 1, 2008 kunye neDisemba 31, 2019 (i-Fig. ) that underwent a coronary angiogram. Patients without videos of the left or right coronary artery were excluded. Angiograms were acquired with Philips (Koninklijke Philips N.V., Amsterdam, Netherlands) and Siemens (Siemens Healthineers, Forchheim, Germany) systems at 15 frames per second using Iopromide contrast. We generated specifically annotated training datasets (either through available meta-data or expert annotation) of Full Dataset subsets for each of the four primary tasks performed by CathAI. 1 Ukuphucula iingcebiso ze-labelling ye-manual efunekayo ukuvelisa idatha ye-training ye-Algorithm eyodwa, sinikezela i-”extracted Full Dataset” ngokuvamile ngokuvela ngokuzenzakalelayo i-frames ngaphakathi kwividiyo ebonakalayo i-peak-contrast ngokucacisa i-structural similarity index measure (SSIM) ukusuka kwividiyo kwi-position ‘0’ apho akukho i-colour isetyenziswa, leyo ebizwa ngokuba yi-”reference” frame. I-SSIM iye aphezulu ukuba iifoto zihlanganisa i-pixel efana ne-difference ephakeme kakhulu. I-frame kunye ne-indice ye-similarity ebonakalayo kwi-frame ye-reference = 1,418,297). Zonke i-frames yevidiyo ziye kubhalwe kwi-images ye-512*512 i-pixel ye-algorithmic analysis. I-subset ye-frames evela kwi-Full Dataset eyenziwa ngexesha elandelayo, njengoko kulandelwe phantsi, ukuze yenza idatha yokufunda ye-Algorithms 1-3. n For all algorithms, except Algorithm 3, data was split randomly for each algorithm into Training (70%), Development (10%) and Test (20%) datasets, each containing non-overlapping patients. The development dataset was used for algorithm tuning, when required. For Algorithm 3, dataset splits were Training (80%) and Test (20%); since we used original hyperparameters and did not require algorithm tuning , . 16 32 I-Algorithm 1 i-labels ziye zithunyelwe ngqo kwi-DICOM metadata ebonakalayo i-craneal-caudal kunye ne-LAO-RAO orientations. I-Algorithm 2 kunye ne-3 zithunyelwe i-annotations ye-cardiologist ebonakalayo (i-Fig. I-Table ye-Extra Ukucinga and for definitions). For Algorithm 4, the stenoses were taken directly from the procedural report. 8 1 2 3 The Report dataset Okwangoku emva kokwenza inqubo, i-cardiologists-interventional ngokuvamile zihlanganisa i-angiogram ngokusebenzisa ukubuyekezwa kwe-visual, njengoko ngokuvamile lwezonyango lwezonyango, kwaye zihlanganisa ubunzima lwe-stenosis ye-coronary kwi-procedure report. Le nqaku le-procedure report lithathwe (bheka phantsi) ukuze zihlanganise: nayiphi na ukubuyekezwa kwe-stenosis ye-coronary artery, inqaku le-stenosis ephakeme (eyaziwa ngokuba yi- REPORT-stenosis) kunye nenkundla yayo kwi-one of 11 segments ye-coronary artery (i-Table Supplementary) ). We identified 9782 coronary stenoses in artery segments (REPORT-stenoses) and identified in 1766 non-stenosed complete vessels yielding a total of 10,088 non-stenosed vessel segments, derived from 84,153 images. Then we randomly sampled 10,000 images corresponding to healthy artery subsegments (Supplementary Fig. ). Metadata was extracted from each angiogram video including the procedure date, the primary (Right Anterior Oblique [RAO]/Left Anterior Oblique [LAO]) and secondary (cranio-caudal) angles of rotation, a unique exam identifier and a unique patient identifier. Non-matched REPORT-stenoses were removed from the dataset. We also excluded videos where an intra-coronary guidewire was present in more than 4 frames, as automatically determined by Algorithm 3 (6,076 videos, 41,780 stenosis-frames identified with guidewires), since these videos likely represent percutaneous coronary interventions which could alter the stenosis percentage within that video (Supplementary Fig. ); iividiyo ziquka izifundo ngaphambi kokufaka i-intracoronary guidewire. 3 1 1 Text parsing methods I-text free evela kwi-procedural report yenzelwe ekuqaleni ngokusebenzisa i-commas (“,”) okanye i-periods (“.”). Ngoko ke sisetyenziswa izindlela ze-text parsing ukuze zibonise iingxaki ze-coronary ezahlukileyo (i-Table Supplementary) ). Xa i-segment ye-coronary yaziwa, siye yaziwa zonke iinkcukacha ze-stenosis ezifanelekileyo ngokusebenzisa i-localizing "%" kunye neenombolo ye-1 ukuya kwi-three-digit kakhulu kwelinye ingxelo. Thina ekuqaleni kwenziwa ngokusebenzisa iingcebiso ze-standard (njenge- "equal coronary artery"), ngoko kwandisa iingcebiso nge-manual review of the text, nge- iterations ezininzi, ukuze kuquka iingcebiso ezininzi eziqhelekileyo, iingcebiso ze-alternate kunye ne-permutations (njenge-"RCA"). Iingcebiso ze-100% ze-stenosis ze-obstructive CHD, njenge-"mild", "moder Uyaziqhelekanga i-chronic total occlusions kunye ne-stenoses kwi-diagonal, i-marginal, i-septals, i-ramus, i-left posterior descending artery, i-left posterolateral okanye kwi-bypass transplant. 3 3 Ukucaciswa kwe-human subjects This study was reviewed by the University of California, San Francisco Institutional Review Board and need for informed consent was waived. The external validation was reviewed and approved by the University of Ottawa Institutional Review Board. Ukulungiselela i-algorithm I-CathAI system ifumaneka kwi-4 i-algorithms ye-neural network ezihlangene kwi-pipeline (i-Fig. ). Angiographic images are analyzed by each algorithm and “flow” sequentially to the next to accomplish the four foundational tasks for automated angiogram analysis: (1) classification of angiographic projection angle; (2) classification of an angiogram’s primary anatomic structure; (3) localization of relevant objects within an angiogram, including coronary artery sub-segments and stenoses; (4) prediction of coronary artery stenosis severity (as a percentage of artery narrowing). 1A I-Algorithm 1 ifumaneka imifanekiso eyodwa (i-coronary angiogram video frames) njenge-input kunye nokufumanisa i-angiographic projection angle esetyenziswa yi-LAO-RAO kunye ne-craneal-caudal axes (LAO cranial, RAO caudal, njl); i-etiquette ifumaneka kwi-metadata yevidiyo yehlabathi. I-Algorithm 2 ifumaneka isakhiwo yokuqala ye-anatomic (i-Table Supplementary) ), njengoko kubandakanya iividiyo ye-angiogram ezinezinto ze-anatomic non-cardiac ezifana ne-aorta okanye i-femoral artery. I-Algorithm 2 ivumela i-CathAI ukuyifaka kwiividiyo ze-angiogram kuphela ezininzi ezinezinto ze-left and right coronary arteries (LCA kunye ne-RCA, ngokulinganayo). I-Algorithm 3 iye zithunyelwe izicwangciso ezinxulumene ze-LCA kunye ne-RCA ngokusebenzisa izicwangciso ze-box boundary kwi-object ezaziwayo (i-Video Supplementary). Iindawo ye-stenosis ye-coronary iye ilawulwa ngokubanzi kweeyure ezimbini ze-Algorithm 3 zibonakalayo ze-bo ). I-Algorithm 4 ithathwe iifoto ezincinane kwi-arterial segments ezincinane (ngokusebenzisa i-Algorithm 3 bounding boxes) kwaye ithathwe i-stenosis yentlawulo ephakeme kwisithombe njenge-value epheleleyo phakathi kwe-0 kunye ne-100 ngalinye imifanekiso. Iifoto ziye zithunywe kwi-video ukuze inikeze ukuhlaziywa kwinqanaba le-video; kwaye ubunzima kwezifundo zeevidiyo zeevidiyo zeevidiyo zeevidiyo ezibonisa i-arterial segment ngaphakathi kwizifundo zithunyelwa ukuhlaziywa kwinqanaba le-artery. 2 1B I-Algorithm 1: Ukulinganiswa kwe-angiographic projection angle Algorithm 1 accepted individual images (video frames) as its input and identified the angiographic projection angle used. The projection angle refers to the fluoroscopic angulation used to obtain the image, commonly described on two axes defined by LAO-RAO and cranial-caudal (LAO cranial, RAO caudal, etc). For Algorithm 1 training data, all images from the extracted Full Dataset were categorized into 12 categories of left-right and cranio-caudal projection angles based on the primary and secondary angles extracted from each video’s metadata (−180 and 180 degrees for the primary angle and −50 and 50 degrees for secondary; Supplementary Table Emva koko, sincoma i-Full Dataset eyenziwe kwi-training (990,082), i-development (128,590) kunye ne-test datasets (299,625). 1 I-Algorithm 1 ye-architecture yayisekelwe kwi-Xception, i-convolutional neural network eyenziwe ngempumelelo kwi-state-of-the-art kwiingxaki ze-image recognition . Iye yasungulwa nge 'ImageNet' izicwangciso , as commonly performed to initialize weights for faster algorithm convergence in image classification settings; all layers were trainable. Images were augmented by random zoom (range=0.2) and shear rotate (range=0.2). The development dataset was used to iteratively compare algorithm performance and fine tune hyperparameters using grid search (Supplementary Table ). Sifundoza kunye ne-architectures ezahlukeneyo ezifana ne-VGG-16, i-ResNet50 kunye ne-InceptionNet-V3, kodwa asikwazi ukufumana izinzuzo ezininzi ngaphezu kwe-Xception. Ukusetyenziswa kwe-grid ukucacisa i-hyperparameter. I-Test Dataset ayisetyenziswa nangona ngexesha lokufunda kwaye isetyenziswe kuphela ukuba isebenze ukusebenza lokugqibela. I-prediction engapheliyo kwi-frames eyenziwe ngalinye yevidiyo ilawulwe njenge-angiographic projection yevidiyo; iingxaki zithathwe ngokuchofoza i-projection kunye ne-middle probability ephakeme kumgangatho bonke i-frames. Siza kusetyenziswa i-Algor 33 34 5 I-Algorithm 2: Ukulungiselela isakhiwo se-anatomic yokuqala I-Algorithm 2 yenzelwe ukucacisa isakhiwo se-anatomic eyona kwividiyo ye-angiographic (i-Table Supplementary) ), njengoko iividiyo ye-angiogram zihlanganisa izakhiwo ze-anatomy non-cardiac ezifana ne-aorta okanye i-femoral artery ngexesha le nqubo. Ukwenza iinkcukacha ze-Algorithm 2 yokufunda, sincoma ngempumelelo iifoto ze-14,366 kwi-Full Dataset eyongezelelweyo, kwaye i-cardiologist i-catalogged iifoto ngamnye kwi-one of the 11 classes describing the primary anatomic structure (i-Table Supplementary) ). Le dataset yahlukaniswa kwi-Training/Development/Test datasets, ebandakanya i-9887 (70%), i-1504 (10%), kunye ne-2 975 (20%) imifanekiso, ngokufanayo. Sitholisa i-Algorithm 2 usebenzisa i-Architecture ye-Xception, izicwangciso ze-Algorithm 1 ezidlulileyo, kunye ne-hyper-parameter eyenziwe (i-Table Supplementary) ). Iifoto zilungiswa yi-zoom ye-random (i-range = 0.2) kunye ne-shoar rotate (i-range = 0.2). Ukwakhiwa kwe-anatomic yokuqala ye-video yaba ukhuseleko lwe-mode yeenkcukacha zayo ze-Full Dataset ze-extracted. Iifoto kuphela ezininzi eziquka i-coronary arteries ezininzi okanye ezininzi ziye zithunyelwa kwi-Algorithm 3 ye-CathAI ye-analysis ezilandelayo (i-Fig. Ukulinganiswa kwe-F1 kunye nokusebenza kwe-model zihlanganisa kwi-class ye-anatomics, kodwa kwizilwanyana ezininzi kunye ne-frames ezincinane zithembisa (i-Fig. Supplementary. ), ukucacisa inkqubo yokuphucula ukusebenza ukuba idatha ezininzi ezaziwayo ziyafumaneka. 2 2 5 1 3B I-Algorithm 3: Lokalization yeengxaki ze-angiogram Algorithm 3 aimed to localize relevant objects within images of the left and right coronary arteries (the output of Algorithm 2). While Algorithm 3 was trained to localize multiple objects (Supplementary Table ), iingxaki ezininzi ezibalulekileyo kwi-CathAI pipeline ziya (i) ukucacisa iingxaki ze-coronary artery, (ii) ukucacisa i-stenosis (ngokufumaneka) kunye (iii) ukucacisa iimpawu ezininzi ezinxulumeneyo ezifana ne-guidewires okanye i-sternotomy. Ukwenza idatha yemfundo ye-Algorithm 3, i-2338, iifoto ze-contrast-containing ye-LCA kunye ne-RCA ngaphandle kwe-stenosis (ngokucaciswa yi-Algorithm 2) ziye zithunyelwe ngama-cardiologist eyenza iifoto ezininzi ezininzi ezifanelekileyo kwiifoto (i-Table Supplementary) ). I-stenosis kuphela kwiindawo eziphambili ze-epicardal, ngaphandle kweengxaki ze-side, njenge-diagonal okanye i-marginal, i-labeled. Kwi-100% ye-2338, i-LCA okanye i-RCA yaba isakhiwo yokuqala ye-anatomic ebonakalayo, kwaye i-artery iye yaziwa kakuhle, i-opacified kakuhle, kwaye ayikho i-underfilled, ngokutsho kwe-cardiologist ebonakalayo. 3 3 Kwinkqubo yethu lokugqibela ye-CathAI i-pipeline ethandwa iiyure ezimbini ye-Algorithm 3: I-Algorithm 3a iye yandiswa kunye nokufumana iifoto ze-LCA kunye ne-RCA njenge-input. Ngenxa yokuba i-RCA kwi-LAO ye-projection ibandakanya iifoto ezininzi ze-anotated kwi-dataset yethu, siye yandiswa i-Algorithm 3b ye-dedicated kwi-projection yaye yandisa iziphumo ezininzi ze-algorithm eyenza kwi-artery/projection ye-LAO (i-RCA kwi-LAO). Ukwenza ukufundisa i-Algorithms 3a/b, sisahlukanisa iifoto zethu ze-label I-algorithm 3a/b usebenzisa i-Architecture ye-RetinaNet kwaye ibandwe usebenzisa i-hyperparameter yokuqala ; a development dataset was not used. RetinaNet has achieved state-of-the-art performance for object localization such as the pedestrian detection for self-driving cars Ukusetyenziswa kwimeko yethu, i-Algorithms 3a/b output bounding boxes zihlanganisa iimeko ezininzi ezibonakalayo kwimifanekiso ye-input. Ngenxa yokuba ezinye iingxaki ze-artery kwiingcali zonyango ze-angiographic zaziwa a priori ukuba zibonakalayo okanye engabonakaliwe, sisetyenziswa i-post-hoc heuristic ukunceda ezinye iingxaki ze-artery ze-Algorithm 3a/b zibonakalayo kwiingxaki ze-angiographic njengoko zibonakalayo yi-Algorithm 1 (i-Table Supplementary) ). Oku kwakhona ibonisa ukuxhaswa kwezibuyekezo ezimbini ze-CathAI i-pipeline algorithms ukufumana ukusebenza kwe-pipeline epheleleyo. Ukuhlola ukusebenza kwe-Algorithm 3a/b, izibuyekezo ezidlulileyo zilinganiswa kunye nezibuyekezo ze-foot-truth usebenzisa i-ratio ye-area of intersection phezu kwindawo ye-union (eyaziwa ngokuba yi-Intersection-over-union [IoU]) . I-IoU ≥0.5 phakathi kwama-coordinates ezidlulileyo kunye ne-anotated ibizwa ngokuba yi-true positive. Emva koko, sincoma i-middle average precision (mAP), nto leyo ibonisa i-ratio ye-true positives kwi-true positives kunye ne-false positives kwiintlobo ezahlukeneyo ze-IoU, kumazwe ngamnye. I-mAP ye-50% ibandakanya iziphumo ze-state-of-the-art kwi-type ye-task , . 16 35 4 36 37 16 35 I-Algorithm 4: Ukuhlaziywa kwe-percentage of coronary artery stenosis I-Algorithm 4 yenzelwe ukuhlaziywa ukuhlaziywa kwe-stenosis ye-artery coronary njengentengiso, njengoko iifoto ze-input zihlanganisa malunga neengxaki ze-artery ze-Algorithm 3. I-Algorithm 3a/b yayisetyenziselwa zonke iividiyo ze-Report ze-dataset ukuze zibonise iingxaki ze-artery kunye ne-stenosis. Zonke iifoto ezine-stenosis ekubeni kwikhasi ye-artery coronary kunye ne-IoU ≥0.20 zihlanganisa iifomu ze-potential input ye-Algorithm 4. I-stenosis yayisetyenziswa kwi-artery ingxaki ze-Algorithm 3 ebonakalisa ukuba i-IoU ). Matched procedural report values served as labels to train Algorithm 4 with input images cropped around stenosed artery segments according to Algorithm 3a/b bounding boxes. Non-matched stenoses were removed from our dataset. We also excluded all videos where an intra-coronary guidewire was present in more than 4 frames, as automatically determined by Algorithm 3a/b (6,076 videos, 41,780 stenosis-frames identified with guidewires), since these videos likely represent percutaneous coronary interventions which could alter the stenosis percentage within that video (Supplementary Fig. ); iividiyo zithunyelwe kwizifundo ezidlulileyo ngaphambi kokufaka i-intracoronary guidewire. Ukuqeqesha kunye nokuqinisekisa i-Algorithm 4, sinxulumene i-6258 iifoto zeengxaki ze-coronary artery non-stenosed kunye ne-98,756 iifoto ze-stenoses. 1 1 Emva kokufumana i-stenosis, i-bounding box i-coordinates yandiswa ngama-12 ipixels kwiintlobo ezininzi, emva koko i-cuts kunye ne-re-size ukuya kwiintlobo ezintathu ezidlulileyo: i-256*256 ipixels (i-aspect ratio no.1), i-256*128 ipixels (i-aspect ratio no.2) kunye ne-128*256 ipixels (i-aspect ratio no.3). Oku yenzelwe ukwandisa i-signal-to-noise (i-vessel-to-background) ratio, ngenxa yeengxaki ezahlukeneyo ze-vascular orientations kunye ne-stenosis sizes. I-"Report Dataset" esetyenziselwa ukuqeqesha i-Algorithm 4 Ngokuba iingxaki zeengcingo ze-non-stenosed zilungele ukuba ziquka iingxaki zeengcingo zeengcingo, leyo ingxaki zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo zeengcingo. 1 I-Algorithm 4 ibekwe kwi-architecture ye-Xception eyenziwe ngexesha lokugqibela (i-Softmax layer, ezisetyenziswa yokuhlanganisa) yaye yaye yahlukile kwi-"average pool" yaye yahlukileyo kunye ne-linear activation function ukuze kufumanise ukuhlaziywa kwe-stenosis njenge-percentage value. I-imaging metadata ebandakanya i-coronary artery segment label kunye ne-croped aspect ratios ziye kubandakanya njengengxaki kwi-ultimate layer ye-Algorithm 4, nto leyo yandisa ukusebenza. I-algorithm ivumela i-stenosis i-percentage value phakathi kwe-0 kunye ne-100 ngalinye imifanekiso ebandakanya i-stenosis Iimifanekiso zithunyelwe ngempumelelo nge-flip ngempumelelo (both ephakathi kunye ne-vertical), i-contrast ngempumelelo, i-gamma kunye ne-brightness, ukusetyenziswa ngempumelelo kwi-adaptive histogram equalization (Ukuvelisa i-contrast kwiimifanekiso). Iimifanekiso zithunyelwe ukunciphisa ukusetyenziswa kwe-square phakathi kwe-predicted (AI-stenosis) kunye ne-report-stenosis usebenzisa i-RADAM optimizer with an initial learning rate of 0.001, momentum of 0.9 and batch size of 12, trained for 50 epochs. Training was halted when loss stopped improving for 8 consecutive epochs in the test dataset. 38 Kwimeko ye-Algorithm 4, i-modified i-training scheme ukuze zonke i-epoche ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye ziye zibe zibe zibe zibe zibe zibe zibe zibe zibe zibe zibe . Oku lithunyelwe ukuya ku-convergence. Sithunyelwe ukusebenza kwe-algorithm kwi-test dataset epheleleyo, ebandakanya iindidi ezintathu. Sithunyelwe ukuba ukuxhaswa kwe-input training ye-multi-size lithunyelwe njenge-algorithms ezininzi ezisetyenziswayo kwi-training. Sithunyelwe kwakhona iindlela ezahlukeneyo ze-pre-processing kunye ne-sequences ngaphandle kokuphucula ukusebenza kwe-algorithm (i-Table Supplementary) inguqulelo 39 7S Uhlobo lwe-Neural Network Explainability We applied two neural network explainability approaches to the fully-trained CathAI algorithms in order to better understand how algorithms made their predictions, respective to their relative tasks. The GradCAM ubuchwepheshe itheyibhile izixeko ze-image ezininzi ezininzi zokusetyenziswa. Izixeko ze-highlighted ze-red zihlanganisa ukubaluleka kwizixeko ze-algorithm, kwaye izixeko ze-highlighted ezininzi ze-blue zihlanganisa ukubaluleka ezincinane. Kwakhona sinikezela iimaphi ze-signature usebenzisa i-LOVI (Layer Ordered Visualization of Information) indlela , leyo ibonisa iipikseli ezahlukileyo ezahlukileyo ezahlukileyo kwi-algorithm predictions. Iipikseli ezihlangeneyo zibonisa iipikseli ezahlukileyo kwi-algorithm prediction. 17 18 External validation Ukusetyenziswa kwe-albenozyme ye-albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Albenozyme ye-Alben To train CathAI Algorithm 4 for the different population distribution of the QCA dataset, we split the QCA dataset into training (75%), development (12.5%) and testing (12.5%) and fine-tuned the last two fully-connected layers of Algorithm 4, to allow the algorithm to learn to predict the QCA stenosis values from the input stenosis images rather than visually assessed stenosis. We performed a grid-search of initial learning rates from 1e−4 to 1e−8 and selected the rate which produced the lowest loss value on the development set in 100 epochs. We then trained the model starting with a learning rate of 1e-6 and dropping by a factor of 0.1 every 100 epochs, for 300 epochs. I-Quantitative Coronary Angiography Dataset The CathAI system provides an algorithmic foundation to be re-trained for future angiogram-relevant tasks. To demonstrate this, we obtained an external dataset of coronary angiograms of an a priori different patient population adjudicated at the Montreal Heart Institute (MHI) Core laboratory using the CMS QCA system (MEDIS, Leiden, Netherlands). This dataset was comprised of angiograms analyzed by the MHI Angiographic Core lab, obtained during randomized controlled trials (RCT) in ≥18 year old patients that had a coronary angiography intervention as part of the study, and received novel lipid lowering drugs or placebo Iimvavanyo ezidlulileyo iinkcukacha ezisetyenziselwa iingcebiso zokusetyenziswa ziye zithintela izigulane kunye ne-obstructive coronary artery disease (CAD) ekuqaleni kweemvavanyo, ebonakalisa i-majority of mild-to-moderate severity coronary stenosis in this dataset. I-QCA analysis yenzelwe ngu-2 technicians ezidlulileyo kwaye ilawulwe yi-expert physician. Kule dataset, i-stenosis ebonakalayo ilawulwe njenge ≥50% QCA severity stenosis , . 40 6 13 Iifoto ze-angiogram ye-coronary ziye zithunyelwe ngokusebenzisa i-Philips (i-Koninklijke Philips N.V., Amsterdam, Netherlands), i-General Electric Medical Systems (i-General Electric, Chicago, Illinois, USA) kunye ne-Toshiba (i-Toshiba Corporation, Minato City, Tokyo, Japan) ngexesha le-15 iifoto ngenyanga, ngokunyuka i-Iopromide kwi-coronary arteries. Kwi-analyze ye-QCA ye-stenosis kwi-MHI core laboratory, i-end-diastolic frame ebonakalisiwe kunye ne-angulations ezininzi ezibonisa i-stenosis kwinqanaba yayo enzima kunye ne-pre-shortening ezincinane kunye ne-branch overlap. I-QCA software This RCT QCA dataset contained a different patient population a priori from the real-world clinical UCSF dataset, since all stenoses ≥ 50% were not present during the baseline angiogram (but could be present at a follow-up angiogram), leaving primarily mild-CAD with a mean QCA stenosis severity of 31.7% ± 11.6%. This provided an optimal opportunity to examine how CathAI could provide a foundation for retraining using QCA stenosis labels to function as an automated tool for core lab angiogram analysis. We split the QCA dataset into training (75%), development (12.5%) and testing (12.5%) datasets and fine-tuned the last two fully connected layers of CathAI Algorithm 4 to predict QCA values rather than visually assessed stenosis estimates. Ukubalwa i-algorithm kunye ne-statistical analysis Njengoko kufanelekileyo ngalinye umsebenzi, ukusebenza ka-algorithm yeeyure ze-catalogical ilawulwa ngokusebenzisa i-value predictive ye-positive (PPV), i-value predictive ye-negative, i-sensitivity, i-specificity, indawo phantsi kwe-receiver operating characteristic curve (AUC), i-F1 score kunye ne-Bland-Altmann plots. Ukuze i-values engapheliyo, sinikezela i-intra-class (ICC [2,2]) kunye ne-Pearson correlation kunye ne-average error absolute phakathi kwe-CathAI's stenosis prediction kunye ne-report stenosis okanye i-QCA stenosis. Zonke iiyunithi ezintathu zithunyelwe i-stenosis yentsholongwane yentsholongwane njengenxalenye ye-routine ye-clinical care. 41 Neural networks were trained using Keras v.2.24 and TensorFlow v.1.12. Final algorithms performance was reported in the Test Dataset. All analyses were performed using Python 2.7. I-algorithms 1 kunye ne-algorithms 2 zilinganiswa kwinqanaba le-frame/image ngokusebenzisa i-precision (i.e. i-value ye-predictive), i-recall (i-sensitivity) kunye ne-plot yokusebenza ngokusebenzisa i-matrices ye-confusion. Kwakhona sinikezela i-F1 ye-class, nto leyo i-middle ye-precision kunye ne-recall. Ukucacisa i-Algorithm 3a kunye ne-3b, sinikezela indawo ye-intersection phezu kwindawo ye-union (IoU) phakathi kwe-coordinates ye-bounding-box ezidlulileyo kunye ne-coordinates ye-bounding-box ezidlulileyo ze-expert yeengxaki zeengxaki zazo zonke iindidi kwi-test dataset. I-IoU yi-ratio phakathi kwindawo ye-overlay phezu kwindawo ye-union phakathi kweengxaki ze-coordinates ezidlulileyo kunye neengxaki ze-annotated . I-IoU≥0.5 ibonisa ubuncinane i-50% yindawo yokubandakanya phakathi kweengxaki ze-bounding efanelekileyo kunye neengxaki ze-true, leyo yenzelwe yi-true-positive. Ngoko ke, sinokuthumela ukusebenza kwe-Algorithm 3a/b njenge-medium precision (mAP) metric, leyo ibonisa inqaku yeengxaki ze-true-positive kunye neengxaki ze-true kunye neengxaki ze-false kwiintlobo ezahlukeneyo ze-IoU, kumazwe ngamazwe omnye . A mAP value of 50% compares with state-of-the-art results for this type of task , . We also present the mean average precision for algorithm 3a and algorithm 3b by calculating the proportion of correct class prediction with an IoU≥0.1 with our ground-truth labeling across all our classes in our test dataset, as well as the positive predictive value of stenosis localization using the report or QCA dataset as ground truth. 36 37 16 35 Ukucacisa i-Algorithm 4, i-metric yokuqala esithakazelisayo yaba i-average error absolute phakathi kwinqanaba ebhalisiweyo (i-REPORT-stenosis) kunye ne-value ebhalisiweyo (i-AI-stenosis) kwinqanaba ye-artery. Oku ibonelela i-guideline-based standard clinical practice for stenosis estimation, ngokusebenzisa ukucaciswa kwe-stenosis kwi-multiple orthogonal projections kwaye ibonelela i-maximum degree of stenosis narrowing , . I-AI-stenosis kwinqanaba ye-image yabaqhathaniswa kwiividiyo ukufumana i-AI-stenosis kwinqanaba ye-video kunye nokuqhathaniswa kunye ne- REPORT-stenosis usebenzisa i-error square average. I-Pearson kunye ne-Intra-class correlation (ICC) kunye neBland-Altman plots were then calculated between the REPORT-stenosis and AI-stenosis at the video-level and artery-level. The reliability was classified as poor ( < 0.5), moderate (0.50–0.75), good (0.76-0.90), or excellent (0.91-1.0).(52) Finally, we present the mean squared error between REPORT-stenosis and AI-stenosis at the video-level. 2 3 42 Ukubonisa i-binary "obstructive" coronary artery stenosis, ebonakalayo njenge ≥70% stenosis , Ukusetyenziswa kwe-F1, i-threshold ye-0,54 yenza i-F1 score. Ngokusekelwe kwalo, sinokuthunyelwa kwindawo phantsi kwe-receiver operating characteristic curve (AUC), i-sensitivity, i-specificity kunye ne-diagnostic odds-ratio , at the frame level, video level and artery level, based on this threshold. 2 3 43 I-Intervals ye-confidence ye-performance metrics yenzelwe yi-bootstrapping ye-80% yedatha ze-test kwi-1000 iterations ukufumana i-percentile ye-5 kunye ne-95. Sinikezela ukusebenza kwe-Algorithm 4 eyenziwe ngalinye, ngalinye kunye ne-coronary arteries, ngexesha le-artery kunye ne-age group. values for interaction (between the CathAI stenosis, the covariates and reported stenosis calculated by the Wald test). We categorized AI-stenosis and REPORT-stenosis in concordant and discordant lesion groups based on the visual ≥70% cutoff. For discordant lesions we present their prevalence, stratified by coronary artery segment. For lesion/vessel level data, a mixed effects logistic regression model as used to account for within-subject correlation and for repeated angiograms. P Reporting summary Ulwazi oluthe ngexabiso malunga ne-research design ifumaneka kwi ikhonkco kule nqaku. Izifundo zeNature Reporting Data Ukuphepha Iinkcukacha ezisetyenziswe kwinkcukacha zokusetyenziswa kwinkcukacha zokusetyenziswa kwezonyango zonyango kwaye ngoko akuyona ngokubanzi ngenxa yeengxaki ze-privacy data. Imibuzo efanelekileyo yokuxhumana usebenzisa iinkcukacha kunokwenzeka kumadokhumenti, njengoko kunokwenzeka kwaye i-Regents yeYunivesithi yaseCalifornia. ikhowudi Availability I-code ebonakalayo le nqaku i-copyright ye-Regents of the University of California kwaye ingatholakala nge-license. Iinkcukacha ezisetyenziselwa ukuqeqesha kunye nokuqinisekisa i-algorithm ayikwazi ukufumana ngokubanzi njengoko zinezinto zempilo zonyango. Ukucaciswa Virani, S. S. et al. Heart Disease and Stroke Statistics—2021 Update: A Report From the American Heart Association. , e254–e743 (2021). Circulation 143 Manesh, R. P. et al. ACC/AATS/AHA/ASE/ASNC/SCAI/SCCT/STS 2017 Appropriate Use Criteria for Coronary Revascularization in Patients With Stable Ischemic Heart Disease: A Report of the American College of Cardiology Appropriate Use Criteria Task Force, American Association for Thoracic Surgery, American Heart Association, American Society of Echocardiography, American Society of Nuclear Cardiology, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, and Society of Thoracic Surgeons. , 2212–2241 (2017). J. Am. Coll Cardiol 69 Levine, G. N. et al. 2011 ACCF/AHA/SCAI Guideline for Percutaneous Coronary Intervention: executive summary: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines and the Society for Cardiovascular Angiography and Interventions. , 2574–2609 (2011). Circulation 124 Leape, L. L. et al. Effect of variability in the interpretation of coronary angiograms on the appropriateness of use of coronary revascularization procedures. , 106–113 (2000). Am. Heart J. 139 Haibo, Z. et al. Comparison of physician visual assessment with quantitative coronary angiography in assessment of stenosis severity in China. , 239–247 (2018). JAMA Intern Med. 178 Rohan, S. et al. Comparison of visual assessment of coronary stenosis with independent quantitative coronary angiography: Findings from the PROMISE trial. , 1–9 (2017). Am. Heart J. 184 Kussmaul, W. G., Popp, R. L. & John, N. Accuracy and reproducibility of visual coronary stenosis estimates using information from multiple observers. , 154–162 (1992). Clin Cardiol 15 Nallamothu, B. K. et al. Comparison of clinical interpretation with visual assessment and quantitative coronary angiography in patients undergoing percutaneous coronary intervention in contemporary practice. , 1793–1800 (2013). Circulation 127 Elbaz-Greener, G., Masih, S., Fang, J., Roifman, I. & Wijeysundera, H. C. Temporal trends in fractional flow reserve use in patients undergoing coronary angiography: A population-based study. , 10–18 (2019). Cjc Open 1 Tonino, P. A. L. et al. Fractional flow reserve versus angiography for guiding percutaneous coronary intervention. , 213–224 (2009). New Engl J Medicine 360 Islam, Y. E. et al. Temporal trends in inpatient use of intravascular imaging among patients undergoing percutaneous coronary intervention in the United States. . , 913–915 (2018). JACC Cardiovasc. Interv 11 Parikh, R. V. et al. Utilization and outcomes of measuring fractional flow reserve in patients with stable ischemic heart disease. , 409–419 (2020). J. Am. CollegeCardiol. 75 Paolo, G. et al. Quantitative coronary angiography in the current era: principles and applications. , 527–536 (2009). J. Interv. Cardiol 22 Shah, R. et al. Comparison of visual assessment of coronary stenosis with independent quantitative coronary angiography: Findings from the Prospective Multicenter Imaging Study for Evaluation of Chest Pain (PROMISE) trial. , 1–9 (2017). Am. Heart J. 184 Langlais, É. L. et al. Novel artificial intelligence applications in cardiology: Current landscape, limitations, and the road to real-world applications. . 1–13 (2022). J. Cardiovasc. Transl https://doi.org/10.1007/s12265-022-10260-x Tsung-Yi, L., Priya, G., Ross, G., Kaiming, H. & Piotr, D. Focal loss for dense object detection. Preprint at (2017). https://arxiv.org/abs/1708.02002 Selvaraju, R. R. et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization. , 336–359 (2019). Int J Comput Vis 128 Nathan, M. T., Chen, B. Y. & Friedland, G. Efficient saliency maps for explainable AI. Preprint at (2019). https://arxiv.org/abs/1911.11293 Chengyang, Z. et al. Automated deep learning analysis of angiography video sequences for coronary artery disease. Preprint at (2021). https://arxiv.org/abs/2101.12505 Yang, S. et al. Major vessel segmentation on X-ray coronary angiography using deep networks with a novel penalty loss function. , 16897 (2019). Sci. Rep. 9 Nasr-Esfaha, E. et al. Segmentation of vessels in angiograms using convolutional neural networks. , 240–251 (2018). Biomed. Signal Process. Control 40 Benjamin, A. et al. Automated characterization of stenosis in invasive coronary angiography images with convolutional neural networks. Preprint at (2018). https://arxiv.org/abs/1807.10597 Athanasios, V. & Krystian, M. Deep segmentation and registration in X-ray angiography video. Preprint at (2018). https://arxiv.org/abs/1805.06406 Jingfan, F. et al. Multichannel fully convolutional network for coronary artery segmentation in X-Ray angiograms. , 44635–44643 (2018). IEEE Access 6 Moon, J. H. et al. Automatic stenosis recognition from coronary angiography using convolutional neural networks. , 105819 (2021). Comput Methods Programs Biomed. 198 Iyer, K. et al. AngioNet: a convolutional neural network for vessel segmentation in X-ray angiography. , 18066 (2021). Sci. Rep-uk 11 Yang, S. et al. Automatic coronary artery segmentation in X-ray angiograms by multiple convolutional neural networks. 31–35 (2018). Assoc. Comput. Mach. https://doi.org/10.1145/3195588.3195592 Du, T. et al. Training and validation of a deep learning architecture for the automatic analysis of coronary angiography. , 32–40 (2021). Eurointervention 17 Marcus, M. L. et al. Visual estimates of percent diameter coronary stenosis: “a battered gold standard.”. , 882–885 (1988). J. Am. Coll Cardiol. 11 Xu, X. et al. Sex differences in assessing stenosis severity between physician visual assessment and quantitative coronary angiography. , 9–14 (2022). Int. J. Cardiol. 348 Morris, P. D., Curzen, N. & Gunn, J. P. Angiography‐derived fractional flow reserve: More or less physiology? , e015586 (2020). J. Am. Heart Assoc. 9 Jaemin, S., Jun, P., Sang & Kyu-Hwan, J. Retinal vessel segmentation in fundoscopic images with generative adversarial networks. Preprint at (2017). https://arxiv.org/abs/1706.09318 Chollet, F. Xception: Deep Learning with Depthwise Separable Convolutions. Preprint at (2017). https://arxiv.org/abs/1610.02357 Alex, K., Ilya, S. & Geoffrey, E. H. ImageNet classification with deep convolutional neural networks. in 1097–1105 (2012). Advances in Neural Information Processing Systems Minh, H. et al. Deep RetinaNet-based detection and classification of road markings by visible light camera sensors. , 281 (2019). Sensors (Basel) 19 Rezatofighi, H. et al. Generalized intersection over union: A metric and a loss for bounding box regression. . 658–666 (2019). Conference: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/cvpr.2019.00075 Tie-Yan, L. Learning to rank for information retrieval. , 225–331 (2007). Found Trends Inf. Retr 3 Liyuan, L. et al. On the variance of the adaptive learning rate and beyond. Preprint at (2019). https://arxiv.org/abs/1908.03265 Kaiming, H., Xiangyu, Z., Shaoqing, R. & Jian, S. Spatial pyramid pooling in deep convolutional networks for visual recognition. 346–361 (2014). Lect Notes Comput. Sci. Tardif, J.-C. et al. Effects of reconstituted high-density lipoprotein infusions on coronary atherosclerosis: A randomized controlled trial. , 1675–1682 (2007). JAMA 297 Koo, T. K. & Li, M. Y. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. , 155–163 (2016). J. Chiropr. Med. 15 Bland, J. M. & Altman, D. G. Statistical methods for assessing agreement between two methods of clinical measurement. , 307–310 (1986). Lancet 327 Glas, A. S., Lijmer, J. G., Prins, M. H., Bonsel, G. J. & Bossuyt, P. M. M. The diagnostic odds ratio: a single indicator of test performance. , 1129–1135 (2003). J. Clin. Epidemiol. 56 Ukubuyiselwa I-RA ibekwe yi-"Fonds de la recherche en santé du Québec" (i-Grant 274831). I-JEO ibekwe i-NIH (U2CEB021881). I-GHT ibekwe yi-National Institutes of Health NHLBI K23HL135274. I-KS ayikho imveliso efanelekileyo ekubeni le nqaku. I-AW ibekwe yi-National Science Foundation Graduate Research Fellowship phantsi kwe-Grant No. DGE 1752814. I-YS ibekwe yi-Mid-Career Investigator Award ye-Heart and Stroke Foundation of Ontario. I-J-CT ibekwe i-Canada Research Chair kwi-personalized and translational medicine kunye neYunivesithi yaseMontreal ibekwe Le nqaku lula phantsi kwe-CC by 4.0 Deed (i-Attribution 4.0 International). This paper is under CC by 4.0 Deed (Attribution 4.0 International) license. available on nature