Prodvana is in the Cloud Infrastructure Industry

Prodvana

prodvana

Provenance Tags Founder Niels Soerensen Discusses Blockchain’s Role In Combating Counterfeiting

The global art market grew 6% in 2018 to 67.4 billion in sales according to economist Clare McAndrew’s report “The Art Market 2019,” released by Art Basel and UBS, however, despite the 6% growth, 57% of dealers saw their sales decline in 2018. This follows a trend over the past decade, during which the value of all sales has gone up 9%, while the number of artworks sold has gone down 9%. Just 0.2 percent of artists have work that sells for more than $10 million, 32 percent of the $63-plus billion in art sales in 2017 came from works that sold for more than $10 million. The burning question is despite the increase in sales, how come the members of this exclusive guild are still surviving on bare minimum? Art is not evenly distributed, artists today are obligated to art galleries to showcase their work which usually represents the well-established artists, and only a small number of new or still emerging artists have the support of art galleries. There are only a few independent platforms available for artists to showcase their work.Majority of the revenue is made by the art galleries as they demand as much as 50% as commission.Financial transactions are slow, galleries don’t pay artists on time.If an artist does decide to go solo they seldom do make it because of lack of networking or marketing skills due to which they become dependent on the galleries for their sale. The art industry is centralized and conservative because of which artists face consequences. Many art pieces get plagiarized and sold without the knowledge of artists as there is no system in this traditional industry that keeps a record of IP rights or the transactions taking place. BLOCKCHAIN AND ART The immutable nature of Blockchain technology is being widely adopted in diverse industries and one such interesting use case is Art! Merging of Blockchain technology with art has opened new horizons in the Blockchain revolution as well as opening new avenues for artists and collectors alike, it provides a solution to the problems faced by the artists today: Artists don’t require galleries to showcase their work and to attract buyers, blockchain is open and decentralized which means anyone in the network can view the art piece and directly get in touch with the seller, thereby reducing the need of intermediaries. Every transaction is cryptographically recorded in blockchain, it’s a transparent system which proves the authenticity of a work. Once something is copied its value decreases, so to keep the authenticity of an art piece it will be represented by a unique value which can not be copied hence preserving the art piece. It’s an open decentralized network which puts the artists in direct contact with the buyers thereby saving the large commission they have to pay the art galleries. Despite being transparent and recording every transaction it protects the security and enormity of the buyer which helps in increasing the confidence of both the buyer and the seller. When a tokenized painting is bought, this transaction is updated in the network which is available to everyone but the name of the buyer and the value of the token is concealed. ART TOKENIZATION & PROVENANCE By art tokenization, a monolithic painting is defragmented into thousands of digital tokens which are then issued to the buyers, it digitizes the ownership into tradable tokens thereby improving the liquidity of artwork. This transforms art into a lucrative investment option by authorizing fractional ownership. It was done for the first time in 2018 by The Maecenas – they tokenized Andy Warhol’s painting “14 Small Electric chairs”. Initially, the cost of the painting was approx 1.5 million dollars but after tokenization, the painting was sold at 5 million dollars. They organized a private dutch auction in which 100 participants took part, a total of 36 bids were received out of which two were unsuccessful. During the four week auction, 6+ million art tokens were utilized They are now in the process of tokenizing one of Pablo Picasso’s masterpieces that will be auctioned in open Ethereum platform. Offline assets are registered on the system which is converted into digital assets and assigned a unique token which ensures the copyright of the artist. Digital art chain is a platform that lets artists publish their digital art on an Ethereum run platform IPFS- InterPlanetary File System, each digital art uploaded is assigned a non-fungible token which signifies the uniqueness of an art piece. The art token (TAT) is another such platform that is linked to a collection of post-war and contemporary art. This genre has been the most stable from the past fifty years and offers the most return on investment (ROI). Physical artwork is tokenized and stored in a Swiss warehouse backed by the Swiss government. TAT can be bought using ETH, BTC, Dai, SWM, Dash or Fiat via swarm. Money is changed into a 1:1 ratio and the physical artwork is stored in the warehouses. TAT holders can now enjoy the benefits of a historically conservative investment class with newly added liquidity. The token can be either traded on eligible exchanges or redeemed for physical artworks. HOW DOES NFT HELP ? When a painting is tokenized, it is given a specific value which is unique, irreplaceable and uncopyable – these values are called non-fungible tokens. They provide foolproof digital security which prevents the plagiarism of the art piece. They are implemented on the Ethereum blockchain as ERC-721 tokens which have individual traits and identities. A fixed number of fungible tokens (ERC-20) separately represent the shared ownership of the physical painting. Buyers have the fractional ownership of the painting which makes the physical painting inaccessible to everyone, they are sold the digitized certificate of the painting which can be used as a liquid asset. So when a painting is tokenized “Crypto-art” is structured which is a liquid asset. Complete Architecture of a Art Tokenisation platform CONCLUSION Future is prolific to blockchain and art, not only this technology can protect artists by certifying the authenticity or helping artists to profit more from their creative artwork, but it’s also a new exciting source of inspiration! The art world is traditional but no industry can ignore the ascendency of blockchain. Art is becoming more and more accessible for buyers, once reserved to the elites who collected art for its aesthetic qualities or as an investment becoming more of something everyone can afford, blockchain takes its share in the blooming of the art industry. It’s trustless, decentralised, immutable nature makes it an attractive platform for the artists, investors and art collectors. Blockchain and art are still at an early stage, many critics claim it desensitises the art piece. There’s an invisible future ahead for the blockchain and art but one thing for certain, it is definitely going to be absorbing! Thanks for reading. Hopefully this guide has been useful to you and also we are working on like Art, Videos, Songs, e-books, Photography, Academics etc. dNFT, a open source protocol to tokenize the asset At QuillHash , we understand the Potential of Blockchain and have a good team of developers who can develop any blockchain applications like Smart Contracts, dApps, DeFi, DEX on the any Blockchain Platform like Ethereum , EOS and Hyperledger .

Of Art Tokenization, Blockchain, and Provenance with A Sprinkling of Non-Fungible Tokens

Did Craig Wright Fake His Supposed Bitcoin Meeting Notes? 

Forensic Experts Accuse Craig Wright of Manipulating Evidence To Support His Satoshi Claims

Building Advanced Video Search: Frame Search Versus Multi-Modal Embeddings

উন্নত ভিডিও অনুসন্ধান তৈরি করা: ফ্রেম অনুসন্ধান বনাম মাল্টি-মোডাল এম্বেডিং

Erstellen einer erweiterten Videosuche: Frame-Suche im Vergleich zu multimodalen Einbettungen

Creación de búsqueda de vídeo avanzada: búsqueda de fotogramas frente a incrustaciones multimodales

Création d'une recherche vidéo avancée : recherche de trames et intégrations multimodales

उन्नत वीडियो खोज का निर्माण: फ़्रेम खोज बनाम मल्टी-मोडल एम्बेडिंग

高度なビデオ検索の構築: フレーム検索とマルチモーダル埋め込み

고급 비디오 검색 구축: 프레임 검색과 다중 모드 임베딩 비교

Construindo pesquisa avançada de vídeo: pesquisa de quadros versus incorporações multimodais

Создание расширенного поиска видео: поиск кадров в сравнении с мультимодальными встраиваниями

Gelişmiş Video Arama Oluşturma: Çerçeve Aramaya Karşı Çok Modlu Yerleştirmeler

Xây dựng Tìm kiếm video nâng cao: Tìm kiếm khung so với nhúng đa phương thức

构建高级视频搜索：帧搜索与多模态嵌入

Swarm Summit 2024 Recap

Warp Core of Confidence: How Blockchain Creates Trust in the Digital Frontier

Authors:
(1) TIMNIT GEBRU, Black in AI;
(2) JAMIE MORGENSTERN, University of Washington;
(3) BRIANA VECCHIONE, Cornell University;
(4) JENNIFER WORTMAN VAUGHAN, Microsoft Research;
(5) HANNA WALLACH, Microsoft Research;
(6) HAL DAUMÉ III, Microsoft Research; University of Maryland;
(7) KATE CRAWFORD, Microsoft Research. Table of Links 1 Introduction 1.1 Objectives 2 Development Process 3 Questions and Workflow 3.1 Motivation 3.2 Composition 3.3 Collection Process 3.4 Preprocessing/cleaning/labeling 3.5 Uses 3.6 Distribution 3.7 Maintenance 4 Impact and Challenges Acknowledgments and References Appendix Acknowledgments We thank Peter Bailey, Emily Bender, Yoshua Bengio, Sarah Bird, Sarah Brown, Steven Bowles, Joy Buolamwini, Amanda Casari, Eric Charran, Alain Couillault, Lukas Dauterman, Leigh Dodds, Miroslav Dudík, Michael Ekstrand, Noémie Elhadad, Michael Golebiewski, Nick Gonsalves, Martin Hansen, Andy Hickl, Michael Hoffman, Scott Hoogerwerf, Eric Horvitz, Mingjing Huang, Surya Kallumadi, Ece Kamar, Krishnaram Kenthapadi, Emre Kiciman, Jacquelyn Krones, Erik Learned-Miller, Lillian Lee, Jochen Leidner, Rob Mauceri, Brian Mcfee, Emily McReynolds, Bogdan Micu, Margaret Mitchell, Sangeeta Mudnal, Brendan O’Connor, Thomas Padilla, Bo Pang, Anjali Parikh, Lisa Peets, Alessandro Perina, Michael Philips, Barton Place, Sudha Rao, Jen Ren, David Van Riper, Anna Roth, Cynthia Rudin, Ben Shneiderman, Biplav Srivastava, Ankur Teredesai, Rachel Thomas, Martin Tomko, Panagiotis Tziachris, Meredith Whittaker, Hans Wolters, Ashly Yeo, Lu Zhang, and the attendees of the Partnership on AI’s April 2019 ABOUT ML workshop for valuable feedback. References [1] Don A Andrews, James Bonta, and J Stephen Wormith. 2006. The recent past and near future of risk and/or need assessment. Crime & Delinquency 52, 1 (2006), 7–27. [2] Emily M. Bender and Batya Friedman. 2018. Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science. Transactions of the Association for Computational Linguistics 6 (2018), 587–604. [3] Anant P. Bhardwaj, Souvik Bhattacherjee, Amit Chavan, Amol Deshpande, Aaron J. Elmore, Samuel Madden, and Aditya G. Parameswaran. 2014. DataHub: Collaborative Data Science & Dataset Version Management at Scale. CoRR abs/1409.0798 (2014). [4] Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In Advances in Neural Information Processing Systems (NeurIPS). [5] Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*). 77–91. [6] Yang Trista Cao and Hal Daumé. 2020. Toward Gender-Inclusive Coreference Resolution. In Proceedings of the Conference of the Association for Computational Linguistics (ACL). abs/1910.13913. [7] Yang Trista Cao and Hal Daumé, III. 2020. Toward Gender-Inclusive Coreference Resolution. In Proceedings of the Conference of the Association for Computational Linguistics (ACL). [8] James Cheney, Laura Chiticariu, and Wang-Chiew Tan. 2009. Provenance in databases: Why, how, and where. Foundations and Trends in Databases 1, 4 (2009), 379–474. [9] Kasia Chmielinski, Sarah Newman, Matt Taylor, Josh Joseph, Kemi Thomas, Jessica Yurkofsky, and Yue Chelsea Qiu. 2020. The Dataset Nutrition Label (2nd Gen): Leveraging Context to Mitigate Harms in Artificial Intelligence. In NeurIPS Workshop on Dataset Curation and Security. [10] Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. QuAC : Question Answering in Context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. [11] Glennda Chui. 2017. Project will use AI to prevent or minimize electric grid failures. [Online; accessed 14-March-2018]. [12] Jeffrey Dastin. 2018. Amazon scraps secret AI recruiting tool that showed bias against women. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazonscraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G [13] Clare Garvie, Alvaro Bedoya, and Jonathan Frankle. 2016. The Perpetual Line-Up: Unregulated Police Face Recognition in America. Georgetown Law, Center on Privacy & Technology, New Jersey Ave NW, Washington, DC. [14] Michael Hind, Sameep Mehta, Aleksandra Mojsilovic, Ravi Nair, Karthikeyan Natesan Ramamurthy, Alexandra Olteanu, and Kush R. Varshney. 2018. Increasing Trust in AI Services through Supplier’s Declarations of Conformity. CoRR abs/1808.07261 (2018). [15] Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miroslav Dudík, and Hanna M. Wallach. 2019. Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?. In 2019 ACM CHI Conference on Human Factors in Computing Systems. [16] Gary B Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. 2007. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49. University of Massachusetts Amherst. [17] Ivan Krasin, Tom Duerig, Neil Alldrin, Vittorio Ferrari, Sami Abu-El-Haija, Alina Kuznetsova, Hassan Rom, Jasper Uijlings, Stefan Popov, Shahab Kamali, Matteo Malloci, Jordi Pont-Tuset, Andreas Veit, Serge Belongie, Victor Gomes, Abhinav Gupta, Chen Sun, Gal Chechik, David Cai, Zheyun Feng, Dhyanesh Narayanan, and Kevin Murphy. 2017. OpenImages: A public dataset for large-scale multi-label and multi-class image classification. [18] Tom CW Lin. 2012. The new investor. UCLA Law Review 60 (2012), 678. [19] G Mann and C O’Neil. 2016. Hiring Algorithms Are Not Neutral. https://hbr.org/2016/12/hiring-algorithms-are-not-neutral. [20] Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*). 220–229. [21] Mary Catherine O’Connor. 2017. How AI Could Smarten Up Our Water System. [Online; accessed 14-March-2018]. [22] Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics. 271. [23] Ismaïla Seck, Khouloud Dahmane, Pierre Duthon, and Gaëlle Loosli. 2018. Baselines and a datasheet for the Cerema AWP dataset. CoRR abs/1806.04016 (2018). http://arxiv.org/abs/1806.04016 [24] Doha Suppy Systems. 2017. Facial Recognition. [Online; accessed 14-March-2018]. [25] World Economic Forum Global Future Council on Human Rights 2016–2018. 2018. How to Prevent Discriminatory Outcomes in Machine Learning. https://www.weforum.org/whitepapers/how-to-prevent-discriminatory-outcomes-inmachine-learning. [26] Semih Yagcioglu, Aykut Erdem, Erkut Erdem, and Nazli Ikizler-Cinbis. 2018. RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipes. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing This paper is available on arxiv under CC 4.0 license. Authors: (1) TIMNIT GEBRU, Black in AI; (2) JAMIE MORGENSTERN, University of Washington; (3) BRIANA VECCHIONE, Cornell University; (4) JENNIFER WORTMAN VAUGHAN, Microsoft Research; (5) HANNA WALLACH, Microsoft Research; (6) HAL DAUMÉ III, Microsoft Research; University of Maryland; (7) KATE CRAWFORD, Microsoft Research. Authors: Authors: (1) TIMNIT GEBRU, Black in AI; (2) JAMIE MORGENSTERN, University of Washington; (3) BRIANA VECCHIONE, Cornell University; (4) JENNIFER WORTMAN VAUGHAN, Microsoft Research; (5) HANNA WALLACH, Microsoft Research; (6) HAL DAUMÉ III, Microsoft Research; University of Maryland; (7) KATE CRAWFORD, Microsoft Research. Table of Links 1 Introduction 1 Introduction 1.1 Objectives 1.1 Objectives 2 Development Process 2 Development Process 3 Questions and Workflow 3 Questions and Workflow 3.1 Motivation 3.1 Motivation 3.2 Composition 3.2 Composition 3.3 Collection Process 3.3 Collection Process 3.4 Preprocessing/cleaning/labeling 3.4 Preprocessing/cleaning/labeling 3.5 Uses 3.5 Uses 3.6 Distribution 3.6 Distribution 3.7 Maintenance 3.7 Maintenance 4 Impact and Challenges 4 Impact and Challenges Acknowledgments and References Acknowledgments and References Appendix Appendix Acknowledgments We thank Peter Bailey, Emily Bender, Yoshua Bengio, Sarah Bird, Sarah Brown, Steven Bowles, Joy Buolamwini, Amanda Casari, Eric Charran, Alain Couillault, Lukas Dauterman, Leigh Dodds, Miroslav Dudík, Michael Ekstrand, Noémie Elhadad, Michael Golebiewski, Nick Gonsalves, Martin Hansen, Andy Hickl, Michael Hoffman, Scott Hoogerwerf, Eric Horvitz, Mingjing Huang, Surya Kallumadi, Ece Kamar, Krishnaram Kenthapadi, Emre Kiciman, Jacquelyn Krones, Erik Learned-Miller, Lillian Lee, Jochen Leidner, Rob Mauceri, Brian Mcfee, Emily McReynolds, Bogdan Micu, Margaret Mitchell, Sangeeta Mudnal, Brendan O’Connor, Thomas Padilla, Bo Pang, Anjali Parikh, Lisa Peets, Alessandro Perina, Michael Philips, Barton Place, Sudha Rao, Jen Ren, David Van Riper, Anna Roth, Cynthia Rudin, Ben Shneiderman, Biplav Srivastava, Ankur Teredesai, Rachel Thomas, Martin Tomko, Panagiotis Tziachris, Meredith Whittaker, Hans Wolters, Ashly Yeo, Lu Zhang, and the attendees of the Partnership on AI’s April 2019 ABOUT ML workshop for valuable feedback. References [1] Don A Andrews, James Bonta, and J Stephen Wormith. 2006. The recent past and near future of risk and/or need assessment. Crime & Delinquency 52, 1 (2006), 7–27. [2] Emily M. Bender and Batya Friedman. 2018. Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science. Transactions of the Association for Computational Linguistics 6 (2018), 587–604. [3] Anant P. Bhardwaj, Souvik Bhattacherjee, Amit Chavan, Amol Deshpande, Aaron J. Elmore, Samuel Madden, and Aditya G. Parameswaran. 2014. DataHub: Collaborative Data Science & Dataset Version Management at Scale. CoRR abs/1409.0798 (2014). [4] Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In Advances in Neural Information Processing Systems (NeurIPS). [5] Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*). 77–91. [6] Yang Trista Cao and Hal Daumé. 2020. Toward Gender-Inclusive Coreference Resolution. In Proceedings of the Conference of the Association for Computational Linguistics (ACL). abs/1910.13913. [7] Yang Trista Cao and Hal Daumé, III. 2020. Toward Gender-Inclusive Coreference Resolution. In Proceedings of the Conference of the Association for Computational Linguistics (ACL). [8] James Cheney, Laura Chiticariu, and Wang-Chiew Tan. 2009. Provenance in databases: Why, how, and where. Foundations and Trends in Databases 1, 4 (2009), 379–474. [9] Kasia Chmielinski, Sarah Newman, Matt Taylor, Josh Joseph, Kemi Thomas, Jessica Yurkofsky, and Yue Chelsea Qiu. 2020. The Dataset Nutrition Label (2nd Gen): Leveraging Context to Mitigate Harms in Artificial Intelligence. In NeurIPS Workshop on Dataset Curation and Security. [10] Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. QuAC : Question Answering in Context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. [11] Glennda Chui. 2017. Project will use AI to prevent or minimize electric grid failures. [Online; accessed 14-March-2018]. [12] Jeffrey Dastin. 2018. Amazon scraps secret AI recruiting tool that showed bias against women. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazonscraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G [13] Clare Garvie, Alvaro Bedoya, and Jonathan Frankle. 2016. The Perpetual Line-Up: Unregulated Police Face Recognition in America. Georgetown Law, Center on Privacy & Technology, New Jersey Ave NW, Washington, DC. [14] Michael Hind, Sameep Mehta, Aleksandra Mojsilovic, Ravi Nair, Karthikeyan Natesan Ramamurthy, Alexandra Olteanu, and Kush R. Varshney. 2018. Increasing Trust in AI Services through Supplier’s Declarations of Conformity. CoRR abs/1808.07261 (2018). [15] Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miroslav Dudík, and Hanna M. Wallach. 2019. Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?. In 2019 ACM CHI Conference on Human Factors in Computing Systems. [16] Gary B Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. 2007. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49. University of Massachusetts Amherst. [17] Ivan Krasin, Tom Duerig, Neil Alldrin, Vittorio Ferrari, Sami Abu-El-Haija, Alina Kuznetsova, Hassan Rom, Jasper Uijlings, Stefan Popov, Shahab Kamali, Matteo Malloci, Jordi Pont-Tuset, Andreas Veit, Serge Belongie, Victor Gomes, Abhinav Gupta, Chen Sun, Gal Chechik, David Cai, Zheyun Feng, Dhyanesh Narayanan, and Kevin Murphy. 2017. OpenImages: A public dataset for large-scale multi-label and multi-class image classification. [18] Tom CW Lin. 2012. The new investor. UCLA Law Review 60 (2012), 678. [19] G Mann and C O’Neil. 2016. Hiring Algorithms Are Not Neutral. https://hbr.org/2016/12/hiring-algorithms-are-not-neutral. [20] Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*). 220–229. [21] Mary Catherine O’Connor. 2017. How AI Could Smarten Up Our Water System. [Online; accessed 14-March-2018]. [22] Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics. 271. [23] Ismaïla Seck, Khouloud Dahmane, Pierre Duthon, and Gaëlle Loosli. 2018. Baselines and a datasheet for the Cerema AWP dataset. CoRR abs/1806.04016 (2018). http://arxiv.org/abs/1806.04016 [24] Doha Suppy Systems. 2017. Facial Recognition. [Online; accessed 14-March-2018]. [25] World Economic Forum Global Future Council on Human Rights 2016–2018. 2018. How to Prevent Discriminatory Outcomes in Machine Learning. https://www.weforum.org/whitepapers/how-to-prevent-discriminatory-outcomes-inmachine-learning. [26] Semih Yagcioglu, Aykut Erdem, Erkut Erdem, and Nazli Ikizler-Cinbis. 2018. RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipes. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Using Language Models to Simulate Human Samples: Acknowledgments and References

Authors:
(1) TIMNIT GEBRU, Black in AI;
(2) JAMIE MORGENSTERN, University of Washington;
(3) BRIANA VECCHIONE, Cornell University;
(4) JENNIFER WORTMAN VAUGHAN, Microsoft Research;
(5) HANNA WALLACH, Microsoft Research;
(6) HAL DAUMÉ III, Microsoft Research; University of Maryland;
(7) KATE CRAWFORD, Microsoft Research. Table of Links 1 Introduction 1.1 Objectives 2 Development Process 3 Questions and Workflow 3.1 Motivation 3.2 Composition 3.3 Collection Process 3.4 Preprocessing/cleaning/labeling 3.5 Uses 3.6 Distribution 3.7 Maintenance 4 Impact and Challenges Acknowledgments and References Appendix 1 Introduction Data plays a critical role in machine learning. Every machine learning model is trained and evaluated using data, quite often in the form of static datasets. The characteristics of these datasets fundamentally influence a model’s behavior: a model is unlikely to perform well in the wild if its deployment context does not match its training or evaluation datasets, or if these datasets reflect unwanted societal biases. Mismatches like this can have especially severe consequences when machine learning models are used in high-stakes domains, such as criminal justice [1, 13, 24], hiring [19], critical infrastructure [11, 21], and finance [18]. Even in other domains, mismatches may lead to loss of revenue or public relations setbacks. Of particular concern are recent examples showing that machine learning models can reproduce or amplify unwanted societal biases reflected in training datasets [4, 5, 12]. For these and other reasons, the World Economic Forum suggests that all entities should document the provenance, creation, and use of machine learning datasets in order to avoid discriminatory outcomes [25]. Although data provenance has been studied extensively in the databases community [3, 8], it is rarely discussed in the machine learning community. Documenting the creation and use of datasets has received even less attention. Despite the importance of data to machine learning, there is currently no standardized process for documenting machine learning datasets. To address this gap, we propose datasheets for datasets. In the electronics industry, every component, no matter how simple or complex, is accompanied with a datasheet describing its operating characteristics, test results, recommended usage, and other information. By analogy, we propose that every dataset be accompanied with a datasheet that documents its motivation, composition, collection process, recommended uses, and so on. Datasheets for datasets have the potential to increase transparency and accountability within the machine learning community, mitigate unwanted societal biases in machine learning models, facilitate greater reproducibility of machine learning results, and help researchers and practitioners to select more appropriate datasets for their chosen tasks. After outlining our objectives below, we describe the process by which we developed datasheets for datasets. We then provide a set of questions designed to elicit the information that a datasheet for a dataset might contain, as well as a workflow for dataset creators to use when answering these questions. We conclude with a summary of the impact to date of datasheets for datasets and a discussion of implementation challenges and avenues for future work. This paper is available on arxiv under CC 4.0 license. Authors: (1) TIMNIT GEBRU, Black in AI; (2) JAMIE MORGENSTERN, University of Washington; (3) BRIANA VECCHIONE, Cornell University; (4) JENNIFER WORTMAN VAUGHAN, Microsoft Research; (5) HANNA WALLACH, Microsoft Research; (6) HAL DAUMÉ III, Microsoft Research; University of Maryland; (7) KATE CRAWFORD, Microsoft Research. Authors: Authors: (1) TIMNIT GEBRU, Black in AI; (2) JAMIE MORGENSTERN, University of Washington; (3) BRIANA VECCHIONE, Cornell University; (4) JENNIFER WORTMAN VAUGHAN, Microsoft Research; (5) HANNA WALLACH, Microsoft Research; (6) HAL DAUMÉ III, Microsoft Research; University of Maryland; (7) KATE CRAWFORD, Microsoft Research. Table of Links 1 Introduction 1 Introduction 1.1 Objectives 1.1 Objectives 2 Development Process 2 Development Process 3 Questions and Workflow 3 Questions and Workflow 3.1 Motivation 3.1 Motivation 3.2 Composition 3.2 Composition 3.3 Collection Process 3.3 Collection Process 3.4 Preprocessing/cleaning/labeling 3.4 Preprocessing/cleaning/labeling 3.5 Uses 3.5 Uses 3.6 Distribution 3.6 Distribution 3.7 Maintenance 3.7 Maintenance 4 Impact and Challenges 4 Impact and Challenges Acknowledgments and References Acknowledgments and References Appendix Appendix 1 Introduction Data plays a critical role in machine learning. Every machine learning model is trained and evaluated using data, quite often in the form of static datasets. The characteristics of these datasets fundamentally influence a model’s behavior: a model is unlikely to perform well in the wild if its deployment context does not match its training or evaluation datasets, or if these datasets reflect unwanted societal biases. Mismatches like this can have especially severe consequences when machine learning models are used in high-stakes domains, such as criminal justice [1, 13, 24], hiring [19], critical infrastructure [11, 21], and finance [18]. Even in other domains, mismatches may lead to loss of revenue or public relations setbacks. Of particular concern are recent examples showing that machine learning models can reproduce or amplify unwanted societal biases reflected in training datasets [4, 5, 12]. For these and other reasons, the World Economic Forum suggests that all entities should document the provenance, creation, and use of machine learning datasets in order to avoid discriminatory outcomes [25]. Although data provenance has been studied extensively in the databases community [3, 8], it is rarely discussed in the machine learning community. Documenting the creation and use of datasets has received even less attention. Despite the importance of data to machine learning, there is currently no standardized process for documenting machine learning datasets. To address this gap, we propose datasheets for datasets. In the electronics industry, every component, no matter how simple or complex, is accompanied with a datasheet describing its operating characteristics, test results, recommended usage, and other information. By analogy, we propose that every dataset be accompanied with a datasheet that documents its motivation, composition, collection process, recommended uses, and so on. Datasheets for datasets have the potential to increase transparency and accountability within the machine learning community, mitigate unwanted societal biases in machine learning models, facilitate greater reproducibility of machine learning results, and help researchers and practitioners to select more appropriate datasets for their chosen tasks. After outlining our objectives below, we describe the process by which we developed datasheets for datasets. We then provide a set of questions designed to elicit the information that a datasheet for a dataset might contain, as well as a workflow for dataset creators to use when answering these questions. We conclude with a summary of the impact to date of datasheets for datasets and a discussion of implementation challenges and avenues for future work. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Out of One, Many: Using Language Models to Simulate Human Samples

Authors:
(1) Sasun Hambardzumyan, Activeloop, Mountain View, CA, USA;
(2) Abhinav Tuli, Activeloop, Mountain View, CA, USA;
(3) Levon Ghukasyan, Activeloop, Mountain View, CA, USA;
(4) Fariz Rahman, Activeloop, Mountain View, CA, USA;.
(5) Hrant Topchyan, Activeloop, Mountain View, CA, USA;
(6) David Isayan, Activeloop, Mountain View, CA, USA;
(7) Mark McQuade, Activeloop, Mountain View, CA, USA;
(8) Mikayel Harutyunyan, Activeloop, Mountain View, CA, USA;
(9) Tatevik Hakobyan, Activeloop, Mountain View, CA, USA;
(10) Ivo Stranic, Activeloop, Mountain View, CA, USA;
(11) Davit Buniatyan, Activeloop, Mountain View, CA, USA. Table of Links Abstract and Intro
Current Challenges
Tensor Storage Format
Deep Lake System Overview
Machine Learning Use Cases
Performance Benchmarks
Discussion and Limitations
Related Work
Conclusions, Acknowledgement, and References 3. TENSOR STORAGE FORMAT Deep Lake datasets follow columnar storage architecture, with tensors as columns, as shown in Fig. 3. Each tensor is a collection of chunks - binary blobs that contain the data samples. An index map associated with each tensor helps find the right chunk and index of the sample within that chunk for a given sample index. 3.1 Dataset A sample in a dataset represents a single row indexed across parallel tensors. As opposed to a document storage format, sample elements are logically independent, which enables partial access to samples for running performant queries or streaming selected tensors over the network to the GPU training instances. Multiple tensors can be grouped. Groups implement syntactic nesting and define how tensors are related to each other. Syntactic nesting avoids the format complication for hierarchical memory layout. Changes to the dataset’s schema are also tracked over time with version control, similar to dataset content changes. 3.2 Tensors Tensors are typed and can be appended or modified in-place. Default access to an index or a set of indices returns the data as NumPy arrays [55]. Instead of storing 1-D data as seen in Parquet [79] or series in Arrow [13], tensors can accommodate n-dimensional data, where typically the first dimension corresponds to the index or batch dimension. Tensors can contain dynamically shaped arrays, also called ragged tensors, as opposed to other statically chunked array formats such as Zarr [52]. 3.3 Types Htype defines the expectations on samples in a tensor such as data type (dtype as seen in NumPy [55]), shape, number of dimensions, or compression. Typed tensors make interacting with deep learning frameworks straightforward and enable sanity checks and efficient memory layout. By inheriting from a generic tensor htype, we can construct types such as image, video, audio, bbox, dicom, and others. For example, a tensor with image htype would expect samples being appended to it to have dtype as uint8 and shape length 3 (i.e. width, height and number of channels). We further expand on the notion of htypes allowing for meta types that support storing image sequences in tensors (sequence[image]), referencing to remotely stored images, while maintaining the regular behavior of a image tensor (link[image]), or even possible cross-format support. 3.4 Memory Layout A Deep Lake dataset contains a provenance file in JSON format and folders per tensor. A tensor contains chunks, chunk encoder, tile encoder, and tensor metadata. Tensors can be optionally hidden. For instance, hidden tensors can be used to maintain down-sampled versions of images or preserve shape information for fast queries. Tensors are stored in chunks at the storage level. While statically (inferred) shaped chunking avoids maintaining a chunk map table, it introduces significant user overhead during the specification of the tensor, custom compression usage limitations, underutilized storage for dynamically shaped tensors, and post-processing inefficiencies. Deep Lake chunks are constructed based on the lower and upper bound of the chunk size to fit a limited number of samples. This comes with a trade-off of having a compressed index map that preserves the sample index to chunk id mapping per tensor while enabling chunk sizes in the range optimal for streaming while accommodating mixed-shape samples. One could consider the approach taken in this paper as an optimized trade-off between file system page map and compute-defined map-less array storage system. For practical reasons, a single chunk encoder can be scaled to billions of images while maintaining a 150MB chunk encoder per 1PB tensor data. Further scaling can be introduced by sharding the chunk encoder. Chunks contain header information such as byte ranges, shapes of the samples, and the sample data itself. If a sample is larger than the upper bound chunk size, which is the case for large aerial or microscopy images, the sample is tiled into chunks across spatial dimensions. The only exception to tiling is videos. Videos are preserved due to efficient frame mapping to indices, key-frame-only decompression, and range-based requests while streaming. 3.5 Access Patterns The tensor storage format is optimized for deep learning training and inference, including sequential and random access. Sequential access is used for running scan queries, transforming tensors into other tensors, or running inference. Random access use cases include multiple annotators writing labels to the same image or models storing back predictions along with the dataset. While the strict mode is disabled, out-of-the-bounds indices of a tensor can be assigned, thus accommodating sparse tensors. However, random assignment over time will produce inefficiently stored data chunks. To fix the data layout, we implement an on-the-fly re-chunking algorithm to optimize the data layout. One of the key access patterns of Deep Lake is shuffled stream access for training machine learning models. It requires random or custom order access while streaming chunks into the training process. This is achieved by involving range-based requests to access sub-elements inside chunks, running complex queries before training to determine the order, and maintaining a buffer cache of fetched and unutilized data. This avoids having a separate compute cluster for running shuffling algorithm [50]. Each tensor has its own chunks, and the default chunk size is 8MB. A single chunk consists of data from multiple indices when the individual data points (image, label, annotation, etc.) are smaller than the chunk size. Conversely, when individual data points are larger than the chunk size, the data is split among multiple chunks (tiling). Exceptions to chunking logic are video data. Deep Lake format is optimized for maximizing throughput to GPU processing. It includes CPU pre-fetching, decompression or decoding, transformations, and GPU memory transfer in a deep learning framework’s expected layout. 3.6 Storage Providers Deep Lake can be plugged into any storage provider, including object storages such as AWS S3 [1], Google Cloud Storage (GCS) [3], POSIX compatible file systems, or local in-memory storage. Moreover, it constructs memory caching by chaining various storage providers together, for instance - the Least Recently Used (LRU) cache of remote S3 storage with local in-memory data. This paper is available on arxiv under CC 4.0 license. Authors: (1) Sasun Hambardzumyan, Activeloop, Mountain View, CA, USA; (2) Abhinav Tuli, Activeloop, Mountain View, CA, USA; (3) Levon Ghukasyan, Activeloop, Mountain View, CA, USA; (4) Fariz Rahman, Activeloop, Mountain View, CA, USA;. (5) Hrant Topchyan, Activeloop, Mountain View, CA, USA; (6) David Isayan, Activeloop, Mountain View, CA, USA; (7) Mark McQuade, Activeloop, Mountain View, CA, USA; (8) Mikayel Harutyunyan, Activeloop, Mountain View, CA, USA; (9) Tatevik Hakobyan, Activeloop, Mountain View, CA, USA; (10) Ivo Stranic, Activeloop, Mountain View, CA, USA; (11) Davit Buniatyan, Activeloop, Mountain View, CA, USA. Authors: Authors: (1) Sasun Hambardzumyan, Activeloop, Mountain View, CA, USA; (2) Abhinav Tuli, Activeloop, Mountain View, CA, USA; (3) Levon Ghukasyan, Activeloop, Mountain View, CA, USA; (4) Fariz Rahman, Activeloop, Mountain View, CA, USA;. (5) Hrant Topchyan, Activeloop, Mountain View, CA, USA; (6) David Isayan, Activeloop, Mountain View, CA, USA; (7) Mark McQuade, Activeloop, Mountain View, CA, USA; (8) Mikayel Harutyunyan, Activeloop, Mountain View, CA, USA; (9) Tatevik Hakobyan, Activeloop, Mountain View, CA, USA; (10) Ivo Stranic, Activeloop, Mountain View, CA, USA; (11) Davit Buniatyan, Activeloop, Mountain View, CA, USA. Table of Links Abstract and Intro Current Challenges Tensor Storage Format Deep Lake System Overview Machine Learning Use Cases Performance Benchmarks Discussion and Limitations Related Work Conclusions, Acknowledgement, and References Abstract and Intro Abstract and Intro Current Challenges Current Challenges Tensor Storage Format Tensor Storage Format Deep Lake System Overview Deep Lake System Overview Machine Learning Use Cases Machine Learning Use Cases Performance Benchmarks Performance Benchmarks Discussion and Limitations Discussion and Limitations Related Work Related Work Conclusions, Acknowledgement, and References Conclusions, Acknowledgement, and References 3. TENSOR STORAGE FORMAT Deep Lake datasets follow columnar storage architecture, with tensors as columns, as shown in Fig. 3. Each tensor is a collection of chunks - binary blobs that contain the data samples. An index map associated with each tensor helps find the right chunk and index of the sample within that chunk for a given sample index. 3.1 Dataset A sample in a dataset represents a single row indexed across parallel tensors. As opposed to a document storage format, sample elements are logically independent, which enables partial access to samples for running performant queries or streaming selected tensors over the network to the GPU training instances. Multiple tensors can be grouped. Groups implement syntactic nesting and define how tensors are related to each other. Syntactic nesting avoids the format complication for hierarchical memory layout. Changes to the dataset’s schema are also tracked over time with version control, similar to dataset content changes. 3.2 Tensors Tensors are typed and can be appended or modified in-place. Default access to an index or a set of indices returns the data as NumPy arrays [55]. Instead of storing 1-D data as seen in Parquet [79] or series in Arrow [13], tensors can accommodate n-dimensional data, where typically the first dimension corresponds to the index or batch dimension. Tensors can contain dynamically shaped arrays, also called ragged tensors, as opposed to other statically chunked array formats such as Zarr [52]. 3.3 Types Htype defines the expectations on samples in a tensor such as data type (dtype as seen in NumPy [55]), shape, number of dimensions, or compression. Typed tensors make interacting with deep learning frameworks straightforward and enable sanity checks and efficient memory layout. By inheriting from a generic tensor htype, we can construct types such as image, video, audio, bbox, dicom, and others. For example, a tensor with image htype would expect samples being appended to it to have dtype as uint8 and shape length 3 (i.e. width, height and number of channels). We further expand on the notion of htypes allowing for meta types that support storing image sequences in tensors (sequence[image]), referencing to remotely stored images, while maintaining the regular behavior of a image tensor (link[image]), or even possible cross-format support. 3.4 Memory Layout A Deep Lake dataset contains a provenance file in JSON format and folders per tensor. A tensor contains chunks, chunk encoder, tile encoder, and tensor metadata. Tensors can be optionally hidden. For instance, hidden tensors can be used to maintain down-sampled versions of images or preserve shape information for fast queries. Tensors are stored in chunks at the storage level. While statically (inferred) shaped chunking avoids maintaining a chunk map table, it introduces significant user overhead during the specification of the tensor, custom compression usage limitations, underutilized storage for dynamically shaped tensors, and post-processing inefficiencies. Deep Lake chunks are constructed based on the lower and upper bound of the chunk size to fit a limited number of samples. This comes with a trade-off of having a compressed index map that preserves the sample index to chunk id mapping per tensor while enabling chunk sizes in the range optimal for streaming while accommodating mixed-shape samples. One could consider the approach taken in this paper as an optimized trade-off between file system page map and compute-defined map-less array storage system. For practical reasons, a single chunk encoder can be scaled to billions of images while maintaining a 150MB chunk encoder per 1PB tensor data. Further scaling can be introduced by sharding the chunk encoder. Chunks contain header information such as byte ranges, shapes of the samples, and the sample data itself. If a sample is larger than the upper bound chunk size, which is the case for large aerial or microscopy images, the sample is tiled into chunks across spatial dimensions. The only exception to tiling is videos. Videos are preserved due to efficient frame mapping to indices, key-frame-only decompression, and range-based requests while streaming. 3.5 Access Patterns The tensor storage format is optimized for deep learning training and inference, including sequential and random access. Sequential access is used for running scan queries, transforming tensors into other tensors, or running inference. Random access use cases include multiple annotators writing labels to the same image or models storing back predictions along with the dataset. While the strict mode is disabled, out-of-the-bounds indices of a tensor can be assigned, thus accommodating sparse tensors. However, random assignment over time will produce inefficiently stored data chunks. To fix the data layout, we implement an on-the-fly re-chunking algorithm to optimize the data layout. One of the key access patterns of Deep Lake is shuffled stream access for training machine learning models. It requires random or custom order access while streaming chunks into the training process. This is achieved by involving range-based requests to access sub-elements inside chunks, running complex queries before training to determine the order, and maintaining a buffer cache of fetched and unutilized data. This avoids having a separate compute cluster for running shuffling algorithm [50]. Each tensor has its own chunks, and the default chunk size is 8MB. A single chunk consists of data from multiple indices when the individual data points (image, label, annotation, etc.) are smaller than the chunk size. Conversely, when individual data points are larger than the chunk size, the data is split among multiple chunks (tiling). Exceptions to chunking logic are video data. Deep Lake format is optimized for maximizing throughput to GPU processing. It includes CPU pre-fetching, decompression or decoding, transformations, and GPU memory transfer in a deep learning framework’s expected layout. 3.6 Storage Providers Deep Lake can be plugged into any storage provider, including object storages such as AWS S3 [1], Google Cloud Storage (GCS) [3], POSIX compatible file systems, or local in-memory storage. Moreover, it constructs memory caching by chaining various storage providers together, for instance - the Least Recently Used (LRU) cache of remote S3 storage with local in-memory data. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Deep Lake, a Lakehouse for Deep Learning: Tensor Storage Format

ডিপ লেক, গভীর শিক্ষার জন্য একটি লেকহাউস: টেনসর স্টোরেজ ফর্ম্যাট

Deep Lake, ein Lakehouse für Deep Learning: Tensor-Speicherformat

Deep Lake, una casa de lago para el aprendizaje profundo: formato de almacenamiento tensorial

Deep Lake, un Lakehouse pour le Deep Learning : format de stockage Tensor

डीप लेक, डीप लर्निंग के लिए एक लेकहाउस: टेंसर स्टोरेज फॉर्मेट

ディープラーニングのためのレイクハウス「Deep Lake」: テンソルストレージ形式

딥 러닝을 위한 레이크하우스인 Deep Lake: Tensor 스토리지 형식

Deep Lake, um Lakehouse para aprendizado profundo: formato de armazenamento tensor

Deep Lake, домик у озера для глубокого обучения: тензорный формат хранения

Deep Lake, Derin Öğrenmeye Yönelik Bir Göl Evi: Tensör Depolama Formatı

Deep Lake, một Lakehouse cho Deep Learning: Định dạng lưu trữ Tensor

Deep Lake，深度学习的 Lakehouse：张量存储格式

Deep Lake, a Lakehouse for Deep Learning: Deep Lake System Overview

গভীর লেক, গভীর শিক্ষার জন্য একটি লেকহাউস: গভীর লেক সিস্টেম ওভারভিউ

Deep Lake, ein Lakehouse für Deep Learning: Übersicht zum Deep Lake-System

Deep Lake, una casa de lago para el aprendizaje profundo: descripción general del sistema Deep Lake

Deep Lake, une Lakehouse pour le Deep Learning : aperçu du système Deep Lake

डीप लेक, डीप लर्निंग के लिए एक लेकहाउस: डीप लेक सिस्टम अवलोकन

ディープラーニングのためのレイクハウス「Deep Lake」: Deep Lake システムの概要

딥 러닝을 위한 레이크하우스, Deep Lake: Deep Lake 시스템 개요

Deep Lake, um Lakehouse para Deep Learning: Visão geral do sistema Deep Lake

Deep Lake, дом для глубокого обучения: обзор системы Deep Lake

Deep Lake, Derin Öğrenme için Bir Göl Evi: Deep Lake Sistemine Genel Bakış

Deep Lake, Lakehouse cho Deep Learning: Tổng quan về hệ thống Deep Lake

Deep Lake，深度学习的Lakehouse：Deep Lake系统概述

OpenCitations Meta: Methodology

OpenCitations Meta: পদ্ধতি

OpenCitations Meta: Methodik

Meta de OpenCitations: Metodología

Méta OpenCitations : Méthodologie

ओपनसाइटेशन्स मेटा: कार्यप्रणाली

OpenCitations メタ: 方法論

OpenCitations Meta: 방법론

OpenCitations Meta: Metodologia

Мета OpenCitations: методология

OpenCitations Metası: Metodoloji

OpenCites Meta: Phương pháp luận

OpenCitations Meta：方法论

Authors:
(1) Arcangelo Massari, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {arcangelo.massari@unibo.it};
(2) Fabio Mariani, Institute of Philosophy and Sciences of Art, Leuphana University, Lüneburg, Germany {fabio.mariani@leuphana.de};
(3) Ivan Heibi, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy and Digital Humanities Advanced Research Centre (/DH.arc), Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {ivan.heibi2@unibo.it};
(4) Silvio Peroni, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy and Digital Humanities Advanced Research Centre (/DH.arc), Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {silvio.peroni@unibo.it};
(5) David Shotton, Oxford e-Research Centre, University of Oxford, Oxford, United Kingdom {david.shotton@opencitations.net}. Table of Links Abstract and Intro
Related Works
Methodology
Data and services
Discussion
Conclusion, Acknowledgements, and References 5. Discussion As shown in Section 2, when considering only semantic publishing datasets, OpenCitations Meta, which currently includes data from Crossref, DataCite, and the NIH Open Citation Collection (ICite et al., 2022), is first in data volume. Moreover, work is already underway to ingest data from new sources, such as the Japan Link Center (Hara, 2020), the OpenAIRE Research Graph (Atzori et al., 2017), and the Dryad Digital Repository (Vision, 2010). When compared to the OpenAIRE Research Graph, OpenCitations Meta has advantages in terms of functionality: namely the use of OMIDs, globally unique persistent identifiers used internally to identify every entity within OpenCitations Meta. This usage makes it possible to represent and index citations between bibliographic resources that lack an external persistent identifier such as a Digital Object Identifier (DOI). This feature adds significant value for the OpenCitations Indexes, as it allows for the first time the ingestion of many citations which until now were not possible to be characterised, particularly citations between publications from the humanities and social sciences (Gorraiz et al., 2016), and citations involving primary sources, e.g. a statue, a painting, or a codex, which typically lack a persistent identifier. Importantly, having an OMID also permits the identified resource to be assigned a unique URL, for example https://w3id.org/oc/meta/br/061401975837 for omid:br/061401975837. Another feature that, to the best of our knowledge, is only present in OpenCitations Meta is the mechanism for change-tracking management within the provenance information stored in RDF. This information can be queried using the Python timeagnostic-library software (Massari & Peroni, 2022). It can perform time-traversal SPARQL queries, i.e. queries across different snapshots together with provenance information. As far as other bibliographic datasets that do not use Semantic Web technologies go, OpenAlex (Priem et al., 2022) is an important case to consider for comparison with OpenCitations Meta. OpenAlex uses web crawls to add missing metadata, a feature that allows it to automatically correct a higher number of errors appearing in the data of the sources, when compared to OpenCitations Meta. Indeed, currently, the main limitation of OpenCitations Meta concerns the quality of the data, which is strictly dependent on the quality of the sources. Crossref does not double-check the metadata provided by publishers, and thus many errors are preserved. For instance, it is possible to encounter articles published in the future (the metadata available at https://api.crossref.org/v1/works/10.12960/tsh.2020.0006 say that the article will be published in print in 2029). Some of these errors can be corrected automatically without any background knowledge, while others require either the use of web crawlers or manual intervention. While OpenAlex is pursuing the path of web crawls, OpenCitations is working on a framework that will allow the editing and curation of data by trusted human domain experts (such as academic librarians). OpenCitations Meta fulfils its primary purpose by holding the bibliographic metadata required to describe the citing and cited publications involved in the citations within the OpenCitations Indexes. In addition to these bibliographic metadata elements, however, we are well aware that there are additional metadata elements of great importance for the academic community: Abstracts, for text mining, domain and subject field determination, and indexing (even if the full texts of the publications are available open access elsewhere), and Funder IDs, Funding information and Institutional identifiers, essential for determining performance metrics and undertaking research assessment. Once we have completed the provision of our textual search operations, expanded our coverage in the ways indicated, and enhanced the computational infrastructure upon which OpenCitations Meta and the OpenCitations Indexes run, we will proceed to integrate and populate these additional metadata fields. The provision of high-quality bibliographic metadata is a complex and difficult goal to achieve by automated operations, while the scale of the operations precludes manual curation except for a minority of records. No bibliographic dataset is currently able to achieve this goal on its own. For this reason, all the available bibliographic databases should be viewed as complementary. For example, while at the moment OpenAlex provides better quality metadata, OpenCitations Meta has complete provenance data openly available, and enables more complex searches, thanks to the potentialities given by Semantic Web technologies. For example, "Search for all authors who co-authored with Silvio Peroni or Fabio Vitali in conference proceedings that were published by Springer after 2009". Furthermore, OpenAlex is only partially free, since a fee must be paid to make more than a hundred thousand requests per day via the API and to access data updated every hour via the API (instead of every month via the dump)[9]. In contrast, users can make unlimited requests to the latest version of OpenCitations Meta for free. Also, although the OpenAIRE Research Graph currently contains more metadata, such data are released under a CC-BY attribution licence, while the data released by OpenCitations Meta is under a CC0 public domain waiver, permitting complete freedom for reuse, including commercial reuse, and for machine processing without any requirement for attribution. This paper is available on arxiv under CC 4.0 DEED license. Authors: (1) Arcangelo Massari, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {arcangelo.massari@unibo.it}; (2) Fabio Mariani, Institute of Philosophy and Sciences of Art, Leuphana University, Lüneburg, Germany {fabio.mariani@leuphana.de}; (3) Ivan Heibi, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy and Digital Humanities Advanced Research Centre (/DH.arc), Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {ivan.heibi2@unibo.it}; (4) Silvio Peroni, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy and Digital Humanities Advanced Research Centre (/DH.arc), Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {silvio.peroni@unibo.it}; (5) David Shotton, Oxford e-Research Centre, University of Oxford, Oxford, United Kingdom {david.shotton@opencitations.net}. Authors: Authors: (1) Arcangelo Massari, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {arcangelo.massari@unibo.it}; (2) Fabio Mariani, Institute of Philosophy and Sciences of Art, Leuphana University, Lüneburg, Germany {fabio.mariani@leuphana.de}; (3) Ivan Heibi, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy and Digital Humanities Advanced Research Centre (/DH.arc), Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {ivan.heibi2@unibo.it}; (4) Silvio Peroni, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy and Digital Humanities Advanced Research Centre (/DH.arc), Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {silvio.peroni@unibo.it}; (5) David Shotton, Oxford e-Research Centre, University of Oxford, Oxford, United Kingdom {david.shotton@opencitations.net}. Table of Links Abstract and Intro Related Works Methodology Data and services Discussion Conclusion, Acknowledgements, and References Abstract and Intro Abstract and Intro Related Works Related Works Methodology Methodology Data and services Data and services Discussion Discussion Conclusion, Acknowledgements, and References Conclusion, Acknowledgements, and References 5. Discussion As shown in Section 2, when considering only semantic publishing datasets, OpenCitations Meta, which currently includes data from Crossref, DataCite, and the NIH Open Citation Collection (ICite et al., 2022), is first in data volume. Moreover, work is already underway to ingest data from new sources, such as the Japan Link Center (Hara, 2020), the OpenAIRE Research Graph (Atzori et al., 2017), and the Dryad Digital Repository (Vision, 2010). When compared to the OpenAIRE Research Graph, OpenCitations Meta has advantages in terms of functionality: namely the use of OMIDs, globally unique persistent identifiers used internally to identify every entity within OpenCitations Meta. This usage makes it possible to represent and index citations between bibliographic resources that lack an external persistent identifier such as a Digital Object Identifier (DOI). This feature adds significant value for the OpenCitations Indexes, as it allows for the first time the ingestion of many citations which until now were not possible to be characterised, particularly citations between publications from the humanities and social sciences (Gorraiz et al., 2016), and citations involving primary sources, e.g. a statue, a painting, or a codex, which typically lack a persistent identifier. Importantly, having an OMID also permits the identified resource to be assigned a unique URL, for example https://w3id.org/oc/meta/br/061401975837 for omid:br/061401975837. Another feature that, to the best of our knowledge, is only present in OpenCitations Meta is the mechanism for change-tracking management within the provenance information stored in RDF. This information can be queried using the Python timeagnostic-library software (Massari & Peroni, 2022). It can perform time-traversal SPARQL queries, i.e. queries across different snapshots together with provenance information. As far as other bibliographic datasets that do not use Semantic Web technologies go, OpenAlex (Priem et al., 2022) is an important case to consider for comparison with OpenCitations Meta. OpenAlex uses web crawls to add missing metadata, a feature that allows it to automatically correct a higher number of errors appearing in the data of the sources, when compared to OpenCitations Meta. Indeed, currently, the main limitation of OpenCitations Meta concerns the quality of the data, which is strictly dependent on the quality of the sources. Crossref does not double-check the metadata provided by publishers, and thus many errors are preserved. For instance, it is possible to encounter articles published in the future (the metadata available at https://api.crossref.org/v1/works/10.12960/tsh.2020.0006 say that the article will be published in print in 2029). Some of these errors can be corrected automatically without any background knowledge, while others require either the use of web crawlers or manual intervention. While OpenAlex is pursuing the path of web crawls, OpenCitations is working on a framework that will allow the editing and curation of data by trusted human domain experts (such as academic librarians). OpenCitations Meta fulfils its primary purpose by holding the bibliographic metadata required to describe the citing and cited publications involved in the citations within the OpenCitations Indexes. In addition to these bibliographic metadata elements, however, we are well aware that there are additional metadata elements of great importance for the academic community: Abstracts, for text mining, domain and subject field determination, and indexing (even if the full texts of the publications are available open access elsewhere), and Funder IDs, Funding information and Institutional identifiers, essential for determining performance metrics and undertaking research assessment. Once we have completed the provision of our textual search operations, expanded our coverage in the ways indicated, and enhanced the computational infrastructure upon which OpenCitations Meta and the OpenCitations Indexes run, we will proceed to integrate and populate these additional metadata fields. The provision of high-quality bibliographic metadata is a complex and difficult goal to achieve by automated operations, while the scale of the operations precludes manual curation except for a minority of records. No bibliographic dataset is currently able to achieve this goal on its own. For this reason, all the available bibliographic databases should be viewed as complementary. For example, while at the moment OpenAlex provides better quality metadata, OpenCitations Meta has complete provenance data openly available, and enables more complex searches, thanks to the potentialities given by Semantic Web technologies. For example, "Search for all authors who co-authored with Silvio Peroni or Fabio Vitali in conference proceedings that were published by Springer after 2009". Furthermore, OpenAlex is only partially free, since a fee must be paid to make more than a hundred thousand requests per day via the API and to access data updated every hour via the API (instead of every month via the dump)[9]. In contrast, users can make unlimited requests to the latest version of OpenCitations Meta for free. Also, although the OpenAIRE Research Graph currently contains more metadata, such data are released under a CC-BY attribution licence, while the data released by OpenCitations Meta is under a CC0 public domain waiver, permitting complete freedom for reuse, including commercial reuse, and for machine processing without any requirement for attribution. This paper is available on arxiv under CC 4.0 DEED license. This paper is available on arxiv under CC 4.0 DEED license. available on arxiv

OpenCitations Meta: Discussion

OpenCitations Meta: আলোচনা

OpenCitations Meta: Diskussion

Meta de OpenCitations: Discusión

Méta OpenCitations : discussion

ओपनसाइटेशन्स मेटा: चर्चा

OpenCitations メタ: ディスカッション

OpenCitations Meta: 토론

OpenCitations Meta: Discussão

Мета OpenCitations: обсуждение

OpenCitations Meta: Tartışma

OpenCites Meta: Thảo luận

OpenCitations Meta：讨论

Authors:
(1) Arcangelo Massari, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {arcangelo.massari@unibo.it};
(2) Fabio Mariani, Institute of Philosophy and Sciences of Art, Leuphana University, Lüneburg, Germany {fabio.mariani@leuphana.de};
(3) Ivan Heibi, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy and Digital Humanities Advanced Research Centre (/DH.arc), Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {ivan.heibi2@unibo.it};
(4) Silvio Peroni, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy and Digital Humanities Advanced Research Centre (/DH.arc), Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {silvio.peroni@unibo.it};
(5) David Shotton, Oxford e-Research Centre, University of Oxford, Oxford, United Kingdom {david.shotton@opencitations.net}. Table of Links Abstract and Intro
Related Works
Methodology
Data and services
Discussion
Conclusion, Acknowledgements, and References 4. Data and services At the time of its initial release in December 2022, OpenCitations Meta included Crossref (Hendricks et al., 2020), DataCite (Brase, 2010), and the NIH Open Citation Collection (ICite et al., 2022) as its primary sources for the bibliographic metadata describing the publications involved in citations within the following OpenCitations Indexes: COCI (https://opencitations.net/index/coci) (OpenCitations, 2022), DOCI (https: //opencitations.net/index/doci), and POCI (https://opencitations.net/index/poci). From a quantitative point of view, there are within this initial release of OpenCitations Meta 98,243,101 bibliographic entities (fabio:Expression), 309,881,223 authors (pro:author), 2,406,510 editors (pro:editor), 19,076 publishers (pro:publisher), and 659,214 venues (e.g. resources of type fabio:AcademicProceedings, fabio:ExpressionCollection, fabio:Book, fabio:BookSeries, fabio:Journal, fabio:ReferenceBook, or fabio:Series). Thus, on average, each bibliographic resource has three authors. Typically no editor is recorded, as the latter metadata are little used in our sources. In total, the triplestore consists of 3,749,729,755 triples (excluding provenance). Editors and authors have been counted as roles, without disambiguating the individuals holding these roles. Conversely, bibliographic entities, publishers, and venues were counted by OMID. However, for venues (e.g. journals), we have taken an extra precaution: many are duplicated in OpenCitations Meta because they have no identifiers other than the OMID. Therefore, in the figures shown above, we found it reasonable to disambiguate the venues by title in the absence of other identifiers. As shown in Table 2, Springer Science is the publishing entity with the highest number of venues (2097), followed by Elsevier BV (1961) and IEEE (1775). When counting the number of publications, Elsevier is in the lead (16,933,610), followed by Springer Science (11,507,498) and Wiley (7,262,893) in Table 3. Considering the venues in Table 4, Wiley’s ChemInform has the most publications (421,735), followed by Elsevier’s SSRN Electronic Journal (337,223) and Springer’s Journal On Data Semantics (330,093). Table 5 lists all the types of bibliographic resources in OpenCitations Meta. The current dataset contains mostly journal articles (67,904,323), which exceed the number of book chapters in second place (6,476,623) by about ten times, and proceedings articles in third place (5,046,165) by about thirteen times. Table 6, which lists the number of publications per year, shows an increasing trend, with a greater number of publications from year to year. OpenCitations Meta allows the users to explore such data either via SPARQL (https://opencitations.net/meta/sparql) or via an API (https://opencitations.net/meta/ api/v1). In particular, the OpenCitations Meta API retrieves a list of bibliographic resources and related metadata starting from one or more publication identifiers, an author’s ORCID, or an editor’s ORCID. Textual searches are currently under testing and will be released in the future as one further operation of the OpenCitations Meta API. In particular, text searches on titles, authors, editors, publishers, IDs, and venues can be performed. They can also be achieved on volume and issue numbers, provided the venue is first specified. Indeed, searches on multiple fields can be combined using the Boolean conjunction and disjunction operators. For example, once the operation is released, the user will be able to search for all bibliographic resources whose title contains the word “micro-chaos” published either by Philosophical Studies or the Journal of Nonlinear Science: title=micro-chaos&&venue=philosophical%20studies||title=microchaos&&venue=journal%20of%20nonlinear%20science, where “&&” is the conjunction operator, while || is the disjunction operator. Finally, all data and provenance are available as dumps in RDF (JSON-LD) (OpenCitations, 2023b) or CSV format (OpenCitations, 2023a) under a CC0 licence. This paper is available on arxiv under CC 4.0 DEED license. Authors: (1) Arcangelo Massari, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {arcangelo.massari@unibo.it}; (2) Fabio Mariani, Institute of Philosophy and Sciences of Art, Leuphana University, Lüneburg, Germany {fabio.mariani@leuphana.de}; (3) Ivan Heibi, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy and Digital Humanities Advanced Research Centre (/DH.arc), Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {ivan.heibi2@unibo.it}; (4) Silvio Peroni, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy and Digital Humanities Advanced Research Centre (/DH.arc), Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {silvio.peroni@unibo.it}; (5) David Shotton, Oxford e-Research Centre, University of Oxford, Oxford, United Kingdom {david.shotton@opencitations.net}. Authors: Authors: (1) Arcangelo Massari, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {arcangelo.massari@unibo.it}; (2) Fabio Mariani, Institute of Philosophy and Sciences of Art, Leuphana University, Lüneburg, Germany {fabio.mariani@leuphana.de}; (3) Ivan Heibi, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy and Digital Humanities Advanced Research Centre (/DH.arc), Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {ivan.heibi2@unibo.it}; (4) Silvio Peroni, Research Centre for Open Scholarly Metadata, Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy and Digital Humanities Advanced Research Centre (/DH.arc), Department of Classical Philology and Italian Studies, University of Bologna, Bologna, Italy {silvio.peroni@unibo.it}; (5) David Shotton, Oxford e-Research Centre, University of Oxford, Oxford, United Kingdom {david.shotton@opencitations.net}. Table of Links Abstract and Intro Related Works Methodology Data and services Discussion Conclusion, Acknowledgements, and References Abstract and Intro Abstract and Intro Related Works Related Works Methodology Methodology Data and services Data and services Discussion Discussion Conclusion, Acknowledgements, and References Conclusion, Acknowledgements, and References 4. Data and services At the time of its initial release in December 2022, OpenCitations Meta included Crossref (Hendricks et al., 2020), DataCite (Brase, 2010), and the NIH Open Citation Collection (ICite et al., 2022) as its primary sources for the bibliographic metadata describing the publications involved in citations within the following OpenCitations Indexes: COCI (https://opencitations.net/index/coci) (OpenCitations, 2022), DOCI (https: //opencitations.net/index/doci), and POCI (https://opencitations.net/index/poci). From a quantitative point of view, there are within this initial release of OpenCitations Meta 98,243,101 bibliographic entities (fabio:Expression), 309,881,223 authors (pro:author), 2,406,510 editors (pro:editor), 19,076 publishers (pro:publisher), and 659,214 venues (e.g. resources of type fabio:AcademicProceedings, fabio:ExpressionCollection, fabio:Book, fabio:BookSeries, fabio:Journal, fabio:ReferenceBook, or fabio:Series). Thus, on average, each bibliographic resource has three authors. Typically no editor is recorded, as the latter metadata are little used in our sources. In total, the triplestore consists of 3,749,729,755 triples (excluding provenance). Editors and authors have been counted as roles, without disambiguating the individuals holding these roles. Conversely, bibliographic entities, publishers, and venues were counted by OMID. However, for venues (e.g. journals), we have taken an extra precaution: many are duplicated in OpenCitations Meta because they have no identifiers other than the OMID. Therefore, in the figures shown above, we found it reasonable to disambiguate the venues by title in the absence of other identifiers. As shown in Table 2, Springer Science is the publishing entity with the highest number of venues (2097), followed by Elsevier BV (1961) and IEEE (1775). When counting the number of publications, Elsevier is in the lead (16,933,610), followed by Springer Science (11,507,498) and Wiley (7,262,893) in Table 3. Considering the venues in Table 4, Wiley’s ChemInform has the most publications (421,735), followed by Elsevier’s SSRN Electronic Journal (337,223) and Springer’s Journal On Data Semantics (330,093). Table 5 lists all the types of bibliographic resources in OpenCitations Meta. The current dataset contains mostly journal articles (67,904,323), which exceed the number of book chapters in second place (6,476,623) by about ten times, and proceedings articles in third place (5,046,165) by about thirteen times. Table 6, which lists the number of publications per year, shows an increasing trend, with a greater number of publications from year to year. OpenCitations Meta allows the users to explore such data either via SPARQL (https://opencitations.net/meta/sparql) or via an API (https://opencitations.net/meta/ api/v1). In particular, the OpenCitations Meta API retrieves a list of bibliographic resources and related metadata starting from one or more publication identifiers, an author’s ORCID, or an editor’s ORCID. Textual searches are currently under testing and will be released in the future as one further operation of the OpenCitations Meta API. In particular, text searches on titles, authors, editors, publishers, IDs, and venues can be performed. They can also be achieved on volume and issue numbers, provided the venue is first specified. Indeed, searches on multiple fields can be combined using the Boolean conjunction and disjunction operators. For example, once the operation is released, the user will be able to search for all bibliographic resources whose title contains the word “micro-chaos” published either by Philosophical Studies or the Journal of Nonlinear Science: title=micro-chaos&&venue=philosophical%20studies||title=microchaos&&venue=journal%20of%20nonlinear%20science, where “&&” is the conjunction operator, while || is the disjunction operator. Finally, all data and provenance are available as dumps in RDF (JSON-LD) (OpenCitations, 2023b) or CSV format (OpenCitations, 2023a) under a CC0 licence. This paper is available on arxiv under CC 4.0 DEED license. This paper is available on arxiv under CC 4.0 DEED license. available on arxiv

OpenCitations Meta: Data and services

OpenCitations মেটা: ডেটা এবং পরিষেবা

OpenCitations Meta: Daten und Dienste

OpenCitations Meta: Datos y servicios

Méta OpenCitations : données et services

OpenCitations मेटा: डेटा और सेवाएँ

OpenCitations Meta: データとサービス

OpenCitations Meta: 데이터 및 서비스

OpenCitations Meta: Dados e serviços

Мета OpenCitations: данные и сервисы

OpenCitations Meta: Veriler ve hizmetler

OpenCites Meta: Dữ liệu và dịch vụ

OpenCitations Meta：数据和服务

OpenCitations Meta: Conclusion, Acknowledgements, and References

OpenCitations Meta: উপসংহার, স্বীকৃতি, এবং রেফারেন্স

OpenCitations Meta: Schlussfolgerung, Danksagungen und Referenzen

Meta de OpenCitations: conclusión, agradecimientos y referencias

Méta OpenCitations : conclusion, remerciements et références

ओपनसाइटेशन्स मेटा: निष्कर्ष, आभार और संदर्भ

OpenCitations Meta: 結論、謝辞、参考文献

OpenCitations Meta: 결론, 감사의 말, 참고 문헌

OpenCitations Meta: Conclusão, Agradecimentos e Referências

Мета OpenCitations: заключение, благодарности и ссылки

OpenCitations Meta: Sonuç, Teşekkür ve Referanslar

OpenCites Meta: Kết luận, Lời cảm ơn và Tài liệu tham khảo

OpenCitations Meta：结论、致谢和参考文献

OpenCitations Meta: Abstract and Intro

OpenCitations মেটা: বিমূর্ত এবং ভূমিকা

OpenCitations Meta: Zusammenfassung und Einleitung

Meta de OpenCitations: resumen e introducción

Méta OpenCitations : résumé et introduction

ओपनसाइटेशंस मेटा: सार और परिचय

OpenCitations Meta: 概要と序文

OpenCitations Meta: 초록 및 소개

OpenCitations Meta: Resumo e introdução

Мета OpenCitations: Аннотация и введение

OpenCitations Meta: Özet ve Giriş

OpenCites Meta: Tóm tắt và giới thiệu

OpenCitations Meta：摘要和简介

OpenCitations Meta: Related Works

OpenCitations Meta: সম্পর্কিত কাজ

OpenCitations Meta: Verwandte Werke

Meta de OpenCitations: trabajos relacionados

Méta OpenCitations : travaux connexes

ओपनसाइटेशंस मेटा: संबंधित कार्य

OpenCitations メタ: 関連作品

OpenCitations Meta: 관련 저작물

OpenCitations Meta: Trabalhos Relacionados

Мета OpenCitations: Связанные работы

OpenCitations Meta: İlgili Çalışmalar

OpenCites Meta: Tác phẩm liên quan

OpenCitations Meta：相关著作

OpenAI truly revolutionized the field of AI in March 2022 by releasing its AI chat model GBT3. Although developments in the field of AI have been underway for a long time, humanity is truly on the verge of AI being able to replace humans in many areas such as design, automation, software development, and even in the areas of decision making. However, such advanced AI comes with ethical problems that need to be solved. In this essay, I propose to consider the ethical problems of using AI in software development, proposing solutions to potential risks. Ethical implications need to be considered when developing software to ensure fairness, transparency, and accountability in the deployment of AI-driven systems, even though AI offers many benefits to AI development. The first is data security and privacy. According to research (Boulemtafes, Derhab, & Challal, 2020), privacy concerns are particularly related to sensitive input data either during training or inference and to the sharing of the trained model with others. Typically, models used for AI are trained on huge amounts of data. According to research (Arnott, 2023), ChatGPT collects both your account-level information as well as your conversation history. This includes records such as your email address, device, IP address, and location, as well as any public or private information you use in your ChatGPT prompts. This cannot but cause concern since this data may be confidential. Therefore, developers of AI systems must obtain consent to process personal data. Typically, when it comes to processing personal data, protection from unauthorized access is necessary, so it is necessary to implement a strong encryption method. The second is bias and fairness. As mentioned earlier, large amounts of data are used to train AI. But are there any guarantees that AI will not inherit the bias of their information sources? This can lead to highly unpredictable consequences, unfair outcomes, discrimination, and the perpetuation of social inequality. For example, a person turning to an AI language model expects to receive pure information that does not convey anyone’s opinion but receives biased and reversed information. The dataset leans towards specific demographics, and the algorithm might exhibit biases, leading to unfair outcomes. Therefore, according to research (Li, 2023) collecting widely varied data representing different races, genders, ages,  cultures, and social backgrounds is key to ensuring algorithmic fairness. The danger lies in the fact that this can control the opinion of all humanity about certain facts. It is diversity in data presentation and regular bias checks to identify and mitigate discriminatory patterns. Therefore, continuous commitment and effort are vital to ensuring algorithmic fairness. According to Li (2023), only through continuous learning, improvement, and adaptation can the field of  AI  achieve genuine fairness and equity. Data scientists, on the other hand, are responsible for analyzing data, ensuring the algorithm's performance is fair across various demographics. Third is accountability and transparency. These days, AI algorithms have a complex structure, which makes it quite difficult to determine the degree of responsibility for errors in them. Issues surrounding transparency, debates on job displacement, and global disparities in AI  development exacerbate these ethical dilemmas. According to Li (2023) to solve this problem, it is necessary to introduce clear standards and, of course, reporting. This same demand will differentiate the roles of developers, data scientists, decision-makers, and end users. Also, the process of documenting key processes can enable tracking and reporting of AI results. The concept of transparency refers to the degree to which the decisions and actions of AI systems are understandable, interpretable, and understandable not only by the specialists who developed the AI model, but also by all interested parties. An example of the use of interpretable machine learning models could be decision trees or even linear models. This will allow us to evaluate the factors influencing AI forecasts and decisions. Association for Computing Machinery outlined seven principles that emphasize the importance of ethical considerations in the design, implementation, and use of analytic computer systems. These principles are provided and explained in greater detail in the article by Garfinkel et al (2017). According to Matthews (2020), these principles encompass various facets of ethical considerations in the design, implementation, and use of analytic systems. Access and Redress should encourage the adoption of mechanisms that enable questioning and redress for individuals and groups that are adversely affected by algorithmically informed decisions. Accountability entails that institutions should be held responsible for decisions made by the algorithms that they use, even if it is not feasible to explain in detail how the algorithms produce their results. Explanations of systems and institutions that use algorithmic decision-making are encouraged to produce explanations regarding both the procedures followed by the algorithm and the specific decisions that are made. Data Provenance means a description of the way in which the training data collected should be maintained by the builders of the algorithms, accompanied by an exploration of the potential biases induced by the human or algorithmic data-gathering process. Audibility of models, algorithms, data, and decisions should be recorded so that they can be audited in cases where harm is suspected. Validation and Testing of institutions should use rigorous methods to validate their models and document those methods and results. In particular, they should routinely perform tests to assess and determine whether the model generates discriminatory harm. In conclusion, developments in the field of AI have been underway for a long time, humanity is truly on the verge of AI being able to replace humans in many areas such as design, automation, software development, and even in the areas of decision making. However, such advanced AI comes with ethical problems that need to be solved. In this essay, I considered the ethical problems of using AI in software development, proposing solutions to potential risks. References Arnott, B. (2023, September 13). Yes, ChatGPT Saves Your Data. Here’s How to Keep It Secure. Retrieved May 11, 2024, from Forcepoint website: https://www.forcepoint.com/blog/insights/does-chatgpt-save-data#:~:text=ChatGPT%20collects%20both%20your%20account Boulemtafes, A., Derhab, A., & Challal, Y. (2020). A review of privacy-preserving techniques for deep learning. Neurocomputing,
384, 2–5. https://doi.org/10.1016/j.neucom.2019.11.041 Garfinkel, S.; Matthews, J.; Shapiro, S.; and Smith, J. 2017. Toward Algorithmic Transparency and Accountability. Communications of the ACM 60(9): 5. doi.org/10.1145/ 3125780. Li, N. (2023). Ethical Considerations in Artificial Intelligence: A Comprehensive Discussion from the Perspective of Computer Vision. SHS Web of Conferences,
179. https://doi.org/10.1051/shsconf/202317904024 Matthews, J. (2020). Patterns and Antipatterns, Principles, and Pitfalls: Accountability and Transparency in Artificial Intelligence. AI Magazine, 41(1), 82–89.
https://doi.org/10.1609/aimag.v41i1.5204 OpenAI truly revolutionized the field of AI in March 2022 by releasing its AI chat model GBT3. Although developments in the field of AI have been underway for a long time, humanity is truly on the verge of AI being able to replace humans in many areas such as design, automation, software development, and even in the areas of decision making. However, such advanced AI comes with ethical problems that need to be solved. In this essay, I propose to consider the ethical problems of using AI in software development, proposing solutions to potential risks. Ethical implications need to be considered when developing software to ensure fairness, transparency, and accountability in the deployment of AI-driven systems, even though AI offers many benefits to AI development. The first is data security and privacy. According to research (Boulemtafes, Derhab, & Challal, 2020), privacy concerns are particularly related to sensitive input data either during training or inference and to the sharing of the trained model with others. Typically, models used for AI are trained on huge amounts of data. According to research (Arnott, 2023), ChatGPT collects both your account-level information as well as your conversation history. This includes records such as your email address, device, IP address, and location, as well as any public or private information you use in your ChatGPT prompts. This cannot but cause concern since this data may be confidential. Therefore, developers of AI systems must obtain consent to process personal data. Typically, when it comes to processing personal data, protection from unauthorized access is necessary, so it is necessary to implement a strong encryption method. The second is bias and fairness. As mentioned earlier, large amounts of data are used to train AI. But are there any guarantees that AI will not inherit the bias of their information sources? This can lead to highly unpredictable consequences, unfair outcomes, discrimination, and the perpetuation of social inequality. For example, a person turning to an AI language model expects to receive pure information that does not convey anyone’s opinion but receives biased and reversed information. The dataset leans towards specific demographics, and the algorithm might exhibit biases, leading to unfair outcomes. Therefore, according to research (Li, 2023) collecting widely varied data representing different races, genders, ages,  cultures, and social backgrounds is key to ensuring algorithmic fairness. The danger lies in the fact that this can control the opinion of all humanity about certain facts. It is diversity in data presentation and regular bias checks to identify and mitigate discriminatory patterns. Therefore, continuous commitment and effort are vital to ensuring algorithmic fairness. According to Li (2023), only through continuous learning, improvement, and adaptation can the field of  AI  achieve genuine fairness and equity. Data scientists, on the other hand, are responsible for analyzing data, ensuring the algorithm's performance is fair across various demographics. Third is accountability and transparency. These days, AI algorithms have a complex structure, which makes it quite difficult to determine the degree of responsibility for errors in them. Issues surrounding transparency, debates on job displacement, and global disparities in AI  development exacerbate these ethical dilemmas. According to Li (2023) to solve this problem, it is necessary to introduce clear standards and, of course, reporting. This same demand will differentiate the roles of developers, data scientists, decision-makers, and end users. Also, the process of documenting key processes can enable tracking and reporting of AI results. The concept of transparency refers to the degree to which the decisions and actions of AI systems are understandable, interpretable, and understandable not only by the specialists who developed the AI model, but also by all interested parties. An example of the use of interpretable machine learning models could be decision trees or even linear models. This will allow us to evaluate the factors influencing AI forecasts and decisions. Association for Computing Machinery outlined seven principles that emphasize the importance of ethical considerations in the design, implementation, and use of analytic computer systems. These principles are provided and explained in greater detail in the article by Garfinkel et al (2017). According to Matthews (2020), these principles encompass various facets of ethical considerations in the design, implementation, and use of analytic systems. Access and Redress should encourage the adoption of mechanisms that enable questioning and redress for individuals and groups that are adversely affected by algorithmically informed decisions. Accountability entails that institutions should be held responsible for decisions made by the algorithms that they use, even if it is not feasible to explain in detail how the algorithms produce their results. Explanations of systems and institutions that use algorithmic decision-making are encouraged to produce explanations regarding both the procedures followed by the algorithm and the specific decisions that are made. Data Provenance means a description of the way in which the training data collected should be maintained by the builders of the algorithms, accompanied by an exploration of the potential biases induced by the human or algorithmic data-gathering process. Audibility of models, algorithms, data, and decisions should be recorded so that they can be audited in cases where harm is suspected. Validation and Testing of institutions should use rigorous methods to validate their models and document those methods and results. In particular, they should routinely perform tests to assess and determine whether the model generates discriminatory harm. In conclusion, developments in the field of AI have been underway for a long time, humanity is truly on the verge of AI being able to replace humans in many areas such as design, automation, software development, and even in the areas of decision making. However, such advanced AI comes with ethical problems that need to be solved. In this essay, I considered the ethical problems of using AI in software development, proposing solutions to potential risks. References References Arnott, B. (2023, September 13). Yes, ChatGPT Saves Your Data. Here’s How to Keep It Secure. Retrieved May 11, 2024, from Forcepoint website: https://www.forcepoint.com/blog/insights/does-chatgpt-save-data#:~:text=ChatGPT%20collects%20both%20your%20account Boulemtafes, A., Derhab, A., & Challal, Y. (2020). A review of privacy-preserving techniques for deep learning. Neurocomputing , 384 , 2–5. https://doi.org/10.1016/j.neucom.2019.11.041 Neurocomputing 384 Garfinkel, S.; Matthews, J.; Shapiro, S.; and Smith, J. 2017. Toward Algorithmic Transparency and Accountability. Communications of the ACM 60(9): 5. doi.org/10.1145/ 3125780. Li, N. (2023). Ethical Considerations in Artificial Intelligence: A Comprehensive Discussion from the Perspective of Computer Vision. SHS Web of Conferences , 179 . https://doi.org/10.1051/shsconf/202317904024 Matthews, J. (2020). Patterns and Antipatterns, Principles, and Pitfalls: Accountability and Transparency in Artificial Intelligence. AI Magazine , 41 (1), 82–89.
https://doi.org/10.1609/aimag.v41i1.5204 SHS Web of Conferences 179 AI Magazine 41

Artificial Intelligence in Software Development: Discussing the Ethics

Stories, News & Mentions about PRODVANA

Prodvana’s Stories on HackerNoon