Techstars `22 Portfolio Company\n\nHuman potential is being squandered

ByDesign

By 2023, government regulations requiring organizations to provide free and accessible consumer privacy rights will cover 5 billion citizens globally. Fundamental principles such as privacy rights and purposeful processing are included in almost all of these regulatory changes. We’ve learned from cybersecurity that shifting left works by automating security scanners in order to ensure protection throughout the development process. Based on that principle, ‘privacy by design’ builds observability and tracking into the code from day one, in order to ensure the potential for continuous compliance with any and all relevant privacy laws and policies. This is in contrast to traditional post-production solutions, which have a number of integral weaknesses, including their reliance on manual processes, expense, and the fact that they are usually implemented too late to mitigate data risk. Here are five reasons why privacy by design is a must for companies and organizations in 2023: 1) The cost of not taking action: Data breaches, fines and reputation management in the aftermath of such events present very serious potential – and avoidable – financial burdens.  By building protection into the process of designing, developing and deploying technology, privacy by design proactively guarantees personal data is being protected by means of preventative measures. As of September 2022, the to date totalled more than $4.4 billion. That figure does not include thousands of other cases globally each year. According to “State of Privacy — The European Union”, an August 2022 report published by Gartner, in 2021, EU sanctions for privacy violations increased sevenfold as compared to the previous year, resulting in $1.2 billion in fines. 12 largest data breach fines, penalties, and settlements By 2026, organizations mishandling personal data will suffer three times more financial damage from class actions and mass claims than from enforcement sanctions.  By taking an active role in protecting customer data, companies and organizations ensure they are meeting all applicable laws and regulations. 2) CPRA, a comprehensive update of the CCPA: In 2022, cosmetics retailer Sephora agreed to pay $1.2 million in the first public enforcement action under California’s landmark consumer privacy law, known as the CCPA. On January 1, 2023, the new and expanded California Privacy Rights Act (CPRA) will require further technological and operational capabilities to ensure proper data security and compliance. Though technically an act passed by the California legislature, the CPRA’s impact will be felt globally. To successfully manage the complexities of the CPRA, organizations must develop an understanding of the data they collect, store, and transfer, as well as implement a comprehensive data governance strategy to meet the new requirements of the CPRA. Due to the new challenges of the CPRA, the core tenets of privacy by design are no longer just ‘nice to have’, but are foundational to ensuring comprehensive compliance. This includes but is not limited to, data mapping of personal and sensitive information, mapping the details of personal information (PI) data that has been shared to third parties, assisting in building a process for data subject rights (DSR), accuracy, data minimization and data retention. 3) Customers demand it: Privacy by design improves customer service and loyalty. By taking steps to protect customer data, organizations are able to provide customers with experiences that build trust in their services, increase customer loyalty and subsequently drive revenues. Organizations with underserved privacy user experiences, missing proper transparency, consent and preference management, and lacking the automation of DSR responses, result in an abundance of complaints and the highest rates of customer attrition. A top recommendation from Gartner's 2021 is to drive automation in the privacy UX to make it efficient for individuals to exercise their rights and keep operational costs manageable. Security and Risk Survey 4) Automation is key in a global economic crisis: According to Gartner’s survey, the average annual budget for managing privacy risks was $2.2 million, with 55% percent of respondents indicating that this was an increase over the previous year. Organizations that have at least 100 employees and $50 million in total annual revenue for fiscal year 2020 were spending, on average, $1,524 per data subject access request (DSR), and the majority of respondents (85%) were able to process a request within two weeks of receipt. Half of the respondents indicated their organization received 51 to 100 data subject rights requests (DSR) per month. Privacy by design can be used to enable mapping of relevant storage and services as part of development life cycle, helping engineers develop DSR-related issues as an integral part of the product, thus dramatically increasing DSR-related efficiency. 5) The more manual work is involved in managing data privacy, the greater the risk of costly data and compliance breaches. Privacy by design identifies, streamlines and manages potential risks associated with personal data and takes steps to mitigate them before they become a costly and seemingly insurmountable hurdle. Automation includes assessing potential threats to customer data, such as unauthorized access or exposure of PI data to unauthorized storage and databases, and developing measures to protect it. A privacy professional talent shortage calls for automation: By 2024, a privacy professional talent shortage will lead to 100,000 positions unfulfilled, forcing fragmented automation investment. Gartner’s 2021 Security and Risk Survey showed that 63% of surveyed organizations planned on increasing staff levels for their privacy programs. With a privacy talent shortage on the rise, the automation of privacy by design keeps up to the pace and the breadth of changes across organizations. As was successfully done in security a few years ago, shifting left and giving responsibility to engineering is undoubtedly the solution for privacy, as well. Moreover, many privacy professionals approach the issues from a legal context, lacking the technical background required to understand complex issues related to big data and related technologies. This further underlines the operational resource crunch and the need for privacy by design. Final Thoughts As the data landscape continues to evolve, leveraging privacy by design will ensure that privacy considerations are taken into account from the very start and throughout the entire lifecycle of a product or service. Simultaneously, data privacy and security measures remain effective and up to date, as technology and the associated risks and regulations rapidly advance. As we enter 2023, privacy by design proves to be critical in identifying and mitigating potential risks associated with personal data, improving customer service, and ensuring compliance with data privacy regulations. About Me My experienced encouraged me to establish , a data privacy startup focused on ensuring that data privacy and compliance could be built into companies' digital products from day one. Please check out my work and let me know what you think in the comments below! Privya

5 Reasons Why Privacy by Design is More Critical Than Ever in 2023

Hi Beni, welcome to our series “Behind the Startup.” Please tell us about yourself and the story behind ? Ishan Pandey: Concordium My name is Beni Issembert, born and raised in France but living in Tel Aviv for the last 26 years. Happily married to Olivia and proud father of two princesses. Beni Issembert: My background is mainly made of a strong anchor in the academy with doctoral studies at John Hopkins University and Paris 8 in the field of philosophy of sciences and expertise in the following fields: ethics in tech, collective intelligence and Friedrich Nietzsche. Since 2005 I’ve been a web entrepreneur with a marketing and storytelling orientation. I’ve been active in the following industries: video streaming, online gaming, ad-tech and most recently, in the blockchain industry. Before my time at Concordium, I was the founding CMO at Beam Privacy, a Mimblewimble implementation and I’m heading the board of advisors at Kylin Network. Plus, very recently, I’ve created a little Venture Capital Fund called 314.VC, which deals with ethical investments in tech. The story behind Concordium is simple. Let’s begin with the following postulate: businesses don’t adopt blockchain-based technologies and the mission was to solve this issue because we believe that mainstream adoption will come from the business world in general and from the SMEs in particular. In our analysis of the “why”, we’ve concluded that businesses aren’t ready to jump in without a strong privacy orientation combined with a regulatory-compliant approach. And this is why we’ve created Concordium, whose design has been first developed in the academy via our science partners at ETH Zurich, Aarhus University and the Indian Institute of Science at Bengaluru. The solution is a technological expression of a sensible balance between a holistic privacy-centric infrastructure meant to protect users’ privacy using applied encryption and ZK proof; and the necessary regulatory-compliant readiness in the form of Concordium ID layer at the protocol level. In other words, the blockchain is a smart equation between Privacy and Identity Techs.  Plus, the platform’s 2-Layer consensus offers better scalability, security and finalisation. The blockchain’s innovative identification layer strikes a harmony between anonymity and disclosure that is focused on enforcement. On-chain, a user’s identity is private, however, it may be removed, and their real-world identity exposed in response to a legitimate request by a government body made through existing legal channels. Anonymity concerning the general population is maintained from the user’s viewpoint, and blockchain’s identification layer can serve identity suppliers and anonymity revokers based in various jurisdictions around the globe. As a result, the blockchain provides a multinational, multijurisdictional solution for blockchain adoption through regulatory regimes. With so many frauds happening where unscrupulous people are duping investors, professional ethics and morals need to play a more prominent role in our industry. What are your views on ethics in the blockchain/cryptocurrency industry particularly? Ishan Pandey: You are right. Cybercrimes are one of the most threatening challenges of our time. But before we speak about ethics in the blockchain world, let’s define ethics. If we refer to ethics by means of speaking of good vs evil, then we are wrong. This form of ethics is a pure product of the Judeo-Christian morality approach, meaning a subjective judgment. By ethics, I mean what is allowed and what is forbidden within a structure governed by a social contract, the same way Frederich Nietzsche has expressed it in Beyond Good and Evil (1886). If we get back to the tech landscape in general and the blockchain in particular, by ethics, we need to understand a work-in-progress that will enable us, the builders, to define some paradigms that will lead us to state what is allowed vs what is forbidden. Finally, after having defined “ethics” we need to enforce it and this where the regulator is playing a role. Beni Issembert: In our industry today, everything is allowed and very few things are forbidden. This so-called anarchist situation - I understand anarchy the way its “father” the French Philosopher Joseph Proudhon has defined it in the first part of the 19th century, meaning as “order without power” - leads all of us to a dangerous position where people are scammed, VCs govern and nobody is protected. We miss here a tangible and community-built social contract that will help us live together in this space. This is what I call ethics. But again, ethics as an idea is not much. We need the “regulator” to enforce this ethical design by creating laws and applying them. Ishan Pandey: Concordium works on a next-generation privacy-focused, public, and permissionless blockchain architecture whereby users can verify that something is true without physically seeing the evidence. Can you explain how consensus will be established on Concordium between network participants? Concordium is indeed the first public and permissionless layer-1 blockchain that expresses this sensible balance between identity and privacy. The whole is quite simple. To use blockchain and its various products, i.e. a Rust-based Smart Contract, users must be identified using our zk-proof-based identity layer. In other words, users are identified and create an SSI object that can be deployed everywhere, so they don’t have to renounce their privacy. Thanks to ZK-proof, the blockchain’s Identity layer enables users to deploy an “identity object” without sharing the data that sits inside. Beni Issembert: That said, the Identity layer offers an Identity Revoking design, decentralised, that can be used to revoke the “anonymity” of a user upon a legal court request, and only if the user is suspicious to have acted illegally. But to recap, if you play by the rules, your activities are private in a singular way. If you don’t play according to the rules, well, like in the legacy world, you need to explain yourself in front of the authorities. But more than everything else, this design provides users with the most regulatory-compliant ecosystem, enabling businesses but not only to manage their activities legally on the network. How will revoking a user’s anonymity for complying with regulations or court order work? Further, do you think that Compliance-as-a-Design is a crucial feature for next-generation blockchains? Ishan Pandey: I would say that Compliance is not a question. It’s a fact, whether you like it or not, this is not the question. But living in a society with a “social contract” means that you have a set of common rules that all must respect. Compliance in the crypto world is ineluctable. That’s the key to mainstream adoption, as far as I’m concerned and we still have room to impact the upcoming regulation to fit with our times. Beni Issembert: It’s designed to work with existing financial and business networks that need to recognise a user’s identification. The blockchain platform lets application developers, people, and companies design products that conform with local legislation while maintaining the advantages of a privacy-focused, public, and permissionless blockchain by developing special protocol-level identity algorithms. Any person that has an account on the platform is considered a member. Individuals or legitimate bodies, such as companies, are qualified, although they must have a clear form of identification to complete the off-chain identity procedure. Can you explain what is Global Transaction Unit (GTU)? Further, how will baking and shielded transfer work on the Concordium blockchain? Ishan Pandey: The GTU is the native token built on top of Concordium using our token standard, the equivalent of the famous erc20. It’s a payment token that enables smart contract deployers and users to use the network. Beni Issembert: Bakers play an important role in our network and will be rewarded for every new-minted block. In the first year, the blockchain’s inflation will be 10%, the second year 5% and from the third one 2%. Bakers are getting 60% of each new block. Block finalisers are getting 30% of it and the Foundation 10%. When it comes to transactions, the blockchain’s approach has always been clear from day 1: flat and fiat-based fees for cheaper transactions but mainly for a predictable cost. A critical feature for businesses, actually. A simple transition of x GTU from Account A to Account B is the most basic form of transaction. If Account A’s public balance is at least x + transaction cost, the transaction is legal. The result of the exchange is that Account A’s public balance is lowered by x + transaction fee, whereas Account B’s public balance is raised by x. For this particular type of trade, both the records and the amount x are available to the public. When you think about what’s going on with our dear friends at Ethereum when it comes to gas fees, that’s insane. Although encrypted transactions conceal the sum of GTU transferred, it is easy to see which accounts are involved. Through anonymous transactions, you get an extra layer of anonymity. The usability of an anonymous transfer is similar to that of an encrypted transfer, except that the sender and recipient of a transmission cannot be connected. The blockchain intends to promote anonymous transfer using various recent and well-known methods, including a mix-net solution similar to Fauzi et al. work’s on Quisquis. Authorities can also access details regarding a transaction concealed in an encrypted move by using an anonymity revocation method. Concordium will use Rust as the programming language for deploying smart contracts. According to you, why is Rust a better programming language as compared to Solidity or Vyper for writing smart contracts? Ishan Pandey: : The RustLang ecosystem is one of the most impressive landscapes I ever saw for several reasons. Beni Issembert First of all, this framework is incredibly secure and flexible, enabling coders and users to enjoy a non-complex environment. In other words, no stupid elitism in this ecosystem but a nova linga by the people for the people. Plus, as a self-taught coder that codes in RustLang, I can testify that Rust is one of the simplest languages to learn and use. And combined with WebAssembly, RustLang offers some great opportunities when it comes to cross-chain communication. Although I can talk about RustLang for hours, RustLang offers one of the readiest logos for ethics in technology due to its flexibility and thanks to its builders that are real ontological leaders. I’m speaking clearly about Florian Glitcher and Josh Triplett as the best examples of what I just said. WebAssembly (Wasm), a compact, well-defined assembly-like language, is the blockchain’s fundamental on-chain language. Wasm is an internet standard that has gained much popularity in recent years and is now endorsed by many of the main browsers. Many programming languages may already be compiled to Wasm, which means we will accept a wide variety of smart contract languages in the future. What advice and insights will you give to developers interested in building applications in the Concordium ecosystem? Ishan Pandey: I won’t say welcome homes like my competitors and friends. That’s simply silly. Beni Issembert: I would say more that if they want to contribute to the development of the ecosystem, they must understand that the best way to do it is to deploy the principle of reciprocity, the Golden Rule as Karl Popper mentioned, based on which contributing to the collective brings direct individual benefits. In other words, coders willing to deploy a Dapp on top of Concordium have to look at the RustLang ecosystem, too, to fit and bring real value. By strengthening the RustLang Ecosystem, the blockchain developers are strengthening the ecosystem. We invite them to contribute to the ecosystem while working on their dapp on Concordium with a grant from the Concordium Foundation. Can you highlight certain business use cases of Concordium? Further, how can organisations operating in the healthcare industry or finance industry leverage and at the same time comply with regulations by building on Concordium? Ishan Pandey: Beni Issembert: We are a layer-1 blockchain, meaning we are not going to deploy any use-case. That said, we work with big system integrators that will onboard businesses and use-cases on the blockchain. The first natural verticals that could enjoy an incredible user experience and an adequate platform to build are identity, privacy, gaming, “regulated” defi and healthcare. How will the governance of the Concordium blockchain look like? Ishan Pandey: It looks good, lol. Well, the Foundation board is very impressive and will govern the network until decentralisation. We feel that full decentralisation at launch is dangerous for the project. Our friends and competitors are living proof that too fast decentralisation could be very challenging. That’s why we’ve created a Steering Committee that will assist the Foundation Board. This steering committee is made of professionals, developers, RustLang Foundation members, researchers and community members. Beni Issembert: What are your views on the recent NFT craze? Further, according to you, what new trends are we going to see moving forward? Ishan Pandey: NFT is not new to me, actually. We are working on a token standard that will support the tokenisation of art. For the first time in our short blockchain history, we have a real use case that doesn’t serve the past, but that opens the doors of the future. This is really new and refreshing. Because if you think, besides BTC and Elon Musk, what are the current use-cases running on top of Blockchain? Not a lot. But wait, Concordium is coming and will enable the Steve Jobs of tomorrow to create the unicorns of the future. Beni Issembert: The purpose of this article is to remove informational asymmetry existing today in our digital markets by performing due diligence by asking the right questions and equipping readers with better opinions to make informed decisions. The material does not constitute any investment, financial, or legal advice. Please do your research before investing in any digital assets or tokens, etc. The writer does not have any vested interest in the company. Ishan Pandey, legal researcher at Karm Legal Consultants.

Compliance by Design is the future of Blockchains - Beni Issembert, CMO at Concordium

Earlier this month, WhatsApp started showing a notice informing users of an update to the app's privacy policy. The update seemed to indicate that WhatsApp would share user data with its parent company, Facebook. Nevermind that WhatsApp had already been doing it , the notice raised awareness on the data-sharing scheme and pushed millions to look for alternatives to the messaging app. since 2016 Many flocked to Signal, the secure messenger endorsed by Edward Snowden and already used by journalists and activists worldwide (in fact, Signal got so many new users that its servers went down for a few hours on Friday). But even larger numbers turned to Telegram: over 25 million people registered on the app within 72 hours. The same thing happened when Facebook purchased WhatsApp in 2014, and in 2019, when WhatsApp a vulnerability in the app, and thousands of alarmed users--particularly activists--turned to Telegram for safety. discovered This increased awareness of digital privacy risks is certainly a positive development. But Telegram is seen as the gold-standard for secure messaging is deeply concerning. So here are 7 reasons why Telegram isn't as secure as it paints itself to be. 1. Chats are not end-to-end encrypted by default By contrast to Signal or WhatsApp, conversations are not end-to-end encrypted by default. This means that anyone with access to the Telegram servers can read user chats (messages, photos, audio recordings, etc)--whether that's Telegram staff, a hacker who manages to get in, or a government serving the company a subpoena. We don't know whether Telegram staff has accessed user conversations in the past or handed over data to governments, but we don't need to know: if they want to, they can. Telegram does support end-to-end encrypted "secret chats", which keep data encrypted even on Telegram servers. But users need to go out of their way to start a secret chat, and in the process lose significant functionalities, such as cross-device syncing (secret chats started on the phone are not viewable on Telegram desktop, and vice versa). Because they join Telegram expecting an already-secure messenger, casual users are often not aware that default chats are not fully private. (It's also worth noting that secret chat is not a particularly unique feature: Facebook Messenger and Skype support similar features) 2. Group chats are not end-to-end encrypted Just like regular one-on-one chats, group chats are not end-to-end encrypted. Except there, Telegram does not offer an option for a secret group chat at all. So if you are on Telegram and want a truly private group chat, you're out of luck. 3. Full chat histories are stored in the cloud The entirety of users' chat history is stored on Telegram servers. This means that if someone gets access to your account, they are able to read every message of every conversation you've ever had on the app (except for secret chats and those you have manually deleted). of security researcher The Grugq, "this is a security nightmare". In the words To get access, a hacker would need to intercept the verification code that Telegram sends when a user activates the app on a new device. This is not theoretical, it happens in real life-- in places where criticizing the government can land you in jail. There are ways of protecting yourself against this threat, such as enabling an extra password to register a new device, but yet again, most users are unaware of those risks. including This is not an issue unique to Telegram: other messengers that use the phone number as an identifier face the same risks. What is unique about Telegram is that it stores all conversations on its servers. Other apps like WhatsApp or Signal do not store chat history on their servers, leaving it to the user to back up their conversations on their device. Telegram storing chat history in the cloud adds a great deal of convenience (all conversations are available on any device at any time), but presents serious risks that are often not clear to ordinary users. 4. Metadata collection Metadata is data about your conversations. It can include who you talk to, when, and for how long; it can identify members of a group chat, their locations, IP addresses, etc. Even without access to the content of a conversation, metadata can reveal a lot about someone's life--so much so, that a former head of the NSA that the US government kills enemies just based on metadata. When WhatsApp says it shares data about its users to Facebook, the data it's talking about is that metadata. admitted Like WhatsApp, Telegram collects metadata about its users, including IP addresses that can reveal the users' location. To many users, that's much better than Facebook holding that data. But others many not want anyone, not even Telegram, to know who they are talking to, when, and from where. This is all the more regretful because private messaging can work perfectly well without collecting metadata. In 2016, as part of an FBI investigation, the US court subpoenaed Signal to handover all the information it had about two of its users. Because Signal is end-to-end encrypted, and therefore has no access to user conversations, all that the team could share with the court was metadata and all the metadata available: the date the users created their Signal account, and the date they were last connected. That's all. it did share Collecting user metadata is a choice apps make, and Telegram made the privacy-invasive choice. Above: the entirety of the user metadata Signal handed over to the FBI 5. More of a social network than a secure messenger While Telegram calls itself a "messaging app with a focus on security and speed", over the years, it’s become more of a social network. Telegram supports groups of up to 200,000 members and channels where admins can broadcast messages to millions, similar to a Twitter feed or Facebook page. ﻿﻿There is of course nothing wrong with being a social network. The problem is that users are misled to think that Telegram is a particularly secure or private one. In August 2019, when millions of Hongkongers turned to Telegram groups to coordinate protests against Chinese interference, a vulnerability in the app. An attacker was able to identify the phone numbers of those who were part of those Telegram groups--even if those users had set their app preferences to hide their phone number. This meant that if that attacker was the government, they could then use the phone number to track down the users’ real world identities--and detain them for their activism.The issue was , but it showed yet again that many Telegram features of Telegram were not built with privacy as a primary concern. was discovered quickly fixed And like all social networks, Telegram uses censorship: it shuts down groups and channels when it is under public or political pressure--like when the app a hub of ISIS propaganda or, just last week, when white nationalists to Telegram after being booted from all major social networks. became turned 6. A great deal of confusion Telegram brands itself as a "secure" app and says its chats are "highly encrypted". But as all of the above shows, it's a little more complicated than that. Secret chats are end-to-end encrypted, but not regular chats. Telegram's website says nothing about groups being end-to-end encrypted (they are not), so many users may mistakenly believe that they are. Voice calls and video calls are end-to-end encrypted. To an ordinary user--not a security expert or a hacker--this is all pretty confusing and error prone. Users who buy into Telegram's branding are likely to feel like anything they do in the app is safe. Others may mistakenly open a regular chat instead of a secret chat, or forget which feature is end-to-end encrypted and which isn't. All in all, Telegram was either not designed to protect user privacy, or protecting user privacy was poorly executed. 7. Worthy of our trust? A big part of digital security is about trust. It's about asking ourselves questions like: Who are we willing to trust with our data and digital identities? Do they have the right motives? What is their history and reputation? Asking ourselves those questions about Telegram, it's hard to recommend the app. The app was launched by Russian billionaire Pavel Durov and his mathematician brother Nikolai, the creators of VK, Russia's most popular social network. ﻿When in 2011 VK topped 100 million users, Durov starting attracting the anger of the Kremlin for refusing ﻿to censor Putin's political opposition. By 2014, the situation had escalated to the point where Durov was ousted from the company and left Russia. This is when Durov launched Telegram. He that "﻿the No. 1 reason for me to support and help launch Telegram was to build a means of communication that can’t be accessed by the Russian security agencies". But Durov's aim to help Russians bypass censorship says little about his commitment to privacy in general. said at the time The first warning sign about Telegram came early on. When designing the app, the Durov brothers ignored cryptography's golden rule, a consensus among security experts: don't create your own encryption scheme. Cryptography is so complex that designing an encryption algorithm from scratch almost inevitably leads to mistakes, so developers should instead use time-tested, recognized algorithms. But Telegram arrogantly decided to create its own homegrown encryption technology, and as expected it was (which remained unaddressed for years). filled with errors In the following years, despite fixing those initial cryptographic flaws, Telegram did nothing to address the many worrying weaknesses, and continued to paint itself as the most secure and private app out there--often making his case by that Signal cannot be trusted (despite the universal endorsement of the app by cryptographers, security experts, and whistleblowers). arguing Years later, Durov's appetite to cash-in on the app further raised eyebrows. ﻿﻿In 2018, Telegram raised a whopping $1.7 billion from investors to launch a cryptocurrency on the platform--the largest amount ever raised for a cryptocurrency launch. The move was expected to generate a whole lot of cash for Telegram's founders. The project was eventually after a US court ruled that Telegram didn't comply with financial regulations, but the whole saga raised questions about Durov's motives. abandoned Finally, though initially based in Berlin, Telegram has operated out of Dubai since 2017. Durov that it moved to the United Arab Emirates for its fiscally advantageous policy. This was, according to him, out of belief in small government rather than financial motivation--though the billion-dollar cryptocurrency fundraising was announced just months after the move to the tax-free zone. explained Most worrying, the UAE is a , routinely jailing political dissidents and journalists for critizing the ruling family. It is a violent player in regional politics, backing dictatorships against democratic movements, waging a cold war against Iran, and fueling proxy wars in Yemen, Libya, and Syria. The UAE is an unabashed enemy of freedom of speech, with a long list of political enemies. Durov claims that its server infrastructure is distributed around the world and strategically set up to avoid the jurisdiction of any one country, including of the UAE. With no means of verifying those claims, we'll have to take his word for it and hope that the billions of conversations stored in Telegram's servers are, indeed, safe. deeply repressive state Conclusion As the Electronic Frontier Foundation , it would be silly, dangerous even, to give blanket recommendations on what communication tool is the most "secure". Security is subjective and personal. What is it you want to protect--the content of your conversations, who you talk to, your identity? Who do you want to protect it from--Facebook, Google, the US government, the Chinese government, your boss, your stalking ex-lover? Each app has strengths and weaknesses, and different needs call for different apps. explains Telegram may well fit some people's security needs, and it may offer social networking features that people are looking for, free of Facebook surveillance. But the app’s self-branding as "secure" and "highly encrypted" is deceptive and puts users at risk. From to or , there are many apps that were designed from the ground up with security in mind. Telegram is simply not one of them. Signal Wire Briar

Insecure by Design: As Millions Flock to Telegram, 7 Reasons to Question the App's Privacy Claims

This story begins and ends with algorithms, those series of functions so mathy and boring that rather than think about them at all, most of us would prefer listening to our nine-year-old nephew rattle off a list of his 255 most-favorite Pokemon, organized from most to least interesting. is a clear #255, by the way. Psyduck But I think we’re all at least vaguely aware of the importance of algorithms in our techy world. We know they basically run the world: the NSA uses them to sift through all your personal data, crypto whales use them to control prices of whatever coin you’re throwing your life savings into, Facebook uses them to figure out what fake news it will float to the top of the feed to wreck democracy today, and Medium uses them to tell you exactly what you will be reading this week. We’re surrounded by them, inundated with them. And they can be really useful. For example, they enable Youtube to push my favorite content creators’ latest videos to the top. Great, right? I never miss a thing. On Netflix, they (mostly) helpfully suggest new shows or movies based on my viewing history. More entertainment. Good, right? It turns out that algorithms provide all sorts of useful suggestions that make your life easier. They feed you the content you love faster than you could on your own. And they’re absolutely fantastic if you never want to change. You see, for all their predictive power, their fortune-telling ability, and behavior influencing, algorithms are necessarily based on past data. They help shape the future version of you based on the past version of you. They market these suggestions as “tailored to your individual interests,” but the reality is that it’s all a recipe for stagnation. Algorithms that never want me to change are why my Medium feed is a guilt-trip-laden cornucopia of lessons on how to improve my writing/why I’m not improving my writing and why my Youtube feed is one long escapist fantasy of Rocket League videos. But algorithms aren’t destiny, are they? You can throw in a wildcard, mix it up a little bit, right? Sure, you can. And you probably should. But you have to go out of your way for that. And these predictive algorithms, by design, discourage searching outside of the box. They present a laundry list of content you consume, inundating you with more of the same. Why would you go searching for new content, an effortful process, when there’s all this readily-available tailored content already at your fingertips? It’s the Amazon convenience effect, right? could You’re not going to get off your duff and hunt for something when you can just click on and consume what’s right here in front of you. Now, in some ways, this isn’t exactly a new problem. In fact, this entry stalled in the drafting phase for a few weeks once I realized that change has always been hard. Algorithms didn’t start this mess. As a species, we have been complacent. always Sure, there are the visionaries (God bless them) who push us forward and say things like, “What if we explored that fertile-looking grassland over there…?” or “Let’s go to space, y’all!” Without them, we might just keep doing the same things in the same ways and fall back on lazy excuses about “tradition” and “the way things have always been done.” Really, though, that’s just our lazy streak showing through. Thankfully, those visionaries get smart, try new things, and make our lives easier. And sometimes they make things easier by concocting algorithms to automate and simplify everything for us. And here we are again. It’s not that Medium’s or Google’s algorithms represent some break in the chain of human development. It’s not like those companies invented personal stagnation. These predictive algorithms just represent our contemporary, high tech version of an age-old problem: change is hard. It requires effort. If it weren’t Google trying to keep you exactly the same as you were yesterday, it would just be your own complacency, your neighbors, family, or community. In fact, those actors are probably still working at it, too. When we move into a new phase of technological development, it’s not like the old forms and orders stop operating, after all. Sometimes I’ve considered rebranding my Medium blog as “shitting on Google.” That’s how this entry started. After working through the problem a little more, however, I realized it’s not quite fair to just blame Google or Medium. They didn’t create this very human dystopia; they just made it high tech. Now it’s on us to go search for something new and to keep challenging ourselves to change. The whole Internet is at our fingertips, so let’s use it to fulfill the liberatory promise that it holds out, rather than succumbing to the languishing power of algorithmic decay. To that end, today, I’m going deep down the rabbit hole of learning Norse mythology because why the hell not? I have to keep Google on their toes after all.

Predictive Analytics and You: Stagnation by Design

An Introduction To Eugenics There is grandeur in this view of life, with its several powers, having been originally breathed into a few forms or into one; and that, whilst this planet has gone cycling on according to the fixed law of gravity, from so simple a beginning endless forms most beautiful and most wonderful have been, and are being, evolved-Charles Darwin, The Origin of Species There is no scientific study more vital to man than the study of his own brain. Our entire view of the universe depends on it — Francis Crick, Co-Creator of The Double Helix Structure Of Human DNA They say truth is stranger than fiction. I think fiction can be used to make the truth seem stranger. A bit like tinkering with nature to make it even more desirable. Changing the laws of nature is not a very new endeavor. In fact it is ancient and it is a discipline called Eugenics. In fact, Eugenics can be traced as far back as Plato who suggested applying the principles of selective breeding of humans as early as 400 BCE(source:Wikipedia) Modern Origins In a way, the term ‘survival of the fittest’ first laid out in Darwin’s theory of natural selection is a succinct explanation of Eugenics. Darwin, along with Alfred Wallace presented a paper on the concept of Natural Selection in 1858. This concept became the backbone of modern biology. Subsequently, Darwin published an influential book ‘ ’ in 1859. On the Origin of Species by Means of Natural Selection, or the Preservation of Favoured Races in the Struggle for Life Inspired by Darwins work, the term ‘Eugenics’ was coined in 1883 by . Francis Galton Old Wine In a New Bottle Source: Wikipedia Eugenics (/juːˈdʒɛnɪks/; from Greek εὐγενής eugenes ‘well-born’ from εὖ eu, ‘good, well’ and γένος genos, ‘race, stock, kin’)is a set of beliefs and practices that aims at improving the genetic quality of a human population. The exact definition of eugenics has been a matter of debate since the term was coined by Francis Galton in 1883. Broadly speaking, there are two types of Eugenics ie positive (encouraging the fit to breed) and negative(discouraging the unfit to breed). While nature has no such laws, humans are another species altogether. Not surprisingly, this technology is always embroiled in controversy. Consider this: This year, there were a record number of Asian American figure skaters at the Pyeongchang Olympics. Japan ended up wining a gold and a silver for Yuzuru Hanru and Shoma Uno. Then, there were revelations by Adam Rippon about quite starvation in figure skating to have a lean frame that would enable him to compete with the other figure skaters who have mastered the quad jumps. One of the reasons for success of many figure skaters is lean frame and minimal body fat sometimes hovering around 4%. Now, imagine if it was possible to reverse engineer such a body type using technologies such as CRISPR or genetic engineering. It would be like creating custom super heroes on demand. The problem is that not everyone can have access to such a disruptive technology. It’s like taking over nature’s role in creation and evolution without giving everyone an equal chance. Thus, Eugenics has evolved from human selection to gene selection and editing. The concept is an old idea but new discoveries such as the double helix structure of DNA and gene editing tools such as CRISPR have refocused attention on Eugenics. The Advent of Molecular Biology While DNA (deoxy ribonucleic acid) was discovered as early as 1869, its role in determining heredity was not known until 1943. Subsequently, James Watson and Francis Crick, in 1953, announced the double helix structure of DNA. globalgenes.org This discovery heralded a new revolution in molecular biology or a way of looking at biology at the cellular and sub-cellular level. Writing in Nature in 1961, William Astbury described molecular biology as: “…not so much a technique as an approach, an approach from the viewpoint of the so-called basic sciences with the leading idea of searching below the large-scale manifestations of classical biology for the corresponding molecular plan..” CRISPR CAS9 While genetic engineering is not a new field, a new editing technique called CRISPR CAS9 promises to herald a new revolution. From solving problems to enhancing the human species, it has a wide range of applications. CRISPR is an abbreviation of Clustered Regularly Interspaced Short Palindromic Repeats. These are clusters of palindromic DNA sequences used by bacteria to overcome viral infections. Scientists used this natural mechanism to edit human DNA. An enzyme called CAS9 is used as a pair of molecular scissors to cut the ‘bad’ DNA and then a type of RNA called guide RNA or gRNA guides CAS9 to identify the portion of defective DNA that need snipping. Thus, the faulty DNA sequence is removed. Capital Chases The Future Genetics has seen a steady surge of venture capital since 2010. As far as biotech is concerned, VC investments surpassed $10 billion for the first time. In 2017, 68 deals worth $1.5 billion were announced: Pitchbook The new frontier for economic warfare is technology. There is a lot of capital lying idle in the world today. Capital that is chasing above market returns. This means a hunt for risk adjusted and hugely profitable investments. From AI to genetics, there is a race among nations and businesses to unlock nature’s secret and reap profits of incredibly profound research. Of course, the corollary of such seismic changes is the global war for talent. In fact, disruptive technologies can perpetuate a self reinforcing cycle where more investments attracts more talent and more talent results in new discoveries that attract more money. Of Humans, Machines and Hybrids Bizzarocomics on Pinterest UK Many modern technologies such as Artificial Intelligence, Gene Editing, Biohacking or creating cyborgs are extremely disruptive for the entire human species. These technologies have the capability to unleash Frankensteins that may be beyond the control of humans. They can tear apart the fabric of life and society as we know it today. Of course, these technologies will also usher in a new era of increase in productivity, new methods of dealing with chronic diseases and solutions to deal with Artificial General Intelligence (AGI) gone rogue. Undoubtedly, controversial becomes an understatement and volatility becomes the norm. Recently, the MIT Technology Review published . The human genome is far more complicated but that complexity is spurring a global race to replicate it. an article on how scientists are creating completely customizable yeast strain called Sc2.0 While a certain segment of scientists ie computer scientists are busy creating a parallel universe called the virtual world, another section of scientists is busy recreating humanity through genetic engineering. Then there are those who are into biohacking or the use of machines to enhance the human body ie creating a hybrid of man and machine. It is hard to say which endeavor is more exciting. But, the human capacity of replicating and altering nature is literally beyond imagination. Ironically, science fiction has become a guiding light or the fiction that has defined our vision for the future. Truth can be stranger than fiction but today science fiction is guiding our efforts to make the truth seem stranger. Many would say our efforts at biohacking may be premature but at least we are looking at a backup plan. The implications of continued advances in genetic engineering and Eugenics on real life will be profound. Most importantly, man can direct his own evolutionary destiny. This also means that he will have access to technologies of mass disruption. One of the most poignant books on this topic was written in 1896. It was called “The Island of Dr. Moreau” written by HG Wells. The book tells the story of a shipwrecked sailor landing on an island run by a mad scientist that carries out experiments to fuse humans and animals through vivisection. The book was a dystopian view and a work of fiction but it deals with a number of topics including emotions and empathy. It directs the reader, as I am, to consider the other side of technology i.e. morality.

Perfect By Design

Joel Spolsky came up with the term “ ” and its associated law: Leaky Abstraction All non-trivial abstractions, to some degree, are leaky. As programmers and system builders, we leverage abstraction at every turn. When an abstraction leaks, some characteristic of the underlying implementation spoils the purity of the abstraction. A programmer needs to worry about tuning their use of the abstraction to take into account these messy details. Or the program breaks or performs badly. One can quibble with the equivocation in this law even while agreeing with the basic principle. It’s hard to disprove — if the abstraction doesn’t leak, one can claim it is trivial. Alternatively, leaking “to some degree” covers a lot of ground. But the general examples Joel provides resonate. Whether it is trying to abstract a reliable connection over an unreliable transport, or even something so basic as trying to abstract a flat memory address space in the face of processor cache locality effects, having to deal with the characteristics of the implementation underlying the abstractions they build on are issues any developer has wrestled with as they develop a system. Joel’s original post is a long lament with little guidance. Abstraction is key to managing complexity in a large system and yet these leaky abstractions mean things really just keep getting more and more complex. A developer needs to master more and more technologies. The best engineers embrace this leakiness, working to deeply understand why it arises and how they need to account for it. Lamenting it is really a failure of end-to-end design. I wanted to consider a couple issues related to leakiness. In some cases, the problem with the abstraction and the cause of leakiness is that the abstraction tries to abstract or hide “the wrong thing”. A common type of design problem is trying to abstract something that can’t successfully be abstracted to begin with. The most common and perhaps fundamental example is latency — wrapping something in a synchronous API to make it look local and fast when it is really remote and slow. While I was working on Office, we actually shipped a version of Excel that after saving a file would query the file handle for the new modified time stamp. For a local NTFS file, this information was effectively cached in memory and so the call was fast and cheap. When running against the “DAV redirector”, which abstracted the file system across the remote DAV protocol, asking for the last modified time would download the entire file in order to retrieve the metadata associated with it (even though the underlying DAV protocol implementation had provided the new modified time as part of the response to the save operation). So effectively on every save we would upload the entire file and then immediately download it again in order to retrieve a 64-bit value. In Word 2003, 5% of all hangs were caused by a synchronous call to test for the existence of a file prior to displaying the MRU list. For files on network drives, this could take a long time to time out (especially when moving between home and work networks) and the user would kill the application while it was sitting there hung, waiting for the time out. One way of dealing with leakiness is simply not trying to abstract something where the failure of the abstraction is so profound that it impacts (or should impact) the overall structure of how the abstraction is used. Often the best approach is to adapt the abstraction itself, not the usage of it. You can’t “abstract” that P = NP (at least I don’t think so!) A surprisingly hard problem is how to design a system that is “intentionally leaky” — where you can provide higher level functionality while still exposing internal layers that allow someone building on top of your component direct access to those lower layers. This is a common problem in building UI application frameworks. Office worked closely with the XAML and Windows composition teams to come up with the right design that both provided access to high level XAML functionality while allowing direct and highly performant access to composition and rendering. This seems like an eternal problem since every new windowing / control framework seems to start with the perspective that “windows are going to be really cheap so I don’t need to expose a lower-level primitive” and then they keep getting heavier and more expensive as more and more functionality is accreted on. Eventually the framework ends up going down a path of “window-less controls” or other mechanisms to reduce cost. Ultimately the real solution is to add explicit layering — which is the hard design problem. One of the long-running layering controversies inside Microsoft was the topic of “Exchange on SQL”. Exchange runs on “ ”, a database engine that also underlies many other technologies in the company (e.g. Active Directory) but is separate from the commercial SQL Server engine. There were elements of the company that pushed hard to re-platform Exchange on top of SQL (and in fact they long had working prototypes of such an implementation). The real controversy was not which technology to use for low-level IO. The argument was whether (or how) to expose the underlying data model that Exchange implements (as well as how to change that data model so that it was easier to program against). The argument was that building on top of SQL would allow direct access to the Exchange data model through all SQL’s powerful programming technologies. That is, the intent was that the implementation on top of SQL would be “intentionally leaky”. The controversy was whether that was a feature or a bug. ESENT The view of the Exchange team is that Exchange is an app. The “app semantics” are implemented through a combination of specific database table design as well as the code that operates over that data. Some of that code enforces semantics that are not explicit in the data design (as well as not present in database triggers and other mechanisms embedded in the database itself to enforce integrity). The Exchange code is also tuned to operate with that data design to deliver the performance and responsiveness required by the “app”. Virtually every complex application has these characteristics. Exposing the underlying data representation has 3 problems which are universal to this kind of layering problem. . Some of the semantics are implemented at the app level. In fact, the “verbs” — how the data is modified, can be just as important to application semantics as the “nouns” — the structure of the data itself. Accessing and modifying the data directly through the data access level can break assumptions that are made throughout the rest of the app code. Databases provide mechanisms to define and enforce certain integrity characteristics of the data schema, but often properties are more easily and efficiently enforced at the app level. Semantic Integrity . The design of the data model is typically carefully tuned and designed for the access patterns and requirements of the app built on top of it. Allowing access at this level to third-party code exposes significant performance risk. The risk shows up in two ways. Applications that access the lower layers may make use of them in ways that are unoptimized, bringing the performance of the overall system down. Alternatively, the wider set of use cases that the lower levels need to support either prevents or dilutes efforts to optimize for the more limited behaviors of the app itself. Performance . The detailed data model is effectively a hidden implementation detail of the application. As soon as it is exposed to third parties, it becomes an explicit public contract with rigid requirements around how and when it can be changed. This is a huge constraint on future evolution of an application whose performance is highly linked to the careful design of the data model and the ability to tune it for the load patterns of the application and its continually evolving feature set. Agility These challenges and questions often also play out internally to the implementation of an application. A data model is often designed with layers of more restrictive constraints and semantics. For example, at one level the FrontPage data model is just a text buffer and associated ordered array of nodes with pointers into the buffer. At the next level, those nodes define an over the text buffer. At another level, it is a full HTML tree with significant semantic constraints on how and where nodes can be nested. Where exactly to enforce those constraints and whether to expose simpler low-level capabilities at higher levels is an interesting challenge. For example, it was convenient to implement the “blockquote” action by simply inserting a BLOCKQUOTE node into the node array and setting its start and end pointers to straddle the selected paragraphs. It was a simple and expressive way of implementing the action — in fact significantly less complex than the code it replaced that did explicit tree manipulations. But it meant that the integrity of the data structure at the full semantic level was now dependent on the correctness of those 6 lines of code (and the hundreds of other examples of similar manipulations sprinkled through the code base). A significant ongoing source of instability was tree consistency, although typically after doing messy merging or pruning of complex trees in actions like Cut and Paste rather than smaller actions like blockquote. interval tree In an evolving system, you can have two alternate sources of layering stress, both tied to this same dynamic of more and more code deeply “correlated” with the underlying data representation. In the first case, you decide not to expose these internal layers because the overhead of going through these thin extra layers is low. But as these layers because more and more functional and the lower layers become bound to the semantics and load of the higher layers, the promise of transparent access to those lower layers becomes less and less true. In the second case, you start by exposing these lower layers (maybe because initially your mapping from data model to semantic model is quite straight-forward), but over time allowing access to these lower layers is a significant ongoing constraint and overhead on the evolution of the system. There is no free lunch. is how hackers start their afternoons. We’re a part of the family. We are now and happy to opportunities. Hacker Noon @AMI accepting submissions discuss advertising &sponsorship To learn more, , , or simply, read our about page like/message us on Facebook tweet/DM @HackerNoon. If you enjoyed this story, we recommend reading our and . Until next time, don’t take the realities of the world for granted! latest tech stories trending tech stories

Leaky by Design

Disclaimer: The author works at Microsoft. The article is a general exploration of user trust and AI, but it does call out some references to the author’s company’s products such as Copilot/Azure Open AI. “I knew I had just seen the most important advance in technology since the graphical user interface.” — Bill Gates, about ChatGPT acing an AP-level biology test Generative AI’s surge in popularity has opened up new and innovative possibilities, transforming the way we live and work. According to a study conducted by PwC, AI could contribute up to $15.7 trillion to the global economy in 2030. A Forbes Advisor report shows that a staggering 97 percent of business owners believe ChatGPT will help their business. These are exciting times for new startups specializing in Generative AI. However, there is a dichotomy in how end users feel about the Generative AI revolution. For instance, 46 percent of Americans are equally concerned and excited about AI (Artificial Intelligence). This type of user mistrust and skepticism has the potential to encumber AI adoption. What are some ways to mitigate inaccurate or misleading content generated by Large Language Models? How can companies build products that users not only love but trust? Hallucinations Large Language Models (LLMs) have been labeled stochastic parrots — a fitting metaphor for entities that regurgitate convincingly plausible answers without really understanding the meaning behind them. As Stephen Wolfram put it, a model like ChatGPT is simply adding one word at a time. LLMs are trained on massive amounts of data from various sources but aren’t by themselves doing any lookups or referring to external sources of information. This means that an LLM can make misleading claims or factual errors — while sounding fairly confident about the mistake. These counterfactual statements are, somewhat anthropomorphically, nicknamed hallucinations. To some extent, LLMs are meant to hallucinate. It’s Generative AI’s version of “it’s not a bug — it’s by design.” The same capability that makes LLMs hallucinate is also what makes them powerful. On the flip side, there are situations when hallucinations are not desirable, and where it is critical to get truthful and factual responses. Working around the non-determinism of GenAI products continues to be a persistent challenge. But there are ways to ensure that we can still maintain user trust while harnessing all the creativity and potential behind the AI. Adding a Grounding Source of Truth One way to mitigate hallucinations is to augment the knowledge an LLM possesses with a grounding source of truth. Imagine you’re making someone take a test on a complex, previously unseen topic. This person is a fluent English speaker, but they know next to nothing about the topic they’re being tested on. Now, they could simply fabricate answers for all the questions. Or, you can give them an open textbook about the topic they’re being tested on, so they have reference material from which they can answer. This example is a simplified explanation of what grounding represents in the context of LLMs. Grounding is table stakes while creating any kind of custom LLM application, especially for internal data (such as customer service chatbots that rely on non–public-facing knowledge bases) or applications that rely on a corpus to retrieve answers (a.k.a., search engines). As far as implementation goes, Retrieval Augmented Generation is probably the most ubiquitous pattern currently used for grounding LLMs. There are numerous out-of-the-box RAG solutions in the market that take in a custom corpus to do retrieval and generation. Azure Open AI on your own data is one such example, where users can upload a dataset or document from blob storage and “chat” with it. The feature also offers citations to the source text, along with the response. Making an LLM cite an authoritative source as the reason behind a particular answer automatically increases transparency and adds credibility. Grounding can also help with evaluation and validation of response quality, a key component to testing Generative AI applications before making them production-worthy for public release. In the absence of a grounding dataset, there is no way to “compare” the responses of an LLM to see if they are based on truth or not. So, choosing the right implementation of RAG and the right grounding dataset is critical for improving the factualness of the GenAI application. Increasing Explainability Surveys have shown that people think providing too little context on AI decisions can erode trust. Deep Learning models are trained to get accurate and fast predictions, but the end users are often clueless as to why the model generated a particular response. A famous example of this black box behavior occurs in my favorite work of fiction, Douglas Adams’ brilliant Hitchhiker’s Guide to Galaxy. In the book, the supercomputer Deep Thought is tasked with working out the Answer to The Ultimate Question of Life, the Universe, and Everything. After 7.5 million years, Deep Thought responds that the answer is 42. The scientists are puzzled, but Deep Thought is quite confident about the answer: “I think the problem, to be quite honest with you, is that you’ve never actually known what the question was.” In order to then calculate the Ultimate Question, a special computer is built, and hilarities ensue — but I won’t spoil the book any further. Decades after the book was written, we now have ways to peek behind the curtain, to understand why an AI generated a particular response. While traditional Machine Learning (ML) models use techniques like LIME and SHAP for explainability, Chain-of-Thought (CoT) prompting is a popular technique to add more transparency to Generative AI’s reasoning process. In essence, CoT consists of meta-prompts asking the AI to explain itself or interpret why it arrived at a particular output. In fact, researchers have noted that just adding a phrase like “Let’s think step by step” also increases the accuracy of generated responses, even in zero-shot scenarios. Employing Guardrails Apart from grounding, there are also various other techniques to reduce hallucinations including advanced prompt engineering with CoVE (chain of thought verification), feedback loops with iterative improvements (or “humans in the loop”), using a combination of fine-tuned models with RAG, asking an LLM to self-critique its own output, as well as playing around with the “creativity” and “temperature” of LLM APIs. However, none of these methods are infallible since an LLM is essentially non-deterministic and unpredictable. There could be grave ethical as well as practical implications from Artificial Intelligence making decisions without any oversight. Incorrect predictions could lead to potentially life-threatening implications in fields like medical sciences, pharmacology, autonomous vehicle design, law enforcement, or disaster management. In the end, the safest option is for humans to have the final say in accepting or rejecting any AI-driven predictions. Having the end user in charge of vetting AI decisions also serves as a great feedback loop for developers to learn from and iterate continuously. Efforts to reduce hallucinations by improving factualness and transparency will increase user confidence in AI products. In addition, accountability is a key Responsible AI principle that must be at the center of all AI design. AI can be an amazing tool to revolutionize the way we all live and work — but ultimately, not to replace human judgment. Disclaimer: The author works at Microsoft. The article is a general exploration of user trust and AI, but it does call out some references to the author’s company’s products such as Copilot/Azure Open AI. Disclaimer: The author works at Microsoft. The article is a general exploration of user trust and AI, but it does call out some references to the author’s company’s products such as Copilot/Azure Open AI. “I knew I had just seen the most important advance in technology since the graphical user interface.” — Bill Gates, about ChatGPT acing an AP-level biology test “I knew I had just seen the most important advance in technology since the graphical user interface.” — Bill Gates, about ChatGPT acing an AP-level biology test “I knew I had just seen the most important advance in technology since the graphical user interface.” Bill Gates, about ChatGPT acing an AP-level biology test Bill Gates, about ChatGPT acing an AP-level biology test Generative AI’s surge in popularity has opened up new and innovative possibilities, transforming the way we live and work. According to a study conducted by PwC, AI could contribute up to $15.7 trillion to the global economy in 2030 . A Forbes Advisor report shows that a staggering 97 percent of business owners believe ChatGPT will help their business . These are exciting times for new startups specializing in Generative AI . AI could contribute up to $15.7 trillion to the global economy in 2030 97 percent of business owners believe ChatGPT will help their business new startups specializing in Generative AI However, there is a dichotomy in how end users feel about the Generative AI revolution. For instance, 46 percent of Americans are equally concerned and excited about AI (Artificial Intelligence). This type of user mistrust and skepticism has the potential to encumber AI adoption. What are some ways to mitigate inaccurate or misleading content generated by Large Language Models? How can companies build products that users not only love but trust? 46 percent of Americans are equally concerned and excited about Hallucinations Hallucinations Large Language Models (LLMs) have been labeled stochastic parrots — a fitting metaphor for entities that regurgitate convincingly plausible answers without really understanding the meaning behind them. As Stephen Wolfram put it, a model like ChatGPT is simply adding one word at a time. stochastic parrots adding one word at a time. LLMs are trained on massive amounts of data from various sources but aren’t by themselves doing any lookups or referring to external sources of information. This means that an LLM can make misleading claims or factual errors — while sounding fairly confident about the mistake. These counterfactual statements are, somewhat anthropomorphically , nicknamed hallucinations . somewhat anthropomorphically hallucinations To some extent, LLMs are meant to hallucinate. It’s Generative AI’s version of “it’s not a bug — it’s by design.” The same capability that makes LLMs hallucinate is also what makes them powerful. On the flip side, there are situations when hallucinations are not desirable, and where it is critical to get truthful and factual responses. meant Working around the non-determinism of GenAI products continues to be a persistent challenge. But there are ways to ensure that we can still maintain user trust while harnessing all the creativity and potential behind the AI. Adding a Grounding Source of Truth Adding a Grounding Source of Truth One way to mitigate hallucinations is to augment the knowledge an LLM possesses with a grounding source of truth. Imagine you’re making someone take a test on a complex, previously unseen topic. This person is a fluent English speaker, but they know next to nothing about the topic they’re being tested on. Now, they could simply fabricate answers for all the questions. Or, you can give them an open textbook about the topic they’re being tested on, so they have reference material from which they can answer. This example is a simplified explanation of what grounding represents in the context of LLMs. Grounding is table stakes while creating any kind of custom LLM application, especially for internal data (such as customer service chatbots that rely on non–public-facing knowledge bases) or applications that rely on a corpus to retrieve answers (a.k.a., search engines). As far as implementation goes, Retrieval Augmented Generation is probably the most ubiquitous pattern currently used for grounding LLMs. There are numerous out-of-the-box RAG solutions in the market that take in a custom corpus to do retrieval and generation. Retrieval Augmented Generation Azure Open AI on your own data is one such example, where users can upload a dataset or document from blob storage and “chat” with it. The feature also offers citations to the source text, along with the response. Making an LLM cite an authoritative source as the reason behind a particular answer automatically increases transparency and adds credibility. Azure Open AI on your own data Grounding can also help with evaluation and validation of response quality, a key component to testing Generative AI applications before making them production-worthy for public release. In the absence of a grounding dataset, there is no way to “compare” the responses of an LLM to see if they are based on truth or not. So, choosing the right implementation of RAG and the right grounding dataset is critical for improving the factualness of the GenAI application. Increasing Explainability Increasing Explainability Surveys have shown that people think providing too little context on AI decisions can erode trust . Deep Learning models are trained to get accurate and fast predictions, but the end users are often clueless as to why the model generated a particular response. providing too little context on AI decisions can erode trust why A famous example of this black box behavior occurs in my favorite work of fiction, Douglas Adams’ brilliant Hitchhiker’s Guide to Galaxy . Hitchhiker’s Guide to Galaxy In the book, the supercomputer Deep Thought is tasked with working out the Answer to The Ultimate Question of Life, the Universe, and Everything. After 7.5 million years, Deep Thought responds that the answer is 42. The scientists are puzzled, but Deep Thought is quite confident about the answer: “I think the problem, to be quite honest with you, is that you’ve never actually known what the question was.” “ I think the problem, to be quite honest with you, is that you’ve never actually known what the question was. ” I think the problem, to be quite honest with you, is that you’ve never actually known what the question was. question In order to then calculate the Ultimate Question, a special computer is built, and hilarities ensue — but I won’t spoil the book any further. Decades after the book was written, we now have ways to peek behind the curtain, to understand why an AI generated a particular response. While traditional Machine Learning (ML) models use techniques like LIME and SHAP for explainability, Chain-of-Thought (CoT) prompting is a popular technique to add more transparency to Generative AI’s reasoning process. In essence, CoT consists of meta-prompts asking the AI to explain itself or interpret why it arrived at a particular output. In fact, researchers have noted that just adding a phrase like “ Let’s think step by step ” also increases the accuracy of generated responses, even in zero-shot scenarios. Let’s think step by step Employing Guardrails Employing Guardrails Apart from grounding, there are also various other techniques to reduce hallucinations including advanced prompt engineering with CoVE (chain of thought verification), feedback loops with iterative improvements (or “humans in the loop”), using a combination of fine-tuned models with RAG, asking an LLM to self-critique its own output, as well as playing around with the “creativity” and “temperature” of LLM APIs. However, none of these methods are infallible since an LLM is essentially non-deterministic and unpredictable. There could be grave ethical as well as practical implications from Artificial Intelligence making decisions without any oversight. Incorrect predictions could lead to potentially life-threatening implications in fields like medical sciences, pharmacology, autonomous vehicle design, law enforcement, or disaster management. In the end, the safest option is for humans to have the final say in accepting or rejecting any AI-driven predictions. Having the end user in charge of vetting AI decisions also serves as a great feedback loop for developers to learn from and iterate continuously. Efforts to reduce hallucinations by improving factualness and transparency will increase user confidence in AI products. In addition, accountability is a key Responsible AI principle that must be at the center of all AI design. AI can be an amazing tool to revolutionize the way we all live and work — but ultimately, not to replace human judgment.

Pursuit of Trustworthiness: Increasing User Trust in Generative AI Products

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Ghazaleh H. Torbati, Max Planck Institute for Informatics Saarbrucken, Germany & ghazaleh@mpi-inf.mpg.de;
(2) Andrew Yates, University of Amsterdam Amsterdam, Netherlands & a.c.yates@uva.nl;
(3) Anna Tigunova, Max Planck Institute for Informatics Saarbrucken, Germany & tigunova@mpi-inf.mpg.de;
(4) Gerhard Weikum, Max Planck Institute for Informatics Saarbrucken, Germany & weikum@mpi-inf.mpg.de. Table of Links Abstract and Introduction
Related Work
Methodology
Experimental Design
Experimental Results
Conclusion
Ethics Statement and References V. EXPERIMENTAL RESULTS A. Comparison of CUP against Baselines • LLM-Rec also performs poorly. Solely relying on the LLM’s latent knowledge about books is not sufficient when coping with long-tail items that appear sparsely in the LLM’s training data. Popularity and position bias [31] further aggravate this adverse effect. Thus, to address RQ1 (see Section I-B), merely and fully resorting to LLMs alone is not a viable solution. We also ran extended CUP configurations where precomputed CF vectors were incorporated (see Section III-B). These achieved only very minor gains in some of the settings, and there was even a detrimental effect on some cases. We omit these hybrid methods here. We will revisit them in Subsection V-C for sensitivity studies on data density. Another observation is that the weighted training almost always improves performance. Therefore, the following subsections focus on results with weighted training samples. B. Comparison of CUP Configurations • On the GR-1K-rich data, the overall performance is higher than for AM across all configurations. This can be attributed to the fact that GR has longer reviews, and these texts tend to be more informative than the ones in the AM data (which sometimes refer to packaging, shipment and other non-content aspects). C. Influence of Interaction Density The slices AM-1K-rich and GR-1K-rich are constructed with a text-centric stress-test in mind. Both are extremely sparse in terms of user-item interactions. To study the influence of sparseness in isolation, we created two larger samples of both datasets, one still sparse by design and the other denser in terms of users sharing items. We refer to these as AM-10Ksparse and AM-10K-dense, and analogously for GR. All these samples cover 10K users: 10x more users than our previous text-rich slices, to allow more potential for learning from interactions. Table I in Section IV-B shows statistics. The datasets are constructed as follows. To ensure connectivity in the interaction graphs, we first select 500 users and sample 2000 books connected to these users, both uniformly at random. This is the seed for constructing two variants of data, based on the cumulative item degrees of users, that is, the sum of the #users per book for all books that the user has in her collection: • 10K-dense: We randomly select 10K (minus the initial 500) users from all users in proportion to the users’ cumulative item degrees. This favors users with many or popular books. • 10K-sparse: We select users inversely proportional to their cumulative item degrees, thus favoring sparse users. In this sensitivity study, we focus on comparing CF (based on matrix factorization) against our default CUPtext configuration (with idf-selected sentences) and the hybrid combination CUPtext +CF. Tables V and VI show the results for the AM10K data and GR-10K data, comparing the sparse vs. the dense variants with search-based evaluation (corresponding results with standard evaluation are on the project web page). Key insights are: • As the 10K-sparse data is already much denser than the stress-test 1K-rich slices, CF can achieve decent results on the sparse data. Especially in search-based mode, CF is almost on par with CUP. Note that these gains come from the seen items alone, as CF is bound to fail on unseen items. • CUPtext is still the clear winner on the sparse variant. The hybrid configuration, with text and CF vectors combined, is inferior to learning from text alone. • For the 10K-dense data, CF alone performs well, as this is the traditional regime for which CF has been invented. An interesting point arises for the search-based mode and with dense data. Here, the negative test points are closer to the positive sample (after the BM25 search), and pose higher difficulty for text cues alone to discriminate them. Thus, CF becomes more competitive. On GR, hybrid CUP+CF performs best. D. Efficiency of CUP Two architectural choices make CUP efficient: • input length restricted to 128 tokens, and • fine-tuning only the last layer of BERT and the FFN layers. For more analysis, we measured the training time and resulting NDCG on the GR-1K-rich data, comparing different input size budgets and choices of which parameters are tuned and which ones are kept frozen. Figure 2 shows the results. We observe that the 128-token configuration has the lowest training cost: significantly less time per epoch than the other variants and fast convergence (reaching its best NDCG already after 15 epochs in ca. 3000 seconds). The 256- and 512-token models eventually reach higher NDCG, but only by a small margin and after much longer training time. This confirms that concise user profiles, with a small token budget, are the best approach in terms of benefit/cost ratio. As for configurations with more or less tunable parameters, we observe that the variant with frozen BERT takes much longer to converge and is inferior to the preferred CUP method even after more than 50 epochs. The other extreme, allowing all of BERT parameters to be altered together with the FFN parameters, performs best after enough training epochs, better than the CUP configuration that tunes only the last BERT layer. However, it takes almost twice as much time per epoch. So again, from the benefit/cost perspective, our design choice hits a sweet spot in the spectrum of model configurations. This paper is available on arxiv under CC 4.0 license. Authors: (1) Ghazaleh H. Torbati, Max Planck Institute for Informatics Saarbrucken, Germany & ghazaleh@mpi-inf.mpg.de; (2) Andrew Yates, University of Amsterdam Amsterdam, Netherlands & a.c.yates@uva.nl; (3) Anna Tigunova, Max Planck Institute for Informatics Saarbrucken, Germany & tigunova@mpi-inf.mpg.de; (4) Gerhard Weikum, Max Planck Institute for Informatics Saarbrucken, Germany & weikum@mpi-inf.mpg.de. This paper is available on arxiv under CC 4.0 license. Authors: (1) Ghazaleh H. Torbati, Max Planck Institute for Informatics Saarbrucken, Germany & ghazaleh@mpi-inf.mpg.de; (2) Andrew Yates, University of Amsterdam Amsterdam, Netherlands & a.c.yates@uva.nl; (3) Anna Tigunova, Max Planck Institute for Informatics Saarbrucken, Germany & tigunova@mpi-inf.mpg.de; (4) Gerhard Weikum, Max Planck Institute for Informatics Saarbrucken, Germany & weikum@mpi-inf.mpg.de. Table of Links Abstract and Introduction Related Work Methodology Experimental Design Experimental Results Conclusion Ethics Statement and References Abstract and Introduction Abstract and Introduction Related Work Related Work Methodology Methodology Experimental Design Experimental Design Experimental Results Experimental Results Conclusion Conclusion Ethics Statement and References Ethics Statement and References V. EXPERIMENTAL RESULTS A. Comparison of CUP against Baselines A. Comparison of CUP against Baselines • LLM-Rec also performs poorly. Solely relying on the LLM’s latent knowledge about books is not sufficient when coping with long-tail items that appear sparsely in the LLM’s training data. Popularity and position bias [31] further aggravate this adverse effect. Thus, to address RQ1 (see Section I-B), merely and fully resorting to LLMs alone is not a viable solution. We also ran extended CUP configurations where precomputed CF vectors were incorporated (see Section III-B). These achieved only very minor gains in some of the settings, and there was even a detrimental effect on some cases. We omit these hybrid methods here. We will revisit them in Subsection V-C for sensitivity studies on data density. Another observation is that the weighted training almost always improves performance. Therefore, the following subsections focus on results with weighted training samples. B. Comparison of CUP Configurations • On the GR-1K-rich data, the overall performance is higher than for AM across all configurations. This can be attributed to the fact that GR has longer reviews, and these texts tend to be more informative than the ones in the AM data (which sometimes refer to packaging, shipment and other non-content aspects). C. Influence of Interaction Density C. Influence of Interaction Density The slices AM-1K-rich and GR-1K-rich are constructed with a text-centric stress-test in mind. Both are extremely sparse in terms of user-item interactions. To study the influence of sparseness in isolation, we created two larger samples of both datasets, one still sparse by design and the other denser in terms of users sharing items. We refer to these as AM-10Ksparse and AM-10K-dense, and analogously for GR. All these samples cover 10K users: 10x more users than our previous text-rich slices, to allow more potential for learning from interactions. Table I in Section IV-B shows statistics. The datasets are constructed as follows. To ensure connectivity in the interaction graphs, we first select 500 users and sample 2000 books connected to these users, both uniformly at random. This is the seed for constructing two variants of data, based on the cumulative item degrees of users, that is, the sum of the #users per book for all books that the user has in her collection: • 10K-dense: We randomly select 10K (minus the initial 500) users from all users in proportion to the users’ cumulative item degrees. This favors users with many or popular books. • 10K-sparse: We select users inversely proportional to their cumulative item degrees, thus favoring sparse users. In this sensitivity study, we focus on comparing CF (based on matrix factorization) against our default CUPtext configuration (with idf-selected sentences) and the hybrid combination CUPtext +CF. Tables V and VI show the results for the AM10K data and GR-10K data, comparing the sparse vs. the dense variants with search-based evaluation (corresponding results with standard evaluation are on the project web page). Key insights are: • As the 10K-sparse data is already much denser than the stress-test 1K-rich slices, CF can achieve decent results on the sparse data. Especially in search-based mode, CF is almost on par with CUP. Note that these gains come from the seen items alone, as CF is bound to fail on unseen items. • CUPtext is still the clear winner on the sparse variant. The hybrid configuration, with text and CF vectors combined, is inferior to learning from text alone. • For the 10K-dense data, CF alone performs well, as this is the traditional regime for which CF has been invented. An interesting point arises for the search-based mode and with dense data. Here, the negative test points are closer to the positive sample (after the BM25 search), and pose higher difficulty for text cues alone to discriminate them. Thus, CF becomes more competitive. On GR, hybrid CUP+CF performs best. D. Efficiency of CUP D. Efficiency of CUP Two architectural choices make CUP efficient: • input length restricted to 128 tokens, and • fine-tuning only the last layer of BERT and the FFN layers. For more analysis, we measured the training time and resulting NDCG on the GR-1K-rich data, comparing different input size budgets and choices of which parameters are tuned and which ones are kept frozen. Figure 2 shows the results. We observe that the 128-token configuration has the lowest training cost: significantly less time per epoch than the other variants and fast convergence (reaching its best NDCG already after 15 epochs in ca. 3000 seconds). The 256- and 512-token models eventually reach higher NDCG, but only by a small margin and after much longer training time. This confirms that concise user profiles, with a small token budget, are the best approach in terms of benefit/cost ratio. As for configurations with more or less tunable parameters, we observe that the variant with frozen BERT takes much longer to converge and is inferior to the preferred CUP method even after more than 50 epochs. The other extreme, allowing all of BERT parameters to be altered together with the FFN parameters, performs best after enough training epochs, better than the CUP configuration that tunes only the last BERT layer. However, it takes almost twice as much time per epoch. So again, from the benefit/cost perspective, our design choice hits a sweet spot in the spectrum of model configurations.

BYDESIGN

bydesign.io

-- employees

Since n.d.

COMPANY RANKING

BYDESIGN

EVERGREEN INDEX #1661

Representing this company? Click here to claim and customize this page!

ByDesign WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

BYDESIGN

bydesign.io

-- employees

Since n.d.

COMPANY RANKING

BYDESIGN

EVERGREEN INDEX #1661

RELATED COMPANIESRANK

STORIES ABOUT BYDESIGN

Representing this company? Click here to claim and customize this page!

ByDesign WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!