Rioks

rioks

Top 100 Places to Learn to Code for Free Online in 2019.

SETI, Civilizational Longevity, and the Drake Equation: Why AI Risks Matter

Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Methods
2.1 Tokenizer analysis
2.2 Indicators for detecting under-trained tokens and 2.3 Verification of candidate tokens


Results
3.1 Effectiveness of indicators and verification
3.2 Common observations
3.3 Model-specific observations


Closed-source models


Discussion, Acknowledgments, and References Methods
2.1 Tokenizer analysis
2.2 Indicators for detecting under-trained tokens and 2.3 Verification of candidate tokens Methods 2.1 Tokenizer analysis 2.1 Tokenizer analysis 2.2 Indicators for detecting under-trained tokens and 2.3 Verification of candidate tokens 2.2 Indicators for detecting under-trained tokens and 2.3 Verification of candidate tokens Results
3.1 Effectiveness of indicators and verification
3.2 Common observations
3.3 Model-specific observations Results 3.1 Effectiveness of indicators and verification 3.1 Effectiveness of indicators and verification 3.2 Common observations 3.2 Common observations 3.3 Model-specific observations 3.3 Model-specific observations Closed-source models Closed-source models Closed-source models Discussion, Acknowledgments, and References Discussion, Acknowledgments, and References Discussion, Acknowledgments, and References A. Verification details A. Verification details B. A short primer on UTF-8 encoding B. A short primer on UTF-8 encoding C. Outputs for API-based verification C. Outputs for API-based verification 3.3 Model-specific observations In this section we outline some model-specific observations, grouped by the tokenizer used. These examples are mainly intended to illustrate the variety of different under-trained tokens and configuration issues that can be found using our methods, and are not exhaustive. 3.3.1 Models based on the GPT-2 tokenizer 3.3.1 Models based on the GPT-2 tokenizer GPT-2 introduced the framework for much of current-day tokenization and training of LLMs [1], and the tokenizer has been re-used extensively. We confirm previous findings with a significant number of tokens Figure 3: Comparison of (un)embedding based indicators. The scatter plots are coloured by token id, from light green to dark blue. related to (fragments of) usernames (e.g. _TheNitrome, _RandomRedditor, StreamerBot). Although the model is aimed at English text, there are a few under-trained non-English tokens, including the Japanese token _サーティ[4]. In addition to the 13 bytes unused in UTF-8, we detect that all ASCII characters in the 0-31 range, except for the newline character, appear untrained. Although removing or replacing many of these ‘control’ characters is a reasonable normalization step, the tokenizer as published does not perform such normalization. Most notably this means that the horizontal tab character \t as well as the carriage return \r are out of distribution for the models. We also evaluated a few models that base their tokenizer on GPT-2, including Phi-2 and GPT-J 6B [15, 16] [5]. These models share many of the same under-trained tokens, and have significantly more confirmed tokens than GPT-2, likely due to their training data being further removed from the data used to train the tokenizer. These additional tokens include _SolidGoldMagikarp, which is not among verified candidates in GPT-2. 3.3.2 Models based on the GPT-NeoX tokenizer 3.3.2 Models based on the GPT-NeoX tokenizer GPT-NeoX is an open-source library and associated family of models which uses a tokenizer with the same vocabulary size as GPT-2, but trained on the same ‘The Pile’ dataset also used for model training, and with added tokens for multiple spaces [14]. The GPT-NeoX 20B model has very few under-trained tokens, likely in part due to this alignment between tokenizer and model training, with the fragment FFIRMED showing up most consistently. The Pythia 6.7B model based on the same library [17] also shows very similar results. The OLMo open language models [13] also use this tokenizer, but have a much higher rate of under-trained tokens, including a wide range of punctuation-based tokens. We detect over 200 unreachable tokens representing combinations of spaces and line breaks in the tokenizer, which appear to be caused by the aforementioned ‘multiple spaces’ tokens taking precedence. However, many of them appear to have been seen in training, based on both our indicators and provided data on token counts in training[6]. In addition, we noticed that embedding based indicators are not near zero for the NeoX, Pythia, and OLMo v1 models. For the NeoX/Pythia models, this was explained to be due to an specific implementation of weight decay, where only weights that are used in the forward pass are affected. The OLMo v1 model instead applies no weight decay. For the NeoX and Pythia models, we find that having low but non-zero embeddings is still a good predictor for under-trained tokens, and keep the default choice. For the OLMo v1 model, we find a large number of tokens near the minimum of approximately 1, and use the more discriminative metric based on unembeddings. The OLMo v1.7 model does apply weight decay to embeddings, and its embedding norms are near zero for untrained tokens (cf. Figure 2), but we maintain the same choice for more consistent comparison between the two versions. 3.3.3 Llama2 and related models 3.3.3 Llama2 and related models The Llama2 models are a family of models by Meta and use a SentencePiece based BPE tokenizer with 32,000 tokens [18]. We detect no unreachable or partial UTF-8 sequence tokens aside from the expected single byte fallback tokens. The model’s relatively few confirmed under-trained tokens typically relate to specific long non-English words. These include _Mediabestanden, _Portály, and _Расподела. We also observe a few fragments which are ‘occluded’ by more complete words, such as ederbörd and nederbörd (_årsnederbörd, _nederbörd), _gepublic (_gepubliceerd), and oreferrer (noreferrer). Several of these tokens were also found in previous work on optimizing prompts for steering model outputs [8]. 3.3.4 Mistral and related models 3.3.4 Mistral and related models Derived models such as Zephyr beta and variants such as Solar 10.7B use the same tokenizer [12, 21], and show a slight increase in the number of under-trained tokens within our threshold, but no change in the most severely under-trained tokens. The model by Rakuten [22] is an extended-vocabulary Japanese model based on the Mistral 7B base model, with continued pre-training. Among the extended vocabulary we find a few under-trained fragments such as 稲田大学 (早稲田大学, ‘Waseda University’). Their presence is proportional to the extended vocabulary, and the extended vocabulary tokens form a distinct cluster when visualising their indicators (see Figure 3a). 3.3.5 Gemma 3.3.5 Gemma Finally, the Gemma models stand out due to the high similarity between the rows of their (un)embedding matrix, making removal of the first principal component a particularly important step in determining an indicator for detecting under-trained tokens in these models (see Figure 3b. 3.3.6 Command R and R+ 3.3.6 Command R and R+ Cohere’s ‘Command R’ and ‘Command R+’ models [23] also have a large multi-lingual vocabulary with over 255,000 tokens. The most notable discovery in these models was that over 1,400 tokens related to emojis are all clearly untrained according to their indicators, and are categorized as ‘unreachable’ due to the tokenizer using a different representation when encoding text which includes them. Among confirmed under-trained tokens we finds the typical fragments of long words, such as ephritidae (Tephritidae) and ishockeyspieler (Eishockeyspieler). 3.3.7 Models using the OpenAI ‘tiktoken’ tokenizer 3.3.7 Models using the OpenAI ‘tiktoken’ tokenizer A number of models have been published that use the OpenAI ‘cl100k’ tokenizer as used in GPT-3.5 and GPT-4, and published in the ‘tiktoken’ library [25]. The pattern used to ‘pre-tokenize’ text is unusual in allowing not just a single space character in front of words, but any character that is not a number, letter, or line break. This choice results in tokens such as \tTokenNameIdentifier and $PostalCodesNL, which are highly sensitive to pre-tokenization splitting, with leading spaces before the token resulting in different tokenization such as _$, PostalCodesNL. In combination with their specific content, this is likely to have made them more severely under-trained across models. StableLM2 is a model by Stability AI [26], and uses a slightly modified version of this tokenizer, adding digits splitting, as well as special code tokens similar to StarCoder2. Due to the added digit splitting, the original multi-digit tokens in the tokenizer are expected to show up as both unreachable and untrained. However, initially these tokens appeared only as untrained due to a tokenizer configuration error [9]. The Qwen model family by Alibaba significantly extends the tokenizer to over 150,000 tokens with “commonly used Chinese characters and words, as well as those in other languages” [27]. The combination of many thousands of manually added tokens, as well as a training corpus that is likely even further removed from the tokenizer’s training data than usual, results in many under-trained tokens. These include archaic Chinese characters (such as 𬳽) and Korean characters which are typographically valid but never seen in normal text (such as 앐). Llama3 is a recent model by Meta AI which also extends the tiktoken tokenizer with 28,000 additional tokens.[10] Aside from sharing many under-trained tokens with other models using this tokenizer, the newly added tokens include additional under-trained tokens such as ЎыџNЎыџN and krvldkf. 3.3.8 StarCoder2 3.3.8 StarCoder2 StarCoder2 is a series of models resulting from the BigCode project, an open-scientific collaboration collaboration focused on the development of code models [28], and have published both their tokenizer training method and the dataset. The token ittrLoremipumdolorsitametconsecteturadipiscingelitIntegervelvel and fragments thereof are the most eye-catching among the verified under-trained tokens. Being a code focused model, there are a also number of long and specific variable and function names, such as simpleIndexQueryParserTests, and fragments of them. Additional notable under-trained tokens include the words for a number of Swiss German dialects (Ostschwizertütsch, Baseldytsch, _Züritüütsch, _Bärndütsch), and a number of seemingly random tokens such as BjKPZFq. The open nature of the project represents a great opportunity for further investigation, and allowed us to determine the source of these tokens in the tokenizer training data. We find a single document which repeats ‘LoremipumdolorsitametconsecteturadipiscingelitIntegervelvelittr’ to illustrate maximal variable lengths in Java, a single document with base-64 encoded strings as the origin of the random looking tokens, and a single source code file with a list of solutions of a German Wordle game with words categorized by dialect. As mentioned in section 3.2.1, some models exclude bytes not used in UTF-8. The StarCoder2 tokenizer is unique in additionally missing the 0xF1 fallback byte. Although this byte is not used in any defined Unicode block, an example phrase with a Unicode escape sequence in that range such as \U0006baeb encodes to [0, 142, 142, 142] where token id 0 is the special <|endoftext|> token being used in the absence of both an 0xF1 byte and a specialized token. This could potentially be used to disrupt systems which use not only this specific model, but any derived fine-tuned models regardless of their training data. 3.3.9 Yi 3.3.9 Yi Yi-9B is a base model by 01.ai whose training data is focused on English and Chinese [29]. The model has a number of typical under-trained tokens, including punctuation-based tokens, (partial) non-English/Chinese words such as Разпространение, and some apparent user handles including mabaochang. Unique to the model are a number of strange tokens starting with ‘n’, including nConsequently and nInterestingly which may have been caused by incorrectly processing newline characters in some data. In addition, three tokens with Chinese phrases are unusual unreachable tokens, as they re-encode to a different sequence, including 毛泽东 (‘Mao Zedong’), which tokenizes to three separate tokens despite the presence of a dedicated token. Finally, a number of tokens representing HTML tags appear to have been seen in training, although they initially appeared as unreachable when using the ‘fast’ version of the Hugging Face tokenizer[11]. 3.3.10 Jamba 3.3.10 Jamba Authors:
(1) Sander Land, Cohere s(ander@cohere.com);
(2) Max Bartolo, Cohere (max@cohere.com). Authors: Authors: (1) Sander Land, Cohere s(ander@cohere.com); (2) Max Bartolo, Cohere (max@cohere.com). This paper is available on arxiv under CC BY-SA 4.0 DEED license. This paper is available on arxiv under CC BY-SA 4.0 DEED license. available on arxiv available on arxiv [4] A fragment of the token _サーティワン (‘thirty-one’), the Japanese name for the ‘Baskin-Robbins’ ice cream chain. [5] Although GPT-J uses a bias in the unembedding layer, it does not affect our indicators due to its small magnitude. [6] This was in part traced to a breaking change in tokenizers v0.14 (Luca Soldaini, personal communication). [7] This bug was reported and subsequently fixed by the Gemma team, and our results are based on the fixed version. [8] We submitted a fix for this at https://github.com/spencermountain/wtf_wikipedia/pull/573. [9] This was reported to the Stability AI team and has been fixed by disabling the incorrect ‘slow’ tokenizer. [10] We use tokenizers v0.19.1, in which the tokenization of 588 formerly unreachable tokens was fixed. [11] This has been reported to the 01.ai team, who advise not to use the ‘fast’ version.

How Tokenizer Choices Shape Hidden Risks in Popular Language Models

As software engineers, we live in a dependency-driven world. Whether they are libraries, open-source code, or frameworks, these dependencies often provide the boilerplate functionality, integrations with other platforms, and tooling that make our jobs a little easier. But of course, one of the burdens of this dependency-driven world is that we are now at the mercy of our dependencies’ updates and schedules. Especially when we’ve added our own customizations and integrations on top. One common dependency that I (and likely you) have crossed paths with is Salesforce. If Salesforce is part of your ecosystem, you’ve likely added a few of those customizations and integrations. And you’ve likely faced the problem of managing your customizations and integrations when Salesforce issues updates. Salesforce In this article, I want to look at how you can help ensure that Salesforce updates don’t lead to downtime in your app. About Salesforce Seasonal Releases Salesforce issues updates on an established schedule. Currently, Salesforce targets spring (February), summer (June), and winter (October) releases. Along with these seasonal updates, of course, comes teams scrambling to update their own applications and tests in time for the releases. Which can introduce significant risk. Potential Areas of Risk Let’s look in detail at three potential areas of risk: User interface (UI) updates
Applications built on top of Salesforce
Deprecated APIs User interface (UI) updates Applications built on top of Salesforce Deprecated APIs User interface (UI) updates User interface (UI) updates At first glance, teams might downplay the impact of UI updates. However, these changes can become an issue when published instructions are no longer valid or (even worse) functional. In my experience, I have seen challenges from trivial UI changes such as reordering of a menu structure. Applications built on top of Salesforce Applications built on top of Salesforce If you’ve built your solution on top of Salesforce, you’re also at risk. You’re at the mercy of the platform and have no seat at the decision-making table for updates that could break the consuming functionality. An example is when the Apex programming code is used to extend the value of the Salesforce platform. As the language matures, there is always the potential for breaking changes to be introduced. Deprecated APIs Deprecated APIs When a team uses Salesforce APIs, they are required to adopt a specific version of the API which often defaults to the most recent version. Over time, that choice remains static and does not update with the seasonal releases, which is actually by design. However, eventually those versions become deprecated and are subsequently removed. Imagine being woken up due to an incident because the API version you are using is no longer available. Failure to perform pre-release validation can lead to latency concerns (if not high-severity incidents) which not only impacts organizations in the short term, but in the long term too. Each of the three issues above has the potential to put enterprises in an unfavorable position. Tips to Mitigate Areas of Risk Now let’s look at some tips on how to mitigate the above areas of risk: Automated testing, including end-to-end tests
Stay in line with Salesforce best practices
Full suite of test coverage
Use of isolated test data Automated testing, including end-to-end tests Stay in line with Salesforce best practices Full suite of test coverage Use of isolated test data Automated testing, including end-to-end tests Automated testing, including end-to-end tests Teams should make sure to spend the necessary time to automate the testing process. Taking this approach removes the toil associated with trying to remember to run the tests and makes testing part of the development lifecycle. This automation should include some form of regression or end-to-end tests which are executed at least one time before attempting a production release. While it is possible to introduce a process to copy production data into a test environment (obfuscating sensitive data), this isn’t typically the best approach. A better approach is to identify scenarios that should be validated and create those situations exclusively for the tests. Taking this approach allows teams to be 100% in control of their data so that only the pre-release code is getting tested. Stay in line with Salesforce best practices Stay in line with Salesforce best practices When an organization adopts Salesforce, this should be viewed as an “all-in” mission for the enterprise. After all, Salesforce is now one of that organization’s key business partners. Subsequently, the organization should not only be in tune with Salesforce’s roadmap, but they should also make sure feature teams are always implementing the best practices published by Salesforce. Salesforce Trailhead is an interactive example, complete with a built-in test interface. Salesforce Trailhead Full suite of test coverage Full suite of test coverage Teams should be of the mindset that test coverage is equally important as the production code that will be deployed. While Salesforce requires 75% code coverage (for Apex code) to deploy code to production instances, teams should instead strive for 100% coverage across the board. Any gaps increase the chance that something will impact your customers. It’s important to remember that the tests should be easy to maintain and focused on the smallest grain of functionality possible. The suite of tests should cover any customizations made to the platform, especially UI changes. Use of isolated test data Use of isolated test data Finally, while test data can often be mocked (or included in the tests themselves), this is not always an option for end-to-end tests. One challenge teams face is trying to figure out “what” data should be used. By adopting these tips, teams will find themselves in a far better position for a successful seasonal release to the Salesforce platform. AI-Powered Salesforce Testing So far, I’ve identified risks that can surface from a lack of pre-release validation. I’ve also provided tips to mitigate those risks. But what I haven’t provided is a description of “how” this mitigation can actually happen. In 2024, I published the “Why You Need to Shift Left with Mobile Testing” article which focused on the Tosca product by Tricentis. Since I wasn’t quite sure how to do all of this myself, I thought I would check to see what options are available from Tricentis for Salesforce test automation. article Tricentis That’s when I noticed the Testim Salesforce product, which provides the ability to mitigate the risks associated with leveraging the Salesforce platform: Offers a no-code approach to testing your Salesforce implementation, but also allows for Apex and JavaScript code to be used as needed.
Designed for a wide realm of consuming actors and designed for software engineers.
Architected using a scalable-first approach, growing to match your implementation's test expectations.
Includes pre-built test steps to foster creation of test suites. When needed, the no-code approach allows actors to record their tests by using their applications.
The Testim Salesforce product embraces AI as part of its base product. Under the hood are locators that go beyond basic Apex-based unit tests and use Salesforce metadata to provide improved stability for both Salesforce releases as well as dynamic Lightning pages.
Most importantly, this solution can facilitate testing not only across environments and branches, but can take into account different personas and stakeholders within the Salesforce implementation. Offers a no-code approach to testing your Salesforce implementation, but also allows for Apex and JavaScript code to be used as needed. Designed for a wide realm of consuming actors and designed for software engineers. Architected using a scalable-first approach, growing to match your implementation's test expectations. Includes pre-built test steps to foster creation of test suites. When needed, the no-code approach allows actors to record their tests by using their applications. The Testim Salesforce product embraces AI as part of its base product. Under the hood are locators that go beyond basic Apex-based unit tests and use Salesforce metadata to provide improved stability for both Salesforce releases as well as dynamic Lightning pages. Most importantly, this solution can facilitate testing not only across environments and branches, but can take into account different personas and stakeholders within the Salesforce implementation. Testim Salesforce in Action Let’s see this in action. First, we need to install Testim. Let’s set it up as a Chrome extension. Chrome extension You can install Testim directly from the Chrome Web Store. After the Chrome extension is installed, you can connect your Salesforce environment to Testim and the extension will orchestrate tests in a separate Chrome instance. Start a new test Start a new test Now we can use the included Salesforce demo project to see how it works. In the Testim dashboard, click New Test. You can use the no‑code wizard and pre‑built steps, but you can also record a series of actions in your browser that then automatically create test steps. Additionally, you can also drop in custom Apex or JavaScript code. New Test For an existing test, press the play button to execute it against your Salesforce instance. Testim has a set of pre-existing Salesforce tests that can be used out of the box or as a basis for customizing your own testing. A test is a series of test steps, such as opening the login screen, entering credentials, and verifying that your browser navigates to the correct screen after logging in.Record user actions for test creation Record user actions for test creation Testim has a feature that allows a test creator to record their actions in their browser and use that data to generate a new test. With recording enabled, navigate to theSales tab, click New Lead, fill in the First Name, Last Name, and Email fields, then click Create. Next, click Convert. You’ll see in the test editor that each of these steps is now recorded in a new test. Sales New Lead, First Name Last Name Email Create Convert An example of a recorded test is below. You can click on any test step and modify its requirements, including success criteria and UI timeouts. In the above example of editing a test step, you could change the target element that is selected, what is considered “passing” for this step, and when this step runs in the test sequence.Insert additional test steps manually Insert additional test steps manually Now click the + button at the end of the test sequences to manually add an additional test step. There are various built-in test steps that can be added without needing to re-record your steps in the browser. This is useful for adding additional checks into a larger test sequence without having to re-record it completely. + Review and maintain stability. Finally, after execution, use the results dashboard to identify passed and failed steps. Testim flags unstable locators and suggests replacements. Conclusion In the introduction of this article, I noted that we live in a dependency-driven world. For me, I don’t think that is a bad thing at all. We just need to make sure we understand the reason for the dependency and take time to validate if the relationship still makes sense. My readers may recall my personal mission statement, which I feel can apply to any IT professional: “Focus your time on delivering features/functionality that extends the value of your intellectual property. Leverage frameworks, products, and services for everything else.” — J. Vester “Focus your time on delivering features/functionality that extends the value of your intellectual property. Leverage frameworks, products, and services for everything else.” — J. Vester In this article, I’ve noted that organizations should treat Salesforce as a close business partner. I’ve also noted how pre-release validation testing is vital for avoiding service interruptions. We’ve discovered that implementation of Testim Salesforce provides an avenue teams can use to mitigate risks that are often exposed by the seasonal updates to the Salesforce platform. Testim Salesforce fully adheres to my personal mission statement. It utilizes a no-code solution which eases the pain of providing an acceptable level of test coverage, which is often faster to implement than options that require manual coding. Their use of AI provides deeper validation that creates stability and requires less maintenance over time. Most importantly, the solution can be an automated part of the software development lifecycle (SDLC). Have a really great day!

This Chrome Extension Can Help Mitigate Risks of Salesforce Updates to Your Existing Setup

Dartmouth’s AI Therapist has completed a Randomized Controlled Trial (RCT), but what does this mean for the future? Dartmouth’s AI Therapist has completed a Randomized Controlled Trial (RCT), but what does this mean for the future? Dartmouth’s AI Therapist has completed a Randomized Controlled Trial (RCT), but what does this mean for the future? Background Background Since the days of ELIZA, mankind has endeavored to understand itself through the soulless silicon eyes of a machine. It seems like now, in 2025, that reality is finally here. Two articles or studies published in the last month have challenged how I see computer-assisted therapy. However, with my background in data and technology, it has also fueled my fears. I focused those fretful thoughts into my science fiction writing, my cyberpunk book, Above Dark Waters, which is about how someone uses their AI Therapist to improve their brain, sending humanity snowballing towards the singularity. ELIZA articles studies Above Dark Waters The first piece is a study from Dartmouth College called Randomized Trial of a Generative AI Chatbot for Mental Health Treatment, which was published last month. They had a sample size of 200, and split them into control and testing groups of ~100 each. The results were very promising as the control group showed decreases across three types of disorders. Randomized Trial of a Generative AI Chatbot for Mental Health Treatment Randomized Trial of a Generative AI Chatbot for Mental Health Treatment Overall, results from the Therabot RCT are highly promising. We found high engagement and acceptability of the intervention, as well as symptom decreases while maintaining a therapeutic alliance comparable to that of human therapists and their patients. Overall, results from the Therabot RCT are highly promising. We found high engagement and acceptability of the intervention, as well as symptom decreases while maintaining a therapeutic alliance comparable to that of human therapists and their patients. The second is the article from the Harvard Business School titled, “How people were really using gen AI,” which itself was a follow-up of a 2024 article. I have trimmed down their infographic below, but as you can see, the top three use cases in 2025 are self-help related, with Therapy/Companionship taking top spot! How people were really using gen AI, How people were really using gen AI, No matter if you think this is a fad or if it does not remotely interest you, the AI Therapists are coming! I think there are many positives for this technology, but in this article, I want to cover only the pitfalls. there are many positives 5 Potential Problems with AI Therapy 5 Potential Problems with AI Therapy This is only my initial list. I’m sure there are many more, but I will briefly cover each of the following areas of concern: Addiction Risks
Data Privacy Risks
Corporate or Gov’t Control
Dehumanization
Algorithmic Bias Addiction Risks Data Privacy Risks Corporate or Gov’t Control Dehumanization Algorithmic Bias No.5: Addiction Risks No.5: Addiction Risks This one is the most obvious and (thankfully) easiest to control for. Many of these applications tout 24/7 access as the main selling point, but I think this is indeed a negative. Why wait two weeks or a month for therapy when you can have it in your hands right now! In fact, the Dartmouth study showed a nice heatmap of the usage for us. As you can see, there are several user-days with over 100 messages. Other users used it nearly every day of the study. Dartmouth study This is a success as far as getting people to use the product. But is this a success in the sense that the participant no longer needs therapy? I would also love to see this by the time of day. Are people waking up in the middle of the night, using the therapist, and not sleeping well? Because we all know one way to help with anxiety is to get sufficient rest. Fortunately, these kinds of controls are easy to implement technically. However, there seems to be little reason for a private company to disincentivize usage. The goal of therapy is to one day stop using therapy! see this by the time of day No. 4: Data Privacy Risks No. 4: Data Privacy Risks This one is also obvious. If you can’t trust who you’re talking to, you’re never going to say anything of value. And if you’re going to self-censor, then you may never go down a potentially fruitful line of questioning and land on a potentially fruitful and/or nourishing answer. The mere hint of data breaches might be enough to destroy any faith you have in the provider, and as such, you’ve lost the potential history you have with the artificial therapist. And this isn’t a hypothetical scenario, this is the reality. Data from BetterHelp ended up being sold to Meta, based on this FTC complaint. you’re never going to say anything of value FTC complaint To capitalize on these consumers’ health information, Respondent handed it over to numerous third-party advertising platforms, including Facebook, Pinterest, Snapchat, and Criteo, often permitting these companies to use the information for their own research and product development as well. To capitalize on these consumers’ health information, Respondent handed it over to numerous third-party advertising platforms, including Facebook, Pinterest, Snapchat, and Criteo, often permitting these companies to use the information for their own research and product development as well. Now, this isn’t exclusive to digital therapists. This can happen to real therapists, too. A therapist breaching trust in a small town would be devastating to the patient; however, the scope of the  breach would be small. What would a patient do? They would never go back to that therapist. A major data breach might have a chilling effect on the entire AI therapy ecosystem. the entire the entire No. 3: Corporate or Government Control No. 3: Corporate or Government Control Perhaps these AI therapists will be open-sourced or run by a non-profit; regardless of any future hopes, most of these seem to be run by tech startups. And, if there’s one thing a startup desperately needs, it’s cash! They have to make it to the next round of funding. What happens if the corporation sells even the metadata to other companies for ads? Will the fact that you have an account with a company be enough to push pills, teas, creams, retreats, or books? This brings me to a topic which I will also fit into point number 4: Mental Health Data Weaponization. There are cases of people being locked out of their money by banks for dubious reasons. Imagine if what you say to your therapist flags you on the no-fly list? Mental Health Data Weaponization Governments can and do subpoena companies for data. The US recently subpoenaed data for companies in other countries, like in this article, US Lawmakers Subpoena China Telecom Giants.  The UK has put people in jail for their social media posts. US Lawmakers Subpoena China Telecom Giants social media posts There are also attempts to exert more subtle control over what the algorithms are nudging people towards. This is called Data Poisoning and can occur without the host company even realizing it. Imagine that someone makes hundreds of accounts, and constantly push away from the ‘right’ answer by repeatedly responding with ‘well then I’m going to self-terminate’ or something. Will the AI therapist push people to a ‘new thing’ that it thinks is going away from the ‘bad thing’?  Will they suggest a final solution for struggling people like the MAID in Canada? Data Poisoning No. 2: Dehumanization No. 2: Dehumanization I’m so terrible, even a human therapist won’t take me. Does the very act of being pushed to a technical solution hurt some people? I’m so terrible, even a human therapist won’t take me. Perhaps people are simply looking for someone, anyone, to listen to us in a non-judgmental way. Perhaps we’re in this hole precisely because of technology. Are the AI avatars of Replika going to help humanity? Replika Any developers out there? What happens when you layer another technological solution over an existing technological problem? It often becomes worse. Chaotic. Clunky. I have two other problems with an AI therapist: It’s verbose, and it’s always trying to solve a problem. Sometimes the best thing for a therapist to do is wait a few long moments and let the person process and cry it out… Silence and feeling heard… And giving space between the next prompt… Those would be essential to making a true connection. However, LLMs are ravenous for your next prompt. ravenous Wait, wait, you might say, we could program those pauses in. You could, for sure, and I believe they will be successful. And then it would be even more super-human super-empathic than it already is. And then the human user will grow tired of other humans because no one is as good at talking and being emotive and empathic as their own supertherapist. Imagine for a moment we replace ‘therapy’ with ‘sexbot,’ and the algorithms become so good that no human can please you like it does. Would that be a good thing for society? No, society would cease in one generation. Wait, wait, we could program those pauses in The deeper issue may be ‘denaturalization,’ rather than depersonalization. We just don’t fit into the 24/7 content-churning infinite scroll world we’ve built. It’s alien. Going outside and feeling the grass is common advice today. But it is rooted in some truth, as even nature can be therapy; shinrin-yoku, or “forest bathing,” might yield better results for free. Even watching the birds in trees or bees in flowers can help with anxiety. Could the silent swish of snow over a lawn be enough to heal one’s mind better than a human confidant? nature can be therapy shinrin-yoku No. 1: Algorithmic Bias 🎰🧑‍⚖️⚖️🎰 No. 1: Algorithmic Bias 🎰🧑‍⚖️⚖️🎰 Algorithmic bias is defined as the systematic and repeatable harmful tendency in a computerized sociotechnical system to create "unfair" outcomes, such as "privileging" one category over another in ways different from the intended function of the algorithm. Ultimately, this technology shares all the same problems that algorithmic policing does. Much has been made of this in the past five years. One solution is transparency, but it is not clear whether the LLMs actually know how they arrive at those solutions. the systematic and repeatable harmful tendency in a computerized sociotechnical system to create "unfair" outcomes, such as "privileging" one category over another in ways different from the intended function of the algorithm. If you’re rural or poor, then it increases the likelihood that you will use this technology, as access to a normal therapist is probably limited. If you’re young and don’t have/want parental help, then this is likely going to lead you to online solutions. Sometimes it’s good to have something trained on all kinds of people, but the bulk of the data is the English Language Western Technological society. The problem is that many of these things are trained on the output of people like you. As such, it might be pushing people to the most general solution, which does work most of the time, but ends up being poorly fitting for everyone. The alternate problem comes when the data is trained completely on PLY: (People Like You). Then you get an algorithmic black hole and an echo chamber. are Conclusion Conclusion I think the promise of AI therapy is real, but its cyberpunk risks—addiction, surveillance, bias—loom large. Ultimately, I am interested in watching how the promise of AI therapy unfolds, but it helps to keep that dystopian critical eye razor sharp, because reality will always be stranger than fiction. Also, to note, I’m sure that the people making AI therapists are great people trying to do ‘the right thing.’ I hope no one in this field views this article as an attack. I simply want us all to think about the second and third-order effects of the things we’re all building. Will this be a tool for healing, or will it just be one more techie solution to a techie problem, standing upon the already shaky foundation of our technological society? the right thing. second third order Am I a huge wet blanket? Let me know, and if you’re looking for a sci-fi novel exploring the ethical considerations of AI Therapy and dystopian cyberpunk possibilities, then please check out my book, Above Dark Waters. Above Dark Waters

Are AI Therapists Safe? 5 Risks You Should Know

Authors:
(1) Anthi Papadopoulou, Language Technology Group, University of Oslo, Gaustadalleen 23B, 0373 Oslo, Norway and Corresponding author (anthip@ifi.uio.no);
(2) Pierre Lison, Norwegian Computing Center, Gaustadalleen 23A, 0373 Oslo, Norway;
(3) Mark Anderson, Norwegian Computing Center, Gaustadalleen 23A, 0373 Oslo, Norway;
(4) Lilja Øvrelid, Language Technology Group, University of Oslo, Gaustadalleen 23B, 0373 Oslo, Norway;
(5) Ildiko Pilan, Language Technology Group, University of Oslo, Gaustadalleen 23B, 0373 Oslo, Norway. Authors: Authors: (1) Anthi Papadopoulou, Language Technology Group, University of Oslo, Gaustadalleen 23B, 0373 Oslo, Norway and Corresponding author (anthip@ifi.uio.no); (2) Pierre Lison, Norwegian Computing Center, Gaustadalleen 23A, 0373 Oslo, Norway; (3) Mark Anderson, Norwegian Computing Center, Gaustadalleen 23A, 0373 Oslo, Norway; (4) Lilja Øvrelid, Language Technology Group, University of Oslo, Gaustadalleen 23B, 0373 Oslo, Norway; (5) Ildiko Pilan, Language Technology Group, University of Oslo, Gaustadalleen 23B, 0373 Oslo, Norway. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Background 2.1 Definitions 2.1 Definitions 2.2 NLP Approaches 2.2 NLP Approaches 2.3 Privacy-Preserving Data Publishing 2.3 Privacy-Preserving Data Publishing 2.4 Differential Privacy 2.4 Differential Privacy 3 Datasets and 3.1 Text Anonymization Benchmark (TAB) 3 Datasets and 3.1 Text Anonymization Benchmark (TAB) 3.2 Wikipedia Biographies 3.2 Wikipedia Biographies 4 Privacy-oriented Entity Recognizer 4 Privacy-oriented Entity Recognizer 4.1 Wikidata Properties 4.1 Wikidata Properties 4.2 Silver Corpus and Model Fine-tuning 4.2 Silver Corpus and Model Fine-tuning 4.3 Evaluation 4.3 Evaluation 4.4 Label Disagreement 4.4 Label Disagreement 4.5 MISC Semantic Type 4.5 MISC Semantic Type 5 Privacy Risk Indicators 5 Privacy Risk Indicators 5.1 LLM Probabilities 5.1 LLM Probabilities 5.2 Span Classification 5.2 Span Classification 5.3 Perturbations 5.3 Perturbations 5.4 Sequence Labelling and 5.5 Web Search 5.4 Sequence Labelling and 5.5 Web Search 6 Analysis of Privacy Risk Indicators and 6.1 Evaluation Metrics 6 Analysis of Privacy Risk Indicators and 6.1 Evaluation Metrics 6.2 Experimental Results and 6.3 Discussion 6.2 Experimental Results and 6.3 Discussion 6.4 Combination of Risk Indicators 6.4 Combination of Risk Indicators 7 Conclusions and Future Work 7 Conclusions and Future Work Declarations Declarations References References Appendices Appendices A. Human properties from Wikidata A. Human properties from Wikidata B. Training parameters of entity recognizer B. Training parameters of entity recognizer C. Label Agreement C. Label Agreement D. LLM probabilities: base models D. LLM probabilities: base models E. Training size and performance E. Training size and performance F. Perturbation thresholds F. Perturbation thresholds 5 Privacy Risk Indicators Many text sanitization approaches simply operate by masking all detected PII spans. This may, however, lead to overmasking, as the actual risk of re-identification may vary greatly from one span to another. In many documents, a substantial fraction of the detected text spans may be kept in clear text without notably increasing the reidentification risk. For instance, in the TAB corpus, only 4.4 % of the entities were marked by the annotators as direct identifiers, and 64.4 % as quasi-identifiers, thus leaving 31.2 % of the entities in clear text. To assess which text spans should be masked, we need to design privacy risk indicators able to determine which text spans (or combination of text spans) actually constitute a re-identification risk. We present 5 possible approaches for inferring the re-identification risk associated with text spans in a document. Those 5 indicators are respectively based on: LLM probabilities,


Span classification,


Perturbations,


Sequence labelling,


Web search. LLM probabilities, LLM probabilities, Span classification, Span classification, Perturbations, Perturbations, Sequence labelling, Sequence labelling, Web search. Web search. The web search approach can be applied in a zero-short manner without any finetuning. The two methods respectively based on LLM probabilities and perturbations require a small number of labeled examples to adjust a classification threshold or fit a simple binary classification model. Finally, the span classification and sequence labelling approaches operate by fine-tuning an existing language model, and are thus the methods that are most dependent on getting a sufficient amount of labeled training data to reach peak performance. This training data will typically take the form of human decisions to mask or keep in clear text a given text span. We present each method in turn and provide an evaluation and discussion of their relative benefits and limitations in Section 6. 5.1 LLM Probabilities The probability of a span as predicted by a language model is inversely correlated with its informativeness: a text span that is harder to predict is more informative/surprising than one that the language model can easily infer from the context (Zarcone et al., 2016). The underlying intuition is that text spans that are highly informative/surprising are also associated with a high re-identification risk, as they often correspond to specific names, dates or codes that cannot be predicted from the context. Concretely, we calculate the probability of each detected PII span given the entire context of the text by masking all the (sub)words of the span, and returning a list with all log-probabilities (one per token) referring to the span as calculated by a large, bidirectional language model, in our case BERT (large, cased) (Devlin et al., 2019). Those probabilities are then aggregated and employed as features for a binary classifier that outputs the probability of the text span being masked by a human annotator. The list of log-probabilities for each span has a different length depending on the number of tokens comprised in the text span. We therefore aggregate this list by reducing it to 5 features, namely the minimum, maximum, median and mean log probability as well as the sum of log-probabilities in the list. In addition, we also include as feature an encoding of the PII type assigned to the span by the human annotators (see Table 1). minimum maximum median mean sum For the classifier itself, we conduct experiments using a simple logistic regression model as well as a more advanced classification framework based on AutoML. AutoML (He et al., 2021) provides a principled approach to hyper-parameter tuning and model selection, efficiently searching for the hyper-parameters and model combination that yield the best performance. We use the Autogluon toolkit (Erickson et al., 2020) in our experiments, and more specifically AutoGluon’s Tabular predictor (Erickson et al., 2020), which consists of an ordered sequence of base models. After each model is trained and hyper-parameter optimization is performed on each, a weighted ensemble model is trained using a stacking technique. A list of all base models of the Tabular predictor is shown in Table 11, in Appendix D. We use the training splits of the Wikipedia biographies and the TAB corpus to fit the classifier. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

5 Ways to Measure Privacy Risks in Text Data

While developers, DevOps, and DevSecOps teams work tirelessly to ship secure code, the steps that enable collaboration can lead to security risks. In the “old days,” programmers wrote code locally on a desktop or laptop. To gain access to source code, malicious actors needed to be physically present to compromise a device. Today, all they need is access to the internet and a valid credential. Even riskier, developers often share code snippets in public repositories which can expose secrets in plaintext, hardcoded into the source code. With nearly every organization building its own applications, DevSecOps and security teams face unique challenges trying to mitigate risks, especially when trying to mitigate risks outside their control. Dynamic application security testing (DAST) and static application security testing (SAST) tools identify vulnerabilities in source code, but they may not always identify all potentially compromised secrets that sit in the team’s tools. Increasingly, security teams need insights into the risks outside the traditional network perimeter to protect their applications. Secrets: What they are and why they get shared In IT, secrets are typically sensitive data that help provide access to or secure an application or an environment. Some examples of secrets may include: Authentication tokens
API keys
Usernames
Passwords
Encryption keys
Security certificates Authentication tokens API keys Usernames Passwords Encryption keys Security certificates When developers are pushing to ship code faster, data leaks are commonly caused by hardcoded secrets in the source code. Some reasons that programmers may leave these secrets in the source code include: Simplify workflows: Hardcoding secrets in code can allow for faster code sharing and testing since they don’t need to manually input credentials each time.
Training and awareness: Programmers may assume that the internal repository is secure enough to protect the source code containing secrets.
Complex CI/CD pipelines: Hardcoding secrets streamlines the multiple stages of changing keys and secure secrets storage during the phases of development, testing, and deployment, which push code live on tight deadlines. Simplify workflows: Hardcoding secrets in code can allow for faster code sharing and testing since they don’t need to manually input credentials each time. Simplify workflows Training and awareness: Programmers may assume that the internal repository is secure enough to protect the source code containing secrets. Training and awareness Complex CI/CD pipelines: Hardcoding secrets streamlines the multiple stages of changing keys and secure secrets storage during the phases of development, testing, and deployment, which push code live on tight deadlines. Complex CI/CD pipelines When developers hardcode secrets, they create security risks. Secrets stored in plaintext can be vulnerable to unauthorized access. As threat actors scan public GitHub repositories, they can find and exploit these accidentally leaked or hardcoded credentials. Once they have this information, they can compromise the service that the secret is authenticating to. Finally, developers’ failure to remove the hardcoded secrets means this information can spread during the typical development processes of cloning, forking, or checking out source code. Finding a hardcoded needle in a GitHub haystack Security teams are already overwhelmed trying to keep the organization’s IT resources secure. While DAST and SAST tools should help identify hardcoded secrets, many companies use a combination of automated and manual processes which can lead to undetectable human error when trying to review thousands of lines of code. Additionally, these tools only identify hardcoded secrets in locations that the organization knows or can control. According to a 2025 report, 39 million secrets were leaked across GitHub in 2024. In this report, GitHub explains that secret leaks remain one of the most common—and preventable—causes of security incidents. In a similar 2024 report, 3,066,304 GitHub repositories, accounting for only 4.6% of active repositories, leaked a secret in 2023. The same report noted that more than 1 in 10 commit authors leaked a secret, essentially 1,749,398 distinct contributing authors. The sheer volume and speed at which programmers develop code makes identifying all potentially leaked secrets a challenge. 2025 report 2025 report 2024 report 2024 report Often, AppSec and security teams need to review multiple locations for hardcoded secrets, including: Source code: Large projects can have hundreds — or even thousands — of lines of code that need scanning.
Configuration files: File paths uploaded to repositories can expose secrets when they trace back to resources.
Environment variables: Values affecting how processes run and behave can have hardcoded secrets that pass to the function code at runtime, exposing the secrets.
Container images: Accidentally copying sensitive files with hardcoded secrets can expose them during the image build.
Developers’ personal repositories: Team members may accidentally push code to their personal repositories and default settings are public. Source code: Large projects can have hundreds — or even thousands — of lines of code that need scanning. Source code Configuration files: File paths uploaded to repositories can expose secrets when they trace back to resources. Configuration files Environment variables: Values affecting how processes run and behave can have hardcoded secrets that pass to the function code at runtime, exposing the secrets. Environment variables Container images: Accidentally copying sensitive files with hardcoded secrets can expose them during the image build. Container images Developers’ personal repositories: Team members may accidentally push code to their personal repositories and default settings are public. Developers’ personal repositories Taking off the blinders with data leak monitoring Security teams work ceaselessly to monitor the organization’s enterprise IT environment yet still face challenges trying to protect development and testing environments. Monitoring repositories and identifying leaked information is critical but time-consuming. Security teams should incorporate auditing their code repositories in their data leak monitoring. While typical data loss prevention (DLP) tools are a good addition to DAST/SAST, these traditional enterprise environment tools focus more on monitoring sensitive files so they often fail to account for external risks like public code. To gain full insight into leaked hardcoded credentials, organizations need solutions to augment their DLP solutions. Data leaks can easily turn into data breaches, and organizations' security teams need to mitigate these risks. Source code leaks and hardcoded secrets are often overlooked because teams face time and staffing limitations. However, automated threat intelligence solutions that scan the clear, dark, and deep web as well as illicit channels on the messaging app Telegram where malicious actors sell and share stolen data can provide the additional coverage necessary. These solutions can help security teams identify data leaks and remediate risks by monitoring outside the network to detect potential exposures, like: Compromised credentials that attackers could use to gain access to the repository itself.
Infected devices with stealer malware collecting sensitive information, including secrets and credentials.
Leaked source code containing hardcoded secrets.
Accidental commits from full-time employees or contractors that contain hardcoded secrets. Compromised credentials that attackers could use to gain access to the repository itself. Infected devices with stealer malware collecting sensitive information, including secrets and credentials. Leaked source code containing hardcoded secrets. Accidental commits from full-time employees or contractors that contain hardcoded secrets. As security teams work to mitigate risk across the ever-expanding attack surface, they need to monitor more than just internally controlled resources, like the enterprise IT environment. They need to expand their reach to understand the external threat exposures that exist outside the traditional network perimeter, like GitHub. By augmenting traditional DLP, DAST, and SAST tools with clear, deep, and dark web monitoring, as well as monitoring illicit Telegram channels, security teams gain full visibility into risk across internal and external locations. Beyond the perimeter: Strengthening security against hardcoded secrets Hardcoded secrets in public code repositories remain a persistent security risk, exposing organizations to potential breaches and unauthorized access. While traditional security tools like DAST, SAST, and DLP provide a strong foundation, they often fail to capture the full scope of external threats. To stay ahead, security teams must expand their monitoring beyond internal environments, leveraging automated threat intelligence solutions that scan public GitHub repositories, the dark web, and illicit Telegram channels. By taking a proactive, data-driven approach, organizations can mitigate the risks of leaked secrets and safeguard their applications from evolving cyber threats.

From Repos to Risks: How Hardcoded Secrets in GitHub Source Code Create Security Risks

The ability to control electronics without touching them or paying with a swipe of the hand may seem like sci-fi, but it’s a growing reality. Microchip implants have existed in some form for decades, and technology is now advanced enough to make them practical. Still, this field carries significant security concerns. The ability to control electronics without touching them or paying with a swipe of the hand may seem like sci-fi, but it’s a growing reality. Microchip implants have existed in some form for decades, and technology is now advanced enough to make them practical. Still, this field carries significant security concerns. The Growing Field of Implanted Microchips The Growing Field of Implanted Microchips Veterinarians have implanted radiofrequency identification chips in pets since the 1990s. Human microchips got FDA approval in 2004 for medical use, though the technology did not catch on at the time. got FDA approval in 2004 got FDA approval in 2004 Fast-forward to today, and several leading tech companies and startups are researching microchip implants. Most famously, Elon Musk’s Neuralink has begun trials for brain-computer interfaces, which let users control electronic devices with their thoughts. Simpler chips that use RFID or near-field communication to store information or manage electronic payments are already available for commercial use. These chips can stand in for debit cards, give doctors access to vital medical information or work as keycards to access digital locks. Though they may not necessarily be widespread yet, they’re becoming increasingly prevalent. Cybersecurity Risks of Microchip Implants While chip implants can be highly convenient, they also open the door to unique cybersecurity risks. People should consider these potential dangers before proceeding with the technology. RFID Skimming Contactless payments are one of the most practical ways to use an implanted microchip. However, this application means users are theoretically always vulnerable to skimming. Cybercriminals could intercept financial information if they get close enough to the implant, and because it’s essentially part of the user’s body, physical protections are minimal. Thankfully, while skimming has garnered much attention from RFID-blocking wallets, it’s not a common threat. NFC is even more secure, as it can only happen within four centimeters of the chip, making it difficult for criminals to get close enough. Still, the possibility may worry some consumers. only happen within four centimeters only happen within four centimeters Malware A more concerning threat is the risk of cybercriminals installing malware onto an implanted chip. Researchers have proved that it’s possible to transfer malware through NFC, even when the chip does not connect to an actively running device. Attackers may still need to get close to do so, but the damage might go far beyond skimming card details. it’s possible to transfer malware through NFC it’s possible to transfer malware through NFC Malware could cause a chip to malfunction. Depending on what people use their microchips for, that could leave them locked out of rooms or unable to access some electronics. The malware could also spread to anything that connects to the implant, including smart home devices or users’ phones. Data Privacy Concerns Finally, storing sensitive data in a wirelessly accessible chip in people’s bodies raises privacy questions, especially when it comes to BCIs. A BCI could theoretically give companies access to brain data, even including users’ thoughts if technology advances enough. give companies access to brain data give companies access to brain data A breach at a business storing its customers’ brainwaves could give criminals access to a massive amount of highly sensitive information. Even outside such an event, targeted advertising based on brain data may feel like a privacy violation. The possibility of government bodies accessing this information further complicates the ethical implications. How Can Microchip Implants Become More Secure? Given these risks, implanted chips should implement a few extra security measures to protect their users. More best practices will emerge as the technology develops, but there are still a few must-have options to include. Zero-Trust Security The companies offering microchip implants must adhere to zero-trust policies. Government agencies began requiring zero trust in 2021, and private businesses would be wise to adopt the same standards. began requiring zero trust in 2021 began requiring zero trust in 2021 Zero-trust architecture treats every transaction as a possible breach, verifying all data transmissions before allowing them. It also restricts access permissions as much as possible. That level of attention and responsiveness is crucial for anything involving as much sensitive information and potential complications as a chip in users’ bodies. Tighter Access Controls Implementing stronger defenses for accessing the microchip and its connected services is also critical. Even a PIN is better than nothing. Hackers can brute-force through a four-digit code instantly, but they would still have to get within centimeters of an implanted NFC chip to do so. brute-force through a four-digit code instantly brute-force through a four-digit code instantly Setting and changing a PIN and managing the information and permissions of an implanted chip should require more robust protections. Strong passwords and multi-factor authentication are essential. Services should also restrict access attempts to stop unauthorized users from changing what it takes to get into an implant-based system. Functionality Limits Finally, microchip implants will be more secure when they do less. While some organizations may not want to sacrifice functionality, limiting what a chip can do minimizes the ways an attacker can take advantage of it. Restricting long-range communications is vital for protection. The most robust defense implanted chips have today is that they only work within a few centimeters. A microchip with remote accessibility or Wi-Fi functionality suddenly becomes vulnerable to a much wider range of attacks and potential entry points. Maintaining a relative air gap by relying on NFC or RFID and restricting the kinds of data a chip can access and transmit make them far safer. Microchip Implants Are Exciting But Potentially Risky Despite being around since the ‘90s, implanted microchip technology is still in its infancy. As this field grows and develops, its possibilities will expand, but so might its potential dangers. Cybersecurity considerations must come before expanded functionality and adoption. When these systems follow a security-by-design approach, people can use them confidently and safely. Failing to consider such risks in design and implementation could lead to catastrophic consequences.

Think Before You Link: Security Risks of Microchip Implants

**Austin, TX, United States, March 19th, 2025/CyberNewsWire/--**SpyCloud, the leading identity threat protection company, today released its 2025 SpyCloud Annual Identity Exposure Report, highlighting the rise of darknet-exposed identity data as the primary cyber risk facing enterprises today. 2025 SpyCloud Annual Identity Exposure Report 2025 SpyCloud Annual Identity Exposure Report As cybercriminals move beyond single data points and leverage stolen data from a number of sources – breaches, malware and phishes – they are embracing a more sophisticated approach to identity exploitation, and organizations must shift their focus to a comprehensive and holistic defense strategy that accounts for the interconnected nature of digital identities. Holistic Identity: The New Cyber Battleground Organizations have traditionally focused on securing individual account credentials, but SpyCloud’s research indicates that cybercriminals have expanded their tactics beyond conventional account takeover. Attackers now have access to extensive identity data from multiple sources—including data breaches, infostealer malware infections, phishing campaigns, and combolists—posing a challenge for organizations whose security measures have not yet adapted to address the full scope of interconnected identity exposures holistically. SpyCloud’s collection of recaptured darknet data grew 22% in the past year, now encompassing more than 53.3 billion distinct identity records and over 750+ billion total stolen assets that are now circulating in the criminal underground, fueling identity-based cybercrime. These assets are a vast array of personal and professional credentials, session cookies, personally identifiable information (PII), financial data, IP addresses, national IDs and more that criminals are weaponizing in attacks against individuals and businesses. "The cybersecurity industry has spent years defending against traditional credential-based threats, but the reality is that attackers have advanced as the data they have access to has exploded in volume,” said Damon Fleury, Chief Product Officer, SpyCloud. “Identity is the ultimate frontier of cyber risk, with users’ exposure across past and present, personal and professional identities the new attack surface. It requires organizations to rethink the risks posed by employees, consumers, partners and suppliers.” "The cybersecurity industry has spent years defending against traditional credential-based threats, but the reality is that attackers have advanced as the data they have access to has exploded in volume,” said Damon Fleury, Chief Product Officer, SpyCloud. “Identity is the ultimate frontier of cyber risk, with users’ exposure across past and present, personal and professional identities the new attack surface. It requires organizations to rethink the risks posed by employees, consumers, partners and suppliers.” Fleury continues, “At SpyCloud, we’ve created holistic identity analytics built on the industry’s largest collection of recaptured darknet data, enabling our customers to correlate disparate data points that encompass an individual’s digital footprint—providing a truly holistic view of identity risk.” Fleury continues, “At SpyCloud, we’ve created holistic identity analytics built on the industry’s largest collection of recaptured darknet data, enabling our customers to correlate disparate data points that encompass an individual’s digital footprint—providing a truly holistic view of identity risk.” New Definition for Identity Risk Emerges With the explosion of available identity data, attackers can now piece together historical and present-day records to bypass security barriers. Traditionally, cybersecurity teams were only able to see a fraction of an individual’s darknet exposures – primarily only the exposed assets tied to a corporate identity – which were not comprehensive nor in correlation with other exposures. SpyCloud’s report shows that an individual’s identity exposure is more expansive than traditional cyber risk tools would indicate; in fact, it’s a sprawling web of interrelated assets that provide cybercriminals with a roadmap to exploit vulnerabilities and the keys to unlock valuable access. Of particular concern for businesses, a single corporate user now has an average of 146 stolen records linked to their identity – across 13 unique emails and 141 credential pairs (a username or email and its associated password) per corporate user, which highlights how attackers correlate historical data to uncover active enterprise access points.
In the consumer realm, the numbers are even higher with 229 records per consumer, frequently including exposed PII such as full names, dates of birth, and phone numbers, as well as Social Security/ID numbers, addresses, and credit card or bank information. Consumer exposure averages 27 unique emails and 227 credential pairs per user. Of particular concern for businesses, a single corporate user now has an average of 146 stolen records linked to their identity – across 13 unique emails and 141 credential pairs (a username or email and its associated password) per corporate user, which highlights how attackers correlate historical data to uncover active enterprise access points. In the consumer realm, the numbers are even higher with 229 records per consumer, frequently including exposed PII such as full names, dates of birth, and phone numbers, as well as Social Security/ID numbers, addresses, and credit card or bank information. Consumer exposure averages 27 unique emails and 227 credential pairs per user. "The record-breaking breaches of 2024, including the Mother of All Breaches (MOAB) and the National Public Data Breach, along with the growing use of infostealing malware and crafty phishing campaigns illustrate just how vast the pool of exposed identity data has become,” said Trevor Hilligoss, Senior Vice President of Security Research, SpyCloud Labs at SpyCloud. "The record-breaking breaches of 2024, including the Mother of All Breaches (MOAB) and the National Public Data Breach, along with the growing use of infostealing malware and crafty phishing campaigns illustrate just how vast the pool of exposed identity data has become,” said Trevor Hilligoss, Senior Vice President of Security Research, SpyCloud Labs at SpyCloud. “By understanding how cybercriminals aggregate stolen data and the new tactics and trends they are leveraging to assume even more valuable information and access, organizations can take proactive steps to mitigate identity-based threats from these large underground sources before they escalate." “By understanding how cybercriminals aggregate stolen data and the new tactics and trends they are leveraging to assume even more valuable information and access, organizations can take proactive steps to mitigate identity-based threats from these large underground sources before they escalate." Additional Report Findings: 17.3 billion cookies were recaptured from malware-infected devices, enabling attackers to bypass MFA and hijack active user sessions.
548 million credentials were exfiltrated via infostealer malware, highlighting the growing role of stealthy, targeted data theft in enterprise attacks.
3.1 billion passwords were recaptured in 2024, marking a 125% increase from the previous year.
70% of users whose credentials were exposed in breaches last year reused previously compromised passwords, significantly increasing their risk of account takeover attacks – a 9+ jump from 2023.
44.8 billion PII assets – a 39% increase from 2023 are opening the door for new fraudulent activities.
97% of recaptured phished data logs in 2024, from popular phishing-as-a-service (PHaaS) platforms like ONNX, included an email address and 64% had an associated IP address, giving criminals direct opportunities to perpetrate as the user and make lateral movements within an organization.
In the public sector, SpyCloud recaptured 127K .gov credentials and observed a 67% all-time password reuse rate – an increase of 13% over the previous year – highlighting persistent security risks for our federal agencies and national security. 17.3 billion cookies were recaptured from malware-infected devices, enabling attackers to bypass MFA and hijack active user sessions. 548 million credentials were exfiltrated via infostealer malware, highlighting the growing role of stealthy, targeted data theft in enterprise attacks. 3.1 billion passwords were recaptured in 2024, marking a 125% increase from the previous year. 70% of users whose credentials were exposed in breaches last year reused previously compromised passwords, significantly increasing their risk of account takeover attacks – a 9+ jump from 2023. 44.8 billion PII assets – a 39% increase from 2023 are opening the door for new fraudulent activities. 97% of recaptured phished data logs in 2024, from popular phishing-as-a-service (PHaaS) platforms like ONNX, included an email address and 64% had an associated IP address, giving criminals direct opportunities to perpetrate as the user and make lateral movements within an organization. In the public sector, SpyCloud recaptured 127K .gov credentials and observed a 67% all-time password reuse rate – an increase of 13% over the previous year – highlighting persistent security risks for our federal agencies and national security. Evolving Cybersecurity Strategies The findings highlight that cybercriminals are moving well-beyond their own legacy tactics and businesses must recognize that traditional defenses are no longer enough. SpyCloud’s approach leverages holistic identity analytics, powered by the industry’s largest collection of recaptured darknet data, to help organizations correlate disparate identity elements and shore up identity threat protection measures, while mitigating risk more effectively. For further insights, the full 2025 SpyCloud Identity Exposure Report is available here. here here About SpyCloud SpyCloud transforms recaptured darknet data to disrupt cybercrime. Its automated holistic identity threat protection solutions leverage advanced analytics to proactively prevent ransomware and account takeover, safeguard employee and consumer accounts, and accelerate cybercrime investigations. SpyCloud’s data from breaches, malware-infected devices, and successful phishes also powers many popular dark web monitoring and identity theft protection offerings. SpyCloud SpyCloud Customers include seven of the Fortune 10, along with hundreds of global enterprises, mid-sized companies, and government agencies worldwide. Headquartered in Austin, TX, SpyCloud is home to more than 200 cybersecurity experts whose mission is to protect businesses and consumers from the stolen identity data criminals are using to target them now. To learn more and see insights, users can visit spycloud.com. spycloud.com spycloud.com Contact Emily Brown REQ on behalf of SpyCloud spycloud@req.co This story was distributed as a release by Cybernewswire under HackerNoon’s Business Blogging Program. Learn more about the program here This story was distributed as a release by Cybernewswire under HackerNoon’s Business Blogging Program. Learn more about the program here here here

SpyCloud’s 2025 Identity Exposure Report: The Scale And Hidden Risks Of Digital Identity Threats

تقرير SpyCloud حول كشف الهوية لعام 2025: حجم ومخاطر التهديدات الرقمية الخفية

Справаздача аб выкрыцці ідэнтыфікацыйных дадзеных SpyCloud за 2025 год: маштабы і схаваныя рызыкі пагроз лічбавай ідэнтыфікацыйнай інфармацыі

Informe de exposición de identidad de SpyCloud 2025: La magnitud y los riesgos ocultos de las amenazas a la identidad digital

SpyCloud-ի 2025 թվականի ինքնության բացահայտման հաշվետվություն. թվային ինքնության սպառնալիքների մասշտաբները և թաքնված ռիսկերը

Laporan Paparan Identitas SpyCloud 2025: Skala dan Risiko Tersembunyi dari Ancaman Identitas Digital

SpyCloud の 2025 年個人情報漏洩レポート: デジタル ID 脅威の規模と隠れたリスク

របាយការណ៍បង្ហាញអត្តសញ្ញាណឆ្នាំ 2025 របស់ SpyCloud៖ មាត្រដ្ឋាន និងហានិភ័យលាក់បាំងនៃការគំរាមកំហែងអត្តសញ្ញាណឌីជីថល

Laporan Pendedahan Identiti 2025 SpyCloud: Skala Dan Risiko Tersembunyi Ancaman Identiti Digital

Ripoti ya SpyCloud ya 2025 ya Mfichuo wa Utambulisho: Kiwango na Hatari Zilizofichwa za Vitisho vya Utambulisho wa Dijiti

รายงานการเปิดเผยข้อมูลส่วนตัวปี 2025 ของ SpyCloud: ขนาดและความเสี่ยงที่ซ่อนเร้นของภัยคุกคามต่อข้อมูลส่วนตัวดิจิทัล

SpyCloud-ning 2025-yilda shaxsiy ma'lumotlarga duchor bo'lish hisoboti: raqamli identifikatsiya tahdidlarining ko'lami va yashirin xavflari

SpyCloud 的 2025 年身份暴露报告：数字身份威胁的规模和隐藏风险

More Yield, More Problems: The Restaking Risks You Can't Afford to Ignore

هل يستحق العائد الإضافي بنسبة 0.05% حقًا المخاطرة بوضع عملتك المشفرة في محفظتك؟

০.০৫% অতিরিক্ত ফলন কি সত্যিই আপনার ক্রিপ্টো বাজি ধরার ঝুঁকির যোগ্য?

¿Realmente vale la pena correr el riesgo de apostar sus criptomonedas por ese rendimiento adicional del 0,05 %?

क्या 0.05% अतिरिक्त लाभ वास्तव में आपके क्रिप्टो को दांव पर लगाने के जोखिम के लायक है?

Արդյո՞ք այդ 0,05% հավելյալ եկամտաբերությունը իսկապես արժե ձեր կրիպտո վտանգը դնելու համար:

0.05% の追加利回りは本当に暗号通貨をステーキングするリスクに見合う価値があるのでしょうか?

Adakah Hasil Tambahan 0.05% Itu Benar-Benar Berbaloi dengan Risiko Mengambil Kripto Anda?

के त्यो ०.०५% अतिरिक्त उपज साँच्चै तपाईंको क्रिप्टो दांव लगाउने जोखिमको लायक छ?

Na Poelo Yeo ya 0.05% e Eketsegilego e Ruri e Swanelega Kotsi ya go Bea Crypto ya Gago Bekeng?

Naozaj stojí tých 0,05 % výnosu navyše za riziko stávkovania vašich kryptomien?

Iyo 0.05% Yekuwedzera Goho Inonyatso Kukodzera Njodzi Yekudzvanya Crypto Yako?

Je, Mavuno ya Ziada ya 0.05% yanastahili Hatari ya Kuweka Crypto yako?

Stories, News & Mentions about RIOKS

Categories

Trending Topics

Stories, News & Mentions about RIOKS

Rioks’s Stories on HackerNoon

Categories

Trending Topics