Graphika

graphika

The last few years exposed more than ever the fragility of our information ecosystem. Elections, Brexit, social unrest, COVID-19 conspiracies, and the war in Ukraine have been used as disinformation operations by certain actors to undermine faith in governments, raise fear and anger, and confuse and manipulate us. Disinformation is information meant to deceive (trolls posting fake news). In contrast, misinformation is false information with no malicious intent (our friends and family genuinely believe in disinformative content and willingly share it). Why Is Disinformation Effective? Most disinformation efforts work through a so-called ‘censorship through noise’. Often, disinformation actors push for a story and repeat it across different news outlets that are massively shared on social media by fake accounts of troll farms. The exponential and relatively cheap internet access allowed anyone to post information (be it an article, video, comment, or tweet). Consequently, social media users are overwhelmed by the various channels that parrot the same narrative. Some users might believe these stories as they see multiple sources of information as more credible than one. These stories don’t necessarily need to be accurate and check their facts. Propagandistic efforts are to be among the first to report a story (more chances to warp our beliefs as these accounts prime our attention through ), drown competing reports, or sow doubt and skepticism. mere exposure effect Often, propaganda information is reiterated by more credible and legitimate news sources, reinforcing falsehoods. since it is the best means of competing with the ‘body of fact’ that exists in the minds of the general public. It is also the means of establishing a controversy. Doubt is our product A tobacco industry executive in 1969 about how cigarette companies should deal with the growing ‘body of fact’ linking smoking and diseases. This quote about planting doubt is even more relevant in modern propaganda, where we see our acquittances, friends, or family slip away from us as they fall prey to disorienting news. , former Director of Investigations for network analysis firm , presents his 4D model of predicting disinformation. Ben Nimmo Graphika Dismiss: if you don’t like what your critics say, insult them. Distort: if you don’t like the facts, twist them. Distract: if you’re accused of something, accuse someone else of the same thing (whataboutism). Dismay: if you don’t like what someone else is planning, try to scare them off (as we can see currently with nuclear threats). This video is part of the course from . Disinformation, Misinformation, and Fake News Teach-Out the University of Michigan Interestingly, Nimo is now Global Threat Intel Lead at Meta, the company behind Facebook, Instagram, and WhatsApp. (or the ) gives detailed information about current influence ops targeting Europe or the United States. Nimo’s Twitter feed non-Twitter version How to Counteract Disinformation Nimo also has a few rules for defending against online disinformation campaigns: : Think about the emotional targeting that is going on before clicking on a story because disinformation actors do not care about the truth. Rule #1 “the goal of a lot of these [disinformation] operations is to make people so angry or so afraid that they stop thinking. And once somebody stops thinking, they’re really easy to manipulate.” : Question the motive behind manipulative headlines: “where is this story spreading?” and “who is picking it up?” These questions can give us an idea of the impact of a specific campaign. Rule #2 : Keep calm and carry on. (so that) Rule #3 : Don’t let exposure lead to chaos. Rule #4 Ben Nimmo on defending against online disinformation campaigns We are creatures of habits, and our information diet is nothing different either: When reading news, do we read a story proposed by an algorithm or open our favorite news agency? The outlet news that we usually seek, do they provide fact-checking or retract articles when proven wrong? Did the headline that grabbed our attention trigger a highly negatively charged reaction? Did that emotion make us feel read the article? Did the body of the article have the same tone as the headline? Did the article feel balanced? Or was it confirming our previous (biased) knowledge? Do we dismiss or want to hear more when we hear contradictory information? Were all voices represented in the article? Who was missing? Why? Did we research on our own to check the statistics or other evidence in the article? After reading a news story, do we go and check other sources? How compelled do we feel to share this article with others, especially those we know who share our values? One of the most reputable forums that enforce healthy rules of news consumption is the subreddit , where users discuss national security issues. Some of are: r/credibledefense their guidelines 1.1. Strive to be informative, professional, gracious, and encouraging in your communications with other members here. Imagine writing to a superior in the Armed Forces, or a colleague in a think tank or major investigative journal. 1.2. This is a subreddit dedicated to collating articles, opinion pieces by distinguished authors, historical research, and the research of warfare relating to national security issues. 1.3. The purpose of this subreddit is to learn for ourselves, and to bring better public understanding of related topics. On their , a set of rules are posted, which, among others, state: daily megathreads Please do: * Be curious not judgmental, * Be polite and civil, * Link to the article or source of information that you are referring to, * Make it clear what is your opinion and from what the source actually says, * Read the articles before you comment, and comment on the content of the articles, * Leave a submission statement that justifies the legitimacy or importance of what you are submitting, * Submit articles that will be relevant 5-10 years from now, and not ephemeral news stories Among the do not rules, they recommend not using memes, emojis, excessive swearing, foul imagery, starting fights with other commenters, making it personal, or outing other commenters. How many of us adhere to these rules before posting something on social media, be it an article or responding to somebody who doesn’t share our point of view? How many times do we stop and try to be curious and polite and not judgmental? Yes, we shouldn’t feed trolls, but not all people that don’t share our point of view are trolls. Two other subreddits are related in name to r/credibledefence. One is , which has the greeting message: “Welcome to LessCredibleDefence – the home of links which have failed to pass the quality requirement of r/CredibleDefense”. r/LessCredibleDefence The other one is , home of defense-themed memes, which, unsurprisingly, has a golden rule for their users: “Don’t get us banned.” r/NonCredibleDefense Then, there are a few citizen activism movements that are successfully disrupting disinformation. Extremely relevant is about the Elves (cyber activists) in the Baltic States and Central Europe. this article is a decentralized online that propose shares plenty of NAFO memes. NAFO (North Atlantic Fella Organization) army of dog avatars humor as an effective instrument against propaganda. r/NonCredibleDefense We all need to be aware of our news diet and critical thinking (being able to objectively follow news and change our minds if we come across relevant information) limitations. Thinking of , Morgan Housel‘s quote Tell people what they want to hear, and you can be wrong indefinitely without penalty. We must ask ourselves: does this quote also apply to us? Do we allow ourselves to become actors of disinformation through sharing malicious content, thus enabling misinformation? Resources: Bellingcat Debunkeu Fullfact Graphika Politifact Propaganda Techniques Propaganda Techniques Previously published at https://www.roxanamurariu.com/how-to-counteract-disinformation/

Effective Techniques Against Disinformation

Técnicas efectivas contra la desinformación

Techniques efficaces contre la désinformation

दुष्प्रचार के खिलाफ प्रभावी तकनीकें

偽情報に対する効果的なテクニック

Técnicas eficazes contra a desinformação

Các kỹ thuật hiệu quả chống lại thông tin sai lệch

打击虚假信息的有效技术

In any database environment, there are scenarios, such as performing maintenance, applying schema changes, or conducting emergency troubleshooting, where it's necessary to restrict access to a single user. This operational state, commonly referred to as single-user mode, ensures that only one session can interact with the database at a time, preventing conflicts and ensuring data consistency during critical tasks. single-user mode single-user mode However, once these administrative tasks are complete, transitioning the database back to a multi-user mode is essential to restore regular operations and allow access for applications, developers, and users. This transition process, while conceptually similar, differs significantly depending on the underlying database management system (DBMS). multi-user mode This article provides a comparative guide on how to perform this transition across three widely used relational database management systems (RDBMS) platforms: SQL Server, PostgreSQL, and MySQL. SQL Server offers a built-in mechanism for toggling between single-user and multi-user modes via specific database options. In contrast, PostgreSQL and MySQL don’t provide a direct single-user mode but can simulate it by terminating existing sessions, modifying connection settings, or changing server configurations. SQL Server PostgreSQL MySQL By understanding the unique mechanisms of each DBMS, database administrators can confidently manage user access modes, ensure system stability during maintenance, and safely bring systems back online for general use. How to Implement in SQL Server : How to Implement in SQL Server : For SQL Server, the article demonstrates two methods, one using SQL commands and another utilizing the graphical interface (SSMS). In the command-line example, the article shows how first to verify the current access state using: -- Check current access mode for AdventureWorks database
SELECT DATABASEPROPERTYEX('AdventureWorks', 'UserAccess') AS AccessMode; -- Check current access mode for AdventureWorks database
SELECT DATABASEPROPERTYEX('AdventureWorks', 'UserAccess') AS AccessMode; After confirming that the database is in single-user mode, it recommends running: -- Enable multi-user access
ALTER DATABASE AdventureWorks SET MULTI_USER; -- Enable multi-user access
ALTER DATABASE AdventureWorks SET MULTI_USER; This command immediately restores the default state allowing multiple concurrent connections. The graphical method described involves navigating the Object Explorer by expanding the Databases folder, accessing database properties, and then selecting the appropriate option (MULTI_USER) within the state menu. Changing in GUI via SQL Server Management Studio (SSMS) Open SQL Server Management Studio
Navigate to Databases → AdventureWorks
Right-click → Properties → Options
Under the State section, locate "Restrict Access"
Select MULTI_USER from the dropdown
Click OK to apply changes Open SQL Server Management Studio Navigate to Databases → AdventureWorks AdventureWorks Right-click → Properties → Options Properties Options Under the State section, locate "Restrict Access" Restrict Access Select MULTI_USER from the dropdown MULTI_USER Click OK to apply changes Limit Connections in PostgreSQL Database PostgreSQL: PostgreSQL does not offer a direct single-user mode switch. Instead, the article outlines a two-step workaround: PostgreSQL: Managing concurrent connections is an important part of database administration. PostgreSQL provides a built-in way to limit the number of concurrent connections to a specific database using the CONNECTION LIMIT parameter in the CREATE DATABASE or ALTER DATABASE statement. limit the number of concurrent connections to a specific database CONNECTION LIMIT CREATE DATABASE ALTER DATABASE This is especially useful in scenarios where you want to avoid excessive load on your server, control access for development or testing environments, or reserve resources for high-priority databases. Syntax You can specify the connection limit when creating or altering a database using the following syntax: Create a database with a connection limit: Create a database with a connection limit: CREATE DATABASE db_name
WITH
   CONNECTION LIMIT = max_allowed_connection; CREATE DATABASE db_name
WITH
   CONNECTION LIMIT = max_allowed_connection; Alter an existing database to set a connection limit: Alter an existing database to set a connection limit: ALTER DATABASE db_name
WITH
   CONNECTION LIMIT = max_allowed_connection; ALTER DATABASE db_name
WITH
   CONNECTION LIMIT = max_allowed_connection; Parameters db_name: Name of the database.
max_allowed_connection: The maximum number of concurrent connections allowed to the database. Set it to any positive integer. The default value is -1, which means unlimited connections. db_name: Name of the database. db_name max_allowed_connection: The maximum number of concurrent connections allowed to the database. Set it to any positive integer. The default value is -1, which means unlimited connections. max_allowed_connection -1 Example: Creating a Database with Connection Limit CREATE DATABASE sales_db
WITH
   OWNER = sales_admin
   CONNECTION LIMIT = 50; CREATE DATABASE sales_db
WITH
   OWNER = sales_admin
   CONNECTION LIMIT = 50; In the above example, a new database named sales_db is created, and it allows up to 50 concurrent connections. sales_db up to 50 concurrent connections Example: Altering an Existing Database ALTER DATABASE hr_db
WITH
   CONNECTION LIMIT = 25; ALTER DATABASE hr_db
WITH
   CONNECTION LIMIT = 25; This statement modifies the hr_db database to allow no more than 25 concurrent connections. hr_db no more than 25 concurrent connections Important Notes This setting applies per database, not per user or role.
When the connection limit is reached, new connection attempts will fail with an error.
Superusers are not exempt from the connection limit for a specific database—they also count toward the total.
To view the current connection limits, query the pg_database catalog: This setting applies per database, not per user or role. When the connection limit is reached, new connection attempts will fail with an error. Superusers are not exempt from the connection limit for a specific database—they also count toward the total. not exempt To view the current connection limits, query the pg_database catalog: pg_database SELECT datname, datconnlimit
FROM pg_database; SELECT datname, datconnlimit
FROM pg_database; By managing connection limits, DBAs can ensure resource stability and predictability, especially in multi-tenant or resource-constrained PostgreSQL environments. ensure resource stability and predictability Terminate Active Sessions: The article instructs administrators to run a query based on the pg_terminate_backend function. This command terminates all active connections except the current session, ensuring that no old sessions obstruct the change.
SELECT pg_terminate_backend(pid)
FROM pg_stat_activity
WHERE datname = 'University' AND pid <> pg_backend_pid();



Control Connection Limit: Once sessions are terminated, the connection limit is set to one via:
ALTER DATABASE University WITH CONNECTION LIMIT 1;

After administrative tasks are complete, the limit is reverted by setting the value to -1:
ALTER DATABASE University WITH CONNECTION LIMIT -1; Terminate Active Sessions: The article instructs administrators to run a query based on the pg_terminate_backend function. This command terminates all active connections except the current session, ensuring that no old sessions obstruct the change.
SELECT pg_terminate_backend(pid)
FROM pg_stat_activity
WHERE datname = 'University' AND pid <> pg_backend_pid(); Terminate Active Sessions: The article instructs administrators to run a query based on the pg_terminate_backend function. This command terminates all active connections except the current session, ensuring that no old sessions obstruct the change. Terminate Active Sessions: pg_terminate_backend SELECT pg_terminate_backend(pid)
FROM pg_stat_activity
WHERE datname = 'University' AND pid <> pg_backend_pid(); SELECT pg_terminate_backend(pid)
FROM pg_stat_activity
WHERE datname = 'University' AND pid <> pg_backend_pid(); Control Connection Limit: Once sessions are terminated, the connection limit is set to one via:
ALTER DATABASE University WITH CONNECTION LIMIT 1;

After administrative tasks are complete, the limit is reverted by setting the value to -1:
ALTER DATABASE University WITH CONNECTION LIMIT -1; Control Connection Limit: Once sessions are terminated, the connection limit is set to one via: Control Connection Limit: ALTER DATABASE University WITH CONNECTION LIMIT 1; ALTER DATABASE University WITH CONNECTION LIMIT 1; After administrative tasks are complete, the limit is reverted by setting the value to -1: ALTER DATABASE University WITH CONNECTION LIMIT -1; ALTER DATABASE University WITH CONNECTION LIMIT -1; This maneuver effectively mimics single-user mode behavior without having a built-in single-user command. MySQL Configuration: MySQL Configuration: Unlike PostgreSQL, MySQL does not have a dedicated single-user mode. The article details a process relying on terminating connections and adjusting global connection limits: Listing and Terminating Sessions: The command SHOW PROCESSLIST; is used to view active database sessions. Specific sessions can then be terminated by using:
KILL <session_id>;



Switching Connection Modes: Initially, to restrict the database for administrative tasks, the global connection parameter is reduced to one:
SET GLOBAL max_connections = 1;

Once the necessary changes or backups are made, the connections are reinstated by resetting the variable to its typical default (often 151):
SET GLOBAL max_connections = 600; Listing and Terminating Sessions: The command SHOW PROCESSLIST; is used to view active database sessions. Specific sessions can then be terminated by using:
KILL <session_id>; Listing and Terminating Sessions: The command SHOW PROCESSLIST; is used to view active database sessions. Specific sessions can then be terminated by using: Listing and Terminating Sessions: SHOW PROCESSLIST; KILL <session_id>; KILL <session_id>; Switching Connection Modes: Initially, to restrict the database for administrative tasks, the global connection parameter is reduced to one:
SET GLOBAL max_connections = 1;

Once the necessary changes or backups are made, the connections are reinstated by resetting the variable to its typical default (often 151):
SET GLOBAL max_connections = 600; Switching Connection Modes: Initially, to restrict the database for administrative tasks, the global connection parameter is reduced to one: Switching Connection Modes: SET GLOBAL max_connections = 1; SET GLOBAL max_connections = 1; Once the necessary changes or backups are made, the connections are reinstated by resetting the variable to its typical default (often 151): SET GLOBAL max_connections = 600; SET GLOBAL max_connections = 600; Core Themes and Best Practices: Core Themes and Best Practices: Verification: It is crucial to check the current database mode (or current active sessions) before making changes.
Controlled Termination: When working with PostgreSQL or MySQL, terminating active sessions in a controlled manner is essential to prevent conflicts.
Reversion Post-Administration: After administrative tasks are conducted in the restricted mode, reset connection limits so that the database resumes normal multi-user operations.
Documentation and Monitoring: Regular monitoring and clear documentation ensure that administrators can track changes and identify issues with concurrency. Verification: It is crucial to check the current database mode (or current active sessions) before making changes. Verification: Controlled Termination: When working with PostgreSQL or MySQL, terminating active sessions in a controlled manner is essential to prevent conflicts. Controlled Termination: Reversion Post-Administration: After administrative tasks are conducted in the restricted mode, reset connection limits so that the database resumes normal multi-user operations. Reversion Post-Administration: Documentation and Monitoring: Regular monitoring and clear documentation ensure that administrators can track changes and identify issues with concurrency. Documentation and Monitoring: Summary: The article provides a comparative analysis of handling multi-user mode in SQL databases. For SQL Server, the transition is straightforward using system commands or a graphical interface. In contrast, PostgreSQL and MySQL require a temporary adjustment by terminating active sessions and modifying connection limits. Adherence to best practices such as verifying the mode, managing sessions carefully, and properly reverting limits ensures that system integrity is maintained during these transitions. Summary:

Want Everyone Out of the Database? Here’s How the Pros Do It

Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Related Work 2 Related Work 3 Method and 3.1 Phase 1: Taxonomy Generation 3 Method and 3.1 Phase 1: Taxonomy Generation 3.2 Phase 2: LLM-Augmented Text Classification 3.2 Phase 2: LLM-Augmented Text Classification 4 Evaluation Suite and 4.1 Phase 1 Evaluation Strategies 4 Evaluation Suite and 4.1 Phase 1 Evaluation Strategies 4.2 Phase 2 Evaluation Strategies 4.2 Phase 2 Evaluation Strategies 5 Experiments and 5.1 Data 5 Experiments and 5.1 Data 5.2 Taxonomy Generation 5.2 Taxonomy Generation 5.3 LLM-Augmented Text Classification 5.3 LLM-Augmented Text Classification 5.4 Summary of Findings and Suggestions 5.4 Summary of Findings and Suggestions 6 Discussion and Future Work, and References 6 Discussion and Future Work, and References A. Taxonomies A. Taxonomies B. Additional Results B. Additional Results C. Implementation Details C. Implementation Details D. Prompt Templates D. Prompt Templates 6 DISCUSSION AND FUTURE WORK This work has the potential to create significant impact for research and application of AI technologies in text mining. Our framework has demonstrated the ability to use LLMs as taxonomy generators, as well as data labelers and evaluators. These automations could lead to significant efficiency gains and cost savings for a variety of domains and applications that rely on understanding, structuring and analyzing massive volumes of unstructured text. It could also broadly democratize the process of mining knowledge from text, empowering non-expert users and enterprises to interact with and interpret their data through natural language, thereby leading to better insights and data-driven decision making for a range of industries and sectors. Additionally, our framework and research findings relate to other work that leverages LLMs for taxonomy creation and text clustering, and has important empirical lessons for the efficient use of instruction-following models in these scenarios. Despite these initial successes, there are some important challenges and future directions that are worth exploring. As we have already noted, LLMs are expensive and slow. In future work, we hope to explore ways to improve the speed, efficiency and robustness of our framework, through hybrid approaches that further explore the combination of LLMs with embedding-based methods, or model distillation that fine-tunes a smaller model through instructions from a larger one. Evaluation continues to be a crucial and open challenge for future work, and we plan to explore ways of performing more robust LLM-aided evaluations in future work, for example by fine-tuning a model to expand its reasoning capabilities beyond pairwise judgement tasks. While this work has focused largely on text mining in the conversational domain, we also hope to explore the extensibility of our framework to other domains as well. Finally, many domains have ethical considerations from the perspective of privacy and security that must be taken into account when performing large-scale automated text mining, and we hope to engage with these challenges more deeply in future work. REFERENCES [1] Charu C Aggarwal and ChengXiang Zhai. 2012. A survey of text clustering algorithms. Mining text data (2012), 77–128. [2] David Arthur, Sergei Vassilvitskii, et al. 2007. k-means++: The advantages of careful seeding. In Soda, Vol. 7. 1027–1035. [3] Léon Bottou. 1998. Online algorithms and stochastic approximations. Online learning in neural networks (1998). [4] B Barla Cambazoglu, Leila Tavakoli, Falk Scholer, Mark Sanderson, and Bruce Croft. 2021. An intent taxonomy for questions asked in web search. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval. 85–94. [5] Jonathan Chang, Sean Gerrish, Chong Wang, Jordan Boyd-Graber, and David Blei. 2009. Reading tea leaves: How humans interpret topic models. Advances in neural information processing systems 22 (2009). [6] Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1 (1960), 37–46. [7] Joseph L Fleiss and Jacob Cohen. 1973. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and psychological measurement 33, 3 (1973), 613–619. [8] Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. Chatgpt outperforms crowd-workers for text-annotation tasks. arXiv preprint arXiv:2303.15056 (2023). [9] Simon Haykin. 1998. Neural networks: a comprehensive foundation. Prentice Hall PTR. [10] Andreas Hotho, Andreas Nürnberger, and Gerhard Paaß. 2005. A brief survey of text mining. Journal for Language Technology and Computational Linguistics 20, 1 (2005), 19–62. [11] Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, and Tomas Mikolov. 2016. FastText.zip: Compressing text classification models. arXiv preprint arXiv:1612.03651 (2016). [12] Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016. Bag of Tricks for Efficient Text Classification. arXiv preprint arXiv:1607.01759 (2016). [13] Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems 30 (2017). [14] Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014). https://api.semanticscholar.org/CorpusID:6628106 https://api.semanticscholar.org/CorpusID: [15] Dong-Ho Lee, Jay Pujara, Mohit Sewak, Ryen White, and Sujay Jauhar. 2023. Making Large Language Models Better Data Creators. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 15349–15360. https://doi.org/10.18653/v1/2023.emnlp-main.948 https://doi.org/10.18653/v1/2023.emnlp-main.948 [16] Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the middle: How language models use long contexts. arXiv preprint arXiv:2307.03172 (2023). [17] Geoffrey J McLachlan and Kaye E Basford. 1988. Mixture models: Inference and applications to clustering. Vol. 38. M. Dekker New York. [18] Chau Minh Pham, Alexander Hoyle, Simeng Sun, and Mohit Iyyer. 2023. TopicGPT: A Prompt-based Topic Modeling Framework. arXiv preprint arXiv:2311.01449 (2023). [19] Reid Pryzant, Dan Iter, Jerry Li, Yin Lee, Chenguang Zhu, and Michael Zeng. 2023. Automatic Prompt Optimization with “Gradient Descent” and Beam Search. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 7957–7968. https://doi.org/10.18653/v1/2023.emnlp-main.494 https://doi.org/10.18653/v1/ [20] Daniel E Rose and Danny Levinson. 2004. Understanding user goals in web search. In Proceedings of the 13th international conference on World Wide Web. 13–19. [21] Peter J Rousseeuw. 1987. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics 20 (1987), 53–65. [22] Evan Sandhaus. 2008. The new york times annotated corpus. Linguistic Data Consortium, Philadelphia 6, 12 (2008), e26752. [23] Chirag Shah, Ryen W White, Reid Andersen, Georg Buscher, Scott Counts, Sarkar Snigdha Sarathi Das, Ali Montazer, Sathish Manivannan, Jennifer Neville, Xiaochuan Ni, et al. 2023. Using large language models to generate, validate, and apply user intent taxonomies. arXiv preprint arXiv:2309.13063 (2023). [24] Jingbo Shang, Xinyang Zhang, Liyuan Liu, Sha Li, and Jiawei Han. 2020. Nettaxo: Automated topic taxonomy construction from text-rich network. In Proceedings of the Web Conference 2020. 1908–1919. [25] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing. 1631–1642. [26] Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2022. One Embedder, Any Task: Instruction-Finetuned Text Embeddings. https://arxiv.org/abs/2212.09741 https://arxiv.org/abs/2212.0 [27] Ah-Hwee Tan et al. 1999. Text mining: The state of the art and the challenges. In Proceedings of the PAKDD 1999 workshop on knowledge discovery from advanced databases, Vol. 8. 65–70. [28] Paul Thomas, Seth Spielman, Nick Craswell, and Bhaskar Mitra. 2023. Large language models can accurately predict searcher preferences. arXiv preprint arXiv:2309.10621 (2023). [29] Ike Vayansky and Sathish AP Kumar. 2020. A review of topic modeling methods. Information Systems 94 (2020), 101582. [30] Zihan Wang, Jingbo Shang, and Ruiqi Zhong. 2023. Goal-Driven Explainable Clustering via Language Descriptions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 10626–10649. https://doi.org/10.18653/v1/2023.emnlp-main.657 https://doi.org/10.18653/v1/2023.emnlp-main.657 [31] Anuradha Welivita and Pearl Pu. 2020. A Taxonomy of Empathetic Response Intents in Human Social Conversations. In Proceedings of the 28th International Conference on Computational Linguistics. 4886–4899. [32] Qingkai Zeng, Jinfeng Lin, Wenhao Yu, Jane Cleland-Huang, and Meng Jiang. 2021. Enhancing taxonomy completion with concept generation via fusing relational representations. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2104–2113. [33] Chao Zhang, Fangbo Tao, Xiusi Chen, Jiaming Shen, Meng Jiang, Brian Sadler, Michelle Vanni, and Jiawei Han. 2018. Taxogen: Unsupervised topic taxonomy construction by adaptive term embedding and clustering. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2701–2709. [34] Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. 2023. Can Large Language Models Transform Computational Social Science? arXiv preprint arXiv:2305.03514 (2023). This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. available on arxiv Authors:
(1) Mengting Wan, Microsoft Corporation and Microsoft Corporation;
(2) Tara Safavi (Corresponding authors), Microsoft Corporation;
(3) Sujay Kumar Jauhar, Microsoft Corporation;
(4) Yujin Kim, Microsoft Corporation;
(5) Scott Counts, Microsoft Corporation;
(6) Jennifer Neville, Microsoft Corporation;
(7) Siddharth Suri, Microsoft Corporation;
(8) Chirag Shah, University of Washington and Work done while working at Microsoft;
(9) Ryen W. White, Microsoft Corporation;
(10) Longqi Yang, Microsoft Corporation;
(11) Reid Andersen, Microsoft Corporation;
(12) Georg Buscher, Microsoft Corporation;
(13) Dhruv Joshi, Microsoft Corporation;
(14) Nagu Rangan, Microsoft Corporation. Authors: Authors: (1) Mengting Wan, Microsoft Corporation and Microsoft Corporation; (2) Tara Safavi (Corresponding authors), Microsoft Corporation; (3) Sujay Kumar Jauhar, Microsoft Corporation; (4) Yujin Kim, Microsoft Corporation; (5) Scott Counts, Microsoft Corporation; (6) Jennifer Neville, Microsoft Corporation; (7) Siddharth Suri, Microsoft Corporation; (8) Chirag Shah, University of Washington and Work done while working at Microsoft; (9) Ryen W. White, Microsoft Corporation; (10) Longqi Yang, Microsoft Corporation; (11) Reid Andersen, Microsoft Corporation; (12) Georg Buscher, Microsoft Corporation; (13) Dhruv Joshi, Microsoft Corporation; (14) Nagu Rangan, Microsoft Corporation.

TnT-LLM: Democratizing Text Mining with Automated Taxonomy and Scalable Classification

SeaTunnel + Bedrock + OpenSearch = AI That Gets What You’re Saying

The Markup and the Curious Case of the Tricky Timeout Bug

According to CoinMarketCap (CMC), there are over 10,000 coins and tokens available to trade in different markets worldwide. Among them, we can find all kinds of things, from old-established cryptocurrencies like Bitcoin and Obyte to experimental DeFi platforms and memecoins. However, you might not know that, in addition to a vast array of tokens, there’s also a diverse selection of free crypto-related tools —ranging from practical to quirky and entertaining. CMC and CoinGecko are, for instance, two quite popular and useful crypto tools, but they’re far from being the only ones. We can discover some interesting things if we just look around a bit. Let’s see some platforms in this category. When Lambo? Memecoins are quite the thing now, but the crypto community always had its own internal memes and jokes without related tokens. “When Lambo” is an old one and a very popular one too. It implies a momentous question: when will we be able to afford an expensive Lamborghini car with our current crypto holdings? It’s known that prices change rapidly in the crypto space, so can you maybe get richer tomorrow with the same number of coins? We can dream, at least. “When Lambo” is so widespread around here that some years ago CMC changed all the prices from USD to LAMBOs in April’s Fools. And the question has its own calculator, to help you find out once and for all how much of a Lambo you can buy with your current holdings in different coins. The site Crypto to Lambo allows you to type the amount, choose between 100 coins and three Lamborghini models, and calculate, well, when Lambo? CMC changed CMC changed The site Crypto to Lambo allows you to type the amount, choose between 100 coins and three Lamborghini models, and calculate, well, when Lambo? Crypto to Lambo Crypto to Lambo The results will include what Lambo you can buy with your coins (at least a toy Lambo?), how many more tokens you need to buy it, the needed price per coin you’ll need to afford it, and the current percentage you have in your wallet. It’ll also lead you to trade on the exchanges Binance or Bitvavo, but that’s an additional option. Do your own research first! Wallet Scrutiny Cryptocurrencies are mostly decentralized, which means you have full control over your holdings if you have a private key. But wait a second, because there might be a trick there. The coins and their ledger are decentralized, yes, but the wallet you use to interact with crypto networks can be a completely different story. Wallets are often offered by middlemen (companies and developer teams), who generally have good intentions —but that’s not always the case. Some awful things (backdoors and other traps) could be hidden in their codes, beyond what we can see. Even if the wallets are open-source software, which means anyone can check their codes, not everyone knows how to. And worse: cybercriminals are designing fake wallets with the same name and branding as legitimate ones, aiming to trick users into downloading this malicious software from official app stores to steal their private keys. Some awful things (backdoors and other traps) could be hidden in their codes, beyond what we can see. Even if the wallets are open-source software, which means anyone can check their codes, not everyone knows how to Now, of course, there’s a way to avoid these bad scenarios, and we have several easy and free tools available for it. WalletScrunity is one of them, focused primarily on Bitcoin wallets, but basic information about others is available too. On this site, anyone can just type the name of their preferred wallet (software or hardware) and discover if it passed the tests and questions from their developers. Among those tests, they verify who’s the team behind the brand, if it’s open-source or not, if it’s non-custodial (offer private keys) or not, and if it’s the original product. They also offer safe links to download or buy the correct version. WalletScrunity WalletScrunity Crypto Bubbles With all their detailed data and long lists, sites to check cryptocurrency market caps, like CMC and CoinGecko, can be a tad overwhelming for beginners. Crypto Bubbles offers a more visual, dynamic experience to check the same data, using colorful bubbles and user-friendly interfaces to do it. The first thing you’ll see on this site is a lot of crypto logos and tickers floating around in bubbles of different sizes and colors, indeed. Crypto Bubbles offers a more visual, dynamic experience to check the same data, using colorful bubbles and user-friendly interfaces to do it. Crypto Bubbles Crypto Bubbles Green bubbles mean price increases —the larger the bubble, the higher the percentage change or market cap. Red bubbles imply negative percentages, and they also change in size, depending on the figures. It’s possible to order them per period (hour, day, week, month, year) or market cap size. In the Settings, they’ll let you change the currency for the prices (USD, EUR, CAD, etc.), languages, colors, and watchlists. If you click a bubble, more specific data, charts, and links about that currency will appear, summed up in a very graphical manner. Below the bubbles, a more traditional list of coins is also available to order by market cap, price, volume, or time changes. Or you can use their search bar to find your favorite coins. An app for iOS and Android can be downloaded too. TxCity Network explorers aren’t exactly the most colorful and user-friendly thing in the world. And let’s not talk about those long, incomprehensible alphanumeric strings that represent transactions and addresses. TxCity was created to see crypto transactions in a new, clearer light. By representing each transaction as a person attempting to board a bus—where buses symbolize blocks in a blockchain— this platform offers users a unique, real-time perspective on the available crypto networks. By representing each transaction as a person attempting to board a bus—where buses symbolize blocks in a blockchain— this platform offers users a unique, real-time perspective on the available crypto networks. Obviously, those networks are only blockchains — they are the kind of networks that have buses and bus drivers. This approach demystifies on-chain data, effectively converting mere transactional networks into visual ecosystems. https://x.com/tastycrypto/status/1887589349677744200?embedable=true https://x.com/tastycrypto/status/1887589349677744200?embedable=true By using TxCity, anyone can visualize transactions across multiple networks, including Bitcoin, Ethereum, Bitcoin Cash, Monero, and Litecoin. The platform also plans to introduce features like bridge visualization between Ethereum and LUKSO, providing users with a real-time experience of their cross-chain transactions. Additionally, TxCity aims to represent wallets as characters or avatars, simplifying user interactions. TxCity TxCity To begin using TxCity, simply visit their website and select the chain you wish to explore. The platform will display live transactions as animated characters attempting to board buses, offering an engaging way to observe how transactions are processed and included in blocks. This visual representation not only makes it easier to understand crypto operations but also serves as an educational tool for those new to this space. Order a Pizza in Obyte Did you know you can order a Hawaiian, Pepperoni, or Mexican pizza using an Obyte chatbot? Well, at least as a role-play! The Obyte Testnet Wallet offers a fun and risk-free way to explore cryptocurrency transactions, including interacting with chatbots. By following simple steps, users can simulate making a purchase and experience how payments work within the Obyte ecosystem. It’s an engaging way to see how digital payments function while learning about Obyte’s capabilities in a relaxed and playful environment. The Obyte Testnet Wallet offers a fun and risk-free way to explore cryptocurrency transactions, including interacting with chatbots. Obyte Testnet Wallet Obyte Testnet Wallet A testnet is a duplicate version of a crypto network designed for learning, development, and testing. Unlike the mainnet, where transactions involve real assets, the testnet allows users to practice without any financial risk. The Obyte Testnet Wallet is a safe space where developers can refine their applications and everyday users can experiment with different features. To start, users can claim free testnet coins—tokens without market value—to send transactions, explore smart contracts, and even test advanced security measures like two-factor authentication (2FA) or multi-signature accounts. A testnet A testnet free testnet coins free testnet coins Beyond simulated pizza purchases, the testnet wallet offers a variety of features to explore. Users can try chatbots from the Bot Store, set up conditional payments, and experiment with wallet settings. It’s also possible to add new devices and test secure account recovery options. Everything is recorded on the testnet explorer, providing insights into how transactions are processed. Since no real money is involved, users can experiment freely and gain confidence before switching to the actual Obyte wallet. testnet explorer testnet explorer Obyte wallet Obyte wallet Featured Vector Image by Freepik Featured Vector Image by Freepik Freepik Freepik

5 Quirky and Useful Crypto Tools You Didn’t Know Before

It was in early 2025 that I initiated my own blog, OpenExploit.in, with one simple aim: to make cybersecurity easy to learn and available to all—particularly students and curious minds who are new to this field. I decided to host my blog using Ghost CMS and pledged to update it on a regular basis, discussing what I already know and what I'm currently learning. It's been a thrilling, at times daunting, but enormously rewarding ride. In this article, I'd like to share what I've learned up until now from setting up and maintaining a cybersecurity blog—from technical configuration to publishing content, interacting with readers, and how to stay motivated. If you're considering setting up your own blog or simply wondering what it's like, keep reading. Starting Was the Hardest Part As with all things in life, the initial step was the most difficult one. I went weeks deliberating over names, designs, and tools. I wanted everything to be "perfect" before launching. But after some time, I came to understand that perfection is the arch-nemesis of progress. So I settled on a name that was fitting—OpenExploit.in—and used Ghost CMS for its sleek, minimal look and speed. Lesson learned: Don't wait for perfection. Start small, and improve along the way. Choose the Right Platform Early I tried a few: WordPress, Blogger, and Ghost. I chose Ghost CMS because: It's open-source
It's fast
The writing experience is distraction-free
It's ideal for developers and technical blogs It's open-source It's fast The writing experience is distraction-free It's ideal for developers and technical blogs Setting up Ghost on a server was a bit of a learning curve, but it taught me about server configuration, domain management, SSL, and more. In hindsight, that technical knowledge became part of my own cybersecurity learning journey. Lesson learned: Choosing a platform that fits your technical comfort level matters a lot. You’ll spend a lot of time with it, so make sure it feels right. Writing Is a Skill—And It Gets Better With Practice I was anxious writing my initial post. Would anyone read it? Would anyone judge it? What if I got something wrong? The reality? None of that initially. I just had to write. My initial post was simple, detailing how to know if a website is safe to click on. Not ideal, but useful. As I wrote more, I found myself becoming more skilled at making things simple. The more I wrote, the surer I became. Lesson learned: Writing is similar to exercising. The more you exercise it, the more skilled you become. And the more clearly you write, the better understood you will be by people. Simplify Complex Subjects Cybersecurity can get very technical. When I was in the mode of writing for beginners, I found that making things simple and accurate in wording is an art. You can't simply plagiarize from a book. You need to: Use analogies
Avoid jargon
Break content into bite-sized sections
Include visuals or code snippets
Point to further resources Use analogies Avoid jargon Break content into bite-sized sections Include visuals or code snippets Point to further resources For instance, rather than merely stating "SQL injection enables attackers to manipulate database queries," I employed an everyday analogy: "It's like writing your own ending to someone else's sentence." Lesson learned: When teaching others, you're compelled to make your own thinking simple. And that's a good thing. Consistency > Perfection One of the things I wanted to do was post frequently—daily, if possible. Life doesn't always agree, but I attempted to be consistent. Some posts were lengthy and informative; others were brief tips or references to good resources. Consistency allowed me to gain momentum, remain motivated, and gradually accumulate a reader base. Lesson learned: Posting a small useful article every few days is preferable to waiting weeks to put out a single "perfect" post. There's a Huge Community Out There There are cybersecurity enthusiasts all over—on Twitter (X), LinkedIn, Reddit, Discord, and YouTube. After I began posting my posts and interacting with other people, I knew I was part of an enormous community. Individuals began responding, offering feedback, posing questions, and even proposing new subjects. That interaction made me feel connected and encouraged me to continue. Lesson learned: Don't blog in isolation. Share your post, engage in discussions, assist others. You'll receive much in return. Use Multimedia: Code, Images, and Videos Cybersecurity is tactile and graphical. So I began including: Code snippets (e.g., Python scripts, Nmap scans)
Screenshots of software such as Wireshark or Burp Suite
YouTube Shorts or video tutorials Code snippets (e.g., Python scripts, Nmap scans) Screenshots of software such as Wireshark or Burp Suite YouTube Shorts or video tutorials That not only helped readers understand better but also increased the time they spent on my posts. Now, I’m planning to embed more demo videos and maybe even start a companion YouTube channel. Lesson learned: People learn in different ways. Give them multiple options—text, video, images, code. It’s Okay to Not Know Everything Initially, I was anxious: "What if I mess something up?" But the thing is, nobody knows everything about cybersecurity. It's too vast, too dynamic. Rather than acting like an expert, I was a learner who explained what they were learning. That made me more down-to-earth and authentic. I even posted about things I didn't completely understand yet—just going through my learning process. Lesson learned: Be real. It's alright to say, "I'm still learning this." People value honesty. SEO, Titles, and Thumbnails Matter As much as I emphasized the content, I finally realized that titles, descriptions, and images count as well. A blog post with the title "Basics of Cybersecurity" is less interesting than one with the title "5 Cybersecurity Myths You Still Believe (And What's Actually True)." Good SEO makes people discover your content. I began learning: How to write catchy headlines
How to use keywords naturally
How to write summaries and meta descriptions
How to design simple but clickable thumbnails How to write catchy headlines How to use keywords naturally How to write summaries and meta descriptions How to design simple but clickable thumbnails Lesson learned: Content is king, but presentation is queen. Your Blog Is Your Resume I didn't know this initially, but my blog was now part of my personal brand. When I was applying for jobs or meeting people online, I could simply send them a link to OpenExploit.in. It demonstrated that I: Know how to write about technical subjects
Can dedicate myself to a project
Keep up to date with trends
Care about assisting others Know how to write about technical subjects Can dedicate myself to a project Keep up to date with trends Care about assisting others Lesson learned: A blog isn't a side project—it can be a career booster. Final Thoughts Opening OpenExploit.in has been one of the greatest things I've ever done. It's helped me become a better communicator, a more self-assured learner, and part of an incredible cybersecurity community. If you've ever considered opening your own blog—do it. Don't worry about being perfect or knowing everything. Just begin. Write what you know. Someone out there needs to know it. Thanks for reading. And if you're on the same journey—let's connect.

Here’s Everything I Learned Building a Blog to Teach Cybersecurity

Table of Links Abstract and 1 Introduction 1.1 Related literature Types of Metrics and Their Hypothesis and 2.1 Types of metrics
2.2 Hypotheses for different types of metrics


Type I and Type II Error Rates for Decision Rules including Superiority and Non-Inferiority Tests
3.1 The composite hypotheses of superiority and non-inferiority tests
3.2 Bounding the type I and type II error rates for UI and IU testing
3.3 Bounding the error rates for a decision rule including both success and guardrail metrics
3.4 Power corrections for non-inferiority testing


Extending the Decision Rule with Deterioration and Quality Metrics


Monte Carlo Simulation Study
5.1 Results


Discussion and Conclusions APPENDIX A: IMPROVING THE EFFICIENCY OF PROPOSITION 4.1 WITH ADDITIONAL ASSUMPTIONS APPENDIX B: EXAMPLES OF GLOBAL FALSE AND TRUE POSITIVE RATES APPENDIX C: A NOTE ON SEQUENTIAL TESTING OF DETERIORATION APPENDIX D: USING NYHOLT’S METHOD OF EFFICIENT NUMBER OF INDEPENDENT TESTS Acknowledgments and References APPENDIX A: IMPROVING THE EFFICIENCY OF PROPOSITION 4.1 WITH ADDITIONAL ASSUMPTIONS It is possible to improve the efficiency by making assumptions that makes the inferiority test rejection region not overlap with the rejection region of the superiority or non-inferiority test, respectively, for success metrics and guardrail metrics. In the sections below, we describe how deterioration tests affect the type I and type II error rates for superiority and non-inferiority tests and under what conditions the rates are not affected. We then use this information to update Proposition 4.1. A.1 Type I and type II error rates for a success metric with superiority and deterioration tests To be specific about the consequences of testing a success metric with an additional deterioration test, we require some new notation. We focus on a single success metric and how this metric contributes to the shipping decision. There are now two tests and four hypotheses for the success metric, defined as: The deterioration test is here an inferiority test that mirrors the main superiority test, but with the hypotheses reversed. Because of these two tests, there are now two possible false positive results for this metric: a false positive inferiority test, and a false positive superiority test. The false positive risk in terms of the shipping decision for a set of success metrics with superiority and inferiority tests is given by the probability that the superiority test rejects the null hypothesis while, simultaneously, the inferiority test does not. The additional criterion of non-significant inferiority tests can only make the superiority test more conservative, i.e., reduce the false positive rate and the true positive rate of the shipping decision. However, it is easy to restrict the rejection regions of the deterioration test and the superiority test such that the deterioration test cannot affect the rejection rate of the superiority test under any hypothesis. Lemma A.1 formalizes this. Lemma A.1 says that if the same test statistic is used for the superiority test and the deterioration test, and α_ < 1− α, the rejection regions cannot overlap. This is somewhat trivial for the success metric case. In the next section, we present similar results for guardrail metrics with non-inferiority and deterioration tests. A.2 Type I and type II error rates for a guardrail metric with non-inferiority and deterioration tests For guardrail metrics, we can reason analogously to the success metrics case, but there are additional nuances due to the potential overlap between rejection regions of the deterioration and non-inferiority tests. There are four relevant hypotheses given by The fact that β ≥ α− is a sufficient additional criterion for the rejection regions to not overlap in the non-inferiority case is perhaps most easily seen graphically. Figure 2 displays the relation between the regions when β = α_. From the figure it is clear that the regions would overlap for a fixed β if α_ increases. Analogously, a decrease in β would make the regions overlap for a fixed α_. Note that as long as α_ ≤ 1 − α+, the value of α+ does not affect the overlap. This follows since if α+ changes, the implied NIM for which the non-inferiority test has power 1 − β changes accordingly. A.3 An updated version of Proposition 4.1 Here we improve the α and β corrections slightly by enforcing non-overlapping rejections regions. PROOF. The proof is almost identical to the proof of Proposition 4.1. Now we have under the global null Which implies We can adjust our correction of α to make the test less conservative by solving which implies that we should correct α according to to minimize the conservativeness without any assumptions on the data generating process. for the type II error rate we follow the proof of Proposition 4.1 closely. The only changes are that A.4 Additional approximate improvements The β correction in the updated Proposition bounds the type II error rate under a quite extreme worst-case scenario. This is because under the alternative, and given that at least one success metric has improved, the probability that any of the remaining success metrics is significantly inferior is very low. In fact, the probability is so low that it is usually negligible. We formalize this in Remark A.1. from which the remark follows immediately from the previous proof. Authors:
(1) Mårten Schultzberg, Experimentation Platform team, Spotify, Stockholm, Sweden;
(2) Sebastian Ankargren, Experimentation Platform team, Spotify, Stockholm, Sweden;
(3) Mattias Frånberg, Experimentation Platform team, Spotify, Stockholm, Sweden. This paper is available on arxiv under CC BY 4.0 DEED license. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 1.1 Related literature 1.1 Related literature Types of Metrics and Their Hypothesis and 2.1 Types of metrics
2.2 Hypotheses for different types of metrics Type I and Type II Error Rates for Decision Rules including Superiority and Non-Inferiority Tests
3.1 The composite hypotheses of superiority and non-inferiority tests
3.2 Bounding the type I and type II error rates for UI and IU testing
3.3 Bounding the error rates for a decision rule including both success and guardrail metrics
3.4 Power corrections for non-inferiority testing Extending the Decision Rule with Deterioration and Quality Metrics Monte Carlo Simulation Study
5.1 Results Discussion and Conclusions Types of Metrics and Their Hypothesis and 2.1 Types of metrics 2.2 Hypotheses for different types of metrics Types of Metrics and Their Hypothesis and 2.1 Types of metrics Types of Metrics and Their Hypothesis and 2.1 Types of metrics 2.2 Hypotheses for different types of metrics 2.2 Hypotheses for different types of metrics Type I and Type II Error Rates for Decision Rules including Superiority and Non-Inferiority Tests 3.1 The composite hypotheses of superiority and non-inferiority tests 3.2 Bounding the type I and type II error rates for UI and IU testing 3.3 Bounding the error rates for a decision rule including both success and guardrail metrics 3.4 Power corrections for non-inferiority testing Type I and Type II Error Rates for Decision Rules including Superiority and Non-Inferiority Tests Type I and Type II Error Rates for Decision Rules including Superiority and Non-Inferiority Tests 3.1 The composite hypotheses of superiority and non-inferiority tests 3.1 The composite hypotheses of superiority and non-inferiority tests 3.2 Bounding the type I and type II error rates for UI and IU testing 3.2 Bounding the type I and type II error rates for UI and IU testing 3.3 Bounding the error rates for a decision rule including both success and guardrail metrics 3.3 Bounding the error rates for a decision rule including both success and guardrail metrics 3.4 Power corrections for non-inferiority testing 3.4 Power corrections for non-inferiority testing Extending the Decision Rule with Deterioration and Quality Metrics Extending the Decision Rule with Deterioration and Quality Metrics Extending the Decision Rule with Deterioration and Quality Metrics Monte Carlo Simulation Study 5.1 Results Monte Carlo Simulation Study Monte Carlo Simulation Study 5.1 Results 5.1 Results Discussion and Conclusions Discussion and Conclusions Discussion and Conclusions APPENDIX A: IMPROVING THE EFFICIENCY OF PROPOSITION 4.1 WITH ADDITIONAL ASSUMPTIONS APPENDIX A: APPENDIX A: IMPROVING THE EFFICIENCY OF PROPOSITION 4.1 WITH ADDITIONAL ASSUMPTIONS APPENDIX B: EXAMPLES OF GLOBAL FALSE AND TRUE POSITIVE RATES APPENDIX B: APPENDIX B: EXAMPLES OF GLOBAL FALSE AND TRUE POSITIVE RATES APPENDIX C: A NOTE ON SEQUENTIAL TESTING OF DETERIORATION APPENDIX C: APPENDIX C: A NOTE ON SEQUENTIAL TESTING OF DETERIORATION APPENDIX D: USING NYHOLT’S METHOD OF EFFICIENT NUMBER OF INDEPENDENT TESTS APPENDIX D: APPENDIX D: USING NYHOLT’S METHOD OF EFFICIENT NUMBER OF INDEPENDENT TESTS Acknowledgments and References Acknowledgments and References APPENDIX A: IMPROVING THE EFFICIENCY OF PROPOSITION 4.1 WITH ADDITIONAL ASSUMPTIONS It is possible to improve the efficiency by making assumptions that makes the inferiority test rejection region not overlap with the rejection region of the superiority or non-inferiority test, respectively, for success metrics and guardrail metrics. In the sections below, we describe how deterioration tests affect the type I and type II error rates for superiority and non-inferiority tests and under what conditions the rates are not affected. We then use this information to update Proposition 4.1. A.1 Type I and type II error rates for a success metric with superiority and deterioration tests To be specific about the consequences of testing a success metric with an additional deterioration test, we require some new notation. We focus on a single success metric and how this metric contributes to the shipping decision. There are now two tests and four hypotheses for the success metric, defined as: The deterioration test is here an inferiority test that mirrors the main superiority test, but with the hypotheses reversed. Because of these two tests, there are now two possible false positive results for this metric: a false positive inferiority test, and a false positive superiority test. The false positive risk in terms of the shipping decision for a set of success metrics with superiority and inferiority tests is given by the probability that the superiority test rejects the null hypothesis while, simultaneously, the inferiority test does not. The additional criterion of non-significant inferiority tests can only make the superiority test more conservative, i.e., reduce the false positive rate and the true positive rate of the shipping decision. However, it is easy to restrict the rejection regions of the deterioration test and the superiority test such that the deterioration test cannot affect the rejection rate of the superiority test under any hypothesis. Lemma A.1 formalizes this. Lemma A.1 says that if the same test statistic is used for the superiority test and the deterioration test, and α_ < 1− α, the rejection regions cannot overlap. This is somewhat trivial for the success metric case. In the next section, we present similar results for guardrail metrics with non-inferiority and deterioration tests. A.2 Type I and type II error rates for a guardrail metric with non-inferiority and deterioration tests For guardrail metrics, we can reason analogously to the success metrics case, but there are additional nuances due to the potential overlap between rejection regions of the deterioration and non-inferiority tests. There are four relevant hypotheses given by The fact that β ≥ α− is a sufficient additional criterion for the rejection regions to not overlap in the non-inferiority case is perhaps most easily seen graphically. Figure 2 displays the relation between the regions when β = α_. From the figure it is clear that the regions would overlap for a fixed β if α_ increases. Analogously, a decrease in β would make the regions overlap for a fixed α_. Note that as long as α_ ≤ 1 − α+, the value of α+ does not affect the overlap. This follows since if α+ changes, the implied NIM for which the non-inferiority test has power 1 − β changes accordingly. A.3 An updated version of Proposition 4.1 Here we improve the α and β corrections slightly by enforcing non-overlapping rejections regions. PROOF. The proof is almost identical to the proof of Proposition 4.1. Now we have under the global null Which implies We can adjust our correction of α to make the test less conservative by solving which implies that we should correct α according to to minimize the conservativeness without any assumptions on the data generating process. for the type II error rate we follow the proof of Proposition 4.1 closely. The only changes are that A.4 Additional approximate improvements The β correction in the updated Proposition bounds the type II error rate under a quite extreme worst-case scenario. This is because under the alternative, and given that at least one success metric has improved, the probability that any of the remaining success metrics is significantly inferior is very low. In fact, the probability is so low that it is usually negligible. We formalize this in Remark A.1. from which the remark follows immediately from the previous proof. Authors: (1) Mårten Schultzberg, Experimentation Platform team, Spotify, Stockholm, Sweden; (2) Sebastian Ankargren, Experimentation Platform team, Spotify, Stockholm, Sweden; (3) Mattias Frånberg, Experimentation Platform team, Spotify, Stockholm, Sweden. Authors: Authors: (1) Mårten Schultzberg, Experimentation Platform team, Spotify, Stockholm, Sweden; (2) Sebastian Ankargren, Experimentation Platform team, Spotify, Stockholm, Sweden; (3) Mattias Frånberg, Experimentation Platform team, Spotify, Stockholm, Sweden. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv

How to Improve Accuracy in Success and Safety Testing

Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Survey with Industry Professionals 2 Survey with Industry Professionals 3 RQ1: Real-World use cases that necessitate output constraints 3 RQ1: Real-World use cases that necessitate output constraints 4 RQ2: Benefits of Applying Constraints to LLM Outputs and 4.1 Increasing Prompt-based development Efficiency 4 RQ2: Benefits of Applying Constraints to LLM Outputs and 4.1 Increasing Prompt-based development Efficiency 4.2 Integrating with Downstream Processes and Workflows 4.2 Integrating with Downstream Processes and Workflows 4.3 Satisfying UI and Product Requirements and 4.4 Improving User Experience, Trust, and Adoption 4.3 Satisfying UI and Product Requirements and 4.4 Improving User Experience, Trust, and Adoption 5 How to Articulate output constraints to LLMS and 5.1 The case for GUI: A Quick, Reliable, and Flexible Way of Prototyping Constraints 5 How to Articulate output constraints to LLMS and 5.1 The case for GUI: A Quick, Reliable, and Flexible Way of Prototyping Constraints 5.2 The Case for NL: More Intuitive and Expressive for Complex Constraints 5.2 The Case for NL: More Intuitive and Expressive for Complex Constraints 6 The Constraint maker Tool and 6.1 Iterative Design and User Feedback 6 The Constraint maker Tool and 6.1 Iterative Design and User Feedback 7 Conclusion and References 7 Conclusion and References A. The Survey Instrument A. The Survey Instrument ABSTRACT Large language models can produce creative and diverse responses. However, to integrate them into current developer workflows, it is essential to constrain their outputs to follow specific formats or standards. In this work, we surveyed 51 experienced industry professionals to understand the range of scenarios and motivations driving the need for output constraints from a user-centered perspective. We identified 134 concrete use cases for constraints at two levels: low-level, which ensures the output adhere to a structured format and an appropriate length, and high-level, which requires the output to follow semantic and stylistic guidelines without hallucination. Critically, applying output constraints could not only streamline the currently repetitive process of developing, testing, and integrating LLM prompts for developers, but also enhance the user experience of LLM-powered features and applications. We conclude with a discussion on user preferences and needs towards articulating intended constraints for LLMs, alongside an initial design for a constraint prototyping tool. 1 INTRODUCTION Over the past few years, we have witnessed the extraordinary capability of Large Language Models (LLMs) to generate responses that are not only creative and diverse but also highly adaptable to various user needs [5, 7, 18, 21, 22, 29, 31, 32]. For example, researchers can prompt ChatGPT [25] to condense long articles into concise summaries for fast digestion; while video game developers can generate detailed character profiles with rich personality traits, backstories, and unique abilities on demand, simply by dynamically prompting an LLM with the game context and players’ preferences. As much as end-users appreciate the unbounded creativity of LLMs, recent field studies examining the development of LLMpowered applications have repeatedly demonstrated the necessity to impose constraints on LLM outputs [10, 30]. For instance, a user might require a summary of an article to be “strictly less than 20 words” to meet length constraints, or a generated video game character profile to be “a valid JSON that can be parsed by Python” for a development pipeline. However, as evidenced by many recent NLP benchmarks and evaluations [16, 36, 42, 43], current state-of-the-art LLMs still lack the ability to guarantee that the generated output will invariably conform to user-defined constraints in the prompt (sometimes referred to as controllability). Although researchers have proposed various methods to improve controllability, such as supervised finetuning with specialized datasets [35] or controlled decoding strategies [1, 4, 24, 40], they tend to only focus on addressing a narrow range of constraints without taking into consideration the diverse usage scenarios and rationale that real-world developers and end-users encounter when prototyping and building practical LLM-powered functionalities and applications [12–14, 23]. In this work, we took the first step to systematically investigate the scenarios and motivations for applying output constraints from a user-centered perspective. Specifically, we sought to understand: • RQ1: What real-world use cases would necessitate or benefit from being able to constrain LLM outputs? RQ1 • RQ2: What are the benefits and impacts of being able to apply constraints to LLM outputs? RQ2 • RQ3: How would users like to articulate their intended constraints to LLMs? RQ3 We investigated these research questions by distributing a survey to a broad population of industry professionals (software engineers, researchers, designers, project managers, etc.) who have experience building LLM-powered applications. Our analysis identified six primary categories of output constraints that users desire, each supported by detailed usage scenarios and illustrative examples, summarized in Table 1. In a nutshell, users not only need low-level constraints, which mandate the output to conform to a structured format and an appropriate length, but also desire high-level constraints, which involve semantic and stylistic guidelines that users would like the model output to adhere to without hallucinating. Notably, developers often have to write complex code to handle ill-formed LLM outputs, a chore that could be simplified or eliminated if LLMs could strictly follow output constraints. In addition, the ability to apply constraints could help ease the integration of LLMs with existing pipelines, meet UI and product specifications, and enhance user trust and experience with LLM-powered features. Moreover, we discovered that describing constraints in natural language (NL) within prompts is not always the preferred method of control for LLM users. Instead, they seek alternative mechanisms, such as using graphical user interfaces, or GUIs, to define and test constraints, which could offer greater flexibility and a heightened sense of assurance that the constraints will be strictly followed. Informed by these results, we present an early design of ConstraintMaker, a prototype tool that enables LLM users to experiment, test, and apply constraints on the format of LLM outputs (see Figure 2 for more details), along with feedback and insights from preliminary user tests. Overall, this paper contributes: • a comprehensive taxonomy summarizing both low-level and high-level output constraints desired by LLM users (Table 1), derived from 134 real-world use cases reported by our survey respondents (RQ1), • an overview of both developer and user-facing benefits of being able to impose constraints on LLM outputs (RQ2), • an exploration of LLM users’ preferences for expressing constraints, whether via GUIs or natural language (RQ3), • a initial design of the tool ConstraintMaker, which enables users to visually prototype LLM output constraints, accompanied by a discussion of preliminary user feedback. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv Authors:
(1) Michael Xieyang Liu, Google Research, Pittsburgh, PA, USA (lxieyang@google.com);
(2) Frederick Liu, Google Research, Seattle, Washington, USA (frederickliu@google.com);
(3) Alexander J. Fiannaca, Google Research, Seattle, Washington, USA (afiannaca@google.com);
(4) Terry Koo, Google, Indiana, USA (terrykoo@google.com);
(5) Lucas Dixon, Google Research, Paris, France (ldixon@google.com);
(6) Michael Terry, Google Research, Cambridge, Massachusetts, USA (michaelterry@google.com);
(7) Carrie J. Cai, Google Research, Mountain View, California, USA (cjcai@google.com). Authors: Authors: (1) Michael Xieyang Liu, Google Research, Pittsburgh, PA, USA (lxieyang@google.com); (2) Frederick Liu, Google Research, Seattle, Washington, USA (frederickliu@google.com); (3) Alexander J. Fiannaca, Google Research, Seattle, Washington, USA (afiannaca@google.com); (4) Terry Koo, Google, Indiana, USA (terrykoo@google.com); (5) Lucas Dixon, Google Research, Paris, France (ldixon@google.com); (6) Michael Terry, Google Research, Cambridge, Massachusetts, USA (michaelterry@google.com); (7) Carrie J. Cai, Google Research, Mountain View, California, USA (cjcai@google.com).

"We Need Structured Output": Towards User-Centered Constraints on Large Language Model Output

“Fascism: a far-right, authoritarian, and ultranationalist political ideology and movement, characterized by a dictatorial leader, centralized autocracy, militarism, forcible suppression of opposition, belief in a natural social hierarchy, subordination of individual interests for the perceived good of the nation or race, and strong regimentation of society and the economy.” [Wikipedia] “Fascism: a far-right, authoritarian, and ultranationalist political ideology and movement, characterized by a dictatorial leader, centralized autocracy, militarism, forcible suppression of opposition, belief in a natural social hierarchy, subordination of individual interests for the perceived good of the nation or race, and strong regimentation of society and the economy.” [Wikipedia] Fascism: “Tyranny: cruel and oppressive government or rule. ‘Refugees fleeing tyranny and oppression’.” [Oxford Dictionaries] “Tyranny: cruel and oppressive government or rule. ‘Refugees fleeing tyranny and oppression’.” [Oxford Dictionaries] Tyranny At this point in history, we only have to look around to discover the embodiment of these concepts in more than a few places, in all its disgusting splendor. In the middle of it, software, all kinds of software, really, has been used for evil, but also for good. We can fight back (with something more than ranting on social media), and it’s time to pick our team and resources. Those resources, of course, can include software specially designed to preserve our rights. more than ranting on social media more than ranting on social media We’ll explore here some open-source and free software tools available for anyone to use against fascist and tyrannical governments all around the world. They work to protect your identity, your files, your communications, your connections, and more, so you can avoid censorship, surveillance, and oppression online. You’d be able to organize your community to defend yourselves and engage in activism if you wish. We’ll explore here some open-source and free software tools available for anyone to use against fascist and tyrannical governments all around the world. In the end, if you find them useful, you can support them with another useful software tool: Kivach. This open platform lets any open-source project on GitHub receive decentralized cryptocurrencies for funding, without major setups or requirements. Let’s start! Kivach Kivach Invisible Internet Project (I2P) Originally created by Lance James in 2002, the Invisible Internet Project (I2P) has grown in the hands of a whole team of dedicated open-source developers. Its primary goal is to create an anonymous network layer that enables censorship-resistant, peer-to-peer (P2P) communication. By encrypting user traffic and routing it through a global network of volunteer-run computers, I2P ensures that both the sender’s and receiver’s identities remain concealed, promoting secure and private interactions online. Lance James Lance James By encrypting user traffic and routing it through a global network of volunteer-run computers, I2P ensures that both the sender’s and receiver’s identities remain concealed, promoting secure and private interactions online. I2P offers a range of features designed to enhance user privacy. It supports various applications, including anonymous web hosting (known as “eepsites”), email, instant messaging, and file sharing. The network employs end-to-end encryption and utilizes unidirectional tunnels (encrypted pathways) for data transmission, making it challenging for third parties to trace communications. Additionally, I2P uses a distributed network database based on the Kademlia algorithm, eliminating central points of failure and enhancing the network’s resilience. I2P I2P Unlike similar projects, like Tor, I2P doesn’t have a proper organizational structure to receive donations, but that doesn’t mean they don’t appreciate support from its community. Indeed, they entirely rely on global volunteers and urge contributors to fund secondary applications or hire others to work on I2P. You can support their developers on GitHub, directly donating to them via Kivach. donating to them via Kivach donating to them via Kivach Briar This is a private messaging application introduced in 2018 by Michael Rogers. It’s tailored for activists, journalists, and individuals seeking a secure and dependable communication method. Unlike conventional messaging platforms that depend on central servers, Briar facilitates direct device-to-device synchronization, ensuring that conversations remain confidential and resistant to censorship. Unlike conventional messaging platforms that depend on central servers, Briar facilitates direct device-to-device synchronization, ensuring that conversations remain confidential and resistant to censorship. Briar Briar One of Briar’s standout features is its adaptability in various connectivity scenarios. When Internet access is available, it utilizes the Tor network to safeguard users’ identities and communications. In situations where the Internet is inaccessible, Briar can synchronize messages via Bluetooth or Wi-Fi, ensuring uninterrupted information flow during crises. All communications are protected with end-to-end encryption, and data is securely stored on users’ devices, eliminating reliance on cloud services. The development of this software has been supported by various organizations dedicated to Internet freedom and open-source initiatives. Notable contributors include the Small Media Foundation, the Open Internet Tools Project, the Open Technology Fund, the Next Generation Internet program, and the ISC Project. They also accept donations in cryptocurrency, and you can send them some coins via Kivach. send them some coins via Kivach send them some coins via Kivach SecureDrop At some point between 2010 and 2011, Aaron Swartz, a known hacktivist, decided to download a large number of academic papers from the digital library JSTOR using MIT’s network, allegedly without permission. He intended to make them freely available but was charged with computer fraud and unauthorized access, even threatened with 35 years in prison for it. He refused a plea bargain (admission of being guilty) for a sentence of six months and took his own life in January 2013. That’s the American justice system in action. Aaron Swartz Aaron Swartz Before his tragic death, along with Kevin Poulsen and James Dolan, he was developing DeadDrop: a free software designed for secure communications between journalists and their most vulnerable sources (whistleblowers). Swartz’s legacy was taken over by The Freedom of the Press Foundation, renamed to SecureDrop, and released in October 2013. Before his tragic death, along with Kevin Poulsen and James Dolan, he was developing DeadDrop: a free software designed for secure communications between journalists and their most vulnerable sources (whistleblowers). SecureDrop SecureDrop Now, it’s been used by very prominent media, including The Guardian, Forbes, The Washington Post, Associated Press, The New York Times, and Bloomberg. Anyone, everywhere, can use SecureDrop to share confidential, valuable information and evidence with the journalists of any of these newspapers, without fear of being fired, arrested, or even killed for it. This software uses Tor for anonymity, encrypts submissions, lessens metadata, and ensures no direct contact between sources and reporters, minimizing surveillance risks and protecting confidentiality. Its team accepts contributions in the form of code and translations and also accepts donations with credit cards and PayPal through the foundation. To send them cryptocurrencies, you can find them on Kivach as freedomofpress/securedrop. freedomofpress/securedrop freedomofpress/securedrop InterPlanetary File System (IPFS) IPFS, short for InterPlanetary File System, is a decentralized protocol designed to store and share files across a distributed network. It was created by Juan Benet and launched in February 2015. Unlike traditional web protocols that rely on centralized servers, IPFS uses a P2P system where data is identified by its content rather than its location. This approach makes file storage and retrieval more efficient, resilient, and resistant to failures or censorship. Unlike traditional web protocols that rely on centralized servers, IPFS uses a P2P system where data is identified by its content rather than its location. At its core, IPFS operates using content-addressing and a distributed hash table (DHT). Each file is assigned a unique cryptographic hash, ensuring that data remains unchanged and verifiable. When a file is requested, nodes in the network search for its hash rather than a specific server, enabling efficient distribution similar to BitTorrent. Being open-source, the project has evolved into numerous implementations and has enabled the creation of independent applications as well. IPFS IPFS So far, IPFS has played a key role in bypassing online censorship, including hosting a mirror of Wikipedia during Turkey's ban and preserving the Tornado Cash website after U.S. sanctions. The ecosystem is supported by Protocol Labs, which funds core development, while independent teams contribute to alternative implementations. Filecoin, a blockchain-based storage network, also incentivizes IPFS adoption by rewarding users for hosting data. Beyond them, a wide variety of IPFS implementations and apps can be found on GitHub, and you can support their developers via Kivach. You’ll just need to paste their repositories on the donation website. So far, IPFS has played a key role in bypassing online censorship, including hosting a mirror of Wikipedia during Turkey's ban and preserving the Tornado Cash website after U.S. sanctions. during Turkey's ban during Turkey's ban Tornado Cash website Tornado Cash website a wide variety a wide variety support their developers via Kivach support their developers via Kivach OpenSnitch OpenSnitch is a firewall application specifically designed for Linux systems, developed as an open-source alternative to macOS’s Little Snitch. It was created around 2017 by Simone Margaritelli (evilsocket) and is currently maintained by Gustavo Iñiguez Goya. Its main purpose is to monitor and control Internet access at the application level, allowing users to decide which programs can connect online. This is particularly useful for detecting software that might be transmitting data without the user’s knowledge. The software works by intercepting outgoing connections from installed applications and prompting users to approve or deny them. OpenSnitch offers fine-grained control by letting users create specific rules, such as blocking a single request, an entire domain, or all connections from a particular app. It stores rules in easily editable JSON files and features a graphical interface for real-time monitoring. The firewall also supports SIEM (Security Information and Event Management) tools, making it suitable for both personal and enterprise security. OpenSnitch offers fine-grained control by letting users create specific rules, such as blocking a single request, an entire domain, or all connections from a particular app. OpenSnitch OpenSnitch OpenSnitch can be a powerful tool against government surveillance or spyware, as it allows users to identify and block unauthorized data transmissions. It offers a layer of protection by ensuring that no application sends data without explicit permission. As free software, it’s primarily funded through community donations, which can be made via the project’s GitHub page. Users can also contribute by donating crypto via Kivach. donating crypto via Kivach donating crypto via Kivach Special Mentions Freedom and privacy tools are many, and we’ve already mentioned several of them in previous episodes of our Kivach series. These open-source, free software projects can also work to fight against fascism and tyranny: previous episodes previous episodes Tor – Anonymizes internet traffic by routing it through multiple servers.
Whonix – A privacy-focused OS that routes all traffic through Tor.
Signal – A secure messaging app with end-to-end encryption.
GrapheneOS – A privacy-focused, hardened Android OS.
Linux Kodachi – A secure Linux distribution with built-in anonymity tools.
Berty – A decentralized, encrypted messaging app.
Agorakit – A self-hosted collaboration tool for activists (communications and events).
QubesOS – A security-focused OS that isolates apps in virtual machines.
WireGuard – A fast, modern Virtual Privacy Network (VPN) protocol.
Cryptomator – Encrypts cloud storage files for privacy. Tor – Anonymizes internet traffic by routing it through multiple servers. Tor Whonix – A privacy-focused OS that routes all traffic through Tor. Whonix Signal – A secure messaging app with end-to-end encryption. Signal GrapheneOS – A privacy-focused, hardened Android OS. GrapheneOS Linux Kodachi – A secure Linux distribution with built-in anonymity tools. Linux Kodachi Berty – A decentralized, encrypted messaging app. Berty Agorakit – A self-hosted collaboration tool for activists (communications and events). Agorakit QubesOS – A security-focused OS that isolates apps in virtual machines. QubesOS WireGuard – A fast, modern Virtual Privacy Network (VPN) protocol. WireGuard Cryptomator – Encrypts cloud storage files for privacy. Cryptomator All of them, of course, are available on GitHub and Kivach as well, if you want to support their developers. Support Freedom With Kivach! Kivach makes crypto donations to open-source projects easy. Just set up an Obyte wallet to manage tokens like USDC, ETH, BNB, or GBYTE, then find the GitHub project you want to support, go to the platform, paste the repo, and donate. The funds stay secure in an Obyte’s Autonomous Agent (AA) until the recipient claims them with their own Obyte wallet —which they'll need only after receiving the donation. You’ll have to tell them about it! Kivach Kivach Obyte wallet Obyte wallet The funds stay secure in an Obyte’s Autonomous Agent (AA) until the recipient claims them with their own Obyte wallet This open platform empowers developers and upholds freedom. By channeling contributions to innovative projects, Kivach helps counter censorship, fascism, and tyranny. Every donation builds a stronger, democratic digital space, supporting resistance against oppression while fostering an environment of open collaboration and positive change. And remember that the whole Obyte ecosystem is also there as another tool for freedom, fully decentralized and available for everyone. Obyte ecosystem Obyte ecosystem Check out some previous lists to discover other useful software tools: 5 Open-Source Projects You Can Donate to via Kivach, Episode IV: Privacy Tools
5 Open-Source Tools You Can Donate to via Kivach, Episode VI: Decentralized Services
5 Cybersecurity Tools to Use for Free and Donate to Via Kivach
5 Open-Source Software for Global Teams To Donate via Kivach
5 Open-source and Free Operating Systems to Donate via Kivach
Ditch the Phone Line and Try These 5 Free and Open-Source Platforms for Voice Calls Instead 5 Open-Source Projects You Can Donate to via Kivach, Episode IV: Privacy Tools 5 Open-Source Projects You Can Donate to via Kivach, Episode IV: Privacy Tools 5 Open-Source Projects You Can Donate to via Kivach, Episode IV: Privacy Tools 5 Open-Source Projects You Can Donate to via Kivach, Episode IV: Privacy Tools 5 Open-Source Tools You Can Donate to via Kivach, Episode VI: Decentralized Services 5 Open-Source Tools You Can Donate to via Kivach, Episode VI: Decentralized Services 5 Open-Source Tools You Can Donate to via Kivach, Episode VI: Decentralized Services 5 Open-Source Tools You Can Donate to via Kivach, Episode VI: Decentralized Services 5 Cybersecurity Tools to Use for Free and Donate to Via Kivach 5 Cybersecurity Tools to Use for Free and Donate to Via Kivach 5 Cybersecurity Tools to Use for Free and Donate to Via Kivach 5 Cybersecurity Tools to Use for Free and Donate to Via Kivach 5 Open-Source Software for Global Teams To Donate via Kivach 5 Open-Source Software for Global Teams To Donate via Kivach 5 Open-Source Software for Global Teams To Donate via Kivach 5 Open-Source Software for Global Teams To Donate via Kivach 5 Open-source and Free Operating Systems to Donate via Kivach 5 Open-source and Free Operating Systems to Donate via Kivach 5 Open-source and Free Operating Systems to Donate via Kivach 5 Open-source and Free Operating Systems to Donate via Kivach Ditch the Phone Line and Try These 5 Free and Open-Source Platforms for Voice Calls Instead Ditch the Phone Line and Try These 5 Free and Open-Source Platforms for Voice Calls Instead Ditch the Phone Line and Try These 5 Free and Open-Source Platforms for Voice Calls Instead Ditch the Phone Line and Try These 5 Free and Open-Source Platforms for Voice Calls Instead Featured Vector Image by Freepik Featured Vector Image by Freepik Freepik Freepik

Stories, News & Mentions about GRAPHIKA

Categories

Trending Topics

Stories, News & Mentions about GRAPHIKA

Graphika’s Stories on HackerNoon

Categories

Trending Topics