EVCS is leading the automotive electrification revolution with one of the fastest growing networks of level 2 and DC Fast charging points in California, Oregon and Washington

EVCS

ZKP Meets ML in Verifiability: Existing Scheme & Experimental Analysis

This example project has everything you need to localize your native Android application on platform, using a fully automated approach. Smartcat continuous localization See >>> bootstrap project at github What we offer here is a pre-configured Docker container with our command-line localization automation tool, , which will pull the changes from your code repository, parse the source file, send translation data over to Smartcat, get translations back, integrate them into its local database, generate localized files and push them back to your repo, all in a single sweep. Serge (String Extraction and Resource Generation Engine) strings.xml This approach to localization: Doesn't require writing custom low-level integration code: you write declarative configuration files instead Doesn't require you to change your CI/CD build process: translations are integrated statically into your source code repository Is secure: you don't give Smartcat direct access to your source code repositories, and you control all the synchronization Is vendor lock-in free: you own your translation data, and can generate localized resource files completely offline Is flexible: you can manually run it on your computer, or run it on a schedule on a dedicated host Is powerful: you can implement localization workflows that otherwise would require many months of development (see below). Features With our continuous localization solution, you have an unprecedented flexibility and power. Some ideas that might inspire you are: Automatic discovery and localization of multiple product branches Ability to prohibit localized file updates unless they are 100% translated (handy for marketing materials!) Ability to specify target languages right in each file, to enable automated self-service scenarios Pseudo-localization for easier internationalization (i18n) QA Conditional exclusion of certain strings by mask Ability to auto-generate comments and preview links for each string Ability to group multiple repositories and different file types under a same logical project Ability to preprocess source files for greater flexibility Ability to post-process localized files so that the final result is CI/CD-ready Ability to email developers if source files are broken ... and much more! Installation 1. for your Windows, Mac, or Linux. Install Docker 2. Run this command to install Serge for the first time, or to update it to a latest version: $ docker pull smartcatcom/serge 3. Clone this repository into any directory: $ git clone https: //github.com/smartcatai/smartcat-serge-bootstrap-android-native.git 4. Create an account at Smartcat. 5. Create a new project in Smartcat to synchronize your data with, with as a source and and as target languages (don't worry, you will be able to change the settings later). English German Russian Getting Started Running Serge in interactive mode In the root directory of this repo we have provided you a convenience script to run a Serge shell, from which you will have access to command. Run it as follows (both under Windows, Mac or Linux): serge $ ./serge-shell When you enter your shell, the root directory of your bootstrap project will be mounted as / and you will see the following prompt: data [Serge Shell] /data $ Here and below, it is assumed that you're running the commands inside this shell. At the same time, you can continue editing the configuration files or browse the directory outside of the shell in Explorer, Finder, or any other favorite file management tool and code editor. Note: Now you that you are inside this interactive shell, you can run Serge (the command below will just show some short usage synopsis and a list of available commands): $ serge To exit the shell later, use the command: exit $ exit Getting around This repository contains a sample project, . First things to do are: android-native See the configuration file: configs/android-native.serge Look at the contents of the source project folder: vcs/android-native This sample project is set up to have (English) as a source language and (German) and (Russian) as target languages, and will process all keys in the resource file in directory. en de ru strings.xml vcs/android-native/src/main/res/values Initial run Go to the directory and run the localization step: configs $ cd configs
$ serge localize This will create localized files under and directories. These files will have English content, since translations have not been provided just yet. vcs/android-native/de vcs/android-native/ru The same localization step will generate translation files under and directories. You can examine the generated files and also see their initial state with no translations. ts/android-native/de ts/android-native/ru .po Doing translations locally Edit e.g. file and provide a translation for a single string (for testing purposes, any random "translation" will work). ts/android-native/de/strings.xml.po Run once again from the directory. If you now open the localized resource file, e.g. , you will see that your translation has been integrated into the XML file. serge localize configs vcs/android-native/src/main/res/values-de/strings.xml Connecting with Smartcat Create a new project in Smartcat with as a source language and and as target ones. Languages must be registered on the project before you can upload files targeting those languages. You can later change the list of target languages at any time. English German Russian Edit the file and fill out your account parameters and credentials under config section: , , and . Instructions next to each parameter will help you understand where to get the values from. Once defined, these settings will be shared across your future localization projects. configs/common.serge.inc common-settings → ts base_url account_id token Edit the file and specify the , following the instructions in the configuration file. configs/android-native.serge project_id Now you can push your translation files to Smartcat: $ serge push-ts Go to Smartcat, open the test project you created, and there you will see your and files available for translation, one for German, and another one for Russian. strings.xml_de strings.xml_ru Open e.g. and edit the translation for the string that you translated previously. strings.xml_de Now you can pull translations from Smartcat and run a localization cycle at once: $ serge pull-ts localize If you now open the localized resource file, , you will see that the new translation has been integrated into the XML file. vcs/android-native/src/main/res/values-de/strings.xml Running the localization continuously This project has two convenience scripts, and that you can use as a starting point for your continuous localization process. sync-once.sh sync-loop.sh Run to run the sync/localization cycle just once. ./sync-once.sh Run to run the sync/localization cycle every 5 minutes. You can edit the file and put any delay you want. ./sync-loop.sh While the localization is running in background (eventually you will want to run it on a dedicated server), you can modify your example project (add, change or delete keys in your source XML file) and provide translations on Smartcat side at the same time, and Serge will merge in all these changes, gathering the new translations, updating the localized resources, and pushing new strings for translation. Adding VCS synchronization The last missing step is to make Serge not only generate the localized files locally, but also pull the sources from your actual source code repository, and push localized files back to the remote repo. Since Git is the most popular version control system, this sample project includes a stub for connecting to a Git repo. Edit the file so that parameter points to the actual remote repository URL. Now initialize the data from the remote repo (this will remove the previous contents of folder with the contents of your repository): configs/android-native.serge → sync → vcs → data → remote_path vcs/android-native $ serge pull --initialize android-native.serge The parameter in this command is needed only once, when you want to delete and re-populate the contents of the project folder. --initialize Now your project has a different directory structure, and maybe different files (not JSON), so you need to edit your file accordingly (see , , parameters, and the section). At this point we recommend you to spend some time reading the for the list of available parsers and their parameters, and the . configs/android-native.serge → jobs → (first job section) → source_dir source_match output_file_path parser Serge documentation configuration file syntax and reference Once you have your job parameters tweaked, it's time to test this by running a localization cycle once again, and also cleaning up the outdated translation files: $ serge localize
$ serge clean-ts If you see that a set of localized files was properly created in your folder, and that translation files in folder also look good, you can push the localized files back to the remote repository: vcs/android-native ts/android-native $ serge push The last and final step is to enable the full synchronization cycle. Edit the script by disabling the line and enabling the line. The latter command is equivalent to . In human language, this means that running will do everything a continuous localization is expected to do: pull the new contents from both your Git server and Smartcat, update localized resources and translation files, and push changes back to Git and Smartcat, respectively. sync-once.sh serge pull-ts localize push-ts serge sync serge pull pull-ts localize push-ts push serge sync Finally, run: ./sync-loop.sh Congratulations! You now live in the world of smooth continuous localization . (press Ctrl+C anytime to get back to reality) Running Serge in a non-interactive mode When you don't need an interactive shell (especially when you're running Serge on a dedicated server), you can use a provided wrapper, as follows: run-in-docker $ ./run- -docker serge localize configs/android-native.serge in or: $ ./run- -docker ./sync-loop.sh in Adding new projects In the paradigm of continuous localization, your Smartcat projects will be permanent; content in these projects will be constantly updated, reflecting the changes in your repositories and external content sources. You can create one project for your iOS application, another one for Android, for your website, documentation, and so on. To set up a new project (besides creating one in Smartcat), you will need to make a copy of to (of course, file and project names are up to you), and edit a new configuration file accordingly. configs/android-native.serge configs/project-b.serge Getting professional support At Smartcat, we strive to build delightful localization processes that work seamlessly and don't slow you down. We offer managed solutions and localization engineering services , so you don't have to figure this out all by yourself! as a part of the subscription Previously published at https://github.com/smartcatai/smartcat-serge-bootstrap-android-native

How To Localize Your Android App: A Step by Step Guide

VCs, institutional investors and family offices descend on South Beach for the weeklong rave with Christo, Calvin Harris, Kaskade, Die Antwoord and Zedd If all of your crypto friends are racing down to Florida next week, it’s because world’s hippest art show, , is bringing together 77,000 fans, celebrities, collectors, institutional investors, family offices and venture capitalists for a week of epic partying and talks on decentralized art. Art Basel Miami There will be 4,000 artists from 268 galleries showcasing their works but the real story will unfold afterhours in popup nightclubs, elevators of the ultra chic hotels: , , , , , , , , , , , , and maybe, just maybe, Jeremy Gardner’s Crypto Castle Miami. Soho Beach House W South Beach Delano Miami Beach Edition Fontainebleau 1 Hotel Faena The Plymouth Freehand Sagamore SLS Brickell JW Marriott Marquis This year headliners announced include Calvin Harris, Kaskade, Die Antwoord, Action Bronson and Zedd! Last year, All Day I Dream/Lee Burridge, Bob Moses, Bjork, Chromeo, Claude von Stroke, Cut Copy, Diplo, Doug E Fresh & The Sugarhill Gang, Major Lazer, Markus Schulz, Paris Hilton, Rick Ross, Wu-Tang Clan, 2 Chainz, and 50 Cent performed. The Art Basel epicenter takes place December 5 through December 9 at the Miami Convention Center with satellite shows sprawling beyond which include Art Miami, Aqua Art Miami, CONTEXT Art Miami, The Directed Art Modern, INK Miami Art Fair, Faena Festival, Miami River Art Fair, NADA , PINTA, PRIZM Art Fair, PULSE, Red Dot Miami, SCOPE, Spectrum, Superfine! and UNTITLED. To get a sense of what Art Basel is like, check out by Florum Fashion. this 2017 pictorial review I’ll be on the scene speaking at the SCOPE Art Show on and tweeting all of the shenanigans . When Deals Turn Digital: New Technologies, Banking, and the Future of the Art Market @contentnow Here’s what’s on for crypto: Bass Museum, 2100 Collins, 33139 WHEN: Wednesday, December 5, 6–8pm/8–10pmWHAT: Bass Museum Opening Party featuring The Haas BrothersWHERE: HOW: https://thebass.org/event/member-reception-haas-brothers/ Soho Studios, 2136 NW 1st Avenue, Miami, FL, 33127 WHEN: Wednesday, December 5, 10pmWHAT: Die AntwoordWHERE: HOW: https://www.seetickets.us/event/DIE-ANTWOORD-at-Soho-Studios/373623 Lemon City Studios, 261 NE 73rd, 33138 WHEN: Wednesday, December 5, 11pmWHAT: ARIA with Guy Laliberte, Alex Neri: Maria Callas TributeWHERE: HOW: https://www.residentadvisor.net/events/1189938 disrupting a $64 billion marketplace. Featuring visual artist, .2–3pm Tokenizing Art with Forbes’ Rachel Wolfson and artists Jesselyn Contreras, Johnny Dollar, Carrie Eldridge and Rob Charles3:15–3:45pm Investors Panel with TheStreet’s Jordan French, Ausum Ventures’ Jeremy Gardner, Bluesky Capital’s Andrea Lecesse, Lex Capital’s Alex Mendoza, Kryptal Group’s Edna Yuan, Park Capital Partners’ Alexander Lopatine, Level 7’s Michael Blin, Crowdsmart.io VC’s Richard Swart. Nikki Beach, 1 Ocean Drive, Miami Beach, 33139 WHEN: Thursday, December 6, 9–6pm WHAT: Art Decentralized, The World of Distributed Masterpieces Vesa WHERE: HOW: https://www.eventbrite.com/e/art-decentralized-the-world-of-distributed-masterpieces-blockchain-art-registration-51810706217 Soho Studios, 2136 NW 1st Avenue, Miami, FL, 33127 WHEN: Thursday, December 6, 10pmWHAT: Action BronsonWHERE: HOW: https://www.seetickets.us/event/ACTION-BRONSON-at-Soho-Studios/373655 LIV Nightclub at Fontainebleau, 4441 Collins Avenue WHEN: Thursday, December 6, 11pm-5am WHAT: Zedd Basel Edition at LIV WHERE: HOW: https://www.tixr.com/groups/liv/events/zedd-basel-edition-11166 E11EVEN, 29 NE 11th Street WHEN: Thursday, December 6, 11pm-8amWHAT: G-Eazy Basel Edition at E11EVEN WHERE: HOW: https://www.tixr.com/groups/11miami/events/art-basel-g-eazy-11073 with Kiki Del Valle, SVP Digital Future, Mastercard Katya Fisher, Tech and Corporate Attorney, Fisher Catalotti Vanessa Grellet, Executive Director, Consensys Elena Zhavelev, Forbes contributor and Founder, New Art Academy, Martine Paris, Tech Reporter, The FinTech Times MasterCard Tent, 801 Ocean Drive, Miami Beach, 33139 WHEN: Friday, December 7, 10–12pm WHAT: SCOPE Art Show presented by MasterCard, When Deals Turn Digital: New Technologies, Banking, and the Future of the Art Market WHERE: HOW: https://scope-art.com/show/miami-beach-2018/programming , Christo best known for massive environmental installations like the Umbrellas in California and Japan, the Wrapped Reichstag in Berlin and the Floating Piers in Lake Iseo, Italy will talk about Miami Beach Convention Center, Auditorium, West Lobby, 1901 Convention Center Drive, Miami Beach 33139 FREE, open to the public, or watch live: , WHEN: Friday, December 7, 10–11:30am WHAT: Artist Talk - Christo, Tomás Saraceno, and Hans Ulrich Obrist WHERE: HOW: facebook.com/artbasel https://www.artbasel.com/events/detail/7854/Conversations-Artist-Talk-Artists-Influencers with Ruth Catlow, Furtherfield/DECL London, Matt Hall, Cryptopunks New York, Harm van den Dorpel, Artist, Berlin, and Orit Gat, Writer, New York, London — What are the social and political implications of decentralized financial networks? Is blockchain a reliable source for fundraising? How are artists using it in their artistic practice? Miami Beach Convention Center, Auditorium, West Lobby, 1901 Convention Center Drive, Miami Beach 33139 FREE, open to the public, or watch live: WHEN: Friday, December 7, 2:30–3:30pm WHAT: Art Market Talk — Which Art Problems Can Blockchain Solve? WHERE: HOW: facebook.com/artbasel **WHEN: Friday, December 7, 4:30–5pmWHAT: Brian Kelly, CNBC’s Fast Money Keynote at CryptoProExpo**This conference provides a market update and fundamentals, how to set up a crypto fund, how to identify blockchain investment opportunities including STOs, and the intersection of art, crypto and blockchain Fontainebleau WHERE: HOW: https://cryptoproexpo.com/miami-art-week-2018 Foxhole, 1218 14th Court, Miami Beach 33139 WHEN: Friday, December 7, 9pm–5amWHAT: DJ Jazzy Jeff WHERE: HOW: https://www.eventbrite.com/e/jazzy-jeff-at-foxhole-bar-art-basel-2018-tickets-52617016914 Basement at Miami Beach EDITION Hotel, 2901 Collins Avenue In List App WHEN: Friday, December 7, 10pm–5amWHAT: Saint Vie + Coss + Mateo WHERE: HOW: Historic Virginia Key Beach WHEN: Saturday, December 8, 3pm-7amWHAT: Rakastella feat Dixon + DJ Tennis WHERE: HOW: https://www.eventbrite.com/e/rakastella-2018-by-innervisions-life-and-death-tickets-46128112426 Nikki Beach, 1 Ocean Drive WHEN: Saturday, December 8, 8pmWHAT: Airthereum party WHERE: HOW: https://www.eventbrite.com/o/fly-by-airthereum-18269671711 Soho Studios, 2136 NW 1st Avenue, Miami, FL, 33127 WHEN: Saturday, December 8, 10pm-3amWHAT: KaskadeWHERE: HOW: https://wl.seetickets.us/event/2018-BLNK-CNVS-Basel-Pass/373660 Racket, 150 NW 24th Street, Miami Beach, 33127 WHEN: Saturday, December 8, 9pm-3amWHAT: DJ Jazzy Jeff WHERE: HOW: https://www.eventbrite.com/e/jazzy-jeff-at-racket-art-basel-2018-tickets-52616606687 Basement at Miami Beach EDITION Hotel, 2901 Collins Avenue In List App WHEN: Saturday, December 8, 10pm–5amWHAT: Bedouin + Guy Laliberte WHERE: HOW: LIV Nightclub at Fontainebleau, 4441 Collins Avenue WHEN: Saturday, December 8, 11pm-5am WHAT: Calvin Harris Basel Edition at LIV WHERE: HOW: https://www.tixr.com/groups/liv/events/calvin-harris-basel-edition-11066

Art Basel Miami on the Blockchain

How to Build an e-Commerce App : Entrepreneur’s Guide

Authors: (1) Maria Rigaki, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and maria.rigaki@fel.cvut.cz; (2) Sebastian Garcia, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and sebastian.garcia@agents.fel.cvut.cz. Table of Links Abstract & Introduction Threat Model Background and Related Work Methodology Experiments Setup Results Discussion Conclusion, Acknowledgments, and References Appendix 8 Conclusions By employing model-based reinforcement learning, MEME generates adversarial malware samples that successfully evade antivirus systems and train a surrogate model mimicking the target classifier accurately. Our experiments show that MEME surpasses existing methods in evasion rate, suggesting its potential for various applications, such as testing model robustness and enhancing cyber security against advanced persistent threats. Future work may involve exploring ensemble surrogates and other optimizations to enhance MEME’s performance further. Acknowledgments The authors acknowledge support from the Strategic Support for the Development of Security Research in the Czech Republic 2019–2025 (IMPAKT 1) program, by the Ministry of the Interior of the Czech Republic under No. VJ02010020 – AI-Dojo: Multi-agent testbed for the research and testing of AI-driven cyber security technologies. The authors acknowledge the support of NVIDIA Corporation with the donation of a Titan V GPU used for this research. References 1. Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: A Next-generation Hyperparameter Optimization Framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. pp. 2623–2631. KDD ’19, Association for Computing Machinery, New York, NY, USA (Jul 2019). https://doi.org/10.1145/3292500.3330701 2. Anderson, H.S., Kharkar, A., Filar, B., Evans, D., Roth, P.: Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning (Jan 2018). https://doi.org/10.48550/arXiv.1801.08917, arXiv:1801.08917 [cs] 3. Anderson, H.S., Roth, P.: EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models (Apr 2018). https://doi.org/10.48550/arXiv.1804.04637, arXiv:1804.04637 [cs] 4. Bergstra, J., Bardenet, R., Bengio, Y., K´egl, B.: Algorithms for Hyper-Parameter Optimization. In: Advances in Neural Information Processing Systems. vol. 24. Curran Associates, Inc. (2011) 5. Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: OpenAI Gym (Jun 2016). https://doi.org/10.48550/arXiv.1606.01540, arXiv:1606.01540 [cs] 6. Ceschin, F., Botacin, M., Gomes, H.M., Oliveira, L.S., Gr´egio, A.: Shallow Security: on the Creation of Adversarial Variants to Evade Machine Learning-Based Malware Detectors. In: Proceedings of the 3rd Reversing and Offensive-oriented Trends Symposium. pp. 1–9. ROOTS’19, Association for Computing Machinery, New York, NY, USA (Feb 2020). https://doi.org/10.1145/3375894.3375898 7. Chandrasekaran, V., Chaudhuri, K., Giacomelli, I., Jha, S., Yan, S.: Exploring connections between active learning and model extraction. In: Proceedings of the 29th USENIX Conference on Security Symposium. pp. 1309–1326. SEC’20, USENIX Association, USA (Aug 2020) 8. Correia-Silva, J.R., Berriel, R.F., Badue, C., de Souza, A.F., Oliveira-Santos, T.: Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Data. In: 2018 International Joint Conference on Neural Networks (IJCNN). pp. 1–8 (Jul 2018). https://doi.org/10.1109/IJCNN.2018.8489592, iSSN: 2161-4407 9. Demetrio, L., Biggio, B., Lagorio, G., Roli, F., Armando, A.: FunctionalityPreserving Black-Box Optimization of Adversarial Windows Malware. IEEE Transactions on Information Forensics and Security 16, 3469–3478 (2021). https://doi.org/10.1109/TIFS.2021.3082330, conference Name: IEEE Transactions on Information Forensics and Security 10. Demetrio, L., Coull, S.E., Biggio, B., Lagorio, G., Armando, A., Roli, F.: Adversarial exemples: A survey and experimental evaluation of practical attacks on machine learning for windows malware detection. ACM Transactions on Privacy and Security (TOPS) 24(4), 1–31 (2021), publisher: ACM New York, NY, USA 11. Dowling, S., Schukat, M., Barrett, E.: Using Reinforcement Learning to Conceal Honeypot Functionality. In: Brefeld, U., Curry, E., Daly, E., MacNamee, B., Marascu, A., Pinelli, F., Berlingerio, M., Hurley, N. (eds.) Machine Learning and Knowledge Discovery in Databases. pp. 341–355. Lecture Notes in Computer Science, Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-10997-4 21 12. Fang, Y., Zeng, Y., Li, B., Liu, L., Zhang, L.: DeepDetectNet vs RLAttackNet: An adversarial method to improve deep learning-based static malware detection model. PLOS ONE 15(4), e0231626 (Apr 2020). https://doi.org/10.1371/journal.pone.0231626, publisher: Public Library of Science 13. Fang, Z., Wang, J., Geng, J., Kan, X.: Feature Selection for Malware Detection Based on Reinforcement Learning. IEEE Access 7, 176177–176187 (2019). https://doi.org/10.1109/ACCESS.2019.2957429, conference Name: IEEE Access 14. Fang, Z., Wang, J., Li, B., Wu, S., Zhou, Y., Huang, H.: Evading Anti-Malware Engines With Deep Reinforcement Learning. IEEE Access 7, 48867–48879 (2019). https://doi.org/10.1109/ACCESS.2019.2908033, conference Name: IEEE Access 15. Harang, R., Rudd, E.M.: SOREL-20M: A Large Scale Benchmark Dataset for Malicious PE Detection. arXiv:2012.07634 [cs] (Dec 2020), arXiv: 2012.07634 16. Hu, W., Tan, Y.: Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN. In: Tan, Y., Shi, Y. (eds.) Data Mining and Big Data. pp. 409–423. Communications in Computer and Information Science, Springer Nature, Singapore (2022). https://doi.org/10.1007/978-981-19-8991-9 29 17. Huang, L., Zhu, Q.: Adaptive Honeypot Engagement Through Reinforcement Learning of Semi Markov Decision Processes. In: Alpcan, T., Vorobeychik, Y., Baras, J.S., D´an, G. (eds.) Decision and Game Theory for Security. pp. 196– 216. Lecture Notes in Computer Science, Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-32430-8 13 18. Institute, A.T.: AV-ATLAS - Malware & PUA (2023), https://portal.av-atlas.org/malware 19. Jagielski, M., Carlini, N., Berthelot, D., Kurakin, A., Papernot, N.: High Accuracy and High Fidelity Extraction of Neural Networks. pp. 1345–1362 (2020) 20. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., Liu, T.Y.: LightGBM: A Highly Efficient Gradient Boosting Decision Tree. In: Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017) 21. Kurutach, T., Clavera, I., Duan, Y., Tamar, A., Abbeel, P.: Model-Ensemble TrustRegion Policy Optimization. In: International Conference on Learning Representations (2018) 22. Labaca-Castro, R., Franz, S., Rodosek, G.D.: AIMED-RL: Exploring Adversarial Malware Examples with Reinforcement Learning. In: Dong, Y., Kourtellis, N., Hammer, B., Lozano, J.A. (eds.) Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track. pp. 37–52. Lecture Notes in Computer Science, Springer International Publishing, Cham (2021). https://doi.org/10.1007/978-3-030-86514-6 3 23. Li, X., Li, Q.: An IRL-based malware adversarial generation method to evade anti-malware engines. Computers & Security 104, 102118 (May 2021). https://doi.org/10.1016/j.cose.2020.102118 24. Ling, X., Wu, L., Zhang, J., Qu, Z., Deng, W., Chen, X., Qian, Y., Wu, C., Ji, S., Luo, T., Wu, J., Wu, Y.: Adversarial attacks against Windows PE malware detection: A survey of the state-of-the-art. Computers & Security 128, 103134 (May 2023). https://doi.org/10.1016/j.cose.2023.103134 25. Lundberg, S.M., Lee, S.I.: A Unified Approach to Interpreting Model Predictions. In: Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017) 26. Nguyen, T.T., Reddi, V.J.: Deep Reinforcement Learning for Cyber Security. IEEE Transactions on Neural Networks and Learning Systems pp. 1–17 (2021). https://doi.org/10.1109/TNNLS.2021.3121870, conference Name: IEEE Transactions on Neural Networks and Learning Systems 27. Orekondy, T., Schiele, B., Fritz, M.: Knockoff Nets: Stealing Functionality of BlackBox Models. pp. 4954–4963 (2019) 28. Pal, S., Gupta, Y., Shukla, A., Kanade, A., Shevade, S., Ganapathy, V.: ActiveThief: Model Extraction Using Active Learning and Unannotated Public Data. Proceedings of the AAAI Conference on Artificial Intelligence 34(01), 865–872 (Apr 2020). https://doi.org/10.1609/aaai.v34i01.5432, number: 01 29. Phan, T.D., Duc Luong, T., Hoang Quoc An, N., Nguyen Huu, Q., Nghi, H.K., Pham, V.H.: Leveraging Reinforcement Learning and Generative Adversarial Networks to Craft Mutants of Windows Malware against Black-box Malware Detectors. In: Proceedings of the 11th International Symposium on Information and Communication Technology. pp. 31–38. SoICT ’22, Association for Computing Machinery, New York, NY, USA (Dec 2022) 30. Quertier, T., Marais, B., Morucci, S., Fournel, B.: MERLIN – Malware Evasion with Reinforcement LearnINg (Mar 2022), arXiv:2203.12980 [cs] 31. Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., Dormann, N.: StableBaselines3: Reliable Reinforcement Learning Implementations. Journal of Machine Learning Research 22(268), 1–8 (2021) 32. Rigaki, M., Garcia, S.: Stealing and evading malware classifiers and antivirus at low false positive conditions. Computers & Security 129, 103192 (Jun 2023). https://doi.org/10.1016/j.cose.2023.103192 33. Rosenberg, I., Meir, S., Berrebi, J., Gordon, I., Sicard, G., Omid David, E.: Generating End-to-End Adversarial Examples for Malware Classifiers Using Explainability. In: 2020 International Joint Conference on Neural Networks (IJCNN). pp. 1–10 (Jul 2020). https://doi.org/10.1109/IJCNN48605.2020.9207168, iSSN: 2161-4407 34. Sanyal, S., Addepalli, S., Babu, R.V.: Towards Data-Free Model Stealing in a Hard Label Setting. pp. 15284–15293 (2022) 35. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017) 36. Security.org, T.: 2023 Antivirus Market Annual Report (Feb 2023), https://www.security.org/antivirus/antivirus-consumer-report-annual/ 37. Severi, G., Meyer, J., Coull, S., Oprea, A.: Explanation-Guided Backdoor Poisoning Attacks Against Malware Classifiers. In: 30th USENIX Security Symposium (USENIX Security 21). pp. 1487–1504. USENIX Association (2021) 38. Song, W., Li, X., Afroz, S., Garg, D., Kuznetsov, D., Yin, H.: MAB-Malware: A Reinforcement Learning Framework for Blackbox Generation of Adversarial Malware. In: Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security. pp. 990–1003. ASIA CCS ’22, Association for Computing Machinery, New York, NY, USA (May 2022). https://doi.org/10.1145/3488932.3497768 39. Sussman, B.: New Malware Is Born Every Minute (May 2023), https://blogs.blackberry.com/en/2023/05/new-malware-born-every-minute 40. Sutton, R.S., Barto, A.G.: Reinforcement Learning, second edition: An Introduction. MIT Press (Nov 2018) 41. Total, V.: VirusTotal - Stats, https://www.virustotal.com/gui/stats 42. Uprety, A., Rawat, D.B.: Reinforcement Learning for IoT Security: A Comprehensive Survey. IEEE Internet of Things Journal 8(11), 8693–8706 (Jun 2021). https://doi.org/10.1109/JIOT.2020.3040957, conference Name: IEEE Internet of Things Journal 43. Wu, C., Shi, J., Yang, Y., Li, W.: Enhancing Machine Learning Based Malware Detection Model by Reinforcement Learning. In: Proceedings of the 8th International Conference on Communication and Network Security. pp. 74–78. ICCNS 2018, Association for Computing Machinery, New York, NY, USA (Nov 2018). https://doi.org/10.1145/3290480.3290494 44. Zolotukhin, M., Kumar, S., H¨am¨al¨ainen, T.: Reinforcement Learning for Attack Mitigation in SDN-enabled Networks. In: 2020 6th IEEE Conference on Network Softwarization (NetSoft). pp. 282–286 (Jun 2020). https://doi.org/10.1109/NetSoft48620.2020.9165383 45. Sembera, V., Paquet-Clouston, M., Garcia, S., Erquiaga, M.J.: Cybercrime Spe- ˇ cialization: An Expos´e of a Malicious Android Obfuscation-as-a-Service. In: 2021 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW. pp. 213–236 (2021). https://doi.org/10.1109/EuroSPW54576.2021.00029 This paper is under CC BY-NC-SA 4.0 DEED license. available on arxiv

Enhancing Cybersecurity with MEME: Reinforcement Learning for Adversarial Malware Evasion

Authors: (1) Maria Rigaki, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and maria.rigaki@fel.cvut.cz; (2) Sebastian Garcia, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and sebastian.garcia@agents.fel.cvut.cz. Table of Links Abstract & Introduction Threat Model Background and Related Work Methodology Experiments Setup Results Discussion Conclusion, Acknowledgments, and References Appendix 6 Results 6.1 Malware Evasion To evaluate MEME, we tested its evasion capabilities compared to four other algorithms. Figure 3 shows the evasion rates of all algorithms against four different targets. Generally, we can see that some targets are more challenging than others. For example, the Sorel-FFNN target seems the hardest to evade, while Ember seems the easiest. MEME outperforms the PPO algorithm in all four targets, with a difference in mean evasion rate between 5-10% depending on the target. This result indicates that creating a surrogate model and running additional training steps is beneficial and produces a better policy. MEME also outperforms MAB in three out of four targets and GAMMA in all four, showing stable overall performance. GAMMA is the only method affected by the time constraint and, in most experiments, manages to process less than 100 binaries. However, it performs almost as well as MEME on the Sorel-FFNN. This shows that the RL methods, including MAB, did not learn the simple strategy of section injection that GAMMA uses, which is part of their action set. Average number of binary modifications to evade the target. The numbers for MAB correspond to the minimal samples created. Random, PPO, and MEME numbers correspond to the mean episode length of the final evaluation on the test set. Table 2. The AV evasion rate was over 50% for both PPO and MEME, with MEME achieving an average evasion rate of 59%. This demonstrates that even though AV is more complex than a single classifier, it can still be bypassed. In terms of required modifications for a malicious binary to appear benign (Table 2), MEME has the lowest average changes against Sorel-FFNN and Microsoft Defender. MAB performs better with lower average changes against Ember and Sorel-LGB, as it aims to minimize the required actions. However, if MAB struggles with evasion, the average modifications increase. Unfortunately, it was not possible to get the number of modifications for each binary from the GAMMA attack since the framework that implements the attack does not provide this metric. Surrogate agreement metrics with the test set for each target model. Table 3. 6.2 Surrogate Evaluation Table 3 shows the agreement scores of the surrogate models with their respective targets. The label agreement scores were higher than 97% for all targets. In the feature agreement metrics, 9 out of 10 top features, according to SHAP feature importances were the same for the Ember target and the respective surrogates. The percentage is slightly lower for Sorel-LGB, with 7.2 out of the top 10 feature agreement and 16.2 out of the top 20. However, the feature agreement metrics get significantly lower regarding the Sorel-FFNN target, even though the label agreement is higher than 98%. This target is also the harder one to evade for the RL-based methods. This high level of label agreement was reached with only 2,048 queries, significantly lower than reported in prior work (25k) [32]. This paper is under CC BY-NC-SA 4.0 DEED license. available on arxiv

MEME vs. PPO: Analyzing Evasion Rates and Binary Modifications in Malware Detection

Authors: (1) Maria Rigaki, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and maria.rigaki@fel.cvut.cz; (2) Sebastian Garcia, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and sebastian.garcia@agents.fel.cvut.cz. Table of Links Abstract & Introduction Threat Model Background and Related Work Methodology Experiments Setup Results Discussion Conclusion, Acknowledgments, and References Appendix 5 Experiments Setup To evaluate MEME, several experiments were conducted in different configurations. First, there was a selection of four different malware detection solutions as targets to evade. Second, there was a comparison of MEME and four other evasion techniques on these four targets. 5.1 Targets The selection of targets was made to include three highly cited malware detection models together with a real implementation of a popular free Antivirus solution. 1. Ember. A LightGBM [20] model that was released as part of the Ember dataset [3] which was used for training that same model. The decision threshold was set to 0.8336, which corresponds to a 1% false positive rate (FPR) on the Ember 2018 test set. 2. Sorel-LGB. A LightGBM model that was distributed as part of the Sorel20M [15] dataset, which was used for training that same model. The decision threshold was set to 0.5, which corresponds to a 0.2% false positive rate (FPR) on the Sorel-20M test set. 3. Sorel-FFNN. A feed-forward neural network (FFNN) that was also released as part of the Sorel 20M dataset and was trained using the same data. The decision threshold was set to 0.5, which corresponds to 0.6% false positive rate (FPR) on the Sorel-20M test set. 4. Microsoft Defender. An antivirus product that comes pre-installed with the Windows operating system. According to [36], it is the most used free antivirus product for personal computers. All tests were performed using a virtual machine (VM) running an updated version of the product. The VM had no internet connectivity during the binary file scanning. 5.2 Datasets Our experiments required the use of the following datasets: 1. Ember 2018. A dataset that consists of features extracted from one million Windows Portable Executable (PE) files [3]. The dataset is split into training, testing, and ”unlabeled sets”. The training set consists of 300,000 clean samples, 300,000 malicious samples, and 200,000 “unlabeled” samples. The so-called unlabeled part of the dataset was truly unlabeled in the first version of the dataset, however, in the 2018 release, the authors provided an avclass label for all malicious samples, including those in the unlabeled set. Each sample has 2,381 static features related to byte and entropy histograms, PE header information, strings, imports, data directories, etc. 2. Sorel-20M. The Sorel dataset [15] was released in 2020 and contains the extracted features of 20 million binary files (malicious and benign). The feature set used was the same as the one from the Ember dataset. 3. Malware Binary Files. In addition to the Ember features used for training the surrogate, we also obtained 1,000 malicious binary files whose hashes were part of Ember 2018 and we use them for generating the evasive malware with all the methods. 4. Benign Binary Files. All methods require a benign set of data from where they extract benign strings, sections, and other elements that are used for the binary modifications. The same set of 100 benign binaries were used in all experiments. The files were obtained from a Windows 10 virtual machine after installing known benign software. MEME created two versions of the Daux dataset to train the surrogate models. For the Ember and AV surrogates, Daux contains the unlabeled part of the Ember dataset. For the Sorel surrogates, Daux contains 200,000 samples from the Sorel-20M validation set. These datasets were chosen to create the surrogate because they were not used to train the corresponding targets. During the evaluation, a subset of the Sorel-20M test set was used to evaluate the performance of the Sorel surrogate models and the Ember test set was used to evaluate the Ember and AV models. The 1,000 malware binaries were split into training and test sets with a 70-30% ratio using five different seeds. The test sets were used to test all the methods, while the 700 binaries in the training set were used to train each of the RL policies for PPO and MEME (these were the binaries to which modifier actions were applied). 5.3 Adversarial Malware Generation Comparison With regard to the generation of adversarial malware, MEME is compared with four algorithms in total. Two baseline reinforcement learning algorithms that use the Malware-Gym environment: a random agent and an agent that learns a policy using the vanilla Proximal Policy Optimization (PPO) algorithm [35], and two state-of-the-art (SOTA) algorithms: MAB [38] and GAMMA [9]. The two SOTA algorithms were selected based on the fact that they are relatively recently released and the fact that they seem to perform well in the malware evasion task. In addition, their source code is available. The detailed setup used for each of the algorithms, as well as any modifications, are presented below: 1. Random Agent The random agent is the simplest baseline used in our experiments. It uses the Malware-Gym environment and randomly samples the next modification action from the available action space. The agent is evaluated in the test environments using the 300 test malware samples. 2. PPO This is an agent that uses the PPO [35] algorithm as implemented in the Stable-Baselines3 software package. The agent was trained for 2,048 steps on the malware training set and evaluated on the malware test set. To select the hyper-parameters related to PPO training, we used the Tree-structured Parzen Estimator (TPE) method [4] as implemented in the software package Optuna [1]. The TPE algorithm was executed with the Ember dataset, but the settings performed well in the other targets. The tuned hyper-parameters were γ, the learning rate, the maximum gradient norm, the activation function, and the neural network size for the actor and critic models. The search space of each parameter and the final values are presented in the Appendix. 3. MAB RL algorithm that treats the evasive malware generation as a multiarmed bandit problem2. It operates in two stages: evasion and minimization. It samples the action space, which includes generic actions (similar to Malware-Gym) and any successful evasive actions along with the specific modifiers, e.g., appending a specific benign section. MAB directly manipulates each binary without generating a learned policy. Therefore, all experiments were conducted directly on the malicious binary test set. 4. GAMMA An algorithm that injects benign binary sections into malicious PE files while preserving their functionality. It modifies features like section count, byte histograms, and strings, leaving features related to, e.g., certificates and debugging data unaffected. By employing genetic algorithms, GAMMA searches for optimal benign sections to reduce the target model’s confidence by minimizing the content and location of injected sections. The attack is implemented in the SecML library3. Though effective, it has a significantly longer runtime than other tested methods. The attack uses a restricted set of 30 available benign sections and a population size of 20. The λ parameter, impacting the injected data size, was set to 10−6. GAMMA directly operates on each binary and does not generate a learned policy. Hence, all experiments were conducted on the malicious binary test set. 5.4 MEME Experimental Setup The initial training steps n in Algorithm 1 were set to 1,024, and the total number of loops k was two. For evaluation, the test set of 300 malware binaries was used. The surrogate training steps m were set to 2,048 (step 6 of Algorithm 1). MEME utilizes PPO for training and updating the policy (πθ). The PPO settings remained the same as in the baseline experiments, enabling a comparison of the impact of using a surrogate model for additional PPO training. In total, there were a total of 2,048 queries to the target and 4,096 training steps using the surrogate environment. The surrogate was always a LightGBM model. Surrogate training involved two datasets: Daux from an external dataset (e.g., Ember 2018 or Sorel-20M) and Dsur generated during lines 3 and 7 of Algorithm 1. These datasets were mixed with a ratio α, a hyperparameter for LGB surrogate tuning. Other hyperparameters, such as the number of boosting trees, learning rate, tree depth, minimum child samples, and feature fraction, were also tuned separately for each target using TPE and Optuna. Appendix A provides the detailed search space and selected values. Surrogate models were evaluated using the respective target test sets, and a decision threshold matching target FPR levels was calculated. For the AV target without a representative dataset, the surrogate’s decision threshold was set to 0.5. 5.5 General Experiment Settings All the algorithms were tested under a common set of constraints. The maximum allowed modifications to a binary file were set to 15 for the RL-based algorithms. Similarly, for GAMMA, the number of iterations was set to 15; for MAB, the number of ”pulls” was also 15. The second constraint was to set the maximum running time for all experiments to 4 hours. For MAB and GAMMA, this setting means that the algorithms must manage to handle as many malicious binaries as possible in that time. At the same time, for PPO and MEME, this time included both the policy training time and the evaluation time. For PPO and MEME, we set the query budget to 2,048. This budget does not include the final evaluation queries on the test set. MAB and GAMMA were not constrained in the total number of queries because it is not supported by the respective frameworks. Finally, all experiments were run with five different seeds. The random seeds controlled the split of the 1,000 malicious binaries into train and test, and therefore, all methods were tested in the same files. This paper is under CC BY-NC-SA 4.0 DEED license. available on arxiv

MEME Algorithm's Impact on Adversarial Malware Generation and Model Evasion

Authors:
(1) Maria Rigaki, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and maria.rigaki@fel.cvut.cz;
(2) Sebastian Garcia, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and sebastian.garcia@agents.fel.cvut.cz. Table of Links Abstract & Introduction Threat Model Background and Related Work Methodology Experiments Setup Results Discussion Conclusion, Acknowledgments, and References Appendix 4 Methodology The MEME algorithm combines model extraction and reinforcement learning (RL) techniques to improve the generation of evasive malware binaries. The algorithm takes advantage of the fact that the attacker fully controls the binary modifier part of the environment but does not control the target model, which is a black box. However, an attacker can use data collected during the training of the RL agent to create and train a surrogate model of the target. The algorithm is implemented as several training/testing rounds until a final reinforcement learning policy is learned that can produce adversarial binaries for a given target. Figure 2 presents a high-level description of the algorithm, and the detailed listing is presented in Algorithm 1. Following Algorithm 1, MEME initializes a Proximal Policy Optimization (PPO) RL algorithm [35 (lines 1 and 2) to perform a first train of the PPO policy using the target model and a limited amount of steps (using MalwareGym). The goal of this step is to store a small number of observations and labels in Dsur (line 3). Then, Dsur is combined with an auxiliary set of labeled data (Daux) that the attacker should possess, and this combined dataset is used to train a surrogate model for the model extraction attack (line 5). The improved surrogate replaces the target model in a new Malware-Gym environment that trains a better policy (agent) for evasion (line 6). Last, the improved policy is evaluated against the original target model (line 7) to obtain the final evasion metrics. In this last step, we take advantage of the queries done to the target to append their output to the Dsur dataset. This loop is repeated for k rounds. During the last round, the evaluation is performed using the malware binary test set to produce the final evasion metrics (instead of the malware binary evaluation set used in the inner loop). The total number of queries to the target during training is k ∗ n, where k is the total number of rounds. For learning the policy, we use the PPO algorithm, which is a model-free onpolicy gradient method that clips the objective function, limiting the updates done to the policy to avoid too large positive changes or a minimum of negative changes in order to stabilize it. We used the Stable Baselines3 [31] implementation of PPO that uses an Actor-Critic (A2C) architecture. To train the surrogate model, it is not enough to use the data collected during the learning or evaluation of the agent in Dsur. The number of samples is not enough since we aim to minimize the queries to the target. In addition, all observations come from the extracted features of the malicious binary dataset or their adversarial modifications. Therefore using an auxiliary dataset is necessary to learn a good surrogate. This dataset is assumed to be from a similar distribution as the training data distribution of the target, however, this is not easy to achieve when the target is an AV or an unknown system. Adaptations to the Gym Environment MEME was implemented around the Malware-Gym environment (Section 3.1) in its latest version1. Several modifications were applied to the environment to fit the assumptions and constraints for this work: – The ”modify machine type” action was removed because our tests showed that it produces invalid binaries for Windows 10 systems. – All targets were set to return hard labels (0 or 1), not scores. – Regarding the benign sections, the Malware-Gym implementation used only data from ”.text” sections. In our implementation, we use data from other sections if the ”.text” section is unavailable. – The latest environment supports the Ember and Sorel-LGB classifiers as targets. We added new environments to support the Sorel-FFNN, surrogate, and AV targets. The AV requires a web service in a virtual machine that invokes the AV static scanning capabilities. – For all target environments, we added support for saving the observations (features) and scores during training and evaluation runs so that they can be used for the training of the surrogate. Our version of the environment and the experiments performed as part of this work can be found in (https://github.com/stratosphereips/meme malware rl/releases/tag/v1.0). Please note that we are releasing the source code of our implementation for reproducibility and improvement; however, we do not release the trained models or agents to avoid potential misuse. 4.1 Evasion Evaluation To evaluate the performance of the evasion models reasonably and realistically, we limited how many queries they do to the target model and how long they run. The idea behind liming running time is that the ratio at which malware authors create new malware is high, and a method that takes too long to create evasive malware is impractical. Industry measurements such as the ones provided by AV-ATLAS [18] report that 180 new malware are generated per minute. Virus Total [41] provides an even higher measurement of 560 distinct new files uploaded per minute as of the 21st of May, 2023. These global measurements vary but show how quickly new malware variants appear, possibly due to the proliferation of Malware-as-a-Service frameworks. A recent and more conservative measurement from Blackberry mentions that their customers get attacked by 1.5 new malware per minute [39]. Given the above measurements, we decided to constrain the running time of each experiment to four hours in total. Given that the test set consists of 300 malware binaries, this corresponds to 1.25 processed binaries per minute which is lower than the most conservative reported metric we could find. The primary metric used to evaluate the malware evasion task was the evasion rate E, which is the fraction of malware that becomes evasive within the time window of four hours over the total number of malware that were initially detected by each target: E = nev/ndet . We also report the average number of binary modifications required for a malware binary to evade the target. For the Random, PPO, and MEME methods, this is equivalent to the mean episode length over all the detected malware binaries in the test set. The episode length for a non-evasive binary is equal to the maximum number of attempts, and for an evasive one is the number of changes required to become evasive. For the MAB framework, for the evasive binaries, we considered only the minimal binaries with the least amount of actions. The average number of binary modifications was impossible to measure for the GAMMA attack since the SecML framework (through which GAMMA was implemented) does not provide a straightforward way to measure this metric. 4.2 Surrogate Evaluation The surrogate models trained in MEME were evaluated using two different metrics: label agreement and explainability-based feature agreement [37]. The label agreement of two models f and ˆf, respectively, is defined as the average number of similar predictions over a test set Xtest, and it is a standard metric for model extraction fidelity attacks [19]: The feature agreement metric computes the fraction of common features between the sets of top-k features of two explanations. Given two explanations Et (target) and Es (surrogate), the feature agreement metric can be formally defined as: where top features(E, k) returns the set of top-k features of the explanation E based on the magnitude of the feature importance values. The maximum value of the feature agreement is 1. For this work, we measured the feature agreement for k = 10 and k = 20, and the explainability method used was SHAP (SHapley Additive exPlanations) [25]. SHAP employs game theory principles to explain machine learning model outputs by linking effective credit allocation with localized explanations via Shapley values that originated from game theory. SHAP is model-agnostic and has been used effectively in the past for adversarial malware creation [33]. For the LightGBM targets we used TreeSHAP which is designed specifically for tree models, while for Sorel FFNN we used the KernelSHAP variant. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Maria Rigaki, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and maria.rigaki@fel.cvut.cz; (2) Sebastian Garcia, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and sebastian.garcia@agents.fel.cvut.cz. Authors: Authors: (1) Maria Rigaki, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and maria.rigaki@fel.cvut.cz; (2) Sebastian Garcia, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and sebastian.garcia@agents.fel.cvut.cz. Table of Links Abstract & Introduction Abstract & Introduction Threat Model Threat Model Background and Related Work Background and Related Work Methodology Methodology Experiments Setup Experiments Setup Results Results Discussion Discussion Conclusion, Acknowledgments, and References Conclusion, Acknowledgments, and References Appendix Appendix 4 Methodology The MEME algorithm combines model extraction and reinforcement learning (RL) techniques to improve the generation of evasive malware binaries. The algorithm takes advantage of the fact that the attacker fully controls the binary modifier part of the environment but does not control the target model, which is a black box. However, an attacker can use data collected during the training of the RL agent to create and train a surrogate model of the target. The algorithm is implemented as several training/testing rounds until a final reinforcement learning policy is learned that can produce adversarial binaries for a given target. Figure 2 presents a high-level description of the algorithm, and the detailed listing is presented in Algorithm 1. Following Algorithm 1, MEME initializes a Proximal Policy Optimization (PPO) RL algorithm [35 (lines 1 and 2) to perform a first train of the PPO policy using the target model and a limited amount of steps (using MalwareGym). The goal of this step is to store a small number of observations and labels in Dsur (line 3). Then, Dsur is combined with an auxiliary set of labeled data (Daux) that the attacker should possess, and this combined dataset is used to train a surrogate model for the model extraction attack (line 5). The improved surrogate replaces the target model in a new Malware-Gym environment that trains a better policy (agent) for evasion (line 6). Last, the improved policy is evaluated against the original target model (line 7) to obtain the final evasion metrics. In this last step, we take advantage of the queries done to the target to append their output to the Dsur dataset. This loop is repeated for k rounds. During the last round, the evaluation is performed using the malware binary test set to produce the final evasion metrics (instead of the malware binary evaluation set used in the inner loop). The total number of queries to the target during training is k ∗ n, where k is the total number of rounds. For learning the policy, we use the PPO algorithm, which is a model-free onpolicy gradient method that clips the objective function, limiting the updates done to the policy to avoid too large positive changes or a minimum of negative changes in order to stabilize it. We used the Stable Baselines3 [31] implementation of PPO that uses an Actor-Critic (A2C) architecture. To train the surrogate model, it is not enough to use the data collected during the learning or evaluation of the agent in Dsur. The number of samples is not enough since we aim to minimize the queries to the target. In addition, all observations come from the extracted features of the malicious binary dataset or their adversarial modifications. Therefore using an auxiliary dataset is necessary to learn a good surrogate. This dataset is assumed to be from a similar distribution as the training data distribution of the target, however, this is not easy to achieve when the target is an AV or an unknown system. Adaptations to the Gym Environment MEME was implemented around the Malware-Gym environment (Section 3.1) in its latest version1. Several modifications were applied to the environment to fit the assumptions and constraints for this work: Adaptations to the Gym Environment – The ”modify machine type” action was removed because our tests showed that it produces invalid binaries for Windows 10 systems. – All targets were set to return hard labels (0 or 1), not scores. – Regarding the benign sections, the Malware-Gym implementation used only data from ”.text” sections. In our implementation, we use data from other sections if the ”.text” section is unavailable. – The latest environment supports the Ember and Sorel-LGB classifiers as targets. We added new environments to support the Sorel-FFNN, surrogate, and AV targets. The AV requires a web service in a virtual machine that invokes the AV static scanning capabilities. – For all target environments, we added support for saving the observations (features) and scores during training and evaluation runs so that they can be used for the training of the surrogate. Our version of the environment and the experiments performed as part of this work can be found in (https://github.com/stratosphereips/meme malware rl/releases/tag/v1.0). Please note that we are releasing the source code of our implementation for reproducibility and improvement; however, we do not release the trained models or agents to avoid potential misuse. 4.1 Evasion Evaluation To evaluate the performance of the evasion models reasonably and realistically, we limited how many queries they do to the target model and how long they run. The idea behind liming running time is that the ratio at which malware authors create new malware is high, and a method that takes too long to create evasive malware is impractical. Industry measurements such as the ones provided by AV-ATLAS [18] report that 180 new malware are generated per minute. Virus Total [41] provides an even higher measurement of 560 distinct new files uploaded per minute as of the 21st of May, 2023. These global measurements vary but show how quickly new malware variants appear, possibly due to the proliferation of Malware-as-a-Service frameworks. A recent and more conservative measurement from Blackberry mentions that their customers get attacked by 1.5 new malware per minute [39]. Given the above measurements, we decided to constrain the running time of each experiment to four hours in total. Given that the test set consists of 300 malware binaries, this corresponds to 1.25 processed binaries per minute which is lower than the most conservative reported metric we could find. The primary metric used to evaluate the malware evasion task was the evasion rate E, which is the fraction of malware that becomes evasive within the time window of four hours over the total number of malware that were initially detected by each target: E = nev/ndet . We also report the average number of binary modifications required for a malware binary to evade the target. For the Random, PPO, and MEME methods, this is equivalent to the mean episode length over all the detected malware binaries in the test set. The episode length for a non-evasive binary is equal to the maximum number of attempts, and for an evasive one is the number of changes required to become evasive. For the MAB framework, for the evasive binaries, we considered only the minimal binaries with the least amount of actions. The average number of binary modifications was impossible to measure for the GAMMA attack since the SecML framework (through which GAMMA was implemented) does not provide a straightforward way to measure this metric. 4.2 Surrogate Evaluation The surrogate models trained in MEME were evaluated using two different metrics: label agreement and explainability-based feature agreement [37]. The label agreement of two models f and ˆf, respectively, is defined as the average number of similar predictions over a test set Xtest, and it is a standard metric for model extraction fidelity attacks [19]: The feature agreement metric computes the fraction of common features between the sets of top-k features of two explanations. Given two explanations Et (target) and Es (surrogate), the feature agreement metric can be formally defined as: where top features(E, k) returns the set of top-k features of the explanation E based on the magnitude of the feature importance values. The maximum value of the feature agreement is 1. For this work, we measured the feature agreement for k = 10 and k = 20, and the explainability method used was SHAP (SHapley Additive exPlanations) [25]. SHAP employs game theory principles to explain machine learning model outputs by linking effective credit allocation with localized explanations via Shapley values that originated from game theory. SHAP is model-agnostic and has been used effectively in the past for adversarial malware creation [33]. For the LightGBM targets we used TreeSHAP which is designed specifically for tree models, while for Sorel FFNN we used the KernelSHAP variant. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv

EVCS

evcs.com

-- employees

Since n.d.

COMPANY RANKING

EVCS

EVERGREEN INDEX #308

Representing this company? Click here to claim and customize this page!

EVCS WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

EVCS

evcs.com

-- employees

Since n.d.

COMPANY RANKING

EVCS

EVERGREEN INDEX #308

RELATED COMPANIESRANK

STORIES ABOUT EVCS

Representing this company? Click here to claim and customize this page!

EVCS WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!