Optimity is an award-winning tech company that helps B2B clients modernize their offerings for health and financial wellness, using wearable data to drive improved member/customer experience, mitigate ongoing risk and increase adoption of financial wellbeing as it relates to health and longevity. Optimity provides SaaS products (apps, admin-reporting-portal, APIs, SDKs & webflow modules) that engage members (policyholders, clients, plan members, and/or employees) to support and achieve their evolving life goals. Our IP includes proprietary health scores (i.e. Optimity Health Grade™), personalized physical, mental, and financial wellness programs (i.e. Optimity Micro-learning content modules), rewards & gamification methodologies (i.e. Optimity Micro-nudging & Journey System™), and 3+ Billion first-party data points powering insights to support client business outcomes. We have 8 published peer-reviewed studies on health population improvement using our apps & rewards systems, and 10+ industry awards along with case studies from notable clients.

Optimity

By now, most of us are accustomed to using CSS Custom Properties on projects. Since its introduction in 2017, CSS Custom Properties, also known as CSS variables, have optimised the code we work with and allowed engineers a finer degree of control when managing values within rules. CSS Custom Properties can be tricky to deal with and aren’t always as intuitive as one might hope. However, when used right, they can significantly reduce boilerplate code and keep things efficient, making our work more productive. While the approach below is not perfect, it works well when writing CSS for building projects. Why Use CSS Custom Properties As far as my experience goes, I can only think of three possible reasons to implement a CSS Variable: We want to use a given value in more than one place,


We know that a value will change in the future and want to reduce our efforts for refactoring,


We need to store dynamic values from JS execution. With this in mind, we can be more assertive about our use cases. Point 3 is less of a concern for this article, but points 1 and 2 are essential. Let's illustrate this using the familiar HTML button element. The Infamous Button Buttons are notoriously tricky to style consistently for such a simple HTML element. Perhaps it’s because they have several different states and applications, regardless - it's usually the first component we Frontend Engineers take on when it comes to a fresh build. Let's have a look at how we could write the style for a button using CSS Custom Properties: :root {
 --color-primary: purple;
 --color-text: black;
 --color-text-inverted: white;
 --color-border: gray;
}

.button {
 --button-bg: var(--color-primary);
 --button-color: var(--color-text-inverted);
 --button-border: var(--color-border);

 align-items: center;
 background-color: var(--button-bg);
 border: 1px solid var(--button-border);
 color: var(--button-color);
 display: inline-flex;
 line-height: 1;
} As far as this example goes, from a technical standpoint, it’s great. It’s clean and straightforward and would get a 👍 from me in a code review. But could we optimise it further? Well, yes. The var() function in CSS is pretty nifty…we’re used to using it in a somewhat one-dimensional way by just rendering out a value; however, it can be more dynamic. The way to do this is by using a fallback value. If we considered the above example, we could rewrite it like so: :root {
 --color-primary: purple;
 --color-text: black;
 --color-text-inverted: white;
 --color-border: gray;
}

.button {
  align-items: center;
  background-color: var(--button-bg, var(--color-primary));
  border: 2px solid var(--button-border, var(--color-border));
  color: var(--button-color, var(--color-text-inverted));
  cursor: pointer;
  display: inline-flex;
  line-height: 1;
  padding-block: 0.875rem;
  padding-inline: 1rem;
} This is far cleaner and will lead to the same outcomes as the previous example - with less boilerplate code. Overriding it in a variation makes each locally scoped CSS Variable (effectively) optional. We can easily now optionally define the variables defined in .button as separate variations: .button:where(.button-secondary) {
  --button-bg: orange;
  --button-border: teal;
}

.button:where(.button-tertiary) {
  --button-bg: teal;
  --button-border: orange;
} Although this technique is not revolutionary, we care how many lines of compressed text are heading over the wire. The less physical memory a CSS file takes up, the better, which ultimately translates to how many lines of CSS exist. In the first example, we’re looking at almost double the amount of code to achieve the same thing in our optimised example. This compound effect has an influential impact on performance and further improves readability and developer experience. If you'd like to play around a bit further, feel free to fork the following Code Pen: https://codepen.io/iamdainemawer/pen/NWmQZMz Support Since April 2017, according to Google Baseline, this feature is well-supported across all major browsers. If you're interested to learn more about Baseline, read my post about Google Baseline and Its Impact You can check for browser support by adding the following statement to your CSS stylesheets: @supports(--css: variable) {} Can I Use - CSS Nesting
CSS Specification - CSS Nesting Module Resources MDN - CSS Custom Properties
CSS Variables Polyfill By now, most of us are accustomed to using CSS Custom Properties on projects. Since its introduction in 2017, CSS Custom Properties, also known as CSS variables, have optimised the code we work with and allowed engineers a finer degree of control when managing values within rules. CSS Custom Properties can be tricky to deal with and aren’t always as intuitive as one might hope. However, when used right, they can significantly reduce boilerplate code and keep things efficient, making our work more productive. While the approach below is not perfect, it works well when writing CSS for building projects. Why Use CSS Custom Properties As far as my experience goes, I can only think of three possible reasons to implement a CSS Variable: We want to use a given value in more than one place, We know that a value will change in the future and want to reduce our efforts for refactoring, We need to store dynamic values from JS execution. We want to use a given value in more than one place, We want to use a given value in more than one place, We know that a value will change in the future and want to reduce our efforts for refactoring, We know that a value will change in the future and want to reduce our efforts for refactoring, We need to store dynamic values from JS execution. We need to store dynamic values from JS execution. With this in mind, we can be more assertive about our use cases. Point 3 is less of a concern for this article, but points 1 and 2 are essential. Let's illustrate this using the familiar HTML button element. The Infamous Button Buttons are notoriously tricky to style consistently for such a simple HTML element. Perhaps it’s because they have several different states and applications, regardless - it's usually the first component we Frontend Engineers take on when it comes to a fresh build. Let's have a look at how we could write the style for a button using CSS Custom Properties: :root {
 --color-primary: purple;
 --color-text: black;
 --color-text-inverted: white;
 --color-border: gray;
}

.button {
 --button-bg: var(--color-primary);
 --button-color: var(--color-text-inverted);
 --button-border: var(--color-border);

 align-items: center;
 background-color: var(--button-bg);
 border: 1px solid var(--button-border);
 color: var(--button-color);
 display: inline-flex;
 line-height: 1;
} :root {
 --color-primary: purple;
 --color-text: black;
 --color-text-inverted: white;
 --color-border: gray;
}

.button {
 --button-bg: var(--color-primary);
 --button-color: var(--color-text-inverted);
 --button-border: var(--color-border);

 align-items: center;
 background-color: var(--button-bg);
 border: 1px solid var(--button-border);
 color: var(--button-color);
 display: inline-flex;
 line-height: 1;
} As far as this example goes, from a technical standpoint, it’s great. It’s clean and straightforward and would get a 👍 from me in a code review. But could we optimise it further? Well, yes. The var() function in CSS is pretty nifty…we’re used to using it in a somewhat one-dimensional way by just rendering out a value; however, it can be more dynamic. The way to do this is by using a fallback value . var() fallback value If we considered the above example, we could rewrite it like so: :root {
 --color-primary: purple;
 --color-text: black;
 --color-text-inverted: white;
 --color-border: gray;
}

.button {
  align-items: center;
  background-color: var(--button-bg, var(--color-primary));
  border: 2px solid var(--button-border, var(--color-border));
  color: var(--button-color, var(--color-text-inverted));
  cursor: pointer;
  display: inline-flex;
  line-height: 1;
  padding-block: 0.875rem;
  padding-inline: 1rem;
} :root {
 --color-primary: purple;
 --color-text: black;
 --color-text-inverted: white;
 --color-border: gray;
}

.button {
  align-items: center;
  background-color: var(--button-bg, var(--color-primary));
  border: 2px solid var(--button-border, var(--color-border));
  color: var(--button-color, var(--color-text-inverted));
  cursor: pointer;
  display: inline-flex;
  line-height: 1;
  padding-block: 0.875rem;
  padding-inline: 1rem;
} This is far cleaner and will lead to the same outcomes as the previous example - with less boilerplate code. Overriding it in a variation makes each locally scoped CSS Variable (effectively) optional. We can easily now optionally define the variables defined in .button as separate variations: .button:where(.button-secondary) {
  --button-bg: orange;
  --button-border: teal;
}

.button:where(.button-tertiary) {
  --button-bg: teal;
  --button-border: orange;
} .button:where(.button-secondary) {
  --button-bg: orange;
  --button-border: teal;
}

.button:where(.button-tertiary) {
  --button-bg: teal;
  --button-border: orange;
} Although this technique is not revolutionary, we care how many lines of compressed text are heading over the wire. The less physical memory a CSS file takes up, the better, which ultimately translates to how many lines of CSS exist. In the first example, we’re looking at almost double the amount of code to achieve the same thing in our optimised example. This compound effect has an influential impact on performance and further improves readability and developer experience. If you'd like to play around a bit further, feel free to fork the following Code Pen: https://codepen.io/iamdainemawer/pen/NWmQZMz https://codepen.io/iamdainemawer/pen/NWmQZMz Support Since April 2017, according to Google Baseline , this feature is well-supported across all major browsers. If you're interested to learn more about Baseline, read my post about Google Baseline and Its Impact Google Baseline Google Baseline and Its Impact You can check for browser support by adding the following statement to your CSS stylesheets: @supports(--css: variable) {} @supports(--css: variable) {} Can I Use - CSS Nesting CSS Specification - CSS Nesting Module Can I Use - CSS Nesting Can I Use - CSS Nesting CSS Specification - CSS Nesting Module CSS Specification - CSS Nesting Module Resources MDN - CSS Custom Properties CSS Variables Polyfill MDN - CSS Custom Properties MDN - CSS Custom Properties CSS Variables Polyfill CSS Variables Polyfill

Using CSS Custom Properties To Optimize Styling Efficiency and Readability

An investment portfolio is a collection of financial assets, such as stocks, bonds, or cryptocurrency, in which an individual invests. An investment is mostly identified by its risk (how volatile the value is) and its return (what is the expected gain). Investors aim to build a portfolio that minimizes risk while maximizing return. Since investments are all about understanding the numbers, expert traders utilize data science techniques and models to optimize their investment strategy. One such model is the Modern Portfolio Theory (MPT), also known as the Markowitz Mean-Variance Theory. The model provides the optimal investment portfolio using risk assessment and maximizes return for the investor. Let’s understand the role of data science in making efficient investments, look at modern portfolio theory in detail, and discuss the assumptions and risks associated with data science models. More On The Markowitz Mean-Variance Theory The Markowitz Mean-Variance Theory was first published by Harry Markowitz in 1952. The theory presents a data-based model that analyzes financial trends to estimate risk and return. As a rule of thumb, investments are categorized as low-risk, low-return, and high-risk, high-return. In simpler terms, it establishes that investments with a higher risk factor carry a higher reward and vice versa. MPT provides an optimal selection of investments that balances out risk for reward. The final selection of investments and their share in the portfolio represents the ideal investment strategy based on the data trends. The Science Behind the Modern Portfolio Theory Let’s understand the mathematics behind MPT. However, first, we must understand a few key terms that make the mathematical model possible. Expected Return: This is the percentage return expected from an investment. It can be calculated using statistical analysis of historical trends. Standard Deviation: This quantifies the volatility of a particular financial asset. It is the measure of risk associated with an investment, i.e., a high variance asset carries high risk and high reward. It is also estimated using statistical analysis of the data trends. Covariance: This estimates the relationship between the different assets. Covariance helps optimize the portfolio distribution by changing the asset weights depending on the covariances. Given three stocks, A, B, and C, let’s build a portfolio. An investor aims to figure out how many funds to allocate to either stock. For the given stocks, let's assume each stock holds the following features. If the total investment amount is $1000, $200 is for Stock A, $300 for B, and $500 for C. Given the distribution, the mean portfolio return comes out to be. The allocation percentages are also considered the profile’s weights as they determine how much investment goes into which asset. The second important factor to consider here is the portfolio's variance or Risk. Portfolio risk is trickier to calculate as it considers the covariance of the different assets. An optimal portfolio under the Markowitz model includes assets with a negative correlation. If a particular asset declines, another will rise and counter its loss, reducing the overall portfolio's risk. The formula for a portfolio variance becomes The covariance needs to be calculated for each asset pair in the portfolio. Let’s assume our assets have the following correlation matrix. Considering the correlation values and the above standard deviation, we can calculate the covariances using the following formula: The covariance matrix becomes Using the above-calculated values, our portfolio co-variance becomes Efficient Frontier The above example displays one possibility for an investment portfolio. Markowitz’s theory creates multiple combinations of such portfolios using different allocation (weights) values. The different portfolios display various levels of returns for a given risk value (variance). These different portfolios are visualized on a chart called the Efficient Frontier. The curve represents a risk-reward trade-off where investors are interested in everything above the line. Another interesting factor of this chart is the Capital Allocation line (CAL) that runs from the risk-free point (Zero Standard-Deviation) and forms a tangent across the curve. The tangent point has the highest reward-to-risk ratio and is the best possible portfolio for investment. Key Takeaways An investment portfolio comprises various assets such as stocks and bonds. Every investor starts with a fixed investment capital and decides how much to invest in each asset. Data science techniques such as the Markowitz mean-variance theory help determine the optimal share allocation to build the optimal portfolio. The theory formulates a mathematical model to optimize the asset allocations to gain the maximum return for a given risk-level. It analyzes different financial assets and considers their rate of return and risk factors, given their historical trends. The rate of return is an approximation of how much profit the asset will generate over a given time period. The risk factor is quantified using the standard deviation of the asset value. A higher deviation represents a volatile asset and, hence, higher risk. The return and risk values are calculated for various portfolio combinations and are represented on the efficient frontier curve. The curve helps investors determine the highest returns against their selected risk. An investment portfolio is a collection of financial assets, such as stocks, bonds, or cryptocurrency, in which an individual invests. An investment is mostly identified by its risk (how volatile the value is) and its return (what is the expected gain). Investors aim to build a portfolio that minimizes risk while maximizing return. Since investments are all about understanding the numbers, expert traders utilize data science techniques and models to optimize their investment strategy. One such model is the Modern Portfolio Theory (MPT), also known as the Markowitz Mean-Variance Theory. The model provides the optimal investment portfolio using risk assessment and maximizes return for the investor. Let’s understand the role of data science in making efficient investments, look at modern portfolio theory in detail, and discuss the assumptions and risks associated with data science models. More On The Markowitz Mean-Variance Theory More On The Markowitz Mean-Variance Theory The Markowitz Mean-Variance Theory was first published by Harry Markowitz in 1952. The theory presents a data-based model that analyzes financial trends to estimate risk and return. As a rule of thumb, investments are categorized as low-risk, low-return, and high-risk, high-return. In simpler terms, it establishes that investments with a higher risk factor carry a higher reward and vice versa. MPT provides an optimal selection of investments that balances out risk for reward. The final selection of investments and their share in the portfolio represents the ideal investment strategy based on the data trends. The Science Behind the Modern Portfolio Theory The Science Behind the Modern Portfolio Theory Let’s understand the mathematics behind MPT. However, first, we must understand a few key terms that make the mathematical model possible. Expected Return: This is the percentage return expected from an investment. It can be calculated using statistical analysis of historical trends. Expected Return: This is the percentage return expected from an investment. It can be calculated using statistical analysis of historical trends. Expected Return: Standard Deviation: This quantifies the volatility of a particular financial asset. It is the measure of risk associated with an investment, i.e., a high variance asset carries high risk and high reward. It is also estimated using statistical analysis of the data trends. Standard Deviation: This quantifies the volatility of a particular financial asset. It is the measure of risk associated with an investment, i.e., a high variance asset carries high risk and high reward. It is also estimated using statistical analysis of the data trends. Standard Deviation: Covariance: This estimates the relationship between the different assets. Covariance helps optimize the portfolio distribution by changing the asset weights depending on the covariances. Covariance: This estimates the relationship between the different assets. Covariance helps optimize the portfolio distribution by changing the asset weights depending on the covariances. Covariance: Given three stocks, A, B, and C, let’s build a portfolio. An investor aims to figure out how many funds to allocate to either stock. For the given stocks, let's assume each stock holds the following features. Given three stocks, A, B, and C, let’s build a portfolio. An investor aims to figure out how many funds to allocate to either stock. For the given stocks, let's assume each stock holds the following features. If the total investment amount is $1000, $200 is for Stock A, $300 for B, and $500 for C. Given the distribution, the mean portfolio return comes out to be. The allocation percentages are also considered the profile’s weights as they determine how much investment goes into which asset. The second important factor to consider here is the portfolio's variance or Risk. Portfolio risk is trickier to calculate as it considers the covariance of the different assets. An optimal portfolio under the Markowitz model includes assets with a negative correlation. If a particular asset declines, another will rise and counter its loss, reducing the overall portfolio's risk. The formula for a portfolio variance becomes The covariance needs to be calculated for each asset pair in the portfolio. Let’s assume our assets have the following correlation matrix. Considering the correlation values and the above standard deviation, we can calculate the covariances using the following formula: The covariance matrix becomes Using the above-calculated values, our portfolio co-variance becomes Efficient Frontier Efficient Frontier The above example displays one possibility for an investment portfolio. Markowitz’s theory creates multiple combinations of such portfolios using different allocation (weights) values. The different portfolios display various levels of returns for a given risk value (variance). These different portfolios are visualized on a chart called the Efficient Frontier. The curve represents a risk-reward trade-off where investors are interested in everything above the line. Another interesting factor of this chart is the Capital Allocation line (CAL) that runs from the risk-free point (Zero Standard-Deviation) and forms a tangent across the curve. The tangent point has the highest reward-to-risk ratio and is the best possible portfolio for investment. Key Takeaways Key Takeaways An investment portfolio comprises various assets such as stocks and bonds. Every investor starts with a fixed investment capital and decides how much to invest in each asset. Data science techniques such as the Markowitz mean-variance theory help determine the optimal share allocation to build the optimal portfolio. The theory formulates a mathematical model to optimize the asset allocations to gain the maximum return for a given risk-level. It analyzes different financial assets and considers their rate of return and risk factors, given their historical trends. The rate of return is an approximation of how much profit the asset will generate over a given time period. The risk factor is quantified using the standard deviation of the asset value. A higher deviation represents a volatile asset and, hence, higher risk. The return and risk values are calculated for various portfolio combinations and are represented on the efficient frontier curve. The curve helps investors determine the highest returns against their selected risk.

Data Science for Portfolio Optimization: Markowitz Mean-Variance Theory

If you had to pick a key determinant for your satisfaction on a web page, speed would undoubtedly be in the top two of all your arguments. You’re not alone either, more than half of web users worldwide require a website’s load time to be under three seconds or they lose interest. Beyond load times, a page’s responsiveness also contributes to the perception of speed for users. However, due to server round trip times, network glitches, and sometimes the processing power of the machine, response times can be wonky. But nobody wants to listen to technical jargon, they just need their technology to work. To fix this issue, front-end developers turned to optimistic updates. You know when you like an image on Instagram, and the heart appears even when your network is down? Those are optimistic updates. Typically, the User interface (UI) waits until the server confirms an action before reflecting an update, however, optimistic updates bypass that sequence. In one line, Optimistic updates is a tool used to enhance user experience by initiating UI changes without backend validation. This article looks to dive deeper into optimistic updates. How do they work? Why use them? What use cases render them ineffective? Why do we use optimistic updates? Why embrace this tool, you might ask? The advantages are persuasive. As mentioned earlier in this article, the tool updates the UI instantly, creating a smoother and more responsive user experience, eliminating delays, and keeping users engaged. It also takes away the wait time for server confirmation making it possible for faster interaction, and improving the user's satisfaction. Optimistic updates can be applied in various functions like common actions; liking a post, updating a profile picture, or adding an item to a cart. Further, offline editing allows users to make changes without access to the internet, and later sync when they're back online. An example of this is Google Docs. How optimistic updates work Once an action is implemented, the tool identifies it and triggers an update. Next, it updates the local state, modifying the local UI structure as if the change were successful. A request is sent to the server to communicate the desired change, which in turn sends back a response of either Success or Failure. User initiates action (clicks like) -> UI updates immediately (instantly reflects 1Like) -> Server communication (request is sent to the server notifying it of the action) -> Server response (server processes the request and sends a response, success or failure). If it's successful then the optimistic update is validated, if it's a failure then the optimistic update is reverted (the like indicator is removed.). Learn more here When not to use optimistic updates While Optimistic Update is a good tool, it's only ideal for frequent, low-risk action where the server validation is simple and quick. Optimistic updates require careful implementation. It's not to be used in transactions where failed updates could have significant backfires (e.g. financial transactions). Avoid using them when maintaining strict data integrity across multiple users. Optimistic updates are valuable tools in a developer toolkit. By understanding their functions and importance, you have an advantage in using them to create responsive user experiences that leave your users satisfied. If you had to pick a key determinant for your satisfaction on a web page, speed would undoubtedly be in the top two of all your arguments. You’re not alone either, more than half of web users worldwide require a website’s load time to be under three seconds or they lose interest. Beyond load times, a page’s responsiveness also contributes to the perception of speed for users. However, due to server round trip times, network glitches, and sometimes the processing power of the machine, response times can be wonky. But nobody wants to listen to technical jargon, they just need their technology to work. To fix this issue, front-end developers turned to optimistic updates. You know when you like an image on Instagram, and the heart appears even when your network is down? Those are optimistic updates. Typically, the User interface (UI) waits until the server confirms an action before reflecting an update, however, optimistic updates bypass that sequence. front-end developers User interface In one line, Optimistic updates is a tool used to enhance user experience by initiating UI changes without backend validation. This article looks to dive deeper into optimistic updates. How do they work? Why use them? What use cases render them ineffective? Why do we use optimistic updates? Why embrace this tool, you might ask? The advantages are persuasive. As mentioned earlier in this article, the tool updates the UI instantly, creating a smoother and more responsive user experience, eliminating delays, and keeping users engaged. It also takes away the wait time for server confirmation making it possible for faster interaction, and improving the user's satisfaction. Optimistic updates can be applied in various functions like common actions; liking a post, updating a profile picture, or adding an item to a cart. Further, offline editing allows users to make changes without access to the internet, and later sync when they're back online. An example of this is Google Docs. How optimistic updates work Once an action is implemented, the tool identifies it and triggers an update. Next, it updates the local state, modifying the local UI structure as if the change were successful. A request is sent to the server to communicate the desired change, which in turn sends back a response of either Success or Failure. User initiates action (clicks like) -> UI updates immediately (instantly reflects 1Like) -> Server communication (request is sent to the server notifying it of the action) -> Server response (server processes the request and sends a response, success or failure). If it's successful then the optimistic update is validated, if it's a failure then the optimistic update is reverted (the like indicator is removed.). Learn more here here When not to use optimistic updates While Optimistic Update is a good tool, it's only ideal for frequent, low-risk action where the server validation is simple and quick. Optimistic updates require careful implementation. It's not to be used in transactions where failed updates could have significant backfires (e.g. financial transactions). Avoid using them when maintaining strict data integrity across multiple users. Optimistic updates are valuable tools in a developer toolkit. By understanding their functions and importance, you have an advantage in using them to create responsive user experiences that leave your users satisfied.

Optimistic Updates in Frontend Applications

Feedback Loop Optimization:
From Customer Insights to Product Development

How Optimistic Rollups Enhance Ethereum's Performance and Security

Authors:
(1) Maria Rigaki, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and maria.rigaki@fel.cvut.cz;
(2) Sebastian Garcia, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and sebastian.garcia@agents.fel.cvut.cz. Table of Links Abstract & Introduction Threat Model Background and Related Work Methodology Experiments Setup Results Discussion Conclusion, Acknowledgments, and References Appendix 4 Methodology The MEME algorithm combines model extraction and reinforcement learning (RL) techniques to improve the generation of evasive malware binaries. The algorithm takes advantage of the fact that the attacker fully controls the binary modifier part of the environment but does not control the target model, which is a black box. However, an attacker can use data collected during the training of the RL agent to create and train a surrogate model of the target. The algorithm is implemented as several training/testing rounds until a final reinforcement learning policy is learned that can produce adversarial binaries for a given target. Figure 2 presents a high-level description of the algorithm, and the detailed listing is presented in Algorithm 1. Following Algorithm 1, MEME initializes a Proximal Policy Optimization (PPO) RL algorithm [35 (lines 1 and 2) to perform a first train of the PPO policy using the target model and a limited amount of steps (using MalwareGym). The goal of this step is to store a small number of observations and labels in Dsur (line 3). Then, Dsur is combined with an auxiliary set of labeled data (Daux) that the attacker should possess, and this combined dataset is used to train a surrogate model for the model extraction attack (line 5). The improved surrogate replaces the target model in a new Malware-Gym environment that trains a better policy (agent) for evasion (line 6). Last, the improved policy is evaluated against the original target model (line 7) to obtain the final evasion metrics. In this last step, we take advantage of the queries done to the target to append their output to the Dsur dataset. This loop is repeated for k rounds. During the last round, the evaluation is performed using the malware binary test set to produce the final evasion metrics (instead of the malware binary evaluation set used in the inner loop). The total number of queries to the target during training is k ∗ n, where k is the total number of rounds. For learning the policy, we use the PPO algorithm, which is a model-free onpolicy gradient method that clips the objective function, limiting the updates done to the policy to avoid too large positive changes or a minimum of negative changes in order to stabilize it. We used the Stable Baselines3 [31] implementation of PPO that uses an Actor-Critic (A2C) architecture. To train the surrogate model, it is not enough to use the data collected during the learning or evaluation of the agent in Dsur. The number of samples is not enough since we aim to minimize the queries to the target. In addition, all observations come from the extracted features of the malicious binary dataset or their adversarial modifications. Therefore using an auxiliary dataset is necessary to learn a good surrogate. This dataset is assumed to be from a similar distribution as the training data distribution of the target, however, this is not easy to achieve when the target is an AV or an unknown system. Adaptations to the Gym Environment MEME was implemented around the Malware-Gym environment (Section 3.1) in its latest version1. Several modifications were applied to the environment to fit the assumptions and constraints for this work: – The ”modify machine type” action was removed because our tests showed that it produces invalid binaries for Windows 10 systems. – All targets were set to return hard labels (0 or 1), not scores. – Regarding the benign sections, the Malware-Gym implementation used only data from ”.text” sections. In our implementation, we use data from other sections if the ”.text” section is unavailable. – The latest environment supports the Ember and Sorel-LGB classifiers as targets. We added new environments to support the Sorel-FFNN, surrogate, and AV targets. The AV requires a web service in a virtual machine that invokes the AV static scanning capabilities. – For all target environments, we added support for saving the observations (features) and scores during training and evaluation runs so that they can be used for the training of the surrogate. Our version of the environment and the experiments performed as part of this work can be found in (https://github.com/stratosphereips/meme malware rl/releases/tag/v1.0). Please note that we are releasing the source code of our implementation for reproducibility and improvement; however, we do not release the trained models or agents to avoid potential misuse. 4.1 Evasion Evaluation To evaluate the performance of the evasion models reasonably and realistically, we limited how many queries they do to the target model and how long they run. The idea behind liming running time is that the ratio at which malware authors create new malware is high, and a method that takes too long to create evasive malware is impractical. Industry measurements such as the ones provided by AV-ATLAS [18] report that 180 new malware are generated per minute. Virus Total [41] provides an even higher measurement of 560 distinct new files uploaded per minute as of the 21st of May, 2023. These global measurements vary but show how quickly new malware variants appear, possibly due to the proliferation of Malware-as-a-Service frameworks. A recent and more conservative measurement from Blackberry mentions that their customers get attacked by 1.5 new malware per minute [39]. Given the above measurements, we decided to constrain the running time of each experiment to four hours in total. Given that the test set consists of 300 malware binaries, this corresponds to 1.25 processed binaries per minute which is lower than the most conservative reported metric we could find. The primary metric used to evaluate the malware evasion task was the evasion rate E, which is the fraction of malware that becomes evasive within the time window of four hours over the total number of malware that were initially detected by each target: E = nev/ndet . We also report the average number of binary modifications required for a malware binary to evade the target. For the Random, PPO, and MEME methods, this is equivalent to the mean episode length over all the detected malware binaries in the test set. The episode length for a non-evasive binary is equal to the maximum number of attempts, and for an evasive one is the number of changes required to become evasive. For the MAB framework, for the evasive binaries, we considered only the minimal binaries with the least amount of actions. The average number of binary modifications was impossible to measure for the GAMMA attack since the SecML framework (through which GAMMA was implemented) does not provide a straightforward way to measure this metric. 4.2 Surrogate Evaluation The surrogate models trained in MEME were evaluated using two different metrics: label agreement and explainability-based feature agreement [37]. The label agreement of two models f and ˆf, respectively, is defined as the average number of similar predictions over a test set Xtest, and it is a standard metric for model extraction fidelity attacks [19]: The feature agreement metric computes the fraction of common features between the sets of top-k features of two explanations. Given two explanations Et (target) and Es (surrogate), the feature agreement metric can be formally defined as: where top features(E, k) returns the set of top-k features of the explanation E based on the magnitude of the feature importance values. The maximum value of the feature agreement is 1. For this work, we measured the feature agreement for k = 10 and k = 20, and the explainability method used was SHAP (SHapley Additive exPlanations) [25]. SHAP employs game theory principles to explain machine learning model outputs by linking effective credit allocation with localized explanations via Shapley values that originated from game theory. SHAP is model-agnostic and has been used effectively in the past for adversarial malware creation [33]. For the LightGBM targets we used TreeSHAP which is designed specifically for tree models, while for Sorel FFNN we used the KernelSHAP variant. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Maria Rigaki, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and maria.rigaki@fel.cvut.cz; (2) Sebastian Garcia, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and sebastian.garcia@agents.fel.cvut.cz. Authors: Authors: (1) Maria Rigaki, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and maria.rigaki@fel.cvut.cz; (2) Sebastian Garcia, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic and sebastian.garcia@agents.fel.cvut.cz. Table of Links Abstract & Introduction Abstract & Introduction Threat Model Threat Model Background and Related Work Background and Related Work Methodology Methodology Experiments Setup Experiments Setup Results Results Discussion Discussion Conclusion, Acknowledgments, and References Conclusion, Acknowledgments, and References Appendix Appendix 4 Methodology The MEME algorithm combines model extraction and reinforcement learning (RL) techniques to improve the generation of evasive malware binaries. The algorithm takes advantage of the fact that the attacker fully controls the binary modifier part of the environment but does not control the target model, which is a black box. However, an attacker can use data collected during the training of the RL agent to create and train a surrogate model of the target. The algorithm is implemented as several training/testing rounds until a final reinforcement learning policy is learned that can produce adversarial binaries for a given target. Figure 2 presents a high-level description of the algorithm, and the detailed listing is presented in Algorithm 1. Following Algorithm 1, MEME initializes a Proximal Policy Optimization (PPO) RL algorithm [35 (lines 1 and 2) to perform a first train of the PPO policy using the target model and a limited amount of steps (using MalwareGym). The goal of this step is to store a small number of observations and labels in Dsur (line 3). Then, Dsur is combined with an auxiliary set of labeled data (Daux) that the attacker should possess, and this combined dataset is used to train a surrogate model for the model extraction attack (line 5). The improved surrogate replaces the target model in a new Malware-Gym environment that trains a better policy (agent) for evasion (line 6). Last, the improved policy is evaluated against the original target model (line 7) to obtain the final evasion metrics. In this last step, we take advantage of the queries done to the target to append their output to the Dsur dataset. This loop is repeated for k rounds. During the last round, the evaluation is performed using the malware binary test set to produce the final evasion metrics (instead of the malware binary evaluation set used in the inner loop). The total number of queries to the target during training is k ∗ n, where k is the total number of rounds. For learning the policy, we use the PPO algorithm, which is a model-free onpolicy gradient method that clips the objective function, limiting the updates done to the policy to avoid too large positive changes or a minimum of negative changes in order to stabilize it. We used the Stable Baselines3 [31] implementation of PPO that uses an Actor-Critic (A2C) architecture. To train the surrogate model, it is not enough to use the data collected during the learning or evaluation of the agent in Dsur. The number of samples is not enough since we aim to minimize the queries to the target. In addition, all observations come from the extracted features of the malicious binary dataset or their adversarial modifications. Therefore using an auxiliary dataset is necessary to learn a good surrogate. This dataset is assumed to be from a similar distribution as the training data distribution of the target, however, this is not easy to achieve when the target is an AV or an unknown system. Adaptations to the Gym Environment MEME was implemented around the Malware-Gym environment (Section 3.1) in its latest version1. Several modifications were applied to the environment to fit the assumptions and constraints for this work: Adaptations to the Gym Environment – The ”modify machine type” action was removed because our tests showed that it produces invalid binaries for Windows 10 systems. – All targets were set to return hard labels (0 or 1), not scores. – Regarding the benign sections, the Malware-Gym implementation used only data from ”.text” sections. In our implementation, we use data from other sections if the ”.text” section is unavailable. – The latest environment supports the Ember and Sorel-LGB classifiers as targets. We added new environments to support the Sorel-FFNN, surrogate, and AV targets. The AV requires a web service in a virtual machine that invokes the AV static scanning capabilities. – For all target environments, we added support for saving the observations (features) and scores during training and evaluation runs so that they can be used for the training of the surrogate. Our version of the environment and the experiments performed as part of this work can be found in (https://github.com/stratosphereips/meme malware rl/releases/tag/v1.0). Please note that we are releasing the source code of our implementation for reproducibility and improvement; however, we do not release the trained models or agents to avoid potential misuse. 4.1 Evasion Evaluation To evaluate the performance of the evasion models reasonably and realistically, we limited how many queries they do to the target model and how long they run. The idea behind liming running time is that the ratio at which malware authors create new malware is high, and a method that takes too long to create evasive malware is impractical. Industry measurements such as the ones provided by AV-ATLAS [18] report that 180 new malware are generated per minute. Virus Total [41] provides an even higher measurement of 560 distinct new files uploaded per minute as of the 21st of May, 2023. These global measurements vary but show how quickly new malware variants appear, possibly due to the proliferation of Malware-as-a-Service frameworks. A recent and more conservative measurement from Blackberry mentions that their customers get attacked by 1.5 new malware per minute [39]. Given the above measurements, we decided to constrain the running time of each experiment to four hours in total. Given that the test set consists of 300 malware binaries, this corresponds to 1.25 processed binaries per minute which is lower than the most conservative reported metric we could find. The primary metric used to evaluate the malware evasion task was the evasion rate E, which is the fraction of malware that becomes evasive within the time window of four hours over the total number of malware that were initially detected by each target: E = nev/ndet . We also report the average number of binary modifications required for a malware binary to evade the target. For the Random, PPO, and MEME methods, this is equivalent to the mean episode length over all the detected malware binaries in the test set. The episode length for a non-evasive binary is equal to the maximum number of attempts, and for an evasive one is the number of changes required to become evasive. For the MAB framework, for the evasive binaries, we considered only the minimal binaries with the least amount of actions. The average number of binary modifications was impossible to measure for the GAMMA attack since the SecML framework (through which GAMMA was implemented) does not provide a straightforward way to measure this metric. 4.2 Surrogate Evaluation The surrogate models trained in MEME were evaluated using two different metrics: label agreement and explainability-based feature agreement [37]. The label agreement of two models f and ˆf, respectively, is defined as the average number of similar predictions over a test set Xtest, and it is a standard metric for model extraction fidelity attacks [19]: The feature agreement metric computes the fraction of common features between the sets of top-k features of two explanations. Given two explanations Et (target) and Es (surrogate), the feature agreement metric can be formally defined as: where top features(E, k) returns the set of top-k features of the explanation E based on the magnitude of the feature importance values. The maximum value of the feature agreement is 1. For this work, we measured the feature agreement for k = 10 and k = 20, and the explainability method used was SHAP (SHapley Additive exPlanations) [25]. SHAP employs game theory principles to explain machine learning model outputs by linking effective credit allocation with localized explanations via Shapley values that originated from game theory. SHAP is model-agnostic and has been used effectively in the past for adversarial malware creation [33]. For the LightGBM targets we used TreeSHAP which is designed specifically for tree models, while for Sorel FFNN we used the KernelSHAP variant. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv

OPTIMITY

myoptimity.com

11-50 employees

Since 2014

COMPANY RANKING

OPTIMITY

EVERGREEN INDEX #7775

Representing this company? Click here to claim and customize this page!

Optimity WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

OPTIMITY

myoptimity.com

11-50 employees

Since 2014

COMPANY RANKING

OPTIMITY

EVERGREEN INDEX #7775

RELATED COMPANIESRANK

STORIES ABOUT OPTIMITY

Representing this company? Click here to claim and customize this page!

Optimity WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!