Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Background and Related Work Research Method Results Discussion Conclusion and References Background and Related Work Background and Related Work Background and Related Work Research Method Research Method Research Method Results Results Results Discussion Discussion Discussion Conclusion and References Conclusion and References Conclusion and References 2 Background and Related Work Organizations of all industries and sizes perform various combinations of activities to achieve desired results, may it be the production of physical goods or the provision of services. These combinations of activities are called business processes [1]. They are often standardized and documented, meaning that each time an organization tries to achieve a particular result, they use a similar combination of activities. Such a standardized business process is often modeled graphically with the Business Process Model and Notation (BPMN) [22]. BPI is a central part of BPM [6], and also a core topic of this study. The traditional BPM lifecycle (see non-blue areas and dotted arrow of Figure 1) is generally sequential [6] and does not consider failures as “first-class citizens”. This means that failures are not considered a necessary part of improvement and evaluated systematically, but rather a nuisance caused by insufficient planning in the redesign phase; and if failure happens, the whole lifecycle has to be repeated. A more comprehensive approach to assessing the effects of change is AB testing. The main goal of AB testing is to quickly determine whether a particular change to a system component will improve important performance metrics [2]. The initial and the updated version of the component are tested in parallel using randomized experiments in a production environment (A vs B). The new version is often only made available to a select group of consumers, limiting any potential adverse impacts. The method has emerged as a popular approach when updating business-to-consumer software systems [11]. A particularly dynamic approach to AB testing can be facilitated by RL. While supervised learning aims to learn how to label elements from a collection of labeled data, and unsupervised learning tries to find hidden structures in unlabeled data, RL has the goal of optimizing the behavior of a software agent based on a numeric reward in an interactive environment [26]. In RL, the agent is situated in a specific environment and has to decide on an action. This action is then evaluated, and a reward is calculated based on the action’s consequence. The goal of the agent is to maximize the total reward it obtains. Learning which choices to make in what situation happens, essentially, through a systematic approach to trial and error [26]. As mentioned above, the sequential approach of the traditional BPI lifecycle fails to rapidly react to improvement hypotheses that are falsified in reality. This is a crucial shortcoming: research on BPI has shown that 75 percent of BPI ideas did not lead to an improvement: half of them had no effect, and a quarter even had detrimental outcomes [7]. This issue can be observed across domains: in a study conducted at Microsoft, only a third of the website improvement ideas actually had a positive impact [12]. Furthermore, comparing process performance before and after the implementation is problematic in and of itself because changing environmental factors may be the primary driver of changes in process performance (or lack thereof). To mitigate these problems, [24] proposes using AB testing when transitioning from the analysis to the implementation phase. This would mean that the redesigned version is deployed in parallel to the old process version, allowing for a fair comparison. Since AB testing is not traditionally used in such a high-risk and long-running setting as BPM, the authors [24] apply RL to facilitate dynamic testing. With RL algorithms, we can make decisions based on the obtained data faster, by already dynamically routing process instantiation requests to the better-performing version during the experiment itself, thereby minimizing the risk of exposing customers to suboptimal process versions for too long. Altogether, AB-BPM should also allow for a shorter theoretical analysis of the redesign, in line with the DevOps mantra “fail fast, fail cheap”. Figure 1 (including the blue areas, excluding the dotted arrow) presents the improved AB-BPM lifecycle. In addition to the RL-supported AB testing of improvement hypotheses, the complete AB-BPM method proposes some more test and analysis techniques. Our inquiry focuses on the RL-supported AB testing of process variants: it is at the AB-BPM method’s core, whereas the other steps in AB-BPM merely support the design of the RL-supported AB tests. References to AB-BPM in this work solely refer to the RL-supported AB testing of business process variants. Authors: (1) Aaron Friedrich Kurz [0000 −0002 −2547 −6780], Technische Universitat Berlin, Berlin, Germany and SAP Signavio, Berlin, Germany (aaron.kurz@sap.com); (2) Timotheus Kampik, SAP Signavio, Berlin, Germany (timotheus.kampik@sap.com); (3) Luise Pufahl, Technische Universitat Munchen, Munich, Germany (luise.pufahl@tum.de); (4) Ingo Weber [0000 −0002 −4833 −5921], Technische Universitat Munchen, Munich, Germany and Fraunhofer Gesellschaft, Munich, Germany (ingo.weber@tum.de). Authors: Authors: (1) Aaron Friedrich Kurz [0000 −0002 −2547 −6780], Technische Universitat Berlin, Berlin, Germany and SAP Signavio, Berlin, Germany (aaron.kurz@sap.com); (2) Timotheus Kampik, SAP Signavio, Berlin, Germany (timotheus.kampik@sap.com); (3) Luise Pufahl, Technische Universitat Munchen, Munich, Germany (luise.pufahl@tum.de); (4) Ingo Weber [0000 −0002 −4833 −5921], Technische Universitat Munchen, Munich, Germany and Fraunhofer Gesellschaft, Munich, Germany (ingo.weber@tum.de). This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv
