Helping enterprise salespeople close more deals faster by providing rich buyer intelligence and recommending the most effective content by sales stage

Contiq

contiq

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Hyerin Cho (조혜린), Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA;
(2) Ben S. Prather, CCS-2, Los Alamos National Laboratory, PO Box 1663, Los Alamos, NM 87545, USA;
(3) Ramesh Narayan, enter for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA;
(4) Priyamvada Natarajan, Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA, Department of Astronomy, Yale University, Kline Tower, 266 Whitney Avenue, New Haven, CT 06511, USA and Department of Physics, Yale University, P.O. Box 208121, New Haven, CT 06520, USA;
(5) Kung-Yi Su, Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA;
(6) Angelo Ricarte, Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA;
(7) Koushik Chatterjee, Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA. Table of Links Abstract and Introduction Numerical Methods Hydrodynamic Bondi Accretion Magnetized Bondi Accretion Feedback Via Reconnection-Driven Convection Summary and Conclusions Acknowledgements Appendix A. GRMHD Primer and Definitions B. Numerical Set-up C. Resolution and Initial Condition Study References REFERENCES Angl´es-Alc´azar, D., Quataert, E., Hopkins, P. F., et al. 2021, ApJ, 917, 53, doi: 10.3847/1538-4357/ac09e8 Anile, A. M. 1989, Relativistic fluids and magneto-fluids: With applications in astrophysics and plasma physics Bisnovatyi-Kogan, G. S., & Ruzmaikin, A. A. 1974, Ap&SS, 28, 45, doi: 10.1007/BF00642237 Bondi, H. 1952, MNRAS, 112, 195, doi: 10.1093/mnras/112.2.195 Booth, C. M., & Schaye, J. 2009, MNRAS, 398, 53, doi: 10.1111/j.1365-2966.2009.15043.x Chatterjee, K., & Narayan, R. 2022, ApJ, 941, 30, doi: 10.3847/1538-4357/ac9d97 Chatterjee, K., Chael, A., Tiede, P., et al. 2023, Galaxies, 11, 38, doi: 10.3390/galaxies11020038 Dav´e, R., Angl´es-Alc´azar, D., Narayanan, D., et al. 2019, MNRAS, 486, 2827, doi: 10.1093/mnras/stz937 Event Horizon Telescope Collaboration, Akiyama, K., Algaba, J. C., et al. 2021, ApJL, 910, L13, doi: 10.3847/2041-8213/abe4de Ferrarese, L., & Merritt, D. 2000, ApJL, 539, L9, doi: 10.1086/312838 Ferri`ere, K. 2020, Plasma Physics and Controlled Fusion, 62, 014014, doi: 10.1088/13616587/ab49eb Fiacconi, D., Sijacki, D., & Pringle, J. E. 2018, MNRAS, 477, 3807, doi: 10.1093/mnras/sty893 Gammie, C. F., McKinney, J. C., & T´oth, G. 2003, ApJ, 589, 444, doi: 10.1086/374594 Gebhardt, K., Bender, R., Bower, G., et al. 2000, ApJL, 539, L13, doi: 10.1086/312840 Grete, P., Dolence, J. C., Miller, J. M., et al. 2023, 37, 465, doi: 10.1177/10943420221143775 Guerra, J. A., Lopez-Rodriguez, E., Chuss, D. T., Butterfield, N. O., & Schmelz, J. T. 2023, AJ, 166, 37, doi: 10.3847/1538-3881/acdacd Guo, M., Stone, J. M., Kim, C.-G., & Quataert, E. 2023, ApJ, 946, 26, doi: 10.3847/1538-4357/acb81e Hopkins, P. F., & Quataert, E. 2010, MNRAS, 407, 1529, doi: 10.1111/j.1365-2966.2010.17064.x Hopkins, P. F., Grudic, M. Y., Su, K.-Y., et al. 2023, arXiv e-prints, arXiv:2309.13115. https://arxiv.org/abs/2309.13115 Igumenshchev, I. V., & Narayan, R. 2002, ApJ, 566, 137, doi: 10.1086/338077 Igumenshchev, I. V., Narayan, R., & Abramowicz, M. A. 2003, ApJ, 592, 1042, doi: 10.1086/375769 Kaaz, N., Murguia-Berthier, A., Chatterjee, K., Liska, M. T. P., & Tchekhovskoy, A. 2023, ApJ, 950, 31, doi: 10.3847/1538-4357/acc7a1 Komissarov, S. S. 1999, MNRAS, 303, 343, doi: 10.1046/j.1365-8711.1999.02244.x Kormendy, J., & Ho, L. C. 2013, ARA&A, 51, 511, doi: 10.1146/annurev-astro-082708-101811 Lalakos, A., Gottlieb, O., Kaaz, N., et al. 2022, ApJL, 936, L5, doi: 10.3847/2041-8213/ac7bed Li, Y., & Bryan, G. L. 2014, ApJ, 789, 153, doi: 10.1088/0004-637X/789/2/153 Magorrian, J., Tremaine, S., Richstone, D., et al. 1998, AJ, 115, 2285, doi: 10.1086/300353 Meszaros, P. 1975, A&A, 44, 59 Michel, F. C. 1972, Ap&SS, 15, 153, doi: 10.1007/BF00649949 Narayan, R., Chael, A., Chatterjee, K., Ricarte, A., & Curd, B. 2022, MNRAS, 511, 3795, doi: 10.1093/mnras/stac285 Narayan, R., Igumenshchev, I. V., & Abramowicz, M. A. 2000, ApJ, 539, 798, doi: 10.1086/309268—. 2003, PASJ, 55, L69, doi: 10.1093/pasj/55.6.L69 Ni, Y., Di Matteo, T., Bird, S., et al. 2022, MNRAS, 513, 670, doi: 10.1093/mnras/stac351 Pen, U.-L., Matzner, C. D., & Wong, S. 2003, ApJL, 596, L207, doi: 10.1086/379339 Penna, R. F., Kulkarni, A., & Narayan, R. 2013, A&A, 559, A116, doi: 10.1051/0004-6361/201219666 Porth, O., Chatterjee, K., Narayan, R., et al. 2019, ApJS, 243, 26, doi: 10.3847/1538-4365/ab29fd Prather, B. S., Wong, G. N., Dhruv, V., et al. 2021, 6, 3336, doi: 10.21105/joss.03336 Quataert, E., & Gruzinov, A. 2000, ApJ, 539, 809, doi: 10.1086/309267 Ressler, S. M., Quataert, E., White, C. J., & Blaes, O. 2021, MNRAS, 504, 6076, doi: 10.1093/mnras/stab311 Ressler, S. M., White, C. J., Quataert, E., & Stone, J. M. 2020, ApJL, 896, L6, doi: 10.3847/2041-8213/ab9532 Ricarte, A., Tremmel, M., Natarajan, P., & Quinn, T. 2019, MNRAS, 489, 802, doi: 10.1093/mnras/stz2161 Ripperda, B., Liska, M., Chatterjee, K., et al. 2022, ApJL, 924, L32, doi: 10.3847/2041-8213/ac46a1 Rosas-Guevara, Y., Bower, R. G., Schaye, J., et al. 2016, MNRAS, 462, 190, doi: 10.1093/mnras/stw1679 Russell, H. R., Fabian, A. C., McNamara, B. R., & Broderick, A. E. 2015, MNRAS, 451, 588, doi: 10.1093/mnras/stv954 Shapiro, S. L., & Teukolsky, S. A. 1983, Black holes, white dwarfs, and neutron stars : the physics of compact objects Shvartsman, V. F. 1971, Soviet Ast., 15, 377 Sijacki, D., Vogelsberger, M., Genel, S., et al. 2015, MNRAS, 452, 575, doi: 10.1093/mnras/stv1340 Talbot, R. Y., Bourne, M. A., & Sijacki, D. 2021, MNRAS, 504, 3619, doi: 10.1093/mnras/stab804 Tchekhovskoy, A., McKinney, J. C., & Narayan, R. 2012, in Journal of Physics Conference Series, Vol. 372, Journal of Physics Conference Series, 012040, doi: 10.1088/1742-6596/372/1/012040 Tchekhovskoy, A., Narayan, R., & McKinney, J. C. 2011, MNRAS, 418, L79, doi: 10.1111/j.1745-3933.2011.01147.x Trott, C. R., Lebrun-Grandi, D., Arndt, D., et al. 2022, 33, 805, doi: 10.1109/TPDS.2021.3097283 Wang, Q. D., Nowak, M. A., Markoff, S. B., et al. 2013, Science, 341, 981, doi: 10.1126/science.1240755 Weinberger, R., Springel, V., Pakmor, R., et al. 2018, MNRAS, 479, 4056, doi: 10.1093/mnras/sty1733 Weinberger, R., Su, K.-Y., Ehlert, K., et al. 2023, MNRAS, 523, 1104, doi: 10.1093/mnras/stad1396 Wellons, S., Faucher-Gigu`ere, C.-A., Hopkins, P. F., et al. 2023, MNRAS, 520, 5394, doi: 10.1093/mnras/stad511 Wong, G. N., Du, Y., Prather, B. S., & Gammie, C. F. 2021, ApJ, 914, 55, doi: 10.3847/1538-4357/abf8b8 Yuan, F., Gan, Z., Narayan, R., et al. 2015, ApJ, 804, 101, doi: 10.1088/0004-637X/804/2/101 This paper is available on Arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. Authors: (1) Hyerin Cho (조혜린), Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA; (2) Ben S. Prather, CCS-2, Los Alamos National Laboratory, PO Box 1663, Los Alamos, NM 87545, USA; (3) Ramesh Narayan, enter for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA; (4) Priyamvada Natarajan, Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA, Department of Astronomy, Yale University, Kline Tower, 266 Whitney Avenue, New Haven, CT 06511, USA and Department of Physics, Yale University, P.O. Box 208121, New Haven, CT 06520, USA; (5) Kung-Yi Su, Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA; (6) Angelo Ricarte, Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA; (7) Koushik Chatterjee, Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA. This paper is available on arxiv under CC 4.0 license. Authors : Authors (1) Hyerin Cho (조혜린), Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA; (2) Ben S. Prather, CCS-2, Los Alamos National Laboratory, PO Box 1663, Los Alamos, NM 87545, USA; (3) Ramesh Narayan, enter for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA; (4) Priyamvada Natarajan, Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA, Department of Astronomy, Yale University, Kline Tower, 266 Whitney Avenue, New Haven, CT 06511, USA and Department of Physics, Yale University, P.O. Box 208121, New Haven, CT 06520, USA; (5) Kung-Yi Su, Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA; (6) Angelo Ricarte, Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA; (7) Koushik Chatterjee, Center for Astrophysics | Harvard & Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA and Black Hole Initiative at Harvard University, 20 Garden Street, Cambridge, MA 02138, USA. Table of Links Abstract and Introduction Abstract and Introduction Numerical Methods Numerical Methods Hydrodynamic Bondi Accretion Hydrodynamic Bondi Accretion Magnetized Bondi Accretion Magnetized Bondi Accretion Feedback Via Reconnection-Driven Convection Feedback Via Reconnection-Driven Convection Summary and Conclusions Summary and Conclusions Acknowledgements Acknowledgements Appendix Appendix A. GRMHD Primer and Definitions A. GRMHD Primer and Definitions B. Numerical Set-up B. Numerical Set-up C. Resolution and Initial Condition Study C. Resolution and Initial Condition Study References References REFERENCES Angl´es-Alc´azar, D., Quataert, E., Hopkins, P. F., et al. 2021, ApJ, 917, 53, doi: 10.3847/1538-4357/ac09e8 Anile, A. M. 1989, Relativistic fluids and magneto-fluids: With applications in astrophysics and plasma physics Bisnovatyi-Kogan, G. S., & Ruzmaikin, A. A. 1974, Ap&SS, 28, 45, doi: 10.1007/BF00642237 Bondi, H. 1952, MNRAS, 112, 195, doi: 10.1093/mnras/112.2.195 Booth, C. M., & Schaye, J. 2009, MNRAS, 398, 53, doi: 10.1111/j.1365-2966.2009.15043.x Chatterjee, K., & Narayan, R. 2022, ApJ, 941, 30, doi: 10.3847/1538-4357/ac9d97 Chatterjee, K., Chael, A., Tiede, P., et al. 2023, Galaxies, 11, 38, doi: 10.3390/galaxies11020038 Dav´e, R., Angl´es-Alc´azar, D., Narayanan, D., et al. 2019, MNRAS, 486, 2827, doi: 10.1093/mnras/stz937 Event Horizon Telescope Collaboration, Akiyama, K., Algaba, J. C., et al. 2021, ApJL, 910, L13, doi: 10.3847/2041-8213/abe4de Ferrarese, L., & Merritt, D. 2000, ApJL, 539, L9, doi: 10.1086/312838 Ferri`ere, K. 2020, Plasma Physics and Controlled Fusion, 62, 014014, doi: 10.1088/13616587/ab49eb Fiacconi, D., Sijacki, D., & Pringle, J. E. 2018, MNRAS, 477, 3807, doi: 10.1093/mnras/sty893 Gammie, C. F., McKinney, J. C., & T´oth, G. 2003, ApJ, 589, 444, doi: 10.1086/374594 Gebhardt, K., Bender, R., Bower, G., et al. 2000, ApJL, 539, L13, doi: 10.1086/312840 Grete, P., Dolence, J. C., Miller, J. M., et al. 2023, 37, 465, doi: 10.1177/10943420221143775 Guerra, J. A., Lopez-Rodriguez, E., Chuss, D. T., Butterfield, N. O., & Schmelz, J. T. 2023, AJ, 166, 37, doi: 10.3847/1538-3881/acdacd Guo, M., Stone, J. M., Kim, C.-G., & Quataert, E. 2023, ApJ, 946, 26, doi: 10.3847/1538-4357/acb81e Hopkins, P. F., & Quataert, E. 2010, MNRAS, 407, 1529, doi: 10.1111/j.1365-2966.2010.17064.x Hopkins, P. F., Grudic, M. Y., Su, K.-Y., et al. 2023, arXiv e-prints, arXiv:2309.13115. https://arxiv.org/abs/2309.13115 Igumenshchev, I. V., & Narayan, R. 2002, ApJ, 566, 137, doi: 10.1086/338077 Igumenshchev, I. V., Narayan, R., & Abramowicz, M. A. 2003, ApJ, 592, 1042, doi: 10.1086/375769 Kaaz, N., Murguia-Berthier, A., Chatterjee, K., Liska, M. T. P., & Tchekhovskoy, A. 2023, ApJ, 950, 31, doi: 10.3847/1538-4357/acc7a1 Komissarov, S. S. 1999, MNRAS, 303, 343, doi: 10.1046/j.1365-8711.1999.02244.x Kormendy, J., & Ho, L. C. 2013, ARA&A, 51, 511, doi: 10.1146/annurev-astro-082708-101811 Lalakos, A., Gottlieb, O., Kaaz, N., et al. 2022, ApJL, 936, L5, doi: 10.3847/2041-8213/ac7bed Li, Y., & Bryan, G. L. 2014, ApJ, 789, 153, doi: 10.1088/0004-637X/789/2/153 Magorrian, J., Tremaine, S., Richstone, D., et al. 1998, AJ, 115, 2285, doi: 10.1086/300353 Meszaros, P. 1975, A&A, 44, 59 Michel, F. C. 1972, Ap&SS, 15, 153, doi: 10.1007/BF00649949 Narayan, R., Chael, A., Chatterjee, K., Ricarte, A., & Curd, B. 2022, MNRAS, 511, 3795, doi: 10.1093/mnras/stac285 Narayan, R., Igumenshchev, I. V., & Abramowicz, M. A. 2000, ApJ, 539, 798, doi: 10.1086/309268—. 2003, PASJ, 55, L69, doi: 10.1093/pasj/55.6.L69 Ni, Y., Di Matteo, T., Bird, S., et al. 2022, MNRAS, 513, 670, doi: 10.1093/mnras/stac351 Pen, U.-L., Matzner, C. D., & Wong, S. 2003, ApJL, 596, L207, doi: 10.1086/379339 Penna, R. F., Kulkarni, A., & Narayan, R. 2013, A&A, 559, A116, doi: 10.1051/0004-6361/201219666 Porth, O., Chatterjee, K., Narayan, R., et al. 2019, ApJS, 243, 26, doi: 10.3847/1538-4365/ab29fd Prather, B. S., Wong, G. N., Dhruv, V., et al. 2021, 6, 3336, doi: 10.21105/joss.03336 Quataert, E., & Gruzinov, A. 2000, ApJ, 539, 809, doi: 10.1086/309267 Ressler, S. M., Quataert, E., White, C. J., & Blaes, O. 2021, MNRAS, 504, 6076, doi: 10.1093/mnras/stab311 Ressler, S. M., White, C. J., Quataert, E., & Stone, J. M. 2020, ApJL, 896, L6, doi: 10.3847/2041-8213/ab9532 Ricarte, A., Tremmel, M., Natarajan, P., & Quinn, T. 2019, MNRAS, 489, 802, doi: 10.1093/mnras/stz2161 Ripperda, B., Liska, M., Chatterjee, K., et al. 2022, ApJL, 924, L32, doi: 10.3847/2041-8213/ac46a1 Rosas-Guevara, Y., Bower, R. G., Schaye, J., et al. 2016, MNRAS, 462, 190, doi: 10.1093/mnras/stw1679 Russell, H. R., Fabian, A. C., McNamara, B. R., & Broderick, A. E. 2015, MNRAS, 451, 588, doi: 10.1093/mnras/stv954 Shapiro, S. L., & Teukolsky, S. A. 1983, Black holes, white dwarfs, and neutron stars : the physics of compact objects Shvartsman, V. F. 1971, Soviet Ast., 15, 377 Sijacki, D., Vogelsberger, M., Genel, S., et al. 2015, MNRAS, 452, 575, doi: 10.1093/mnras/stv1340 Talbot, R. Y., Bourne, M. A., & Sijacki, D. 2021, MNRAS, 504, 3619, doi: 10.1093/mnras/stab804 Tchekhovskoy, A., McKinney, J. C., & Narayan, R. 2012, in Journal of Physics Conference Series, Vol. 372, Journal of Physics Conference Series, 012040, doi: 10.1088/1742-6596/372/1/012040 Tchekhovskoy, A., Narayan, R., & McKinney, J. C. 2011, MNRAS, 418, L79, doi: 10.1111/j.1745-3933.2011.01147.x Trott, C. R., Lebrun-Grandi, D., Arndt, D., et al. 2022, 33, 805, doi: 10.1109/TPDS.2021.3097283 Wang, Q. D., Nowak, M. A., Markoff, S. B., et al. 2013, Science, 341, 981, doi: 10.1126/science.1240755 Weinberger, R., Springel, V., Pakmor, R., et al. 2018, MNRAS, 479, 4056, doi: 10.1093/mnras/sty1733 Weinberger, R., Su, K.-Y., Ehlert, K., et al. 2023, MNRAS, 523, 1104, doi: 10.1093/mnras/stad1396 Wellons, S., Faucher-Gigu`ere, C.-A., Hopkins, P. F., et al. 2023, MNRAS, 520, 5394, doi: 10.1093/mnras/stad511 Wong, G. N., Du, Y., Prather, B. S., & Gammie, C. F. 2021, ApJ, 914, 55, doi: 10.3847/1538-4357/abf8b8 Yuan, F., Gan, Z., Narayan, R., et al. 2015, ApJ, 804, 101, doi: 10.1088/0004-637X/804/2/101 This paper is available on Arxiv under CC 4.0 license. This paper is available on Arxiv under CC 4.0 license. Arxiv

All the References That We Used in Our Black Hole Study So You Can Continue to Learn

Authors:
(1) Philipp Straubinger, University of Passau, Passau, Germany;
(2) Gordon Fraser, University of Passau, Passau, Germany. Table of Links Abstract and Introduction
Gamification Elements of Gamekins
Intellij Plugin for Gamekins
Conclusions
Acknowledgement and References 5 ACKNOWLEDGEMENTS This work is supported by the DFG under grant FR 2955/2-1 REFERENCES [1] André N. Meyer, Earl T. Barr, Christian Bird, and Thomas Zimmermann. 2021. Today Was a Good Day: The Daily Life of Software Developers. IEEE Trans. Software Eng. 47, 5 (2021), 863–880. https://doi.org/10.1109/TSE.2019.2904957 [2] André N. Meyer, Thomas Fritz, Gail C. Murphy, and Thomas Zimmermann. 2014. Software developers’ perceptions of productivity. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, (FSE-22), Hong Kong, China, November 16 - 22, 2014, Shing-Chi Cheung, Alessandro Orso, and Margaret-Anne D. Storey (Eds.). ACM, 19–29. https://doi.org/10.1145/2635868. 2635892 [3] Philipp Straubinger and Gordon Fraser. 2022. Gamekins: Gamifying Software Testing in Jenkins. In 44th IEEE/ACM International Conference on Software Engineering: Companion Proceedings, ICSE Companion 2022, Pittsburgh, PA, USA, May 22-24, 2022. ACM/IEEE, 85–89. https://doi.org/10.1145/3510454.3516862 [4] Philipp Straubinger and Gordon Fraser. 2024. Gamifying a Software Testing Course with Continuous Integration. In 46th IEEE/ACM International Conference on Software Engineering, ICSE 2024, Lisbon, Portugal, May 14-20, 2024. https: //doi.org/10.1145/3639474.3640054 [5] Philipp Straubinger and Gordon Fraser. 2024. Improving Testing Behavior by Gamifying IntelliJ. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, ICSE 2024, Lisbon, Portugal, April 14-20, 2024. ACM, 49:1– 49:13. https://doi.org/10.1145/3597503.3623339 This paper is available on arxiv under CC BY-SA 4.0 DEED license. Authors: (1) Philipp Straubinger, University of Passau, Passau, Germany; (2) Gordon Fraser, University of Passau, Passau, Germany. Authors: Authors: (1) Philipp Straubinger, University of Passau, Passau, Germany; (2) Gordon Fraser, University of Passau, Passau, Germany. Table of Links Abstract and Introduction Gamification Elements of Gamekins Intellij Plugin for Gamekins Conclusions Acknowledgement and References Abstract and Introduction Abstract and Introduction Gamification Elements of Gamekins Gamification Elements of Gamekins Intellij Plugin for Gamekins Intellij Plugin for Gamekins Conclusions Conclusions Acknowledgement and References Acknowledgement and References 5 ACKNOWLEDGEMENTS This work is supported by the DFG under grant FR 2955/2-1 REFERENCES [1] André N. Meyer, Earl T. Barr, Christian Bird, and Thomas Zimmermann. 2021. Today Was a Good Day: The Daily Life of Software Developers. IEEE Trans. Software Eng. 47, 5 (2021), 863–880. https://doi.org/10.1109/TSE.2019.2904957 [2] André N. Meyer, Thomas Fritz, Gail C. Murphy, and Thomas Zimmermann. 2014. Software developers’ perceptions of productivity. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, (FSE-22), Hong Kong, China, November 16 - 22, 2014, Shing-Chi Cheung, Alessandro Orso, and Margaret-Anne D. Storey (Eds.). ACM, 19–29. https://doi.org/10.1145/2635868. 2635892 [3] Philipp Straubinger and Gordon Fraser. 2022. Gamekins: Gamifying Software Testing in Jenkins. In 44th IEEE/ACM International Conference on Software Engineering: Companion Proceedings, ICSE Companion 2022, Pittsburgh, PA, USA, May 22-24, 2022. ACM/IEEE, 85–89. https://doi.org/10.1145/3510454.3516862 [4] Philipp Straubinger and Gordon Fraser. 2024. Gamifying a Software Testing Course with Continuous Integration. In 46th IEEE/ACM International Conference on Software Engineering, ICSE 2024, Lisbon, Portugal, May 14-20, 2024. https: //doi.org/10.1145/3639474.3640054 [5] Philipp Straubinger and Gordon Fraser. 2024. Improving Testing Behavior by Gamifying IntelliJ. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, ICSE 2024, Lisbon, Portugal, April 14-20, 2024. ACM, 49:1– 49:13. https://doi.org/10.1145/3597503.3623339 This paper is available on arxiv under CC BY-SA 4.0 DEED license. This paper is available on arxiv under CC BY-SA 4.0 DEED license. available on arxiv

An IDE Plugin for Gamified Continuous Integration: Acknowledgement and References

Authors:
(1) Philipp Straubinger, University of Passau, Passau, Germany;
(2) Gordon Fraser, University of Passau, Passau, Germany. Table of Links Abstract and Introduction
Gamification Elements of Gamekins
Intellij Plugin for Gamekins
Conclusions
Acknowledgement and References CONCLUSIONS The Gamekins IntelliJ plugin seamlessly incorporates the functionalities of Gamekins into the IDE, presenting essential information such as leaderboards, challenges, quests, and achievements directly within the developer’s view. This integration minimizes context switches and associated productivity losses. We are currently enhancing the achievements system of Gamekins and introducing new ones. These improvements, along with future enhancements, will be integrated into Gamekins IntelliJ plugin. Additionally, we plan to assess the plugin in our university courses to evaluate its effectiveness. Lastly, we aim to introduce new types of challenges and achievements, building upon our experience with IntelliGame. Gamekins IntelliJ plugin is available at: https://github.com/se2p/Gamekins-IntelliJ-Plugin This paper is available on arxiv under CC BY-SA 4.0 DEED license. Authors: (1) Philipp Straubinger, University of Passau, Passau, Germany; (2) Gordon Fraser, University of Passau, Passau, Germany. Authors: Authors: (1) Philipp Straubinger, University of Passau, Passau, Germany; (2) Gordon Fraser, University of Passau, Passau, Germany. Table of Links Abstract and Introduction Gamification Elements of Gamekins Intellij Plugin for Gamekins Conclusions Acknowledgement and References Abstract and Introduction Abstract and Introduction Gamification Elements of Gamekins Gamification Elements of Gamekins Intellij Plugin for Gamekins Intellij Plugin for Gamekins Conclusions Conclusions Acknowledgement and References Acknowledgement and References CONCLUSIONS The Gamekins IntelliJ plugin seamlessly incorporates the functionalities of Gamekins into the IDE, presenting essential information such as leaderboards, challenges, quests, and achievements directly within the developer’s view. This integration minimizes context switches and associated productivity losses. We are currently enhancing the achievements system of Gamekins and introducing new ones. These improvements, along with future enhancements, will be integrated into Gamekins IntelliJ plugin. Additionally, we plan to assess the plugin in our university courses to evaluate its effectiveness. Lastly, we aim to introduce new types of challenges and achievements, building upon our experience with IntelliGame. Gamekins IntelliJ plugin is available at: https://github.com/se2p/Gamekins-IntelliJ-Plugin This paper is available on arxiv under CC BY-SA 4.0 DEED license. This paper is available on arxiv under CC BY-SA 4.0 DEED license. available on arxiv

An IDE Plugin for Gamified Continuous Integration: Conclusions

Authors:
(1) Philipp Straubinger, University of Passau, Passau, Germany;
(2) Gordon Fraser, University of Passau, Passau, Germany. Table of Links Abstract and Introduction
Gamification Elements of Gamekins
Intellij Plugin for Gamekins
Conclusions
Acknowledgement and References 3 INTELLIJ PLUGIN FOR GAMEKINS The Gamekins IntelliJ plugin can be installed on any IntelliJ Community Edition or Ultimate IDE, adding a new tab on the right side of the IDE interface. Upon initial access to this tab, a login prompt will appear, prompting users to enter their username, password, URL, and project name to establish a connection with the relevant Jenkins instance. Subsequent logins are streamlined through the use of a token, eliminating the need for repeated manual logins, although users have the flexibility to log out as needed. Project and user configurations are centrally managed within Gamekins on a Jenkins server rather than within the Gamekins IntelliJ plugin. This allows for extensive collaboration within the project, and the analyses conducted by Gamekins need not be executed locally. A customized API integrated into Gamekins empowers the plugin to fetch challenges, quests, and user data from Gamekins, as well as transmit data to it. The Gamekins IntelliJ plugin tab encompasses various pages presenting all the information and gamification elements generated and provided by Gamekins, along with the buttons on the bottom to change between these pages (Fig. 1). 3.1 Challenges All challenges are visible in the Gamekins IntelliJ plugin, with the availability of different information and actions based on the challenge type. For instance, a Test Challenge solely displays information about the challenge without further interactions with IntelliJ (Fig. 1). Similarly, a Build Challenge provides information and includes an additional link to the log files in Jenkins. The Class and Method Coverage Challenges offer a button to navigate to and highlight the respective class or method. This feature is also available for Line and Branch Coverage challenges, highlighting the specified line. The Branch Coverage Challenge additionally provides information about covered and uncovered branches for the tested line. The Mutation and Smell Challenges allow developers to go to and highlight the original line of code or the start of the test smell, respectively. These challenges also have an expandable area containing details such as the mutant or a smell description. The challenges are not only visible in the Gamekins IntelliJ plugin tab but also feature yellow highlighting in the source files, aiding developers in easily identifying challenges in their current code. Hovering over the highlighted code snippet reveals a tooltip with the challenge description and a button to open the challenges page in the Gamekins IntelliJ plugin (Fig. 2). This page displays current challenges along with completed and rejected ones (Fig. 1). Developers can reject challenges with an explanation if they deem them irrelevant, prompting the generation of new challenges. Once rejected, challenges cannot be reinstated, except for Class Coverage Challenges, which can be restored using the Undo button (Fig. 1). If a developer finds a challenge interesting but unsuitable for the current workflow, they can store a configurable number of challenges. These stored challenges can also be shared with a colleague working on the same project, especially if the source code is segmented into specialized fields unknown to the participant. To address a challenge, developers simply need to commit and push their changes to their version control system, triggering Jenkins to run the CI pipeline in the background. Developers can seamlessly continue their work in the project as Gamekins automatically executes after the pipeline concludes, providing notifications upon completion. General notifications about the finished build and specific notifications for each solved and generated challenge or quest are displayed (Fig. 3) in IntelliJ. These notifications include links to view the corresponding elements in the Gamekins IntelliJ plugin. 3.2 Quests Each quest in the Gamekins IntelliJ plugin is displayed on the quests page (Fig. 4) along with its description and a progress bar. Both ongoing and completed quests are shown. The progress bar indicates the percentage of quest completion and is dynamically updated whenever relevant actions are undertaken. 3.3 Achievements Presently, the Gamekins IntelliJ plugin encompasses all achievements from Gamekins, allowing developers to access a dedicated tab to view both acquired and available achievements (Fig. 5). Each achievement consists of an icon, a title, a description, and the date and time it was awarded. 3.4 Leaderboards To incentivize developers to solve challenges and complete quests, points are awarded based on the task’s difficulty. These points are accumulated for each user and team, prominently displayed in a dedicated leaderboard within the Gamekins IntelliJ plugin tab (Fig. 6). The leaderboard showcases rankings for individual users as well as teams, emphasizing their scores in the logged-in project. Information regarding completed challenges, quests, achievements, and earned points is presented on each leaderboard. Additionally, users have the option to personalize their appearance by selecting one of 50 avatars available within Gamekins. This paper is available on arxiv under CC BY-SA 4.0 DEED license. Authors: (1) Philipp Straubinger, University of Passau, Passau, Germany; (2) Gordon Fraser, University of Passau, Passau, Germany. Authors: Authors: (1) Philipp Straubinger, University of Passau, Passau, Germany; (2) Gordon Fraser, University of Passau, Passau, Germany. Table of Links Abstract and Introduction Gamification Elements of Gamekins Intellij Plugin for Gamekins Conclusions Acknowledgement and References Abstract and Introduction Abstract and Introduction Gamification Elements of Gamekins Gamification Elements of Gamekins Intellij Plugin for Gamekins Intellij Plugin for Gamekins Conclusions Conclusions Acknowledgement and References Acknowledgement and References 3 INTELLIJ PLUGIN FOR GAMEKINS The Gamekins IntelliJ plugin can be installed on any IntelliJ Community Edition or Ultimate IDE, adding a new tab on the right side of the IDE interface. Upon initial access to this tab, a login prompt will appear, prompting users to enter their username, password, URL, and project name to establish a connection with the relevant Jenkins instance. Subsequent logins are streamlined through the use of a token, eliminating the need for repeated manual logins, although users have the flexibility to log out as needed. Project and user configurations are centrally managed within Gamekins on a Jenkins server rather than within the Gamekins IntelliJ plugin. This allows for extensive collaboration within the project, and the analyses conducted by Gamekins need not be executed locally. A customized API integrated into Gamekins empowers the plugin to fetch challenges, quests, and user data from Gamekins, as well as transmit data to it. The Gamekins IntelliJ plugin tab encompasses various pages presenting all the information and gamification elements generated and provided by Gamekins, along with the buttons on the bottom to change between these pages (Fig. 1). 3.1 Challenges All challenges are visible in the Gamekins IntelliJ plugin, with the availability of different information and actions based on the challenge type. For instance, a Test Challenge solely displays information about the challenge without further interactions with IntelliJ (Fig. 1). Similarly, a Build Challenge provides information and includes an additional link to the log files in Jenkins. The Class and Method Coverage Challenges offer a button to navigate to and highlight the respective class or method. This feature is also available for Line and Branch Coverage challenges, highlighting the specified line. The Branch Coverage Challenge additionally provides information about covered and uncovered branches for the tested line. The Mutation and Smell Challenges allow developers to go to and highlight the original line of code or the start of the test smell, respectively. These challenges also have an expandable area containing details such as the mutant or a smell description. The challenges are not only visible in the Gamekins IntelliJ plugin tab but also feature yellow highlighting in the source files, aiding developers in easily identifying challenges in their current code. Hovering over the highlighted code snippet reveals a tooltip with the challenge description and a button to open the challenges page in the Gamekins IntelliJ plugin (Fig. 2). This page displays current challenges along with completed and rejected ones (Fig. 1). Developers can reject challenges with an explanation if they deem them irrelevant, prompting the generation of new challenges. Once rejected, challenges cannot be reinstated, except for Class Coverage Challenges, which can be restored using the Undo button (Fig. 1). If a developer finds a challenge interesting but unsuitable for the current workflow, they can store a configurable number of challenges. These stored challenges can also be shared with a colleague working on the same project, especially if the source code is segmented into specialized fields unknown to the participant. To address a challenge, developers simply need to commit and push their changes to their version control system, triggering Jenkins to run the CI pipeline in the background. Developers can seamlessly continue their work in the project as Gamekins automatically executes after the pipeline concludes, providing notifications upon completion. General notifications about the finished build and specific notifications for each solved and generated challenge or quest are displayed (Fig. 3) in IntelliJ. These notifications include links to view the corresponding elements in the Gamekins IntelliJ plugin. 3.2 Quests Each quest in the Gamekins IntelliJ plugin is displayed on the quests page (Fig. 4) along with its description and a progress bar. Both ongoing and completed quests are shown. The progress bar indicates the percentage of quest completion and is dynamically updated whenever relevant actions are undertaken. 3.3 Achievements Presently, the Gamekins IntelliJ plugin encompasses all achievements from Gamekins, allowing developers to access a dedicated tab to view both acquired and available achievements (Fig. 5). Each achievement consists of an icon, a title, a description, and the date and time it was awarded. 3.4 Leaderboards To incentivize developers to solve challenges and complete quests, points are awarded based on the task’s difficulty. These points are accumulated for each user and team, prominently displayed in a dedicated leaderboard within the Gamekins IntelliJ plugin tab (Fig. 6). The leaderboard showcases rankings for individual users as well as teams, emphasizing their scores in the logged-in project. Information regarding completed challenges, quests, achievements, and earned points is presented on each leaderboard. Additionally, users have the option to personalize their appearance by selecting one of 50 avatars available within Gamekins. This paper is available on arxiv under CC BY-SA 4.0 DEED license. This paper is available on arxiv under CC BY-SA 4.0 DEED license. available on arxiv

An IDE Plugin for Gamified Continuous Integration: Intellij Plugin for Gamekins

Authors:
(1) Philipp Straubinger, University of Passau, Passau, Germany;
(2) Gordon Fraser, University of Passau, Passau, Germany. Table of Links Abstract and Introduction
Gamification Elements of Gamekins
Intellij Plugin for Gamekins
Conclusions
Acknowledgement and References 2 GAMIFICATION ELEMENTS OF GAMEKINS 2.1 Challenges Gamekins offers a variety of different challenges, which are test and quality-oriented tasks for the developer to solve. There are currently eight different types of challenges available[2]: • Build Challenge: This challenge shows the developer that the build on the CI failed and has to be fixed. • Test Challenge: This generic challenge shows that the developer has to write a new test. • Class Coverage Challenge: This challenge tasks the developer to cover more lines in a specific class. • Method Coverage Challenge: This challenge focuses on improving the coverage of a specific method. • Line Coverage Challenge: This challenge assigns the developer an uncovered line that they have to cover. • Branch Coverage Challenge: This challenge focuses on the improvement of branch coverage of a covered line. • Mutation Challenge: This challenge tasks the developer to detect a mutant generated by PIT[3] by writing a new test. • Smell Challenge: This challenge focuses on the removal of code and test smell with the help of SonarLint[4]. 2.2 Quests Since the original inception as described in prior work [3, 4], the way quests are implemented has changed. Instead of having multiple challenges solved after each other, quests are now test- and qualityrelated tasks that involve making multiple quality improvements and interacting with gamification elements. It is therefore not bound to challenges anymore, but rather on improving different test and quality-related metrics. The following types of quests are available: • Add Tests Quest: This quest tasks the developer to add a specified number of tests to the existing test suite. • Cover Branches Quest: This quest focuses on covering an additional specified number of branches with new tests. • Cover Lines Quest: This quest focuses on covering an additional specified number of lines with new tests. • Receive Challenges Quest: This quest promotes interaction with other developers by receiving a challenge from another developer. • Send Challenges Quest: This quest promotes interaction with other developers by sending a challenge to them. • Solve Achievements Quest: This quest focuses on the completion of a new achievement. • Solve Challenges Quest: This quest tasks the developer to solve a specified number of new challenges of one type. • Solve Challenges Without Rejection Quest: This quest focuses on solving a specified number of challenges regardless of their types without rejecting one in between. 2.3 Achievements Developers are rewarded based on their testing accomplishments, encompassing various achievements of differing difficulty levels. These achievements range from simple tasks like adding a test to a project to more challenging objectives such as achieving 100% coverage. Certain achievements remain hidden until they are completed and are tied to individual actions, like the addition of new tests. Achievements are triggered when users commit to the project repository. There are two main types of achievements: individual achievements, which involve completing a specific number of challenges, and project-level achievements, which are linked to reaching a particular threshold within the project. This paper is available on arxiv under CC BY-SA 4.0 DEED license. [2] Detailed information can be found in prior work [3, 4] [3] https://pitest.org/ [4] https://www.sonarsource.com/products/sonarlint/ Authors: (1) Philipp Straubinger, University of Passau, Passau, Germany; (2) Gordon Fraser, University of Passau, Passau, Germany. Authors: Authors: (1) Philipp Straubinger, University of Passau, Passau, Germany; (2) Gordon Fraser, University of Passau, Passau, Germany. Table of Links Abstract and Introduction Gamification Elements of Gamekins Intellij Plugin for Gamekins Conclusions Acknowledgement and References Abstract and Introduction Abstract and Introduction Gamification Elements of Gamekins Gamification Elements of Gamekins Intellij Plugin for Gamekins Intellij Plugin for Gamekins Conclusions Conclusions Acknowledgement and References Acknowledgement and References 2 GAMIFICATION ELEMENTS OF GAMEKINS 2.1 Challenges Gamekins offers a variety of different challenges, which are test and quality-oriented tasks for the developer to solve. There are currently eight different types of challenges available[2]: • Build Challenge: This challenge shows the developer that the build on the CI failed and has to be fixed. • Test Challenge: This generic challenge shows that the developer has to write a new test. • Build Challenge: • Class Coverage Challenge: This challenge tasks the developer to cover more lines in a specific class. • Class Coverage Challenge: • Method Coverage Challenge: This challenge focuses on improving the coverage of a specific method. • Method Coverage Challenge: • Line Coverage Challenge: This challenge assigns the developer an uncovered line that they have to cover. • Line Coverage Challenge: • Branch Coverage Challenge: This challenge focuses on the improvement of branch coverage of a covered line. • Branch Coverage Challenge: • Mutation Challenge: This challenge tasks the developer to detect a mutant generated by PIT[3] by writing a new test. • Mutation Challenge: • Smell Challenge : This challenge focuses on the removal of code and test smell with the help of SonarLint[4]. • Smell Challenge 2.2 Quests Since the original inception as described in prior work [3, 4], the way quests are implemented has changed. Instead of having multiple challenges solved after each other, quests are now test- and qualityrelated tasks that involve making multiple quality improvements and interacting with gamification elements. It is therefore not bound to challenges anymore, but rather on improving different test and quality-related metrics. The following types of quests are available: • Add Tests Quest: This quest tasks the developer to add a specified number of tests to the existing test suite. • Add Tests Quest: • Cover Branches Quest: This quest focuses on covering an additional specified number of branches with new tests. • Cover Branches Quest: • Cover Lines Quest: This quest focuses on covering an additional specified number of lines with new tests. • Receive Challenges Quest: This quest promotes interaction with other developers by receiving a challenge from another developer. • Send Challenges Quest: This quest promotes interaction with other developers by sending a challenge to them. • Send Challenges Quest: • Solve Achievements Quest: This quest focuses on the completion of a new achievement. • Solve Achievements Quest: • Solve Challenges Quest: This quest tasks the developer to solve a specified number of new challenges of one type. • Solve Challenges Quest: • Solve Challenges Without Rejection Quest: This quest focuses on solving a specified number of challenges regardless of their types without rejecting one in between. • Solve Challenges Without Rejection Quest: 2.3 Achievements Developers are rewarded based on their testing accomplishments, encompassing various achievements of differing difficulty levels. These achievements range from simple tasks like adding a test to a project to more challenging objectives such as achieving 100% coverage. Certain achievements remain hidden until they are completed and are tied to individual actions, like the addition of new tests. Achievements are triggered when users commit to the project repository. There are two main types of achievements: individual achievements, which involve completing a specific number of challenges, and project-level achievements, which are linked to reaching a particular threshold within the project. This paper is available on arxiv under CC BY-SA 4.0 DEED license. This paper is available on arxiv under CC BY-SA 4.0 DEED license. available on arxiv [2] Detailed information can be found in prior work [3, 4] [3] https://pitest.org/ [4] https://www.sonarsource.com/products/sonarlint/

An IDE Plugin for Gamified Continuous Integration: Gamification Elements of Gamekins

Authors:
(1) Philipp Straubinger, University of Passau, Passau, Germany;
(2) Gordon Fraser, University of Passau, Passau, Germany. Table of Links Abstract and Introduction
Gamification Elements of Gamekins
Intellij Plugin for Gamekins
Conclusions
Acknowledgement and References ABSTRACT Interruptions and context switches resulting from meetings, urgent tasks, emails, and queries from colleagues contribute to productivity losses in developers’ daily routines. This is particularly challenging for tasks like software testing, which are already perceived as less enjoyable, prompting developers to seek distractions. To mitigate this, applying gamification to testing activities can enhance motivation for test writing. One such gamification tool is Gamekins, which integrates challenges, quests, achievements, and leaderboards into the Jenkins CI (continuous integration) platform. However, as Gamekins is typically accessed through a browser, it introduces a context switch. This paper presents an IntelliJ plugin designed to seamlessly integrate Gamekins’ gamification elements into the IDE, aiming to minimize context switches and boost developer motivation for test writing. CCS CONCEPTS • Software and its engineering → Software testing and debugging; Integrated and visual development environments. KEYWORDS Gamification, IDE, IntelliJ, Software Testing, Continuous Integration ACM Reference Format: Philipp Straubinger and Gordon Fraser. 2024. An IDE Plugin for Gamified Continuous Integration. In 2024 First IDE Workshop (IDE ’24), April 20, 2024, Lisbon, Portugal. ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/ 3643796.3648462 1 INTRODUCTION Gamekins [3] is a tool that integrates gamification into the software testing process within the Jenkins continuous integration platform, to encourage developers to improve their testing practices. Addressing the challenge of achieving high software quality and recognizing the common lack of motivation for testing among developers, Gamekins incorporates gamification elements like challenges, quests, leaderboards, and achievements. Developers earn points by completing test-related challenges and quests, engaging in leaderboard competition, and seeking achievements, with the integration into Jenkins aiming to make gamification easily accessible to developers without additional training or resources. In a recent study [4] we integrated Gamekins into an undergraduate software testing course to motivate students to engage more with testing. This study revealed a positive correlation between students’ testing behavior and the use of Gamekins, resulting in a significant improvement in correct results compared to a nongamified cohort. Additionally, students expressed satisfaction with the tool, confirming its effectiveness as a teaching tool. However, a notable issue reported by the students in this experiment was the necessity to switch from their Integrated Development Environment (IDE) to a browser with Gamekins open, causing a productivity dip and time loss navigating to Jenkins. Such context switches, which represent a known productivity challenge [2], are a concern not only for students but also for developers, who spend approximately 50% of their work time in the IDE [1]. In other work, we established that gamification of testing is possible directly in the IDE. Our IntelliGame plugin for the IntelliJ Java IDE[1] implements a multi-level achievement system rewarding positive testing behavior [5]. In a controlled experiment with 49 participants IntelliGame demonstrated clear improvements in testing behavior, for example, causing users to write more tests, achieve higher coverage and mutation scores, run tests more frequently, and achieve functionality earlier. In this paper, we aim to use the experience of gamifying testing directly in the IDE to address the problem of context switches caused by gamification in the CI. We therefore developed a new plugin for the IntelliJ Java IDE that seamlessly integrates the functionality of Gamekins directly into the IDE, offering enhanced features to improve developers’ workflow. This paper is available on arxiv under CC BY-SA 4.0 DEED license. [1] https://www.jetbrains.com/idea/ Authors: (1) Philipp Straubinger, University of Passau, Passau, Germany; (2) Gordon Fraser, University of Passau, Passau, Germany. Authors: Authors: (1) Philipp Straubinger, University of Passau, Passau, Germany; (2) Gordon Fraser, University of Passau, Passau, Germany. Table of Links Abstract and Introduction Gamification Elements of Gamekins Intellij Plugin for Gamekins Conclusions Acknowledgement and References Abstract and Introduction Abstract and Introduction Gamification Elements of Gamekins Gamification Elements of Gamekins Intellij Plugin for Gamekins Intellij Plugin for Gamekins Conclusions Conclusions Acknowledgement and References Acknowledgement and References ABSTRACT Interruptions and context switches resulting from meetings, urgent tasks, emails, and queries from colleagues contribute to productivity losses in developers’ daily routines. This is particularly challenging for tasks like software testing, which are already perceived as less enjoyable, prompting developers to seek distractions. To mitigate this, applying gamification to testing activities can enhance motivation for test writing. One such gamification tool is Gamekins, which integrates challenges, quests, achievements, and leaderboards into the Jenkins CI (continuous integration) platform. However, as Gamekins is typically accessed through a browser, it introduces a context switch. This paper presents an IntelliJ plugin designed to seamlessly integrate Gamekins’ gamification elements into the IDE, aiming to minimize context switches and boost developer motivation for test writing. CCS CONCEPTS CCS CONCEPTS • Software and its engineering → Software testing and debugging; Integrated and visual development environments. • Software and its engineering → Software testing and debugging; Integrated and visual development environments. KEYWORDS KEYWORDS Gamification, IDE, IntelliJ, Software Testing, Continuous Integration ACM Reference Format: ACM Reference Format: Philipp Straubinger and Gordon Fraser. 2024. An IDE Plugin for Gamified Continuous Integration. In 2024 First IDE Workshop (IDE ’24), April 20, 2024, Lisbon, Portugal. ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/ 3643796.3648462 1 INTRODUCTION Gamekins [3] is a tool that integrates gamification into the software testing process within the Jenkins continuous integration platform, to encourage developers to improve their testing practices. Addressing the challenge of achieving high software quality and recognizing the common lack of motivation for testing among developers, Gamekins incorporates gamification elements like challenges, quests, leaderboards, and achievements. Developers earn points by completing test-related challenges and quests, engaging in leaderboard competition, and seeking achievements, with the integration into Jenkins aiming to make gamification easily accessible to developers without additional training or resources. In a recent study [4] we integrated Gamekins into an undergraduate software testing course to motivate students to engage more with testing. This study revealed a positive correlation between students’ testing behavior and the use of Gamekins, resulting in a significant improvement in correct results compared to a nongamified cohort. Additionally, students expressed satisfaction with the tool, confirming its effectiveness as a teaching tool. However, a notable issue reported by the students in this experiment was the necessity to switch from their Integrated Development Environment (IDE) to a browser with Gamekins open, causing a productivity dip and time loss navigating to Jenkins. Such context switches, which represent a known productivity challenge [2], are a concern not only for students but also for developers, who spend approximately 50% of their work time in the IDE [1]. In other work, we established that gamification of testing is possible directly in the IDE. Our IntelliGame plugin for the IntelliJ Java IDE[1] implements a multi-level achievement system rewarding positive testing behavior [5]. In a controlled experiment with 49 participants IntelliGame demonstrated clear improvements in testing behavior, for example, causing users to write more tests, achieve higher coverage and mutation scores, run tests more frequently, and achieve functionality earlier. In this paper, we aim to use the experience of gamifying testing directly in the IDE to address the problem of context switches caused by gamification in the CI. We therefore developed a new plugin for the IntelliJ Java IDE that seamlessly integrates the functionality of Gamekins directly into the IDE, offering enhanced features to improve developers’ workflow. This paper is available on arxiv under CC BY-SA 4.0 DEED license. This paper is available on arxiv under CC BY-SA 4.0 DEED license. available on arxiv [1] https://www.jetbrains.com/idea/

An IDE Plugin for Gamified Continuous Integration: Abstract and Introduction

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel;
(2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. Table of Links Introduction
Background
Related Work
POMCP for Stochastic Contingent Planning
Domain Independent Heuristics for POMCP
Empirical Evaluation
Conclusion and References 7 Conclusion In this paper we suggested to model goal POMDPs as stochastic contingent planning models, which allows us to use domain independent heuristics developed in the automated planning community to estimate the utility of belief states. We implemented our domain independent heuristics into the rollout mechanism of POMCP — a well known online POMDP planner that constructs a search tree to evaluate which action to take next. We provide an empirical evaluation showing how heuristics provide much leverage, especially in complex domains that require a long planning horizon, compared to the standard uniform rollout policy that is often used in POMCP. For future research we intend to integrate our methods into other solvers, such as RTDP-BEL, or into point-based planners as a method to gather good belief points. We can also experiment with additional heuristics, other than the hadd heuristic used in this paper. References [1] Alexandre Albore, Héctor Palacios & Hector Geffner (2009): A Translation-Based Approach to Contingent Planning. In: IJCAI, 9, pp. 1623–1628, doi:10.5555/1661445.1661706. Available at https://dl.acm. org/doi/10.5555/1661445.1661706. [2] Blai Bonet & Héctor Geffner (2001): Planning as heuristic search. Artificial Intelligence 129(1-2), pp. 5–33, doi:10.1016/S0004-3702(01)00108-4. [3] Blai Bonet & Hector Geffner (2009): Solving POMDPs: RTDP-Bel vs. Point-based Algorithms. In Craig Boutilier, editor: IJCAI 2009, Proceedings of the 21st International Joint Conference on Artificial Intelligence, Pasadena, California, USA, July 11-17, 2009, pp. 1641–1646, doi:10.5555/1661445.1661709. Available at https://dl.acm.org/doi/10.5555/1661445.1661709. [4] Blai Bonet & Hector Geffner (2011): Planning under Partial Observability by Classical Replanning: Theory and Experiments. In: IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, July 16-22, 2011, pp. 1936–1941, doi:10.5591/978-1-57735-516-8/IJCAI11-324. [5] Ronen I Brafman & Guy Shani (2016): Online belief tracking using regression for contingent planning. Artificial Intelligence 241, pp. 131–152, doi:10.1016/j.artint.2016.08.005. [6] Hector Geffner & Blai Bonet (2022): A concise introduction to models and methods for automated planning. Springer Nature. [7] Malte Helmert & Carmel Domshlak (2009): Landmarks, Critical Paths and Abstractions: What’s the Difference Anyway? In Lubos Brim, Stefan Edelkamp, Eric A. Hansen & Peter Sanders, editors: Graph Search Engineering, 29.11. - 04.12.2009, Dagstuhl Seminar Proceedings 09491, Schloss Dagstuhl -LeibnizZentrum für Informatik, Germany, doi:10.1609/icaps.v19i1.13370. Available at http://drops.dagstuhl. de/opus/volltexte/2010/2432/. [8] Malte Helmert, Patrik Haslum, Jörg Hoffmann & Raz Nissim (2014): Merge-and-shrink abstraction: A method for generating lower bounds in factored state spaces. Journal of the ACM (JACM) 61(3), pp. 1–63, doi:10.1145/2559951. [25] Guy Shani, Joelle Pineau & Robert Kaplow (2013): A survey of point-based POMDP solvers. Autonomous Agents and Multi-Agent Systems 27, pp. 1–51, doi:10.1007/s10458-012-9200-2. [26] William Shen, Felipe Trevizan, Sam Toyer, Sylvie Thiébaux & Lexing Xie (2019): Guiding search with generalized policies for probabilistic planning. In: Proceedings of the International Symposium on Combinatorial Search, 10, pp. 97–105, doi:10.1609/socs.v10i1.18507. [27] David Silver & Joel Veness (2010): Monte-Carlo planning in large POMDPs. Advances in neural information processing systems 23, doi:10.5555/2997046.2997137. [28] Richard D Smallwood & Edward J Sondik (1973): The optimal control of partially observable Markov processes over a finite horizon. Operations research 21(5), pp. 1071–1088, doi:10.1287/opre.21.5.1071. [29] Adhiraj Somani, Nan Ye, David Hsu & Wee Sun Lee (2013): DESPOT: Online POMDP planning with regularization. Advances in neural information processing systems 26, doi:10.1613/jair.5328. [30] Zachary Sunberg & Mykel Kochenderfer (2018): Online algorithms for POMDPs with continuous state, action, and observation spaces. In: Proceedings of the International Conference on Automated Planning and Scheduling, 28, pp. 259–263, doi:10.48550/arXiv.1709.06196. [31] Yuchen Xiao, Sammie Katt, Andreas ten Pas, Shengjian Chen & Christopher Amato (2019): Online planning for target object search in clutter under partial observability. In: 2019 International Conference on Robotics and Automation (ICRA), IEEE, pp. 8241–8247, doi:10.1109/ICRA.2019.8793494. [32] Nan Ye, Adhiraj Somani, David Hsu & Wee Sun Lee (2017): DESPOT: Online POMDP planning with regularization. Journal of Artificial Intelligence Research 58, pp. 231–266, doi:10.1613/jair.5328. This paper is available on arxiv under CC 4.0 license. Authors: (1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel; (2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel; (2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. Table of Links Introduction Background Related Work POMCP for Stochastic Contingent Planning Domain Independent Heuristics for POMCP Empirical Evaluation Conclusion and References Introduction Introduction Background Background Related Work Related Work POMCP for Stochastic Contingent Planning POMCP for Stochastic Contingent Planning Domain Independent Heuristics for POMCP Domain Independent Heuristics for POMCP Empirical Evaluation Empirical Evaluation Conclusion and References Conclusion and References 7 Conclusion In this paper we suggested to model goal POMDPs as stochastic contingent planning models, which allows us to use domain independent heuristics developed in the automated planning community to estimate the utility of belief states. We implemented our domain independent heuristics into the rollout mechanism of POMCP — a well known online POMDP planner that constructs a search tree to evaluate which action to take next. We provide an empirical evaluation showing how heuristics provide much leverage, especially in complex domains that require a long planning horizon, compared to the standard uniform rollout policy that is often used in POMCP. For future research we intend to integrate our methods into other solvers, such as RTDP-BEL, or into point-based planners as a method to gather good belief points. We can also experiment with additional heuristics, other than the h add heuristic used in this paper. add References [1] Alexandre Albore, Héctor Palacios & Hector Geffner (2009): A Translation-Based Approach to Contingent Planning. In: IJCAI, 9, pp. 1623–1628, doi:10.5555/1661445.1661706. Available at https://dl.acm. org/doi/10.5555/1661445.1661706. [2] Blai Bonet & Héctor Geffner (2001): Planning as heuristic search. Artificial Intelligence 129(1-2), pp. 5–33, doi:10.1016/S0004-3702(01)00108-4. [3] Blai Bonet & Hector Geffner (2009): Solving POMDPs: RTDP-Bel vs. Point-based Algorithms. In Craig Boutilier, editor: IJCAI 2009, Proceedings of the 21st International Joint Conference on Artificial Intelligence, Pasadena, California, USA, July 11-17, 2009, pp. 1641–1646, doi:10.5555/1661445.1661709. Available at https://dl.acm.org/doi/10.5555/1661445.1661709. [4] Blai Bonet & Hector Geffner (2011): Planning under Partial Observability by Classical Replanning: Theory and Experiments. In: IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, July 16-22, 2011, pp. 1936–1941, doi:10.5591/978-1-57735-516-8/IJCAI11-324. [5] Ronen I Brafman & Guy Shani (2016): Online belief tracking using regression for contingent planning. Artificial Intelligence 241, pp. 131–152, doi:10.1016/j.artint.2016.08.005. [6] Hector Geffner & Blai Bonet (2022): A concise introduction to models and methods for automated planning. Springer Nature. [7] Malte Helmert & Carmel Domshlak (2009): Landmarks, Critical Paths and Abstractions: What’s the Difference Anyway? In Lubos Brim, Stefan Edelkamp, Eric A. Hansen & Peter Sanders, editors: Graph Search Engineering, 29.11. - 04.12.2009, Dagstuhl Seminar Proceedings 09491, Schloss Dagstuhl -LeibnizZentrum für Informatik, Germany, doi:10.1609/icaps.v19i1.13370. Available at http://drops.dagstuhl. de/opus/volltexte/2010/2432/. [8] Malte Helmert, Patrik Haslum, Jörg Hoffmann & Raz Nissim (2014): Merge-and-shrink abstraction: A method for generating lower bounds in factored state spaces. Journal of the ACM (JACM) 61(3), pp. 1–63, doi:10.1145/2559951. [25] Guy Shani, Joelle Pineau & Robert Kaplow (2013): A survey of point-based POMDP solvers. Autonomous Agents and Multi-Agent Systems 27, pp. 1–51, doi:10.1007/s10458-012-9200-2. [26] William Shen, Felipe Trevizan, Sam Toyer, Sylvie Thiébaux & Lexing Xie (2019): Guiding search with generalized policies for probabilistic planning. In: Proceedings of the International Symposium on Combinatorial Search, 10, pp. 97–105, doi:10.1609/socs.v10i1.18507. [27] David Silver & Joel Veness (2010): Monte-Carlo planning in large POMDPs. Advances in neural information processing systems 23, doi:10.5555/2997046.2997137. [28] Richard D Smallwood & Edward J Sondik (1973): The optimal control of partially observable Markov processes over a finite horizon. Operations research 21(5), pp. 1071–1088, doi:10.1287/opre.21.5.1071. [29] Adhiraj Somani, Nan Ye, David Hsu & Wee Sun Lee (2013): DESPOT: Online POMDP planning with regularization. Advances in neural information processing systems 26, doi:10.1613/jair.5328. [30] Zachary Sunberg & Mykel Kochenderfer (2018): Online algorithms for POMDPs with continuous state, action, and observation spaces. In: Proceedings of the International Conference on Automated Planning and Scheduling, 28, pp. 259–263, doi:10.48550/arXiv.1709.06196. [31] Yuchen Xiao, Sammie Katt, Andreas ten Pas, Shengjian Chen & Christopher Amato (2019): Online planning for target object search in clutter under partial observability. In: 2019 International Conference on Robotics and Automation (ICRA), IEEE, pp. 8241–8247, doi:10.1109/ICRA.2019.8793494. [32] Nan Ye, Adhiraj Somani, David Hsu & Wee Sun Lee (2017): DESPOT: Online POMDP planning with regularization. Journal of Artificial Intelligence Research 58, pp. 231–266, doi:10.1613/jair.5328.

Rollout Heuristics for Online Stochastic Contingent Planning: Conclusion and References

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel;
(2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. Table of Links Introduction
Background
Related Work
POMCP for Stochastic Contingent Planning
Domain Independent Heuristics for POMCP
Empirical Evaluation
Conclusion and References 2 Background We now provide the required background on POMDPs, contingent planning, domain-independent heuristics, and the POMCP algorithm. 2.1 POMDPs Because the state of a POMDP is partially observable, the agent typically does not know what the true underlying state of the world is. Hence, it can maintain a belief state b, which is a distribution over S, i.e., b(s) is the likelihood that s is the current state. When a goal belief is reached, that is sums∈Gb(s) = 1, then the agent is sure that it is at a goal state, and the execution terminates. A solution to a POMDP, called a policy, is a function that assigns an action to every belief state. The optimal policy minimizes the expected cost, i.e., the expected number of steps before a goal belief has been reached. 2.2 POMCP The Partially Observable Monte-Carlo Planning (POMCP) online re-planning approach uses a Monte Carlo tree search (MCTS) approach to select the next action to execute [27]. At each re-planning episode POMCP constructs a tree, where the root node is the current belief state. Then, POMCP runs forward simulations, where a state is sampled from the current belief, and actions are chosen using an exploration strategy that initially selects actions that were not executed a sufficient amount of times, but gradually moves to select the seemingly best action. Observations in the simulations are selected based on the current state-action observation distribution. When reaching a leaf node, POMCP begins a so-called rollout. This rollout is designed to provide a value estimate for the leaf, based on some predefined rollout policy. The value of the leaf is updated using the outcome of the rollout, and then the values of the nodes along the branch that were visited during the simulation are updated given their descendants. Obviously, the values of all nodes in the tree are hence highly dependent on the values obtained by the rollout policy. POMCP is an anytime algorithm, that is, it continues to run simulations until a timeout, and then returns the action that seems best at the root of the search tree. POMCP was designed for large problems. Hence, POMCP does not maintain and update a belief state explicitly. Instead, POMCP uses a particle filter approach, where a set of states is sampled at the initial belief, and this set is progressed through the tree. 2.3 Contingent Planning under Partial Observability Preconditions allow us to restrict our attention only to applicable actions. An action is applicable in a belief b is all possible states in b satisfy the preconditions of the action. Obviously, one can avoid specifying preconditions for actions, allowing for actions that can always be executed, as is typically the case in flat POMDP representations. However, in domains with many actions, preconditions are a useful tool for drastically limiting the amount of actions that should be considered. 2.4 Regression-based Belief Maintenance Updating a belief can be costly. Alternatively, one can avoid the computation of new formulas representing the updated belief, by maintaining only the initial belief formula, and the history —the sequence of executed actions and sensed observations [5]. When the agent needs to query whether the preconditions of an action or the goal hold at the current node, the formula is regressed [20] through the action observation sequence back towards the initial belief. Then, one can apply SAT queries to check whether the query formula holds. We now briefly review the regression process for deterministic actions. This approach can be highly useful for larger POMDPs, complementing the particle filter approach used in POMCP. Regression maintains the equivalence of the formula [20, 5]. For any two formulas φ1 and φ2 we have: Hence, we can produce a regression over formulas, and compare the regressed formulas, making conclusions about the original formulas. The regression can be recursively applied to a sequence of actions and observations (history) h as follows: where ε is the empty sequence. This allows us to perform a regression of a formula through an entire plan, and analyze the required conditions for a plan to apply. In addition, the regression mechanism of [5] maintains a cached list of fluents F(n) that are known to hold at node n, given the action effects or observations. Following an actuation action a, F(a(n)) contains all fluents in F(n) that were not modified by a, as well as effects of a that are not conditioned on hidden fluents. For a sensing action revealing the value l, F(a(n,l)) = F(n) ∪ {l}. During future regression queries, when a value at a particular node becomes known, e.g. when regressing a later observation, it is added to F(n). All fluents p such that p ∈/ F(n)∧ ¬p ∈/ F(n) are said to be hidden at n. The cached list is useful for simplifying future regressed formulas. This paper is available on arxiv under CC 4.0 license. Authors: (1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel; (2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel; (2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. Table of Links Introduction Background Related Work POMCP for Stochastic Contingent Planning Domain Independent Heuristics for POMCP Empirical Evaluation Conclusion and References Introduction Introduction Background Background Related Work Related Work POMCP for Stochastic Contingent Planning POMCP for Stochastic Contingent Planning Domain Independent Heuristics for POMCP Domain Independent Heuristics for POMCP Empirical Evaluation Empirical Evaluation Conclusion and References Conclusion and References 2 Background We now provide the required background on POMDPs, contingent planning, domain-independent heuristics, and the POMCP algorithm. 2.1 POMDPs Because the state of a POMDP is partially observable, the agent typically does not know what the true underlying state of the world is. Hence, it can maintain a belief state b, which is a distribution over S, i.e., b(s) is the likelihood that s is the current state. When a goal belief is reached, that is sums∈Gb(s) = 1, then the agent is sure that it is at a goal state, and the execution terminates. A solution to a POMDP, called a policy, is a function that assigns an action to every belief state. The optimal policy minimizes the expected cost, i.e., the expected number of steps before a goal belief has been reached. 2.2 POMCP The Partially Observable Monte-Carlo Planning (POMCP) online re-planning approach uses a Monte Carlo tree search (MCTS) approach to select the next action to execute [27]. At each re-planning episode POMCP constructs a tree, where the root node is the current belief state. Then, POMCP runs forward simulations, where a state is sampled from the current belief, and actions are chosen using an exploration strategy that initially selects actions that were not executed a sufficient amount of times, but gradually moves to select the seemingly best action. Observations in the simulations are selected based on the current state-action observation distribution. When reaching a leaf node, POMCP begins a so-called rollout. This rollout is designed to provide a value estimate for the leaf, based on some predefined rollout policy. The value of the leaf is updated using the outcome of the rollout, and then the values of the nodes along the branch that were visited during the simulation are updated given their descendants. Obviously, the values of all nodes in the tree are hence highly dependent on the values obtained by the rollout policy. POMCP is an anytime algorithm, that is, it continues to run simulations until a timeout, and then returns the action that seems best at the root of the search tree. POMCP was designed for large problems. Hence, POMCP does not maintain and update a belief state explicitly. Instead, POMCP uses a particle filter approach, where a set of states is sampled at the initial belief, and this set is progressed through the tree. 2.3 Contingent Planning under Partial Observability Preconditions allow us to restrict our attention only to applicable actions. An action is applicable in a belief b is all possible states in b satisfy the preconditions of the action. Obviously, one can avoid specifying preconditions for actions, allowing for actions that can always be executed, as is typically the case in flat POMDP representations. However, in domains with many actions, preconditions are a useful tool for drastically limiting the amount of actions that should be considered. 2.4 Regression-based Belief Maintenance Updating a belief can be costly. Alternatively, one can avoid the computation of new formulas representing the updated belief, by maintaining only the initial belief formula, and the history —the sequence of executed actions and sensed observations [5]. When the agent needs to query whether the preconditions of an action or the goal hold at the current node, the formula is regressed [20] through the action observation sequence back towards the initial belief. Then, one can apply SAT queries to check whether the query formula holds. We now briefly review the regression process for deterministic actions. This approach can be highly useful for larger POMDPs, complementing the particle filter approach used in POMCP. Regression maintains the equivalence of the formula [20, 5]. For any two formulas φ1 and φ2 we have: Hence, we can produce a regression over formulas, and compare the regressed formulas, making conclusions about the original formulas. The regression can be recursively applied to a sequence of actions and observations (history) h as follows: where ε is the empty sequence. This allows us to perform a regression of a formula through an entire plan, and analyze the required conditions for a plan to apply. In addition, the regression mechanism of [5] maintains a cached list of fluents F(n) that are known to hold at node n, given the action effects or observations. Following an actuation action a, F(a(n)) contains all fluents in F(n) that were not modified by a, as well as effects of a that are not conditioned on hidden fluents. For a sensing action revealing the value l, F(a(n,l)) = F(n) ∪ {l}. During future regression queries, when a value at a particular node becomes known, e.g. when regressing a later observation, it is added to F(n). All fluents p such that p ∈/ F(n)∧ ¬p ∈/ F(n) are said to be hidden at n. The cached list is useful for simplifying future regressed formulas.

Rollout Heuristics for Online Stochastic Contingent Planning: Background

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel;
(2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. Table of Links Introduction
Background
Related Work
POMCP for Stochastic Contingent Planning
Domain Independent Heuristics for POMCP
Empirical Evaluation
Conclusion and References 6 Empirical Evaluation We conduct an empirical study to evaluate our methods. Our methods are implemented in C#. 6.1 Benchmark Domains We extended the following contingent planning benchmarks to stochastic settings: Doors: In the door domain the agent must move in a grid to reach a target position. Odd levels in the grid are all open, while in even levels there are doors, and only one door is open. The agent can sense whether a door is open when it is at adjacent cells. The agent must identify the open doors and get to the target position. In the stochastic version the agent can open a closed door with some probability of success. The agent can hence either search for the already open door, or attempt to open a closed door. Blocks World: In the contingent blocks world problem, the agent does not know the structure of the initial block configuration, but it can sense whether one block is on top of another one, and whether a block is clear. In the stochastic version moving a block from one block to another has a 0.3 probability of success, while moving blocks to and from the table succeeds deterministicly. Hence, it is often preferable to use the table as an intermediate position. Unix: In this domain the agent must search for a file in a file system, and copy it to a destination folder. In the stochastic version there is a non-uniform distribution over the possible locations of the file. MedPks: The agent here needs to identify which illness a patient has and treat it. To do so, the agent tests for each illness independently, until the proper illness is found. The stochastic version here has non-uniform distribution over the possible illnesses as well. Localize: In this domain the agent must reach a goal position in a grid. The agent does not know where it initially is, and can only sense adjacent walls. In the stochastic version there several places in the grid where the agent may slip and stay in place. This makes the localization in the grid more difficult. Wumpus: In this challenging problem the agent must reach a target position in a grid infested by monsters called Wumpuses. Cells may be unsafe to travel as they may contain either a Wumpus or a pit. Wumpuses emit a stench, and pits emit a breeze, both of which can be sensed in adjacent cells. The agent must sense in multiple positions to identify the safe cells. The stochastic version here also has non-uniform distribution over the safe cells. 6.2 Results For each domain above we run 20 online episodes, and compute the success rate, the average run time for computing the next action, and the average cost to the goal. We did not use a timeout, but runs longer than 100 steps were considered to be stuck in a loop, and terminated. We begin by looking at the quality of the policy — the average cost to the goal. As can be seen, the random rollout policy is best only in the MedPks domain, and close to best in unix. This is not too surprising, because in these two domains the best strategy is very simple, and random strategies easily stumble upon the goal. In blocks all methods achieved similar performance, because the optimal strategy is very short, and rollouts are less important. This domain has many possible actions, and hence a huge branching factor, making it difficult to scale up using POMCP. On doors and localize, which require lengthier trajectories to reach the goal, but do not need long information-gathering efforts, the single state hadd strategy operates very well. However, on Wumpus, where long sequences of actions are needed for information gathering, the multiple-state heuristic works best. We expected the results to be that way, although we expected more significant difference between the "smart" heuristics and the random rollout. With respect to the required time to run the simulations for a single decision, the results are mixed. While obviously the random strategy requires no time to compute the next action during a rollout, it often results in lengthy rollouts, which reduce this effect. The single state heuristic is almost always faster than the multi-state heuristic, but not by much. This paper is available on arxiv under CC 4.0 license. Authors: (1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel; (2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel; (2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. Table of Links Introduction Background Related Work POMCP for Stochastic Contingent Planning Domain Independent Heuristics for POMCP Empirical Evaluation Conclusion and References Introduction Introduction Background Background Related Work Related Work POMCP for Stochastic Contingent Planning POMCP for Stochastic Contingent Planning Domain Independent Heuristics for POMCP Domain Independent Heuristics for POMCP Empirical Evaluation Empirical Evaluation Conclusion and References Conclusion and References 6 Empirical Evaluation We conduct an empirical study to evaluate our methods. Our methods are implemented in C#. 6.1 Benchmark Domains We extended the following contingent planning benchmarks to stochastic settings: Doors : In the door domain the agent must move in a grid to reach a target position. Odd levels in the grid are all open, while in even levels there are doors, and only one door is open. The agent can sense whether a door is open when it is at adjacent cells. The agent must identify the open doors and get to the target position. In the stochastic version the agent can open a closed door with some probability of success. The agent can hence either search for the already open door, or attempt to open a closed door. Doors Blocks World: In the contingent blocks world problem, the agent does not know the structure of the initial block configuration, but it can sense whether one block is on top of another one, and whether a block is clear. In the stochastic version moving a block from one block to another has a 0.3 probability of success, while moving blocks to and from the table succeeds deterministicly. Hence, it is often preferable to use the table as an intermediate position. Unix: In this domain the agent must search for a file in a file system, and copy it to a destination folder. In the stochastic version there is a non-uniform distribution over the possible locations of the file. MedPks : The agent here needs to identify which illness a patient has and treat it. To do so, the agent tests for each illness independently, until the proper illness is found. The stochastic version here has non-uniform distribution over the possible illnesses as well. MedPks Localize : In this domain the agent must reach a goal position in a grid. The agent does not know where it initially is, and can only sense adjacent walls. In the stochastic version there several places in the grid where the agent may slip and stay in place. This makes the localization in the grid more difficult. Localize Wumpus : In this challenging problem the agent must reach a target position in a grid infested by monsters called Wumpuses. Cells may be unsafe to travel as they may contain either a Wumpus or a pit. Wumpuses emit a stench, and pits emit a breeze, both of which can be sensed in adjacent cells. The agent must sense in multiple positions to identify the safe cells. The stochastic version here also has non-uniform distribution over the safe cells. Wumpus 6.2 Results For each domain above we run 20 online episodes, and compute the success rate, the average run time for computing the next action, and the average cost to the goal. We did not use a timeout, but runs longer than 100 steps were considered to be stuck in a loop, and terminated. We begin by looking at the quality of the policy — the average cost to the goal. As can be seen, the random rollout policy is best only in the MedPks domain, and close to best in unix. This is not too surprising, because in these two domains the best strategy is very simple, and random strategies easily stumble upon the goal. In blocks all methods achieved similar performance, because the optimal strategy is very short, and rollouts are less important. This domain has many possible actions, and hence a huge branching factor, making it difficult to scale up using POMCP. On doors and localize, which require lengthier trajectories to reach the goal, but do not need long information-gathering efforts, the single state h add strategy operates very well. However, on Wumpus, where long sequences of actions are needed for information gathering, the multiple-state heuristic works best. We expected the results to be that way, although we expected more significant difference between the "smart" heuristics and the random rollout. add With respect to the required time to run the simulations for a single decision, the results are mixed. While obviously the random strategy requires no time to compute the next action during a rollout, it often results in lengthy rollouts, which reduce this effect. The single state heuristic is almost always faster than the multi-state heuristic, but not by much.

Rollout Heuristics for Online Stochastic Contingent Planning: Empirical Evaluation

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel;
(2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. Table of Links Introduction
Background
Related Work
POMCP for Stochastic Contingent Planning
Domain Independent Heuristics for POMCP
Empirical Evaluation
Conclusion and References 4 POMCP for Stochastic Contingent Planning We focus here on goal-oriented POMDP domains specified as stochastic contingent planning problems. We now define this concept formally. This definition does not support noisy observations, however, this is not truly a limitation. One can compile a noisy observation into a deterministic observation over an artificial fact whose value changes stochastically. Consider, for example, a sensor that nosily detects whether there is a wall in front of a robot. Instead of noisily observing whether there is a wall, we can deterministically detect a green light that is lit when the sensor (stochastically) detects a wall. That is, we can observe the green light without noise, but the green light is only noisily correlated with the existence of a wall. Algorithm 1 describes the POMCP implementation for stochastic contingent planning problems. When the agent needs to act, it calls Search. Search repeatedly samples a state (line 3-4) and simulates forward execution given this state is the true underlying system state. The Simulate procedure is recursive. We first check whether the current tree node is a goal belief. This is done be regressing the negation of the goal formula ¬G through the history of the current node. For goal beliefs, the value is 0, and we can stop. Our implementation of POMCP also stops deepening the tree after a predefined threshold Max TreeDepth. If that threshold is reached (line 12), we run a Rollout to compute an estimation for the cost of reaching the goal from this node (line 13). In line 15 we check whether this node has already been expanded, and if not, we compute its children. We do so only for applicable actions whose preconditions are satisfied in the current belief (line 17). Again, this is computed using regression over the history. We now select an action a using the UCT exploration-exploitation criterion (line 21), and sample a next state and an observation (lines 22-27). We call Simulate recursively in line 28. Lines 29-32 update the value of the current node. Lines 28,29 update the counters for the executed action and received observation. Line 31 computes the value for the action as a weighted average over all observations. Line 32 computes the value of the node as the minimal cost among all actions. Our value update, which we empirically found to be more useful, is different than the original POMCP, which uses incremental updates, and more similar to the value update in DESPOT [18]. The Rollout procedure receives as input the current simulated state s, as well as a set B of states (particles) in the node from which the rollout begins. B is used by some of our rollout heuristics, as we explain below. The rollout executes actions given the heuristic rollout policy until the goal has been reached, or a maximal number of steps has been reached. This paper is available on arxiv under CC 4.0 license. Authors: (1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel; (2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel; (2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. Table of Links Introduction Background Related Work POMCP for Stochastic Contingent Planning Domain Independent Heuristics for POMCP Empirical Evaluation Conclusion and References Introduction Introduction Background Background Related Work Related Work POMCP for Stochastic Contingent Planning POMCP for Stochastic Contingent Planning Domain Independent Heuristics for POMCP Domain Independent Heuristics for POMCP Empirical Evaluation Empirical Evaluation Conclusion and References Conclusion and References 4 POMCP for Stochastic Contingent Planning We focus here on goal-oriented POMDP domains specified as stochastic contingent planning problems. We now define this concept formally. This definition does not support noisy observations, however, this is not truly a limitation. One can compile a noisy observation into a deterministic observation over an artificial fact whose value changes stochastically. Consider, for example, a sensor that nosily detects whether there is a wall in front of a robot. Instead of noisily observing whether there is a wall, we can deterministically detect a green light that is lit when the sensor (stochastically) detects a wall. That is, we can observe the green light without noise, but the green light is only noisily correlated with the existence of a wall. Algorithm 1 describes the POMCP implementation for stochastic contingent planning problems. When the agent needs to act, it calls Search. Search repeatedly samples a state (line 3-4) and simulates forward execution given this state is the true underlying system state. The Simulate procedure is recursive. We first check whether the current tree node is a goal belief. This is done be regressing the negation of the goal formula ¬G through the history of the current node. For goal beliefs, the value is 0, and we can stop. Our implementation of POMCP also stops deepening the tree after a predefined threshold Max TreeDepth. If that threshold is reached (line 12), we run a Rollout to compute an estimation for the cost of reaching the goal from this node (line 13). In line 15 we check whether this node has already been expanded, and if not, we compute its children. We do so only for applicable actions whose preconditions are satisfied in the current belief (line 17). Again, this is computed using regression over the history. We now select an action a using the UCT exploration-exploitation criterion (line 21), and sample a next state and an observation (lines 22-27). We call Simulate recursively in line 28. Lines 29-32 update the value of the current node. Lines 28,29 update the counters for the executed action and received observation. Line 31 computes the value for the action as a weighted average over all observations. Line 32 computes the value of the node as the minimal cost among all actions. Our value update, which we empirically found to be more useful, is different than the original POMCP, which uses incremental updates, and more similar to the value update in DESPOT [18]. The Rollout procedure receives as input the current simulated state s, as well as a set B of states (particles) in the node from which the rollout begins. B is used by some of our rollout heuristics, as we explain below. The rollout executes actions given the heuristic rollout policy until the goal has been reached, or a maximal number of steps has been reached.

Rollout Heuristics: POMCP for Stochastic Contingent Planning

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel;
(2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. Table of Links Introduction
Background
Related Work
POMCP for Stochastic Contingent Planning
Domain Independent Heuristics for POMCP
Empirical Evaluation
Conclusion and References 3 Related Work Augmenting MCTS methods with heuristics in the context of fully observable MDPs was previously suggested. [14] describe an MCTS tree based approach that uses planning based heuristics. The PROST planner [13] uses a heuristic for estimating the value of states. The DP-UCT approach [26] use a planning heuristic based on deep learning for the rollout phase. We are not aware of previous attempts to adapt these approaches to POMDPs. There were several extensions suggested for POMCP [16]. For example [17] considers dynamic environments, and [11] consider non-linear dynamics. All these methods rely on rollouts, and can hence leverage the rollout strategies that we suggest here. POMCPOW [30] extends POMCP to the challenge of solving POMDPs with continuous state, action, and observation spaces. It constructs the search tree incrementally to explore additional regions of the observation and action spaces. It also requires a rollout policy to evaluate the utility of leaf nodes. Our rollout strategies relay on classical planning approaches which are discrete, and it is hence unlikely that our methods can be extended to continuous domains. DESPOT [29, 32, 18] is an online POMDP solver based on tree search, similar to POMCP. DESPOT uses a different strategy than the UCB rule for constructing the tree, designed to avoid the overly greedy nature of POMCP exploration strategy. DESPOT also requires a so-called default policy to evaluate the utility of a leaf in the tree, and the authors stress the importance of a strong default policy to improve the convergence. Thus, our methods can be directly applied to DESPOT as well. We chose here to focus on POMCP rather than DESPOT, because POMCP is a simpler method, which allows us to better focus on the importance of the heuristic function, independent of the effect of the various augmentations that DESPOT adds on top of POMCP. [22] suggest a method called PSG to evaluate the proximity of states to the goal. They suggest using PSG in several places within POMCP including rollouts. PSG assumes that states are defined in a factored manner using state features, and computes a function from features to the goal using subgoals. In essence, their approach can be considered as a type of heuristic, which is highly related to the concept of landmarks in classical planning [19]. Representing the POMDP as a stochastic contingent planning problem, as we do, allows us to use any heuristic developed in the planning community, and can hence be considered to be an extension of PSG. Similarly, [31] also find it difficult to provide good rollout strategies to compute the value of POMCP leaves. Focusing on a robotics motion planning domain, they suggest SVE, state value estimator, that attempts to evaluate the utility of a state directly. [15] also focus on the need to use heuristics for guiding search in POMDPs. They focus on RTDPBEL [4], an algorithm that runs forward trajectories in belief space to produce a policy. They show that using a heuristic can significantly improve RTDP-BEL. They use domain specific heuristics, and as such, our domain independent approach can also be applied to their methods. This paper is available on arxiv under CC 4.0 license. Authors: (1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel; (2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel; (2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. Table of Links Introduction Background Related Work POMCP for Stochastic Contingent Planning Domain Independent Heuristics for POMCP Empirical Evaluation Conclusion and References Introduction Introduction Background Background Related Work Related Work POMCP for Stochastic Contingent Planning POMCP for Stochastic Contingent Planning Domain Independent Heuristics for POMCP Domain Independent Heuristics for POMCP Empirical Evaluation Empirical Evaluation Conclusion and References Conclusion and References 3 Related Work Augmenting MCTS methods with heuristics in the context of fully observable MDPs was previously suggested. [14] describe an MCTS tree based approach that uses planning based heuristics. The PROST planner [13] uses a heuristic for estimating the value of states. The DP-UCT approach [26] use a planning heuristic based on deep learning for the rollout phase. We are not aware of previous attempts to adapt these approaches to POMDPs. There were several extensions suggested for POMCP [16]. For example [17] considers dynamic environments, and [11] consider non-linear dynamics. All these methods rely on rollouts, and can hence leverage the rollout strategies that we suggest here. POMCPOW [30] extends POMCP to the challenge of solving POMDPs with continuous state, action, and observation spaces. It constructs the search tree incrementally to explore additional regions of the observation and action spaces. It also requires a rollout policy to evaluate the utility of leaf nodes. Our rollout strategies relay on classical planning approaches which are discrete, and it is hence unlikely that our methods can be extended to continuous domains. DESPOT [29, 32, 18] is an online POMDP solver based on tree search, similar to POMCP. DESPOT uses a different strategy than the UCB rule for constructing the tree, designed to avoid the overly greedy nature of POMCP exploration strategy. DESPOT also requires a so-called default policy to evaluate the utility of a leaf in the tree, and the authors stress the importance of a strong default policy to improve the convergence. Thus, our methods can be directly applied to DESPOT as well. We chose here to focus on POMCP rather than DESPOT, because POMCP is a simpler method, which allows us to better focus on the importance of the heuristic function, independent of the effect of the various augmentations that DESPOT adds on top of POMCP. [22] suggest a method called PSG to evaluate the proximity of states to the goal. They suggest using PSG in several places within POMCP including rollouts. PSG assumes that states are defined in a factored manner using state features, and computes a function from features to the goal using subgoals. In essence, their approach can be considered as a type of heuristic, which is highly related to the concept of landmarks in classical planning [19]. Representing the POMDP as a stochastic contingent planning problem, as we do, allows us to use any heuristic developed in the planning community, and can hence be considered to be an extension of PSG. Similarly, [31] also find it difficult to provide good rollout strategies to compute the value of POMCP leaves. Focusing on a robotics motion planning domain, they suggest SVE, state value estimator, that attempts to evaluate the utility of a state directly. [15] also focus on the need to use heuristics for guiding search in POMDPs. They focus on RTDPBEL [4], an algorithm that runs forward trajectories in belief space to produce a policy. They show that using a heuristic can significantly improve RTDP-BEL. They use domain specific heuristics, and as such, our domain independent approach can also be applied to their methods.

Rollout Heuristics for Online Stochastic Contingent Planning: Related Work

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel;
(2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. Table of Links Introduction
Background
Related Work
POMCP for Stochastic Contingent Planning
Domain Independent Heuristics for POMCP
Empirical Evaluation
Conclusion and References Partially observable Markov decision processes (POMDP) are a useful model for decision-making under partial observability and stochastic actions. Partially Observable Monte-Carlo Planning is an online algorithm for deciding on the next action to perform, using a Monte-Carlo tree search approach, based on the UCT (UCB applied to trees) algorithm for fully observable Markov-decision processes. POMCP develops an action-observation tree, and at the leaves, uses a rollout policy to provide a value estimate for the leaf. As such, POMCP is highly dependent on the rollout policy to compute good estimates, and hence identify good actions. Thus, many practitioners who use POMCP are required to create strong, domain-specific heuristics. In this paper, we model POMDPs as stochastic contingent planning problems. This allows us to leverage domain-independent heuristics that were developed in the planning community. We suggest two heuristics, the first is based on the well-known hadd heuristic from classical planning, and the second is computed in belief space, taking the value of information into account. 1 Introduction Many autonomous agents operate in environments where actions have stochastic effects, and important information that is required for obtaining the goal is hidden from the agent. Agents in such environments typically execute actions and sense some observations that result from these actions. Based on the accumulated observations the agents can better estimate their current state and decide on the next action to execute. Such environments are often modeled as partially observable Markov decision processes (POMDPs) [28]. POMPD models allow us to reason about the hidden state of the system, typically using a belief state – a distribution over the possible environment states. The belief state can be updated given the executed action and the received observation. One can compute a policy, a mapping from beliefs to actions, that dictates which action should be executed given the current belief. Many algorithms were suggested for computing such policies [25]. However, in larger environments, it often becomes difficult to maintain a belief state, let alone compute a policy for all possible belief states. In such cases, one can use an online re-planning approach, where after every action is executed, the agent computes which action to execute next. Such online approaches replace the lengthy single policy computation which is done offline, before the agent begins to act, with a sequence of shorter computations, which are executed online, during execution, after each action [21]. POMCP [27] is such an online replanning approach, extending the UCT algorithm for fully observable Markov decision processes (MDPs) to POMDPs. POMCP operates by constructing online a search tree, interleaving decision and observation nodes. The root of the tree is a decision node. Each decision node has an outgoing edge for every possible action, ending at an observation node. Then, the outgoing edges from an observation node denote the possible observations that result from the incoming action. The agent computes a value for each node in the tree, and then, the agent can choose, from the root node, the action associated with the edge leading to the highest value child node. To evaluate the value of leaf nodes, POMCP executes a random walk in belief space, known as a rollout, where the agent selects actions from some rollout policy to construct a trajectory in belief space and obtain an estimation of the quality of the leaf node. Clearly, this evaluation is highly dependant on the ability of the rollout policy to reach the agent goals. In complex problems, obtaining the goal may require a lengthy sequence of actions [16], and until the goal is reached, no meaningful rewards are obtained. Indeed, practitioners that use POMCP often implement complex domain-specific heuristics for the rollout policy. In this paper we focus on suggesting domain-independent heuristics for rollout policies. We leverage work in automated planning, using heuristics defined for classical and contingent planning problems [7, 8, 6]. We thus represent POMDP problems in a structured manner, using boolean facts to capture the state of the environments. This allows both for a compact representation of large problems, compared with standard flat representations that do not scale, as well as the ability to use classical planning heuristics. We begin by suggesting using the well-known hadd heuristic for choosing rollout actions [2]. This heuristic searches forward in a delete relaxation setting, until the goal has been reached. Then, the value of an action is determined by the number of steps in the delete relaxation space following the action, required for obtaining the goal. Next, we observe that any state-based rollout policy is inherently limited in its ability to evaluate the missing information required for reaching the goal, and hence, provide some estimate as to the value of information [12] of an action. We hence suggest a multi-state rollout policy, where actions are executed on a set of states jointly, and observations are used to eliminate states that are incompatible with the observed value. We show that this heuristic is much more informed in domains that require complex information-gathering strategies. For an empirical evaluation, we extend domains from the contingent planning community with stochastic effects. We evaluate our heuristics, comparing them to random rollouts, showing that they allow us to provide significantly better behavior. This paper is available on arxiv under CC 4.0 license. Authors: (1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel; (2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Oded Blumenthal, Software and Information Systems Engineering, Ben Gurion University, Israel; (2) Guy Shani, Software and Information Systems Engineering, Ben Gurion University, Israel. Table of Links Introduction Background Related Work POMCP for Stochastic Contingent Planning Domain Independent Heuristics for POMCP Empirical Evaluation Conclusion and References Introduction Introduction Background Background Related Work Related Work POMCP for Stochastic Contingent Planning POMCP for Stochastic Contingent Planning Domain Independent Heuristics for POMCP Domain Independent Heuristics for POMCP Empirical Evaluation Empirical Evaluation Conclusion and References Conclusion and References Partially observable Markov decision processes (POMDP) are a useful model for decision-making under partial observability and stochastic actions. Partially Observable Monte-Carlo Planning is an online algorithm for deciding on the next action to perform, using a Monte-Carlo tree search approach, based on the UCT (UCB applied to trees) algorithm for fully observable Markov-decision processes. POMCP develops an action-observation tree, and at the leaves, uses a rollout policy to provide a value estimate for the leaf. As such, POMCP is highly dependent on the rollout policy to compute good estimates, and hence identify good actions. Thus, many practitioners who use POMCP are required to create strong, domain-specific heuristics. In this paper, we model POMDPs as stochastic contingent planning problems. This allows us to leverage domain-independent heuristics that were developed in the planning community. We suggest two heuristics, the first is based on the well-known h add heuristic from classical planning, and the second is computed in belief space, taking the value of information into account. add 1 Introduction Many autonomous agents operate in environments where actions have stochastic effects, and important information that is required for obtaining the goal is hidden from the agent. Agents in such environments typically execute actions and sense some observations that result from these actions. Based on the accumulated observations the agents can better estimate their current state and decide on the next action to execute. Such environments are often modeled as partially observable Markov decision processes (POMDPs) [28]. POMPD models allow us to reason about the hidden state of the system, typically using a belief state – a distribution over the possible environment states. The belief state can be updated given the executed action and the received observation. One can compute a policy, a mapping from beliefs to actions, that dictates which action should be executed given the current belief. Many algorithms were suggested for computing such policies [25]. However, in larger environments, it often becomes difficult to maintain a belief state, let alone compute a policy for all possible belief states. In such cases, one can use an online re-planning approach, where after every action is executed, the agent computes which action to execute next. Such online approaches replace the lengthy single policy computation which is done offline, before the agent begins to act, with a sequence of shorter computations, which are executed online, during execution, after each action [21]. POMCP [27] is such an online replanning approach, extending the UCT algorithm for fully observable Markov decision processes (MDPs) to POMDPs. POMCP operates by constructing online a search tree, interleaving decision and observation nodes. The root of the tree is a decision node. Each decision node has an outgoing edge for every possible action, ending at an observation node. Then, the outgoing edges from an observation node denote the possible observations that result from the incoming action. The agent computes a value for each node in the tree, and then, the agent can choose, from the root node, the action associated with the edge leading to the highest value child node. To evaluate the value of leaf nodes, POMCP executes a random walk in belief space, known as a rollout, where the agent selects actions from some rollout policy to construct a trajectory in belief space and obtain an estimation of the quality of the leaf node. Clearly, this evaluation is highly dependant on the ability of the rollout policy to reach the agent goals. In complex problems, obtaining the goal may require a lengthy sequence of actions [16], and until the goal is reached, no meaningful rewards are obtained. Indeed, practitioners that use POMCP often implement complex domain-specific heuristics for the rollout policy. In this paper we focus on suggesting domain-independent heuristics for rollout policies. We leverage work in automated planning, using heuristics defined for classical and contingent planning problems [7, 8, 6]. We thus represent POMDP problems in a structured manner, using boolean facts to capture the state of the environments. This allows both for a compact representation of large problems, compared with standard flat representations that do not scale, as well as the ability to use classical planning heuristics. We begin by suggesting using the well-known h add heuristic for choosing rollout actions [2]. This heuristic searches forward in a delete relaxation setting, until the goal has been reached. Then, the value of an action is determined by the number of steps in the delete relaxation space following the action, required for obtaining the goal. add Next, we observe that any state-based rollout policy is inherently limited in its ability to evaluate the missing information required for reaching the goal, and hence, provide some estimate as to the value of information [12] of an action. We hence suggest a multi-state rollout policy, where actions are executed on a set of states jointly, and observations are used to eliminate states that are incompatible with the observed value. We show that this heuristic is much more informed in domains that require complex information-gathering strategies. For an empirical evaluation, we extend domains from the contingent planning community with stochastic effects. We evaluate our heuristics, comparing them to random rollouts, showing that they allow us to provide significantly better behavior.

Rollout Heuristics for Online Stochastic Contingent Planning: Introduction

Act 2: Life '“As the shackles of progress began to take shape, art began to guide itself to a deeper truth. Being responsible for the laws and flaws that governed the system. The shapes took distinct interpretations based on their approach to form the depth. A requirement for the power required to truly live and share the art of doing so.” Part 1: Biology ‘As the girls delved into their assignments, the world around them continued to evolve, shaped by the collective actions of individuals striving to understand and improve their surroundings. Through the exploration of science, these young students and others like them would contribute to the progress and stability of humankind.’- Rewinder Lyra, do you believe that fish transform water into breathable oxygen, and it is this process that doesn’t allow for higher degrees of thinking to develop? Uhh… I believe fishes embraced their medium. I mean… the water’s embrace seems nice. Yeah… I believe that’s why this approach is for those who like to dive into it. I can’t imagine myself doing such a thing to a fish. That’s why I like mathematics. You don’t cut anything here. Well… maybe if you want to take a closer look. But hey, at least you don’t have to deal with fish. Yeah, I guess you’re right. But what is the true magic in mathematics if we make it? The magic is… weird but you’ll get used to it. I suppose. Breaking News! Science has been able to discover a new range of particles. A new understanding of colors was able to shed light on the physical interactions of perception. Mom! You forgot the TV open! A sight gasp hidden in Elara’s voice Again… Elara got up to find the remote to continue their lessons further. Huh, what’s with the rush Elara? I hate when I try to understand something but I get distracted. I feel like I have to take it from the beginning each time. Well… that’s kind of the magic. You never know why your formula drove you this way and not the other. Yes, but the fish can still dive for a long time in the water. Imagine how interesting their lives must be. There is no right or wrong there, it just… is. If I understand correctly… you’re talking about your mom here? I’m talking about everything. My mom doesn’t understand that I don’t need her protection anymore, my teachers consider that I must achieve their dreams… What if I am just… me? “A moment of shock for Lyra. She felt that Elara was more than this. She knew it in her inner core. But how could she express it? In all her attempts, she failed. Looking back, she realized that what she was missing, was just herself. “ Elara, look. I know you see nothing more than just you. For you, the world is this way. It is your norms and normal. Keep tight to them. I can’t show you the results as you would lose all their beauty. Maybe one day, we could cut a holographic fish. Elara’s gaze was ablaze. Her hands holding her own face That would be… AMAZING! Imagine the possibilities. From medical systems to biological understanding to To geometrical exploration. Lyra Interrupted. It is in our balanced perspectives that we explore this land. You have your insights, while I have mine. Maybe this is what makes us unique across an entire whole. Your thinking is… twisting. But in the right way. I like it. Part 2: Randomness Meanwhile at the Quanta Research Academy “The question between the randomness of the system and its order is based on the relative positions of the points within the system.”-Thomas “No, we shouldn’t pick our first assumption. The world of quantum is always unpredictable.”-Alfred “That’s why it denotes a collective understanding of all points”-Genova “But when seeing all points you can hardly see the patterns”-Lituanin Each of them argued on their terms. Each of them failed to keep hold of their own beauty. Circular reasons are indeed weird. They do seem to bend our reality. But then, why is reality so… straightforward? Because it is a moment of “agreement”? Like… “Yes, this is the rarest path.” Maybe one in which they stopped arguing x.x Lyra excitedly said. Elara! I believe I found a way to differentiate the universal space factor from the self-perspective factor of any individual group. Umm… meaning…? Meaning that I found a new way of scanning the world! Intriguing. What do you mean by scanning? Like… how to better see the world? In a sense yes… but more like… wait. Let’s rewind. Imagine you have to lower a bit your expectations in your answer x.x Uh… then you mean a new way of interpretation? . This is a story without ending in the end. Too much discussion will exclude the narrator from the view. Now, what should a good story need? Remember, we unleashed all this potential into the story. What can we do now? Uhh… It’s weird when you ask me about it… It can only come out when thought about before all else I know! You can envision this scanner as a part of your eye. Differentiating the other world from your being. _ But AGI could require complexities only you know about Elara. Those are also a kind of depth I assume.** This way, our curse as living might be to be restricted by technology. In this sense, biology loses in the face of AI. Could be. But hey, first, I want to pass my exam this week. You said we are in this together. So you too have to help me. I knew this would come. But you know what comes after you? The digital frog. Haha. I’d love to see it coming. Especially running. I too have legs. We’ll see who’s faster. :) Act 3: Breaking the Norms “The night sky was alight with stars, and at the center of it all hung the moon, glowing with a soft, luminous light. But this was no ordinary moon—it was alive with a sentience all its own, watching over the world below with a keen and knowing gaze. As the characters embarked on their journey, they found themselves guided by this celestial being, who seemed to understand their thoughts and desires in a way that no human ever could. And as they delved deeper into the mysteries of the universe, they began to realize that this moon was more than just a passive observer—it was an active participant in the unfolding story, shaping their destinies in ways they could never have imagined.” Elara exclaimed. Hello Moon! Without realizing it, she was dreaming. But not dreaming in the sense of being just herself. The Moon is also an important character in this story. However, their interactions seemed to be significant because they were so distant. She never felt something similar to it before. This moment was the moment in which Elara experienced for the first time what it meant to dream. To mix reality with fiction. The Moon asked intrigued Hello Elara. How are you, my dear? Elara responded with the same blaze on her face as before Well… I’m honestly amazed. The Moon slowly turned Her gaze toward Elara Why so Elara? Elara curiously admitted I remember how drawn I was to taking photos of you a while ago. I was drawn to your beauty. I am wondering now why I stopped doing it. The Moon revealed with a small laugh So you are sorry that you can’t describe the beauty you saw in me back then? Haha. I knew you would be special. Elara responded with a sense of curiosity Special? What do you mean? , the Moon explained. Special, in the sense that you have a unique ability to see beauty and meaning in the world around you, Elara And now, in this dream, you have the chance to explore that beauty in a way that would be impossible in the waking world. You can commune with me, a celestial being, and learn from my ancient wisdom. You can transcend the limitations of your physical form and experience a reality that is beyond the reach of your senses. Elara was silent for a moment, letting this line of thoughts slowly fade into forgetfulness. she said at last. I never thought of myself as special I always felt like I was just a regular person, trying to make sense of a confusing world. But now, with you here, I feel like anything is possible. Like I can explore the depths of the universe and unlock secrets that have been hidden for millennia. The Moon smiled, its glow seeming to brighten the entire dreamscape. That's the spirit, Elara. Together, we can break free from the norms and expectations that have held you back, and forge a new path into the unknown. Elara. Elara? Oh, yes Lyra. Sorry. What? Wasn’t I dreaming? Wasn’t I supposed to be in bed? Uh… no… we were discussing how this hologram is special because it takes into account all the biological-like data from the external medium… well, except wind. And I wanted to ask you if you would like to take a look at the specifications. Sure, I would, but let’s leave this for another day. Right now, I feel a bit weird. Is tomorrow a good day? Yes, should be. Alright, see you tomorrow then. Good night Elara! Good Night Lyra! Act 4: Still Me, (Not the Narrator) “I feel like… exploring the world beyond our own seems in a sense both warm and cold. Quanta Research Team helps me in a sense make sense of where I should not look for warmth, whereas Elara and Lyra are the only lucid neurons I have left. Is it an empathetic view of craziness? I do not know. Maybe craziness becomes craziness only when material life is drawn too much towards its end. Uncertainty could arise because the conclusions are either rushed or drawn without truly tapping the general map across the play. Each story requires a stage before playing it. Maybe I sit in the space that differentiates the play of the story and the play on the stage? The last one indeed shapes the story. But if the story itself is not purely good, it cannot envelop true creativity. Imagine a world where war is not restricting our forms of expression. Where negativity is not a pre-requirement for emotional building. That could be the domain of pure adrenaline. I could give an example. Let’s jump to… Book 5, Act 2: “The Reality of Duality.” A chapter in which the duality is found in the mere fact that what is not alive, is its opposite and vice-versa. That’s why ethics is not able to be formed there. Balance is a requirement rather than a choice. The Book of Transcendence is in a sense a question about the importance of the view in mathematics and why is quantum considered more powerful. “ As Elara drifted into her world of dreams, she began to imagine strange creatures that formed along the drift. The trust in her own actions, offered by her understanding of the world allows her to truly rest. Especially after a day filled with progress and steps climbed toward achieving both her goals. Lyra, although being her close friend, couldn’t see their friendship flourishing forever, no matter how much she tried. This was a thought that kept Lyra awake night by night. Her interactions with Elara were able to somehow erase this irrational fear. Or at least, put a break to this thought. After all, Lyra loved strict and always-right abstract formulas, whereas Elara saw the beauty that pushed every existing being further. A drive to love the unattainable and a pursuit to bring the unattainable to light.

Transcendence Book 2: The Origins of Continuity 

Authors: (1) Muzhaffar Hazman, University of Galway, Ireland; (2) Susan McKeever, Technological University Dublin, Ireland; (3) Josephine Griffith, University of Galway, Ireland. Table of Links Abstract and Introduction Related Works Methodology Results Limitations and Future Works Conclusion, Acknowledgments, and References A Hyperparameters and Settings B Metric: Weighted F1-Score C Architectural Details D Performance Benchmarking E Contingency Table: Baseline vs. Text-STILT E Contingency Table: Baseline vs. Text-STILT Table 8 shows the contingency table – as one would prepare for a McNemar’s Test between two classifiers (McNemar, 1947) – between the model trained with Text-STILT on 60% Memes and Baseline trained on 100% Memes available which had the most similar performance. While the two models performed similarly in terms of Weighted F1- scores, Text-STILT correctly classified a notable number of memes that Baseline did not and vice versa. Examples of such memes are discussed in Section 4.1. Furthermore, approximately 40% of memes in the testing set were incorrectly classified by both models. This suggests that these memes convey sentiment in a way that cannot be reliably predicted by either approach. This paper is under CC 4.0 license. available on arxiv

Contingency Table: Baseline vs. Text-STILT

I confess! I use ChatGPT, and here’s why Like anyone in the communications industry (or probably just anyone), I couldn’t ignore ChatGPT for too long. I tried it for everything: articles’ outlines, short-to-large social media posts, work on content strategy, work emails, search for needed platforms or tools, and even internal and external agency documentation. And I should say, even though I never presumed something written by the ‘Chat’ would be the final version, it helps me work faster. I’ll try to answer how & why further. Every day is the Chat’s day Let me explain why I’ve tried ChatGPT for so many different purposes. First, I firmly believe that one cannot lead a team without understanding the specifics of their work. So, to lead the communications team, I should be a communications expert myself. Second, as a C-level person in a small boutique-type agency, especially in its early stages, you must catch up with everything before you find a dedicated expert. From Every Day to One in 5 Days: My Frequency of Using ChatGPT I think I’ve explained why I used ChatGPT almost daily when it started operating. But now, I still have loads of work and a personal blog, which I ignore for most days of the week. That’s because, from what I think, ChatGPT at its current stages is only good for avoiding procrastination. This means that if your task is too complex, requires the creativity you lack, or you don’t know where to start, ChatGPT will solve the problem by providing a rough draft. Besides this, it’s weaker than search engines and has nothing to compare with the human mind. Will I be Using It for the foreseeable future? Months From Now? Years? Yes, I will soon. But I hope it will improve more, and I also hope for more opportunities in unique image creation. I tried this function a little bit, but my impression is the same as that of textual functionality. If I’m So Critical, Why Do I Still Use It? Despite the imperfections, ChatGPT can actually act as a junior assistant. Yes, you’ll need to fact-check its work, and yes, you will find critical mistakes, but it saves time. For me, it is a savior in content plan creation. Indicating all the details of your professional background and interests will help you develop ideas for the following articles. It won’t make the Newsjacking for you, though. Chat is also quite okay for creating rough templates for cold pitching and social media posts. Therefore, it can be very helpful if, for example, you are writing a script for your employee and want to show the examples without spending much time. There Are Other Chatbots Out There. What Makes ChatGPT Stand Out From the Competition? Assecibility. Lots of user feedback. Clear UI. That’s enough for a user like me—those who use AI on a more advanced level might and will say more. The Most Significant Downside The most significant downside is not the ChatGPT itself but its users. Since its very start, many people have thought AI can replace human writers when it can’t. When I was looking for a good writer to join my team, I had to read numerous GPT-generated texts, and none of those were good. Such materials do not provide any insights, have a bit of a robotic tone of voice, and are just not interesting at all. Closing note: How Would I Describe ChatGPT to Someone Who Doesn’t Know Anything About the World of AI? I’d say ChatGPT doesn’t need any introduction. Everybody knows it. But if I were to introduce it to someone like my mom or grandma, I’d say it’s a digital Junior Assistant — not the smartest in town, but always ready to help.

Why I Continue to Use ChatGPT Despite its Limitations — A C-level Perspective 

Hello Hackers! It’s no secret that AI has changed the world and will continue to do so. It has touched and affected nearly every field imaginable, including writing. Some writers view AI as soulless, something that can never replicate the human spirit, and something that should be nowhere near writing. Others view it as a godsend that can write full, complex articles for them. But the truth is, it all depends on how you use it. Today, we’ll go over 3 ways you can use AI to improve your writing and writing process. P.S. HackerNoon does not accept articles 100% written by an AI. However, articles that were AI-assisted are okay. 1. Coming Up With Article Ideas Say you’re a video game writer. You have some general sense of what you want your article to be about but are having trouble finding a specific topic. Use a chatbot, like ChatGPT, to find exactly what you’re looking for. We asked ChatGPT to give us 10 article ideas about Nintendo, and this was te result: Now, all of these seem like they could be decent articles. With some tweaking, hard work, and research, they could be great. And if you don’t like any of these article ideas, that’s a good thing too. You know what you don’t like which will make it easier to find what you’re looking for.
Of course, relying on a chatbot to find all your article ideas isn’t probably the best idea as it could become repetitive, but it’s a great tool when you’re stuck in a rut. How do you feel about ChatGPT? Let us know using this ChatGPT Review writing template! 2. Find What’s Missing From Your Article There’s no better feeling than when you’ve finished writing and you're staring at your hard work. However, it can also feel wrong; like something is missing. You’ve written about everything you wanted to write about, but it feels incomplete. You rack your brain trying to find the missing piece in your article, but nothing comes to mind. Enter AI. List all the key points you wrote about, and ask the AI which are missing or what sections could be included to flesh out your article. Using this, improve your article by adding more to it. If none of the extra topics sound like they would be a right fit, it could be a sign that your article has everything it already needs, and you can breathe a sigh of relief. Your work is finished. If you are still struggling with structuring your article, check out the How to Write a Guide on HackerNoon template. 3. Improving Your Grammar You can re-read your article 10 times over, but sometimes, those little mistakes will get the better of you and will get past you. That’s where an AI writing tool like Grammarly can come in handy. It automatically checks your article for mistakes and will let you know how to fix them. It’s great if English isn’t your first language, if you struggle with spelling, or if you just want a second pair of eyes. However, all of its suggestions won’t be correct. Use your judgment, and if something feels off, stick with your gut. Already incorporate AI into your work? Let us know how with this template. What’s New On HackerNoon? Writing Contests You don’t need AI to tell you how to make money writing because we’ve got you covered. There are currently two HackerNoon writing contests going on: The Crypto API Writing Contest by CoinGecko and The Opt-Out Writing Contest by Aut Labs. Join either (or both!) of these contests to potentially earn some hard-earned money. New Pages to Browse HackerNoon is a global hub for entrepreneurs, venture capitalists, anarchists, and more. It's a community for everyone - check out our new page to see who browses with you. https://hackernoon.com/p/publish/people?embedable=true And that’s a wrap for this article. Take care Hackers, and see you next time! Hello Hackers! It’s no secret that AI has changed the world and will continue to do so. It has touched and affected nearly every field imaginable, including writing. Some writers view AI as soulless, something that can never replicate the human spirit, and something that should be nowhere near writing. Others view it as a godsend that can write full, complex articles for them. But the truth is, it all depends on how you use it. Today, we’ll go over 3 ways you can use AI to improve your writing and writing process. P.S. HackerNoon does not accept articles 100% written by an AI. However, articles that were AI-assisted are okay. P.S. HackerNoon does not accept articles 100% written by an AI. However, articles that were AI-assisted are okay. 1. Coming Up With Article Ideas Say you’re a video game writer. You have some general sense of what you want your article to be about but are having trouble finding a specific topic. Use a chatbot, like ChatGPT, to find exactly what you’re looking for. We asked ChatGPT to give us 10 article ideas about Nintendo, and this was te result: Now, all of these seem like they could be decent articles. With some tweaking, hard work, and research, they could be great. And if you don’t like any of these article ideas, that’s a good thing too. You know what you don’t like which will make it easier to find what you’re looking for.
Of course, relying on a chatbot to find all your article ideas isn’t probably the best idea as it could become repetitive, but it’s a great tool when you’re stuck in a rut. How do you feel about ChatGPT? Let us know using this ChatGPT Review writing template! How do you feel about ChatGPT? Let us know using this ChatGPT Review writing template! ChatGPT Review 2. Find What’s Missing From Your Article There’s no better feeling than when you’ve finished writing and you're staring at your hard work. However, it can also feel wrong; like something is missing. You’ve written about everything you wanted to write about, but it feels incomplete. You rack your brain trying to find the missing piece in your article, but nothing comes to mind. Enter AI. List all the key points you wrote about, and ask the AI which are missing or what sections could be included to flesh out your article. Using this, improve your article by adding more to it. If none of the extra topics sound like they would be a right fit, it could be a sign that your article has everything it already needs, and you can breathe a sigh of relief. Your work is finished. If you are still struggling with structuring your article, check out the How to Write a Guide on HackerNoon template. If you are still struggling with structuring your article, check out the How to Write a Guide on HackerNoon template. How to Write a Guide on HackerNoon 3. Improving Your Grammar You can re-read your article 10 times over, but sometimes, those little mistakes will get the better of you and will get past you. That’s where an AI writing tool like Grammarly can come in handy. It automatically checks your article for mistakes and will let you know how to fix them. It’s great if English isn’t your first language, if you struggle with spelling, or if you just want a second pair of eyes. However, all of its suggestions won’t be correct. Use your judgment, and if something feels off, stick with your gut. Already incorporate AI into your work? Let us know how with this template . this template What’s New On HackerNoon? Writing Contests Writing Contests You don’t need AI to tell you how to make money writing because we’ve got you covered. There are currently two HackerNoon writing contests going on: The Crypto API Writing Contest by CoinGecko and The Opt-Out Writing Contest by Aut Labs . Join either (or both!) of these contests to potentially earn some hard-earned money. The Crypto API Writing Contest by CoinGecko The Opt-Out Writing Contest by Aut Labs New Pages to Browse New Pages to Browse HackerNoon is a global hub for entrepreneurs, venture capitalists, anarchists, and more. It's a community for everyone - check out our new page to see who browses with you. new page new page https://hackernoon.com/p/publish/people?embedable=true https://hackernoon.com/p/publish/people?embedable=true And that’s a wrap for this article. Take care Hackers, and see you next time!

Stories, News & Mentions about CONTIQ

Contiq’s Stories on HackerNoon