Siam Computing is a product development company specializing in the strategy, design, and engineering of digital solutions from concept to market launch.

Product Development Services | IT Consulting Services

Siam Computing | 23,524 followers on LinkedIn. We help companies grow exponentially by developing custom software solutions needed to disrupt their industry. | Siam Computing is a complete product development studio with strong imprints in over 5 major domains.

Siam Computing | LinkedIn

Since its inception, Siam Computing has been an IT partner to organizations to develop and enhance software solutions and systems.

Siam Computing - Crunchbase Company Profile & Funding

Siam Computing is built on a strong vision, corporate philosophy, and core values that shape our culture, best opportunity to innovate, learn and grow.

Careers | Siam Computing

We are a Web Development company and we are working together to build amazing products for our clients as well as some of our own (hisabing.com). We love learning new technologies and have some lofty ambitions and set ideals. You can take a look at our website- www.siamcomputing.com to know ...

5 Siam Computing Jobs in India | Siam Computing careers | Cutshort

Siam Computing is best Enterprise Web App Development Company. We provide best web app solution that streamline their business operation.

Enterprise Web App Development Company in Chennai | Siam Computing

We provide the best MVP development service! We help you test the market in real time, gauge the feedback, and stay ahead of the competition.

MVP Development Service | Siam Computing

Best Web Portal Development Company! We deliver client success by bringing together the best programmers and IT strategists in the industry.

Web Portal Development Company | Siam Computing

Siam Computing is the best React Native Development Company. We provide top-notch React Native development services. Contact our expert today!

React Native Development Company | Siam Computing

Android app development company in Chennai. Siam Computing Specializes in custom, offshore android application development for enterprises and SMEs.

Android app development company in Chennai - Siamcomputing

Industry 4.0 solutions from Siam Computing. Industry 4.0 is the best example of digital transformation in the manufacturing industry. Talk to our experts today!

Industry 4.0 Development Company Chennai | Industry 4.0 ...

Mobile app development services. We build tailored mobility solutions that are scalable and provide an end-user experience. Call us today!

Best Mobile App Development Company in Chennai

Microsoft SQL Server Development Company. We deliver MS SQL database development, maintenance, auditing, and optimization services.

Microsoft SQL Server Development Company | Hire a MSSQL Developers ...

IoT app development company in Chennai. We offer a wide range of IoT web and mobile applications and help companies to utilize them potentially.

iOT Development Company in Chennai - Siam Computing

Siam Computing is a Product Development Company based in Chennai

Siam Computing

This is paper is available on arxiv under CC 4.0 DEED license. Authors: (1) Leah Chrestien, Czech Technical University in Prague; (2) Tomá˘s Pevný, Czech Technical University in Prague, and Gen Digital, Inc.; (3) Stefan Edelkamp, Czech Technical University in Prague; (4) Antonín Komenda, Czech Technical University in Prague. Table of Links Abstract & Introduction Preliminaries Conditions on strictly optimally efficient heuristic Loss functions Related Work Experiments Conclusion and References Appendix 7 Conclusion The motivation of this paper stemmed from the observation that even the cost-to-goal, considered to be an optimal heuristic, fails to guarantee a strictly optimally efficient search. Since a large body of existing research optimizes this quantity, we are effectively lost with respect to what should be optimized. To fill this gap, we have stated the necessary and sufficient conditions guaranteeing the forward search to be strictly optimally efficient. These conditions show that the absolute value of the heuristic is not important, but that the ranking of states in the Open list is what controls the efficiency. Ranking can be optimized by minimizing the ranking loss functions, but its concrete implementation needs to correspond to a variant of the forward search. In case of mismatch, the resulting heuristic can perform poorly, which has been demonstrated when the heuristic optimized for BGFS search was used with A* search. The other benefit of ranking losses is that from the point of view of statistical learning theory, they solve a simpler problem than ubiquitous regression in estimating the cost-to-goal. We do not question the existence of a strictly optimally efficient heuristic. Given our experiments, we believe that if the space of heuristic functions over which the loss is optimized is sufficiently rich, the result will be sufficiently close to the optimal for the needs of the search. 7.1 Acknowledgements This work has been supported by project numbers 22-32620S and 22-30043S from Czech Science Foundation and OP VVV project CZ.02.1.01/0.0/0.0/16_019/0000765 "Research Center for Informatics". This work has also received funding from the European Union’s Horizon Europe Research and Innovation program under the grant agreement TUPLES No 101070149. References [1] Forest Agostinelli, Stephen McAleer, Alexander Shmakov, and Pierre Baldi. Solving the rubik’s cube with deep reinforcement learning and search. Nature Machine Intelligence, 1(8):356–363, 2019. [2] Thomas Anthony, Zheng Tian, and David Barber. Thinking fast and slow with deep learning and tree search. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5360–5370, 2017. [3] Shahab Jabbari Arfaee, Sandra Zilles, and Robert C Holte. Learning heuristic functions for large state spaces. Artificial Intelligence, 175(16-17):2075–2098, 2011. [4] Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B Shah. Julia: A fresh approach to numerical computing. SIAM review, 59(1):65–98, 2017. [5] Mohak Bhardwaj, Sanjiban Choudhury, and Sebastian Scherer. Learning heuristic search via imitation. In Conference on Robot Learning, pages 271–280. PMLR, 2017. [6] Blai Bonet and Héctor Geffner. Planning as heuristic search. Artificial Intelligence. 2001 Jun; 129 (1-2): 5-33., 2001. [7] Vladimir Cherkassky, Xuhui Shao, Filip M Mulier, and Vladimir N Vapnik. Model complexity control for regression using vc generalization bounds. IEEE transactions on Neural Networks, 10(5):1075–1089, 1999. [8] Leah Chrestien, Tomáš Pevný, Antonín Komenda, and Stefan Edelkamp. Heuristic search planning with deep neural network using imitation, attention and curriculum learning. In PRLIJCAI workshop, 2021. [9] Lukás Chrpa. Combining learning techniques for classical planning: Macro-operators and entanglements. In ICTAI, pages 79–86. IEEE Computer Society, 2010. [10] Rina Dechter and Judea Pearl. The optimality of A* revisited. In Michael R. Genesereth, editor, AAAI, pages 95–99. AAAI Press, 1983. [11] Edsger W. Dijkstra. A note on two problems in connexion with graphs. Numerische Mathematik, 1:269–271, 1959. [12] Stefan Edelkamp. Automated creation of pattern database search heuristics. In Model Checking and Artificial Intelligence, pages 35–50, 2006. [13] Stefan Edelkamp and Stefan Schrödl. Heuristic Search - Theory and Applications. Academic Press, 2012. [14] Ariel Felner, Uzi Zahavi, Robert Holte, Jonathan Schaeffer, Nathan Sturtevant, and Zhifu Zhang. Inconsistent heuristics in theory and practice. Artificial Intelligence, 175(9):1570–1603, 2011. [15] Dieqiao Feng, Carla P. Gomes, and Bart Selman. Solving hard AI planning instances using curriculum-driven deep reinforcement learning. In Christian Bessiere, editor, Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pages 2198–2205. ijcai.org, 2020. [16] Patrick Ferber, Malte Helmert, and Jörg Hoffmann. Neural network heuristics for classical planning: A study of hyperparameter space. In ECAI, pages 2346–2353. IOS Press, 2020. [17] Santiago Franco, Álvaro Torralba, Levi H. S. Lelis, and Mike Barley. On creating complementary pattern databases. In IJCAI, pages 4302–4309, 2017. [18] Caelan Reed Garrett, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. Learning to rank for synthesizing planning heuristics. page 3089–3095, 2016. [19] Edward Groshev, Aviv Tamar, Maxwell Goldstein, Siddharth Srivastava, and Pieter Abbeel. Learning generalized reactive policies using deep neural networks. 2018. [20] Arthur Guez, Théophane Weber, Ioannis Antonoglou, Karen Simonyan, Oriol Vinyals, Daan Wierstra, Rémi Munos, and David Silver. Learning to search with mctsnets. In International conference on machine learning, pages 1822–1831. PMLR, 2018. [21] Peter E Hart, Nils J Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Science and Cybernetics, 4(2):100–107, 1968. [22] Patrik Haslum, Adi Botea, Malte Helmert, Blai Bonet, and Sven Koenig. Domain-independent construction of pattern database heuristics for cost-optimal planning. In AAAI, pages 1007–1012, 2007. [23] Malte Helmert and Gabriele Röger. How good is almost perfect? In AAAI, pages 944–949. AAAI Press, 2008. [24] Robert C. Holte and Sandra Zilles. On the optimal efficiency of cost-algebraic A. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, pages 2288–2295. AAAI Press, 2019. [25] Michael Katz, Shirin Sohrabi, Horst Samulowitz, and Silvan Sievers. Delfi: Online planner selection for cost-optimal planning. IPC-9 planner abstracts, pages 57–64, 2018. [26] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv:1412.6980, 2014. [27] Donald E Knuth and Ronald W Moore. An analysis of alpha-beta pruning. Artificial intelligence, 6(4):293–326, 1975. [28] Richard E. Korf. Iterative-deepening-A*: An optimal admissible tree search. In IJCAI, pages 1034–1036. Morgan Kaufmann, 1985. [29] Richard E Korf. Macro-operators: A weak method for learning. Artificial intelligence, 26(1):35–77, 1985. [30] Šimon Mandlík, Matej Ra ˇ cinsk ˇ y, Viliam Lis ` y, and Tomáš Pevn ` y. Jsongrinder. jl: automated ` differentiable neural architecture for embedding arbitrary json data. Journal of Machine Learning Research, 23(298):1–5, 2022. [31] Drew V. McDermott. A heuristic estimator for means-ends analysis in planning. In AIPS, pages 142–149. AAAI, 1996. [32] David Ménager, Dongkyu Choi, Mark Roberts, and David W. Aha. Learning planning operators from episodic traces. In 2018 AAAI Spring Symposia. AAAI Press, 2018. [33] Ionut Moraru, Stefan Edelkamp, Santiago Franco, and Moisés Martínez. Simplifying automated pattern selection for planning with symbolic pattern databases. In KI, pages 249–263, 2019. [34] Christian Muise. Collection of pddl files of classical problems. https://github.com/AI-Planning/classical-domains, 2023. [35] XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan. On surrogate loss functions and f-divergences. The Annals of Statistics, pages 876–904, 2009. [36] Laurent Orseau, Levi Lelis, Tor Lattimore, and Théophane Weber. Single-agent policy tree search with guarantees. Advances in Neural Information Processing Systems, 31, 2018. [37] Laurent Orseau and Levi HS Lelis. Implementation of the algorithms described in "policyguided heuristic search with guarantees" by l. orseau and l. lelis, published at aaai’21. https://github.com/levilelis/h-levin, 2021. [38] Laurent Orseau and Levi HS Lelis. Policy-guided heuristic search with guarantees. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12382–12390, 2021. [39] Arthur L Samuel. Some studies in machine learning using the game of checkers. ii—recent progress. IBM Journal of research and development, 11(6):601–617, 1967. [40] Max-Philipp B. Schrader. gym-sokoban. https://github.com/mpSchrader/gym-sokoban, 2018. [41] Jendrik Seipp, Florian Pommerening, and Malte Helmert. New optimization functions for potential heuristics. In ICAPS, 2015. [42] Jendrik Seipp and Silvan Sievers. Fast downward stone soup 2023. [43] Jendrik Seipp, Álvaro Torralba, and Jörg Hoffmann. PDDL generators. https://doi.org/10.5281/zenodo.6382173, 2022. [44] William Shen. Strips-hgn. https://github.com/williamshen-nz/STRIPS-HGN, 2021. [45] William Shen, Felipe Trevizan, and Sylvie Thiébaux. Learning domain-independent planning heuristics with hypergraph networks. In ICAPS, volume 30, pages 574–584, 2020. [46] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676):354–359, 2017. [47] Kristian Smith. Maze creator / solver. https://github.com/ravenkls/Maze-Generator-and Solver, 2022. [48] Gustav Sourek, Vojtech Aschenbrenner, Filip Zelezny, Steven Schockaert, and Ondrej Kuzelka. Lifted relational neural networks: Efficient learning of latent relational structures. Journal of Artificial Intelligence Research, 62:69–100, 2018. [49] Simon Ståhlberg, Blai Bonet, and Hector Geffner. Learning general optimal policies with graph neural networks: Expressive power, transparency, and limits. In Akshat Kumar, Sylvie Thiébaux, Pradeep Varakantham, and William Yeoh, editors, Proceedings of the Thirty-Second International Conference on Automated Planning and Scheduling, ICAPS 2022, Singapore (virtual), June 13-24, 2022, pages 629–637. AAAI Press, 2022. [50] Ingo Steinwart. Consistency of support vector machines and other regularized kernel classifiers. IEEE transactions on information theory, 51(1):128–142, 2005. [51] Takeshi Takahashi, He Sun, Dong Tian, and Yebin Wang. Learning heuristic functions for mobile robot path planning using deep neural networks. In ICAPS, volume 29, pages 764–772, 2019. [52] Alvaro Torralba, Vidal Alcázar, Daniel Borrajo, Peter Kissmann, and Stefan Edelkamp. SymBA*: A symbolic bidirectional A* planner. In International Planning Competition, pages 105–108, 2014. [53] Sam Toyer, Sylvie Thiébaux, Felipe Trevizan, and Lexing Xie. Asnets: Deep learning for generalised planning. Journal of Artificial Intelligence Research, 68:1–68, 2020. [54] Vladimir Vapnik. Principles of risk minimization for learning theory. Advances in neural information processing systems, 4, 1991. [55] Marin Vlastelica, Michal Rolinek, and Georg Martius. Neuro-algorithmic policies enable fast combinatorial generalization. In International Conference on Machine Learning, pages 10575– 10585. PMLR, 2021. [56] Xuemei Wang. Learning planning operators by observation and practice. In Kristian J. Hammond, editor, AIPS, pages 335–340. AAAI, 1994. [57] Christopher Wilt and Wheeler Ruml. When does weighted A* fail? In International Symposium on Combinatorial Search, volume 3, 2012. [58] Christopher Makoto Wilt and Wheeler Ruml. Effective heuristics for suboptimal best-first search. J. Artif. Intell. Res., 57:273–306, 2016. [59] Yuehua Xu, Alan Fern, and Sungwook Yoon. Learning linear ranking functions for beam search with application to planning. Journal of Machine Learning Research, 10(7), 2009. [60] Ryo Yonetani, Tatsunori Taniai, Mohammadamin Barekatain, Mai Nishimura, and Asako Kanezaki. Path planning using neural A* search. In International Conference on Machine Learning, pages 12029–12039. PMLR, 2021. [61] Ziqi Zhang and Florian Geißer. Extending graph neural networks for generalized stochastic planning. In Bridging the Gap Between AI Planning and Reinforcement Learning. 2021. [62] Tan Zhi-Xuan. PDDL. jl: An Extensible Interpreter and Compiler Interface for Fast and Flexible AI Planning. PhD thesis, Massachusetts Institute of Technology, 2022. [63] Juntang Zhuang, Tommy Tang, Yifan Ding, Sekhar C Tatikonda, Nicha Dvornek, Xenophon Papademetris, and James Duncan. Adabelief optimizer: Adapting stepsizes by the belief in observed gradients. Advances in neural information processing systems, 33:18795–18806, 2020.

Optimize Planning Heuristics to Rank, not to Estimate Cost-to-Goal: Conclusion and References

This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license.
Authors:
(1) U Jin Choi, Department of mathematical science, Korea Advanced Institute of Science and Technology & ujchoi@kaist.ac.kr;
(2) Kyung Soo Rim, Department of mathematics, Sogang University & ksrim@sogang.ac.kr. Table of Links Introduction
Organization and notation
Problem Setting and Preliminaries
Generalized Wiener algebra and Fréchet derivative
Characterization of coefficients
Coefficients from an ergodic process
Conclusion & References Conclusion In this mathematical study, we delve into the realm of statistical inference and introduce a novel approach to variational non-Bayesian inference. Most significantly, we propose a new method for uniquely determining the hidden PDF solely from a random sample while leveraging theory. Beyond merely approximating the shape of the unrevealed PDF, we yield results that provide practical assistance in inferring important moments, such as the mean and variance, from the estimated PDF. These days, methods utilizing artificial intelligence are widely employed, but they generally exhibit a drawback of depending on initial conditions, i.e., a prior distribution, and iterative computations, i.e., a backpropagation. The limitations of such approaches lie in their reliance on these conditions to achieve global optimization. We emphasize the significant contribution of the proposed method to predictive and classification models, even without any information on populations, highlighting its potential applications in various domains such as finance, economics, weather forecasting, and machine learning, all of which present unique challenges. The most well-known method for approximating hidden PDFs is the variational approach in a Bayesian context. In contrast, our study extends this problem by seeking to precisely determine unknown PDFs through a system of equations. Our approach includes proving the Fréchet differentiability of entropy to establish the uniqueness of the energy function space in the Wiener algebra. We then derive the unique determination of the energy function through the minimization of KL-divergence. Leveraging the Ergodic theorem, we elucidate that solutions to equations comprising polynomial function series are the coefficients of the energy function and numerically substantiate the convergence of partial sums of the energy function obtained from a finite number of equations. In summary, our mathematical exploration has unveiled the potential of variational Non-Bayesian inference in Wiener space. We anticipate that these mathematical ideas can offer an innovative framework for probability density estimation and predictive modeling. Ultimately, our research emphasizes its potential contribution to reshaping mathematical horizons and expanding the boundaries of knowledge in the field of statistical methodology. REFERENCES [1] K. Bharath A. Saha and S. Kurtek, A geometric variational approach to bayesian inference, Journal of the American Statistical Association 115 (2020), no. 530, 822–835. [2] P. Alquier and J. Ridgway, Concentration of tempered posteriors and of their variational approximations, The Annals of Statistics 48 (2020), no. 3, 1475–1497. [3] A. R. Barron and C. Sheu, Approximation of density functions by sequences of exponential families, The Annals of Statistics 19 (1991), no. 3, 1347–1369. [4] C. M. Bishop and M. E. Tipping, Probabilistic principal component analysis, J. R. Statist. Soc. B 61 (1999), no. 3, 611–622. [5] F. W. Olver, D. Lozier, R. F. Boisvert and C. W. Clark, NIST Handbook of Mathematical Functions, Cambridge University Press, 2010. [6] C Brouder and F Patras, One-particle irreducibility with initial correlations, (2011). [7] I. Csiszar, I-divergence geometry of probability distributions and minimization problems, Ann. Probab. 3 (1975), no. 1, 146–158. [8] G. Cybenko, Approximation by superpositions of a sigmoidal function, Mathematics of Control, Signals and Systems 2 (1989), 303–314. [9] L. Grafakos, Classical and Modern Fourier Analysis, Prentice Hall, 2003. [10] D. E. Rumelhart, G. E. Hinton and R. Williams, Learning representations by back-propagating errors, Nature 323 (1986), 533–536. [11] G. E. Hinton, A Practical Guide to Training Restricted Boltzmann Machines, University of Toronto, 2010. [12] K. Hornik, Approximation capabilities of multilayer feedforward networks, Neural Networks 4 (1991), no. 2, 251–257. [13] Y. Katznelson, An introduction to harmonic analysis (third ed.), New York: Cambridge Mathematical Library, 2004. [14] D. M. Blei, A. Kucukelbir and J. D. McAuliffe, Variational inference: A review for statisticians, Jour. of the American Statistical Association 112 (2017), no. 518, 859–877. [15] S. Kullback and R. A. Leibler, On information and sufficiency, Ann. Math. Statistics 22 (1951), no. 1, 79–86. [16] H. M. Möller and R. Tenberg, Multivariate polynomial system solving using intersections of eigenspaces, Journal of Symbolic Computation 32 (2001), no. 5, 513–531. [17] B. Mourrain, V. Y. Pan and O. Ruatta, Accelerated solution of multivariate polynomial systems of equations, SIAM Journal on Computing 32 (2003), no. 2, 435–454. [18] W. Rudin, Functional Analysis, McGraw-Hill, 1991. [19] D. W. Stroock, Probability theory, an analytic view, Cambridge University Press, 1993. [20] B. Sturmfels, Solving systems of polynomial equations (cbms regional conference series in mathematics), Amer. Math. Soc., 2002. [21] N. Wiener, Tauberian theorems, Annals of Mathematics 33 (1932), no. 1, 1–100. [22] F. Zhang and C. Gao, Convergence rates of variational posterior distributions, The Annals of Statistics 48 (2020), no. 4, 2180–2207. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Authors: (1) U Jin Choi, Department of mathematical science, Korea Advanced Institute of Science and Technology & ujchoi@kaist.ac.kr; (2) Kyung Soo Rim, Department of mathematics, Sogang University & ksrim@sogang.ac.kr. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Authors: Authors: (1) U Jin Choi, Department of mathematical science, Korea Advanced Institute of Science and Technology & ujchoi@kaist.ac.kr; (2) Kyung Soo Rim, Department of mathematics, Sogang University & ksrim@sogang.ac.kr. Table of Links Introduction Organization and notation Problem Setting and Preliminaries Generalized Wiener algebra and Fréchet derivative Characterization of coefficients Coefficients from an ergodic process Conclusion & References Introduction Introduction Organization and notation Organization and notation Problem Setting and Preliminaries Problem Setting and Preliminaries Generalized Wiener algebra and Fréchet derivative Generalized Wiener algebra and Fréchet derivative Characterization of coefficients Characterization of coefficients Coefficients from an ergodic process Coefficients from an ergodic process Conclusion & References Conclusion & References Conclusion In this mathematical study, we delve into the realm of statistical inference and introduce a novel approach to variational non-Bayesian inference. Most significantly, we propose a new method for uniquely determining the hidden PDF solely from a random sample while leveraging theory. Beyond merely approximating the shape of the unrevealed PDF, we yield results that provide practical assistance in inferring important moments, such as the mean and variance, from the estimated PDF. These days, methods utilizing artificial intelligence are widely employed, but they generally exhibit a drawback of depending on initial conditions, i.e., a prior distribution, and iterative computations, i.e., a backpropagation. The limitations of such approaches lie in their reliance on these conditions to achieve global optimization. We emphasize the significant contribution of the proposed method to predictive and classification models, even without any information on populations, highlighting its potential applications in various domains such as finance, economics, weather forecasting, and machine learning, all of which present unique challenges. The most well-known method for approximating hidden PDFs is the variational approach in a Bayesian context. In contrast, our study extends this problem by seeking to precisely determine unknown PDFs through a system of equations. Our approach includes proving the Fréchet differentiability of entropy to establish the uniqueness of the energy function space in the Wiener algebra. We then derive the unique determination of the energy function through the minimization of KL-divergence. Leveraging the Ergodic theorem, we elucidate that solutions to equations comprising polynomial function series are the coefficients of the energy function and numerically substantiate the convergence of partial sums of the energy function obtained from a finite number of equations. In summary, our mathematical exploration has unveiled the potential of variational Non-Bayesian inference in Wiener space. We anticipate that these mathematical ideas can offer an innovative framework for probability density estimation and predictive modeling. Ultimately, our research emphasizes its potential contribution to reshaping mathematical horizons and expanding the boundaries of knowledge in the field of statistical methodology. REFERENCES [1] K. Bharath A. Saha and S. Kurtek, A geometric variational approach to bayesian inference, Journal of the American Statistical Association 115 (2020), no. 530, 822–835. [2] P. Alquier and J. Ridgway, Concentration of tempered posteriors and of their variational approximations, The Annals of Statistics 48 (2020), no. 3, 1475–1497. [3] A. R. Barron and C. Sheu, Approximation of density functions by sequences of exponential families, The Annals of Statistics 19 (1991), no. 3, 1347–1369. [4] C. M. Bishop and M. E. Tipping, Probabilistic principal component analysis, J. R. Statist. Soc. B 61 (1999), no. 3, 611–622. [5] F. W. Olver, D. Lozier, R. F. Boisvert and C. W. Clark, NIST Handbook of Mathematical Functions, Cambridge University Press, 2010. [6] C Brouder and F Patras, One-particle irreducibility with initial correlations, (2011). [7] I. Csiszar, I-divergence geometry of probability distributions and minimization problems, Ann. Probab. 3 (1975), no. 1, 146–158. [8] G. Cybenko, Approximation by superpositions of a sigmoidal function, Mathematics of Control, Signals and Systems 2 (1989), 303–314. [9] L. Grafakos, Classical and Modern Fourier Analysis, Prentice Hall, 2003. [10] D. E. Rumelhart, G. E. Hinton and R. Williams, Learning representations by back-propagating errors, Nature 323 (1986), 533–536. [11] G. E. Hinton, A Practical Guide to Training Restricted Boltzmann Machines, University of Toronto, 2010. [12] K. Hornik, Approximation capabilities of multilayer feedforward networks, Neural Networks 4 (1991), no. 2, 251–257. [13] Y. Katznelson, An introduction to harmonic analysis (third ed.), New York: Cambridge Mathematical Library, 2004. [14] D. M. Blei, A. Kucukelbir and J. D. McAuliffe, Variational inference: A review for statisticians, Jour. of the American Statistical Association 112 (2017), no. 518, 859–877. [15] S. Kullback and R. A. Leibler, On information and sufficiency, Ann. Math. Statistics 22 (1951), no. 1, 79–86. [16] H. M. Möller and R. Tenberg, Multivariate polynomial system solving using intersections of eigenspaces, Journal of Symbolic Computation 32 (2001), no. 5, 513–531. [17] B. Mourrain, V. Y. Pan and O. Ruatta, Accelerated solution of multivariate polynomial systems of equations, SIAM Journal on Computing 32 (2003), no. 2, 435–454. [18] W. Rudin, Functional Analysis, McGraw-Hill, 1991. [19] D. W. Stroock, Probability theory, an analytic view, Cambridge University Press, 1993. [20] B. Sturmfels, Solving systems of polynomial equations (cbms regional conference series in mathematics), Amer. Math. Soc., 2002. [21] N. Wiener, Tauberian theorems, Annals of Mathematics 33 (1932), no. 1, 1–100. [22] F. Zhang and C. Gao, Convergence rates of variational posterior distributions, The Annals of Statistics 48 (2020), no. 4, 2180–2207.

Variational Non-Bayesian Inference of the Probability Density Function: Conclusion & References

Verifiable Privacy-Preserving Computing: Conclusion & References

Authors: (1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada; (3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Table of Links Abstract & Introduction Optimization of the Optical Interpolation Materials and Methods Results and Discussion Conclusion Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Acknowledgments & References ACKNOWLEDGMENTS The proof for the equidistant case was adapted from a proof by David J. Fleet and Allan D. Jepson from the University of Toronto [18]. REFERENCES [1] C. Haley, “Missing-data nonparametric coherency estimation,” IEEE Signal Processing Letters, vol. 28, pp. 1704–1708, 2021. [2] S. Dilmaghani, I. C. Henry, P. Soonthornnonda, E. R. Christensen, and R. C. Henry, “Harmonic analysis of environmental time series with missing data or irregular sample spacing,” Environmental science & technology, vol. 41, no. 20, pp. 7030–7038, 2007. [3] J. W. Cooley and J. W. Tukey, “An algorithm for the machine calculation of complex fourier series,” Mathematics of computation, vol. 19, no. 90, pp. 297–301, 1965. [4] F. Rasheed, P. Peng, R. Alhajj, and J. Rokne, “Fourier transform based spatial outlier mining,” in Intelligent Data Engineering and Automated Learning-IDEAL 2009: 10th International Conference, Burgos, Spain, September 23-26, 2009. Proceedings 10. Springer, 2009, pp. 317–324. [5] C. Van Loan, Computational frameworks for the fast Fourier transform. SIAM, 1992. [6] H. Carslaw, “A historical note on gibbs’ phenomenon in fourier’s series and integrals,” Bulletin of the American Mathematical Society, 1925. [7] J. Markel, “Fft pruning,” IEEE transactions on Audio and Electroacoustics, vol. 19, no. 4, pp. 305–311, 1971. [8] J. C. Bowman and Z. Ghoggali, “The partial fast fourier transform,” Journal of Scientific Computing, vol. 76, pp. 1578–1593, 2018. [9] M. Kircheis and D. Potts, “Direct inversion of the nonequispaced fast fourier transform,” Linear Algebra and its Applications, vol. 575, pp. 106–140, 2019. [10] S. Kunis and D. Potts, “Stability results for scattered data interpolation by trigonometric polynomials,” SIAM Journal on Scientific Computing, vol. 29, no. 4, pp. 1403–1419, 2007. [11] K. B. Petersen, M. S. Pedersen et al., “The matrix cookbook,” Technical University of Denmark, vol. 7, no. 15, p. 510, 2008. [12] C. M. Bishop, Neural networks for pattern recognition. Oxford university press, 1995. [13] H. Wickham, M. Averick, J. Bryan, W. Chang, L. D. McGowan, R. Franc¸ois, G. Grolemund, A. Hayes, L. Henry, J. Hester, M. Kuhn, T. L. Pedersen, E. Miller, S. M. Bache, K. Muller, J. Ooms, D. Robin- ¨ son, D. P. Seidel, V. Spinu, K. Takahashi, D. Vaughan, C. Wilke, K. Woo, and H. Yutani, “Welcome to the tidyverse,” Journal of Open Source Software, vol. 4, no. 43, p. 1686, 2019. [14] R Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2023. [Online]. Available: https://www.R-project.org/ [15] C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. F. del R´ıo, M. Wiebe, P. Peterson, P. Gerard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, ´ H. Abbasi, C. Gohlke, and T. E. Oliphant, “Array programming with NumPy,” Nature, vol. 585, no. 7825, pp. 357–362, Sep. 2020. [Online]. Available: https://doi.org/10.1038/s41586-020-2649-2 [16] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, ˙I. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors, “SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python,” Nature Methods, vol. 17, pp. 261– 272, 2020. [17] J. D. Hunter, “Matplotlib: A 2d graphics environment,” Computing in Science & Engineering, vol. 9, no. 3, pp. 90–95, 2007. [18] “Notes on fourier analysis,” http://www.cs.toronto.edu/ ∼jepson/csc320/notes/fourier.pdf, accessed: 2023-09-01. This paper is under CC 4.0 license. available on arxiv

Groundbreaking Research in Signal Theory

This paper is CC 4.0 license. available on arxiv under Authors: (1) Amir Noorizadegan, Department of Civil Engineering, National Taiwan University; (2) D.L. Young, Core Tech System Co. Ltd, Moldex3D, Department of Civil Engineering, National Taiwan University & dlyoung@ntu.edu.tw; (3) Y.C. Hon, Department of Mathematics, City University of Hong Kong; (4) C.S. Chen, Department of Civil Engineering, National Taiwan University & dchen@ntu.edu.tw. Table of Links Abstract & Introduction Neural Networks PINN for Solving Inverse Burgers’ Equation Residual Network Numerical Results Results, Acknowledgments & References 6 Conclusion Throughout this study, we conducted a series of experiments to assess how different neural network setups, including Plain NN and SQR-SkipResNet, perform when it comes to interpolating both smooth and complex functions. Our findings consistently showed that SQR-SkipResNet outperforms other architectures in terms of accuracy. This was especially evident when dealing with non-smooth functions, where SQR-SkipResNet displayed improved accuracy, although it might take slightly more time to converge. We also applied our approach to real-world examples, like interpolating the shape of a volcano and the Stanford bunny. In both cases, SQR-SkipResNet exhibited better accuracy, convergence, and computational time compared to Plain NN. Furthermore, while opting for a deeper network might at times lead to reduced accuracy for both Plain NN and SQR-SkipResNet, we observed that this outcome is influenced by the specific problem. For instance, when dealing with the complicated geometry of the Stanford Bunny and its smooth function, we noticed that deeper networks yielded enhanced accuracy, quicker convergence, and improved CPU efficiency. Regardless of whether deeper networks are suitable, the proposed method demonstrated superior performance. As the effectiveness of network depth varies based on the problem, our approach offers a more favorable architecture choice for networks of different depths. Additionally, when applied to solve the inverse Burgers’ equation using a physicsinformed neural network, our proposed architecture showcased significant accuracy and stability improvements across different numbers of hidden layers, unlike Plain NN. Prospective studies might delve into further optimizations, extensions, and applications of the SQR-SkipResNet framework across diverse domains, particularly for addressing a broad range of inverse problems coupled with PINN methodologies. Acknowledgments Authors gratefully acknowledge the financial support of the Ministry of Science and Technology (MOST) of Taiwan under grant numbers 111-2811-E002-062, 109-2221-E002- 006-MY3, and 111-2221-E-002 -054 -MY3. References [1] Raissi, M., P. Perdikaris, and G.E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378: p. 686-707, 2019. [2] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385, 2015. [3] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. arXiv preprint arXiv:1603.05027, 2016. [4] Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, Tom Goldstein: Visualizing the Loss Landscape of Neural Nets, arxiv.1712.09913v3, 2018. [5] Veit, A., M. Wilber, and S. Belongie, Residual networks behave like ensembles of relatively shallow networks, in Proceedings of the 30th International Conference on Neural Information Processing Systems. Curran Associates Inc.: Barcelona, Spain. p. 550–558, 2016. [6] S. Jastrzębski, Arpit D, Ballas N, Verma V, Che T, Bengio Y: Residual Connections Encourage Iterative Inference. arXiv:1710.04773, 2017. [7] Lu, Lu., M. Dao, P. Kumar, U. Ramamurty, G.E. Karniadakis, and S. Suresh, Extraction of mechanical properties of materials through deep learning from instrumented indentation. Proceedings of the National Academy of Sciences, 117(13): p. 7052-7062, 2020. [8] Wang, S., Y. Teng, and P. Perdikaris, Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks. SIAM Journal on Scientific Computing, 43(5): p. A3055-A3081, 2021. [9] R. Franke, Scattered data interpolation: tests of some methods, Mathematics of Computation 38 , 181–200, 1982. [10] C.-S. Chen, A. Noorizadegan, C.S. Chen, D.L. Young, On the selection of a better radial basis function and its shape parameter in interpolation problems, Applied Mathematics and Computation 442, 12771, 2023. [11] denizunlusu / Getty Images. (n.d.). Mt Eden, or Maungawhau, is a significant M¯aori site. Retrieved from https://www.lonelyplanet.com/articles/top-things-to-doin-auckland [12] Mira Bozzini, Milvia Rossini. Testing methods for 3d scattered data interpolation. Multivariate Approximation and Interpolation with Applications, 20, 2002.

Physics-Informed with Power-Enhanced Residual Network: Results, Acknowledgments & References

Efficient Neural Network Approaches for Conditional Optimal Transport: Discussion and Reference

Authors:
(1) Ankur Nath, Department of Computer Science and Engineering, Texas A&M University;
(2) Alan Kuhnle, Department of Computer Science and Engineering, Texas A&M University. Table of Links Abstract & Introduction Related work Evaluation for Max-Cut Evaluation for SAT Summary and Outlook, References Supplementary Materials 5 SUMMARY and OUTLOOK Through our empirical evaluations, our goal is to promote an insightful comparison within the research focusing on the intersection of combinatorial optimization and machine learning. In order to provide the research community with valuable guidance, we believe it is imperative to communicate both the strengths and weaknesses of the proposed approaches. Poor instances and baseline selection may give the wrong impression about the performance of learned heuristics. Specifically, it is important to articulate the degree of improvement achieved through integrating classical heuristics with deep learning architectures and conduct a thorough comparison with classical heuristics. This will aid in elucidating the degree to which deep learning architectures enhance integrated heuristics. It assists in ascertaining whether the integration endeavor is justified and warrants the allocation of computational resources, time, and investment necessary for integrating deep learning with classical heuristics. References Albert, R. and Barab´asi, A.-L. (2002). Statistical mechanics of complex networks. Reviews of modern physics, 74(1):47. Barrett, T., Clements, W., Foerster, J., and Lvovsky, A. (2020). Exploratory combinatorial optimization with reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 3243–3250. Barrett, T. D., Parsonson, C. W., and Laterre, A. (2022). Learning to solve combinatorial graph partitioning problems via efficient exploration. arXiv preprint arXiv:2205.14105. Battiti, R. and Protasi, M. (1997). Reactive search, a history-sensitive heuristic for max-sat. Journal of Experimental Algorithmics (JEA), 2:2–es. Battiti, R. and Tecchiolli, G. (1994). The reactive tabu search. ORSA journal on computing, 6(2):126–140. Bello, I., Pham, H., Le, Q. V., Norouzi, M., and Bengio, S. (2016). Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940. Bengio, Y., Lodi, A., and Prouvost, A. (2021). Machine learning for combinatorial optimization: a methodological tour d’horizon. European Journal of Operational Research, 290(2):405–421. Benlic, U. and Hao, J.-K. (2013). Breakout local search for the max-cutproblem. Engineering Applications of Artificial Intelligence, 26(3):1162–1173. Boettcher, S. and Percus, A. G. (2001). Extremal optimization for graph partitioning. Physical Review E, 64(2):026114. CPLEX (2023). Cplex. https://www.ibm.com/products/ilog-cplexoptimization-studio/cplex-optimizer. Dong, Y., Goldberg, A. V., Noe, A., Parotsidis, N., Resende, M. G., and Spaen, Q. (2021). New instances for maximum weight independent set from a vehicle routing application. In Operations Research Forum, volume 2, pages 1–6. Springer. E´en, N. and S¨orensson, N. (2003). An extensible satsolver. In International conference on theory and applications of satisfiability testing, pages 502–518. Springer. Elsokkary, N., Khan, F. S., La Torre, D., Humble, T. S., and Gottlieb, J. (2017). Financial portfolio management using d-wave quantum optimizer: The case of abu dhabi securities exchange. Technical report, Oak Ridge National Lab.(ORNL), Oak Ridge, TN (United States). Erd˝os, P., R´enyi, A., et al. (1960). On the evolution of random graphs. Publ. math. inst. hung. acad. sci, 5(1):17–60. Festa, P. and Resende, M. G. (2009). Hybrid grasp heuristics. In Foundations of Computational Intelligence Volume 3: Global Optimization, pages 75–100. Springer. Flint, A. and Blaschko, M. (2012). Perceptron learning of sat. Advances in neural information processing systems, 25. Fujishige, S. (2005). Submodular functions and optimization. Elsevier. Ganesh, V., Singh, R., Near, J. P., and Rinard, M. (2009). Avatarsat: An auto-tuning boolean sat solver. Glover, F. (1990). Tabu search: A tutorial. Interfaces, 20(4):74–94. Goemans, M. X. and Williamson, D. P. (1995). Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM (JACM), 42(6):1115–1145. Grozea, C. and Popescu, M. (2014). Can machine learning learn a decision oracle for np problems? a test on sat. Fundamenta Informaticae, 131(3- 4):441–450. Haim, S. and Walsh, T. (2009). Restart strategy selection using machine learning techniques. In International Conference on Theory and Applications of Satisfiability Testing, pages 312–325. Springer. Helmberg, C. and Rendl, F. (2000). A spectral bundle method for semidefinite programming. SIAM Journal on Optimization, 10(3):673–696. Hoos, H. H. et al. (2002). An adaptive noise mechanism for walksat. In AAAI/IAAI, pages 655–660. Jaszczur, S., Luszczyk, M., and Michalewski, H. (2020). Neural heuristics for sat solving. arXiv preprint arXiv:2005.13406. Karp, R. M. (1972). Reducibility among Combinatorial Problems, pages 85–103. Springer US, Boston, MA. Khalil, E., Dai, H., Zhang, Y., Dilkina, B., and Song, L. (2017). Learning combinatorial optimization algorithms over graphs. Advances in neural information processing systems, 30. KhudaBukhsh, A. R., Xu, L., Hoos, H. H., and LeytonBrown, K. (2016). Satenstein: Automatically building local search sat solvers from components. Artificial Intelligence, 232:20–42. Kirkpatrick, S., Gelatt Jr, C. D., and Vecchi, M. P. (1983). Optimization by simulated annealing. science, 220(4598):671–680. Kurin, V., Godil, S., Whiteson, S., and Catanzaro, B. (2020). Can q-learning with graph networks learn a generalizable branching heuristic for a sat solver? Advances in Neural Information Processing Systems, 33:9608–9621. Lauria, M., Elffers, J., Nordstr¨om, J., and Vinyals, M. (2017). Cnfgen: A generator of crafted benchmarks. In Theory and Applications of Satisfiability Testing–SAT 2017: 20th International Conference, Melbourne, VIC, Australia, August 28–September 1, 2017, Proceedings 20, pages 464–473. Springer. Lederman, G., Rabe, M., Seshia, S., and Lee, E. A. (2019). Learning heuristics for quantified boolean formulas through reinforcement learning. In International Conference on LearningRepresentations. Leleu, T., Yamamoto, Y., McMahon, P. L., and Aihara, K. (2019). Destabilization of local minima in analog spin systems by correction of amplitude heterogeneity. Physical review letters,122(4):040607. Li, C. M., Wei, W., and Zhang, H. (2007). Combining adaptive noise and look-ahead in local search for sat. In Theory and Applications of Satisfiability Testing– SAT 2007: 10th International Conference, Lisbon, Portugal, May 28-31, 2007. Proceedings 10, pages 121–133. Springer. Liang, J. H., Ganesh, V., Poupart, P., and Czarnecki, K. (2016). Learning rate based branching heuristic for sat solvers. In Theory and Applications of Satisfiability Testing–SAT 2016: 19th International Conference, Bordeaux, France, July 5-8, 2016, Proceedings 19, pages 123–140. Springer. Manchanda, S., Mittal, A., Dhawan, A., Medya, S., Ranu, S., and Singh, A. (2019). Learning heuristics over large graphs via deep reinforcement learning. arXiv preprint arXiv:1903.03332. Mazure, B., Sais, L., and Gr´egoire, E. (1997). Tabu ´ search for sat. In Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence, pages 281–285. McAllester, D., Selman, B., Kautz, H., et al. (1997). Evidence for invariants in local search. In AAAI/IAAI, pages 321–326. Rhode Island, USA. Mitchell, D., Selman, B., Levesque, H., et al. (1992). Hard and easy distributions of sat problems. In Aaai, volume 92, pages 459–465. Perdomo-Ortiz, A., Dickson, N., Drew-Brook, M., Rose, G., and Aspuru-Guzik, A. (2012). Finding low-energy conformations of lattice protein models by quantum annealing. Scientific reports, 2(1):1–7. Selman, B., Kautz, H. A., Cohen, B., et al. (1993). Local search strategies for satisfiability testing. Cliques, coloring, and satisfiability, 26:521–532. Selman, B., Mitchell, D. G., and Levesque, H. J. (1996). Generating hard satisfiability problems. Artificial intelligence, 81(1-2):17–29. Tiunov, E. S., Ulanov, A. E., and Lvovsky, A. (2019). Annealing by simulating the coherent ising machine. Optics express, 27(7):10288–10295. Toenshoff, J., Ritzert, M., Wolf, H., and Grohe, M. (2019). Graph neural networks for maximum constraint satisfaction. Frontiers in artificial intelligence, 3:580607. Venturelli, D. and Kondratyev, A. (2019). Reverse quantum annealing approach to portfolio optimization problems. Quantum Machine Intelligence, 1(1- 2):17–30. Wolpert, D. H. and Macready, W. G. (1997). No free lunch theorems for optimization. IEEE transactions on evolutionary computation, 1(1):67–82. Ye, Y. (2003). The gset dataset. https://web.stanford.edu/ yyye/yyye/Gset/. Yolcu, E. and P´oczos, B. (2019). Learning local search heuristics for boolean satisfiability. Advances in Neural Information Processing Systems, 32. This paper is available on arxiv under CC 4.0 DEED license. Authors: (1) Ankur Nath, Department of Computer Science and Engineering, Texas A&M University; (2) Alan Kuhnle, Department of Computer Science and Engineering, Texas A&M University. Authors: Authors: (1) Ankur Nath, Department of Computer Science and Engineering, Texas A&M University; (2) Alan Kuhnle, Department of Computer Science and Engineering, Texas A&M University. Table of Links Abstract & Introduction Abstract & Introduction Related work Related work Evaluation for Max-Cut Evaluation for Max-Cut Evaluation for SAT Evaluation for SAT Summary and Outlook, References Summary and Outlook, References Supplementary Materials Supplementary Materials 5 SUMMARY and OUTLOOK Through our empirical evaluations, our goal is to promote an insightful comparison within the research focusing on the intersection of combinatorial optimization and machine learning. In order to provide the research community with valuable guidance, we believe it is imperative to communicate both the strengths and weaknesses of the proposed approaches. Poor instances and baseline selection may give the wrong impression about the performance of learned heuristics. Specifically, it is important to articulate the degree of improvement achieved through integrating classical heuristics with deep learning architectures and conduct a thorough comparison with classical heuristics. This will aid in elucidating the degree to which deep learning architectures enhance integrated heuristics. It assists in ascertaining whether the integration endeavor is justified and warrants the allocation of computational resources, time, and investment necessary for integrating deep learning with classical heuristics. References Albert, R. and Barab´asi, A.-L. (2002). Statistical mechanics of complex networks. Reviews of modern physics, 74(1):47. Barrett, T., Clements, W., Foerster, J., and Lvovsky, A. (2020). Exploratory combinatorial optimization with reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 3243–3250. Barrett, T. D., Parsonson, C. W., and Laterre, A. (2022). Learning to solve combinatorial graph partitioning problems via efficient exploration. arXiv preprint arXiv:2205.14105. Battiti, R. and Protasi, M. (1997). Reactive search, a history-sensitive heuristic for max-sat. Journal of Experimental Algorithmics (JEA), 2:2–es. Battiti, R. and Tecchiolli, G. (1994). The reactive tabu search. ORSA journal on computing, 6(2):126–140. Bello, I., Pham, H., Le, Q. V., Norouzi, M., and Bengio, S. (2016). Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940. Bengio, Y., Lodi, A., and Prouvost, A. (2021). Machine learning for combinatorial optimization: a methodological tour d’horizon. European Journal of Operational Research, 290(2):405–421. Benlic, U. and Hao, J.-K. (2013). Breakout local search for the max-cutproblem. Engineering Applications of Artificial Intelligence, 26(3):1162–1173. Boettcher, S. and Percus, A. G. (2001). Extremal optimization for graph partitioning. Physical Review E, 64(2):026114. CPLEX (2023). Cplex. https://www.ibm.com/products/ilog-cplexoptimization-studio/cplex-optimizer. Dong, Y., Goldberg, A. V., Noe, A., Parotsidis, N., Resende, M. G., and Spaen, Q. (2021). New instances for maximum weight independent set from a vehicle routing application. In Operations Research Forum, volume 2, pages 1–6. Springer. E´en, N. and S¨orensson, N. (2003). An extensible satsolver. In International conference on theory and applications of satisfiability testing, pages 502–518. Springer. Elsokkary, N., Khan, F. S., La Torre, D., Humble, T. S., and Gottlieb, J. (2017). Financial portfolio management using d-wave quantum optimizer: The case of abu dhabi securities exchange. Technical report, Oak Ridge National Lab.(ORNL), Oak Ridge, TN (United States). Erd˝os, P., R´enyi, A., et al. (1960). On the evolution of random graphs. Publ. math. inst. hung. acad. sci, 5(1):17–60. Festa, P. and Resende, M. G. (2009). Hybrid grasp heuristics. In Foundations of Computational Intelligence Volume 3: Global Optimization, pages 75–100. Springer. Flint, A. and Blaschko, M. (2012). Perceptron learning of sat. Advances in neural information processing systems, 25. Fujishige, S. (2005). Submodular functions and optimization. Elsevier. Ganesh, V., Singh, R., Near, J. P., and Rinard, M. (2009). Avatarsat: An auto-tuning boolean sat solver. Glover, F. (1990). Tabu search: A tutorial. Interfaces, 20(4):74–94. Goemans, M. X. and Williamson, D. P. (1995). Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM (JACM), 42(6):1115–1145. Grozea, C. and Popescu, M. (2014). Can machine learning learn a decision oracle for np problems? a test on sat. Fundamenta Informaticae, 131(3- 4):441–450. Haim, S. and Walsh, T. (2009). Restart strategy selection using machine learning techniques. In International Conference on Theory and Applications of Satisfiability Testing, pages 312–325. Springer. Helmberg, C. and Rendl, F. (2000). A spectral bundle method for semidefinite programming. SIAM Journal on Optimization, 10(3):673–696. Hoos, H. H. et al. (2002). An adaptive noise mechanism for walksat. In AAAI/IAAI, pages 655–660. Jaszczur, S., Luszczyk, M., and Michalewski, H. (2020). Neural heuristics for sat solving. arXiv preprint arXiv:2005.13406. Karp, R. M. (1972). Reducibility among Combinatorial Problems, pages 85–103. Springer US, Boston, MA. Khalil, E., Dai, H., Zhang, Y., Dilkina, B., and Song, L. (2017). Learning combinatorial optimization algorithms over graphs. Advances in neural information processing systems, 30. KhudaBukhsh, A. R., Xu, L., Hoos, H. H., and LeytonBrown, K. (2016). Satenstein: Automatically building local search sat solvers from components. Artificial Intelligence, 232:20–42. Kirkpatrick, S., Gelatt Jr, C. D., and Vecchi, M. P. (1983). Optimization by simulated annealing. science, 220(4598):671–680. Kurin, V., Godil, S., Whiteson, S., and Catanzaro, B. (2020). Can q-learning with graph networks learn a generalizable branching heuristic for a sat solver? Advances in Neural Information Processing Systems, 33:9608–9621. Lauria, M., Elffers, J., Nordstr¨om, J., and Vinyals, M. (2017). Cnfgen: A generator of crafted benchmarks. In Theory and Applications of Satisfiability Testing–SAT 2017: 20th International Conference, Melbourne, VIC, Australia, August 28–September 1, 2017, Proceedings 20, pages 464–473. Springer. Lederman, G., Rabe, M., Seshia, S., and Lee, E. A. (2019). Learning heuristics for quantified boolean formulas through reinforcement learning. In International Conference on LearningRepresentations. Leleu, T., Yamamoto, Y., McMahon, P. L., and Aihara, K. (2019). Destabilization of local minima in analog spin systems by correction of amplitude heterogeneity. Physical review letters,122(4):040607. Li, C. M., Wei, W., and Zhang, H. (2007). Combining adaptive noise and look-ahead in local search for sat. In Theory and Applications of Satisfiability Testing– SAT 2007: 10th International Conference, Lisbon, Portugal, May 28-31, 2007. Proceedings 10, pages 121–133. Springer. Liang, J. H., Ganesh, V., Poupart, P., and Czarnecki, K. (2016). Learning rate based branching heuristic for sat solvers. In Theory and Applications of Satisfiability Testing–SAT 2016: 19th International Conference, Bordeaux, France, July 5-8, 2016, Proceedings 19, pages 123–140. Springer. Manchanda, S., Mittal, A., Dhawan, A., Medya, S., Ranu, S., and Singh, A. (2019). Learning heuristics over large graphs via deep reinforcement learning. arXiv preprint arXiv:1903.03332. Mazure, B., Sais, L., and Gr´egoire, E. (1997). Tabu ´ search for sat. In Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence, pages 281–285. McAllester, D., Selman, B., Kautz, H., et al. (1997). Evidence for invariants in local search. In AAAI/IAAI, pages 321–326. Rhode Island, USA. Mitchell, D., Selman, B., Levesque, H., et al. (1992). Hard and easy distributions of sat problems. In Aaai, volume 92, pages 459–465. Perdomo-Ortiz, A., Dickson, N., Drew-Brook, M., Rose, G., and Aspuru-Guzik, A. (2012). Finding low-energy conformations of lattice protein models by quantum annealing. Scientific reports, 2(1):1–7. Selman, B., Kautz, H. A., Cohen, B., et al. (1993). Local search strategies for satisfiability testing. Cliques, coloring, and satisfiability, 26:521–532. Selman, B., Mitchell, D. G., and Levesque, H. J. (1996). Generating hard satisfiability problems. Artificial intelligence, 81(1-2):17–29. Tiunov, E. S., Ulanov, A. E., and Lvovsky, A. (2019). Annealing by simulating the coherent ising machine. Optics express, 27(7):10288–10295. Toenshoff, J., Ritzert, M., Wolf, H., and Grohe, M. (2019). Graph neural networks for maximum constraint satisfaction. Frontiers in artificial intelligence, 3:580607. Venturelli, D. and Kondratyev, A. (2019). Reverse quantum annealing approach to portfolio optimization problems. Quantum Machine Intelligence, 1(1- 2):17–30. Wolpert, D. H. and Macready, W. G. (1997). No free lunch theorems for optimization. IEEE transactions on evolutionary computation, 1(1):67–82. Ye, Y. (2003). The gset dataset. https://web.stanford.edu/ yyye/yyye/Gset/. Yolcu, E. and P´oczos, B. (2019). Learning local search heuristics for boolean satisfiability. Advances in Neural Information Processing Systems, 32. This paper is available on arxiv under CC 4.0 DEED license. This paper is available on arxiv under CC 4.0 DEED license. available on arxiv

Assessing the Justification for Integrating Deep Learning in Combinatorial Optimization

Authors: (1) Sha-Bo Lin, Center for Intelligent Decision-Making and Machine Learning, School of Management, Xi’an Jiaotong University; (2) Xingping Sun, Department of Mathematics, Missouri State University; (3) Di Wang, §Center for Intelligent Decision-Making and Machine Learning, School of Management, Xi’an Jiaotong University. Content Overview Abstract & Introduction Uncertainty Relation of Kernel Interpolation on Spheres Distributed Kernel Interpolation Operator Differences via Quadrature Rules Proofs Numerical Verifications References Abstract For radial basis function (RBF) kernel interpolation of scattered data, Schaback [30] in 1995 proved that the attainable approximation error and the condition number of the underlying interpolation matrix cannot be made small simultaneously. He referred to this finding as an “uncertainty relation”, an undesirable consequence of which is that RBF kernel interpolation is susceptible to noisy data. In this paper, we propose and study a distributed interpolation method to manage and quantify the uncertainty brought on by interpolating noisy spherical data of non-negligible magnitude. We also present numerical simulation results showing that our method is practical and robust in terms of handling noisy data from challenging computing environments. Kernel interpolation, distributed uncertainty mitigation, scattered spherical data Key words. Uncertainty Relation of Kernel Interpolation on Spheres 3. Distributed Kernel Interpolation. Corollary 2.2 shows that kernel interpolation performs poorly while confronting noisy data of non-negligible magnitude. To overcome this major drawback, we propose and study in this section a distributed kernel interpolation (DKI) method, which is motivated by the “distributed learning” in the literature [37, 19]. Figuratively speaking, this is a divide-and-conquer strategy for uncertainty quantification. To elaborate, we describe the method in three steps. 4. Operator Differences via Quadrature Rules. In this section, we first briefly exposit an integral-operator approach initiated in [8] and then derive tight upper bounds for differences of operators of our interest, obtaining a certain type of Sobolev sampling inequalities [12] as a byproduct. Highlights of the section include Proposition 4.5) and Lemma 4.8. 6. Numerical Verifications Four simulations are carried out in this section to verify the excellent performance of DKI. The first one shows that DKI succeeds in circumventing the uncertainty of kernel interpolation. The second one exhibits the role of m in DKI. The third one focuses on the role of the division strategy in DKI. The last one compares DKI with several popular spherical data fitting schemes including the distributed filtered hyperinterpolation (DFH) [21], sketching with s ∗ -designs [20], and distributed kernel ridge regression (DKRR) [8]. In this simulation, we show the role of the parameter m in DKI. We generate 10014 training samples (with 141-designs as inputs). The number of divisions, m, ranges from {5, 10, · · · , 200}. Figure 6.2 shows the relation between RMSE of DKI and the number of local machines under different levels of Gaussian noise, provided that the total number of training samples is given. From Figure 6.2, we can conclude the following assertions: 1) For training samples with higher levels of noise, the testing RMSE generally decreases at first and then increases slowly as the number of local machines increases. Moderate values of m are more conductive to good approximation property for DKI. The reason is that too small m does not successfully address the uncertainty issue in kernel interpolation; too large m increases the fitting error, resulting in slightly worse generalization performance. 2) The optimal number m with the lowest RMSE grows with increasing Gaussian noise. This verifies the equation (3.3) of Theorem 3.2, in which the approximation error is primarily concerned with the sample error for large noise (i.e., large M) and can be reduced using a large m. Simulation 2: REFERENCES [1] R. Bhatia, Matrix Analysis, vol. 169, Springer Science & Business Media, 2013. [2] J. S. Brauchart and K. Hesse, Numerical integration over spheres of arbitrary dimension, Constructive Approximation, 25 (2007), pp. 41–71. [3] G. Brown and F. Dai, Approximation of smooth functions on compact two-point homogeneous spaces, Journal of Functional Analysis, 220 (2005), pp. 401–423. [4] A. Chernih, I. H. Sloan, and R. S. Womersley, Wendland functions with increasing smoothness converge to a gaussian, Advances in Computational Mathematics, 40 (2014), pp. 185– 200. [5] F. Dai, Multivariate polynomial inequalities with respect to doubling weights and a∞ weights, Journal of Functional Analysis, 235 (2006), pp. 137–170. [6] F. Dai, On generalized hyperinterpolation on the sphere, Proceedings of the American Mathematical Society, 134 (2006), pp. 2931–2941. [7] H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems, vol. 375, Springer Science & Business Media, 1996. [8] H. Feng, S.-B. Lin, and D.-X. Zhou, Radial basis function approximation with distributively stored data on spheres, arXiv:2112.02499, (2021). [9] T. Hangelbroek, F. J. Narcowich, X. Sun, and J. D. Ward, Kernel approximation on manifolds ii: The l∞ norm of the l2 projector, SIAM Journal on Mathematical Analysis, 43 (2011), pp. 662–684. [10] T. Hangelbroek, F. J. Narcowich, and J. D. Ward, Kernel approximation on manifolds i: bounding the lebesgue constant, SIAM Journal on Mathematical Analysis, 42 (2010), pp. 1732–1760. [11] K. Hesse, I. H. Sloan, and R. S. Womersley, Radial basis function approximation of noisy scattered data on the sphere, Numerische Mathematik, 137 (2017), pp. 579–605. [12] K. Hesse, I. H. Sloan, and R. S. Womersley, Local rbf-based penalized least-squares approximation on the sphere with noisy scattered data, Journal of Computational and Applied Mathematics, 382 (2021), p. 113061. [13] S. Hubbert, Q. T. Le Gia, and T. M. Morton ˆ , Spherical radial basis functions, theory and applications, Springer, 2015. [14] M. A. King, R. J. Bingham, P. Moore, P. L. Whitehouse, M. J. Bentley, and G. A. Milne, Lower satellite-gravimetry estimates of antarctic sea-level contribution, Nature, 491 (2012), pp. 586–589. [15] Q. T. Le Gia, F. J. Narcowich, J. D. Ward, and H. Wendland, Continuous and discrete least-squares approximation by radial basis functions on spheres, Journal of Approximation Theory, 143 (2006), pp. 124–133. [16] P. Leopardi, A partition of the unit sphere into regions of equal area and small diameter, Electronic Transactions on Numerical Analysis, 25 (2006), pp. 309–327. [17] J. Levesley, Z. Luo, and X. Sun, Norm estimates of interpolation matrices and their inverses associated with strictly positive definite functions, Proceedings of the American Mathematical Society, (1999), pp. 2127–2134. [18] S.-B. Lin, X. Chang, and X. Sun, Kernel interpolation of high dimensional scattered data, arXiv preprint arXiv:2009.01514, (2020). [19] S.-B. Lin, X. Guo, and D.-X. Zhou, Distributed learning with regularized least squares, The Journal of Machine Learning Research, 18 (2017), pp. 3202–3232. [20] S.-B. Lin, D. Wang, and D.-X. Zhou, Sketching with spherical designs on spheres, SIAM Journal on Scientific Computation, in press (2023). [21] S.-B. Lin, Y. G. Wang, and D.-X. Zhou, Distributed filtered hyperinterpolation for noisy data on the sphere, SIAM Journal on Numerical Analysis, 59 (2021), pp. 634–659. [22] J. D. McEwen and Y. Wiaux, A novel sampling theorem on the sphere, IEEE Transactions on Signal Processing, 59 (2011), pp. 5876–5887. [23] H. Mhaskar, F. Narcowich, and J. Ward, Spherical marcinkiewicz-zygmund inequalities and positive quadrature, Mathematics of Computation, 70 (2001), pp. 1113–1130. [24] H. N. Mhaskar, Weighted quadrature formulas and approximation by zonal function networks on the sphere, Journal of Complexity, 22 (2006), pp. 348–370. [25] C. Muller ¨ , Spherical Harmonics, vol. 17, Springer, 1966. [26] F. J. Narcowich, N. Sivakumar, and J. D. Ward, Stability results for scattered-data interpolation on euclidean spheres, Advances in Computational Mathematics, 8 (1998), pp. 137– 163. [27] F. J. Narcowich, X. Sun, J. D. Ward, and H. Wendland, Direct and inverse sobolev error estimates for scattered data interpolation via spherical basis functions, Foundations of Computational Mathematics, 7 (2007), pp. 369–390. [28] F. J. Narcowich and J. D. Ward, Scattered data interpolation on spheres: error estimates and locally supported basis functions, SIAM Journal on Mathematical Analysis, 33 (2002), pp. 1393–1410. [29] A. Rudi, R. Camoriano, and L. Rosasco, Less is more: Nystr¨om computational regularization., in NIPS, 2015, pp. 1657–1665. [30] R. Schaback, Error estimates and condition numbers for radial basis function interpolation, Advances in Computational Mathematics, 3 (1995), pp. 251–264. [31] S. Smale and D.-X. Zhou, Shannon sampling and function reconstruction from point values, Bulletin of the American Mathematical Society, 41 (2004), pp. 279–305. [32] S. Smale and D.-X. Zhou, Shannon sampling ii: Connections to learning theory, Applied and Computational Harmonic Analysis, 19 (2005), pp. 285–302. [33] Y.-T. Tsai and Z.-C. Shih, All-frequency precomputed radiance transfer using spherical radial basis functions and clustered tensor approximation, ACM Transactions on Graphics (TOG), 25 (2006), pp. 967–976. [34] H. Wendland, Scattered data approximation, vol. 17, Cambridge university press, 2004. [35] M. A. Wieczorek and R. J. Phillips, Potential anomalies on a sphere: Applications to the thickness of the lunar crust, Journal of Geophysical Research: Planets, 103 (1998), pp. 1715–1724. [36] R. S. Womersley, Efficient spherical designs with good geometric properties, in Contemporary computational mathematics-A celebration of the 80th birthday of Ian Sloan, Springer, 2018, pp. 1243–1285. [37] Y. Zhang, J. Duchi, and M. Wainwright, Divide and conquer kernel ridge regression: A distributed algorithm with minimax optimal rates, The Journal of Machine Learning Research, 16 (2015), pp. 3299–3340. In this Appendix, we present the detailed implementation of the select-and-judge (SAJ) strategy. Our aim is to derive a series of subsets of similar Cardinality with separation radius not smaller than a given tolerance c0. There are two stages for SAJ. SM1. Appendix A: Select-and-Judge Strategy for Data Division. This paper is under CC0 1.0 DEED license. available on arxiv

SIAM COMPUTING

siamcomputing.com

-- employees

Since n.d.

COMPANY RANKING

SIAM COMPUTING

EVERGREEN INDEX #9512

Representing this company? Click here to claim and customize this page!

Siam Computing WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

SIAM COMPUTING

siamcomputing.com

-- employees

Since n.d.

COMPANY RANKING

SIAM COMPUTING

EVERGREEN INDEX #9512

RELATED COMPANIESRANK

STORIES ABOUT SIAM COMPUTING

Representing this company? Click here to claim and customize this page!

Siam Computing WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!