Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Abstract and 1. Introduction Related Works
2.1 Traditional Index Selection Approaches
2.2 RL-based Index Selection Approaches


Index Selection Problem


Methodology
4.1 Formulation of the DRL Problem
4.2 Instance-Aware Deep Reinforcement Learning for Efficient Index Selection


System Framework of IA2
5.1 Preprocessing Phase
5.2 RL Training and Application Phase


Experiments
6.1 Experimental Setting
6.2 Experimental Results
6.3 End-to-End Performance Comparison
6.4 Key Insights


Conclusion and Future Work, and References Related Works
2.1 Traditional Index Selection Approaches
2.2 RL-based Index Selection Approaches Related Works Related Works 2.1 Traditional Index Selection Approaches 2.1 Traditional Index Selection Approaches 2.2 RL-based Index Selection Approaches 2.2 RL-based Index Selection Approaches Index Selection Problem Index Selection Problem Index Selection Problem Index Selection Problem Methodology
4.1 Formulation of the DRL Problem
4.2 Instance-Aware Deep Reinforcement Learning for Efficient Index Selection Methodology Methodology 4.1 Formulation of the DRL Problem 4.1 Formulation of the DRL Problem 4.2 Instance-Aware Deep Reinforcement Learning for Efficient Index Selection 4.2 Instance-Aware Deep Reinforcement Learning for Efficient Index Selection System Framework of IA2
5.1 Preprocessing Phase
5.2 RL Training and Application Phase System Framework of IA2 System Framework of IA2 System Framework of IA2 5.1 Preprocessing Phase 5.1 Preprocessing Phase 5.2 RL Training and Application Phase 5.2 RL Training and Application Phase Experiments
6.1 Experimental Setting
6.2 Experimental Results
6.3 End-to-End Performance Comparison
6.4 Key Insights Experiments Experiments 6.1 Experimental Setting 6.1 Experimental Setting 6.2 Experimental Results 6.2 Experimental Results 6.3 End-to-End Performance Comparison 6.3 End-to-End Performance Comparison 6.4 Key Insights 6.4 Key Insights Conclusion and Future Work, and References Conclusion and Future Work, and References Conclusion and Future Work, and References Conclusion and Future Work, and References 6 Experiments Our experiments are designed to evaluate the IA2 on several critical aspects of database optimization and index selection. Specifically, we aim to (1) analyze the performance of IA2’s core algorithm, TD3-TD-SWAR, against other reinforcement learning algorithms, showcasing its unique strengths and contributions; (2) assess the efficiency of the action masking technique in IA2 for action space reduction and learning process acceleration; and (3) measure the end-to-end (E2E) workload runtime improvements achieved with IA2, highlighting its practical impact on database performance. 6.1 Experimental Setting Implementation and Environment: Our prototype is implemented in Python, utilizing PyTorch for model development. Interfaced with PostgreSQL 15.6, it integrates HypoPG for what-if analysis, aiding in query cost estimation. Experiments are conducted on a virtual machine powered by a shared Nvidia Quadro RTX8000 GPU and equipped with 8 CPU cores, within a single-threaded SQL-DB environment. Implementation and Environment Benchmark Workloads: TPC-H (SF1) forms the basis for seven workloads (W1 - W7), derived from its 22 query templates plus additional queries for a broad evaluation scope. Each workload contains 50 queries, with complexity reflected in the diversity of tables and attributes. W7, uniquely, serves as a test for IA2’s ability to generalize, being unseen during training. W1-W6 are used for standard training and evaluation, while W7 undergoes slight fine-tuning on a subset of the training set for performance assessment on novel queries. Workload outlines are depicted in Figure 4. Benchmark Workloads Competitors: Our evaluation includes comparisons with SWIRL, DRLinda, Extend, and Lan et al., as discussed in Section 2, to benchmark IA2 against the state-of-the-art in index selection. Comparison of these selected RL-based index advisors is shown in Tabel 1 Competitors SWIRL DRLinda Extend Evaluation Metrics: The primary metric for assessing IA2 and its competitors is the end-to-end runtime of workloads, using the performance gain ratio for direct optimization comparisons across index advising methods. The evaluation covers trends in Storage Budget (2-8), Workloads, and Training Episodes (50-400), with storage quantified in units where 1 unit equals 128MB. Evaluation Metrics: Authors:
(1) Taiyi Wang, University of Cambridge, Cambridge, United Kingdom (Taiyi.Wang@cl.cam.ac.uk);
(2) Eiko Yoneki, University of Cambridge, Cambridge, United Kingdom (eiko.yoneki@cl.cam.ac.uk). Authors: Authors: (1) Taiyi Wang, University of Cambridge, Cambridge, United Kingdom (Taiyi.Wang@cl.cam.ac.uk); (2) Eiko Yoneki, University of Cambridge, Cambridge, United Kingdom (eiko.yoneki@cl.cam.ac.uk). This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Arweave

PostgreSQL & HypoPG: The Experimental Foundation of IA2 Index Selection

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

3D Mapping Initialization: Using RGB-D Images and Camera Parameters

Reducing TPC-H Workload Runtime by 40% with IA2 Deep Reinforcement Learning

Adaptive Action Masking: Accelerating Decision-Making in Database Tuning

Beyond Preprocessing: Sequential Indexing and Temporal Consistency in IA2

It Is Okay If You Don't Know What You Like. We Do (feat. Deep Recommendation Algorithms)

10 Machine Learning, Data Science, and Deep Learning Courses for Programmers in 2020

3D Mapping Initialization: Using RGB-D Images and Camera Parameters

Reducing TPC-H Workload Runtime by 40% with IA2 Deep Reinforcement Learning

Adaptive Action Masking: Accelerating Decision-Making in Database Tuning

Beyond Preprocessing: Sequential Indexing and Temporal Consistency in IA2

It Is Okay If You Don't Know What You Like. We Do (feat. Deep Recommendation Algorithms)

10 Machine Learning, Data Science, and Deep Learning Courses for Programmers in 2020

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps