Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Motivation and design goals


Related Work


Conformal prediction
4.1. Mondrian conformal prediction (MCP)
4.2. Evaluation metrics


Mondrian conformal prediction for Disk Scrubbing: our approach
5.1. System and Storage statistics
5.2. Which disk to scrub: Drive health predictor
5.3. When to scrub: Workload predictor


Experimental setting and 6.1. Open-source Baidu dataset
6.2. Experimental results


Discussion
7.1. Optimal scheduling aspect
7.2. Performance metrics and 7.3. Power saving from selective scrubbing


Conclusion and References Motivation and design goals Motivation and design goals Motivation and design goals Related Work Related Work Related Work Conformal prediction
4.1. Mondrian conformal prediction (MCP)
4.2. Evaluation metrics Conformal prediction Conformal prediction 4.1. Mondrian conformal prediction (MCP) 4.1. Mondrian conformal prediction (MCP) 4.2. Evaluation metrics 4.2. Evaluation metrics Mondrian conformal prediction for Disk Scrubbing: our approach
5.1. System and Storage statistics
5.2. Which disk to scrub: Drive health predictor
5.3. When to scrub: Workload predictor Mondrian conformal prediction for Disk Scrubbing: our approach Mondrian conformal prediction for Disk Scrubbing: our approach 5.1. System and Storage statistics 5.1. System and Storage statistics 5.2. Which disk to scrub: Drive health predictor 5.2. Which disk to scrub: Drive health predictor 5.3. When to scrub: Workload predictor 5.3. When to scrub: Workload predictor Experimental setting and 6.1. Open-source Baidu dataset
6.2. Experimental results Experimental setting and 6.1. Open-source Baidu dataset Experimental setting and 6.1. Open-source Baidu dataset 6.2. Experimental results 6.2. Experimental results Discussion
7.1. Optimal scheduling aspect
7.2. Performance metrics and 7.3. Power saving from selective scrubbing Discussion Discussion 7.1. Optimal scheduling aspect 7.1. Optimal scheduling aspect 7.2. Performance metrics and 7.3. Power saving from selective scrubbing 7.2. Performance metrics and 7.3. Power saving from selective scrubbing Conclusion and References Conclusion and References Conclusion and References 7. Discussion The proposed method for disk identification for scrubbing offers a dual benefit. Firstly, it can be utilized to assess the reliability of the storage system. Secondly, it employs a disk ranking mechanism to assign relative health scores to individual disks. The choice of classification algorithm depends on factors such as dataset size and available compute resources. However, the decision can be guided by the expertise of the system administrator. In addition, we discuss how the use of the Mondrian conformal predictor can aid in identifying latent failures of disks, which could be a potential area for future research. Furthermore, we identify three key aspects for designing optimal scheduling and cover performance metrics, including effective coverage and size of the average prediction set. Lastly, we provide a hypothetical evaluation of energy and power savings resulting from selective scrubbing. This showcases the potential benefits of the proposed method in terms of reduced power and energy consumption, highlighting its effectiveness in optimizing disk scrubbing operations. 7.1. Optimal scheduling aspect With respect to disk scrubbing frequency scheduling, we can design three aspects of scheduling: time window, frequency, and space allocation. Each of them is described below: • Time window focuses on scheduling the time window for scrubbing based on the workload pattern. Scrubbing is done when the system is predicted to be idle. • Frequency involves scheduling the frequency of scrubbing based on the health status of the drive. For drives with the best health, scrubbing is done less frequently. For drives with medium health, scrubbing is done more frequently. • Space deals with scheduling space allocation based on the spatial and temporal locality of sector errors. Instant scrubbing is performed on problematic chunks to ensure efficient disk scrubbing. This paper is available on arxiv under CC BY-NC-ND 4.0 Deed (Attribution-Noncommercial-Noderivs 4.0 International) license. This paper is available on arxiv under CC BY-NC-ND 4.0 Deed (Attribution-Noncommercial-Noderivs 4.0 International) license. available on arxiv Authors:
(1) Rahul Vishwakarma, California State University Long Beach, 1250 Bellflower Blvd, Long Beach, CA 90840, United States (rahuldeo.vishwakarma01@student.csullb.edu);
(2) Jinha Hwang, California State University Long Beach, 1250 Bellflower Blvd, Long Beach, CA 90840, United States (jinha.hwang01@student.csulb.edu);
(3) Soundouss Messoudi, HEUDIASYC - UMR CNRS 7253, Universit´e de Technologie de Compiegne, 57 avenue de Landshut, 60203 Compiegne Cedex - France (soundouss.messoudi@hds.utc.fr);
(4) Ava Hedayatipour, California State University Long Beach, 1250 Bellflower Blvd, Long Beach, CA 90840, United States (ava.hedayatipour@csulb.edu). Authors: Authors: (1) Rahul Vishwakarma, California State University Long Beach, 1250 Bellflower Blvd, Long Beach, CA 90840, United States (rahuldeo.vishwakarma01@student.csullb.edu); (2) Jinha Hwang, California State University Long Beach, 1250 Bellflower Blvd, Long Beach, CA 90840, United States (jinha.hwang01@student.csulb.edu); (3) Soundouss Messoudi, HEUDIASYC - UMR CNRS 7253, Universit´e de Technologie de Compiegne, 57 avenue de Landshut, 60203 Compiegne Cedex - France (soundouss.messoudi@hds.utc.fr); (4) Ava Hedayatipour, California State University Long Beach, 1250 Bellflower Blvd, Long Beach, CA 90840, United States (ava.hedayatipour@csulb.edu).

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Baidu

What If Your Hard Drive Could Predict Its Own Failures?

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

How Machine Learning Optimizes Data Center Disk Health and Power Efficiency

Will Data Centers Ruin Your Neighborhood?

Optimizing Disk Scrubbing in Data Centers with Machine Learning

Mondrian Conformal Prediction for Disk Health Scoring and Scrubbing Optimization

How to Teach Machine Learning Models a Sense of Uncertainty

Why Predictive AI Might Be the Future of Disk Hygiene

How Machine Learning Optimizes Data Center Disk Health and Power Efficiency

Will Data Centers Ruin Your Neighborhood?

Optimizing Disk Scrubbing in Data Centers with Machine Learning

Mondrian Conformal Prediction for Disk Health Scoring and Scrubbing Optimization

How to Teach Machine Learning Models a Sense of Uncertainty

Why Predictive AI Might Be the Future of Disk Hygiene

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps