Authors:
(1) Pavan L. Veluvali, Max Planck Institute for Dynamics of Complex Technical Systems, Sandtorstr. 1, 39106 Magdeburg;
(2) Jan Heiland, Max Planck Institute for Dynamics of Complex Technical Systems, Sandtorstr. 1, 39106 Magdeburg;
(3) Peter Benner, Max Planck Institute for Dynamics of Complex Technical Systems, Sandtorstr. 1, 39106 Magdeburg. Authors: Authors: (1) Pavan L. Veluvali, Max Planck Institute for Dynamics of Complex Technical Systems, Sandtorstr. 1, 39106 Magdeburg; (2) Jan Heiland, Max Planck Institute for Dynamics of Complex Technical Systems, Sandtorstr. 1, 39106 Magdeburg; (3) Peter Benner, Max Planck Institute for Dynamics of Complex Technical Systems, Sandtorstr. 1, 39106 Magdeburg. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Existing Solutions Existing Solutions 2. MaRDIFlow 2. MaRDIFlow Minimum working examples Minimum working examples Spinodal decomposition in a binary A-B alloy Spinodal decomposition in a binary A-B alloy Summary and Outlook, Acknowledgments, Data Availability, and References Summary and Outlook, Acknowledgments, Data Availability, and References 2 MaRDIFlow The design principle of MaRDIFlow is to handle the components as abstract objects described by their input to output behavior and metadata. By means of the metadata and by matching the I/O interfaces, the objects can be chained together to form a workflow; see Fig. 1. Each item then can have different descriptions or realizations that are, in the best case, equivalent and redundant; see Fig. 2 for an example of such a vertical dimension in the workflow. We note that the redundancy is meant from a theoretical perspective. Practically, the different realizations or representations can be used in different scenarios like compiling a mathematical description of the workflow or using lookup tables as a shortcut during a simulation. Additionally, this multi-level description enhances or even enables reproducibility in scenarios where, for example, certain software components of a workflow are not available but will be replaced by data or a description and vice versa. • Defined by metadata • Realized as Fig. 1 and Fig. 2 The working prototype of MaRDIFlow is designed as a command line tool that enables researchers and users to execute, document and maintain the provenance for reproduction and replication of computer-based experiments. As shown in Fig. 3 via a screenshot, MaRDIFlow --help option provides a help message with an immediate sense of what MaRDIFlow is. Through the following list we discuss some of the important arguments required to configure and perform the most common tasks with our RDM tool. • --workflow-title provides the working title for the workflow, as default we have, workflow title = This is a CSE workflow description under MaRDIFlow. • --input requires an inputs object file in .json format such that it consists of all the numerical parameters for the workflow. An example inputs object file is shown in Fig. 4, where the key value pairs are a valid string representing the parameter name, and the values found to the right side of the colon are the absolute values for the given input parameter. As shown in Fig. 4, the JSON object represents the simulation and material parameters which are then passed on to the required workflow component. • --output-directory argument allows the user to specify the desired output directory. If not provided, then the workflow output is collected in the root working directory, namely Output. • MaRDIFlow can also be configured using a .ini config parser file, which includes both the required default and user-defined sections. It also acts as a list of parameters that governs how the tool is run and configured. • --config example config file.ini will execute MaRDIFlow through terminal. An example configparser file is showcased via a screenshot in Fig. 5. • --component argument is necessary to execute the desired workflow component in the multilevel framework, as previously discussed. In present version, users can choose between --math-data or --math-solver components for executing a given I/O data or a numerical model, respectively. • --display represents the descriptive part of our RDM tool. Herein, --display html=TRUE or --display pdf=TRUE will convert the .md into desired format. An empty string will parse FALSE bool value. The file path for workflow description in .md file format is passed on to the workflow tool via --inputmarkdown flag. For this flag, it is important that the absolute file path for input as well as for output is provided. • --data configures the second component of the workflow description for a given I/O data. By utilizing --get-data and --get-url-data flags, users can furnish a workflow description for a lookup table or a database, thereby presenting an alternative approach to a numerical model. Overall, the working prototype of MaRDIFlow ideally provides the user with a specification description, that elucidates the meta-data for a given use case. In addition, it also consists of a computational part that acts as a mechanism to call and execute the given meta-data. In order to further understand our RDM tool, the following minimum working examples implemented in MaRDIFlow are discussed below. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

The code in this story is for educational purposes. The readers are solely responsible for whatever they build with it.

Technical Implementation of MaRDIFlow: Metadata-Driven Workflow Abstraction

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

From Prototype to Promise: MaRDIFlow Charts the Future of Math Computing

New Framework Makes Scientific Computing Workflows Truly Reproducible

From Prototype to Promise: MaRDIFlow Charts the Future of Math Computing

New Framework Makes Scientific Computing Workflows Truly Reproducible

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps