Introducing Paceline — the first-ever fitness rewards platform that adds financial benefit to your physical activity

Paceline

Building a DevOps Automated Deployment Pipeline using Flask, Aptible, CircleCI, and Prometheus

Data is widely considered the lifeblood of an organization; however, it is worthless—and useless—in its raw form. Help is needed to turn this data into life-giving information. Part of this help is to move this data from its source to destination via data pipelines. But managing the end-to-end data pipeline lifecycle is not easy: ones are challenging to scale on demand, others can result in delays in detecting and addressing issues or be difficult to control. These and the other problems may be solved in the declarative programming paradigm, and in this article we will see it on Terraform. Where Does This Data Come From? The data in these extensive datasets come from multiple disparate sources and are typically loaded into datasets in a data storage like ClickHouse via data pipelines. These pipelines play a critical role in this process by efficiently moving, transforming, and processing the data. For instance, imagine you own a data analytics startup that provides organizations with real-time data analytics. Your client owns an eBike hire startup that wants to integrate weather data into its existing data to inform its marketing messaging strategies. Undoubtedly, your data engineers will set up many different data pipelines, pulling data from many sources. The two we are concerned with are designed to extract data from your client’s CRM and the local weather station. The Layers of the Modern Data Stack A data stack is a collection of different technologies, logically stacked on top of each other, that provides end-to-end data processing capabilities. As described here, a data stack can have many different layers. The modern data stack consists of ingestion (data pipelines), storage (OLAP database), and business intelligence (data analytics) layers. Lastly, there is an orchestration layer—for example, Kubernetes or Docker Compose—that sits on top of these layers, orchestrating everything together so that the stack does what it is supposed to do; ergo, ingest raw data and egest advanced predictive analytics. Data Pipelines are typically structured either as Extract, Transform, Load (ETL) or Extract, Load Transform (ELT) pipelines. However, developing, monitoring, and managing these data pipelines, especially in international enterprises with many branches worldwide, can be highly challenging, especially if the end-to-end data pipeline monitoring process is manual. Developing Data Pipelines, the Traditional Way Data pipelines are broadly categorized into two groups: code-based and deployed in a Direct Acyclic Graph or DAG-like tools such as Airflow and Apache Fink or non-code-based, which are  often developed in a SaaS-based application via a drag-and-drop UI. Both have their own problems. Code-based data pipelines are challenging to scale on demand. The development/test/release cycle is typically too long, especially in critical environments with thousands of pipelines, loading petabytes of data into data stores daily. Additionally, they are typically manually implemented, deployed, and monitored by human data engineers, allowing the risk of errors creeping in and significant downtime. With non-code-based data pipelines engineers don’t have access to the backend code, there can be limited visibility and monitoring, resulting in delays in detecting and addressing issues and failures. Moreover, it is challenging to reproduce errors, as making an exact copy of the pipeline is impossible. Lastly, version control using a tool like Git is complex. It is harder to control the evolution of a pipeline as the code is not stored in a repository such as a GitHub repo. The Solution: Improving Your Data Pipelines with Declarative, Reproducible, Modular Engineering The good news is that there is a sustainable solution to these challenges. It is important to remind ourselves of the difference between imperative and declarative programming. In imperative programming, you control how things happen. In declarative programming, you express the logic without specifying the control flow. The answer to the end-to-end data pipeline development, deployment, monitoring, and maintenance challenges is to utilize the declarative programming paradigm, that is, abstracting the computation logic. In practice, this is best achieved by using Terraform by HashiCorp — an open-source project that is an infrastructure-as-code development tool. Because modern world data-intensive application architecture is very similar to the data stack the applications run on, Terraform provides the framework that will make your data stack declarative, reproducible, and modular. Let’s return to our data analytics startup example described above to elucidate these concepts further. Imagine your data analytics app is a web application that executes inside Terraform. As expected, the most significant part of the application is the ClickHouse database. We must build two data pipelines to ETL data from the CRM system—stored in a PostgreSQL database—into the ClickHouse database, transforming and enriching the data before loading it into ClickHouse. The second pipeline extracts real-time weather data from the local weather service—through an API and transforms it—ensuring there are no errors and enriching it—before loading it into ClickHouse. Using Terraform to Code the Data Pipelines Classic infrastructure for modern cloud-based applications looks like: This is an oversimplification, but a more or less solid way of building applications. You have a public subnet with a user facing API / UI, which is connected to a private subnet where data is stored. To simplify things, let’s look at how to use Terraform to build the first data pipeline—to ETL data from your RDBMS (in this case PostgreSQL) system into the ClickHouse database. This diagram visualizes the data pipeline that ELTs the data from the RDBMS database into the ClickHouse database, and then exposed as a connection into Visualization service. This workflow must be reproducible as your company’s information technology architecture has at least three environments: dev, QA, and prod. Enter Terraform. Let’s look at how to implement this data pipeline in Terraform: It is best practice to develop modular code, distilled down to the smallest unit, which can be reused many times. The first step is to start with a , containing nothing more than the provider’s definition. main.tf file provider "doublecloud" {
  endpoint       = "api.double.cloud:443"
  authorized_key = file(var.dc-token)
}
provider "aws" {
  profile = var.profile
} The second step is to create a BYOA netwok: data "aws_caller_identity" "self" {}

data "aws_region" "self" {}

# Prepare BYOC VPC and IAM Role
module "doublecloud_byoc" {
  source  = "doublecloud/doublecloud-byoc/aws"
  version = "1.0.2"
  providers = {
    aws = aws
  }
  ipv4_cidr = var.vpc_cidr_block
}

# Create VPC to peer with
resource "aws_vpc" "peered" {
  cidr_block                       = var.dwh_ipv4_cidr
  provider                         = aws
}

# Get account ID to peer with
data "aws_caller_identity" "peered" {
  provider = aws
}

# Create DoubleCloud BYOC Network
resource "doublecloud_network" "aws" {
  project_id = var.dc_project_id
  name       = "alpha-network"
  region_id  = module.doublecloud_byoc.region_id
  cloud_type = "aws"
  aws = {
    vpc_id          = module.doublecloud_byoc.vpc_id
    account_id      = module.doublecloud_byoc.account_id
    iam_role_arn    = module.doublecloud_byoc.iam_role_arn
    private_subnets = true
  }
}

# Create VPC Peering from DoubleCloud Network to AWS VPC
resource "doublecloud_network_connection" "example" {
  network_id = doublecloud_network.aws.id
  aws = {
    peering = {
      vpc_id          = aws_vpc.peered.id
      account_id      = data.aws_caller_identity.peered.account_id
      region_id       = var.aws_region
      ipv4_cidr_block = aws_vpc.peered.cidr_block
      ipv6_cidr_block = aws_vpc.peered.ipv6_cidr_block
    }
  }
}

# Accept Peering Request on AWS side
resource "aws_vpc_peering_connection_accepter" "own" {
  provider                  = aws
  vpc_peering_connection_id = time_sleep.avoid_aws_race.triggers["peering_connection_id"]
  auto_accept               = true
}

# Confirm Peering creation
resource "doublecloud_network_connection_accepter" "accept" {
  id = doublecloud_network_connection.example.id

  depends_on = [
    aws_vpc_peering_connection_accepter.own,
  ]
}

# Create ipv4 routes to DoubleCloud Network
resource "aws_route" "ipv4" {
  provider                  = aws
  route_table_id            = aws_vpc.peered.main_route_table_id
  destination_cidr_block    = doublecloud_network_connection.example.aws.peering.managed_ipv4_cidr_block
  vpc_peering_connection_id = time_sleep.avoid_aws_race.triggers["peering_connection_id"]
}

# Sleep to avoid AWS InvalidVpcPeeringConnectionID.NotFound error
resource "time_sleep" "avoid_aws_race" {
  create_duration = "30s"

  triggers = {
    peering_connection_id = doublecloud_network_connection.example.aws.peering.peering_connection_id
  }
} This is preparing our Stage for later clusters. Architecture after terraform apply looks follows: Once we prepare a stage, we can create a cluster in this private-subnet: resource "doublecloud_clickhouse_cluster" "alpha-clickhouse" {
  project_id = var.dc_project_id
  name       = "alpha-clickhouse"
  region_id  = var.aws_region
  cloud_type = "aws"
  network_id = doublecloud_network.aws.id

  resources {
    clickhouse {
      resource_preset_id = "s1-c2-m4"
      disk_size          = 34359738368
      replica_count      = 1
    }
  }

  config {
    log_level       = "LOG_LEVEL_TRACE"
    max_connections = 120
  }

  access {
    ipv4_cidr_blocks = [
      {
        value       = doublecloud_network.aws.ipv4_cidr_block
        description = "DC Network interconnection"
      },
      {
        value       = aws_vpc.tutorial_vpc.cidr_block
        description = "Peered VPC"
      },
      {
        value       = "${var.my_ip}/32"
        description = "My IP"
      }
    ]
    ipv6_cidr_blocks = [
      {
        value       = "${var.my_ipv6}/128"
        description = "My IPv6"
      }
    ]
  }
} This is added to our Stage for simple Clickhouse Cluster. But this cluster still empty, so we must enable transfer between PostgreSQL and ClickHouse: resource "doublecloud_transfer_endpoint" "pg-source" {
  name       = "chinook-pg-source"
  project_id = var.dc_project_id
  settings {
    postgres_source {
      connection {
        on_premise {
          tls_mode {
            ca_certificate = file("global-bundle.pem")
          }
          hosts = [
            aws_db_instance.tutorial_database.address
          ]
          port = 5432
        }
      }
      database = aws_db_instance.tutorial_database.db_name
      user     = aws_db_instance.tutorial_database.username
      password = var.db_password
    }
  }
}

data "doublecloud_clickhouse" "dwh" {
  name       = doublecloud_clickhouse_cluster.alpha-clickhouse.name
  project_id = var.dc_project_id
}

resource "doublecloud_transfer_endpoint" "dwh-target" {
  name       = "alpha-clickhouse-target"
  project_id = var.dc_project_id
  settings {
    clickhouse_target {
      connection {
        address {
          cluster_id = doublecloud_clickhouse_cluster.alpha-clickhouse.id
        }
        database = "default"
        user     = data.doublecloud_clickhouse.dwh.connection_info.user
        password = data.doublecloud_clickhouse.dwh.connection_info.password
      }
    }
  }
}

resource "doublecloud_transfer" "pg2ch" {
  name       = "postgres-to-clickhouse-snapshot"
  project_id = var.dc_project_id
  source     = doublecloud_transfer_endpoint.pg-source.id
  target     = doublecloud_transfer_endpoint.dwh-target.id
  type       = "SNAPSHOT_ONLY"
  activated  = false
} This is adding a ELT snapshot process between PostgreSQL and ClickHouse. As a last piece we can add a connection to this newly created cluster to visualization service: resource "doublecloud_workbook" "dwh-viewer" {
  project_id = var.dc_project_id
  title      = "DWH Viewer"

  config = jsonencode({
    "datasets" : [],
    "charts" : [],
    "dashboards" : []
  })

  connect {
    name = "main"
    config = jsonencode({
      kind          = "clickhouse"
      cache_ttl_sec = 600
      host          = data.doublecloud_clickhouse.dwh.connection_info.host
      port          = 8443
      username      = data.doublecloud_clickhouse.dwh.connection_info.user
      secure        = true
      raw_sql_level = "off"
    })
    secret = data.doublecloud_clickhouse.dwh.connection_info.password
  }
} As you can see, most of the code consists of variables. Therefore, executing this code to set up the different environments is easy by adding a file and running with it: stage_name.tfvars terraform apply See the for more information on .tvars files. Terraform documentation // This variable is to set the
// AWS region that everything will be
// created in
variable "aws_region" {
  default = "eu-west-2" // london
}

// This variable is to set the
// CIDR block for the VPC
variable "vpc_cidr_block" {
  description = "CIDR block for VPC"
  type        = string
  default     = "10.0.0.0/16"
}

// This variable holds the
// number of public and private subnets
variable "subnet_count" {
  description = "Number of subnets"
  type        = map(number)
  default = {
    public  = 1,
    private = 2
  }
}

// This variable contains the configuration
// settings for the EC2 and RDS instances
variable "settings" {
  description = "Configuration settings"
  type        = map(any)
  default = {
    "database" = {
      allocated_storage   = 10            // storage in gigabytes
      engine              = "postgres"    // engine type
      engine_version      = "15.4"        // engine version
      instance_class      = "db.t3.micro" // rds instance type
      db_name             = "chinook"     // database name
      identifier           = "chinook"     // database identifier
      skip_final_snapshot = true
    },
    "web_app" = {
      count         = 1          // the number of EC2 instances
      instance_type = "t3.micro" // the EC2 instance
    }
  }
}

// This variable contains the CIDR blocks for
// the public subnet. I have only included 4
// for this tutorial, but if you need more you
// would add them here
variable "public_subnet_cidr_blocks" {
  description = "Available CIDR blocks for public subnets"
  type        = list(string)
  default = [
    "10.0.1.0/24",
    "10.0.2.0/24",
    "10.0.3.0/24",
    "10.0.4.0/24"
  ]
}

// This variable contains the CIDR blocks for
// the public subnet. I have only included 4
// for this tutorial, but if you need more you
// would add them here
variable "private_subnet_cidr_blocks" {
  description = "Available CIDR blocks for private subnets"
  type        = list(string)
  default = [
    "10.0.101.0/24",
    "10.0.102.0/24",
    "10.0.103.0/24",
    "10.0.104.0/24",
  ]
}

// This variable contains your IP address. This
// is used when setting up the SSH rule on the
// web security group
variable "my_ip" {
  description = "Your IP address"
  type        = string
  sensitive   = true
}
// This variable contains your IP address. This
// is used when setting up the SSH rule on the
// web security group
variable "my_ipv6" {
  description = "Your IPv6 address"
  type        = string
  sensitive   = true
}

// This variable contains the database master user
// We will be storing this in a secrets file
variable "db_username" {
  description = "Database master user"
  type        = string
  sensitive   = true
}

// This variable contains the database master password
// We will be storing this in a secrets file
variable "db_password" {
  description = "Database master user password"
  type        = string
  sensitive   = true
}

// Stage 2 Variables

variable "dwh_ipv4_cidr" {
  type        = string
  description = "CIDR of a used vpc"
  default     = "172.16.0.0/16"
}
variable "dc_project_id" {
  type        = string
  description = "ID of the DoubleCloud project in which to create resources"
} In Conclusion… There you have it. A small modern data stack that ETLs data from a PostgreSQL database into a ClickHouse database. The best news is that you can set up multiple data pipelines—as many as you need—using the code in this example. There is also a complete example that you can find , with additional features like adding a peer connection to your existing VPC and creating a sample ClickHouse database with a replication transfer between the two databases. here Have fun playing with your small modern data stack! This article was also . published on DZone

Declarative Engineering: Using Terraform to Code Your Data Pipelines

Introduction Databricks is emerging as one of the main players in the MLOps and DevOps world. In the last month, I experienced this as part of my project at Areto and decided to write a series of hands-on blog posts on how to implement a MLOps Pipeline with Databricks. This is an ongoing learning process for me as well. I write each blog post after I experiment with different features and tools. So, I like to see this blog series as collective learning. Any comments and suggestions are welcome. In this blog post, we'll explore how to build a pipeline using the Databricks DBX tool and GitLab. This guide is designed for data engineers and scientists who are looking to streamline their data processing workflows. We'll cover key concepts, practical applications, and step-by-step instructions to help you integrate these tools effectively into your data projects. Continuous Integration (CI) CI/CD (Continuous Integration/Continuous Delivery): Continuous Integration and Continuous Delivery/Deployment (CI/CD) have become fundamental in modern software development practices. At its core, CI/CD is a method that emphasizes short, frequent updates to software through automated pipelines. While traditionally associated with software development, its principles are increasingly being adopted in data engineering and data science. The essence of CI/CD lies in its ability to automate the stages of software development, particularly in building, testing, and deploying code.  Continuous Integration begins with regular code commits to a shared repository branch, ensuring collaborative development without version conflicts. Each commit undergoes an automated build and test process, validating changes to enhance the quality and reliability of the final product. Databricks: Databricks has emerged as a premier cloud-based platform, uniting the realms of data engineering, machine learning, and analytics. Databricks excels in handling large-scale data processing and complex analytical tasks, but to leverage its full potential, teams need to navigate its multifaceted environment effectively. Integrating CI/CD processes within Databricks environments can streamline workflows, ensuring consistent, reproducible, and scalable data operations. This integration is crucial for teams aiming to maintain agility and efficiency in their data projects, enabling them to deliver reliable, high-quality data solutions consistently. Databricks Workflows: Databricks workflows are particularly significant in the context of CI/CD, as they offer a robust platform for automating and orchestrating data pipelines crucial to continuous integration and deployment. These workflows allow for the seamless scheduling and execution of tasks, which is essential for maintaining the frequent, automated update cycles characteristic of CI/CD. In a CI/CD pipeline, Databricks workflows can be used to automatically process and test large datasets, ensuring that data transformations and analyses are consistently accurate and up-to-date. The diversity of tasks that can be executed within Databricks workflows, such as Notebooks for exploratory data analysis, Python scripts for structured data processing, and Python wheels for custom package deployment, aligns well with the varied needs of CI/CD pipelines. By integrating these tasks into CI/CD workflows, data teams can ensure that every aspect of their data processing and analysis is continuously tested and integrated into the larger data strategy. This integration is key for developing resilient, scalable, and efficient data operations, enabling teams to deliver high-quality, reliable data products rapidly. Databricks CLI Extension (DBX): The Databricks CLI Extension, or DBX, is a pivotal tool in integrating Databricks with CI/CD pipelines, enhancing the automation and management of data workflows. The ability to programmatically control and manipulate data processes is crucial to implementing a CI/CD workflow, and DBX fills this role effectively. It provides a command-line interface for interacting with various Databricks components, such as workspaces, workflows, and clusters, facilitating seamless integration into automated pipelines. Hands-on In our hands-on example, we will create a (very) minimal project that uses dbx to deploy and run a Databricks workflow to manipulate, analyze, and test some data. This practical exercise showcases how Databricks can be used for real-world data analysis tasks. Our CI pipeline will play a crucial role in this process, as it will automate the deployment of our code as a Databricks workflow and validate its output. This validation step is vital to ensure the accuracy and reliability of our analysis. By the end of this exercise, you'll have a clear understanding of how to set up and run a Databricks workflow within a CI pipeline and how such a setup can be beneficial in analyzing and deriving insights from large datasets. This example will not only demonstrate the technical application of Databricks and CI principles but also offer a glimpse into the practical benefits of automated data analysis in a business context. Development pattern using Databricks GUI There are various development patterns to implement our CI pipeline. The common development workflows with dbx are to develop, test, and debug your code on your local environments and then use dbx to batch-run the local code on a target cluster. By integrating a remote repository with Databricks, we use a CI/CD platform such as , , or to automate running our remote repo’s code on our clusters. GitHub Actions Azure DevOps GitLab In this tutorial, for the sake of simplicity, we use Databricks GUI to develop and test our code. We follow these steps: Create a remote repository and clone it into our Databricks workspace. We use here. Gitlab Develop the program logic and test it inside the Databricks GUI. This includes Python scripts to build a Python Wheel package, scripts to test the data quality using Pytest, and a notebook to run the Pytest. Push the code to GitLab. The git push will trigger a Gitlab Runner to build, deploy and launch our workflows on Databricks using dbx Git integration As the first step, we . Next, we create a remote repository and . To allow our Gitlab runner to communicate with Databricks API through dbx, we should add two environment variables, and to our CI/CD pipeline configurations. configure Git credentials & connect a remote repo to Databricks clone it to our Databricks repo DATABRICKS_HOST DATABRICKS_TOKEN to generate a Databricks token, in your Databricks, go to User Settings →  Developer → Access tokens → manage → Generate new token. The Databricks host is the URL when you login into your Databricks workspace. It would be something like . The last part of the URL is your workspace id, and you should ignore that. https://dbc-dc87ke16-h5h2.cloud.databricks.com/ Finally we add the token and host to our Gitlab CI/CD setting. For that open your repo in Gitlab, go to Settings→ CI/CD → Variables → Add variables The project skeleton the project is structured into several folders and key files, each serving a specific purpose: .dbx folder: : defining the configuration of your DBX project. It contains environment settings, dependencies, and other project-specific parameters. project.json conf folder: : outlining the deployment configurations and environment settings. It defines workflows and their respective parameters and cluster configurations. deployment.yml : our wheel package. This folder includes: my_package folder A subfolder containing the main ETL task script . The ETL task load our dataset and create two new tables. tasks sample_etl_job.py The file. it includes common utilities that provide access to components such as common.py SparkSession Contains two Jupyter notebooks: notebooks folder: : plot the distribution of different features in our dataset explorative_analysis : used to execute for unit testing run_unit_test pytest This folder is dedicated to testing: tests folder: : includes pytest fixtures conftest.py : contains unit tests to validate the data structure in our tables. test_data.py In addition to these folders, there are two important files in the root directory: : the configuration file for GitLab's Continuous Integration (CI) service, defining the instructions and commands the CI pipeline should execute. .gitlab-ci.yml : for building my Python wheel package. It defines the package's metadata, dependencies, and build instructions. setup.py dbx-tutorial/
├─ .dbx/
│  ├─ project.json
├─ conf/
│  ├─ deployment.yml
├─ my_package/
│  ├─ tasks/
│  │  ├─ __init__.py
│  │  ├─ sample_etl_job.py
│  ├─ __init__.py
│  ├─ common.py
├─ tests/
│  ├─ conftest.py
│  ├─ test_data.py
├─ notebooks/
│  ├─ explorative_analysis
│  ├─ run_unit_test
├─ .gitlab-ci.yml
├─ setup.py Deployment configuration file is crucial for defining your DBX project's configuration. This file could be generated automatically when you run . Check the dbx website for more details about the . project.json dbx --init Profile file reference The “ ” option is used for local development. If you run the command inside the CI tool, you need to specify the and environment variables, and they will the profile variable. Since we don’t want to use the local environment and develop and run our code on Databricks UI, we don’t need to specify this option here. Instead, the dbx will pick them up from the CI/CD setting that we set earlier. profile dbx databricks host databricks token overwrite Your files that dbx will automatically upload to the system will be stored in They will be stored in .  You can read more about this in ml experiment. artifact_location dbx documentation {
    "environments": { 
        "default": {
            "profile": "dbx-tutorial",
            "storage_type": "mlflow",
            "properties": {
                "workspace_directory": "/Shared/dbx/dbx-tutorial", 
                "artifact_location": "dbfs:/Shared/dbx/projects/dbx-tutorial"
            }
        }
    },
    "inplace_jinja_support": false,
    "failsafe_cluster_reuse_with_assets": false,
    "context_based_upload_for_execute": false
} outlines the deployment configurations and environment settings. Here, we use and format to define the workflows. Each workflow defines an object inside the Databricks workspace: or deployment.yml 2.1 API wheel_task job delta live tables Pipeline. custom:
  existing_cluster_id: &existing_cluster_id
    existing_cluster_id: "1064-xxxxxx-xxxxxxxxx"

environments:
  default: #this should be same envoriment name as you defined in the project.json
    workflows:
      - name: "etl_job"
        tasks:
          - task_key: "main"
            <<: *existing_cluster_id
            python_wheel_task:
              package_name: "my_package"
              entry_point: "etl_job" # take a look at the setup.py entry_points section for details on how to define an entrypoint
          - task_key: "eda"
            <<: *existing_cluster_id
            notebook_task:
              notebook_path: "/Repos/<your DB username>/dbx-tutorial/notebooks/explorative_analysis"
              source: WORKSPACE
            depends_on:
              - task_key: "main"    
      - name: "test_job"
        tasks:
          - task_key: "main"
            <<: *existing_cluster_id
            notebook_task:
              notebook_path: "/Repos/<your DB username>/dbx-tutorial/notebooks/run_unit_test"
              source: WORKSPACE
            libraries:
            - pypi:
                package: pytest Here, we define two workflows: : consists of two tasks/jobs. "etl_job" The first task ( ) is of type . It used to run ETL task using the entry point that we define in the file. main python_wheel_task setup.py The second task ( ) is of type . It is used to run the notebook after the main tasks run successfully. Notice the use of property. etl notebook_task explorative_analysis depends_on : consists of only one job of type . It is used to run the notebook that is "test_job" notebook_task responsible for running the pytest. : in the above example, we run the workflow on an existing cluster. You can also create and run a new cluster for every deployment and launch of your work. For that, you should change the code as follows: Note custom:
  basic-cluster-props: &basic-cluster-props
    spark_version: "11.3.x-cpu-ml-scala2.12"

  basic-static-cluster: &basic-static-cluster
    new_cluster:
      <<: *basic-cluster-props
      num_workers: 1
      node_type_id: "Standard_E8_v3"

environments:
  default: 
    workflows:
      - name: "et_job"
        tasks:
          - task_key: "main"
            <<: *basic-static-cluster
						.... CI pipeline configuration Your GitLab CI pipeline is structured to automate the testing and deployment processes of your Databricks project. It consists of two main stages: and . In the stage, the runs the unit tests and deploys a separate workflow for testing. The stage, activated upon successful completion of the test stage, handles the deployment of your main ETL workflow. In general, the pipeline follows these steps: test deployment test unit-test-job deploy Build the project Push the build artifacts to the Databricks workspace Install the wheel package on your cluster Create the jobs on Databricks Workflows Run the jobs image: python:3.9

stages:          # List of stages for jobs, and their order of execution
  - test
  - deploy

unit-test-job:   # This job runs in the test stage.
  stage: test    # It only starts when the job in the build stage completes successfully.
  script:
    - echo "Running unit tests... This will take about 60 seconds."
    - echo "Code coverage is 90%"
    - pip install -e ".[local]"
    - dbx deploy --deployment-file conf/deployment.yml test_job --assets-only
    - dbx launch test_job --from-assets --trace

deploy-job:      # This job runs in the deploy stage.
  stage: deploy  # It only runs when *both* jobs in the test stage complete successfully.
  script:
    - echo "Deploying application..."
    - echo "Install dependencies"
    - pip install -e ".[local]"
    - echo "Deploying Job"
    - dbx deploy --deployment-file conf/deployment.yml etl_job
    - dbx launch etl_job --trace
    - echo "Application successfully deployed."
    - echo "remove all workflows." For workflow deployment and lunch, we use two commands: : to deploy the workflow definitions to the given environment. dbx deploy : to launch the given workload by its name in a given environment. dbx lunch In the we use the flag to avoid creating a job definition on our Databricks workflows during deployment. For the lunch command, we should use the flag. If this flag is provided, the launch command will search for the latest deployment that was assets-only. Lunching this, we can not see anything in the Workflows UI, but it will be a non-titled job  running by submit API. You can see this on the Job Runs page **** of your Databricks workspace. Check out the dbx documentation to read about and unit-test-job --assets-only --from-assets assest-based workflows other commands If you check the “Experiments” page of your Databricks workspace, you will find the build artifacts of each run of your CI pipeline. Databricks Asset Bundles currently, Databricks recommends using Asset Bundles for CI/CD and offers a migration guide from dbx to bundles.  If you understand the concepts in this post, it would be easy to switch to bundles. In the next post, I am going to explain how we can convert this project to an asset bundle project. Resources for this tutorial. Make sure you update the file with the correct and Repository conf/deployment.yml cluster_id DB username dbx documentation Databricks Best Practices: Development loop and CI/CD on Databricks with dbx Databricks Jobs API CI/CD techniques with Git and Databricks Repos What is CI/CD on Azure Databricks?

Building a CI Pipeline with Databricks dbx Tool and GitLab

The pipeline design pattern in is a valuable tool for software engineers looking to optimize data processing. By breaking down a complex process into multiple stages, and then executing those stages in parallel, engineers can dramatically reduce the processing time required. This also simplifies complex operations and enables engineers to build scalable data processing pipelines. C# design pattern In this article, I’ll provide a detailed overview of how to implement the pipeline design pattern in C#. I’ll share the fundamental concepts behind the pattern, provide example code to illustrate implementation and provide tips for optimizing its performance. We’ll also highlight some common pitfalls and share guides on how to avoid them. Finally, we’ll discuss real-world scenarios where this pattern can be applied and provide specific use cases to illustrate. Understanding the Pipeline Design Pattern The pipeline design pattern is commonly used in software engineering for efficient data processing. This design pattern utilizes a series of stages to process data, with each stage passing its output to the next stage as input. The pipeline structure is made up of three components: The source: Where the data enters the pipeline The stages: Each stage is responsible for processing the data in a particular way The sink: Where the final output goes Implementing the pipeline design pattern offers several benefits, with one of the most significant benefits the efficiency of processing large amounts of data. By breaking down the data processing into smaller stages, the pipeline can handle larger datasets. The pattern also allows for easy scalability, making it easy to add additional stages as needed. The pipeline design pattern offers us a flexible and efficient way to process large datasets. With its straightforward structure made up of three components, you can easily create pipelines that meet your specific needs and scale as their data processing requirements grow. Follow along with this video on the pipeline design pattern in C#: https://www.youtube.com/watch?v=Z4qUWxTH0Mw&embedable=true Implementing the Pipeline Design Pattern in C# To implement the pipeline design pattern in C#, there are specific steps that you’ll need to follow. First, you must define each stage of the pipeline. After creating the stages, you’ll need to chain them together in the correct order, connecting the output of each stage to the input of the next. Finally, you’re going to need to define a sink component to receive the output after the final stage has processed the data. Creating Pipeline Stages To create each stage of the pipeline, you can utilize the C# delegate method. Of course, we can get more specific with creating dedicated APIs through interfaces for the pipeline stages… But using a delegate is quick and easy. First, you’ll define the delegate’s input and output types. Next, you’ll need to code the stage to handle the input data and process it as required. The stage’s output data type must match the next step’s input data type in the pipeline. Here is an example of how to define a delegate for pipeline stages: delegate OutputType MyPipelineStage(InputType input); Chaining Pipeline Stages To execute the pipeline stages in sequence, you’re going to need to chain each stage to the next. To do this, we define the input delegate for each stage to receive the output of the previous stage. Here is an example of how to chain two pipeline stages together: MyPipelineStage firstStage = (InputType input) =>
{
   // process input and return OutputType
};

MyPipelineStage secondStage = (InputType input) =>
{
   var outputFromFirst = firstStage(input);
   // process outputFromFirst and return OutputType
}; We can create a pipeline with multiple stages by repeating the process of defining each stage and chaining them together. The final step is to send the output of the last stage to the sink component, which will look the same. The difference is we don’t continue passing data beyond that point. Example of the Pipeline Design Pattern in C# We’re going to look at an example of the Pipeline Design Pattern in C# that tackles text analysis! I find it’s often best to understand concepts when we can apply them to a practical situation. In this case, we’ll require several stages in a pipeline that can work together: Sanitize the text Some type of frequency analysis Summarization of the results With these roughly as the stages of the pipeline, let’s see how we can get started! Example C# Pipeline Overview Let’s start by defining delegates for each stage of the pipeline. We could declare a specific interface that the pipeline needs to implement, but we’re going to simplify this example by keeping things lightweight and flexible: public delegate string TextCleaner(string input);
public delegate Dictionary<string, int> WordCounter(string input);
public delegate string TextSummarizer(Dictionary<string, int> wordFrequency); Next, we’d have code for each stage. I’ll go into more detail on this in the next section, but for now, we can mark these as follows: TextCleaner cleaner = text =>
{
    /* normalization logic */
    return cleanedText;
};

WordCounter counter = cleanedText =>
{
    /* validation logic */
    return wordFrequency; 
};

TextSummarizer summarizer = wordFrequency =>
{
    /* transformation logic */
    return summary;
}; Next, we need to chain the stages from one to the next. Again, given that this is a simple example, we’ll manually set these stages up to be configured how we need them. Consider though that you could write code that automatically wires these up! Here’s the manual approach: var inputText = "Your input text here";
var cleanedText = cleaner(inputText);
var wordFrequency = counter(cleanedText);
var summary = summarizer(wordFrequency); Each stage is a specific task in text processing. The text cleaner removes unnecessary characters, the word counter creates a frequency map of words, and the summarizer generates a summary based on the most frequent words. The pipeline processes the text through each stage in sequence, demonstrating how different tasks can be modularly connected in a pipeline. Implementations of Pipeline Stages The following are just for demonstration purposes, but here are some implementations that you could consider for the pipeline stages that I listed above: TextCleaner cleaner = text =>
{
    // Example: Remove punctuation and convert to lower case
    var cleanedText = new string(text.Where(c => !char.IsPunctuation(c)).ToArray());
    return cleanedText.ToLower();
}; This stage modifies the text to our liking and returns it as the result of the stage. Next, we’ll look at the counting stage: WordCounter counter = cleanedText =>
{
    var wordFrequency = new Dictionary<string, int>();
    var words = cleanedText.Split(' ');

    foreach (var word in words)
    {
        if (string.IsNullOrWhiteSpace(word))
        {
            continue;
        }

        if (wordFrequency.ContainsKey(word))
        {
            wordFrequency[word]++;
        }
        else
        {
            wordFrequency[word] = 1;
        }
    }

    return wordFrequency;
}; This stage keeps a count of the different words that we have in the text. TextSummarizer summarizer = wordFrequency =>
{
    // Example: Summarize by picking top 3 frequent words
    var topWords = wordFrequency
        .OrderByDescending(kvp => kvp.Value)
        .Take(3)
        .Select(kvp => kvp.Key);
    return $"Top words: {string.Join(", ", topWords)}";
}; Finally, the summary step gets a string together with information about the input data based on the pipeline stages. With these implementations, we now have something that can process some text input with a pipeline! Optimizing the Pipeline Design Pattern in C# To make the pipeline design pattern as efficient as possible, you can make some optimizations in your approach. This section will cover techniques for optimizing the pipeline design pattern in C#. Optimizing Pipeline Performance One way to optimize pipeline performance is through parallelism. Developers can execute multiple stages of the pipeline concurrently. They can implement this technique using the . TPL provides a simple API for executing on multiple processors. Task Parallel Library (TPL) in C# tasks concurrently Another way to optimize pipeline performance is through batch processing. The idea is to batch the input data and apply the pipeline stages to the batches. This technique ensures that the pipeline uses all available resources, which increases efficiency. Developers can control the batch size to find the optimal balance between resource usage and performance. If you’re considering running things in parallel, check out this video for some interesting results on Task.WhenAll and Parallel.ForEachAsync: https://www.youtube.com/watch?v=GE-phyI6lEM&embedable=true Avoiding Common Pitfalls While implementing the pipeline design pattern, you may come across some common pitfalls that hurt pipeline performance. One pitfall is not designing stages that can execute concurrently. It is important to design each stage to be independent from other stages to ensure parallel execution. Another pitfall is not considering the effect of a stage on other stages’ performance. For example, if a stage introduces a lot of processing overhead, it could slow down the pipeline’s overall throughput. To avoid this, you must create stages that are as efficient as possible. Even if you have concurrent steps, this one may become the bottleneck! Optimizing the pipeline design pattern involves leveraging parallelism, batch processing, and designing stages that balance performance with efficiency. By avoiding common pitfalls, you can improve the overall effectiveness of the pipeline. You can use tools like BenchmarkDotNet to measure the performance of your code too! Check out this video to see BenchmarkDotNet in action: https://www.youtube.com/watch?v=Wa3sdKGp3wE&embedable=true Real-world scenarios and Use Cases Pipeline design pattern has numerous real-world applications where it can significantly improve the efficiency of data processing. Here are some examples of scenarios where the pipeline design pattern can be applied: : In this scenario, data scientists and analysts can use a pipeline to process large datasets efficiently. The pipeline can help preprocess data, filter outliers, scale datasets, standardize feature representations, train models, or make predictions. Data Analytics Pipeline : Image processing involves multiple steps such as resizing, cropping, filtering, and enhancing. The pipeline can be tailored to the specific image processing workflow and optimized it for running on large datasets. Image Processing Pipeline : Automated software testing pipelines provide early bug detection and reduce the risk of regressions during software development. The pipeline can be divided into multiple stages, each responsible for running a specific type of test, such as unit or integration tests. Automated Testing Pipeline : Real-time data streaming is becoming increasingly popular in various industries, including finance, healthcare, and telecommunications. The pipeline can help to preprocess and filter incoming data in real-time, enabling quick decision-making. Data Streaming Pipeline : Video processing involves multiple tasks such as decoding, encoding, resizing, and filtering. The pipeline can assist in optimizing these processes, making it easier to handle large amounts of video files. Video Processing Pipeline Wrapping Up Implementing the pipeline design pattern in C# can greatly improve the efficiency of data processing in software development. By breaking down complex tasks into smaller stages, pipelines can reduce latency and improve overall performance. Some of the key takeaways from this article include understanding the structure of a pipeline, creating pipeline stages, and optimizing pipeline performance. It is important to avoid common pitfalls when implementing the pipeline design pattern and monitor and resolve bottlenecks as they arise — And make sure you benchmark your code before prematurely optimizing! By implementing the pipeline design pattern, software engineers and developers can achieve faster and more efficient data processing. If you’re interested in more learning opportunities, and ! subscribe to my free weekly newsletter check out my YouTube channel Also published here.

The Pipeline Design Pattern - Examples in C#

Seamless Implementation of Safety Practices in the DevOps Pipeline

In the modern fast-paced digital landscape, there is a high demand for streamlined and efficient data management tools and services. With the exponential growth of data-driven development and decision-making in organizations, there is a need for robust solutions to optimize data pipelines. Jupyter Notebooks have emerged as one of the most popular choices for organizations to use in data exploration and analysis due to their interactive and user-friendly interface. However, as the scale and complexity of data operations grow, the requirement to easily migrate Jupyter Notebooks into production environments becomes more essential. Here comes the (VDK), a framework that simplifies data ingestion and data processing. A toolset enabling you to run data jobs & comprehensive solutions for the productionization of Jupyter Notebooks. With its powerful features and capabilities, VDK serves as a game-changing tool that enables organizations to easily integrate Jupyter Notebooks into complex data pipelines while ensuring scalability, reproducibility, and enhanced workflow efficiency. Versatile Data Kit In this blog, we see all about , exploring the potential of this open-source toolkit to revolutionize the way organizations manage and process their data. Productionizing Jupyter Notebooks with VDK Understanding the Challenges with Jupyter Notebooks Despite their popularity in data analysis and exploration, Jupyter Notebooks have several inherent limitations when used in production environments. Sometimes, these limitations of the Jupyter Notebook pose significant hurdles during seamless integration into complex data pipelines. Understanding these challenges is important for companies aiming to use Jupyter Notebooks within their production workflows. Some of the biggest challenges include: Jupyter Notebooks sometimes struggle to handle large-scale data processing efficiently. When dealing with large datasets or sophisticated computations, the lack of optimised memory management and the sequential nature of execution might cause performance bottlenecks, impeding the seamless expansion of data pipelines. Scalability and Performance: Version control for Jupyter Notebooks in a collaborative environment can be difficult. Merging changes, managing revisions, and ensuring consistency across various versions of a notebook may become time-consuming, potentially resulting in version conflicts and data inconsistencies, especially when numerous team members are working at the same time. JSON file with lots of irrelevant information as you can see in the image below. Version control and Collaboration: During the deployment of Jupyter Notebooks in production, there is a significant challenge to ensure the reproducibility of results. Variations in the runtime environment, dependencies, and external libraries can all have an impact on the consistency of results, making it difficult to accurately replicate analyses or experiments, especially when switching across computing environments. Reproducibility and Environment Management: If proper security measures and access controls are not implemented in a production environment, Jupyter Notebooks can present security risks. Allowing unrestricted access to notebooks or exposing sensitive data might jeopardize data integrity and confidentiality, potentially leading to security breaches and unauthorized data tampering. Security and Access Control: Few more challenges exist, like irrelevant code, modularization and no proper method for testing due to lack of libraries and tools. Solving Problems with VDK VDK plays an important role in facilitating the seamless integration of Jupyter Notebooks into production pipelines. It is a robust and comprehensive framework designed to simplify the complex process of data ingestion, transformation, and deployment within production pipelines using Python and SQL. Its capabilities for version control, environment management, and scalable data processing enable organizations to overcome the inherent challenges associated with deploying Jupyter Notebooks in production environments. . Check the VDK GitHub repo to learn more We have discussed some of the challenges with Jupyter Notebook above; it’s time to see how we can solve those challenges with VDK. Non-Linear Execution and Hidden State Risks Notebooks support non-linear code execution, which might result in hidden dependencies when cells run out of order. It increases risks when moving to the production environment. Follow this image for understanding. To know about problems, try to focus on the first two objectives — Retrieve Data and Data Cleaning. To retrieve the data, We will first and then load it. After the execution, we can check the data. Now we will clean the data by removing some . import pandas testuser As you can see, while retrieving and cleaning the data, we are unaware of the exact operations that were executed; they are hidden. In the production environment, we aim to ensure clarity about the executed processes, thereby eliminating the need for hidden dependencies or states. Let's explore the VDK solution; we will employ the VDK cell tag (located at the top right in the image). Essentially, it will assign numbering to the tagged cell, indicating the order of execution as 1, 2, 3 when deployed. This approach provides assurance that there are no hidden dependencies or states, and we can only execute what is visible. Irrelevant Code Excessive irrelevant code, like unused statements or unrelated snippets, can be found in notebooks. During the experimental stages of development, a few snippets and useful algorithms that help in interactive changes can be problematic in production. Let’s see the VDK solution; we will do the data classification by assigning into predefined categories for clarity, as you can see in the image below. After executing code from cell 9, we get a new column that contains the types of users. scores To check if something unknown is added to this data, we have defined methods in file. After the visualization, we are sure that only Detractors and Promotors are present and classification is right. Now we have to ingest the organized data using vdk . helper.py job_input # sending data for ingestion
 job_input.send_tabular_data_for_ingestion(
 df.itertuples(index=False),
 destination_table="nps_data",
 column_names=df.columns.tolist()
 ) If we see our whole code, we can know for sure that and will not be executed in production. Some codes are relevant for development and irrelevant to deployment, so we can't just remove these always. VDK helps here to maintain performance without removing irrelevant code. df visualize_data(df) Testing When working with Jupyter Notebooks, there is a lack of proper tools and methods for testing notebooks. It becomes challenging to verify the code you've written, and resorting to alternative solutions is not practical, especially in larger teams. VDK addresses this issue by offering an end-to-end testing solution through the command, as illustrated in the image below. run After running this command, it will execute vdk cells of the above Jupyter Notebook like a production environment. If something fails, it will give the error message and the details of where it originated from. It will be easier for us to fix things. We can also use the testing feature from the create deployment method by checking the box. It will also behave the same, and in case of any failures, it will stop the deployment and show us the error and the exact trace. Run data job before deployment Version Control Version control with notebooks can often become a complex process due to their JSON-based format, which can contain excessive noise. VDK offers a solution to this predicament through two key features. See the version control after using VDK in the image below. Firstly, it implements Noise Reduction, a mechanism that cleanses the notebook’s JSON by eliminating non-essential elements, including execution counts and outputs, known to contribute to the “noise” in the data. This helps streamline the version control process by focusing solely on the relevant content. Secondly, VDK ensures Seamless Integration with Git, facilitating the seamless committing of code to Git during deployment in a cleaner state, free from unnecessary metadata. By simplifying version control and minimizing clutter, VDK enables a more efficient and streamlined workflow for managing notebook-based projects. If you want to understand what we are doing in a step-wise manner with Jupyter Notebook UI, check this . guide Conclusion To sum up, the application of VDK in the productionization of Jupyter Notebooks has numerous benefits. By using the Noise Reduction feature, VDK significantly simplifies the version control process by eliminating unnecessary components that could hinder effective data management. Furthermore, VDK’s seamless integration with Git ensures a tidier and more organized environment for code deployment and collaboration, reducing clutter and simplifying the workflow. Adopting VDK can greatly enhance data pipeline management, enabling users to streamline their operations and improve overall workflow efficiency. It is a valuable resource for those aiming to fully utilize their data-driven projects. Additional Resources 💡Check Versatile Data Kit GitHub Repo 💡Check Youtube Video Tutorial 💡Check the of VDK to learn more Getting Started guide 💡Check VDK in Jupyter Notebook UI guide 💡Go through VDK user guides Also published . here

How to Use Versatile Data Kit to Turn Your Jupyter Notebooks Into Scalable & Reliable Data Pipelines

Telecom's AI Revolution: How Predictive Analytics Optimizes CI/CD Pipelines

How are you, hacker?
🪐 has got you covered with fresh content from our trending stories of the day! Set email preference . Want to know what's trending right now?: The Techbeat by HackerNoon here A Guide to Building an AWS Serverless API and CICD Pipeline By [ 11 Min read ]
This article demonstrates the development of a Serverless API utilizing AWS services @pchandel09 Read More. Everything You Need to Know to Deploy MinIO in Virtualized Environments By [ 4 Min read ]
When deploying MinIO in virtualized environments, it’s important to make sure that the proper conditions are in place. @minio Read More. Enhancing DevOps Efficiency With ChatOps By [ 6 Min read ]
In this article, we will explore how ChatOps tools can be harnessed to supercharge your DevOps practices @vladislavbilay Read More. Bitcoin's Moment in the Sun is Coming By [ 4 Min read ]
Just 2 days ago, Bitcoin’s whitepaper turned 15. Much has happened since those very first days, however, Bitcoin’s promise of decentralisation still stands. @ulriklykke Read More. No Coding Required: 5 Mind-Blowing Uses of GPT-4 By [ 2 Min read ]
What can actually be done using GPT-4? @whatsai Read More. I Trained an AI to Act Like Me: Here's What Happened By [ 3 Min read ]
I trained my official AI by re-claiming my data from social platforms. It was my data in the first place! Now I have an AI version of me that can speak for me. @tomwom Read More. Solving the Hard Problems: A Conversation with Matan Almakis, the Head of Project at DOP By [ 6 Min read ]
A discussion with Matan Almakis, the Head of Project at Data Ownership Protocol. @danstein Read More. The Tech Company News Brief: It's a Bad Time to be Intel By [ 4 Min read ]
As if the underwhelming launch of the "next generation" processors wasn't bad enough, Intel now has to contend with competitors eating at its market share. @sheharyarkhan Read More. Synchronising Asynchronous Remote Work 🌍 By [ 7 Min read ]
Hack smoother remote work with these tested strategies. @walo Read More. Django SaaS Architecture: Single-Tenant vs Multi-Tenant - Which Is Right for You? By [ 9 Min read ]
Single Tenant vs Multi-Tenant SaaS application example using Django @pbityukov Read More. Top 15 Trends of the Upcoming Cryptocurrency Bull Run in 2024-2025 By [ 15 Min read ]
In this article, I want to give my overview of the nascent crypto industry trends, which I believe is coming soon. @serkhitrov Read More. The Pocket Guide to Understanding Docker Containers By [ 2 Min read ]
Docker containers have revolutionized software development and deployment by offering a flexible, efficient, and consistent way to package and run applications. @orioninsist Read More. Open the Doors to the Metaverse With Lado Okhotnikov's Meta Force and Royalty NFT By [ 2 Min read ]
Meta Force and Lado Okhotnikov are scaling the Metaverse and giving away Royalty NFT bonuses. @btcwire Read More. The Ten Commandments of ChatGPT By [ 5 Min read ]
ChatGPT has changed the game for many professions. None more so, perhaps, that in the consulting world. @adrien-book Read More. Making a Good Engineer a Great One: Training Your Skills and Training Your Mindset By [ 7 Min read ]
Navigating challenges in engineering growth: from coding struggles to leadership dilemmas, a narrative of perseverance and self-discovery. @jurajmalenica Read More. Embracing the Future: The Unexplored Synergy of AI in Web Development By [ 8 Min read ]
Explore the dynamic role of AI in reshaping the web development landscape — from streamlining operations to enhancing user experience. Dive into real-world appl @hackerclo3f6rop00003b6vpescpy4b Read More. The Tech Company News Brief: How Much Money Did Tech Companies Make Last Quarter? By [ 4 Min read ]
It's been a busy few weeks, with each of the tech behemoths reporting their earnings for the third quarter of the year. @sheharyarkhan Read More. The State of Webhooks in 2023 By [ 4 Min read ]
In this report on the State of Webhooks in 2023, we looked at over 100 of the top API providers and our own internal data to analyze webhook implementations. @tomhacohen Read More. Newsletter Businesses Are About to 10x Due to AI By [ 7 Min read ]
Are we in a newsletter bubble? I think it’s about to go 10x due to AI and platforms like beehiiv that make it easy to start, scale and monetize. @richard Read More. The Battle Between Proprietary and Open Source AI By [ 7 Min read ]
OpenAI started the AI race with ChatGPT and GPT-4. But after it, many other players have joined the AI race, both in proprietary and open-source AI. 🧑‍💻 What happened in your world this week? It's been said that , , . Feeling stuck? We got you covered ⬇️⬇️⬇️ We hope you enjoy this worth of free reading material. Feel free to forward this email to a nerdy friend who'll love you for it.
See you on Planet Internet! With love,
The Hacker Noon Team ✌️ @juanfrank77 Read More. writing can help consolidate technical knowledge establish credibility and contribute to emerging community standards ANSWER THESE GREATEST INTERVIEW QUESTIONS OF ALL TIME

PACELINE

paceline.fit

-- employees

Since n.d.

COMPANY RANKING

PACELINE

EVERGREEN INDEX #7390

Representing this company? Click here to claim and customize this page!

Paceline WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

PACELINE

paceline.fit

-- employees

Since n.d.

COMPANY RANKING

PACELINE

EVERGREEN INDEX #7390

RELATED COMPANIESRANK

STORIES ABOUT PACELINE

Representing this company? Click here to claim and customize this page!

Paceline WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!