The CMA will explore whether technical barriers, fees to transfer data, volume discounts and software licensing practices are hindering competition in cloud services. The Competition and Markets ...

CMA outlines scope of market investigation into cloud services

Cloud computing is a huge market that continues to grow. It has made it easy for people to deploy and manage their apps. Anyone is free to sign up for a cloud-based app. You can do that directly ...

Best cloud broker of 2023

Additionally, all L. lota used in the present study were quickly transported to the laboratory and then were anesthetized with tricaine methanesulfonate (100 mg/L) before dissection. We used the Trizol Reagent Kit (Vazyme, Nanjing, China) to extract the ...

Transcriptional responses of liver and spleen in Lota lota to polyriboinosinic polyribocytidylic acid

With an increase in popularity of video streaming business, corporate businesses are looking out for the most reliable, secure as well as scalable cloud video streaming. With the professional-sticken video hosting infrastructure, brands can now ...

Video Cloud Streaming – What It Is & Why You Need To Know In 2023?

An envelope. It indicates the ability to send an email. An curved arrow pointing right. The Cloud Sofa. The name tells you everything you need to know. You can imagine how it will feel before you even sit in it, said designer Travis London, principal of ...

9 inexpensive Cloud couch dupes that look and feel like the real thing, according to interior designers

I write about technology here on Forbes and elsewhere. “The cloud” is frequently in the news, but there is also a fair amount of confusion, outside of technology teams. What is cloud computing ...

Cloud Computing: United States Businesses Will Spend $13 Billion On It

By using this website you agree to our terms and conditions and privacy policy. Cloud computing is a service model that allows information technology (IT) customers to obtain computing resources over the internet. In network diagrams and flowcharts ...

Cloud Computing

Sony announces cloud streaming for its PlayStation 5, available exclusively for PlayStation Plus Premium Members. The new function bypasses storage consumption and endless downloads with extra benefits like Game Trials for Premium Members. The move marks ...

Streaming the future: Sony's PS5 cloud debut challenges Xbox's dominance

An effective and reliable method of communication is a cornerstone of any successful business. Cloud-based phone systems, also known as internet calling or Voice over Internet Protocol (VoIP ...

Should You Get A Cloud-Based Business Phone System?

This step-by-step guide will show you how to quickly turn a list of words into a word cloud using the Pro Word Cloud add-in for Microsoft PowerPoint. We may be compensated by vendors who appear on ...

How to Quickly Create a Word Cloud Using PowerPoint

Lota Cloud

lotacloud

Here's a practical guide on how to manage Terraform provider configurations for different Yandex Cloud regions using Terragrunt. What you'll need Terraform >= 1.9.7
Terragrunt >= 0.67.16
Yandex Cloud Provider >= 0.129.0 Setup Steps Let's look at how to use Terragrunt to dynamically create provider configs for Yandex Cloud. I'll break this down into digestible pieces: Basic provider setup
First, we'll set up the base Yandex Cloud config in the root terragrunt.hcl. This will automatically generate versions.tf for each module:
locals {
  tf_providers = {
    yandex = ">= 0.129.0"
  }
}

generate "providers_versions" {
  path      = "versions.tf"
  if_exists = "overwrite"
  contents  = <<EOF
terraform {
  required_version = ">= 1.9.7"

  required_providers {
    yandex = {
      source  = "yandex-cloud/yandex"
      version = "${local.tf_providers.yandex}"
    }
  }
}
EOF
}



Region settings
For regions like the newly created KZ region, additional endpoints need to be specified due to the default configuration for the RU region. We can specify them at the project level, for example env.hcl and the providers.tf is generated dynamically for each module:
locals {
  cloud_id         = "SOME_ID"
  folder_id        = "SOME_ID"
  sa_key_file      = "${get_repo_root()}/key.json"
  endpoint         = "api.yandexcloud.kz:443"       # Region-Specific
  storage_endpoint = "storage.yandexcloud.kz"       # Region-Specific
}

generate "providers_configs" {
  path      = "providers.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "yandex" {
  service_account_key_file = "${local.sa_key_file}"
  cloud_id                 = "${local.cloud_id}"
  folder_id                = "${local.folder_id}"
  endpoint                 = "${local.endpoint}"
  storage_endpoint         = "${local.storage_endpoint}"
}
EOF
}



Additional providers
If you're working with Kubernetes / Kubectl / Helm in Terraform, you'll need these additional provider configs to manage your cluster. The simplest and most straightforward solution would be to pass cluster_id from a terragrunt dependency into the called module:
dependencies {
  paths = ["path/to/your/mks"]
}

dependency "mks" {
  config_path = "path/to/your/mks"

  mock_outputs_allowed_terraform_commands = ["init", "validate", "plan", "destroy"]
  mock_outputs_merge_strategy_with_state  = "shallow"
  mock_outputs = {
    cluster_id = "cluster_id"
  }
}

terraform {
  source = "path/to/your/module"
}

inputs = {
  cluster_id = dependency.mks.outputs.cluster_id
  . . .
  <OTHER_INPUTS>
  . . .
} Then use data resources in the module to configure providers: variable "cluster_id" {
  type        = string
  default     = null
  description = "Managed Kubernetes Service cluster ID"
}

data "yandex_kubernetes_cluster" "this" {
  cluster_id = var.cluster_id
}

data "yandex_client_config" "this" {}

provider "kubernetes" {
  host                   = data.yandex_kubernetes_cluster.this.master.0.external_v4_endpoint
  cluster_ca_certificate = data.yandex_kubernetes_cluster.this.master.0.cluster_ca_certificate
  token                  = data.yandex_client_config.this.iam_token
}

provider "helm" {
  kubernetes {
    host                   = data.yandex_kubernetes_cluster.this.master.0.external_v4_endpoint
    cluster_ca_certificate = data.yandex_kubernetes_cluster.this.master.0.cluster_ca_certificate
    token                  = data.yandex_client_config.this.iam_token
  }
}

provider "kubectl" {
  host                   = data.yandex_kubernetes_cluster.this.master.0.external_v4_endpoint
  cluster_ca_certificate = data.yandex_kubernetes_cluster.this.master.0.cluster_ca_certificate
  token                  = data.yandex_client_config.this.iam_token
} Notes Using Terragrunt for configuration management:
Terragrunt simplifies configuration management for multiple environments by dynamically generating provider configurations via the generate block in the .hcl files. This setup allows for easy handling of multi-region deployments from a single configuration source.


Setup JSON key for Terragrunt:
To access the Yandex Cloud resources, place the JSON key for the service account in the root directory of your project. Don't forget to add it to .gitignore. Alternatively you can use a static access key.


Configuring the module:
Remember that even if you don't manage the terraform module directly, you can almost always override the configuration using generate when calling the module in terragrunt. Conclusion This setup gives you a clean way to manage Terraform configs across different Yandex Cloud regions. It handles authentication properly and works well whether you're just using basic cloud resources or diving into Kubernetes and Helm deployments. Here's a practical guide on how to manage Terraform provider configurations for different Yandex Cloud regions using Terragrunt. What you'll need Terraform >= 1.9.7 Terragrunt >= 0.67.16 Yandex Cloud Provider >= 0.129.0 Terraform >= 1.9.7 Terragrunt >= 0.67.16 Yandex Cloud Provider >= 0.129.0 Setup Steps Let's look at how to use Terragrunt to dynamically create provider configs for Yandex Cloud. I'll break this down into digestible pieces: Basic provider setup
First, we'll set up the base Yandex Cloud config in the root terragrunt.hcl. This will automatically generate versions.tf for each module:
locals {
  tf_providers = {
    yandex = ">= 0.129.0"
  }
}

generate "providers_versions" {
  path      = "versions.tf"
  if_exists = "overwrite"
  contents  = <<EOF
terraform {
  required_version = ">= 1.9.7"

  required_providers {
    yandex = {
      source  = "yandex-cloud/yandex"
      version = "${local.tf_providers.yandex}"
    }
  }
}
EOF
} Region settings
For regions like the newly created KZ region, additional endpoints need to be specified due to the default configuration for the RU region. We can specify them at the project level, for example env.hcl and the providers.tf is generated dynamically for each module:
locals {
  cloud_id         = "SOME_ID"
  folder_id        = "SOME_ID"
  sa_key_file      = "${get_repo_root()}/key.json"
  endpoint         = "api.yandexcloud.kz:443"       # Region-Specific
  storage_endpoint = "storage.yandexcloud.kz"       # Region-Specific
}

generate "providers_configs" {
  path      = "providers.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "yandex" {
  service_account_key_file = "${local.sa_key_file}"
  cloud_id                 = "${local.cloud_id}"
  folder_id                = "${local.folder_id}"
  endpoint                 = "${local.endpoint}"
  storage_endpoint         = "${local.storage_endpoint}"
}
EOF
} Additional providers
If you're working with Kubernetes / Kubectl / Helm in Terraform, you'll need these additional provider configs to manage your cluster. The simplest and most straightforward solution would be to pass cluster_id from a terragrunt dependency into the called module:
dependencies {
  paths = ["path/to/your/mks"]
}

dependency "mks" {
  config_path = "path/to/your/mks"

  mock_outputs_allowed_terraform_commands = ["init", "validate", "plan", "destroy"]
  mock_outputs_merge_strategy_with_state  = "shallow"
  mock_outputs = {
    cluster_id = "cluster_id"
  }
}

terraform {
  source = "path/to/your/module"
}

inputs = {
  cluster_id = dependency.mks.outputs.cluster_id
  . . .
  <OTHER_INPUTS>
  . . .
} Basic provider setup First, we'll set up the base Yandex Cloud config in the root terragrunt.hcl. This will automatically generate versions.tf for each module: locals {
  tf_providers = {
    yandex = ">= 0.129.0"
  }
}

generate "providers_versions" {
  path      = "versions.tf"
  if_exists = "overwrite"
  contents  = <<EOF
terraform {
  required_version = ">= 1.9.7"

  required_providers {
    yandex = {
      source  = "yandex-cloud/yandex"
      version = "${local.tf_providers.yandex}"
    }
  }
}
EOF
} Basic provider setup Basic provider setup First, we'll set up the base Yandex Cloud config in the root terragrunt.hcl . This will automatically generate versions.tf for each module: terragrunt.hcl versions.tf locals {
  tf_providers = {
    yandex = ">= 0.129.0"
  }
}

generate "providers_versions" {
  path      = "versions.tf"
  if_exists = "overwrite"
  contents  = <<EOF
terraform {
  required_version = ">= 1.9.7"

  required_providers {
    yandex = {
      source  = "yandex-cloud/yandex"
      version = "${local.tf_providers.yandex}"
    }
  }
}
EOF
} locals {
  tf_providers = {
    yandex = ">= 0.129.0"
  }
}

generate "providers_versions" {
  path      = "versions.tf"
  if_exists = "overwrite"
  contents  = <<EOF
terraform {
  required_version = ">= 1.9.7"

  required_providers {
    yandex = {
      source  = "yandex-cloud/yandex"
      version = "${local.tf_providers.yandex}"
    }
  }
}
EOF
} Region settings For regions like the newly created KZ region, additional endpoints need to be specified due to the default configuration for the RU region. We can specify them at the project level, for example env.hcl and the providers.tf is generated dynamically for each module: locals {
  cloud_id         = "SOME_ID"
  folder_id        = "SOME_ID"
  sa_key_file      = "${get_repo_root()}/key.json"
  endpoint         = "api.yandexcloud.kz:443"       # Region-Specific
  storage_endpoint = "storage.yandexcloud.kz"       # Region-Specific
}

generate "providers_configs" {
  path      = "providers.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "yandex" {
  service_account_key_file = "${local.sa_key_file}"
  cloud_id                 = "${local.cloud_id}"
  folder_id                = "${local.folder_id}"
  endpoint                 = "${local.endpoint}"
  storage_endpoint         = "${local.storage_endpoint}"
}
EOF
} Region settings Region settings For regions like the newly created KZ region, additional endpoints need to be specified due to the default configuration for the RU region. We can specify them at the project level, for example env.hcl and the providers.tf is generated dynamically for each module: KZ endpoints RU env.hcl providers.tf locals {
  cloud_id         = "SOME_ID"
  folder_id        = "SOME_ID"
  sa_key_file      = "${get_repo_root()}/key.json"
  endpoint         = "api.yandexcloud.kz:443"       # Region-Specific
  storage_endpoint = "storage.yandexcloud.kz"       # Region-Specific
}

generate "providers_configs" {
  path      = "providers.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "yandex" {
  service_account_key_file = "${local.sa_key_file}"
  cloud_id                 = "${local.cloud_id}"
  folder_id                = "${local.folder_id}"
  endpoint                 = "${local.endpoint}"
  storage_endpoint         = "${local.storage_endpoint}"
}
EOF
} locals {
  cloud_id         = "SOME_ID"
  folder_id        = "SOME_ID"
  sa_key_file      = "${get_repo_root()}/key.json"
  endpoint         = "api.yandexcloud.kz:443"       # Region-Specific
  storage_endpoint = "storage.yandexcloud.kz"       # Region-Specific
}

generate "providers_configs" {
  path      = "providers.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "yandex" {
  service_account_key_file = "${local.sa_key_file}"
  cloud_id                 = "${local.cloud_id}"
  folder_id                = "${local.folder_id}"
  endpoint                 = "${local.endpoint}"
  storage_endpoint         = "${local.storage_endpoint}"
}
EOF
} Additional providers If you're working with Kubernetes / Kubectl / Helm in Terraform, you'll need these additional provider configs to manage your cluster. The simplest and most straightforward solution would be to pass cluster_id from a terragrunt dependency into the called module: dependencies {
  paths = ["path/to/your/mks"]
}

dependency "mks" {
  config_path = "path/to/your/mks"

  mock_outputs_allowed_terraform_commands = ["init", "validate", "plan", "destroy"]
  mock_outputs_merge_strategy_with_state  = "shallow"
  mock_outputs = {
    cluster_id = "cluster_id"
  }
}

terraform {
  source = "path/to/your/module"
}

inputs = {
  cluster_id = dependency.mks.outputs.cluster_id
  . . .
  <OTHER_INPUTS>
  . . .
} Additional providers Additional providers If you're working with Kubernetes / Kubectl / Helm in Terraform , you'll need these additional provider configs to manage your cluster. The simplest and most straightforward solution would be to pass cluster_id from a terragrunt dependency into the called module: Kubernetes Kubectl Helm Terraform cluster_id dependencies {
  paths = ["path/to/your/mks"]
}

dependency "mks" {
  config_path = "path/to/your/mks"

  mock_outputs_allowed_terraform_commands = ["init", "validate", "plan", "destroy"]
  mock_outputs_merge_strategy_with_state  = "shallow"
  mock_outputs = {
    cluster_id = "cluster_id"
  }
}

terraform {
  source = "path/to/your/module"
}

inputs = {
  cluster_id = dependency.mks.outputs.cluster_id
  . . .
  <OTHER_INPUTS>
  . . .
} dependencies {
  paths = ["path/to/your/mks"]
}

dependency "mks" {
  config_path = "path/to/your/mks"

  mock_outputs_allowed_terraform_commands = ["init", "validate", "plan", "destroy"]
  mock_outputs_merge_strategy_with_state  = "shallow"
  mock_outputs = {
    cluster_id = "cluster_id"
  }
}

terraform {
  source = "path/to/your/module"
}

inputs = {
  cluster_id = dependency.mks.outputs.cluster_id
  . . .
  <OTHER_INPUTS>
  . . .
} Then use data resources in the module to configure providers: data variable "cluster_id" {
  type        = string
  default     = null
  description = "Managed Kubernetes Service cluster ID"
}

data "yandex_kubernetes_cluster" "this" {
  cluster_id = var.cluster_id
}

data "yandex_client_config" "this" {}

provider "kubernetes" {
  host                   = data.yandex_kubernetes_cluster.this.master.0.external_v4_endpoint
  cluster_ca_certificate = data.yandex_kubernetes_cluster.this.master.0.cluster_ca_certificate
  token                  = data.yandex_client_config.this.iam_token
}

provider "helm" {
  kubernetes {
    host                   = data.yandex_kubernetes_cluster.this.master.0.external_v4_endpoint
    cluster_ca_certificate = data.yandex_kubernetes_cluster.this.master.0.cluster_ca_certificate
    token                  = data.yandex_client_config.this.iam_token
  }
}

provider "kubectl" {
  host                   = data.yandex_kubernetes_cluster.this.master.0.external_v4_endpoint
  cluster_ca_certificate = data.yandex_kubernetes_cluster.this.master.0.cluster_ca_certificate
  token                  = data.yandex_client_config.this.iam_token
} variable "cluster_id" {
  type        = string
  default     = null
  description = "Managed Kubernetes Service cluster ID"
}

data "yandex_kubernetes_cluster" "this" {
  cluster_id = var.cluster_id
}

data "yandex_client_config" "this" {}

provider "kubernetes" {
  host                   = data.yandex_kubernetes_cluster.this.master.0.external_v4_endpoint
  cluster_ca_certificate = data.yandex_kubernetes_cluster.this.master.0.cluster_ca_certificate
  token                  = data.yandex_client_config.this.iam_token
}

provider "helm" {
  kubernetes {
    host                   = data.yandex_kubernetes_cluster.this.master.0.external_v4_endpoint
    cluster_ca_certificate = data.yandex_kubernetes_cluster.this.master.0.cluster_ca_certificate
    token                  = data.yandex_client_config.this.iam_token
  }
}

provider "kubectl" {
  host                   = data.yandex_kubernetes_cluster.this.master.0.external_v4_endpoint
  cluster_ca_certificate = data.yandex_kubernetes_cluster.this.master.0.cluster_ca_certificate
  token                  = data.yandex_client_config.this.iam_token
} Notes Using Terragrunt for configuration management:
Terragrunt simplifies configuration management for multiple environments by dynamically generating provider configurations via the generate block in the .hcl files. This setup allows for easy handling of multi-region deployments from a single configuration source. Setup JSON key for Terragrunt:
To access the Yandex Cloud resources, place the JSON key for the service account in the root directory of your project. Don't forget to add it to .gitignore. Alternatively you can use a static access key. Configuring the module:
Remember that even if you don't manage the terraform module directly, you can almost always override the configuration using generate when calling the module in terragrunt. Using Terragrunt for configuration management: Terragrunt simplifies configuration management for multiple environments by dynamically generating provider configurations via the generate block in the .hcl files. This setup allows for easy handling of multi-region deployments from a single configuration source. Using Terragrunt for configuration management: Using Terragrunt for configuration management: Terragrunt simplifies configuration management for multiple environments by dynamically generating provider configurations via the generate block in the .hcl files. This setup allows for easy handling of multi-region deployments from a single configuration source. Terragrunt .hcl Setup JSON key for Terragrunt: To access the Yandex Cloud resources, place the JSON key for the service account in the root directory of your project. Don't forget to add it to .gitignore. Alternatively you can use a static access key. Setup JSON key for Terragrunt: Setup JSON key for Terragrunt: To access the Yandex Cloud resources, place the JSON key for the service account in the root directory of your project. Don't forget to add it to .gitignore . Alternatively you can use a static access key . JSON key .gitignore static access key Configuring the module: Remember that even if you don't manage the terraform module directly, you can almost always override the configuration using generate when calling the module in terragrunt. Configuring the module: Configuring the module: Remember that even if you don't manage the terraform module directly, you can almost always override the configuration using generate when calling the module in terragrunt. generate Conclusion This setup gives you a clean way to manage Terraform configs across different Yandex Cloud regions. It handles authentication properly and works well whether you're just using basic cloud resources or diving into Kubernetes and Helm deployments.

Setting up Yandex Cloud Provider with Terraform and Terragrunt

To Build, or Not to Build, That is the Question: Techno-Optimism vs. Decelerationism

নির্মাণ করা, না নির্মাণ করা, এটাই প্রশ্ন: টেকনো-অপটিমিজম বনাম মন্থরতাবাদ

Bauen oder nicht bauen, das ist die Frage: Technooptimismus vs. Entschleunigungismus

Construir o no construir, esa es la cuestión: tecnooptimismo versus desaceleracionismo

Construire ou ne pas construire, telle est la question : techno-optimisme contre décélérationnisme

निर्माण करें, या न करें, यही प्रश्न है: तकनीकी-आशावाद बनाम मंदीवाद

構築すべきか、構築すべきでないか、それが問題です: テクノオプティミズム vs. 減速主義

건설하느냐, 건설하지 않느냐, 그것이 문제이다: 기술 낙관주의 vs. 감속주의

Construir ou não construir, eis a questão: Tecno-otimismo vs.

Строить или не строить, вот в чем вопрос: технооптимизм против деселерационизма

İnşa Etmek ya da İnşa Etmemek Sorun Bu: Tekno-İyimserlik ve Yavaşlamacılık

Xây dựng hay không xây dựng, đó là câu hỏi: Lạc quan về công nghệ và chủ nghĩa giảm tốc

建设还是不建设，这就是问题：技术乐观主义与减速主义

Is Software Eating The Useless Class?

Cowen, Andreessen, and Horowitz: Annotated

Generating Unity builds continuously, for game development, using personal computers is no fun-game. Manually generating them is much more vexing because each build takes a lot of time to configure and compile. Moreover, these builds are slow on personal computers due to high memory utilization and are prone to frequent human errors. Automating build pipeline and offloading them from the developer machines into the “cloud” or using build platforms like Travis CI or CircleCI seems plausible. However, due to higher costs, developers are coerced to take an alternative route. Here’s our story of why we built BuildBot, the open source build pipeline tool for Unity builds. The Analysis of Time and Cost of Unity Builds We, at TotalCloud.io, have been leveraging the Unity3D platform to build our immersive visual cloud management platform since inception. Without a doubt, Unity3D is a wonderful multi-platform game engine, be it for game development or non-game interactive simulations and visualizations. However, it has been quite a challenge for us to tackle the high time-to-build on Unity 3D. Sometimes, the time taken to generate one build was close to an hour. At times, we push close to 10 builds per day! Now, imagine the insane amount of time we spent just on generating the builds. We wanted our developers to see changes in the builds as soon as possible, without waiting for hours. To top it, as the time-to-build increased, so did the cost. We were in dire need of a solution that could automate builds in a very short span of time without using costly build platforms. Just like us, one of the significant factors that keep indie game developers and small studios away from build platforms or CI solutions is the cost incurred to generate builds of a game. Unlike big game studios who might leverage on-prem build servers or CI servers to continuously generate their builds, these indie developers or small teams do not have that kind of money to splurge on hardware and other build platforms. So, if a game developer generates 10 builds per day, the total cost incurred per month will run into hundreds of dollars, which is heavy on the pocket. Apart from the cost and time factor, there are other drawbacks of manually generating the Unity builds, such as: Inability to handle complex build pipeline Provides less control over the environment Makes code signing difficult Decreases collaboration in a team All these factors lead us to build TotalCloud BuildBot, the open source build pipeline tool for Unity builds. Now let’s dive deeper into cost per build. The Cost Per Build Comparison: CircleCI Vs. Unity Cloud Vs. AWS Let’s assume you generate ten builds per day, with the ability to do three builds in parallel. This accounts for approximately 300 builds per month. If each build took close to 30 minutes, then you will consume ~9000 minutes of build time per month. Now let’s make a quick cost comparison between different build platforms: With CircleCI, it would cost anywhere between ~$249 USD to ~$449 USD per month With Unity Cloud Build, it would cost ~$125 USD per month for the Pro Licence + few dollars for concurrent builds With AWS, it would cost anywhere between ~$6 USD to $7 USD per month if you use EC2 instance with Windows Base AMI and i3.large machine This clearly shows that using AWS is one of the most cost effective solutions. Hence, we leveraged AWS and built the BuildBot, an open source on-demand build pipeline tool that allows an indie developer or small teams to generate hundreds of builds per month at a fraction of the cost compared to any other solution available in the market. More on Why We Chose AWS Spot Instance with Windows AMI The reason to go with EC2 spot instance with Windows Base AMI and i3.large machine instead of Mac OS is that generating Unity builds in Linux is very unreliable and doesn’t work most of the times, because of headless mode issues. On the other hand, AWS does not have Mac OS AMIs. This leaves us with Windows machines. And, i3.large is a big machine with 15.25 GB memory, 475 SSD storage, high read write capacity and good CPU, which is enough to generate bigger builds. Plus, we are going to leverage NNVM SSD volumes to generate the builds faster instead of using the inbuilt EBS volumes. Since a build machine needs to come up only when required, we are going to leverage AWS Spot instances to achieve cost savings. Because, i3.large spot instances cost ~$0.0468 USD per hour at 70% discounted rates. We can easily stop the machine when not in use and spin it up quickly when a new commit is available. We will talk more about it in our upcoming build pipeline post. For now, let’s walk you through the steps on generating Unity builds in AWS. How to Generate Unity Builds in AWS 1. Launch and connect an EC2 Spot instances from the AWS console with Windows 2016 Base AMI and i3.large instance type. For developers who are new to AWS or want a quick walk through on how to launch an i3.large instance type from AWS console, . click here 2. Inside the Windows machine, launch the Powershell ISE from the Start menu. Create two files with the below Powershell scripts. The first script is to create an additional SSD volume that is required to generate builds; and the second script is to install Unity3D on to the Windows machine quickly. Here’s the script to create the additional NNMV SSD volume. In the script we will name it ‘Jenkins Volume’ because it will be a part of our Jenkins Build Pipeline. If (!(Test-Path D:)) { Get-Disk | Where-Object IsOffline –Eq $True | Set-Disk –IsOffline $False $newdisk = @(get-disk | Where-Object partitionstyle -eq ‘raw’) $Labels = @(‘Jenkins’,’Other1',’Other2') for($i = 0; $i -lt $newdisk.Count ; $i++) { $disknum = $newdisk[$i].Number $dl = get-Disk $disknum | Initialize-Disk -PartitionStyle GPT -PassThru | New-Partition -AssignDriveLetter -UseMaximumSize Format-Volume -driveletter $dl.Driveletter -FileSystem NTFS -NewFileSystemLabel $Labels[$i] -Confirm:$false } } Here’s the script to install Unity3D engine. P.S: Ensure you save this Powershell script in C drive. $uri_editor = “ " ; https://download.unity3d.com/download_unity/a9f86dcd79df/Windows64EditorInstaller/UnitySetup64-2017.3.0f3.exe $uri_linux_support = “ " ; https://download.unity3d.com/download_unity/a9f86dcd79df/TargetSupportInstaller/UnitySetup-Linux-Support-for-Editor-2017.3.0f3.exe $uri_mac_support = “ " ; https://download.unity3d.com/download_unity/a9f86dcd79df/TargetSupportInstaller/UnitySetup-Mac-Support-for-Editor-2017.3.0f3.exe $out_editor = “C:\Users\Administrator\Documents\UnitySetup64.exe” ; $out_linux_support = “C:\Users\Administrator\Documents\UnitySetup-Linux-Support.exe” ; $out_mac_support = “C:\Users\Administrator\Documents\UnitySetup-Mac-Support.exe” ; $start_time = Get-Date ; $wc = New-Object System.Net.WebClient ; $wc.DownloadFile($uri_editor, $out_editor) ; Write-Output “Download complete, $out_editor > Time taken: $((Get-Date).Subtract($start_time).Seconds)s” ; $wc.DownloadFile($uri_linux_support, $out_linux_support) ; Write-Output “Download complete, $out_linux_support > Time taken: $((Get-Date).Subtract($start_time).Seconds)s” ; $wc.DownloadFile($uri_mac_support, $out_mac_support) ; Write-Output “Download complete, $out_mac_support > Time taken: $((Get-Date). Subtract($start_time).Seconds)s”; 3. Look for Unity setup in the Unity folder from C drive and install packages for Windows, Linux as well as MacOS. 4. Wait for Unity to setup. 5. Download your code from the Repo and look for the path of the project you want to compile the builds. 6. Create a BAT file to automatically store the builds as they are generated. Here’s an example of a BAT file. Make sure you add the right paths to the respective commands. Example: Use the project path URL you used to save the project in line 2 of the code (inside quotes). C:\Development\Unity\Editor\Unity.exe -quit -batchmode -projectPath "D:\jenkins\workspace\unity-dev-build-4x"

-buildWindows64Player "D:\jenkins\workspace\unity-dev-build-4x\builds\win\64\TotalCloud.exe" -buildOSXUniversalPlayer "D:\jenkins\workspace\unity-dev-build-4x\builds\osx\64\TotalCloud.app" -buildLinux64Player "D:\jenkins\workspace\unity-dev-build-4x\builds\linux\64\TotalCloud.x86_64" -logFile "D:\jenkins\workspace\unity-dev-build-4x\logs\35.txt" || EXIT /B 1 7. Try generating a new build and check if the new build is available in the Output path and the log file in their designated paths. Do try it out and share your experience. This is Part 1 of the three-part series blog providing the stepping stone to using TotalCloud BuildBot. Stay tuned with us for our second blog, where we provide detailed information on how to go about the CI/CD pipeline using BuildBot. Originally published at blog.totalcloud.io on March 14, 2018.

TotalCloud BuildBot: A Cost-Efficient & Rapid Unity Build Pipeline Open Source Tool (Part 1)

Marv, an IT manager of an e-commerce business and in-charge of controlling cloud computing cost of his department, has been facing a big challenge ever since his company’s cloud usage scaled up. He is finding it difficult to extract detailed information about which team member drove the increase each month and which application or resource caused the spike in AWS bill. He is also having a hard time figuring out what’s happening in the cloud inventory at the moment. His team of twenty, which consists of DevOps and CloudOps, are trying diligently to optimize the cloud usage. Challenges like these are a typical scenario in every IT set up today, irrespective of the size and vertical. You Cannot Save What You Cannot See As the cloud usage of a business grows, the infrastructural complexities grow proportionally and so does the complexity of the bill. Even though Amazon addresses this challenge with consolidated billing and tagging, IT managers, like Marv, still need to segregate development environment as well as testing accounts to identify cost leakages across different projects, departments, various applications running on AWS, etc. Moreover, how can he identify or analyze saving opportunities when he does not have visibility into complete AWS usage? There’s a way out for Marv to resolve it — using , which maps the cost to actual AWS resources used for building the e-commerce site and renders them all on a visual and interactive console. TotalCloud Cost Analyzer Virtually View AWS Cost Spillage in Real-Time and Take Corrective Action The first stepping stone to achieve optimal cost savings while on AWS is to map the cost of each resource as well as the application as a whole, and also analyze resource utilization. With Group-View, Marv can dynamically group his e-commerce website’s resources, using multiple parameters, right from tag value, tag key and VPC ID to resource type and application, and start mapping the cost of these grouped resources to get a better cost perspective; and all these on a visually-rich interface. TotalCloud Cost Analyzer’s Grouping Example for an E-Commerce Website TotalCloud Cost Analyzer’s Marv can analyze spend on each resource or the complete application, and take corrective action. He can get real-time bird’s eye view of AWS spend for different resource groups. He can also drill-down with TotalCloud interactive zoom-in feature, with few mouse scrolls, up to each resource level and perform root-cause analysis. Cost Analyzer’s TotalCloud Cost Analyzer Group-View Here’s the level 1 of cost analysis for Marv to understand AWS spend at the application level (like e-commerce website), or as we call it ‘group-level,’ with rich color codes denoting the spend for each group, namely Web Front-end, Web checkout service, and Marketing engine. Group View Example for an E-Commerce Website TotalCloud Cost Analyzer’s TotalCloud Cost Analyzer Drill-Down View Here’s the level 2 cost analysis for Marv to understand AWS spend at group of resources level, with rich color codes denoting the spend for that particular group of resources. Drill Down View Example for an E-Commerce Website’s Web Checkout Service TotalCloud Cost Analyzer’s can help Marv further drill-down into a particular application’s resource usage (for instance, the same e-commerce website’s checkout service), and understand the cost of the individual resource, visually. TotalCloud Cost Analyzer Best part, visually-rich color codes (denoting the cost of each resource) provide Marv with the first-hand information on AWS cost, which otherwise would have been difficult to comprehend with data-rich reports. Resource Level Cost Analysis Example for an E-Commerce Website’s Web Checkout Service TotalCloud Cost Analyzer’s Marv can also identify all the culprit resources and optimized resources at a glance and thus eliminate the need for bulky reports and dashboards to comprehend overall cost. Get a Clear AWS Cost Perspective Even Without Tagging Resources Tagging plays a pivotal role in identifying cost leakages and understanding the overall resource usage at the application level. So, to get a better cost perspective, IT Managers like Marv and his DevOps team use organization-wide tagging practices and reports. However, at times, DevOps overlook or forget to tag resources, including the new resources dynamically allocated in a auto-scaling group. In such cases, Marv can lose out on all the cost-saving opportunities he was hoping for. TotalCloud addresses this challenge of locating untagged resources with a smart feature. Even if the DevOps and CloudOps team forget to tag a resource, Marv can still get the cost perspective of a particular application. Because, TotalCloud groups resources at the application level, by default. TotalCloud’s Default Application Level Grouping of Resources Know the Most Costliest Region & Spend Across All Regions at a Glance Apart from getting a customized drill-down view of AWS spend, Marv and his team can understand current global spend, projected spend and the previous day’s spend in a snap with ! TotalCloud’s Globe View TotalCloud’s Globe View Get Proactive and Prescriptive Insights Marv can even get key recommendations and deep insights on AWS cost leakages and potential savings based on cloud resource usage. Additionally, his DevOps and CloudOps team can look into these insights, take remedial action, and optimize AWS cloud usage accordingly, to save on AWS spend. TotalCloud Insights Marv can also perform root-cause analysis of the culprit resource and take corrective action with the real-time view of the inventory. TotalCloud Resource Level Insight Thus, with the real-time view of the inventory, Marv and his team can visualize the impacted resources, compare and contrast with other resources to build the right cost perspective. To summarize, TotalCloud Cost Analyzer facilitates users to: 1. Understand AWS spend across regions with its global-view. 2. Get AWS spend visibility, according to their projects and applications hosted on AWS with its group-view. 3. Obtain complete cost perspective based on their business needs with its top-view. 4. Fetch AWS spend data at resource level with its drill-down view. 5. Take corrective actions using proactive insights that provide smart recommendations. Bottom Line: Unlock Maximum Savings on AWS with TotalCloud Cost Analyzer In an on-demand, pay-as-you-go cloud world like AWS, it’s easy for costs to spiral out of control. But with TotalCloud’s powerful interactive and visual console IT managers like Marv and his DevOps and CloudOps team to take complete control over their AWS spend. Additionally, with real-time rendering of resources help them in making quick decisions and adjustments based on the current pricing of the instances. Are you one of those IT Managers like Marv, or a DevOps or a CloudOps engineer trying to figure out how to unlock maximum AWS cost savings. If you are, you need to try out . TotalCloud Cost Analyzer Take a and check it out for yourself. free trial today

A Game Plan to Unlock Maximum AWS Cost Savings with TotalCloud Visual Cost Analyzer

Cost optimization strategies for AWS services are abundant. Prioritizing between your options is necessary to make sure you don’t overload yourself with the wealth of information. Looking at the best practices in the industry right now and the practices that have now become obsolete would help you find stability in your finances. Cost Visibility and Analysis Before diving into the nitty-gritty details of the strategies to be employed, you need to monitor the current status of your infrastructure to gain insight into what you need. Accessing billing information and purchase reports enables you to analyze your expenses and directly restrict additional costs. Additional monitoring of your cloud resources, their utilization, accuracy & security can save massive costs. AWS has a few native tools that point out what you need to be doing by looking at where your money is going. The tools analyze your bill to predict future expenses or give you a detailed report on where your expenses went. Using this information, you can set up budget plans for your reserved instances. Identifying unallocated Elastic IPs or unused objects with these tools also lets you save costs by deleting them. The three valuable native AWS cost optimization tools are: AWS Cost Explorer AWS Trusted Advisor Cost and Usage Report Detailed Billing used to be able to provide this service but now it has been disbanded and instead, Cost and Usage Report does it. Detailed Billing had let you group your expenditure by the features, tools or parts of your infrastructure to see where expenses are high and low. Now that you have an understanding of how to analyze your expenses, it’s redundant if it’s not actionable. Here are the three major changes you can undertake, based on the information you have accessed. These are the Pillars of AWS Cost Optimization Scheduling Rightsizing Storage optimization Scheduling Scheduling is the act of maintaining or limiting the runtime of your resources so that you don’t pay for non-usage.Oftentimes, poor (or non-existing) scheduling strategies are the main culprit for why companies suffer massive losses. How often have there been articles about a server being turned on and left running for months, or even years together?Since the pricing model of many AWS services charges you for the duration a service stays up, every moment counts. This makes scheduling an integral element to cost optimization. The most widely scheduled resource is EC2, but there’s no need to stop at just that. We’ve observed that cost-savings can come from scheduling every other resource as well. Some of them inherently support start/stop functions (like RDS Instances & Redshift Clusters), and some don’t (ECS, EKS, Fargate). But even these can be optimized to save costs. The go-to metric to create a schedule is time. You’re maintaining uptime of your resources based on business hours or the time you deem necessary for them to be up. In a 168-hour week, you’re most likely using your resources for only 40 hours (a typical 8-hour workday x 5). Shutting your resources down the remaining 76% of the time can be the difference between an optimized, cost-effective workload and one that gives you dreadful bill shocks. A more automated & quick method is usage-based scheduling, where resources shut down on the basis of idleness. You can set up your system to detect idle resources and park them in real time. This intricately optimizes resource usage, to save you 2x more costs than the usual time-based scheduling. Now, you can automate the scheduling of resources with CloudWatch events or a third-party AWS scheduler. While CloudWatch is an AWS exclusive option that offers flexibility, it requires customers to write their own code to execute their automated tasks. A third-party AWS Scheduler would be able to offer similar functionalities but with a more user-friendly approach. Rightsizing Rightsizing achieves the lowest possible cost of maintaining your instances by choosing the appropriate instance types and size according to your performance and hardware requirements. Rightsizing is the strategy to adopt to avoid spending excess on the hardware you don’t need. Purchasing the instances you need won’t end the rightsizing process. Keeping a periodical monitoring(monthly recommended) of the resources you have purchased is necessary to plan ahead on future purchases and to break down what works and what doesn’t. The 4 factors to monitor that decide what the right amount of instances to purchase are vCPU utilization Memory utilization Network I/O utilization Disk utilization Purchase Behaviour On-Demand Instances On-Demand Instances are the most commonly purchased instance types, mainly due to how much control the customer is given with it. For long-time users, looking into the updated pricing list can be beneficial to their AWS cost optimization. It is likely they are running instances that are currently more expensive than their alternatives. Instances such as m1,c1, and t1 can now be switched to m3,t2, and c3. This migration comes with superior CPU performance, higher memory, and a lower price. Stagnating from updating your instances could cost you 10–20% higher. Reserved Instances Purchasing the right AWS reserved instances can save costs up to 60% of your current expenditure. Businesses that continuously make use of the AWS cloud environment should opt to evaluate their Reserved Instances every month. Reserving an instance for one year or three-year duration and usage parameters can fetch an hourly rate lower than On-Demand pricing for up to 75%. AWS Reserved instances for DynamoDB charges based on throughput instead of running hours. That is, whether the instance is running or not, the reservations continue to charge. Spot Instances The last instance type to pay attention to for cost optimization is Spot Instance. Spots are spare Amazon EC2 instances that can be purchased at prices less than 90% of its original. The spot instance is terminated if its price exceeds the customer’s stated price or the capacity ends up unavailable. Using the pricing history on the AWS console makes finding the right Spot Instance easy. Spot Instances are best used along with workloads that don’t suffer from interruption and can be replaced with On-Demand instances without backups and data restoration. Deploying a tool like Spot Instance Advisor gives you the benefit of being prepared to purchase the right Spot instance with the least amount of interruptions and fair pricing. Spot Fleet will further help you by having multiple spot instances ready to be deployed according to their value in the case of an interruption. Storage Optimization You are likely spending a lot of money for storage space that you are not utilizing at all or utilizing improperly. The main goal of cutting storage costs is ensuring your services remain functional at optimal conditions. To ensure this, you have to balance your use of Amazon S3 storage tiers and other AWS storage services properly. When evaluating storage requirements, customers should segment data by how available and durable it needs to be, the size of data sets, throughput and IOPS thresholds, and regulatory requirements. Amazon S3 Storage Lifecycle Optimization The ideal way to optimize your storage is to set an S3 storage lifecycle. This will make the optimization automatic. AWS S3 has several storage classes where you move your resources according to the purpose of each class. The various pricing models they offer can effectively cut down your costs as long as you handle distribution wisely. Amazon S3 storage tiers include: Deleting unused disk volumes open up large spaces for backups and moving your data more freely. Keep tabs on your storage allocation and you can save up a ton. Structural Cost Optimization You can also optimize your costs by targeting the expenses made by the individual members or teams of your organization. Miscommunication can lead to poor resource handling which causes excess purchases that you might not need. Complex Infrastructure When your business is large and has a complex infrastructure that spans across the globe, effective communication between overseas teams becomes difficult to achieve. The various departments of an organization and their individual technical specialties are a huge factor in said miscommunication. There are usually multiple teams, one that sets up the infrastructure and another that actually operates it; and the former team has control over their runtime. If there is any miscommunication between the deploying team & the operating team, it can lead to unforeseen charges. Setting up policies and guidelines for your various teams to follow aligns all the employees to common practice. It could remove potential communication errors by encouraging the teams to open educated conversations with peers of other specialties. The Human Element The human element of running a business makes it prone to occasional errors. Forgetting to shut down a resource, deleting unused volumes or using up more expensive resources are all common careless mistakes. The best solution is to balance the manual workload with the automated workload. A lot of the strategies can be implemented using automation tools. Taking some of the burdens of your employees can lead to lower chances of errors. How Cost Optimization can help other aspects of your business Detecting Rogue Infrastructure If you’ve got infrastructures that fulfill purposes that the organization has not assigned it to do, then it has gone rogue. Depending on the complexity of this unauthorized activity, your infrastructure might be something that is racking up some excess expense. Analyzing your bills with cost management tools can help you pinpoint what infrastructure is acting up. Security holes Analyzing your bills could help determine any potential vulnerabilities. Unintended spikes in prices could be because of a potential breach in security. Neglected projects You can identify half-finished projects that continue to charge you but aren’t being worked on. Idle projects can either be discontinued or you can strip its resources off temporarily. With the right strategies for your resources and the right tools to assist you in actualizing these practices, you are on your way to stabilize the finances for your cloud management. Previously published at https://www.totalcloud.io/blog/the-proven-practices-for-successful-aws-cost-optimization

Successful Cost Optimization on AWS: Best Practices

So you’ve decided to choose AWS as the primary provider of your cloud service and now you’re looking into setting up your environment. You’ve got your project to be deployed and all you have left to do is choose an AWS instance that will run your machine image. But now, like many others before you, you are stumped by the countless choices of EC2 instances out there. Choose the wrong ones and you pay the price not only from your wallet but also in the effectiveness of your cloud environment. It is a very common obstacle for new AWS users to overcome. Along with the varied types of instances, the different AWS EC2 pricing options add fuel to the flames of uncertainty. For example, purchasing i3EN 3XLarge for its memory and CPU specs is a choice that will leave you with an extra $2k expense in your annual bill when you can gain the same size for a cheaper price by purchasing Z1D 3XLarge instead. Having a concrete budgetary plan with a deep insight into the specs you are looking for can greatly help narrow down your choices. However, we don’t all come in with solid plans. Sifting through the choices is time-consuming and ineffective. Totalcloud’s Instance Comparison Chart lets you view the instances in a scatter chart with rich customization options. You can compare instances, filter it by your preferences and view the different instances from lowest to highest EC2 prices. Choose from compute-optimized instances, General purpose instances or any other instance type, filter them according to the memory or vCPU specs you have in mind and view your narrowed down the list of instances according to the price from lowest to highest. Change the pricing model from AWS Reserved Instances of Windows or Linux to On-Demand pricing. Click on the data points to compare multiple instances. View the prices as hourly, weekly, monthly or annually. Whether you come with everything you want already in your mind or you’re completely blank on what you’re looking for, you’ll still gain a lot of information from the chart. We hope you can find the right EC2 instances for your projects, you can provide us with feedback for any suggestions you might have. You can check the tool out, . here Previously published at https://medium.com/faun/instance-comparison-chart-find-the-right-aws-ec2-instance-ae6f91de927d

Instance Comparison Chart: Find The Right AWS EC2 Instance

This Gen Z Dev is addicted to emojis so much! He uses them for AWS instance names as well! His Gen X manager is totally annoyed. Do you use emojis in AWS instance names??!! Or, are you looking for ways to set the instance name? If so, here’s one below: ec2addtag i-??????? --tag Name=a-fancy-name For more information, check out this . AWS CLI Command reference document Subscribe to our Newsletter and get comics and other super useful AWS blogs quickly delivered to your inbox. Originally published at blog.totalcloud.io on October 23, 2018.

Emojis in AWS Instance Names

Ever since Amazon announced AWS EC2 C5D instances, we at — as AWS practitioners — have been digging deep into the technicalities of it. We found some interesting stuff, and we’d like to share our findings for the greater good. TotalCloud.io C5D Intro: AWS EC2 Instances with Local NVMe Storage For those unaware, AWS EC2 C5D instances are high-performance block storage instances. Introduced sometime during May 2018, they are said to be ideal for applications that need access to high-speed, low latency local storage. This has made it media vertical’s favorite instance out of the lot. If you want to know more about its applications and other details, . click here Tests To test the performance of C5D instances, we launched a C5D.large instance in US-east 1 region with 9GB Amazon Elastic Block Store + 50GB Amazon EC2 NVMe Instance Storage. In the same region and availability zone, we launched a C5.large instance with 54 GB EBS volume and T2.small instance for comparison with no NACLs in the way, and open security groups between the instances. All instances were running Amazon Linux 2 OS (ami-0a5e707736615003c), and patched up to date as of October 2018. Before we walk you through the benchmarking results, let’s have a look at the pricing comparison between C5D.Large, C5.Large and T2.Small on-demand instances: We ran the Sysbench tool to calculate all prime numbers up to 20,000 to compare and contrast the pure compute performance of C5D instances with C5 instance as well as the the most popular instance type, T2. Here are the kind of tests we ran on all the three instances: To run CPU tests, we made use of the SysBench tool, with a single threaded CPU test. To measure disk performance, we made use of Sysbench file IO benchmark (using random r/w). As these instances are capable of bursting, we conducted few tests to drain the IO balance. Before draining the IO balance, we used IOping to test latency. Then, using FIO, we attempted to drain the balance and test the performance as the burst bucket was emptied and refilled. To benchmark the performance of a typical file server workload, we made use of the blogbench score. To run network tests and demonstrate TCP/IP latency, we used average latency of 100 icmp packets. Additionally, we used iperf to test the bandwidth between devices. To get a more a slightly more “real-world” test, we used the phoronix test suite to run a benchmark compiler test (compiling Linux). This gives real world disk IO across a filesystem alongside general system and CPU performance. Disk IO is often a constraining factor in machine performance, but it very much depends on what particular workload you’re running. Here are the results: Here are Our Key Observations: #1 | The CPUs all come out at the same score, as all three instances have CPUs of the same capability. #2 | The C5D vastly outperforms the C5 and T2 in File IO due to the nvme disk, with nearly 4 times the read and write capability of a C5 instance. AWS uses the same underlying disk technology for C5 and T2. Hence the results for C5 and T2 are similar. #3 | The disk latency measure (IOping) shows that the nvme disk has half the latency of the C5. This is a dramatic difference, and has significant implications for latency-sensitive workloads such as real-time analytics. #4 | The network measures are essentially the same for C5D and C5 instances, as expected, but significantly improved over the T2 instance. #5 | The compile time is interesting. The task hits a CPU bottleneck at this point. Although the C5D is faster every time, it’s hard to find a repeatable real-world task that really stretches the capability of the new nvme disk. #6 | The test showed that the CPU maxes out to 100% to compile Linux. This indicates that C5D is a little faster. Even though compiling Linux C5D is a very disk intensive operation, the CPU was the ultimate bottleneck of the task in our test case. Just to add to these tests, the C5 instance took 8.10 seconds to unpack the Linux kernel. The C5D took 8.06 seconds to unpack. The Wrap UP There were several tests conducted over a week to benchmark C5D.Large instance’s performance. Hope these test results gives you the real perspective of the new C5D instances. If you’d like to know more about these test results, write to us at tech@totalcloud.io or tweet to us at . Also, do share your views on this post. We would love to hear your thoughts. @totalcloudio Originally published at blog.totalcloud.io on November 9, 2018.

Stories, News & Mentions about LOTACLOUD

Lotacloud’s Stories on HackerNoon

Lotacloud's News & Mentions