CloudBind

cloudbind

Use the open-source integrated machine learning in MindsDB and the open-source data integration platform Airbyte to forecast Shopify store metrics. With the volume of data increasing exponentially, it’s critical for businesses focused on e-commerce to leverage that data as quickly and efficiently as possible. Machine learning represents a disruption to increase predictive capabilities and augment human decision making for use cases like price, assortment and supply chain optimization, inventory management, delivery management, and customer support. In this ‘how-to’ guide, we’ll provide step-by-step instructions showing you how to simply and inexpensively integrate machine learning into an existing Shopify account using , an open-source data integration platform, and , an open-source AutoML framework that runs on top of any database. Airbyte MindsDB Our guide will consist of two parts: Using Airbyte to extract your Shopify data and load it into a database. In our case, we’ll use the open-source DB PostgreSQL, but it can also be a different database of your choice. Using MindsDB to automatically build and train a machine learning model for predicting purchase orders. Let’s get started! We will assume you already have Airbyte set up via Docker. If not, here’s a link to a that shows how to do it. Getting Started article Step 1: Replicating Data from Shopify to PostgreSQL with Airbytea. Setting Up the Airbyte Shopify Connector is going to allow us to get your Shopify data and store it in a database or data warehouse. For this article, we will be storing the data obtained via the Shopify connector into a PostgreSQL database. Airbyte’s Shopify connector The Airbyte Shopify connector is capable of syncing abandoned checkouts, collections, custom collections, customers, metafields, orders, products and transactions to the destination of your choice. Let’s get started by spinning up the Airbyte web app. First, go into the directory where you cloned the Airbyte repository, and then run the following in your terminal:‍ docker-compose up Then, visit in your browser to see the Airbyte dashboard. http://localhost:8000 In the section, click on the button in the top right corner of the dashboard to add a new source. Sources + new source In the new screen, enter the name of the source-we’ll call the source but you can name it anything you want. Then, in the dropdown, select shopify-source, Source type Shopify. Selecting the connector will prompt Airbyte to load the configuration parameters that the Airbyte Shopify connector needs. This might take a while if it is the first time you are selecting the Shopify connector. Airbyte will pull the Shopify connector image from Docker Hub behind the scenes. Shopify When it’s done pulling the image, you will see the following: Let’s peek into the parameters the Shopify connector needs: : This will be the name of the shop on Shopify you want to replicate the data from. So if the URL to your shop is , the name of your shop will be . shop https://airbyte.myshopify.com airbyte : This will be the date you want Airbyte to start the replication from, and it takes the date in the format YYYY-MM-DD. start_date : This will be the password of a private app you created on your Shopify dashboard. api_password The shop name and can be filled in readily, but the api_password requires that you create a new private app on Shopify in case you don’t already have one. Let’s walk through creating a private app on Shopify and obtaining the password that we need to set up the Airbyte Shopify connector. start_date b. Creating a Shopify private app To create a Shopify private app, log into your Shopify dashboard and click on in the sidebar menu. Apps Then, click on the link at the bottom of the page. If you have not enabled private apps previously, you will see a screen to enable it. Click on the button. Manage private apps Enable private app development You will be taken to a screen to accept Shopify’s terms on private app development usage. Check the checkboxes if you agree, and then click on once again. Enable private app development Once private app development has been enabled, you will be redirected to a screen where you can create a private app. Click on the button to get started. Create private app In the section, fill in the private app name and emergency developer email. App details Go into to the Admin API section and perform the following: Click on the dropdown, and locate the in the list. Then, grant from the select box. Show inactive Admin API permissions Order API Read access Locate and and grant those as well. Shopify Payments disputes Shopify Payments payouts, Finally, click on the button to save your new private app. Save You will be greeted with a confirmation modal-just click on the button to confirm the creation of the private app. That’s it! Now we just need to get the private app API password. Create app On the same page, scroll down to the section and copy your private app API password. Admin API Then, head over to the Airbyte web UI dashboard and provide the Shopify connector configuration form with the password you just copied. In the Airbyte UI where you left off setting up Shopify, paste in the password you copied from Shopify and then hit . Set up Source c. Setting up the destination Once you have set up the source successfully, Airbyte will navigate to the destination screen. Click on the button in the top right corner. In the dropdown, click on add destination + add a new destination. Airbyte will validate the source and then present a form to enter the destination name and type. Enter as the name and choose as the destination type. To set up our Postgres destination, we will spin up a Postgres container via Docker. So run the following in your terminal: shopify-destination Postgres docker run — rm — name shopify-db -e POSTGRES_PASSWORD=password -v $HOME/docker/volumes/postgres:/var/lib/postgresql/data -p 2000:5432 -d postgres Note that the bit of the above snippet is important to persist the data, as by default, data in a Postgres Docker container is ephemeral and will be lost when the container is destroyed. You can create the directory on your local machine (as we are assuming it doesn’t exist). Run the following to create it before the command to spin off the Docker container: -v $HOME/docker/volumes/postgres:/var/lib/postgresql/data mkdir -p $HOME/docker/volumes/postgres Verify that the container is running. Then fill in the login credentials in the Airbyte dashboard. After Airbyte has set up the destination successfully, you will be presented with the schema of the data to be replicated onto the destination, and a dropdown to specify the syncing frequency with the source. In this tutorial, we chose manual, as we want direct control over when our syncs take place . Then hit the button. Set up connection Airbyte will navigate you to the destination page where you can hit the button to start the replication. To monitor the syncing, you can click on the destination name to see the syncing process and status. Launch Once Airbyte is done replicating the data onto the Postgres database, the status of the sync will change to . To confirm that the syncing was successful, you can run the below command: Succeeded docker exec shopify-source psql -U postgres -c “SELECT COUNT(*) FROM public.transactions;”‍ The above should return the count of all the records on the table. transactions Step 2: Forecasting Purchase orders with MindsDB The next step is to perform predictive analysis on top of the store data that we have imported inside the database. Before we do that, we need to first install MindsDB. There are a few ways to install it as described in the . Our suggestion is to always use as it encapsulates all of the dependencies for running MindsDB. To download the latest production image, run the following command: documentation Docker docker pull mindsdb/mindsdb Alternatively, if you have Python pre-installed on your machine, you can use to install the latest MindsDB version. To do that, create a new virtual environment and run: pip pip install mindsdb Then, to start the server run: docker run -p 47334:47334 mindsdb/mindsdb Or, if you have installed with run: pip python3 -m mindsdb‍ Finally, MindsDB Studio should be automatically started on your default browser on . http://127.0.0.1:47334/ a. Connect MindsDB and PostgreSQL There are three ways that you can use MindsDB. The first is through the graphical user interface called MindsDB Studio. The second way is using the REST APIs. To make the integration simple, you can use our Python or JavaScript SDKs. The third way is to do it directly inside the database through executing SQL statements. In this tutorial, we will explain everything via the MindsDB Studio so you can follow along visually. We will also share the SQL queries in case you want to test directly inside your database. To connect MindsDB and PostgreSQL from the left navigation menu, head over to integrations and click on the button Here, you will need to fill out all of the parameters for connecting to the PostgreSQL database. Add Database . The required parameters to connect to PostgreSQL are: Supported Databases: From the dropdown select PostgreSQL. : Let’s call it ShopifyData. Integration Name : The name of the database where you saved the shopify data. Database Host: The PostgreSQL host name or IP address. : The PostgreSQL port (default is 5432). Port : The user that has access to the database. Username : Password for the above user. Password After filling the form, click on . You have now successfully connected to PostgreSQL. The next step is to create a datasource from the data which you would like to use for model training. CONNECT b. Orders Data Let’s get an overview of the data. After we’ve imported the data inside PostgreSQL, Airbyte created different tables in the public schema. If you list the tables, you will notice that there are tables for , and a few other tables that Airbyte has created before importing the data. orders orders_refunds, products, transactions, abandoned_checkouts, The data that is important to predict the future number of orders is inside the table. Let’s so we can see the list of orders. There are more than 30 columns inside the table, but we will focus only on: orders SELECT * FROM orders; processed_at: The time when the order was processed. vendor: e.g if you are selling shoes, vendors will be Nike, Adidas, Vans, Converse, etc. : We will only the orders that are or already . financial_status SELECT Pending Paid There are a lot of additional options that you can use to do predictive analytics on top of the Shopify data. To keep the showcase simple, we will focus on the above 3 parameters and train a model that estimates the number of orders that the store will receive over the next period. Go to the Database Integrations dashboard and click on . In the popup, you will be able to SELECT the data to train the model:. NEW DATASET Let’s overview our query: SELECT SELECT processed_at::date AS order_day, COUNT(*) AS orders, (unnest(ARRAY(SELECT jsonb_array_elements(line_items)->>’vendor’))) as vendor FROM orders WHERE financial_status IN (‘pending’, ‘paid’) GROUP BY vendor, order_day; This query will select the time when the order was processed and cast it to YY:MM:DD format (without the time of the day, as we will be forecasting at a daily frequency). Then, select the vendor from the line_items column that contains the product info in JSON format. Finally, aggregate the number of orders per day for each vendor. Click on and MindsDB will create a new datasource called ShopifyOrders. CREATE If you preview the data, you will see the number of daily orders grouped by vendor. The above example shows the orders for and stores. It groups the daily orders for the past 5 years for different vendors like NIKE, VANS, ADIDAS, LEGO, MATTEL, etc. SHOES TOYS We now have data connected to MindsDB. The next steps are: Perform data analysis Train an ML model Predict the amount of future orders by querying the model c. Data Analysis In the dashboard, click on the button so that MindsDB can start with its automated data analysis procedure. In the background, it will iterate over each data record and display warnings if there are potential biases or outliers inside the data. If you click on some of the columns, it will display additional information for specific columns such as the type of the data, if there are any missing values, the number of distinct values, and if there are potential biases it will show more information. Dataset QUALITY By performing data analysis, you will get more information about your data and decide if you trust the warnings shown by MindsDB to act upon them (remember, a good dataset is crucial for good ML models), or you can ignore them and continue with the model training. d. Model Training To train a new model click on the tab from the left navigation bar, and then on the button Inside thepopup form, you will need to input all of the required information so that MindsDB can start training a time series forecasting model: Predictors TRAIN NEW . The datasource from which to select the data. In this example, ShopifyOrders. FROM: The name of the new model, e.g shopify_orders. Predictor Name: The target variable that we want to predict. In this example the number of orders count. Select Only The Column to be Predicted: Columns that we want to exclude from the model training. You can leave this empty as we are training the model with only 3 columns. Select Columns to be Removed for Training: Calculate the margin of error in a range from [0.0–1.0]. Leave it empty, so the default value of 0.01 will be used. Sample Margin of Error: Used for testing purposes to speed up the model training. Leave this empty unless you have a time constraint. Stop Training After: Train the model on GPU or CPU. If you have a GPU, select this checkbox to obtain a speedup in the training and querying phases. Use GPU: Select Yes in the checkbox. Is it a Timeseries Prediction Problem?: The columns based on which the data should be ordered. Select processed_at. Order by: The columns based on which to group entities in the data. We will group by vendor, effectively analyzing each different TOY and SHOE vendor on its own to produce accurate forecasts. Depending on how the dimension data is grouped, this can yield anywhere from a handful to hundreds (or more!) time series that the MindsDB predictor will learn to forecast. Group by: Use default value of 1. This means our model will be trained to predict the amount of orders that should come in the next day. Number of Future Predictions: Use the previous values of the target columns as additional information to the time-series specific part of the model. Leave it as checked. Use Previous Target: The number of rows to look back when making a prediction. Set it at 10 to consider the last 10 days’ amount of orders when forecasting. Look Back Window: To start with the training click on TRAIN. Alternatively, you can train the model automatically directly inside the database by just runing the following query: INSERT INTO mindsdb.predictors(name, predict, select_data_query) VALUES(‘shopify_orders’, ‘orders’, ‘SELECT processed_at::date AS order_day, COUNT(*) AS orders, (unnest(ARRAY(SELECT jsonb_array_elements(line_items)->>”vendor”))) as vendor FROM orders WHERE financial_status in (“pending”, “paid”) group by vendor, order_day;’); To get more information about the INSERT query visit . MindsDBs PostgreSQL docs MindsDB will promptly start the automated machine learning pipeline, deducting the data type of each column, instantiating an encoder for each one to obtain informative intermediate representations of the data, and then use this to train a forecasting model based on a mix of recurrent neural networks and gradient boosters. Finally, it will perform a model analysis to offer insights as to how good you can expect the forecasts to be. When this procedure finishes, the status will change to . Next, click on the PREVIEW button. The model preview dashboard provides insights related to the trained model. The first section shows the model accuracy and how MindsDB did the dataset splitting. COMPLETED You can see that the forecasting accuracy for the shopify_orders model is around 74%, which means that 74% of the time, the true amount of orders fell within the bounds of the predicted range for the held-out validation dataset. The next section shows the column importance where MindsDB is rating the columns from 0 as the least important to 10 as the most important column for making a good prediction. Given the settings we used, our time series forecaster needed all three columns to train, so this part of the analysis is not applicable to this predictor. The last section presents a confusion matrix, where if you hover over the matrix you will see the percentage of time where the model predicted the number of orders correctly or the percentage of the time where the model misclassified the number of orders. A perfect forecaster would show a thin straight diagonal. In this case, the diagonal is still well-formed, which reinforces the reported accuracy of 74%. e. Predict the Number of Orders The model was successfully trained. How can we use it to predict the number of orders? From the left navigation tab click on the option,and then on the button Query New Query . Let’s imagine that we want to forecast “ .” How many Converse orders will the store get next Friday? In the dropdown, select the vendor e.g Converse and the date e.g 6/18/2021 for which we want to predict the number of orders. Then, click on the and in less than a second MindsDB will return the predicted value. RUN QUERY If you want to get predictions directly from the database run: SELECT orders AS predicted, orders_confidence AS confidence, orders_explain AS info FROM mindsdb.shopify_ordersWHERE “vendor”=’Converse’ AND “order_day”=’2021–06–18';‍ In this example, for Converse products on Friday 18 June 202, MindsDB thinks that the store will get around 57 orders with a minimum order of 32 and maximum orders of 78. Also, MindsDB is 74% confident that this will be the correct number of orders. The width of the predicted region can be further tuned by the user at the expense of modified confidence: a tighter pair of bounds would imply less confidence in predictions and vice versa. Another thing to note is that queries will be more accurate as you provide more historical context. Wrapping up We have shown how easy it is to use Airbyte to extract and load data inside the database, and then leverage MindsDB to perform predictive analytics to drive better business based decisions. For further information and examples check out the documentation on , and for a limited time sign up for a free MindsDB Cloud account at Airbyte MindsDB http://cloud.mindsdb.com. This article can also be seen at: https://airbyte.io/recipes/implement-machine-learning-for-your-shopify-store-with-mindsdb

How to Forecast Purchase Orders for Shopify Stores Using Open-Source

In today's digital world, where images and videos are pivotal in web applications, optimizing the media delivery process is essential for a seamless user experience. One powerful solution for managing and serving media assets is Cloudinary. In this blog post, we'll explore how to integrate Cloudinary into your Next.js application using the package. next-cloudinary What is Cloudinary? Cloudinary is a cloud-based media management platform that provides an end-to-end solution for storing, managing, optimizing, and delivering media assets such as images, videos, and documents. It offers a robust set of features for transforming and serving media efficiently. Cloudinary also helps developers handle tasks like resizing, cropping, and format conversion without the need for complex server-side code. Why Use Cloudinary with Next.js? Integrating Cloudinary with your Next.js application brings several benefits: Cloudinary optimizes media assets for faster loading, improving your website's performance and user experience. Optimized Image and Video Delivery: Cloudinary allows you to dynamically transform images and videos to suit various device screen sizes and resolutions, reducing the need for manual asset preparation. Media Transformation: Cloudinary leverages a Content Delivery Network (CDN) to deliver assets quickly and efficiently. It also provides caching options for faster load times. CDN and Caching: Cloudinary provides secure and scalable cloud-based storage, reducing the complexity of managing media files on your servers. Secure Storage: With the package, you can easily integrate Cloudinary into your Next.js project. Seamless Integration: next-cloudinary Setting Up a Next.js Project Before we integrate Cloudinary, ensure you have a Next.js project up and running. If not, you can create one using the following steps: Install Node.js if you haven't already. You can download it
from the . official website Create a new Next.js project using the following commands: npx create-next-app my-next-cloudinary-app
cd my-next-cloudinary-app Run the development server with . Your Next.js app should now be running at . npm run dev http://localhost:3000 Integrating Cloudinary with Next.js using next-cloudinary To integrate Cloudinary with your Next.js application, we'll use the package, which streamlines the process. Follow these steps: next-cloudinary 1. Install the Package next-cloudinary In your Next.js project directory, install the package using npm or yarn: next-cloudinary Copy code
npm install next-cloudinary
# or
yarn add next-cloudinary 2. Set Up a Cloudinary Account If you haven't already, create a and obtain your API credentials, including your cloud name and image preset. Cloudinary account 3. Get the Cloud name After logging in to your Cloudinary account, you will see the dashboard. Grab your cloud name and copy it into the environment variables of file in your nextjs app. next.config.js const nextConfig = {
  reactStrictMode: false,
  env: {
    NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME: "*******",
  },
};

module.exports = nextConfig; Make sure that you name the variable as as the next-cloudinary package automatically reads this environment variable. NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME 4. Get an Unsigned Upload Preset Now, we will need an Upload Preset to upload images. Click on the settings icon in the left sidebar of your Cloudinary Account. Now, you will see the Accounts Page. Click on the Upload Tab from the left sidebar to head to upload settings. Now, you will see the upload page. On this page, you will see the Upload Presets section. Click from there to get an unsigned upload preset. Enable unsigned uploading As soon as you click it, you will see an upload preset. Copy that upload preset. Paste this upload preset as an environment variable in the file in your next.js app. next.config.js const nextConfig = {
  reactStrictMode: false,
  env: {
    NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME: "********",
    NEXT_PUBLIC_CLOUDINARY_PRESET_NAME:"********"
  },
};
module.exports = nextConfig; We are going to name the environment variable as . NEXT_PUBLIC_CLOUDINARY_PRESET_NAME That's it. We have all the things that are required for the setup. 5. Add Cloudinary in the allowed domains In Next.js applications, when you serve images from a domain that differs from the current domain of your application, you need to specify the external domain in the file. next.config.js const nextConfig = {
  reactStrictMode: false,
  env: {
    NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME: "********",
    NEXT_PUBLIC_CLOUDINARY_PRESET_NAME:"********"
  },
  images: {
    domains: ["res.cloudinary.com"],
  },
};
module.exports = nextConfig; Add the in the images section. res.cloudinary.com 6. Use in Your Components next-cloudinary Now, we can use Cloudinary in our components to upload the images. The the package exposes a component named as . We are going to use that component for image upload. Here's the code. next-cloudinary CldUploadButton "use client";
import { CldUploadButton } from "next-cloudinary";
export default function Photos() {
  return (
    <div>    
      <CldUploadButton
        options={{ multiple: true }}
        uploadPreset={process.env.NEXT_PUBLIC_CLOUDINARY_PRESET_NAME}
      >
        <span>
          Upload
        </span>
      </CldUploadButton>
    </div>
  );
} Make sure that you have the environment variable in place for uploadPreset. And that's it. With this, we have completed the setup for Cloudianry to upload the images. Here's a demo Conclusion In this comprehensive guide, we've covered the benefits of integrating Cloudinary with Next.js and provided a step-by-step tutorial on using the package to easily manage and serve media assets. Following these steps can enhance your web application's performance, optimize media delivery, and provide a superior user experience. Cloudinary is a powerful tool for managing media, and when combined with Next.js, it becomes a dynamic duo for building efficient and visually stunning web applications. Start harnessing the power of Cloudinary in your Next.js project today! next-cloudinary

The Ultimate Guide to Integrating Cloudinary With Next.js

Having a blog to write and share our knowledge on the internet is essential for visibility and making us known in our field as an authority on a subject. This article will teach you how to build a blog that has dynamic pages for each article. Table of Contents Prerequisites Getting Started What is Nuxt? What is Nuxt Content? What is Cloudinary? Creating a Blog with Nuxt Creating the Blog Notes Fetching the Posts Creating Dynamic Pages Final Thoughts Learn More To follow along, view the codebase and its repo on and the for reference. GitHub demo Prerequisites The following are required to complete this tutorial: Cloudinary account. is free Sign up Basic knowledge of JavaScript Familiarity with Vue Getting Started To get started, open up the terminal and run the command: npx create-nuxt-app blog-for-developers The command above will kick off a command-line interface (CLI) prompt that will allow us to configure various aspects of our project. Now that the boilerplate is set up with all the files and folders, let’s run the development server which is accessible on inside the project. http://localhost:3000 # change directory to project folder cd blog-for-developers # start the development environment npm run dev NuxtJS is an open-source intuitive Vue framework that allows us to build user interfaces, making web development simple and powerful. What is Nuxt? During the initial setup of this project, we selected the Nuxt content module. This module lets us use our codebase as a Git-based headless CMS to access markdown, JSON, YAML, XML, and even CSV files through an API-like interface. With this module , it will inject the instance globally in our project to access it anywhere in a project. What is Nuxt Content? @nuxt/content $content Cloudinary is a cloud-service image and video management tool for web and mobile applications. With Cloudinary, we can upload, create, and manage digital experiences across any browser and device. What is Cloudinary? Creating a Blog with Nuxt Before we create the content for the blog in markdown using Nuxt content, let’s first create the landing page for the blog that will include the navigation links and some text. In the , we can add our favorite font to the head section and use a particular font family as seen below: Add Font Style nuxt.config.js file The above code block will have no effect on the page until we write the CSS that will use the font. Now, add the following code to the which will include the navigation links and some text on the home page. Remove the component, . pages/index.vue <Tutorial /> <template> <section class =" showcase "> < header > < h2 class =" logo "> < nuxt - link to ="/"> ecosurf </ nuxt - link > </ h2 > < nav class =" nav desktop "> < ul > < li class =" nav__list "> < nuxt - link to ="/ blogs "> Stories </ nuxt - link > </ li > </ ul > </ nav > </ header > < video autoplay loop muted src =" https: // res . cloudinary . com / terieyenike / video / upload / v1651393236 / mixkit - tropical - island - landscape - view - 4692 - large_yanvml . mp4 "></ video > < div class =" overlay "></ div > < div class =" text "> < h2 data - type =" uppercase " class =" stroke "> Never Stop </ h2 > < h3 data - type =" uppercase "> Exploring The World </ h3 > < p > View of the tropical island landscape, from a hill with houses, palm trees and many trees, and in the distance the hills that surround the sea, on a sunny day . </ p > </ div > < ul class =" social "> < li > < a href =" https: // twitter . com / terieyenike " rel =" noopener noreferrer " target =" _blank ">< img alt =" twitter profile " src =" https: // i . ibb . co / Wnxq2Nq / twitter . png "/></ a > </ li > < li > < a href =" https: // instagram . com / terieyenike " rel =" noopener noreferrer " target =" _blank ">< img alt =" instagram profile " src =" https: // i . ibb . co / ySwtH4B / instagram . png "/></ a > </ li > </ ul > </ section > </ template > In the above code block, we included a with the source of the video stored in Cloudinary with attributes such as autoplay, loop, and muted that mute the video when it loads. To obtain the media stored in Cloudinary, we upload our desired video to the media library tab. Thereafter, copy the URL which will be attached to the of the . <video> src <video> Also, we referenced the links in the navigation header with the component to both the home and the routes which we will create to view all our blogs. To make sure our page is styled with CSS, create a folder at the root of the project, , and include the following from this . In the file, update the file to include the created CSS file with the following: nuxt-link / /blogs assets/styles/main.css gist nuxt.config.js ... ...
 } export default { css : [ '@/assets/styles/main.css' ], Here is what our page should look like: Creating the Blog Notes To build our blog site, we need to have content to display and render on the page. First, create a directory and then the blog directory that will contain all the markdown files. Writing all the content in markdown supports a lot of the markdown syntax standards such as the headings, links, and code blocks with syntax highlighting for different programming languages. PS: The Nuxt content utilizes the content folder to store all the files. Here’s an example of a markdown file in the content directory. Add the following to the file . /blogs content content/blogs/egghead.md title: Egghead author: Teri Eyenike
    description: All we need to do is open up our terminal and run the command npm install @nuxt/content. Once it's installed, you'll see that inside of our package.json, we see our Nuxt Content module. Next, to finalize the setup, let's go ahead and open up our nuxt.config.js, and let's go ahead and scroll down to the section that's labeled Modules.
    date: May 2, 2021
    publishOn: 2021-05-02T00:00:00
    tags: ["learning", "platform"]
 --- --- cover _image: https://res.cloudinary.com/terieyenike/image/upload/v1651446130/pexels-jeremy-bishop-8241100_ oklfpe.jpg [ Egghead ]( https://www.egghead.io ) is a great platform for developers to enhance their skills!  Markdown files have the ability to add metadata using the concept of front matter which is denoted with the triple-dash syntax at the start and stop of the front matter. The metadata, is what is displayed to readers like the time of publication and excerpt of the blog content, is what is displayed to readers. Other various properties can be included. PS: Inside of the front matter, it uses the YAML syntax. To fetch the posts for the blog page, let’s create a new route under . Create a folder, called blogs, and in there create file for the directory, blogs. Next, copy and add the following code to the file to display the rendered markdown files. Fetching the Posts pages index.vue pages/blogs/index.vue <template> </template> // pages/blogs/index.vue < main > < header > < h2 class = "logo" > < nuxt-link to = "/" > ecosurf </ nuxt-link > </ h2 > < nav class = "nav desktop" > < ul > < li class = "nav__list" > < nuxt-link to = "/blogs" > Stories </ nuxt-link > </ li > </ ul > </ nav > </ header > < div class = "container section" > < div class = "container__grid" > < div v-for = "blog in blogs" :key = "blog.slug + blog.createdAt" class = "card" > < img :src = "blog.cover_image" alt = "blog photographs" /> < div class = "pad__card" > < div class = "author" > < p class = "author__name" > {{ blog.author }} </ p > < span > | </ span > < p > {{ blog.date }} </ p > </ div > < h2 class = "title" > {{ blog.title }} </ h2 > < p > {{ blog.description.substring(0, 150) }}... </ p > < button > < nuxt-link :to = "`/blogs/${blog.slug}`" > Read More </ nuxt-link > </ button > </ div > </ div > </ div > </ div > </ main > < script > export default { async asyncData ( {$content} ) { const blogs = await $content( "blogs" ).sortBy( "publishOn" , "desc" ).fetch() return { blogs } }, head ( ) { return { title : "Read interesting stories as a nomad" , meta : [ { hid : 'description' , name : 'description' , content : 'Daily and juicy content as you learn, work, and relax. WFH' } ] } } } </ script > < style scoped > header { background : #111111 ; position : unset; } .container { width : 85% ; max-width : 75rem ; margin -inline: auto; } .section { padding : 3em 0 ; } .container__grid { display : grid; gap: 2em ; grid-template- columns : repeat (auto-fit, minmax ( 19rem , 1 fr)); } .card { background : #f0f7f4 ; box-shadow : 0px 4px 3px rgba ( 0 , 0 , 0 , 0.1 ); border-radius : 5px ; } .card img { object-fit : cover; border-top-left-radius : 5px ; border-top-right-radius : 5px ; width : 100% } .pad__card { padding : 2em ; } .author { display : flex; align-items : center; } .author , .title { margin-bottom : 1em ; } .author span { margin : 0 0.3em ; } .author__name { text-transform : capitalize; } button { border : unset; padding : 1em 2em ; margin-top : 2em ; background : #0D5159 ; font-weight : 700 ; cursor : pointer; } button a { color : #fff ; } </ style > The code block above does the following: : used to generate our page from Nuxt as it generates individual files statically to users. We pass in the method where the blog notes are stored, the function, and the method. asyncData $content sortBy() fetch : takes two parameters, (sort it by date) and the desc attributes on how you want the content to appear based on the most recent content published sortBy() publishOn To make each blog article dynamic, we will assign it to a URL, where it’s the blog slug on the Read more button using the component. /blogs/ <nuxt-link> The last step for viewing the snippet of the rendered posts from the content directory is to use the v-for directive and loop over the blogs and render each article in a card. Rendering the posts Here is what the blog page should look like now: Creating Dynamic Pages Now let’s create pages for each blog post to have dynamic URL routes. In Nuxt, to create dynamic pages, we append an underscore before the .vue file name so as not to make the URL hardcoded. We then create the file under the blogs directory and add the following code: _slug.vue <template> </template> // pages/blogs/_slug.vue < main > < header > < h2 class = "logo" > < nuxt-link to = "/" > ecosurf </ nuxt-link > </ h2 > < nav class = "nav desktop" > < ul > < li class = "nav__list" > < nuxt-link to = "/blogs" > Stories </ nuxt-link > </ li > </ ul > </ nav > </ header > < div class = "container" > < div class = "return" > < nuxt-link to = "/" > < img alt = "back to home" src = "/home.png" /> </ nuxt-link > < span > / </ span > < nuxt-link to = "/blogs" > Blog </ nuxt-link > < span > / </ span > < p > {{ note.title.substring(0, 15) }}... </ p > </ div > < section > < h1 > {{ note.title }} </ h1 > < nuxt-content :document = "note" cla /> </ section > </ div > </ main > < script > export default { async asyncData ( {$content, route} ) { const note = await $content( `blogs/ ${route.params.slug} ` ).fetch() return { note } } } </ script > < style scoped > header { background : #111111 ; position : unset; } .container { width : 85% ; max-width : 75rem ; margin -inline: auto; } section { padding-bottom : 2em ; } .return { display : flex; margin : 1.5em 0 ; } .return img { width : 20px ; height : 20px ; } .return span { margin : 0 1em ; } h1 { margin-bottom : 1em } </ style > This block of code does the following: : We use the content to fetch a specific file from the directory blogs using the route parameter and the slug which will map to our actual file name ${route.params.slug} To render the markdown as actual content defined in the markdown file, we use the nuxt-content component and pass the prop of the with the entire object document note : Defining the scoped to the style element make sure that the styling only applies to a specific page scoped Final Thoughts The creation of a blog is great when you want to share your thoughts with a wider audience and Nuxt has provided us with a way to do so in this article. There are still other features we can still explore to make sure we have a robust blog website. Learn More Nuxt content Nuxt documentation Try out the live demo

How to Build Your Own Blog with Nuxt, Nuxt Content, and Cloudinary

Cómo crear su propio blog con Nuxt, Nuxt Content y Cloudinary

Comment créer votre propre blog avec Nuxt, Nuxt Content et Cloudinary

Nuxt, Nuxt Content और Cloudinary के साथ अपना खुद का ब्लॉग कैसे बनाएं

Nuxt、Nuxt Content、および Cloudinary を使用して独自のブログを作成する方法

Como construir seu próprio blog com Nuxt, Nuxt Content e Cloudinary

Cách tạo Blog của riêng bạn với Nuxt, Nội dung Nuxt và Cloudinary

如何使用 Nuxt、Nuxt 内容和 Cloudinary 构建自己的博客

while developing a rails you might want to allow your users to upload an image provides an easy and free way of achieving that.... sounds good? let's get started. Application Cloudinary Oh, wait before we start you'll need an API key from if you don't have an account yet, . Alright let's get started now. Cloudinary go ahead and create it Assuming that you have a rails Application with a Post model We'll start by adding cloudinary and carrier wave to Our gem file in the following order. gem 'carrierwave' gem 'cloudinary' then run Next, if you don't have one yet generate an uploader by running bundle i rails g uploader image also, you'll need to have an image column in your post table assuming that you have a post model you can run rails generate migration AddImageToPosts image: string After running let's create a file in the folder, let's go ahead and add our configurations rails db:migrate cloudinary.yml config #config/cloudinary.yml development: cloud_name: CLOUD NAME api_key: 'API KEY' api_secret: API SECRET enhance_image_tag: true static_file_support: false production: cloud_name: CLOUD NAME api_key: 'API KEY' api_secret: API SECRET enhance_image_tag: true static_file_support: true after adding the configurations we need to mount our uploader in the post model # app/models/post.rb class Post < ApplicationRecord mount_uploader :image , ImageUploader end next, we need to configure our uploader to work with Cloudinary. your should look like this app/uploaders/image_uploader.rb #app/uploaders/image_uploader.rb class ImageUploader < CarrierWave::Uploader:: Base include Cloudinary::CarrierWave process convert: 'png' process tags: [ 'post_picture' ] version :standard do process resize_to_fill: [ 100 , 150 , :north ] end version :thumbnail do resize_to_fit( 50 , 50 ) end CarrierWave.configure do |config| config.cache_storage = :file end end And now we are ready to start uploading images to our . app in our form for a post creating < fieldset > <%= f.label :image , 'image' %> <%= f.hidden_field :image_cache %> <%= f.file_field :image %> </ fieldset > to display the image for a given post < img src = "<%= @post .image %>" /> By now we should be able to upload and display from our application. images learn more at https://cloudinary.com/documentation/rails_carrierwave I hope this post was helpful if you have a question or a suggestion leave a comment or find me on twitter

Creating Image Uploader in Rails 6 Using Cloudinary and Carrierwave

Comparing Cloudinary CDN and Amazon AWS CDN CloudFront 

Crypto enthusiasts often face a conundrum when turning to cryptocurrency as a viable investment. An eternal question remains: To invest and forget, or to trade the notorious price swings for even more gains? shows a simple buy-and-hold strategy for Bitcoin the past six years would have yielded 1690% for investors. However, utilizing a long-short trading approach would have seen gains of over 2,500% at a minimum, or even 5,000% using a well-tweaked strategy. Academic research Needless to say, it’s easy to be allured by the prospect of trading cryptocurrencies with those statistics in mind. But as easy as it may look, trading is not for the faint-hearted, with even seasoned investors succumbing to sudden market movements and price drops — look no further than the “ in March this year which caused two very popular crypto funds, Cryptolab Capital and Adamant Capital, to fold overnight with their millions of dollars falling victim to the market. Black Thursday” event This is where a strategy becomes important. One could either take a fundamental approach towards the crypto market — looking at macro factors, correlations with the US equity indices, or Bitcoin miner-led market movements — or choose a technical strategy, meaning indicators and chart-reading to calculate probabilities of a market movement and place trades accordingly. A popular indicator is the Ichimoku Kinko Hyo, a so-called all-in-one trading system that was developed in the 1940s by Goichi Hosoda, a Japanese journalist. Its name translates to “equilibrium at a glance,” and allows traders to trade breakouts, identify support and resistance levels, and generally assess the group think behavior of the markets. “The Ichimoku Cloud has always been a comprehensive framework: it allows trend analysis, tells the trader when to stay out of the market completely, and provides clear signals with a high probability of success.” states Chairman Zen of MachinaTrader. “This combination allows the trader to act dispassionately and mechanistically in the market. And this also means that Ichimoku Cloud is ideal for creating an automated strategy." Here’s what the Ichimoku Kinko Hyo, or colloquially, the Ichimoku Cloud looks like: Daunted? Don’t be. The tool is easy to read and determine once each individual component is singled out. First, is the “Kumo” - the cloud: the big green and red zones in the chart, below, where price always seems to go and either establish a new trend or get rejected. Markets are usually directionless and sideways when trading in the Cloud, meaning traders usually sit out and avoid placing trades while price is inside the Cloud. The cloud is simply a calculation of the support and resistance equilibriums using previous price action of the past “x” periods. Generally, the longer the period “x” is, the less probability of a “false” signal (although absolutely nothing is absolutely certain in markets). Playing the Cloud is easy. Traders can go “long,” or bet on rising prices when price breaks above the Cloud, and “short,” or bet on decline if price breaks below the cloud. A green Cloud is a “support” area, one where the price is expected to see buying and subsequently continue an upward trend. Vice-versa for when the Cloud is red. In addition, the thicker the Cloud, the stronger its support/resistance. A thin Cloud is an indication that the market is unlikely to see much support/resistance when price approaches the Cloud. Hence, the thickness of the Cloud is an important consideration when trading the “Kumo”. Next is the Tenkan sen, or the “conversion line” as singled out (blue) in the chart below: The Tenkan sen calculates the average of the highest high and lowest low during the last “x” number of candles. Some traders feel that the Tenkan sen provides better points of support/resistance than a simple moving average. Here is a 21-period Tenkan sen and a 21-period simple moving average for comparison: Notice that when the Tenkan sen is flat, it indicates a trendless condition in the market during the last 21 candles. While an angled Tenkan sen shows the direction and relative momentum of price action during the last 21 candles. A moving average doesn’t respond in this manner, making the Tenkan sen a line that acts as both support/resistance and a directional indicator. Third, is the Kijun sen or the “base” line, which, similar to the Tenkan sen, is calculated from the highest high and lowest low averaged over the previous “x” number of candles. Why a duplicate? Because this component is calculated over a longer period than the Tenkan sen, thereby providing a second “slower” Tenkan sen for comparison, as well as, a longer timeframe indicator of support/resistance. A possible strategy — used by some traders — is to consider the Tenkan sen and Kijun sen in a similar manner as the popular “moving average crossover”. Trade long when the faster Tenkan sen crosses above the slower Kijun sen (indicating an uptrend) and trade short when the crossover occurs to the downside. Lastly, the Chikou span, or the “lagging” line. This one calculates the current price and shifts that back by “x” number of candles. The main signal given is “trend confirmation”. Once the Chikou span moves from above to below (or below to above) the Ichimoku Cloud it signals trend direction. In the chart, above, the lagging line breaks above the Kumo, providing a strong buy signal for traders. The move is said to be a “bullish” Kumo breakout, while the opposite of this - the Chikou breaking below the Cloud - is usually a “bearish” signal where traders might sell and offload positions. With these four components explained, the Ichimoku Cloud should now be more easy to understand and read. And here is why it is considered to be a “single, integrated strategy”: One glance at the chart provides the trader with an indication of trend (or lack thereof), support and resistance, potential breakouts, and similar past “setups”, making this freely available indicator a Swiss Army knife in any trader’s toolbox. Automate the Cloud? Ichimoku Cloud trading signals don’t necessarily require manual execution. Services like MachinaTrader, an online trading platform that allows traders to automate their strategies. A trader can code the logic of their strategy via the dashboard, or pick from already-coded algorithms (including Ichimoku Cloud), backtest its efficiency before deploying it live in the market. Traders can activate a terminal on the MachinaTrader dashboard and code their own complex trading strategies, define backtesting parameters, and much more. Traders can monetize successful strategies by making them available to other platform users. In addition, it allows you to choose to have a machine learning bot oversee select strategies. Bob, the trading robot can recommend improvements, leading to a more profitable trading strategy with suggestions of optimal parameters, for various market conditions, for indicators like the Ichimoku Cloud. The future of finance is here indeed.

Crypto-Trading with Ichimoku Cloud...And The World of Virtual Trading Assistants

Ruby on Rails has an awesome gem to upload, store and display images: . It's very easy to use and it comes with a local disk-based service, that should be enough to store everything you need if you just want to practice using Rails. Active Storage However, when you start building more elaborated websites and deploy them to production, you'll start noticing the limitations of Active Storage's local disk. In Heroku, for example, although you are able to upload images using disk storage and everything will look fine, these files will disappear after the application is automatically restarted (which happens once every 24 hours on the free plan). Of course, this all makes sense. The purpose of the local disk is for development and testing. For production, Active Storage facilitates uploading files to cloud storage services, such as Amazon S3, Google Cloud Storage, an Microsoft Azure Storage, which are suggested by . Rails Guides Or, if you are a developer student like me and want to save any possible dollar, you can use . Cloudinary Why choose Cloudinary? The first reason that made me choose Cloudinary was the price. It's free! At least for my needs and, I think, for most of yours as well. The Free Classic Plan gives you 10GB of storage, and it's not just for one year, it's forever. The second reason was the possibility to use it on my existing Rails application. Cloudinary provides a that simplifies the integration with Ruby on Rails applications, which, according to the documentation, presents the following features: Ruby Gem Methods to build URLs for image and video manipulations; Rails view helper tags for embedding and transforming images; API wrappers for file upload, administration, sprite generation and more; Server-side file upload plus direct unsigned file upload from the browser using the jQuery plugin; Migration tool; Active Record integration; CarrierWave plugin; Static image syncing for CDN delivery; General Ruby integration for non-Rails frameworks such as Sinatra. Not enough, Cloudinary also allows you to manipulate uploaded images simply by passing a set of transformation parameters on the dynamic URL in your views. For example, you can resize and crop the images, change its format and quality, rotate, and apply a large variety of special effects and filters. All manipulations are performed automatically in the cloud. The free plan offers 20,000 monthly transformations. Setting up Cloudinary on your Rails application The first step, of course, it's to sign up for a Cloudinary free account. Then, to install the Cloudinary Ruby gem on your project, add the following line to the Gemfile: gem 'cloudinary' And run bundle install. To use the library, you need to configure your project to provide Cloudinary's cloud_name, api_key, and api_secret. These credentials can be found on the Dashboard page of your account. Setting the configuration parameters can be done globally by creating a file under the directory of your Rails project. The contents of this file should look like this: cloudinary.yml config production: cloud_name: "sample" api_key: "874837483274837" api_secret: "a676b67565c6767a6767d6767f676fe1" secure: true cdn_subdomain: true You can your customized configuration file through Cloudinary's Management Console, and just place it under the directory. You might also define several optional if relevant for your project. download cloudinary.yml config configuration parameters So, now that everything is configured for using the gem, you will need to create the functionality for uploading and displaying the images on your website. Cloudinary offers many uploading options, such as an upload widget, server-side upload, and direct uploading from the browser. Moreover, it also supports integration with other libraries, like CarrierWage, Attachinary, and Active Storage, which can be useful for integrating image uploads with your models. In my case, my application was almost ready when I realized I needed an alternative to store the images. Until there, I was using Active Storage's local disk, and since I didn't want to change so much of my code, my decision was pretty clear. Integrating Cloudinary with Active Storage The Cloudinary gem allows you to use all its great features, while still using Active Storage to easily upload images from HTML forms to your model. Assuming you have already configured your application to use Active Storage (if not, I recommend reading ), the following steps are pretty straightforward. this article 1. Declare the Cloudinary service in the file by adding a new entry with a custom name (e.g., cloudinary) and the service configuration: config/storage.yml Cloudinary cloudinary: service: 2. In each environment you want to use Cloudinary, you have to tell Active Storage to use this service by updating . For example, if you want to use it on the production environment, you need to have the following line on the : config.active_storage.service config/environments/production.rb config.active_storage.service = # Store uploaded files on the local file system (see config/storage.yml for options). :cloudinary 3.  After setting up Cloudinary as the service, you will use direct uploading, which is already supported by Active Storage. To do that, include in your application's JavaScript bundle: activestorage.js //= require activestorage 4. On the form, include the direct upload URL on the input field: <%= %> <%= %> <%= %> <% %> form_for @post do |f| f.file_field , :picture : ' - ', : class form control direct_upload true f.submit , 'Create' : ' - - - - -100 -3' class btn sign up button btn outline secondary w mt end 5. Now you just need to display the image in the browser. Use the helper method to generate a permanent URL link, that upon access, redirects to a Cloudinary endpoint. url_for <%= %> url_for(@post.picture) One problem and how I solve it: Although with these configurations uploading and storage work fine, in the browser vertical images were being displayed in the wrong orientation. Turns out this happens because pictures captured on cameras, including from mobile devices, are stored with a series of metadata. One of them is called EXIF (exchangeable image file format), which informs the image's orientation. It seems that not all browsers have the support to get this information and apply the rotation automatically. Searching about this problem, I found out that if the image is transformed, Cloudinary will automatically orientate the image based on its EXIF information, and the image will be delivered with the correct orientation. Easy enough, no? As I said before, it's very easy to manipulate uploaded images on Cloudinary, simply by passing a set of transformation parameters on the dynamic URL in your views. Well, not so much. Even though the documentation states that the helper method was enhanced to accept any Cloudinary image transformation as a parameter, when I tried to do that, it results in an error saying this method only accepts 0..1 argument. url_for I couldn't find any answers to this problem yet, but searching through Cloudinary's settings, I found out that is possible to set up default transformations without changing my code. In your account settings, under the tab, you will find a section for . Upload upload presets Click on , and in the tab, you will find When you click on you will see all the available transformation options, including automatic rotation and use EXIF data. add presets Upload Manipulations Incoming Transformations. edit, In my case, I select the following transformations: use EXIF data, a default width, and quality automatic - good. And ta-dah!! Everything looks perfect now and my website it's fully functional, no more vanishing images (and for free!). Update (april 8th 2020) I contacted Cloudinary's support team about this error I was getting,  and it turns out it was a mistake in the documentation. So, instead of using the helper method, you need to use one of these: url_for <%= cl_image_tag(@user.avatar.key, , ) %> OR

<%= cloudinary_url(@user.avatar.key, options = {}) %> width: 200 crop: :scale //

Image Storage in Rails Apps Using  Cloudinary and Active Storage

The web is now dominated with more visual content than ever hence the need to consider ways of delivering these images with the smallest possible file size while maintaining optimal visual quality. The less file size, the faster the browser can download and render the image, the optimal visual quality, the more satisfied your users will be. This, therefore, leads to better user experience, increased retention rate, conversion rate, and user satisfaction. There are many existing techniques and algorithms used to enhance and optimize images already. In this article, I'll introduce you to an amazing technique (based on my opinion and experience), which would help you better enhance and serve your media assets using a cloud-based image and video management platform. Introduction Original Images are often created with the original extra-high resolution to ensure optimal visual quality, thereby resulting in high file size. This would lead to images delivered slowly when used in web pages. Not cool, right? So how then do we deliver images on the web with optimal speed and optimal visual quality? The Need for Optimization The web is filled with tons of images; virtually every website on the internet has more than one image. Optimizing images allows you to deliver high-quality images in the right format, dimension, size, and resolution while keeping the smallest possible size. According to , 47 % of users expect a page to load in 2 seconds or less, and users will abandon a website that takes more than 3 seconds to load. This will drastically reduce your conversion rate (Amount of visitors turned into users). Kissmetrics The conversion rate is the number of conversions divided by the total number of visitors. For example, if an e-commerce site receives 200 visitors in a month and has 50 sales, the conversion rate would be 50 divided by 200, or 25%. ~ Optimizely Optimizing Images will: Improve page load speed Improve user experience Improve SEO ranking And this can be done in different ways, either by resizing the images, caching, or by compressing the image size. Introducing Cloudinary is the market leader in providing a comprehensive cloud-based image and video management platform. Cloudinary is being used by hundreds of thousands of users around the world, from small startups to large enterprises. ~ Cloudinary Source For better understanding, this is how Cloudinary image optimization works: Upload your image(s) to their service You get a URL for the image Apply transformations (if you want) to optimize the images Load your image on your website via the URL Bingo! you have optimized image(s) Whenever you apply any transformation to an image, Cloudinary performs the following optimizations by default: Strips all associated metadata from the transformed image file leaving the original image untouched Applies an automatic quality adjustment to generated WebP, GIF, JPEG, and JPEG-XR image formats. Runs optimization algorithms to minimize the file size without impairing the visual quality when generating images in the PNG, JPEG, or GIF format. Getting Started The process of getting your images on Cloudinary is pretty quick, but before I show you how-to, let's roll through one fact that this is a PAID SERVICE :). Chill, there is a FREE plan for developers and businesses but with limitations (limited credits: sort of internet data for using Cloudinary services) With the FREE plan, you get 25 credits. One credit equals 1000 transformations or 1GB of managed storage or 1GB of monthly viewing bandwidth or 500 SD video processing seconds or 250 HD video processing seconds. For now, you don't need to worry about PAID plans, but as your images and it's bandwidth increase, your credits get maxed out, and you will need to upgrade to either the Plus ($89/month) or Advanced plan ($224/month). See all plans . here Let's get started, yeah? STEP ONE Create a Cloudinary Account . Follow all the prompts and fill them appropriately. here Upon successful creation of your account, you will get a Cloud name, API Key, API Secret, and Environment variable, which can be found in your console dashboard. The API keys and Secrets are useful when you're working with , but in this article, we'll only use the HTTPS CDN method to fetch images. I'll be publishing another article around how to setup Cloudinary with a React application soon. Cloudinary SDKs Subscribe to my to get updated. newsletter here STEP TWO Head to your media library in your Cloudinary console, where all your uploaded media assets will be listed. If your account is newly created, then you will only find a sample image. Treat this media library like a drive where you can create folders, subfolders, and add images in them. STEP THREE Create a new folder and upload your desired image or upload it in the root folder. STEP FOUR Click on the image you uploaded then click on the 🔗 icon at the left top corner of the image. You have now copied the URL of the uploaded image. On my end, I uploaded and here is the generated URL this image https://res.cloudinary.com/bolaji/image/upload/v1583358060/hashnode_banner.png Now you can use this URL as your image source attribute and load on your website. I'll load the URL above like so: < = = = > a href "https://hashnode.com" target "_blank" rel "noopener noreferrer" < = = </ > img src "https://res.cloudinary.com/bolaji/image/upload/v1583358060/hashnode_banner.png" alt "Hashnode Banner" img </ > a Pretty cool, right? Cloudinary Automatic Image Transformations Cloudinary allows you to add some transformation parameters and values to your image.  Its algorithm analyzes the image to find the best quality compression level and optimal encoding settings based on the image content and the viewing browser, to produce an image with good visual quality while minimizing the file size. Automatic Image Quality and Encoding: q_auto (quality) This transformation allows you to control the visual quality and compression level of JPEG, JPEG 2000, JPEG XR, WebP, and GIF images with the quality parameter (q in URLs). I uploaded to Cloudinary with an initial file size of 1.46 MB this image Now I apply the parameter with value (q_auto) just after the and the image is delivered with optimal visual quality while minimizing the file size to 297 KB based on the image content and the viewing browser. That's about 1163 KB lost in 215 ms. q auto /upload/, https://res.cloudinary.com/bolaji/image/upload/q_auto/v1583381262/bolaji_headshot.jpg Automatic Image Fetch Format: f_auto (fetch format) This transformation allows you to analyze the image content and select the best format to deliver with the fetch parameter (f in URLs). Different browsers support different image formats, Cloudinary allows you to deliver the best format according to the browser used by each of your visitors while saving bandwidth and optimizing delivery time. I uploaded to Cloudinary with an initial file size of 1.46 MB this image Now I apply the parameter with value (f_auto) just after the and the image is delivered as WebP to Chrome browsers (376 KB), JPEG-XR to Internet Explorer browsers, or as a regular JPEG to browsers that support neither format. f auto /upload/, https://res.cloudinary.com/bolaji/image/upload/f_auto/v1583381262/bolaji_headshot.jpg If you add the and parameter, then Cloudinary will deliver WebP and JPEG-XR to the compatible browsers but might deliver selected images as PNG-8 or PNG-24 if the quality algorithm determines that as optimal. f_auto q_auto Using the automatic format and automatic quality is most recommended as it allows Cloudinary's algorithm to decide how to deliver the image based on the user's browsers leading to better visual quality and optimal images. https://res.cloudinary.com/bolaji/image/upload/q_auto,f_auto/v1583381262/bolaji_headshot.jpg Conclusion The faster your website loads images, the faster your website loads, the better your SEO ranking becomes, the better your user experience becomes, the more users interact with your website/ product, the increased your conversion rates become. It's a win-win, satisfy your users and increase your conversion rates. I consider this article fundamental as there is so much more you can do with Cloudinary to manage your media assets, and I hope this helps you find your way to the numerous other transformations you can perform with Cloudinary :). Further Resources Cloudinary Image Transformation Documentation Compress Images for Web and Boost Performance on Your Site Optimization and Fast Delivery Automatically Reduce Image Size Without Losing Quality Compression of Image Files With Only One Line of Code How to make your website load lightning fast with WebP images Automatically Reduce Image Size Without Losing Quality Best Image Optimization Techniques: Expert Roundup Previously published at https://bolajiayodeji.com/image-optimization-and-transformation-with-cloudinary-ck7ft0r9100p7nds1ja70s9mu

Image Optimization and Transformation with Cloudinary

So you've developed an impressive rails APP that showcases a gallery of images. Its time to unveil it on the worldwide web via Heroku. Image uploads in a local development environment are currently defaulted to local psql. In production environment (Heroku), a cloud storage service stores images. By popular choice, AWS S3 cloud service would be suitable to host your images. They offer a free plan, but to sign up it requires a credit/debit card. In my experience setting AWS up for image-handling was challenge. If one does not exercise extreme caution; API key and secret will get exposed on Github. Unscrupulous bots are on the prowl for these credentials to exploit compute resources. Hence raking up a hefty bill. Since Heroku does not host pictures, Heroku's addon cloud service  Cloudinary does. It offers a solution to web application’s entire image management pipeline. Cloudinary offers comprehensive APIs and administration capabilities. It is easy to integrate with any web application, existing or new on Heroku. Cloudinary provides URL and HTTP based APIs. Which helps integrates with any Web development framework. As opposed to setting up a separate Cloudinary account. By the end of this article, users will upload pictures over your rails app on Heroku. On the back-end, images uploads to or deleted from Cloudinary. Your rails app displays images hosted on Cloudinary via API exchange. The best part is that Cloudinary will not ask for your credit card to sign up for 10GB free storage. Author’s Assumptions The following assumptions are being made: You are using Heroku to host your rails app
. Heroku CLI is installed on your machine
. Your rails app have been pushed to Heroku. Cloudinary local and cloud setup Edit your Gemfile, add the following line. gem ‘carrierwave’
gem ‘cloudinary’ # ./gemfile Run "run bundle install" to install inserted gems. Login Herokuapp page and click on the app as illustrated below; Navigate to "Resources". Find & select Cloudinary in the search box as illustrated below; Select "Start Plan" for free usage and click provision. You are now redirected to the Cloudinary admin page. Notice generated credentials. Copy cloud name, API key, API secret & Environment Variable. To hide API key and secret, while the resides in Github, we use config variables. First, navigate to settings and click on config vars. Input credentials (CL_ID, CL_key, Cloudinary_URL) as illustrated below; Create a cloudinary.yml file in your ‘config’ folder. Copy and paste the template into new file. Substitute Cloud_name for actual cloud name found in Cloudinary admin dashboard. This configuration specifies credentials as well as other attributes. --- cloud_name <%= ENV[ ] %> <%= ENV[ ] %> cloud_name <%= ENV[ ] %> <%= ENV[ ] %> cloud_name <%= ENV[ ] %> <%= ENV[ ] %> # config/cloudinary.yml development: cloud_name: api_key: "CL_ID" api_secret: "CL_KEY" enhance_image_tag: true static_file_support: false production: cloud_name: api_key: "CL_ID" api_secret: "CL_KEY" enhance_image_tag: true static_file_support: true test: cloud_name: api_key: "CL_ID" api_secret: "CL_KEY" enhance_image_tag: true static_file_support: false Create an initializer cloudinary.rb as below. When rails server starts up, configuration below will take affect. Cloudinary.config config.cloud_name = config.api_key = ENV[ ]
  config.api_secret = ENV[ ]
  config.secure = config.cdn_subdomain = do |config| 'cloud_name' "CL_ID" "CL_KEY" true true end Assuming you have built the image upload feature (as shown in Setup uploader for Cloudinary as below; learn enough tutorial). CarrierWave::MiniMagick Cloudinary::CarrierWave version process => [ , , ] Rails.env.production? Cloudinary::CarrierWave
    CarrierWave.configure config.cache_storage = storage # app/uploader/picture.rb < CarrierWave::Uploader:: class PictureUploader Base include include # Specify dimensions for images to resize to for appropriate display fit :standard do :resize_to_fill 400 400 :north end # Cloudinary will be utilized in production (Heroku) while local psql stored us utilized in development and testing environment. if include do |config| :file end else :file end # Cloudinary public file path to upload and display pictures def store_dir "uploads/ / / " #{model. .to_s.underscore} class #{mounted_as} #{model.id} end # Only files with extension jpg jpeg, gif and png will be uploaded def extension_whitelist %w[jpg jpeg gif png] end end For image upload forms, emulate the following erb form on front-end view; <%= %> <%= %> <% %> # app/views/micropost/_form.html.erb form_with( @post, ) model: local: true do |form| .
. < = > span class "picture" form.file_field , :picture accept: 'image/jpeg,image/gif,image/png' .
. </ > span end For image display, emulate the following erb tag on front-end view; <% @feeds.each %>
.
.
<%= image_tag p.picture.url p.picture? %>
.
.
<% %> # app/views/micropost/_feed.html.erb do |p| if end Now that you are done, push rails app to Heroku and test it. Check out my , to see how it was done. Gitbub Repo

Cloudinary Upload API Introduction: Hassle Free Alternative to AWS S3

Imagine that you're a social media intern. Please, bear with me! You're a social media intern and your manager, instead of including you in on strategy meetings, hands you a laundry list of image-related tasks. It includes: Crop . this Brighten . that Overlay "New arrivals!" on the Twitter header image. Resize the Facebook share card. Get me coffee. Translate "New arrivals!" to Russian, German, and Swahili. ...You get the point Now, you're a smart cookie. There's no way you want to spend your day wasting away on the computer having to manipulate all these images. So, you set out to find a better way. MANUALLY ⏸ Story over, for now Sorry, I get carried away. However, that scenario is no made-up story. It's real! Every day, whether at work or for personal projects, millions of images get created, edited, updated, hosted, taken down, lost, and so on. Services that help manage the chaos or streamline the process can be helpful. incredibly The other day, a friend shared with me . Immediately, I saw it as being an answer to so many image-related inefficiencies suffered by companies and people. Very quickly, I'll mention a and a . Cloudinary's URL API why how Why From a single image, dozens of tweaked versions might need to get created (faded, text-overlay, black-white, etc.). Each of those versions takes time to create, update, and organize. How Cloudinary's URL API takes a dynamic parameter that applies transformations to the image upon retrieval. Think of it like on-demand Photoshop! Personally, this got me excited. Notably, the transformation. After spending a little time playing with it, I wanted to see if it could be extended to incorporate localization (translation) of image text. Text Overlay A working demo came out of the exercise. You can play with it , or keep reading and learn how it works! here First off, let's take a quick look at the anatomy of the URL API. A large section of parameters exists between the and sections. These are a list of image transformations that get executed when the image is requested. Pretty cool! Right? The documentation is if you'd like to dive deeper. upload/ /horse.jpg right here https://res.cloudinary.com/demo/image/upload/c_crop,g_face,ar_16:9,w_1200,h_600/e_auto_contrast/b_rgb:00000099,e_gradient_fade,y_-0.4/co_white,fl_relative,l_text:Times_100_bold_italic:I%20am%20a%20unicorn!,w_0.95/co_black,e_shadow,x_2,y_1/fl_layer_apply,g_south_west,x_20,y_25/dpr_auto,q_auto,f_auto/horse.jpg Now, the image you see below gets rendered using the link above. Moreover, - this is the crucial part - if you change the transformation, a brand new image gets returned! The is easily visible when inspecting the URL. While we can't add a custom transformation tags (that is, on Cloudinary's side), we do have the ability to apply transformations to the URL. Meaning that, in the case of localizing our image overlays, we can coerce the URL before requesting the image. l_text:Times_100_bold_italic:I%20am%20a%20unicorn! A serverless GraphQL resolver function can get deployed to an to accomplish this. It can handle the parsing of the URL and translation. There are many ways to deploy a serverless function. However, 8base made it super simple and straight forward. 8base workspace As a quick specification, let's design the function to behave as follows. If a `local_[2-char-lang-code]` tag precedes the text, translate the text, and update the URL. If a local code does not precede the text, return the original URL. Enough talk, let's do it 1. Create a new 8base project Note: If you have an existing projected, you can always add a new function to it. npm install -g 8base-cli 8base init my-project --functions="resolver:localizer" # Install the CLI globally # Initialize a new project with a GraphQL resolver called "localizer." These commands create a new project with all the files and code we need to start invoking our GraphQL resolver function. We'll need to make a few changes though before it's translating our Cloudinary URL's 😉 2. Update the resolver's graphql.schema Open up the file at . We need to define our query operation and response. In this case, we'll be returning an object with the updated after having received the . Update the file with the following snippet. src/resolvers/localizer/schema.graphql url cloudinaryUrl type LocalizeResult { : !
}

extend type Query {
  localize(cloudinaryUrl: !): LocalizeResult
} url String String 3. Update the mock for invoke-local Update so that the function can get invoked locally with data. The mock file generated has the same schema as what gets passed to the function in production. src/resolvers/localizer/mocks/request.json { : { : }, : { : }, : } "data" "cloudinaryUrl" "https://res.cloudinary.com/cdemo/image/upload/c_crop,g_face,ar_16:9,w_1200,h_600/e_auto_contrast/b_rgb:00000099,e_gradient_fade,y_-0.4/co_white,fl_relative,l_text:Times_100_bold_italic:local_es:Breaking%20news:%208base%20solves%20all%20your%20image%20related%20needs!,w_0.95/co_black,e_shadow,x_2,y_1/fl_layer_apply,g_south_west,x_20,y_25/dpr_auto,q_auto,f_auto/dosh1/img-0.jpg" "headers" "x-header-1" "header value" "body" "{\"cloudinaryUrl\":\"https://res.cloudinary.com/cdemo/image/upload/c_crop,g_face,ar_16:9,w_1200,h_600/e_auto_contrast/b_rgb:00000099,e_gradient_fade,y_-0.4/co_white,fl_relative,l_text:Times_100_bold_italic:local_es:Breaking%20news:%208base%20solves%20all%20your%20image%20related%20needs!,w_0.95/co_black,e_shadow,x_2,y_1/fl_layer_apply,g_south_west,x_20,y_25/dpr_auto,q_auto,f_auto/dosh1/img-0.jpg\"}" 4. The function We're going to need a translation engine. I chose AWS Translate, which offers 2-million free characters per month. Let's add the required library and config to the project. npm install --save aws-sdk # Install AWS SDK Update src/resolvers/localizer/handler.ts. AWS  = ( );

AWS.config.update({ : , : { : process.env.AWS_IAM_SECRET_KEY, : process.env.AWS_IAM_ACCESS_KEY 
    }
}); translate = AWS.Translate({ : }); const require 'aws-sdk' region 'us-east-1' credentials accessKeyId secretAccessKey const new apiVersion '2017-07-01' /* Other code ... */ When developing locally, you'll need to set your AWS credentials as environment variables or static values. The example you see above is what works when the function gets deployed to 8base. Here's the documentation on accessing . 8base environment variables Since we're using TypeScript, the function response needs a . This type match the structure and name of that added to the file. For our scenario, prepend the following to the function body. type must graphql.schema type LocalizeResult = {
  data: {
    url: string
  }
}; The function body is pretty self-explanatory. Instead of describing it and then showing it , please read the inline comments for clarification on what's happening. here there (event: any, : any) : <LocalizeResult> => { REG_EX = url = event.data.cloudinaryUrl matchObj = REG_EX.exec(url); (matchObj) { local = matchObj[ ], text  = matchObj[ ]; { request = translate.translateText({ : local.slice( ), : , : (text)
      }).promise(); data = request; url = url.replace( , data.TranslatedText.replace( , ))
    } (err) { .log(err, err.stack);
    }    
  } { : {
      url
    }
  }
}; export default async ctx Promise /**
   * Regex Statement for matching our custom local_tag and preceeding text
   */ const /(local_[a-z]{2})\:(.*?)([,\/])/g /**
   * Pull the given cloudinary url from our function arguments 
   */ let /**
   * Execute our Regex statement returning a match object
   */ const /**
   * If a local tag is matched, we're in business! If not,
   * we're simply returning the passed url.
   */ if /**
     * Pull out the matched local and text values from
     * the matchObj array.
     */ let 1 2 try /**
       * Make the request to AWS Translate after decoding the given text
       * and slicing the last two characters from the local tag (e.g. local_es)
       */ let TargetLanguageCode -2 SourceLanguageCode 'auto' Text decodeURI let await /**
       * The ACTUAL cloudinary url will break if it has our custom tag. Plus, we
       * need to update the text with the translation! So, let's replace the previously
       * matched locale and text with our tranlsated text, that needs to be escaped.
       */ ` : ` ${local} ${text} /[.,%\`\s]/g '%20' catch console /**
   * Return the final result.
   */ return data 5. Run it! Done! Let's prove it by invoking our function locally. The returned URL's text section translates to the locale specified language! Copy the link and throw it in a browser to see the magic. base invoke-local localize -p src/resolvers/localize/mocks/request.json
invoking...

Result:
{ : { : { : }
  }
} 8 "data" "localize" "url" "https://res.cloudinary.com/demo/image/upload/c_crop,g_face,ar_16:9,w_1200,h_600/e_auto_contrast/b_rgb:00000099,e_gradient_fade,y_-0.4/co_white,fl_relative,l_text:Times_100_bold_italic:¡Soy%20un%20unicornio%20genial!,w_0.95/co_black,e_shadow,x_2,y_1/fl_layer_apply,g_south_west,x_20,y_25/dpr_auto,q_auto,f_auto/horse.jpg" 🏁 Wrap up Sorry, we're going back to storytime. Remember back when you were a social media intern? Well, you ended up finding and using Cloudinary for all your on-the-fly image transformation and 8base for lightening fast serverless deployment of serverless GraphQL functions. Excited by the chance to become "Employee of the Month", you approach your boss and share with him the big news by saying: "I was able to apply dynamic URL transformations to our images using a URL API and extend its functionality to support real-time translations of text overlay!" Seemingly confused, your manager looks at your hands and responds: "You forgot my coffee?" Just so you know; Cloudinary and 8base both do A LOT more than what is in this post. I highly recommend you check them out!

Localizing Image Text Overlays using Cloudinary + 8base + AWS Translate

Earlier this year, I decided to leave my job. While exploring new opportunities, I came across and became interested in a company that specializes in digital asset management (DAM), called Cloudinary. Happily, after a few rounds of interviews, both technical and nontechnical, I received an offer and joined the company in June as a full-time developer evangelist. My ramp-up plan was straightforward: Spend two weeks in Cloudinary’s office in Tel Aviv, Israel and meet the team there: engineers, product managers, senior executives. Out of pure luck, on the first day of my visit, Cloudinary hosted an internal , “ ,” in which I participated. It turned out to be a gem of an experience. hackathon Hack Outside the Crop Two things struck me soon after I’d spent some time in the Tel Aviv office. One, everyone was amicable and genuinely curious about who I am and what I know. I hadn’t expected the many hugs on day one but, hey, no complaints from me. Those warm gestures suit my personality and I like that kind of congenial work environment. Two, I was impressed to see the top executives casually walking around the office often, which gives us many opportunities to talk to them. It was obvious that if they are free, they are absolutely accessible. After being introduced and shown around the office on my first day, I joined the . From an organizational perspective, the hackathon was pretty standard, starting with pitching ideas and forming teams. The primary criteria were to create hacks that would augment, expand, or enhance Cloudinary’s core product offerings. hackathon I teamed up with another developer evangelist so that we could also use our time together to swap knowledge. That was a greatly beneficial for me because I was new to the technologies being adopted at Cloudinary and had a lot to learn. Nonetheless, I found it challenging at the beginning to even grasp the fundamentals. Thanks to the generous help of my team member and the engineers, all located in one section of the office, I made remarkable strides in short order. Gaining knowledge had never been so easy! The project that our team worked on was a Vue.js-based video editor that would enable marketers and bloggers to stitch and edit videos online with Cloudinary as the back end. The goal was to create an easy-to-use interface, as opposed to a fat client app, for downsizing videos for web browsers. Even though it was my very first experience with , I found it appealing and have since used it in multiple projects. Vue.js All in all, I much enjoyed the hackathon. It afforded me an invaluable opportunity to — * Meet and work with several Cloudinary teams. * Gain in-depth knowledge from a fellow evangelist. * Learn new technologies, both Vue.js and Cloudinary’s. And I got to eat a lot of delicious food while doing all that. In a matter of only a few hours, all the hackathon teams managed to whip up great hacks. At the end, each team presented its hack and I was amazed at how technical some of the solutions were. Check out all the . projects created by the teams Spa treatment and fun day with a +1. First place: Dinner with a +1 in a chef restaurant. Second place: The winning team built a digital asset management (DAM) terminal. DAM is a new Cloudinary offering that accords marketing, creative, and development teams the freedom to use rich media to drive customer or user engagement across diverse use cases. Conveniently, with the DAM terminal, you can execute file-system text commands and operations in the browser, just like in a UNIX terminal. Conclusion All in all, I found it both challenging and exciting to join Cloudinary at the time of the hackathon. Challenging, because I had to grasp the technical details fast; exciting, because I was thrown into the deep end, so to speak, but I learn best that way. Above all, it was gratifying to learn from all those knowledgeable, seasoned colleagues.

A Cloudinary Newbie’s First Impressions and First Hackathon

Cloudinary Users Score at Canadian Music Week Hackathon Today’s music industry is now a “new” music industry — And it is a pretty exciting place for artists. The many gatekeepers and barriers to breaking into the business have vanished thanks to new which is leveling the playing field, and creating more access for artists to engage directly with their audiences. technology For the recording industry, this change has been strikingly fast and not without its challenges. Artist’s today demand more creative control, and not just for their music, but in almost every aspect of their own brand. The balance of financial rewards has flipped, with many record labels struggling to survive as many superstar artists cut their connections and attempt to connect with fans directly. New technologies including live media streaming, augmented reality and blockchain transactions offer a promise to address some of the major challenges facing the music industry. Discovery Channel:Daily Plant coverage of the Canadian Music Week Hackathon On May 8–9, 2018, the organizers of –the longest running and second-largest music and technology conference in North America–hosted a for the first time. The purpose of the was to challenge coders, developers, artists, designers, and technical specialists to create innovative solutions that address roadblocks the country’s music business faces. Canadian Music Week hackathon CMW Hackathon Despite being the first hackathon for CMW, the event drew heavy hitters to Toronto. It was hosted by Travis Laurendine, who has overseen similar events for SXSW, Outside Lands, Bonnaroo, and the Super Bowl. The judges were a panel of who’s-who of CXO’s, music executives and investors. was a proud sponsor of the event. What made us even prouder was that two of the top three project winners leveraged our feature-rich solution for image management and delivery. Cloudinary prepares for the event Colin Bendell A key theme across many of the projects was a focus on creating a new user experience with music. # First-Place Winner: SATE First place went to , an immersive, augmented reality (AR)-enabled concert application that uses live concert performances and sound to trigger AR experiences for concert goers on their mobile devices. SATE Artist took advantage of the collaborative spirit of the event. SATE What is interesting about the app is that it allows the artists to take control of the experience, express themselves through that video and allows them to control what the viewer sees while they’re taking the recording of your performance. — SATE # Second-Place Winner: Good Kid The second-place winner was Good Kid, a Toronto-based band who created a platform for artists to determine whether their band name, song title, or album name is the best for their genre. To build their own deep-learning algorithm, the team used data scraped from numerous publicly-available music sources and datasets to pinpoint what genres are associated with band names. Team Good Kid The application matches the results of the band-name search with similar artists and with their matching artwork provided by Cloudinary’s Music Discovery Service (MDS) API. That API is part of an example blueprint application we built with 7Digital and Capitol Music Group during Capitol360 to enable hackathon participants unfettered access to the label’s catalog of music. The team consumed the MDS API and then further transformed the album artwork through Cloudinary’s fetching and transformation APIs. For their winning project, Cloudinary awarded Good Kid support service for one year to help bring their platform to market. # Third-Place Winner: Databots Databots won third prize, transforming their love of Canadian death metal into an online universe and a destination for bands to reach a wider audience. According to Mark Shuldiner of the Databots team, when he arrived at the hackathon, he didn’t have a specific idea of what he wanted to do. However, by the time he had to pitch his proposal, he was “sold on the theme of a computer-generated metal musician.” Mark’s group was tasked with handling the music; he focused on creating an accompanying visual. Team Databots As a hackathon mentor, I see many developers hit that proverbial wall when trying different approaches to solve their problem. Mark reached out to me for help: what would be an innovative and programmatic approach to visualizing AI-generated audio? I suggested turning the audio into video and further visualizing it by leveraging Cloudinary’s audio-waveform feature. The experiment paid off. Mark had this to say: “Dan walked me through the initial setup steps and bore with me as I tried out JavaScript for the first time,” adding, “After a lot of support on Dan’s part and my staying up through the night to stitch together the various transformations and to clean up messy code, the final result was a sound transformation of the song slowly panning across the screen with the song title revealed when the sound-transform graphic reaches the foreground of the song name. Talk about visually engaging!” # A Consensus All the participating teams agreed that even though hacking innovative projects at CMW was extremely challenging, the exercise offered them an invaluable opportunity to be just as creative as the artists they showcased in their projects. Everyone relished the experience. By all accounts, our participation at The Canadian Music Week Hackathon was a natural fit for ; As we are a company that’s obsessively focused on optimizing media delivery and performance to ultimately provide the best possible user experiences for everyone. We look forward to participating in future events, helping SATE build out V2 of her app and if you have any fun creative ideas, comment below. Cloudinary

Cloudinary Users Score at Canadian Music Week Hackathon

A DevOps transformation without implementing Infrastructure as Code will remain incomplete: Infrastructure Automation is a pillar of the modern Data Center. With tools like SaltStack and Docker things are becoming easier. This tutorial is part of a . Painless Docker Course If you are interested in discovering our new courses about AWS, and we will be sharing interesting stuff with you ! register using this form Update: After sharing this story to Reddit, I read some comments saying that I am reinventing AWS Lambda or a serverless system in general. I’d like to say yeah ! why not !? After all, the purpose of this course is educational and you can consider it as a reverse hack. Happy Hacking ! Introduction We are going to use a set of tools like SaltStack, Boto3 (Python3), AWS CLI, EC2 in order to create a master and some minions running a Docker container each. This diagram explains a lot of what we will do. Creating the First EC2 Machine Using AWS CLI We will need the ID of the VPC, the ID of the Security Group, the ID of the Subnet, the name of the key pair and the AMI of the OS we are going to use (Ubuntu 16.04). The latter depends on the zone. Describing VPCs Let’s see what we have as VPCs: aws ec2 describe-vpcs You should get a detailed list of your active VPCs: {"Vpcs": [{"DhcpOptionsId": "dopt-xxxxxx","CidrBlock": "172.31.0.0/16","InstanceTenancy": "default","State": "available","IsDefault": true,"VpcId": "vpc-xxxxx"},{"DhcpOptionsId": "dopt-xxxxx","CidrBlock": "172.20.0.0/16","InstanceTenancy": "default","State": "available","Tags": [{"Key": "xxxxx","Value": "xxxxx"},{"Key": "xxxxxx","Value": "xxxxx"},{"Key": "xxxxxx","Value": "xxxxxx"}],"IsDefault": false,"VpcId": "vpc-xxxxxx"}]} Listing the Security Groups aws ec2 describe-security-groups You should get a list: {"IpPermissionsEgress": [{"IpProtocol": "-1","IpRanges": [{"CidrIp": "0.0.0.0/0"}],"UserIdGroupPairs": [ \],  
      "PrefixListIds": \[  
          
      \]  
    }  
  \],  
  "Tags": \[  
    {  
      "Key": "Name",  
      "Value": "xxxxxx"  
    }  
  \],  
  "OwnerId": "xxxxx",  
  "GroupName": "xxxxxxxx",  
  "VpcId": "vpc-xxxxxxx",  
  "Description": "xxxxxx",  
  "IpPermissions": \[  
    {  
      "IpProtocol": "-1",  
      "IpRanges": \[  
        {  
          "CidrIp": "0.0.0.0/0"  
        }  
      \],  
      "UserIdGroupPairs": \[  
          
      \],  
      "PrefixListIds": \[  
          
      \]  
    },  
    {  
      "IpRanges": \[  
        {  
          "CidrIp": "0.0.0.0/0"  
        }  
      \],  
      "ToPort": 22,  
      "UserIdGroupPairs": \[  
          
      \],  
      "PrefixListIds": \[  
          
      \],  
      "IpProtocol": "tcp",  
      "FromPort": 22  
    },  
    {  
      "IpRanges": \[  
        {  
          "CidrIp": "0.0.0.0/0"  
        }  
      \],  
      "ToPort": 2376,  
      "UserIdGroupPairs": \[  
          
      \],  
      "PrefixListIds": \[  
          
      \],  
      "IpProtocol": "tcp",  
      "FromPort": 2376  
    }  
  \],  
  "GroupId": "sg-xxxxxx"  
}      
... Describing Availability Zones & Choosing the AMI: aws ec2 describe-availability-zones Even if we’re not going to use this directly but to find the AMI, we should get this information first: {"AvailabilityZones": [{"Messages": [],"RegionName": "eu-west-1","ZoneName": "eu-west-1a","State": "available"},{"Messages": [],"RegionName": "eu-west-1","ZoneName": "eu-west-1b","State": "available"},{"Messages": [],"RegionName": "eu-west-1","ZoneName": "eu-west-1c","State": "available"}]} Since I am going to use Ubuntu, I used this website: https://cloud-images.ubuntu.com/locator/ec2/ Describing the Subnets aws ec2 describe-subnets And you get a similar list to this: {"AvailableIpAddressCount": 4091,"MapPublicIpOnLaunch": true,"AvailabilityZone": "eu-west-1b","VpcId": "vpc-xxxxxx","State": "available","DefaultForAz": true,"CidrBlock": "172.31.0.0/20","Tags": [{"Value": "xxxxxxxxxx","Key": "Name"}],"SubnetId": "subnet-xxxxxxxxxx"} Getting the Key Pairs aws ec2 describe-key-pairs And it will show the list of existent key pairs: {"KeyPairs": [{"KeyFingerprint": "xxxxxxxxxxx","KeyName": "xxxxxxxx"},{"KeyFingerprint": "xxxxxxxxx","KeyName": "xxxxxxxx"}]} Let’s Create the Machine aws ec2 run-instances --image-id ami-785db401 --count 1 --instance-type t2.micro --key-name .xxxxx --security-group-ids sg-xxxxx --subnet-id subnet-xxxxx --associate-public-ip-address --query 'Instances[0].InstanceId' --output text This should show us the id of the instance: i-0ed688fc95b1feeb2 Get the Public DNS In order to connect using SSH, we should get this ! aws ec2 describe-instances --instance-ids i-0ed688fc95b1feeb2 --query 'Reservations[0].Instances[0].PublicDnsName' --output text And the output was: ec2-34-253-201-12.eu-west-1.compute.amazonaws.com Let’s Try the Public IP ! Same as DNS, type: aws ec2 describe-instances --instance-ids i-0ed688fc95b1feeb2 --query 'Reservations[0].Instances[0].PublicIpAddress' --output text Output: 34.253.201.12 I Need the Private IP In my case I am going to connect the Salt Minion to the Salt Master using the private IP, you can do the same if you will create both machines in the same VPC. If you want to use the private IP : aws ec2 describe-instances --instance-ids i-0ed688fc95b1feeb2 --query 'Reservations[0].Instances[0].PrivateIpAddress' --output text Output: 172.31.4.106 Installing the Master Using the public DNS and your Key Pair connect to the created machine: apt install salt-master Creating & Provisioning Multiple Minion Machines Using Boto3 & Cloudinit To use Boto3 (Python), you can create a Python virtual environment if you don’t want to install it to your machine. This is what I do in all cases. virtualenv -p python3 learning cd learning/. bin/activatepip install boto3mkdir app && cd app Now let’s create the Python script: import boto3import sys AWS_ACCESS_ID = "xxxxxxxxxx"AWS_SECRET_KEY = "x/xxxxxx"ImageId = "ami-785db401"KeyName = ".xx"InstanceType = "t2.micro" MinCount = 1MaxCount = 1SubnetId = "subnet-xxxxx"SecurityGroupIds = ["sg-xxxxxx"]UserData = open('init.sh') conn = boto3.client('ec2',aws_access_key_id=AWS_ACCESS_ID,aws_secret_access_key=AWS_SECRET_KEY) for i in range( int(sys.argv[1]) ):reservation = conn.run_instances(ImageId=ImageId,KeyName=KeyName,InstanceType=InstanceType,MinCount=MinCount,MaxCount=MaxCount,SecurityGroupIds=SecurityGroupIds,SubnetId = SubnetId,UserData=UserData.read()) This is the user data file: #!/bin/bash # exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1echo BEGIN https://alestic.com/2010/12/ec2-user-data-output/ sudo apt-get update -ysudo apt-get upgrade -y # Install saltstacksudo apt-get install salt-minion -y # Set salt master location and start minionsudo sed -i 's/#master: salt/master: 34.253.201.12/g' /etc/salt/minionsudo salt-minion -d # Install Dockercurl -fsSL | sudo apt-key add - https://download.docker.com/linux/ubuntu/gpg sudo add-apt-repository \"deb [arch=amd64] \$(lsb_release -cs) \stable" -y https://download.docker.com/linux/ubuntu sudo apt-get update -y || :sudo apt-get install docker-ce -y sudo apt install python-pip -y && sudo pip install docker-py -y echo END It will provision the machine on its startup by installing Salt Minion and Docker. Note in the script above that we already set the server of the minion to the private IP address of the master. sudo sed -i 's/#master: salt/master: 34.253.201.12/g' /etc/salt/minion To create 5 machines, just type: python <script_name> 5 To run 1000 machines, just type: python <script_name> NO WAY :) Now we have 5 machines where Salt Minion and Docker are installed and a a Salt Master. Connecting the Minions to the Master On the Master type: salt-key -L and you’ll get the Minions waiting to be accepted: Accepted Keys:Denied Keys:Unaccepted Keys:ip-172-31-5-169.eu-west-1.compute.internalip-172-31-5-168.eu-west-1.compute.internalip-172-31-5-167.eu-west-1.compute.internalip-172-31-5-166.eu-west-1.compute.internalip-172-31-5-165.eu-west-1.compute.internalip-172-31-5-164.eu-west-1.compute.internal..etc Rejected Keys: Accept all using: salt-key -A then uncomment the following part from /etc/salt/master: file_roots:base:- /srv/salt We will have a file server for Salt states and Salt high states in the location mentioned above. Now restart the Master: pkill salt-mastersalt-master -d In order to install Docker containers on the Minions, you should not forget this: salt * pip.install docker-py>=1.4.0 This command will install a superior version to 1.4.0 of Python docker package. Now create two files, the Top file In Salt, the file which contains a mapping between groups of machines on a network and the configuration roles that should be applied to them is called a . top file And the SLS file: The core of the Salt State system is the SLS, or a t tate file. The SLS is a representation of the state in which a system should be in, and is set up to contain this data in a simple format. This is often called configuration management. S L S (both quotes are taken from the official Salt documentations) Here is the structure of file tree: tree /srv/salt/ /srv/salt/├── top.sls└── webserver.sls Starting Docker Containers We are going to run multiple Nginx containers. Nginx is just an example here, you can run whatever image you want. Let’s modify the content of the webserver.sls and tell the Master to run Nginx: my_service:dockerng.running:- name: nginx- image: nginx Now let’s modify the content of the Top file and tell the Master to apply the state to the servers having Ubuntu as an OS. This is a targeting technique, you can choose any other targeting pattern. (read this for more information: ) https://docs.saltstack.com/en/latest/topics/targeting/ #top.sls file base:'os:Ubuntu':- match: grain- webserver In order to start the containers, type: salt '*' state.apply and you will get the execution logs: [...]ip-172-31-5-169.eu-west-1.compute.internal:----------ID: my_serviceFunction: dockerng.runningName: nginxResult: TrueComment: Container 'nginx' is already configured as specifiedStarted: 01:03:40.062033Duration: 28.66 msChanges: Summary for ip-172-31-5-169.eu-west-1.compute.internal------------Succeeded: 5Failed:    0------------Total states run:     5 This Combination of Salt, Docker, Boto3 and AWS could be used in different automation scenarios and even for self-healing and auto-scalable infrastructures. You can find similar tutorials in my . Painless Docker Course That’s all folks ! Connect Deeper If you resonated with this article, you can find more interesting contents in . Painless Docker Course If you liked this course, subscribe to one or more of our newsletters: : The DevOps Ecosystem is Growing Fast. Each Week, We Hand-curate Must-read News, Tutorials, Reviews and More ! DevOpsLinks : An Independent Newsletter Focused On Serverless, Containers, FaaS & Other Interesting Stuff Shipped : A #Kubernetes Community Hub, Hand Curated Newsletter, Team Chat, Training & More Kaptain You can find me on , or my and you can also check my books: . Twitter Clarity website SaltStack For DevOps Don’t forget to join our jobboard ! Jobs For DevOps If you liked this post, please recommend it and share it with your followers.

Creating an Army of Docker Containers using SaltStack, Boto3 & CloudInit on AWS

Web Image Optimization: Best Practices for Speed and SEO in 2024

Build a Dynamic Photo Collection App With Django

Python Security Essentials for Your Apps

Let's explore how temporary tables work in MariaDB. First, we have to connect to the server. For example (use your own connection details): mariadb -h 127.0.0.1 -u root -p"RootPassword!" --database demo Now, just to point something out, let's create a standard (permanent) table. Here's how: CREATE TABLE t (
    c INT
); This table, t, will persist in the database even after we exit the client: exit When we reconnect and check the existing tables using SHOW TABLES;, the table t will still be listed: mariadb -h 127.0.0.1 -u root -p"RootPassword!" --database demo SHOW TABLES; +----------------+
| Tables_in_demo |
+----------------+
| t              |
+----------------+ All this is pretty obvious, but now, let's recreate this table and try something different: CREATE OR REPLACE TEMPORARY TABLE t (
    c INT
); Notice the TEMPORARY keyword. After creating this table, if we run SHOW TABLES;, it appears in the list. We can insert data into it, query it, and join it with other tables. It behaves like a normal table during the current session. However, if we exit the client, then reconnect, and perform a SHOW TABLES; again, the temporary table t will not be listed. A temporary table only exists for the duration of the session in which it was created and other sessions won't be able to see it. Use Case for Temporary Tables Temporary tables are quite useful for transient data operations. For instance, consider a table called products in our database: CREATE TABLE products (
  id INT NOT NULL AUTO_INCREMENT,
  code VARCHAR(100) NOT NULL,
  name VARCHAR(250) NOT NULL,
  description TEXT DEFAULT NULL,
  PRIMARY KEY (id),
  UNIQUE KEY code (code)
) We can create a temporary table that mimics the structure of products: CREATE TEMPORARY TABLE t LIKE products; We can confirm this by running: DESCRIBE t; +-------------+--------------+------+-----+---------+----------------+
| Field       | Type         | Null | Key | Default | Extra          |
+-------------+--------------+------+-----+---------+----------------+
| id          | int(11)      | NO   | PRI | NULL    | auto_increment |
| code        | varchar(100) | NO   | UNI | NULL    |                |
| name        | varchar(250) | NO   |     | NULL    |                |
| description | text         | YES  |     | NULL    |                |
+-------------+--------------+------+-----+---------+----------------+ Initially, t will be empty. However, suppose we want to transfer some data from products to t. Let’s assume we only want to include products that contain the number 0 in their code: INSERT INTO t
SELECT * FROM products
WHERE code LIKE '%0%'; After running this command, if we query the temporary table t: SELECT * FROM t; +----+--------+------------------+---------------------------------------------------+
| id | code   | name             | description                                       |
+----+--------+------------------+---------------------------------------------------+
|  1 | BG2024 | BugBlaster       | Eradicates software bugs with a single scan.      |
|  3 | FW001  | FireWhale        | An oversized, comprehensive firewall solution.    |
|  4 | CLD404 | CloudNine Finder | Find your way back from cloud outages and errors. |
+----+--------+------------------+---------------------------------------------------+ We see the filtered data. Conclusion Temporary tables offer a powerful way to handle data for temporary processing without affecting the persistent data store. They are particularly useful in scenarios where data needs to be manipulated or transformed temporarily. You can use permanent tables for this kind of data manipulation but temporary tables are useful when you need automatic cleanup, reduced risk of naming conflicts, isolation and security, and resource management for query performance. Let's explore how temporary tables work in MariaDB. First, we have to connect to the server. For example (use your own connection details): temporary tables mariadb -h 127.0.0.1 -u root -p"RootPassword!" --database demo mariadb -h 127.0.0.1 -u root -p"RootPassword!" --database demo Now, just to point something out, let's create a standard (permanent) table. Here's how: CREATE TABLE t (
    c INT
); CREATE TABLE t (
    c INT
); This table, t , will persist in the database even after we exit the client: t exit exit When we reconnect and check the existing tables using SHOW TABLES; , the table t will still be listed: SHOW TABLES; t mariadb -h 127.0.0.1 -u root -p"RootPassword!" --database demo mariadb -h 127.0.0.1 -u root -p"RootPassword!" --database demo SHOW TABLES; SHOW TABLES; +----------------+
| Tables_in_demo |
+----------------+
| t              |
+----------------+ +----------------+
| Tables_in_demo |
+----------------+
| t              |
+----------------+ All this is pretty obvious, but now, let's recreate this table and try something different: CREATE OR REPLACE TEMPORARY TABLE t (
    c INT
); CREATE OR REPLACE TEMPORARY TABLE t (
    c INT
); Notice the TEMPORARY keyword. After creating this table, if we run SHOW TABLES; , it appears in the list. We can insert data into it, query it, and join it with other tables. It behaves like a normal table during the current session. However, if we exit the client, then reconnect, and perform a SHOW TABLES; again, the temporary table t will not be listed. TEMPORARY SHOW TABLES; SHOW TABLES; t A temporary table only exists for the duration of the session in which it was created and other sessions won't be able to see it. Use Case for Temporary Tables Temporary tables are quite useful for transient data operations. For instance, consider a table called products in our database: products CREATE TABLE products (
  id INT NOT NULL AUTO_INCREMENT,
  code VARCHAR(100) NOT NULL,
  name VARCHAR(250) NOT NULL,
  description TEXT DEFAULT NULL,
  PRIMARY KEY (id),
  UNIQUE KEY code (code)
) CREATE TABLE products (
  id INT NOT NULL AUTO_INCREMENT,
  code VARCHAR(100) NOT NULL,
  name VARCHAR(250) NOT NULL,
  description TEXT DEFAULT NULL,
  PRIMARY KEY (id),
  UNIQUE KEY code (code)
) We can create a temporary table that mimics the structure of products : products CREATE TEMPORARY TABLE t LIKE products; CREATE TEMPORARY TABLE t LIKE products; We can confirm this by running: DESCRIBE t; DESCRIBE t; +-------------+--------------+------+-----+---------+----------------+
| Field       | Type         | Null | Key | Default | Extra          |
+-------------+--------------+------+-----+---------+----------------+
| id          | int(11)      | NO   | PRI | NULL    | auto_increment |
| code        | varchar(100) | NO   | UNI | NULL    |                |
| name        | varchar(250) | NO   |     | NULL    |                |
| description | text         | YES  |     | NULL    |                |
+-------------+--------------+------+-----+---------+----------------+ +-------------+--------------+------+-----+---------+----------------+
| Field       | Type         | Null | Key | Default | Extra          |
+-------------+--------------+------+-----+---------+----------------+
| id          | int(11)      | NO   | PRI | NULL    | auto_increment |
| code        | varchar(100) | NO   | UNI | NULL    |                |
| name        | varchar(250) | NO   |     | NULL    |                |
| description | text         | YES  |     | NULL    |                |
+-------------+--------------+------+-----+---------+----------------+ Initially, t will be empty. However, suppose we want to transfer some data from products to t . Let’s assume we only want to include products that contain the number 0 in their code: t products t INSERT INTO t
SELECT * FROM products
WHERE code LIKE '%0%'; INSERT INTO t
SELECT * FROM products
WHERE code LIKE '%0%'; After running this command, if we query the temporary table t : t SELECT * FROM t; SELECT * FROM t; +----+--------+------------------+---------------------------------------------------+
| id | code   | name             | description                                       |
+----+--------+------------------+---------------------------------------------------+
|  1 | BG2024 | BugBlaster       | Eradicates software bugs with a single scan.      |
|  3 | FW001  | FireWhale        | An oversized, comprehensive firewall solution.    |
|  4 | CLD404 | CloudNine Finder | Find your way back from cloud outages and errors. |
+----+--------+------------------+---------------------------------------------------+ +----+--------+------------------+---------------------------------------------------+
| id | code   | name             | description                                       |
+----+--------+------------------+---------------------------------------------------+
|  1 | BG2024 | BugBlaster       | Eradicates software bugs with a single scan.      |
|  3 | FW001  | FireWhale        | An oversized, comprehensive firewall solution.    |
|  4 | CLD404 | CloudNine Finder | Find your way back from cloud outages and errors. |
+----+--------+------------------+---------------------------------------------------+ We see the filtered data. Conclusion Temporary tables offer a powerful way to handle data for temporary processing without affecting the persistent data store. They are particularly useful in scenarios where data needs to be manipulated or transformed temporarily. You can use permanent tables for this kind of data manipulation but temporary tables are useful when you need automatic cleanup, reduced risk of naming conflicts, isolation and security, and resource management for query performance.

How to Use Temporary Tables in MariaDB

মারিয়াডিবিতে অস্থায়ী টেবিলগুলি কীভাবে ব্যবহার করবেন

So verwenden Sie temporäre Tabellen in MariaDB

Cómo utilizar tablas temporales en MariaDB

Comment utiliser des tables temporaires dans MariaDB

मारियाडीबी में अस्थायी तालिकाओं का उपयोग कैसे करें

MariaDB で一時テーブルを使用する方法

MariaDB에서 임시 테이블을 사용하는 방법

Como usar tabelas temporárias no MariaDB

Как использовать временные таблицы в MariaDB

MariaDB'de Geçici Tablolar Nasıl Kullanılır?

Cách sử dụng bảng tạm thời trong MariaDB

如何在 MariaDB 中使用临时表

Dear reader, please note that this article was written anonymously, as I am a person deep inside the Enterprise Linux ecosystem. I know "anonymous" is a big word, and if anyone wants to identify the author, they can. I'm against changes to RHEL code redistribution, and I believe that Red Hat is as much a "freeloader" on open source as any other company is on the RHEL codebase. However, I would like to share reasons why I will never install Rocky Linux on my servers. ever In this article, I would like to share some of the stories, misdeeds, and bad practices that explain why so many people do not like Rocky Linux. I also want to show that things are a bit more complicated than most Linux users think. Story 0: The original founder of CentOS The central figure in the case against Rocky Linux is Gregory Kurtzer. He is, for many people, the hero of the RHEL rebuild world. Many believe he is the father of CentOS, and Rocky is/will remain his beloved and, more importantly, free child. The term "The original founder of CentOS" comes from Kurtzer himself and has been spread like wildfire by PR companies (more on illegal, in most countries, and shady PR practices later). The only problem with this statement is that, in the opinion of many, including myself, it is not true. My doubts about Rocky Linux came exactly from this point, as one of my much older colleagues said that this was "not entirely true" (his exact expression was a “little” stronger). I would like to present some basic but important facts: Rebuilds started on the RHEL rebuilds mailing list. The name CentOS was announced by Rocky McGaugh. The Kurtzer was not interested in building a Red Hat Enterprise Linux (then Red Hat Linux) clone. The idea of creating a true distribution from RHEL was not his and had been discussed much earlier. Well, you can read a bit more on HN. And it's written by quite a famous person in the Linux world :) So why does Gregory Kurtzer claim to be the "original founder of CentOS?" Mainly because he started the Chaos Foundation, which took over the efforts to rebuild under a common name. [1] https://www.mail-archive.com/redhat-devel-list@redhat.com/msg07038.html [2] https://web.archive.org/web/20040630213827/http://www.caosity.org/pipermail/caos/2003-December/001205.html [3] - Gregory Kurtzer was not interested in building a RHEL clone https://web.archive.org/web/20040824032738/http://www.caosity.org/pipermail/caos/2003-July/000701.html [4] - Idea of creating Distribution on it's own https://www.mail-archive.com/rhel-rebuild-l@uibk.ac.at/msg00022.html [5] https://news.ycombinator.com/item?id=33907452 Story 1: The Departure One of the main misconceptions about Gregory is that he left CentOS when, in fact, CentOS left his foundation. The people leading CentOS development didn't want to be part of Gregory's scheme. To this day, some of the "old guard" of CentOS are not "on the best of terms" with him. Fifteen years later, the CIQ CEO is much smarter - but unfortunately for the open-source community, in a very bad way. So, when articles like "What happened to CentOS will not happen to Rocky Linux” make the rounds, I think not only about bad changes to the lifecycle, but also about the fact that CentOS was able to leave the Gregory Foundation, but Rocky Linux cannot. Generally speaking, the person in question himself has a few versions of it; my favorite was presented by himself in the Changelog podcast: I was associated with it until Red Hat sued me. And then he said: It wasn’t a lawsuit, it was a threaten of a lawsuit, which was enough in my book to do it. [1] https://lists.centos.org/pipermail/centos-devel/2005-March/077502.html [2] https://www.reddit.com/r/CentOS/comments/s77p49/comment/ht9v86o/ [3] https://changelog.com/podcast/427#transcript-87 Story 2: Foundation You may know that Rocky Linux is part of RESF—Rocky Enterprise Software Foundation. But if you are familiar with "foundations," especially US-based "foundations,” you know that many of them are broken by design. The Rocky Linux Foundation loosely promised to be a non-profit. But it isn't. It's for-profit and it has a single owner (Gregory Kurtzer). But there is more to the smoke and mirrors surrounding RESF. As you may know, they have governance and bylaws. So look at them and count how many people in that governance are not employed by or connected to the CIQ. By the way, the foundation broke its own statutes by not having a diverse governance :). I'm not going to read the bylaws again to see if they still do that, but if you find it, feel free to comment. Also, remember to read older bylaws and older about pages since it’s changed. The whole story is strong in media and PR. Take, for example, the ITWorld Canada article from 2022 In literally the same paragraph, you have the following sentences: The key for an open source initiative to grow and flourish,
Kurtzer said, is to is to register it as a non-profit organisation,
which is what the cAos Foundation. He did the same with Rocky Linux. And then: The Rocky Enterprise Software Foundation (RESF) is a Public Benefit Corporation (PBC). Do you see the "little" problem here? I do not like to be lied to. And I do not like people being lied to, especially when it is such a primitive lie, literally in the same paragraph. [1] Look at the manipulative language used in this article https://www.zdnet.com/article/rocky-linux-foundation-launches/ [2] https://www.resf.org/about [3] https://www.reddit.com/r/linuxadmin/comments/15p1gbt/comment/jvyat0h [4] https://www.itworldcanada.com/article/what-happened-with-centos-will-not-happen-with-rocky-linux-kurtzer/501239 Story 3: Trademark ownership As I said, have you ever wondered why CentOS could leave the Greg Foundation? Well, one of the main reasons was that there was no clear ownership of trademarks/domains. Unfortunately, for the open-source community, that's not the case with RESF and Rocky Linux. As I said before, RESF has a single owner, and the Rocky Linux project does not have its trademarks. Rocky Linux, as an individual project, if it tries to leave RESF, can't use its name, logo, and other trademarks without permission from the RESF owner. So, good luck. So with CIQ/RESF milking Rocky Linux, there is no way the project can become independent. It's not a community project, it's a corporate project. From this point on, I will use arguments against CIQ/RESF/RL interchangeably. [1] https://www.zdnet.com/article/goodbye-centos-hello-rocky-linux/ [2] https://www.reddit.com/r/AlmaLinux/comments/15dr141/giving_rebuilders_a_bad_name_ciq_and_ansible/ [3] https://www.reddit.com/r/linuxadmin/comments/15p1gbt/comment/jw90zhc Story 4: Buying advertising against community-owned open-source projects These are the early days, and RL is losing to AlmaLinux. AlmaLinux is the first diversely governed foundation (and it is non-profit for real). So, what does CIQ do? They buy ads against several Linux distros, including AlmaLinux and CentOS. Is this legal? Yes, it is! Is this something that a "self-imposed non-profit" (this term is a joke) would benefit from? YES! Is this what the open-source community would do? NO! Buying ads against community-owned non-profit open-source projects while pretending to make one is against the general spirit of open source. Of course, Rocky Linux did not buy these ads; CIQ did, and they did it out of the goodness of their hearts. [1] https://twitter.com/ChrisLAS/status/1460668344156114944 [2] https://twitter.com/ChrisLAS/status/1420123460982894599 [3] https://www.reddit.com/r/linux/comments/qv6mg2/comment/hkw046b/ Story 5: PR without disclosure In many cases, the CIQ/RESF/RL use PR firms to spread the word about themselves. Sometimes, they even sponsor Linux influencers, and they tend to leave out some facts :) (like CentOS leaving the former Kurtzer Foundation or RESF being a for-profit company owned by one person). Anyway, some PR tends not to mention the obvious conflict of interest and that the article is sponsored. And since it's very difficult, if not impossible, to verify the source of the PR, it's very hard to say whether the article is sponsored or not if it's not labeled. In my country and many EU countries, it's legal. Even in the US, it's not a legal problem; it shows poor ethics and morals of the: Writer PR company Publisher The company that sponsored it As I said, it's extremely difficult to catch someone red-handed. However, in the case of RL and CIQ, it has become quite well-known in the Linux community. [1] https://twitter.com/GordonMessmer/status/1675997483573612546 Story 6: Spreading FUD One of the biggest damages done to CentOS Stream has been the FUD spread by CIQ and others. The main point is that CentOS Stream is unstable and will never be production-ready. As someone who regularly compares packages and distributions, I can say this: CentOS Stream is stable and, in many ways, even better than RHEL, and that means it is even better than clones. The number of engineers working on CentOS Stream is greater than that of any other RHEL clone. The amount of testing done on CentOS Stream is orders of magnitude greater than any other RHEL clone. The CentOS Stream gives users real power to fix bugs and influence the future of the distribution. There has also been a lot of FUD spread about AlmaLinux. That AlmaLinux has taken "shortcuts,” especially when it comes to the subscription manager. If you're familiar with Subscription Manager, you know that it's designed to connect to Red Hat infrastructure, and Enterprise Linux clones don't install it by default. So, not including it in AlmaLinux is not a bad shortcut but a good decision. Later, it was added because it might also work with Foreman/Katello/Candlepin. The whole "AlmaLinux takes shortcuts" era was FUD in its purest form. Unfortunately, I can't find the source of this ATM, but I've seen it, and I'm sure one of the readers will be able to add it in the comments section. Other popular FUD spread about AlmaLinux, although I did not find CIQ employees saying it publicly: AlmaLinux uses CloudLinux infrastructure (they used it during the bootstrap phase, and it was very well-known) AlmaLinux is controlled by Russians (this one goes strong after Putler attacks Ukraine). CloudLinux controls AlmaLinux (not that CloudLinux CEO literally stepped down from the foundation; also, AlmaLinux has a much healthier board than RESF) Story 7: Being fu****g hypocrites So, you remember all that Red Hat drama about paywalled GPL sources? When Red Hat a "cancer" on open source? And users were "freeloaders"? One of the people who was very vocal about this was Gregory Kurtzer :). No matter what all the shouting was about: "Rocky going strong." "Protect the community." "Protect open source." Or nice PR from RESF: “I believe that open source should always be freely available and completely stable.
It should never be hidden behind a paywall, nor should it be controlled by a single company,”
states Gregory Kurtzer, founder of the Rocky Linux project and chair of the board of the
Rocky Enterprise Software Foundation, What's worse, they are doing the same or even more restrictive things than Red Hat. And other big words that are not worth mentioning because when it comes to real action, they are not there. Now, CIQ is giving back to the community by putting a paywall around the sources of CIQ Rocky Linux 8.8: Restrictions. The license granted in this Section 3 is conditioned upon
Customer’s and its Authorized Users’ compliance with this Agreement. Customer shall
not and shall ensure its Authorized Users do not: (i) permit any third party to use or
access the Software (except for the Authorized Users as permitted herein); (ii) install the
Software on more than the number of Licensed Hosts permitted under the applicable
Order; (iii) share access to the Software (including log in information or notifications)
with anyone who is not intended to be an Authorized User; (iv) provide, license,
sublicense, sell, resell, rent, lease, share, lend, or otherwise transfer or make available
the Software to any third parties, except as expressly permitted by Ctrl IQ in writing; (v)
except with respect to any access to Software that is licensed under an open source
license, modify, copy or create derivative works based on any content accessed through
the Software; (vi) except with respect to any Software that is licensed under an open
source license, disassemble, reverse engineer, decompile or otherwise seek access to
the source code of the Software; (vii) access the Software in order to build a competitive
product or services; (viii) remove, delete, alter, or obscure any copyright, trademark,
patent, or other notice of intellectual property or documentation, including any copy
thereof; (ix) transmit unlawful, infringing or harmful data or code to or from; or (x)
otherwise use the Software except as expressly permitted hereunder There is more. They made rebuilds of other RH products Red Hat Ansible Tower (or whatever they call it, ATM). They made it closed source. Then, after the community started noticing that it should be open source, after two weeks they decided to release code. So that's it when it comes to open-source heroes and giving back to the community. [1] https://www.reddit.com/r/AlmaLinux/comments/1aurwrr/comment/kri9xpc/ [2] https://www.reddit.com/r/AlmaLinux/comments/15dr141/comment/ju3t9mv/ Story 8: Borrowed content - erratas So the CIQ/RESF/RL use Red Hat content, that's obvious. However, you need to remember that components have different licenses. So, if you are using the source code, the most important and, in many cases, only license is the one that is in the source code. If you are using Red Hat Portal or Support, the new restrictive EULA applies. If you use Red Hat errata, the errata text and content are protected. That's why some vendors have rather empty errata information, including one generated for CentOS. The most important parts of erratas are: List of packages CVEs numbers (for security erratas) The text is not that useful; you should probably read the CVE anyway, but some projects cannot help themselves and cannot be used without any consideration. Have a look at where you can read about it: https://errata.rockylinux.org/RLSA-2023:6818 Rocky Enterprise Software Foundation Satellite is a systems management tool for Linux-based systems. You can also find other examples, such as: Rocky Enterprise Software Foundation Identity Management (IdM) or Rocky Enterprise Software Foundation Entitlement Platform. Is it legal? I do not know. Ask a lawyer. Is it ethical? No. Story 9: Killing Open Source - CIQ versus Sylabs In late August 2023, news broke of a lawsuit against CIQ. Note that if I'm right, it's still ongoing. But for the moment, the court "DISMISSES all of Sylabs' claims WITH LEAVE TO AMEND." I'm not a lawyer, but I am an engineer. An open source engineer who's heard "we won't open it because it might hurt our business" or "it makes us vulnerable" time and time again. And I never liked that, but as I got older and saw people like Gregory Kurtzer, I started to understand. What Gregory did is the worst-case scenario for any company trying to build open-source projects. A company employs Kurtzer as CTO. This company makes software vital to its survival and expansion, just like any software company. Gregory was CTO when the company decided to open source its critical assets. (March 2020). Gregory leaves the company with extensive knowledge of customers, system internals, users, and contacts. As a C-level executive, he literally knows it all. Kurtzer started a new company, took part in the development team, and used it against the former company. Kurtzer profits from the old company code that was open-sourced during his time as CTO. By pulling this kind of stunt, one is killing open source—killing trust in open source companies and developers. You work with people you trust, and you never expect them to be your enemy the next day. It leaves a bad taste in your mouth that's hard to describe. Finally, I do not care about the future of Sylab, CIQ, or who is legally right. I care about the future of open source, and this kind of practice is a mortal wound to open source and free software. [1] https://www.theregister.com/2023/08/24/lawsuit_claims_ciq_was_founded/ [2] https://storage.courtlistener.com/recap/gov.uscourts.cand.408847/gov.uscourts.cand.408847.59.0.pdf [3] https://www.courtlistener.com/docket/66863695/sylabs-inc-v-rose/ Other stories There are other stories, like why EPEL stopped doing count_me analyses for some time and why nobody takes them seriously anymore. Or about some key people officially working in public institutions and de facto doing work for a for-profit foundation. But I think the most important ones are covered. Some of these stories are only known to a small circle of people who go to the conference and know each other. Personally, I have left out some of them because it would literally say who I am or where I work. Summary When Red Hat "bought" the CentOS project, there were many red flags. We all know how it ended. Red Hat was then, and for many still is, the champion of open source. RESF/CIQ/RL was never there and never will be with its current smelly track record. And the sheer volume of red flags regarding them is by far more than it ever was with Red Hat. If you read this and install Rocky Linux, you are not only a fool but a dangerous fool. You are the fool who likes to be lied to and is incapable of understanding the bigger picture. I also omitted the technical part of the CIQ/RESF/RL. Perhaps I will return to this to this topic. In the next article, I will cover the joke called OpenELA.

Stories, News & Mentions about CLOUDBIND

Cloudbind’s Stories on HackerNoon