8,336 reads

Understanding Basic Image Processing Algorithms [A Hands-on JavaScript Tutorial]

by Luiz RosaFebruary 26th, 2020

Too Long; Didn't Read

Instagram filters are simple and simple to build your own without any external library, just pure and simple HTML + JS. OpenInsta is a simple HTML and JS tool that lets you create your own image processing engine. It will connect our image to our canvas, convert it into a flat flat array (JPEGEG, PNG, BMP, PNG) and convert it to a flat array. When our user selects an image, we attach the file as the source of a JavaScript node. This will automatically start loading the image and when it is done, when it's done, we draw its contents to the user.

Companies Mentioned

featured image - Understanding Basic Image Processing Algorithms [A Hands-on JavaScript Tutorial]

We've had a lot of digital image processing tools for a long time: Photoshop, Lightroom, GIMP, PhotoScape, and many more. However, in the past few years, one became popular among non-expert users due to its easiness of use and social features: Instagram. Have you ever wondered how Instagram filters work? It is actually pretty simple matrix operations! So simple we can build our own without any external library, just pure and simple HTML + JS. Let's build one now.

First step is to create our HTML file, it contains only 35 lines. Let's call it "OpenInsta", you can check the final source code at my GitHub.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>OpenInsta</title>
</head>
<body>
<h1>OpenInsta</h1>
<input type="file" accept="image/*" id="fileinput" />
<div>
    <label for="red">Red</label>
    <input type="range" min="-255" max="255" value="0" id="red">

    <label for="green">Green</label>
    <input type="range" min="-255" max="255" value="0" id="green">


    <label for="blue">Blue</label>
    <input type="range" min="-255" max="255" value="0" id="blue">

    <label for="brightness">Brightness</label>
    <input type="range" min="-255" max="255" value="0" id="brightness">

    <label for="contrast">Constrast</label>
    <input type="range" min="-255" max="255" value="0" id="contrast">

    <label for="grayscale">Grayscale</label>
    <input type="checkbox" id="grayscale">

    <br/>
    <canvas id="canvas" width="0" height="0"></canvas>
</div>
<script src="main.js"></script>
</body>
</html>

We have one file input, one range controller for each color channel (red, blue and green) plus brightness and contrast range controllers, and finally a grayscale checkbox. We also have a canvas to draw our image while we process it. At the bottom we include a

main.js

JavaScript file which will contain our image processing engine that we will build now.

If you open the HTML file on the browser, that's how it should look like

Let's start writing our

main.js

with the file loader. It will connect our file input with our canvas, converting the image in whatever format it is (JPEG, PNG, BMP, etc.) into a flat unidimensional array.

const fileinput = document.getElementById('fileinput')

const canvas = document.getElementById('canvas')

const ctx = canvas.getContext('2d')

const srcImage = new Image

let imgData = null

let originalPixels = null

fileinput.onchange = function (e) {
  if (e.target.files && e.target.files.item(0)) {
    srcImage.src = URL.createObjectURL(e.target.files[0])
  }
}

srcImage.onload = function () {
  canvas.width = srcImage.width
  canvas.height = srcImage.height
  ctx.drawImage(srcImage, 0, 0, srcImage.width, srcImage.height)
  imgData = ctx.getImageData(0, 0, srcImage.width, srcImage.height)
  originalPixels = imgData.data.slice()
}

We assign two events: When our input changes (i.e. when the user selects an image) we attach the file as the source of a JavaScript Image node. This node will automatically start loading the image and, when it is done, we draw its contents to our canvas while we extract the pixels array on a separate variable we will use for processing.

The Image node we are creating is basically the JS representation of a HTML

<img>

tag.

The

imgData

variable we have extracted using

getImageData()

method is an object representation of the image, having only three attributes: width, height and data, which is the unidimensional pixels array. We store the pixels array on a separate variable to use it as a base for our image modifications.

Suppose our image is 2x2, the array would be something like

[128, 255, 0, 255, 186, 182, 200, 255, 186, 255, 255, 255, 127, 60, 20, 128]

, where the eight first values represents the 2 pixels of the first row and the next eight values represents the 2 pixels of the second row, having each pixel four values varying from 0 to 255 representing red, green, blue and alpha channels respectively. We can, therefore, create a method to get the index of a given pixel as follows:

function getIndex(x, y) {
  return (x + y * srcImage.width) * 4
}

Now we can start building our filters! First of all, let's define a method to perform the transformations and assign it to each range/check input in our HTML page.

const red = document.getElementById('red')
const green = document.getElementById('green')
const blue = document.getElementById('blue')
const brightness = document.getElementById('brightness')
const grayscale = document.getElementById('grayscale')
const contrast = document.getElementById('contrast')

function runPipeline() {
  // Get each input value

  for (let i = 0; i < srcImage.height; i++) {
    for (let j = 0; j < srcImage.width; j++) {
      // Apply grayscale to pixel (j, i) if checked
      // Apply brightness to pixel (j, i) according to selected value
      // Apply contrast to pixel (j, i) according to selected value
      // Add red to pixel (j, i) according to selected value
      // Add green to pixel (j, i) according to selected value
      // Add blue to pixel (j, i) according to selected value
    }
  }
  
  // Draw updated image
}

red.onchange = runPipeline
green.onchange = runPipeline
blue.onchange = runPipeline
brightness.onchange = runPipeline
grayscale.onchange = runPipeline
contrast.onchange = runPipeline

Now every time red, green, blue, brightness, grayscale and contrast input changes, our pipeline will run applying our filters and displaying the result.

Red, Green and Blue

We start with the simplest filters. Going from -255 to 255, the user selects a value to be added on a given channel on every pixel. So, for example, if the pixel's red is currently 128 and the user selects 50 on the input we will have a pixel with red 178. But what happens if the pixel is currently 230? We cannot have a red with value 280, so we must clamp it to keep it between the boundaries.

function clamp(value) {
  return Math.max(0, Math.min(Math.floor(value), 255))
}

We can mathematically define the red, green and blue function as f(x) = x+ɑ, where x is the current pixel value on that channel and alpha is the value the user selected.

Here's how our function to add a blue value to a pixel is defined. The functions to add red and green are left as an exercise, but is pretty much the same, having to change just the offset it uses.

const R_OFFSET = 0
const G_OFFSET = 1
const B_OFFSET = 2

function addBlue(x, y, value) {
  const index = getIndex(x, y) + B_OFFSET
  const currentValue = currentPixels[index]
  currentPixels[index] = clamp(currentValue + value)
}

An image with enhanced blue looks like this:

Brightness

The concept of brightness refers to how next to white our pixels are. Since the pure white pixel is represented by R=255, G=255 and B=255 we can easily verify that highest channel values leads to brighter pixels. Since we don't want to enhance any color channel specifically, we must change all of them, increasing or decreasing the mean of the each pixel.

function addBrightness(x, y, value) {
  addRed(x, y, value)
  addGreen(x, y, value)
  addBlue(x, y, value)
}

An image with decreased brightness looks like this:

Contrast

The most complex filter is the contrast, which refers to how spaced is the colors histogram of an image. A higher contrast means whites whiter and blacks blacker, so the pixels get more "distinguishable". There are plenty options to enhance a picture contrast, but for the learning purposes let's use a simpler one. The user input from -255 to 255 gets normalized where -255 to 0 is mapped to a range of 0 to 1 (less contrast to unchanged contrast) and 0 to 255 mapped to a range of 1 to 2 (from unchanged to double contrast). Let's call this normalized value alpha, so the contrast function can be defined as: f(x) = ɑ * (x - 128) + 128. That's confuse, but can be explained: x is the current channel color, we apply this function on every channel. We subtract 128 which is the half of possible color values (remember it goes from 0 to 255? remember we want to make white values whiter and blacks blacker?) and multiply alpha. We then sum back this removed 128. For example, if the current value is 180 and alpha is 1.4 (increase contrast), we have: 1.4 * (180 - 128) + 128 = 1.4 * 52 + 128 = 72.8 + 128 = 200.8. We made a white value whiter. If the current color is "blacker", x - 128 will result into a negative value, which multiplied by alpha will become even more negative and decrease. Let's see in JavaScript:

function addContrast(x, y, value) {
  const redIndex = getIndex(x, y) + R_OFFSET
  const greenIndex = getIndex(x, y) + G_OFFSET
  const blueIndex = getIndex(x, y) + B_OFFSET

  const redValue = currentPixels[redIndex]
  const greenValue = currentPixels[greenIndex]
  const blueValue = currentPixels[blueIndex]

  const alpha = (value + 255) / 255 // Goes from 0 to 2, where 0 to 1 is less contrast and 1 to 2 is more contrast

  const nextRed = alpha * (redValue - 128) + 128
  const nextGreen = alpha * (greenValue - 128) + 128
  const nextBlue = alpha * (blueValue - 128) + 128

  currentPixels[redIndex] = clamp(nextRed)
  currentPixels[greenIndex] = clamp(nextGreen)
  currentPixels[blueIndex] = clamp(nextBlue)
}

If you try to decrease contrast, on the other hand, you will narrow the colors histogram until all you can see is a plain gray picture. A picture with enhanced contrast looks like:

Grayscale

Our last filter is not scary as the previous one. A pixel can be made grayscale, or monochromatic, by simply converting it from three channels to a single one. What comes in mind to do that? Mean! All you have to do is to take the mean of R, G and B channels and apply it to R, G and B channels.

function setGrayscale(x, y) {
  const redIndex = getIndex(x, y) + R_OFFSET
  const greenIndex = getIndex(x, y) + G_OFFSET
  const blueIndex = getIndex(x, y) + B_OFFSET

  const redValue = currentPixels[redIndex]
  const greenValue = currentPixels[greenIndex]
  const blueValue = currentPixels[blueIndex]

  const mean = (redValue + greenValue + blueValue) / 3

  currentPixels[redIndex] = clamp(mean)
  currentPixels[greenIndex] = clamp(mean)
  currentPixels[blueIndex] = clamp(mean)
}

A colored picture converted to grayscale looks like:

Bringing everything together

Now that we have enough filters, let's update our "pipeline" function to effectively run them. The order here is important, because the result of one filter is used for the next one. Firstly, we apply the grayscale if checked, then brightness and contrast and finally R, G and B (only if not in grayscale).

function runPipeline() {
  currentPixels = originalPixels.slice()

  const grayscaleFilter = grayscale.checked
  const brightnessFilter = Number(brightness.value)
  const contrastFilter = Number(contrast.value)
  const redFilter = Number(red.value)
  const greenFilter = Number(green.value)
  const blueFilter = Number(blue.value)

  for (let i = 0; i < srcImage.height; i++) {
    for (let j = 0; j < srcImage.width; j++) {
      if (grayscaleFilter) {
        setGrayscale(j, i)
      }

      addBrightness(j, i, brightnessFilter)
      addContrast(j, i, contrastFilter)

      if (!grayscaleFilter) {
        addRed(j, i, redFilter)
        addGreen(j, i, greenFilter)
        addBlue(j, i, blueFilter)
      }
    }
  }

  commitChanges()
}

Our commit function simply gets the modified pixels, apply on ImageData object and draws the result.

function commitChanges() {
  for (let i = 0; i < imgData.data.length; i++) {
    imgData.data[i] = currentPixels[i]
  }

  ctx.putImageData(imgData, 0, 0, 0, 0, srcImage.width, srcImage.height)
}

We can't simply do

imgData.data = currentPixels

because it is a constant that cannot be reassigned.

And that's it! Play with the final result and download the images using right click > Save Image As...

Final thoughts

At the end we have developed an app with less than 200 lines of code that can powerfully apply the most common filters we'd want on an image processing tool! But there is something missing: where is those cool city name filters? They are pretty much a predefined combination of these presented here. Why don't you get your hands dirty and upgrade this app with your favorite cities as presets?

Final source code can be checked here and you can play with the software here. All examples were made using this code. Pictures: Joaquina Beach (splash), Cascavel downtown (blue), Curitiba Botanical Garden (brightness), Ilha do Mel (contrast) and Hercílio Luz Bridge (grayscale).