Boosting Your App's Intelligence: Leveraging OpenAI and JS File API

Artificial Intelligence (AI) is rapidly changing the way we interact with technology. One of the most significant recent advances has been the development of ChatGPT by OpenAI - an AI research lab that produces some of the most advanced AI models in the world. This powerful tool has already gained recognition in natural language processing, computer vision, reinforcement learning, and other fields!

ChatGPT is one of the largest and most intelligent language models, used widely in education, mental health support, customer service, personal assistance, and entertainment.

In this article, we'll discuss how OpenAI can enhance JavaScript File API usage to create smarter apps capable of performing complex tasks, thereby increasing business revenue. This discussion will provide you with a new perspective and confidence to utilize OpenAI for future projects.

What is the JavaScript File API?

The JavaScript File API allows developers to access and manipulate files directly from the browser. By using File API, developers can create web apps that can carry out file-related operations, such as uploading and downloading files, validating user input, and performing various other file manipulations.

Here you’ll find the bare essentials we need to know about the File API to confidently proceed to other sections.

Files are represented as FileList objects which include individual File objects. Every File  object provides access to metadata such as file's name, size, type, and last modified date. In order to access file contents we can simply pass File objects to FileReader.

The readAsDataURL method provided by the FileReader interface is used to read the contents of the specified Blob or File. When the read operation is finished, the readyState becomes DONE, and the loadend is triggered. At that time, the result attribute contains the data as a data: URL representing the file's data as a base64 encoded string.

Nowadays there are plenty of advanced tools like FileStack that help you simplify file upload and management and automate much of routine work for you. In this story, we’ll focus on file management using the vanilla File API.

Setup Open AI

OpenAI has a well-written official documentation - platform.openai.com - that opens up a broad range of natural language processing capabilities like chatbots, virtual assistants, and other conversational interfaces. They even have a nice QuickStart guide to get acquainted with key concepts through hands-on and interactive examples. So if you feel like you’re new to it, don’t pass around it.

The next thing we should care about is API keys. In order to use OpenAI API in Javascript, we will need an API Key to authenticate and authorize our requests to their servers. An API Key is a security token that identifies our account when we access OpenAI API services. Without an API Key , your requests will be rejected, and you will not be able to access OpenAI API. So first we must create an account on the OpenAI’s platform and generate an API key - Platform API Keys - see below.

The registration process is fairly straightforward: you merely add a new secret key - and you’re good to go! But should you have any questions or obstacles, don’t hesitate to drop me a line. I’ll be glad to be of service to assist you or create a separate story for that.

Leverage OpenAI with File API

The integration of OpenAI and JavaScript File API empowers developers to build intelligent applications capable of executing intricate tasks such as content curation, data analysis, predictive analytics, and more. By combining the power of OpenAI's language models with the file manipulation capabilities of JavaScript File API, we can develop advanced new-level apps that can automatically generate text content based on user input or perform advanced text analytics on large datasets.

Let’s cut to the chase and look over how OpenAI can be used with the File API in the example of Smart Image Recognition.

With the help of OpenAI's advanced image recognition models, JavaScript File API can be enhanced to identify images and classify them accordingly. To tailor the image recognition feature we’ll be using the following endpoints:

1. https://api.openai.com/v1/images/gpt
2. https://api.openai.com/v1/images/generations

Then with the help of File API, we’re going to read the file selected by the user and convert it to Base64 format. This is what the above endpoints expect. We then send the Base64-encoded image to OpenAI's image recognition model, which identifies the image and returns the classification results.

Let’s take a glance at a general example of how we can utilize Open AI API here and print the image classification results:

// Get the image file from the input element
const input  = document.querySelector('input[type="file"]');
// Get a File object from the FileList
const imageFile = input.files[0];
// Initiate a file reader to analyze file contents further
const reader = new FileReader();
// Define OpenAI API Key we generated earlier
const apiKey = '<HACKERNOON API KEY>';

// Read file contents
reader.readAsDataURL(imageFile);

// Add an "onload" handler that'll be invoked as soon as the file is loaded
reader.onload = async function() {
  // Parse image content as Base64 format 
  const base64Image = reader.result.replace(/^data:image\/(png|jpg|jpeg);base64,/, '');
  // Fetch image classification results
  const response = await fetch('https://api.openai.com/v1/images/gpt', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${apiKey}`,
    },
    body: JSON.stringify({ image: base64Image }),
  });
  // Parse the response from OpenAI as JSON
  const result = await response.json();
  // Print the result in the console
  console.log(result);
};

We can even go far beyond and reap other powerful Open AI benefits with the Images API like:

Images Generation

We can create brand new images from scratch based on a text prompt - Images Generation:

Here’s the code sample:

const input = document.querySelector('input[type="file"]');

input.addEventListener('change', async (event) => {
  const file = input.files[0]; // or event.target.files[0];
  const reader = new FileReader();

  reader.onload = async () => {
    const base64Image = reader.result.replace(/^data:image\/(png|jpg|jpeg);base64,/, '');
    const response = await fetch('https://api.openai.com/v1/images/generations', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${apiKey}`,
      },
      body: JSON.stringify({
        model: 'image-alpha-001',
        prompt: 'a close up, studio photographic portrait of a white siamese cat that looks curious, backlit ears',
        image: base64Image,
      }),
    });

    const result = await response.json();
    const imageUrl = result.data[0].url;

    const generatedImage = document.createElement('img');
    generatedImage.src = imageUrl;
    document.body.appendChild(generatedImage);
  };

  reader.readAsDataURL(imageFile);
});

Images Editing

Apart from that we can extend or create edits of an existing image based on a new text prompt - Edit Images:

Here’s how we can achieve it:

const input = document.querySelector('input[type="file"]');
const imageFile = input.files[0];

const formData = new FormData();
formData.append('image', imageFile);

const response = fetch('https://api.openai.com/v1/images/generations', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${apiKey}`
  },
  body: JSON.stringify({
    model: 'image-alpha-001',
    prompt: 'a sunlit indoor lounge area with a pool containing a flamingo',
    response_format: 'url',
    image_size: [512, 512],
    image_crop: [1.0, 1.0],
    image_rotate: 0.0,
    image_translate: [0.0, 0.0],
    response_format: 'url',
    target_images: [formData],
  }),
});

const result = await response.json();
const imageUrl = result.data[0].url;

const generatedImage = document.createElement('img');
generatedImage.src = imageUrl;
document.body.appendChild(generatedImage);

Images Variations

On top of that, we also create variations of existing images:

Here’s how it can be done:

const input = document.querySelector('input[type="file"]');

input.addEventListener('change', async (event) => {
  const imageFile = input.files[0]; // or event.target.files[0];
  const reader = new FileReader();

  reader.onload = async () => {
    const base64Image = reader.result.replace(/^data:image\/(png|jpg|jpeg);base64,/, '');
    const response = await fetch('https://api.openai.com/v1/images/generations', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${apiKey}`,
      },
      body: JSON.stringify({
        model: 'image-alpha-001',
        prompt: `generate variations of this image: ${base64Image}`,
        size: 512,
        response_format: 'url',
      }),
    });

    const result = await response.json();
    const imageUrl = result.data[0].url;

    const generatedImage = document.createElement('img');
    generatedImage.src = imageUrl;
    document.body.appendChild(generatedImage);
  };

  reader.readAsDataURL(imageFile);
});

Looks impressive, isn’t it? Give it a shot yourself and let me know if you want to see other OpenAI usages!

Conclusion

In this article, we’ve gotten a grasp of how OpenAI and JavaScript File API can be tailored together to create intelligent and innovative apps capable of solving sophisticated tasks. Furthermore, the ability to use these APIs within the browser using JavaScript File API opens up a world of possibilities for creating more intuitive and creative user interfaces. As OpenAI continues to evolve and improve, we can expect to see even more opportunities for leveraging its technology in combination with JavaScript File API to create innovative applications that benefit both developers and end-users.

Here’s one of my other articles you may totally like - 🚀 Boost Your Productivity As A Software Engineer 🚀.

Should you have any questions, don’t hesitate to let me know by email - [email protected]. I’m always willing to help you, my readers! I wish you good luck on this track!