AI Is Hard? It’s Child’s Play With This AWS AI Services Scratch Extension

AWS AI Services Scratch Extension

In my previous post, I have mentioned that we should prepare our children for the future. In the future, to succeed our children should have skills like creative thinking and problem solving. STEM education can improve this skills.

According to MIT Professor Mitchel Resnick, coding is the new literacy. A visual programming language called Scratch could stimulate the curiosity we know it exists in the children from the day they have born. You can read stories about how it changes children’s lives here, here and here.

As I am working with AWS AI services, I have been wondering how we could teach AI to our children easily. Then I came across ScratchX, an extension framework to Scratch for adding new functionality to Scratch. With ScratchX, you can write extensions to integrate Scratch with anything you imagine. There are sample extensions for LEGO WeDo, Arduino, and Raspberry Pi.

Then I thought I can make a Scratch extension for using AWS AI services like Polly and Translate. And it is here.

You can try sample project here that uses Amazon Polly to speak the text you have entered in the language you choose. Also you can translate text between languages and speak the translated text in target language.

Below is a video recording of the sample project.

Developing The Extension

If you wonder how the extension is developed, let’s develop the extension step by step together.

A minimal Scratch Extension is created like below. After creating the extension object, we should register it by calling ScratchExtensions.register with the name of the extension and a descriptor which defines the blocks that will be created. For more information on making Scratch Extensions, see here.

We can develop our extension in three steps:

1. Create the descriptor
2. Implement the methods
3. Try the extension in ScratchX

Let’s start.

1. Create the descriptor

The descriptor defines the blocks that will be created after the extension is registered. Created blocks will be shown in ‘More Blocks’ section of ScratchX.

The blocks of our extension are defined as below.

First block is for initializing the AWS JavaScript SDK in the browser.

Amazon Polly blocks are the next two blocks. And last blocks are for Amazon Translate service.

Parameters of blocks aredefined as ‘say %s’. We can use lists to choose parameters from with ‘%m.sourceLanguages’ syntax, where ‘%m.’ says that we want to use a list and sourceLanguages shows the name of the list. In ScratchX lists are known as menus and menus are defined with ‘menus’ field.

Blocks of our extension will look like below.

Blocks are shown in ‘More Blocks’ section

2. Implement the methods

After we defined our blocks, we will implement the methods that will be executed when blocks are used.

For more information on method details, you can look extension JavaScript file. Usage of AWS services in the browser with AWS JavaScript SDK is explained in detail in my previous blog post.

3. Try the extension in ScratchX

In order to try our extension, we should load it from a URL. I recommend MiniWeb HTTP Server for this.

After you can run the server in your directory at 8080 port, you can try it in your browser by visiting the address below. I recommend Google Chrome browser, ScratchX is not working in Firefox. IE will not load the extension from http: url, although it works over https: url.

https://scratchx.org/?url=http://localhost:8080/awsAiServicesScratchExtension.js

When ScratchX warned you, click ‘I understand, continue’ to load and try the extension.

Summary

In this post, I have shown how a Scratch Extension can be created easily for using AWS AI services in Scratch. You can find the code here. For more information on making Scratch Extensions, see here.

For more information on how AWS AI services can be used in the browser using AWS JavaScript SDK, please see my previous blog post:

You Can’t Fail To Learn A New Language With These AWS AI Services: Translate, Rekognition, Polly

Also, you might like my other post about creating a serverless app for saving allergy information with the picture of children and searching the allergy information later with the picture of the children by using face recognition methods of Amazon Rekognition:

Serverless Allergy Checker with Amazon Rekognition, Lex, Polly, DynamoDB, S3 and Lambda

I will continue to write about different use cases of AWS AI service. I will add object and text detection features to the extension using Amazon Rekognition.

I am developing a Chrome Browser Extension for reading the selected text in the page you are viewing using Amazon Polly and detecting Authy authentication codes from the camera using Amazon Rekognition. Please stay tuned or follow.

If you like this post, please share, clap or follow.

Thanks for reading.