615 reads

Taking the Azure Open AI Challenge - Day 4: How to Set Up the Azure AI Search Service

by Sukhpinder SinghMarch 29th, 2024

Too Long; Didn't Read

Azure AI Search leverages advanced AI and machine learning to enrich indexing and search capabilities. It can automatically understand the content within documents, images, and other media types, extracting valuable information, identifying patterns, and even understanding sentiments. The Microsoft team has recently launched an Open AI challenge, in which a developer can learn how to build Azure AI solutions.

featured image - Taking the Azure Open AI Challenge - Day 4: How to Set Up the Azure AI Search Service

A day dedicated to Azure Open AI, focusing on harnessing Azure AI Search for document indexing and enrichment.

The Microsoft team has recently launched an Open AI challenge, in which a developer can learn how to build Azure AI solutions and apps.

Introduction

The article demonstrates how to use Azure AI Search to develop a solution where documents undergo indexing and enrichment through AI skills, enhancing their searchability.

Why Azure AI Search?

Azure AI Search leverages advanced AI and machine learning to enrich indexing and search capabilities. It can automatically understand the content within documents, images, and other media types, extracting valuable information, identifying patterns, and even understanding sentiments. These capabilities allow for more nuanced and intelligent search results.

Prerequisite

Experience working with Azure and Azure portals.
An understanding of generative AI.
Experience in one high-level programming language like C# or Python

Getting Started

Before starting the actual implementation, 3 different Azure resources need to be set up before moving forward.

Setup Azure AI Search
Setup Azure AI Service
Setup Azure Storage Account

Your Azure AI Search and Azure AI Services resources must be in the same location!

Once the above prerequisites are completed, you need to perform the following steps in the Azure Portal.

Upload documents to Azure Blob Storage
Index the documents
Testing

Setup Azure AI Search

Step 1: Navigate to the Azure Portal

Search for Azure AI Search, and fill out the following details.

Step 2: Scaling Configuration

Since I am using the free pricing tier, I cannot add scaling capabilities or create replicas as shown below.

Step 3: Create Tags

For this exercise, tag names are not required. But in a production environment, it should be added as it’s a best practice.

Step 4: Review & Create

Post validation checks by Azure Cloud, and proceed with creating the resource. Make sure you review the details entered in the previous steps.

Setup Azure AI Service

Step 1: Create Azure AI Service

Now, let's create another service, that searches for **Azure AI Service **and fills out the following details

Now, continue with the remaining steps as default options are selected, create the service, and finally, wait for the service to be deployed.

Step 2: Network Defaults

Step 3: Identity Defaults

Step 4: Review & Create

Post validation checks by Azure Cloud, and proceed with creating the resource. Make sure you review the details entered in the previous steps.

Setup Azure Storage Account

Step 1: Create an Azure Storage Account

Now, let’s create another service, that searches for Azure Storage Account, and fills out the following details.

Step 2: Allow Anonymous Access

In the Advanced tab, check the box next to Allow enabling anonymous access on individual containers.

Step 3: Review & Create

Now, continue with the remaining steps as default options are selected, create the service, and finally, wait for the service to be deployed.

Upload Documents to Azure Blob Storage

With the necessary resources in place, proceed to upload documents to your Azure Storage account.

Follow the article provided by Microsoft.

Quickstart: Upload, download, and list blobs - Azure portal - Azure Storage

Index the Documents

Open Azure AI Service, and under the Overview tab, select “Import Data” as highlighted below.

From the dropdown, select “Azure Blob Storage” as shown below.

The Connect Your Data tab should follow the following rules.

The Add Cognitive Skills tab should follow the following rules.

The Add Customized Target Indexer tab should follow the following rules.

Click on Submit to initiate the creation of the data source, skillset, index, and indexer.

Test the Indexer

Toward the top of your Azure AI Search resource’s Overview page, opt for Search Explorer. Within Search Explorer, input * (a single asterisk) into the Query string box, then proceed to click on the Search button.

Test Case 1: Fetch All

This query retrieves all documents in the index in JSON format.

Test Case 2: Include Count

Above the search results, you’ll find a count indicating the number of documents returned by the search.

Test Case 3: Search Specific Keywords

This search locates documents containing references to “New York” within any searchable fields and provides the document’s filename along with its key phrases.

Conclusion

The step-by-step guidance on setting up necessary Azure resources and detailed instructions for uploading documents, indexing them, and testing the setup is crucial for anyone looking to implement a robust document management solution.