269 reads

Taking the Azure Open AI Challenge, Day 5: Azure Document Intelligence

by Sukhpinder SinghApril 1st, 2024

Too Long; Didn't Read

Designed for developers familiar with Azure and generative AI, the guide walks you through the process of harnessing the power of the prebuilt model for document intelligence.

featured image - Taking the Azure Open AI Challenge, Day 5: Azure Document Intelligence

Designed for developers familiar with Azure and generative AI, the guide walks you through the process of harnessing the power of the prebuilt model for document intelligence.

Introduction

The Microsoft team has recently launched an Open AI challenge in which a developer can learn how to build Azure AI solutions and apps.

Prerequisite

Experience working with Azure and Azure portals.
An understanding of generative AI.
Experience in one high-level programming language like C# or Python

Getting Started

The article demonstrates the use of a Prebuilt Model — Invoice in the document intelligence service.

What is a Prebuilt Model — Invoice?

Use a predefined model to identify and collect standard elements from particular types of documents. In this instance, we employ the built-in invoice model to process and extract information from an invoice.

The guide will cover the following features.

The steps to configure the document intelligence service on the Azure Portal.
A console application in C# to process a sample document.

Reference: Sample document

Step 1: Navigate to the Azure Portal

Search for Azure Document Intelligence and fill out the following details

Step 2: Choose Network

Step 3: Assign Identity

Enable system-assigned identity to grant the resource access to other existing resources. For this demonstration, we don't require identity.

Step 4: Add Tags

For this exercise, tag names are not required. But in a production environment, it should be added as it’s a best practice.

Step 5: Review & Create

Post validation checks by Azure Cloud and proceed with creating the resource. Make sure you review the details entered in the previous steps.

Let’s switch to code

Please find below the sample document highlighting the fields that will be processed as part of this exercise using C# code.

The expected output should be as follows, along with the confidence score

Vendor Name: 'CONTOSO LTD.', with confidence 0.93.
Customer Name: 'MICROSOFT CORPORATION', with confidence 0.915.
Invoice Total: '$110', with confidence 0.97.

Step 1: Create a console application

To test image generation, create a console application in Visual Studio or Visual Studio Code.

dotnet new console

Step 2: Read the configuration

Read the configuration from appsettings.json file

IConfiguration config = new ConfigurationBuilder()
    .AddJsonFile("appsettings.json")
    .Build();
string? endpoint = config["AzureEndpoint"];
string? apiKey = config["AzureKey"];
string? documentUrl = config["DocumentUrl"];

Step 3: Create a client

var cred = new AzureKeyCredential(apiKey);
var client = new DocumentAnalysisClient(new Uri(endpoint), cred);

Step 4: Analyze the Invoice

Install the NuGet package using the following command

dotnet add package Azure.AI.FormRecognizer

and add the following code

AnalyzeDocumentOperation operation = await client.AnalyzeDocumentFromUriAsync(WaitUntil.Completed, "prebuilt-invoice", fileUri);
await operation.WaitForCompletionAsync();

AnalyzeResult result = operation.Value;

Step 5: Display Invoice Information

The final step is to display the invoice information to the user using the following code snippet.

To keep things straightforward, we’re not displaying every key-value pair returned by the service here. For a comprehensive list of all fields supported and their respective types, please refer to the Invoice concept page.

foreach (AnalyzedDocument invoice in result.Documents)
{
    if (invoice.Fields.TryGetValue("VendorName", out DocumentField? vendorNameField))
    {
        if (vendorNameField.FieldType == DocumentFieldType.String)
        {
            string vendorName = vendorNameField.Value.AsString();
            Console.WriteLine($"Vendor Name: '{vendorName}', with confidence {vendorNameField.Confidence}.");
        }
    }


    if (invoice.Fields.TryGetValue("CustomerName", out DocumentField? customerNameField))
    {
        if (customerNameField.FieldType == DocumentFieldType.String)
        {
            string customerName = customerNameField.Value.AsString();
            Console.WriteLine($"Customer Name: '{customerName}', with confidence {customerNameField.Confidence}.");
        }
    }

    if (invoice.Fields.TryGetValue("InvoiceTotal", out DocumentField? invoiceTotalField))
    {
        if (invoiceTotalField.FieldType == DocumentFieldType.Currency)
        {
            CurrencyValue invoiceTotal = invoiceTotalField.Value.AsCurrency();
            Console.WriteLine($"Invoice Total: '{invoiceTotal.Symbol}{invoiceTotal.Amount}', with confidence {invoiceTotalField.Confidence}.");
        }
    }
}
Console.WriteLine("\nAnalysis complete.\n");

Step 6: Run the code

Make sure the document URL is added to the appsettings.json file.

The URL for the example document is provided below.

https://github.com/MicrosoftLearning/mslearn-ai-document-intelligence/blob/main/Labfiles/01-prebuild-models/sample-invoice/sample-invoice.pdf?raw=true

Complete Program.cs file

// See https://aka.ms/new-console-template for more information


// Build a config object and retrieve user settings.
using Azure;
using Azure.AI.FormRecognizer.DocumentAnalysis;
using Microsoft.Extensions.Configuration;

IConfiguration config = new ConfigurationBuilder()
    .AddJsonFile("appsettings.json")
    .Build();
string? endpoint = config["AzureEndpoint"];
string? apiKey = config["AzureKey"];
string? documentUrl = config["DocumentUrl"];

Uri fileUri = new Uri(uriString: documentUrl);

Console.WriteLine("\nConnecting to Forms Recognizer at: {0}", endpoint);
Console.WriteLine("Analyzing invoice at: {0}\n", fileUri.ToString());

// Create the client

var cred = new AzureKeyCredential(apiKey);
var client = new DocumentAnalysisClient(new Uri(endpoint), cred);

// Analyze the invoice
AnalyzeDocumentOperation operation = await client.AnalyzeDocumentFromUriAsync(WaitUntil.Completed, "prebuilt-invoice", fileUri);
await operation.WaitForCompletionAsync();

AnalyzeResult result = operation.Value;

// Display invoice information to the user
foreach (AnalyzedDocument invoice in result.Documents)
{
    if (invoice.Fields.TryGetValue("VendorName", out DocumentField? vendorNameField))
    {
        if (vendorNameField.FieldType == DocumentFieldType.String)
        {
            string vendorName = vendorNameField.Value.AsString();
            Console.WriteLine($"Vendor Name: '{vendorName}', with confidence {vendorNameField.Confidence}.");
        }
    }


    if (invoice.Fields.TryGetValue("CustomerName", out DocumentField? customerNameField))
    {
        if (customerNameField.FieldType == DocumentFieldType.String)
        {
            string customerName = customerNameField.Value.AsString();
            Console.WriteLine($"Customer Name: '{customerName}', with confidence {customerNameField.Confidence}.");
        }
    }

    if (invoice.Fields.TryGetValue("InvoiceTotal", out DocumentField? invoiceTotalField))
    {
        if (invoiceTotalField.FieldType == DocumentFieldType.Currency)
        {
            CurrencyValue invoiceTotal = invoiceTotalField.Value.AsCurrency();
            Console.WriteLine($"Invoice Total: '{invoiceTotal.Symbol}{invoiceTotal.Amount}', with confidence {invoiceTotalField.Confidence}.");
        }
    }
}
Console.WriteLine("\nAnalysis complete.\n");

Complete code on GitHub

Make sure to give it a star on GitHub and provide feedback on how to improve the tool further..!! To access the document intelligence sample, move to the directory samples > Azure.OpenAI.DocumentIntelligence

GitHub - ssukhpinder/AzureOpenAI: Sample for testing Azure Open AI GPT3 Turbo Model