AI Fundamentals: A Practical Guide for All Professionals

This article is an easy way for technical and nontechnical professionals to understand the practical fundamentals of AI. The aim is to keep the discussion simple and relatively high-level for everyone without industry jargon.

Understanding AI: Misconceptions vs. Realities

What is an AI Model?

At its core, an AI model is simply a file stored on a computer. These models can vary in size and complexity:

• Some are small enough to run on laptops or mobile phones
• Others, like GPT, require large data centers and powerful GPUs

Fundamentally, AI models take an input and generate an output. That's their primary function.

Common Misconceptions

• Generative AI will fully replace humans:
  • Reality:
    • AI augments human capabilities rather than entirely replacing them.
    • Effective solutions combine human insight and support with AI efficiency.
• Only experts and technical people can work with AI
  • Reality:
    • Understanding a few key concepts and limitations is sufficient for working with AI.
    • With the rise of user-friendly AI platforms, professionals from various fields can use AI without deep technical expertise.
• AI is very complex
  • Reality:
    • AI basics are based on 4 basic fundamentals, making it seem more straightforward than it looks.
    • Many available tools make AI accessible to a wide range of users

Realities of Working with AI

• We can build an AI feature in 100 different ways

  • There are numerous ways to create an AI feature, similar to coding.

  • Different model types can be mixed and matched to achieve the same result.

  • There's no single "right way" to implement an AI feature.

    
    

• We should leverage existing models for proof of concepts (POCs)

  • Use GPT/LLMs and off-the-shelf models for quick deployment in POCs.

  • Custom LLMs can be time-consuming and resource-intensive but are best reserved for post-product market fit.

  • Building custom models for unproven concepts is high risk, with usually low ROI.

    
    

• Only optimize something when we see it work and deliver value.
  • Test solutions with minimal resources to increase execution speed and iteration.
  • This low-resource approach allows for testing multiple ideas with less risk.

Choosing the Right AI Model

When selecting an AI model, consider the following questions:

• What model are we using?
• What are the limitations of the model?
• What training data are we using?
• How big is the model?
• How fast is the model?
• What are the work and image size maximums for the models?

Common AI-Related Comments and Important Follow-up Questions:

• "Let's use Hugging Face!"

  • What specific task are we addressing?

  • Which Hugging Face model are we considering?

  • Have we compared different models?

    
    

• "The AI model works!"

  • What's the accuracy rate for this model?

  • What are the operational costs?

    
    

• "I need AI!"

  • What problem are you trying to solve?

  • Does solving the problem increase ROI?

  • Is an AI model truly necessary for this task?

    
    

• "I want to build/fine-tune a model!"
  • Have you explored existing models on Hugging Face?
  • Have you considered using GPT or other pre-trained models?

Transformers

Transformers transform one form of data into another. For example:

• Text to speech
• Text to audio
• Speech to text
• Text to video
• Text to text

Popular transformers include GPT, Midjourney, Stable Diffusion, DALL-e, GPT4, Lamma, Alpaca, and Claude.

Transformers are versatile. Initially designed for text-related tasks, they have proven effective for many data types, including images, video, and biological sequences.

The critical innovation of transformers is their "attention" mechanism. This allows them to understand the context and relationships between different parts of the input data.

Are Transformers Smart?

Despite their impressive outputs, transformers are not inherently smart in terms of how humans understand intelligence. Instead of possessing true comprehension or reasoning abilities, they function by memorizing vast amounts of information and using that data to generate plausible guesses. This highlights a fundamental aspect of current AI technology: what appears as "intelligence" is sophisticated pattern recognition and statistical prediction based on enormous datasets rather than genuine understanding or thought.

What are the Different Types of Transformers?

Text to Image

• Diagram -> input text -> output image

• Use cases

  • ChatGPT and Claude are transformers
  • Iterate on architectural designs by modifying text prompts
  • Generate custom illustrations for children's books or graphic novels

  
  

Audio to Text

• Diagram -> input audio -> output text

• Use cases

  • Transcribing calls
  • Provide real-time captioning for live events (for accessibility purposes)
  • Transcribing patient information in a clinic

  
  

Text to Text

• Diagram -> input text -> output text
• Use cases

  • Generate source code

  • Generate concise product descriptions from detailed specifications

  • Generate financial reports from raw data

    
    

Text to Video

• Diagram -> input text -> output video
• Use cases

  • Generate news video summaries from text articles

  • Convert product manuals into instructional videos

  • Create animated explainer videos for complex topics

    
    

Text to Audio

• Diagram -> input text -> output audio
• Use cases
  • Create personalized children's stories with custom voiceovers
  • Transform written articles or scripts into podcast episodes
  • Create voiced alerts and notifications for various applications

Challenges

Despite their capabilities, transformers face significant challenges that are the focus of ongoing research and development. One major concern is addressing biases learned from training data, which can lead to unfair or discriminatory outputs. Researchers are also prioritizing the safety of these AI systems and working to improve their efficiency, as their computational demands can be substantial.

Where Can I Find Transformer Models?

Transformers and other AI models can readily be found on popular machine learning and software development platforms. Hugging Face is a primary hub for accessing and sharing transformer models, offering a vast library of pre-trained models and tools. The widely-used code repository platform GitHub is another valuable resource where developers frequently share transformer implementations and related projects.

Classification

Classification is the process of categorizing things based on their characteristics.

When is Classification Useful?

Classification approaches are valuable for business problems with large amounts of historical data, including labels, that specify if something is in one group or another.

What are the Different Types of Classification?

Binary Classification

Binary classification sorts data into one of two distinct categories, such as yes/no or true/false. It's fundamental in machine learning, useful for spam detection and medical diagnoses, and forms the basis for more complex classification methods.

1. Outcome options such as:

   1. yes/no,

   2. good/bad,

   3. true/false,

   4. 1/0

      
      

2. Use cases:

   1. Based on the given health conditions of a person, we have to determine whether a person has a particular disease.
   2. Figuring out if an exam question has been answered correctly.
   3. Determining whether a crop is ready for harvest based on visual characteristics

Multi-Class Classification

Multi-class classification extends the binary concept to situations where data can belong to one of several categories. It's crucial for problems like image recognition and sentiment analysis, where multiple outcome options are possible.

1. Outputs options such as:

   1. [neutral, happy, sad, confused]

   2. [blue, red, green, yellow, pink]

   3. [sports, business, legal, tech]

      
      

2. Use cases:

   1. Analyzing customer sentiment in AI-powered customer service

   2. Evaluating machinery parts' conditions

   3. Categorizing news articles into topics like politics, sports, entertainment, or technology.

      
      

Multi-Label Classification

Multi-label classification assigns multiple categories or labels to a single data point simultaneously. This approach is valuable for scenarios with overlapping characteristics, such as categorizing movies by genre or tagging social media posts.

1. Use cases:

   1. Describing locations (e.g., San Diego → [beach, ocean, city, California])

   2. Tagging online retail products

   3. Categorizing machinery components

      
      

Imbalanced Classification

Imbalanced classification deals with datasets where one class is significantly underrepresented compared to the other. It's critical in scenarios like fraud detection or rare disease diagnosis, requiring special techniques to ensure the minority class isn't overlooked.

• Use case:

  • Diagnosing rare diseases
  • Detecting mechanical issues in railway systems
  • Manufacturing quality control
  • Earthquake prediction
  • Fraud detection in credit card transactions

  
  

Classification in AI offers powerful tools for categorizing data, with applications ranging from simple binary decisions to complex multi-label assignments. Its versatility is evident in healthcare, customer service, and manufacturing. As AI evolves, classification techniques will be increasingly crucial in solving complex business problems and enhancing decision-making processes.

Extraction

Extraction in AI is a data processing technique that involves identifying and isolating specific pieces of information from larger, often unstructured datasets. This technique can be applied to various data types, including text, images, and audio. The goal is to find relevant details or patterns that are useful for analysis or further processing.

What are the Different Types of Extraction?

1. Named Entity Recognition (Text)

   1. Name Entity Recognition (NER) is a Natural Language Processing technique that identifies and extracts specific information from text. It enables machines to understand and categorize essential elements within unstructured text data.

      1. Use cases:

         1. Email analysis: Extracting purpose, sender, recipient, and timestamp

         2. Business listings: Identifying location, price, broker, and cash flow

         3. Customer Support: Categorizing requests, complaints, and questions

         4. Healthcare: Quickly extracting essential information from medical reports

         5. Search engines: Analyzing search queries and other texts

         6. Human Resources: Categorizing internal processes and summarizing CVs

            
            

2. Image Segmentation

   1. Image Segmentation (also used in video segmentation) is a computer vision technique that divides an image into multiple segments or regions, each corresponding to a distinct object or part of the image. This process involves analyzing the image's pixels and grouping them based on shared characteristics such as color, texture, or intensity. The resulting segments are then labeled, allowing for the identification and isolation of specific objects or areas within the image.

      1. Use cases:

         1. Autonomous driving

         2. Analysis of railway tracks to detect upcoming maintenance needs

         3. Background removal in images Medical imaging analysis (e.g., X-rays, MRIs)

            
            

Similarity

Similarity models in AI quantify how alike different pieces of data are. This allows systems to find related items, match queries to relevant answers, and group similar data. Similarity models are crucial for applications like search engines, recommendation systems, and data clustering, enabling AI to recognize patterns and relationships across large datasets efficiently.

Note: There are differences between embedding (such as BERT) and vector (such as all-MiniLM-L6-v2) models. Both, while different, may be used for similarity-related use cases. This article will use the terms vectorization, embeddings, and similarity synonymously to simplify understanding.

What can be vectorized?

Vectorization in AI can be applied to various data types, including images, audio, and text. This process converts these diverse data types into numerical vector representations, allowing AI models to process and analyze them mathematically. By transforming complex, unstructured data into vectors, AI systems can more easily compare, classify, and manipulate this information for tasks such as image recognition, speech processing, and natural language understanding.

Use cases

• Finding similar-looking products when online shopping
• Suggesting music you might like based on your playlist
• Grouping similar news articles together
• Identifying faces in your photo collection
• Detecting spam emails by comparing them to known spam
• Recommending movies based on ones you've enjoyed
• Translating languages by finding similar phrases
• Understanding the mood of customer reviews (positive or negative)
• Answering questions by finding similar information in a database
• Improving search results by understanding what you're looking for
• Converting your speech into text for voice assistants
• Detecting unusual sounds in machinery that might indicate a problem

Semantic Search

Semantic search is an advanced information retrieval technique focusing on understanding a query's intent, contextual meaning, and relationships between concepts rather than just matching keywords. This approach enables more accurate and relevant search results, even when the exact search terms aren't in the target content.

Semantic search is often used for RAG-based systems. These systems find the top-N number of documents and summarize them to provide a suitable answer. This is how most generative search results are generated today.

Key Takeaway

AI is a powerful tool that, when used correctly, can solve complex problems efficiently. This article provides a straightforward overview of AI fundamentals for both technical and non-technical professionals, aiming to demystify the subject without relying on industry jargon. It breaks down common misconceptions about AI, emphasizing that while AI is powerful, it augments rather than replaces human capabilities, and its basics can be understood by anyone willing to learn a few key concepts.

Conclusion

This guide explains AI basics for both technical and non-technical professionals. Here are the key points:

• AI Essence:
  • AI models are computer files processing inputs to produce outputs.
• Key Misconceptions:
  • AI enhances rather than replaces human work.
  • AI usage doesn't require advanced technical skills.
  • AI is complex
• Practical Implementation:
  • Use existing models for initial testing.
  • Start simple and refine as needed.
  • Only optimize something when we see it work and deliver value.
• Model Selection:
  • Consider the size, speed, and specific task requirements.
• Transformers are versatile AI models that can transform one form of data into another, such as text-to-speech, text-to-image, speech-to-text, text-to-text, and text-to-video.
• Classification is the process of categorizing data based on characteristics and is useful for business problems with large amounts of historical data and labels.
• Extraction in AI involves identifying and isolating specific information from larger, unstructured datasets using techniques like Named Entity Recognition and Image Segmentation.
• Similarity models in AI quantify how alike different pieces of data are, enabling systems to find related items, match queries to relevant answers, and group similar data efficiently.
• Understanding these AI fundamentals empowers professionals from various fields to work with AI without deep technical expertise, leveraging user-friendly platforms and tools to augment their capabilities and drive innovation.

AI is a versatile tool accessible to professionals across various fields.