How LLMs Have Transformed Working with Computers

All Images used in this article were created by the author with the . Bing Image Creator The advent of (LLMs) like OpenAI's GPT-3 is a paradigm shift in the world of computing and artificial intelligence. Large Language Models These models are revolutionizing the way we approach complex tasks, from machine learning and computer vision to more routine programming tasks like parsing. The beauty of LLMs lies in their ability to understand and generate human language, enabling us to interact with them using plain English. This has completely revolutionized work with computers. Suddenly, everyone is a creative genius. In programming, especially for specific use cases, all the old libraries like PyTorch, TensorFlow, and JAX will not be used for simple tasks but as tools to create ever more powerful Transformers. To understand what a Transformer is and the potential of this technology, you can read the article below. https://hackernoon.com/the-dawn-of-the-transformer-neural-networks?embedable=true Here’s what I mean The concept is so simple that a 3-year-old child could do it. And the child would be more creative than most of us. Here are ten common use-cases, in contrast with the long complex multi-stage program creation of 3-month long projects and the corresponding LLM prompts done in seconds! Artificial Intelligence/ Machine Learning Sentiment Analysis: # Import necessary libraries import tweepy from textblob import TextBlob # Twitter API credentials consumer_key = 'YOUR_CONSUMER_KEY' consumer_secret = 'YOUR_CONSUMER_SECRET' access_token = 'YOUR_ACCESS_TOKEN' access_token_secret = 'YOUR_ACCESS_TOKEN_SECRET' # Authenticate with the Twitter API auth = tweepy.OAuthHandler(consumer_key, consumer_secret) auth.set_access_token(access_token, access_token_secret) api = tweepy.API(auth) # Define the search term and the date_since date search_words = "#climatechange" date_since = "2021-11-01" # Collect tweets tweets = tweepy.Cursor(api.search, q=search_words, lang="en", since=date_since).items(1000) # Create a function to get the subjectivity def getSubjectivity(text): return TextBlob(text).sentiment.subjectivity # Create a function to get the polarity def getPolarity(text): return TextBlob(text).sentiment.polarity # Create a function to compute the negative, neutral and positive analysis def getAnalysis(score): if score < 0: return 'Negative' elif score == 0: return 'Neutral' else: return 'Positive' # Collect the tweets and their polarity data = [[tweet.text, getAnalysis(getPolarity(tweet.text))] for tweet in tweets] # Print the tweets and their sentiment for tweet in data: print(f'{tweet[0]}: {tweet[1]}') This program first authenticates with the Twitter API using your credentials. It then defines a search term and a date and collects tweets that match this criterion. It defines three functions to get the subjectivity, polarity, and sentiment analysis of a text. It then applies these functions to the collected tweets and prints the results. LLM Prompt Equivalent: "What is the sentiment of the text 'OpenAI's GPT-4 is amazing!'?" Image Recognition: # Import necessary libraries import numpy as np from keras.applications.vgg16 import VGG16 from keras.preprocessing import image from keras.applications.vgg16 import preprocess_input, decode_predictions # Load the VGG16 model model = VGG16(weights='imagenet') # Define a function to load and preprocess the image def load_and_process_image(image_path): img = image.load_img(image_path, target_size=(224, 224)) img_array = image.img_to_array(img) expanded_img_array = np.expand_dims(img_array, axis=0) preprocessed_img = preprocess_input(expanded_img_array) return preprocessed_img # Define a function to make predictions def predict_image(model, processed_image): predictions = model.predict(processed_image) label = decode_predictions(predictions) return label # Define a function to print the predictions def print_predictions(label): print('Top predictions of the image are:') for prediction_id in range(len(label[0])): print(f'{label[0][prediction_id][1]}: {round(100*label[0][prediction_id][2], 2)}%') # Define a list of image paths for testing image_paths = ['image1.jpg', 'image2.jpg', 'image3.jpg', 'image4.jpg', 'image5.jpg'] # Loop over all images and make predictions for image_path in image_paths: print(f'Processing image: {image_path}') processed_image = load_and_process_image(image_path) label = predict_image(model, processed_image) print_predictions(label) print('---------------------------------------') This program is designed to classify multiple images. It first loads the VGG16 model, and then defines three functions: one to load and preprocess the image, one to make predictions using the model, and one to print the predictions. It then defines a list of image paths and loops over this list, making predictions for each image and printing the results. LLM Prompt Equivalent: "What objects are in the image 'elephant.jpg'?" Chatbots: # Import necessary libraries from chatterbot import ChatBot from chatterbot.trainers import ChatterBotCorpusTrainer from flask import Flask, render_template, request # Create a chatbot chatbot = ChatBot('Example Bot') # Create a trainer for the chatbot trainer = ChatterBotCorpusTrainer(chatbot) # Train the chatbot on the English corpus trainer.train("chatterbot.corpus.english") # Define a function to get a response from the chatbot def get_response(user_input): return chatbot.get_response(user_input) # Create a Flask web application app = Flask(__name__) # Define the home page @app.route("/") def home(): return render_template("index.html") # Define the route to get the chatbot's response @app.route("/get") def get_bot_response(): user_input = request.args.get('msg') return str(get_response(user_input)) # Run the web application if __name__ == "__main__": app.run() LLM Prompt : Equivalent "Generate a response to the message 'Hello, how are you?'" Text Summarization: # Import necessary libraries import requests from bs4 import BeautifulSoup from gensim.summarization import summarize # Fetch the content from a webpage url = 'https://en.wikipedia.org/wiki/Artificial_intelligence' response = requests.get(url) soup = BeautifulSoup(response.text, 'html.parser') # Extract the text from the webpage text = '' for paragraph in soup.find_all('p'): text += paragraph.text # Preprocess the text text = text.replace('\n', ' ').replace('\r', '').strip() # Summarize the text summary = summarize(text) # Print the summary print(summary) This program first fetches the content from a webpage, in this case, the Wikipedia page for Artificial Intelligence. It then extracts the text from the webpage and preprocesses it by replacing newline and carriage return characters with spaces and removing leading and trailing whitespace. It then summarizes the text using the summarize function from the Gensim library and prints the summary. LLM Prompt : Equivalent "Can you summarize this text?" Language Translation: Creating a language translation model from scratch is a complex task that involves deep learning and a large amount of data. It's typically done using sequence-to-sequence (seq2seq) models, a type of model that's ideal for many tasks including language translation. Here's a simplified example of how you might start to approach this task using Python and the Keras library. This example doesn't include the training, testing, and deployment phases, as each of these would require a significant amount of code and computational resources. # Import necessary libraries from keras.models import Model from keras.layers import Input, LSTM, Dense import numpy as np # Parameters for our model batch_size = 64 epochs = 100 latent_dim = 256 num_samples = 10000 # Placeholder for input data input_texts = [] target_texts = [] # Placeholder for character data input_characters = set() target_characters = set() # Load your data here # This could be a parallel corpus of English and French sentences # For example, the English-French sentence pairs from http://www.manythings.org/anki/ # Vectorize your data # Define an input sequence and process it encoder_inputs = Input(shape=(None, num_encoder_tokens)) encoder = LSTM(latent_dim, return_state=True) encoder_outputs, state_h, state_c = encoder(encoder_inputs) # We discard `encoder_outputs` and only keep the states encoder_states = [state_h, state_c] # Set up the decoder, using `encoder_states` as initial state decoder_inputs = Input(shape=(None, num_decoder_tokens)) # We set up our decoder to return full output sequences, # and to return internal states as well. We don't use the # return states in the training model, but we will use them in inference decoder_lstm = LSTM(latent_dim, return_sequences=True, return_state=True) decoder_outputs, _, _ = decoder_lstm(decoder_inputs, initial_state=encoder_states) decoder_dense = Dense(num_decoder_tokens, activation='softmax') decoder_outputs = decoder_dense(decoder_outputs) # Define the model that will turn # `encoder_input_data` & `decoder_input_data` into `decoder_target_data` model = Model([encoder_inputs, decoder_inputs], decoder_outputs) # Compile & run training model.compile(optimizer='rmsprop', loss='categorical_crossentropy') # Note that `decoder_target_data` needs to be one-hot encoded, # rather than sequences of integers like `decoder_input_data`! model.fit([encoder_input_data, decoder_input_data], decoder_target_data, batch_size=batch_size, epochs=epochs, validation_split=0.2) This is a very simplified example and doesn't include many important steps such as data preprocessing, model evaluation, or the deployment phase. Also, it assumes you have a parallel corpus of English and French sentences. LLM Prompt : Equivalent "Translate 'Hello' to French." Speech Recognition: Creating a speech recognition system from scratch is a complex task that involves deep learning and a large amount of data. It's typically done using Recurrent Neural Networks (RNNs), a type of model that's ideal for sequence data like audio. Here's a simplified example of how you might start to approach this task using Python and the Keras library. This example doesn't include the training, testing, and deployment phases, as each of these would require a significant amount of code and computational resources. # Import necessary libraries from keras.models import Model from keras.layers import Input, LSTM, Dense import numpy as np # Parameters for our model batch_size = 64 epochs = 100 latent_dim = 256 num_samples = 10000 # Placeholder for input data input_audios = [] target_texts = [] # Placeholder for character data input_audios_characters = set() target_characters = set() # Load your data here # This could be a parallel corpus of audio and transcript # Vectorize your data # Define an input sequence and process it encoder_inputs = Input(shape=(None, num_encoder_tokens)) encoder = LSTM(latent_dim, return_state=True) encoder_outputs, state_h, state_c = encoder(encoder_inputs) # We discard `encoder_outputs` and only keep the states encoder_states = [state_h, state_c] # Set up the decoder, using `encoder_states` as initial state decoder_inputs = Input(shape=(None, num_decoder_tokens)) # We set up our decoder to return full output sequences, # and to return internal states as well. We don't use the # return states in the training model, but we will use them in inference decoder_lstm = LSTM(latent_dim, return_sequences=True, return_state=True) decoder_outputs, _, _ = decoder_lstm(decoder_inputs, initial_state=encoder_states) decoder_dense = Dense(num_decoder_tokens, activation='softmax') decoder_outputs = decoder_dense(decoder_outputs) # Define the model that will turn # `encoder_input_data` & `decoder_input_data` into `decoder_target_data` model = Model([encoder_inputs, decoder_inputs], decoder_outputs) # Compile & run training model.compile(optimizer='rmsprop', loss='categorical_crossentropy') # Note that `decoder_target_data` needs to be one-hot encoded, # rather than sequences of integers like `decoder_input_data`! model.fit([encoder_input_data, decoder_input_data], decoder_target_data, batch_size=batch_size, epochs=epochs, validation_split=0.2) LLM Prompt : Equivalent "What is being said in this audio?" Anomaly Detection: Creating an anomaly detection system from scratch is a complex task that involves machine learning and a large amount of data. It's typically done using algorithms like Isolation Forest, One-Class SVM, or Autoencoders. Here's a simplified example of how you might start to approach this task using Python and the Scikit-learn library. This example doesn't include the training, testing, and deployment phases, as each would require a significant amount of code and computational resources. # Import necessary libraries from sklearn.ensemble import IsolationForest import numpy as np import pandas as pd # Parameters for our model outliers_fraction = 0.01 num_samples = 10000 # Placeholder for input data input_data = [] # Load your data here # This could be a dataset of network traffic, financial transactions, etc. # Convert your data to a pandas DataFrame df = pd.DataFrame(input_data) # Define the model model = IsolationForest(contamination=outliers_fraction) model.fit(df) # Predict the anomalies in the data df['anomaly'] = model.predict(df) # Print the anomaly prediction for each data point for index, row in df.iterrows(): print(f'Data point {index}: {"Anomaly" if row["anomaly"] == -1 else "Normal"}') LLM Prompt : Equivalent "Are there any anomalies in this data?" Recommendation Systems: Creating an anomaly detection system from scratch is a complex task that involves machine learning and a large amount of data. It typically uses algorithms like Isolation Forest, One-Class SVM, or Autoencoders. Here's a simplified example of how you might start to approach this task using Python and the Scikit-learn library. This example doesn't include the training, testing, and deployment phases, as each of these would require a significant amount of code and computational resources. # Import necessary libraries from surprise import SVD from surprise import Dataset from surprise.model_selection import cross_validate # Load the movielens-100k dataset data = Dataset.load_builtin('ml-100k') # Use the SVD algorithm algo = SVD() # Run 5-fold cross-validation and print results cross_validate(algo, data, measures=['RMSE', 'MAE'], cv=5, verbose=True) # Train the algorithm on the trainset trainset = data.build_full_trainset() algo.fit(trainset) # Predict a certain item userid = str(196) itemid = str(302) actual_rating = 4 print(algo.predict(userid, itemid, actual_rating)) This is a very simplified example and doesn't include many important steps such as data preprocessing, model evaluation, or the deployment phase. Also, it assumes you have a dataset of network traffic, financial transactions, etc., which is not included here. Creating an anomaly detection model from scratch is a complex task that typically requires a deep understanding of machine learning and statistical analysis, as well as access to a large amount of data and computational resources. LLM Prompt : Equivalent "What items would this user like?” Predictive Analytics: # Import necessary libraries from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score import pandas as pd # Load your data here # This could be a dataset of features and labels data = pd.read_csv('data.csv') # Split the data into features and labels X = data.drop('label', axis=1) y = data['label'] # Split the data into training and testing sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) # Define the model model = RandomForestClassifier(n_estimators=100) # Train the model model.fit(X_train, y_train) # Make predictions on the testing set y_pred = model.predict(X_test) # Print the accuracy of the model print("Accuracy:", accuracy_score(y_test, y_pred)) This program first loads a dataset of features and labels, then splits this data into a training set and a testing set. It defines a RandomForestClassifier model, trains this model on the training set, and makes predictions on the testing set. It then prints the accuracy of the model. LLM Prompt : Equivalent “What will happen next based on this data?" Natural Language Processing: # Import necessary libraries import nltk from nltk.tokenize import word_tokenize from nltk.corpus import stopwords from nltk.stem import PorterStemmer from gensim import corpora, models import string import os # Load your data here # This could be a directory of text documents directory = 'documents/' # Placeholder for the documents documents = [] # Read the documents for filename in os.listdir(directory): if filename.endswith('.txt'): with open(os.path.join(directory, filename), 'r') as file: documents.append(file.read()) # Tokenize the text tokenized_text = [word_tokenize(doc) for doc in documents] # Remove punctuation table = str.maketrans('', '', string.punctuation) tokenized_text = [[word.translate(table) for word in doc] for doc in tokenized_text] # Convert to lower case tokenized_text = [[word.lower() for word in doc] for doc in tokenized_text] # Remove non-alphabetic tokens tokenized_text = [[word for word in doc if word.isalpha()] for doc in tokenized_text] # Remove stop words stop_words = set(stopwords.words('english')) tokenized_text = [[word for word in doc if not word in stop_words] for doc in tokenized_text] # Perform stemming porter = PorterStemmer() tokenized_text = [[porter.stem(word) for word in doc] for doc in tokenized_text] # Create a dictionary representation of the documents dictionary = corpora.Dictionary(tokenized_text) # Convert the list of documents (corpus) into Document-Term Matrix using dictionary prepared above doc_term_matrix = [dictionary.doc2bow(doc) for doc in tokenized_text] # Create an object for LDA model Lda = models.LdaModel # Train LDA model on the document term matrix ldamodel = Lda(doc_term_matrix, num_topics=3, id2word = dictionary, passes=50) # Print the topics print(ldamodel.print_topics(num_topics=3, num_words=3)) This program first loads a directory of text documents, and then tokenizes each document into individual words. It removes punctuation, converts to lowercase, removes non-alphabetic tokens, removes stop words, and performs stemming. It then creates a dictionary representation of the documents and converts this into a document-term matrix. It creates an LDA model and trains this model on the document-term matrix. It then prints the topics identified by the model. LLM Prompt : Equivalent "What is the meaning of this text?" As you can see, the code for each task involves several steps and requires a certain level of technical expertise. On the other hand, the LLM prompts are simple English sentences that anyone can understand and use. Python This is the power of LLMs: they democratize access to all types of technology available on the Internet, making it accessible to everyone, not just those with technical skills. Conclusion I said earlier that Llama2 was a new era in human existence. Now I can confidently say, The World Has Changed. It will never again be the same. Engineering and Specialized Skills Can Now Be Done using Plain English! t The echnology landscape is evolving faster and faster. What will be the skills that won’t be replaced by robots or automated? There is a fundamental misunderstanding here. We assume we will be replaced by our creations when they become sentient. Now that is still an open question, but now, I can confidently say: Artificial intelligence will not replace us but complement us. They will transform the human race forever. Don’t think of it as a risk to your career. Think of it as an addition to your skill set and an improvement in your scientific, technological, and artistic capacity. Adapt, Don’t Give Up! Learn to use LLMs. And be happy. Because it’s so easy that a child can do it! All Images Created Using the Bing Image Creator by the author. To Learn Some More About Transformers, you can check out this article: https://hackernoon.com/the-transformer-neural-network-tnn-is-much-much-bigger-than-even-agi?embedable=true Beloved readers, if you’re reached the end, pat yourself on the back - it’s an achievement! The Stay cool - get into AI right now! (my advice for anyone reading this article). next article will be on Quantum Computing and GPT-4!