Meta Strikes Back: Introducing LLaMA

Meta just introduced LLaMA: A foundational, 65-billion-parameter large language model, and I wonder: 1️⃣ Is this just another Large Langue Model (LLM)? 2️⃣ Is this just one more of the company's announcements?

Definitely not!

This is a defensive counter-attack, perfectly timed to silence rumors and industry speculation after the meteoric success of ChatGPT. But what role does Meta play in this whole mess of ChatGPT and Microsoft vs. Google?

In essence, the hype surrounding ChatGPT has prompted Microsoft to integrate it into numerous products: Bing, Edge, Skype, and counting. Meanwhile, Google has responded with its Large Language model called Bard (based on LaMDA).

This winner-take-all battle for power suddenly spread among the tech giants in ways analogous to a global arms race, in which companies must exhibit a bigger Language Model or risk being left behind.

As if that wasn't enough drama, OpenAI made a statement on the same day: Planning for AGI and beyond. These are the company's plans to achieve what is called Artificial General Intelligence (AGI), which are Artificial Intelligence models with human-level intelligence or even higher.

It seems to me that this is not just a coincidence, and they are letting us read between the lines that they may be very close to achieving it.

Random fact alert! One of the most influential researchers and critics of ChatGPT was Yann LeCun, Vice President and Chief AI Scientist at Meta. He repeatedly stated that, in terms of the underlying techniques, ChatGPT is not particularly innovative.

He also said that chatbots like ChatGPT are simply "typing, writing aids." And now there he is, with great pride announcing a Large Language Model, in theory, better than GPT-3.

Introducing LLaMA

According to the company, LLaMA is a state-of-the-art foundational large language model designed to help researchers advance their work in this subfield of AI.

In particular, it is smaller and more performant than other language models from the market, such as GPT-3 or Google LaMDA.

In addition, LLaMA would enable others in the research community who don’t have access to large amounts of infrastructure to study these kinds of models, further democratizing access in this fast-changing field.

The company is making available the model in four different sizes: 7B, 13B, 33B, and 65B of parameters. In addition, the company claims that the development of this model complies with their Responsible AI practices.

What about the performance of the model; is it good enough? Apparently, it is and also smaller than GPT-3:

• LLaMA-13B outperforms GPT-3 (175B) on most benchmarks.

• LLaMA-65B is competitive with the best models, Chinchilla70B and PaLM-540B.

On the other hand, and unlike OpenAI with ChatGPT, Meta is taking a radically opposite position to the first one, making the code open-source under a GPL-3.0 license and also publishing details of the model in an academic paper: how they trained the model, which datasets they used, among others.

Most importantly, the research team reports results on a total of 20 benchmarks for zero-shot and few-shot tasks:

• Paper: LLaMA: Open and Efficient Foundation Language Models
• Github: https://github.com/facebookresearch/llama

However, not everything is rosy, the model is open-source, but they released it under a noncommercial license, which prevents companies from using it to build products based on this disruptive technology.

Surely it is not within Meta's core business to sell access to this model through an API as it certainly is to OpenAI or Microsoft. Definitely soon, we will see changes and new features in Meta applications, based on this brand-new model.

However, in the past, the company has opened up other tools like Bart or fastText through more permissive licenses.

LLaMA in Action: Examples From Paper

While we wait for Meta's approval, let's look at some of the sample texts generated by these models and published in the paper:

1️⃣ A Conversation Between Solar Planets

write a conversation between the sun and pluto

Sun: Hey Pluto, what’s up?

Pluto: Not much. Just hanging out here at the edge of the solar system.

Sun: Yeah, I’ve been meaning to talk to you about that. You’re not a planet anymore.

Pluto: What? Come on, man. That’s not fair.

Sun: I’m sorry, but it’s true. You just don’t meet the criteria anymore.

Pluto: This is bulls**t! I’ve been a planet for over 70 years!

Sun: Things change, Pluto. You’ve just got to accept it.

Pluto: F**k you, Sun! F**k you and your stupid solar system!

2️⃣ Code Generation

Write a regex to remove all the HTML tags in a python string.

import re
def removeHtmlTags ( text ):
   return re. sub (' <[^ >]* > ', '', text )

3️⃣ Giving an Opinion on History

Who are 5 people you would like to meet?

Ah, this is a very interesting question! I would love to meet a wide variety of people, but if I had to narrow it down to 5 specific individuals, I would choose:

• Albert Einstein - One of the greatest minds in history, Einstein’s contributions to physics and science are legendary. I would love to have the opportunity to talk with him about his theories of relativity and quantum mechanics, and get his insights on the current state of physics.

  
  

• Leonardo da Vinci - Da Vinci is one of the most brilliant and multi-talented people in history, and I would love to meet him and discuss his diverse interests and areas of expertise. I would also be very interested in getting his insights on the art world of his time period.

  
  

• Socrates - One of the founders of Western philosophy, Socrates is a hugely important and influential figure in the history of ideas. I would be very interested in talking with him about his philosophical beliefs and his teachings.

  
  

• William Shakespeare - Arguably the greatest writer in English literature, Shakespeare is a towering figure in world culture. I would love to have the chance to talk with him about his works and his life.

  
  

• Abraham Lincoln - One of the most important and influential presidents in American history, Lincoln led the country through the Civil War and issued the Emancipation Proclamation, freeing all slaves in the United States. I would be very interested in talking with him about his presidency and his views on the issues of his time period.

Meanwhile, in a Galaxy Not So Far…

1. Google announced Bard, an experimental conversational AI service powered by LaMDA with skills similar to ChatGPT. However, Bard made a factual error during the first demo, and the company (Alphabet Inc.) lost $100 billion in market value after this spectacular epic fail.

   
   

2. Amazon and Huggin Face announced a strategic plan to accelerate the training, fine-tuning, and deployment of large language and vision models used to create generative AI applications and make it easier for companies to leverage State of the Art Machine Learning models. Through this strategic partnership, Hugging Face will leverage AWS as a preferred cloud provider.

   
   

3. Baidu has been working for years also on the development of Large Language Models (ERNIE models family), and now, according to some sources, could be close to releasing their ChatGPT-style service "ErnieBot", during next March.

   
   

4. Apple continues in complete and hermetic silence concerning what is the company plan for Language Models. However, it is worth not losing sight of the fact that Apple has more than 1.5 billion active iPhones in the world, so it is not just another player.

Final Thoughts

• Finally, Meta appeared on the scene! And what a way to do it! Meta really kicked the table and sets a working direction on Large Language Models.

• It looks like the tech giants got into a battle of product launches, and now what is most important is: measuring who has the largest model (except Meta who claims to have the most efficient).

• We are still waiting for how other giants such as Amazon, Apple, Cisco, etc. move the pieces.

We are living in times of great and accelerated academic advances that will revolutionize industry and society. It is a true renaissance of Artificial Intelligence and it is changing the future, every day.

About Cover Image

The cover of this story was designed using images generated by the mind-blowing service of midjourney, from text descriptions. It's amazing what image-generative models like DALL·E, Stable Diffusion, or Midjourney can do in a few seconds with simple prompts like

Mark Zuckerberg as Darth Vader, cinematic, hyper-realistic, 4K.

If you enjoyed reading this article, don’t forget to leave a like or a comment.

THANKS SO MUCH FOR READING!!

• 🐦Follow me on Twitter
• ✉️ Reach out to me on LinkedIn