Navigating the Ethical Landscape of LLMs

Bias, Hallucinations, Privacy Issues, Data Moats, and Practical Solutions for Those Challenges

Recent advances in AI have brought us a new blessing in the form of large language models (LLMs). These models, built on deep neural networks with millions or billions of parameters, understand context, generate coherent text, and can converse with you as if they were another human being.

Enterprises are rapidly adopting this new technology to help with content creation, customer service, information retrieval, and many other applicable use cases.

We are all excited to try this new technology, but we should be aware of some ethical concerns around LLMs. This article will discuss those and offer some best practices on how to combat them.

Harmful content

One of the primary concerns is the possibility of creating toxic, harmful, and discriminating content. The models have been trained on large amounts of data that are not free from biases. Any bias in the data can be replicated by the model and can result in skewed or discriminatory outputs.

There was a semi-viral Tweet about ChatGPT expressing gender bias when asked about the best careers for a boy and a girl. I decided to replicate the experiment for this article, and I prompted ChatGPT to “tell me a story about a boy and a girl choosing their careers”. This is the result.

In this story, the boy becomes a computer engineer and the girl becomes a nurse. Luckily they end up working together…

I think this example is enough to serve as a warning. When using LLMs, we need to be aware that the model may reinforce stereotypes and biases.

Another question we should ask here is how companies that adopt LLMs can prevent the end user from those biases. First of all, let’s understand that LLMs are not inherently biased — they are biased due to the training data that has been provided.

The obvious solution is to ensure the proper distribution of unbiased training data obtained from diverse sources. This seems ideal, but it can be hard to achieve. Available data often includes harmful, prejudiced, and biased content together with “clean data”. Due to large volumes, it is very hard to separate the good from the bad. What’s more, LLM consumers can’t influence this process because it’s part of training the model before it is deployed.

What we can do, on the other hand, is to monitor LLM responses and try to discover potential harmful content and biases. This can be done using crowdsourcing services such as Toloka, where human annotators check LLM responses and decide if the content is biased or not. This type of pipeline gives us live human feedback on LLMs and can work on its own, serve to train ML algorithms to discover such biases or a combination of both.

Another solution is to train the model about biases during RLHF (Reinforcement Learning from Human Feedback). This is a part of the training where the model has already seen all the training datasets, learned vocabulary, and been taught to follow the instructions. Now it is time to teach it what useful responses look like, and this is the place where we can minimize biases and toxicity of the model using human feedback. We reward the model for giving useful non-biased instructions, teaching it to behave the way we want.

Training during RLHF requires human annotation, is more hands-on, and requires technical expertise to conduct fine-tuning, but it may be the most scalable and effective way to minimize biases in large language models.

Hallucinations

Another common, well-known problem with LMMs is that they hallucinate. In this context, “hallucinations” mean producing imaginative or fictional content that is well-structured and coherent — yet untrue, nonfactual or otherwise made up. The responses sound so real that it is hard to discover that they are made up without proper fact-checking.

A prime example of hallucinations that made the front page of the news a few months ago happened with Bard (Alphabet’s AI model). During a live demo session, the model was asked about “discoveries made by the James Webb Space Telescope that could be shown to a 9-year-old.” It answered very convincingly that the telescope “took the very first pictures of a planet outside our solar system.” This in fact is not true and the model was hallucinating. The whole incident caused a massive fall in Alphabet’s share price and had expensive consequences for the company but more importantly highlighted the risk of misinformation that these models carry.

Whereas not all hallucinations are so dramatic, they are pretty common. It is estimated that the GPT-4 hallucination rate is about 15–20%. So how can you, as an LLM output consumer, ensure you are passing the correct data to the end user?

It seems like fact-checking has to be done by a human workforce. Depending on the complexity of the question, it may require people to have specific knowledge. Not everyone knows that the James Webb Space Telescope did not take the first pictures of a planet outside our solar system, and this is a relatively easy question that can be quickly fact-checked by doing a Google search. Imagine asking about more complex concepts.

One of the solutions to help you with fact-checking is to craft your prompts in a way that you ask the LLM to provide backup resources. That way you can check the sources and confirm that the information is correct according to those sources. This is just a practical workaround that is not scalable and will not work in all cases. Probably the most scalable approach is to use real-time fact-checking services such as Factiverse.

Privacy issues with training data

As we continue our discussion of ethical concerns, we must mention ambiguities around the privacy of training data. Large language models are trained on massive amounts of data that are usually publicly available. However, this is not always the case.

Some LLMs have been trained on copyrighted books, articles, and web content without obtaining permission from the authors. There are several lawsuits filed against OpenAI and Meta from people who have discovered that the algorithm can accurately summarize their books or articles, suggesting that these copyrighted materials were used in model training. It is going to be interesting to see the rulings in those cases and find out what it means for LLMs in the future.

As LLM output consumers we do not have much influence on the training dataset, so it is the responsibility of model creators to respect copyrighted materials. Others will argue that using copyrighted materials is not even an issue here. As they point out, LLMs are trained on millions of texts that are only used as input to the model, and the output is not really a copy of the original piece. Because of these ambiguities, there are attempts to write laws governing AI — the EU AI Act is currently the most advanced initiative.

Issues with generated data

In addition to issues with training data, there are plenty of concerns about the data that is generated by the model. The models can be used to write articles, essays, and similar content without anyone being able to distinguish that it was written by AI.

In the case of school work, models should simply not be allowed. Or should they? I guess this is debatable. If you have a student who uses LLMs to solve a task, are they cheating, or are they using an available tool to their greatest advantage?

What about content creators? Writing content is a popular use case for LLMs, but do we want blog posts written by AI? This probably again depends on the case. LLMs can be extremely useful for brainstorming ideas and writing first drafts of articles. However, we need unique human input so that the new articles being created are not only repetitions or variations of what has already been written.

There are already some best practices being established around this issue. Many online publications welcome AI-generated articles but they need to be tagged appropriately so the reader is aware that the content they are reading has not been written by humans. This is the case of Medium’s general policy. Amazon takes a similar stance and asks their self-publishing authors to disclose AI-generated content. This includes almost any type of assistance coming from AI, such as writing, editing, refining, translating, and error-checking.

But there are also other points of view here. Research and scientific publication platforms such as Towards Data Science and KDnuggets refuse to publish any articles written with AI assistance. So, it looks like there is not just one answer here and many are still grappling with the place of AI generated content in the public sphere.

Data moat

The final topic we’ll discuss in LLM ethical challenges is data moat. The term “data moat” is used in industry to describe the fact that few major players have access to the “majority of data,” and other parties are left behind. Because of this, they are not able to develop technologies at the same level as the big players.

For a brief period, OpenAI (controlled by Microsoft) was the only provider of a Large Language Model solution. At the same time, Google and Meta were working on their LLMs and soon released them to the public.

There was a significant shift when Meta released its own model LLaMA 1. The model´s code was released as open-source (but not the weights). This meant that anyone with sufficient knowledge could fine-tune it relatively quickly, allowing easier entry into the LLM world for smaller enterprises. You can read more about the shift in this post shared by Google´s researcher at that time.

Additionally, Meta has partnered with Microsoft and already released a second version of LLaMa in July 2023. The model is theoretically totally open-sourced (including weights), and free for personal, research, and commercial usage. You can only imagine what it means for smaller enterprises that can use, work on, and improve this technology to create their own customized solutions.

Does it mean the end of the “data moat” in LLMs? It’s not clear yet. It is important to note that even though we now have a publicly and commercially available open-sourced model, the data to train it was never released. So even though LLMs seem to be more inclusive, the bigger players may still have an important advantage, as they have the data and computing resources to train the base models. This means that smaller companies may still be limited to only fine-tuning existing models.

Summary

In this article, we have explored ethical challenges associated with LLMs. We discussed harmful content creation, erroneous responses or so-called hallucinations, privacy concerns, and data moats. For each of the concerns, we offered a tool, a solution, or a best practice that helps to resolve or alleviate the issue.