LLM Basics I

Hey - It’s Michael.

Had the nicest kitesurfing session with my buddy David the other day. Enjoy the newsletter!

Almost anyone can learn to drive a car, but few people understand how cars really work. Most racecar drivers on the other hand have a pretty deep understanding of what’s happening in their worktool.

The analogy is of course that, the better you understand the technology you are using, the more you can get out of it.

LLMs will inevitably affect most knowledge workers, so to use it well, keep the basics in mind.

Understand the core functionality of LLMs [= Large Language Models] that are used in tools like ChatGPT, Claude or Gemini to use them effectively:

Essentially, LLMs are parrots trained on billions of sentences from books, websites & chats. They predict what words come next - and seem to talk like humans.

AI is not intelligent like humans are.

Imagine LLMs like a giant web of switches, stacked in layers. Each layer tweaks the meaning of words fed into it, spotting patterns from endless examples.

So simplified, the training of these models works like this: Feed the LLM trillions of words with blanks - like: “The sky is ___.” It guesses “blue”, checks if wrong and finally tweaks the switches to get better.

So there are no facts stored anywhere.

An LLM just has a hunch of what word might fit next.

And this is what it will produce.

This of course means that you must be aware of what you ask because LLM tools will always give an answer.

And they’ll give the answer with such confidence that they’ll look very convincing. Even if the answer is not true.

Keep in mind: for topics with lots of training data, LLMs work well. For niche topics they do not.

Here are 2 simple examples to illustrate:

There are thousands of physics books in a variety of languages. So when you ask an LLM to explain a basic physics concept you’re most likely going to get a really good answer.

There are only a handful books and very little documented material on Lindy Hop - a Swing Dance I fell in love with many years ago. Essentially it’s being taught by attending in-person classes of local & international teachers that carry the roots of this dance forward. Since there is rather little written information and the dance is obviously taught in person, many answers LLMs give about the dance are simply wrong.

Principle:

Use LLMs only for topics with reliable training data.

So before you use LLMs, take a second to estimate whether it is really able to give you the answer you are looking for.

“Confidence is a feeling, which reflects the coherence of the information and the cognitive ease of processing it.” - Daniel Kahneman

See you next week - Michael