How does Chat GPT 3.5 work?

Hey friends, Happy Wednesday!

Let’s look at how Chat GPT 3.5 works this week. I aim to write my newsletter issues in a way one can follow them while traveling on a bus, having a coffee, waiting for food, etc. Let's jump in!

How it works: 15-second answer

ChatGPT learns a diverse range of information across the internet without intervention from anyone. Following training and testing by thousands of humans, who rank and rate their responses giving feedback, Chat GPT is now operational. However, her static knowledge needs manual updates to stay current with new information.

How does Chat GPT work?

I’m using metaphor to explain Chat GPT 3.5 so we don’t have to go into any jargon or equations.

Internet - The Library

Imagine a vast library near your home that has all the books, papers, blog posts, and Wikipedia articles ever published. Now, think about having a single librarian capable of managing all this information. Impossible feat right?

What if humans could find and train such a librarian?

Enter Chat GPT.

ChatGPT - The Librarian

The library director discovered a unique librarian called Amelie. (Amelie is Chat GPT in this metaphor :D)

She lacks consciousness, emotions, feelings, or self-awareness. Also like a newborn baby who knows nothing about the world.

But the director aims to teach, train, and test her before she can be given a job. Being dumped with all the pieces of text information that they have in the library, she goes through them quickly and thoroughly, forming patterns in parallel.

Learning Information

Consuming the good and bad:

No one supervises Amelie during this process. So she is exposed to all kinds of information- medical advice, a recipe for Thai green curry, quantum physics, psychology, and even potentially harmful topics like building an atomic bomb, guns, etc.

Amelie, having consumed every possible piece of information, also learned some incorrect information. For example, she thought there were 9 planets in the solar system when it’s only 8.

Forming connections between words:

Also while going through the resources, she noticed that “Barack Obama” and “President” occurred more often together. So she started forming patterns and connections between words that go together more often. There is a smaller correlation between “heart” and “player” than “heart” and “disease”. Likewise, Amelie went through all the information she was given and formed patterns and connections.

Testing and correcting Amelie (Chat GPT)

Humans giving their feedback:

It’s time to test the librarian before she gets the job. Thousands of humans were hired by the library to test Amelie. Each of them came in and asked her questions about different topics.

She was asked to give different responses to the same question, which the humans rated and ranked. The right answers earned cash rewards as well. After receiving this feedback, Amelie corrected herself.

*Figure 2: Humans ranking and rating the responses*

Avoids giving harmful information:

Through this process, she also learns to avoid giving harmful information to the customers by denying the sharing of information. Helpfulness, Harmlessness, and Truthfulness were some of the qualities instilled in her in this process.

Amelie gets the job and starts working

Now that she is trained and tested, the director approves of her abilities. Then, she got the job and they opened the library.

Answering questions:

Customers started coming in, asking questions. She understands them by looking at the words and mapping them to her trained knowledge base. Now she answers them by finding the next best word (calculating the probability) that fits in the sentence. And continues to place one word next to another at a time until she completes the response. She follows correct grammar as she has been trained with sentences that have the right grammar.

*Figure 3: Chat GPT answering a question. Source: ByteByteGo*

Remembers the context:

She knows the context. So if you’re asking her “Who is Barack Obama?”. And in the very next question, if you ask “What is his age?”. She’ll answer based on the previous context to give you the age of 62. You can always continue the “same” conversation and she’ll remember the past responses.

Static knowledge:

Though she is in the library, she does not read the new books out of her interest. The library officials have to give her the resources and train her again before she can be ready to serve the customers by answering questions based on the new information.

This is how Chat GPT works in a very basic sense!

Question of the week

What is the total size of data that Chat GPT 3.5 consumed and learned to become what it is today? Reply to this email with your thoughts, and we’ll discuss. I’ll answer this in next week’s newsletter issue.

Gadget of the week

Physical keyboard for an iPhone: The Clicks case offers a built-in QWERTY keyboard for iPhones, preserving screen space by keeping the virtual keyboard hidden until needed. With added height, a stylish design, and backlit keys, it supports Mac and iPad keyboard shortcuts. It is compatible with iPhone 14 Pro, iPhone 15 Pro, and iPhone 15 Pro Max.

That's all folks. Thank you for reading!

Have an amazing rest of the week, and take care!
Until next to next Wednesday,
Chendur