Thoughts on ChatGPT

If you're not sick of hearing other people's opinions on it yet...

The chart above shows the staggering adoption rate of ChatGPT. This unbelievably impressive technology has skyrocketed in it’s user base in very short amount of time. It caught everyone off guard when it launched and represented a massive step change in machine learning, which was already on the bleeding edge of “new”.

I’ve been incorporating ChatGPT into my work where possible now for several months and have spent enough time with it to form some opinions, which I thought I’d share.

“Google Plus”

No, Google+ is not still around, don’t get confused. In fact, I asked ChatGPT just to confirm.

To people who have asked me, ‘What exactly is ChatGPT?’ or ‘What is ChatGPT useful for?’ The easiest way I’ve found to answer this is to call it ‘Google Plus’.

In the days before ChatGPT (you know, a few months ago), all of our research was carried out through search engines. In those days, you could have easily imagined Google rolling out a brand new premier service that promised to take whatever you searched for and perform all the necessary steps to return a complete, summarized answer for you, and one that didn’t require you to review a dozen websites and piece together information yourself.

At its core, this is essentially what ChatGPT does. It’s an enhanced version of ‘search’ that directly compiles its results for you. This is why Google themselves were so worried when ChatGPT initially came out and have since scrambled to roll out their own version called Bard.

ChatGPT is what is known as a large language model (LLM), which is essentially a very complicated neural network containing many billions of parameters that have been trained on massive volumes of text from the internet. This means it captures the knowledge of the internet (good and bad) and can understand plain English input from its users.

If we take the example above, I asked ChatGPT, ‘Is Google+ still around?’ Not only did it return the answer (no), but it also gave me a bunch of additional, very helpful information about when it was halted. Most importantly, it didn’t give me a list of websites I needed to look through to piece this information together. Granted, I could have easily found this information myself, but it would have taken time and effort.

The above is obviously a trivial example, but this same concept scales quite well, and I’ve been very surprised by just how detailed of questions you can ask ChatGPT, and amazingly, it will very often be capable of returning useful results to you.

Here’s what happens when we look at something more relevant. A hot topic at this exact moment is the debt ceiling and whether Congress is going to reach an agreement on how to handle it. The kind of prompt below is something a journalist working on a new article may be interested in asking ChatGPT.

Just imagine how long this answer above would have taken to produce on your own and then you can start to see where ChatGPT’s value comes in and why it’s all over the news.

But it gets more interesting

ChatGPT is called ‘Chat’ GPT because it is meant to be interactive. If you ask it about something and then want it to continue to dive in deeper, you don’t need to start all over. It’s aware of your chat history and can pick up where you left off. It was designed this way by Open AI, the creators of ChatGPT, intentionally as they felt this was likely to be the optimal way to interact with the model.

Here’s what happens when you ask ChatGPT for its opinion on which was the worst example mentioned above.

When something is built in such a way to allow this kind of ongoing interaction it provides a much different user experience and one that is more intimately familiar to humans. It’s clearly an early step towards what interactions could look like with true artificial intelligence.

How am I using it the most right now?

The examples above start to illustrate the general functionality and workflow of ChatGPT. Of course, the best way to understand what it’s capable of is to start trying it for yourself.

The people who will (and are) immediately benefiting the most from ChatGPT are those who regularly retrieve information from online sources. This includes a very wide range of specialties, from writers to historians. One group in particular that has found a high amount of use from ChatGPT is programmers. The reason for this is simple: just ask any programmer how much time they spend on Stack Overflow, and they’ll probably tell you they always have a tab open in their browser with some kind of content from it. Well, now ChatGPT is capable of answering those same questions for them without all the detailed searching.

I have generally found myself falling into the last category above. As someone focused on making sense of data, I’ve found ChatGPT incredibly useful in helping me produce code in various languages that I may be less familiar with, as part of the process in performing operations on data.

Here’s an example mimicking a scenario where I was looking for the correct syntax to filter grouped data in a pandas DataFrame. And yes, here you’ll notice that ChatGPT’s responses don’t have to be purely word-based; it has been trained across a vast array of code bases (mainly through GitHub), and as a result, it understands the syntax for many programming languages quite well.

What can’t it do?

There are a lot of concerns that ChatGPT (and competitors) are going to make many people’s jobs irrelevant. It’s easy to see where this fear comes from and in some cases it’s likely true. This technology is brand new, so it’s difficult to say exactly where it’s going to go in future versions. At this point at least, ChatGPT doesn’t immediately “plug into” much of anything and isn’t readily used to automate processes that humans perform. It requires an input to be supplied and only then will it produce an output. This should make many people feel somewhat more comfortable.

In addition to not directly interacting with processes, some of the areas where ChatGPT is not well suited include:

• Current or future events - It can’t write about most current events. This is a limitation of how the models are trained in that only past data are used to train the model, thus they are only aware of past events or things which have already been documented.

• Math - As mentioned further above, ChatGPT is a large language model. These models are trained on language data (in the case of ChatGPT nearly the entire internet!). But this means that if the answer to “2+2” isn’t discussed quite often through the volume of text it was trained on, it will be unlikely to produce a correct result for you. Or worse, it will produce a result that is wrong, but will make you think it’s confident that answer was correct. By the way, this is referred to as “hallucinating” and here’s an example of it below. The steps here to produce a linear model are ok, but the coefficient and y-intercept are completely wrong!

• Solve complicated business challenges. This one is probably the most important. ChatGPT is a sophisticated reference, but it’s still not capable of understanding complicated multi-step processes or problems that humans face everyday. This is ultimately the reason I am more optimistic about ChatGPT than pessimistic.

Final thoughts

As a data scientist, I’m astonished by the technical achievement of ChatGPT. As a user, I find myself reaching for it more and more. The future is always uncertain, but at this point, I don’t worry that ChatGPT will make everyone’s job’s obsolete. For most people, I think this will be an enhancement to your work helping you be more efficient at finding and utilizing information, ultimately solving the much bigger problems you are faced with. Think of it like Google on steroids.

We started Pontem Analytics to help serve as a liaison between data and discipline. In the broader sense, one of the things that I think ChatGPT has done is open people’s eyes more to Machine Learning. This is important because outside of these large language models, there are still many problems that need solving. More “traditional” problems that rely on combining deep domain expertise with data. In those situations, machine learning may or may not be the best approach, but in the cases where it is, ChatGPT has already helped to make machine learning a more approachable topic, and I think that is a good thing.

So, in conclusion, give ChatGPT a try and see where your own experiments take you. I think you’re likely to find a few use cases for yourself, but either way I can almost guarantee you’ll be impressed.