What Marketers Should Know About GPT-3

This article was written in 2021, way before GPT-3 became a household name.

Carl (one of my first-ever clients) called yesterday. He has a massive privately-held business from which he’s allegedly retired. His sons, whom I not-so-affectionately refer to as Les Petites Princes, now (ostensibly) run the company daily. Yet, instead of Carl asking them about, well, anything, he still likes to call me to discuss whatever he read in WSJ or Financial Times as if I’m his personal Google. (I adore Carl. I’m ok with it. If I was him, I’d probably have imitated Hercules but alas…)

The other day his question was about GPT-3. He asked what it was many moons ago, but he’s been reading about it ever since and had a very long (read: seemingly never-ending) list of questions beginning with “Why aren’t you telling the boys about this? We should be using this right now.” I didn’t have the heart to tell him that I’d rather have a lobotomy than do more work with his offspring, so I went through his list of questions. I’d give you the transcript of the call, but it wasn’t recorded by either of us anyway. (We’ll know if The Machine has it when I get personalized ads about his products, Saint-Exupery, or the goddess Hera.)

First though…

 

WHAT IS GPT-3?

GPT-3 stands for Generative Pre-trained Transformer 3. Quite the mouthful, eh?

Designed by OpenAI, GPT-3 is a language prediction model that uses deep learning to create human-like text.

Bottom Line: GPT-3 writes text that sounds like a human wrote it. (Kind of.)

 

THEY SAY IT USES TRANSFORMERS. LIKE THE ELECTRICAL THING OR THE ACTION FIGURES?

Yes, he really asked about the action figures, but only so I could predictably say, “Yes, Optimus Prime is now straight-up running the world.”

Transformers are a type of neural network. Neural networks mimic the thought patterns of the human brain – thus, the “neural” part. Transformers evaluate the words in a sentence to predict what’s next. (Transforming an input sequence into an output sequence.)

You see this deep learning model used in Computer Vision and Natural Language Processing (NLP.) It gets the most press for NLP, primarily because of how it impacts content generation.

 

WHEN VENDORS LIKE X AND Y SAY THEY USE ARTIFICIAL INTELLIGENCE TO WRITE TEXT, ARE THEY REALLY JUST WHITE-LABELING GPT-3?

Many vendors will tell you whose model they’re white-labeling and what they’ve done with it. Some of the smarmy ones don’t disclose and/or pretend they’ve developed all the technology. That’s why when you’re vetting a new AI vendor, you need to ask a lot of questions.

WHAT CAN GPT-3 HELP MARKETERS DO?

Generate content (everything from a single headline to War and Peace-sized books.)

Write code

Classify and summarize text

Translate languages

SEO/Social

Design websites (kind of)

Right now, GPT-3 writes text that sounds human-ish. It’s designed to seem like it was 100% written by a human, but we’re not quite there yet.

 

WHAT ARE THE MAIN WEAKNESSES OF GPT-3? EVERYTHING I’VE READ MAKES IT SOUND LIKE IT’S THE BEST THING SINCE THE WHEEL.

The wheel represented a significant turning point in human civilization, and so did GPT-3. The wheel changed the world by allowing us to carry heavy things from one place to another. GPT-3 is similar in that it and its competitors are changing how The Machine – and we, as humans – see, hear, and think about the world. The technology is far from perfect, and it’s incredibly promising.

One of GPT-3s biggest strengths is that it produces content at scale. That means there’s a lot of it. One of the weaknesses, however, is that some of its learning is still wonky. It’s great at recognizing and predicting language patterns but doesn’t understand all the words/concepts at a deep-level level yet. It lacks true context about what it’s writing and doesn’t always take the reader into account. Bespoke personalization, as if we’ve been friends for 50 years and I’m writing YOU a personal letter, will be one of its most significant benefits over time.  (And yes, it’s improved dramatically in a short period of time and will continue to evolve at hyperspeed.)

It’s hard to disrupt. GPT-3 is BIG, and because of that, it’s also slow. Plus, due to its size, when it gets something wrong, it’s often hard to figure out how/why it became wrong. Critics claim this is problematic in spotting bias. Almost all AI has either inward (data) or outward (to humans) bias.

It lacks creativity and adequate emotional response. Some people see this as a positive. Others see it as a negative. If you’re automating short-form content creation and double-checking it for accuracy before you publish, perhaps it doesn’t matter. It likely matters a lot if you’re using it in your chatbot to unhappy customers who have already been escalated through three service tiers. Would I use it to write a fiction book today? No. (But it sure could help with idea generation.) Would I use it for things that need a lot of complex reasoning, like financial data? Also, no. However, there are plenty of other things worth automating to save time and money. Start there.

It doesn’t improve over time the way we think it should. For many reasons (ex: the training set was the entire internet), human/manual intervention is still necessary for some of the improvements. Some purists think this defeats the purpose of AI/ML and argue how this creates additional bias.

It’s generating things that it’s not deeply familiar with and/or has little/no understanding of, so it often misses context and other critical pieces of information. Plus, although it’s somewhat exhaustive in its research/digging, it typically does not paint a whole story. In other words, it leaves way too much information on the table; the text it provides can be outdated, incorrect, and insensitive; and it can suffer from bias. (In that regard, it’s very much like a human.

Plus, the information is outdated. (The latest release was trained on data till June 2021. A lot has happened since then.)

Please note: I would be remiss not to say that there are a lot of sleazy vendors in this space. Much like the Internet was in the 90s, a small group of people is taking advantage of the fact that many marketers are overwhelmed by the topic/lingo and/or in a never-ending Captain Ahab-like quest to find their own white whale. If you ask your vendors the right questions, you will avoid (almost) all the drama. Eventually, this will all sort itself out, just like affiliates and other traffic-driving methods did 20 years ago. In the meantime, do your homework.

 

WHERE IS GPT-3 BEING USED IN MARKETING?

With oversight, marketers are using a combination of GPT-3 and other tools/software/practices in Email, Advertising, Social, SEO, and Public Relations. It’s also used in Customer Service, especially chat. (The key here is “with oversight.)

Right now, most of the good stuff is happening in content generation and translation, but there’s also been solid headway in semantic search, customer feedback, and website creation.

 

ARE YOU UNDERSELLING THIS? DID YOU READ THE COMPUTER WROTE ME ARTICLE?

Yes, I’ve read many “AI Wrote This” haikus, short stories, and newspaper articles.

GPT-3 is inspiring. 175 billion model parameters and an Internet-sized training set are impressive. Full stop.

GPT-1 (AKA first generation, the OG) came out in 2018, which loosely translates to IT’S FOUR YEARS OLD. Have you met a toddler? You may love them, but that doesn’t minimize the fact that they are a lot of work.

 

WHERE SHOULD I USE GPT-3 FIRST?

GPT-3 isn’t really plug-and-play for Marketing yet. Its predictive capacity is impressive and sold as an easy way to generate content at scale. However, that’s still a stretch for most eCommerce/online businesses.

With that said, Marketers can use GPT-3 for simple activities like ad headlines, email subject lines, short product descriptions, headlines and titles, and short sentences. In some areas/concentrations, it’s suitable for brief stories and summaries. In narrow categories, where there is a copious amount of deep information/data, it’s also suitable for first drafts. But start small. Use it for something in a blast email, a PPC ad, or a Tweet. Get the process down, and then keep moving on to bigger things. I know the vendors sell it as something that will “fully write all your product copy tonight,” but you’ll be much happier long-term if you build a solid foundation of knowledge and success.

In my experience (read: a lot of previous failures), optimizing your existing content is one of the best ways to learn about GPT-3 or using any AI/ML in content creation. It can become very good – very quickly – at optimizing human content to maximize value. Giving The Machine quality inputs typically yields increased/better/more productive quality outputs.

It’s best suited for areas where the continuation of a sentence is easily handled, the answer is short and straightforward, and the data needed to generate the response is small.

 

WILL GPT-3 REPLACE MY CONTENT TEAM? THAT COULD SAVE ME A LOT OF MONEY.

Hard no. For now, anyway.

The threats from the media of being “replaced by a robot” are never-ending and, frankly, exhausting. Yes, AI has been around for many decades, but it’s only in the past few years that it’s gotten deep traction in ways that are/could be helpful to your online/eCommerce business.

Use GPT-3 and other Artificial Intelligence/Machine Learning tools to help make the lives of your team faster and easier. Let The Machine do the gruntwork and leave your people to do the most essential stuff. Use internal and external resources, software packages, and tools to support your folks. Use the inside people and outside tools to help build a solid foundation. That’s the best play right now… …. Yes, even if you’re Scrooge McDuck.

Gruntwork, you say? AI is well-suited for automating repetitive tasks. Support your folks by using AI/ML to do those things first.

 

OK, FINE. I’LL KEEP THE CONTENT MARKETERS, BUT CAN I BUILD MY OWN GPT-3 MODEL IN-HOUSE TO SAVE MONEY?

Yes.

You can build your models. You can also “rent” the models that vendors have built. First, you need to decide what problem you are trying to solve. Then, find the best solution for it. (Remember, you need solid data to build AI-anything.)

Building something in-house may or may not save money, depending on what you’re trying to do. You need to determine whether you have the proper technology and expertise to build, plan and maintain it. Please ensure you have the maintaining part covered – this is where many companies slip up.

I’m on the record for believing in owning as much of your AI technology as you can. However, when starting with AI, finding an outside solution is often a better choice. You’re likely to avoid more mistakes this way. Plus, it allows you to learn from outside vendors/services.

As an aside, if you use an outside vendor, you should ask if/what outside models they’ve built on top of. Some marketers want to avoid this question at all costs (it can eliminate some of the bigger vendors), but knowing your foundation is essential. If nothing else, this can also help with privacy/bias issues.

Friendly Heads-Up: The first few projects don’t tend to be big money-savers either way.

 

WHAT IS SEMANTIC ANALYSIS, AND WHY DO PEOPLE TALK ABOUT IT WHEN THEY TALK ABOUT GPT-3?

GPT-3 uses semantic analysis to build its language structure. Semantic analysis understands words within a context, not just on their own. This allows it to get to the sentence’s intended (aka Proper) meaning. So, if I say “I’m beat,” it knows that I’m tired and not whipped like an egg.

Semantic analysis is used in things like chatbots and search engines, and it’s also used to extract information from things like emails, customer reviews/feedback, support tickets, and so on. Because it understands the meaning from text, The Machine can interpret sentences, paragraphs, and entire documents. It does this by processing grammatical structure and identifying relationships and context.

 

IS THE GPT-3 HYPE WORTH IT?

Sam Altman, the CEO of OpenAI, said: “The GPT-3 hype is way too much. It’s impressive (thanks for the compliments!) but has serious weaknesses and sometimes makes silly mistakes. AI will change the world, but GPT-3 is a very early glimpse. We still have a lot to figure out.”

GPT-3 and other Marketing Artificial Intelligence topics are moving much faster than eCommerce and online marketing did 20 years ago, so it’s essential to learn about them…  See what you can do with them… and what you can’t. Play around a little. You’ll be glad you did.

Have questions about AI? Have a tip you’d like to share? Tweet @amyafrica or write info@eightbyeight.com.

 

 A Down-and-Dirty Definition for Marketers. (Read more about these here.)