GenAI, explained

A primer for non-tech people

GenAI is actually hard to illustrate. Most clipart you see for it is inane.

First, an explanation: I’m writing this primarily because there’s a blow-up at one of the organizations I belong to, the Horror Writers Association, over the AI policy they’re trying to implement. Personally, I think what they’ve drafted is as good as you could get considering the state of things in generative AI-land, but it also struck me that some people might benefit from an explainer of how genAI works or the game plan of the tech wallahs running these companies.

For over 15 years, my job was explaining emerging technologies to mainly non-technical people in intelligence. I wrote articles and papers on why certain technologies were likely to have a disruptive impact on how that agency’s work was usually done or on society writ large.¹

As I explained in an earlier post, for the last eighteen months or so of my consulting practice, nearly the only tech I wrote about was generative AI (genAI), or as most of you think of it, ChatGPT. In late 2023, I did a presentation for a conference of non-tech people, a primer for understanding what large language models (LLMs) are. And then I retired from tech forecasting. I still follow AI developments through newsletters by my favorite tech analysts but no longer do the deep-dive kind of analysis I used to. Since putting that 2023 presentation together, developments in genAI have sped up. It’s gotten more complicated, as always happens as a technology matures but genAI is in a class by itself.

Still, from what I see on Twitter and elsewhere, a basic understanding of what genAI is and isn’t would probably benefit a lot of folks, so I’m condensing my presentation below.

What is artificial intelligence (AI)? A complicated question. The effort to define it actually started in the 1950s, at a gathering of experts in all the fields thought to contribution to how concept of human intelligence. They came up with over 50 fields that a program would need to exhibit to be considered to have human-like intelligence. Over the years, that has basically boiled down to seven fields (the list running down the center, below. To the left are sub-fields of the main categories):

Generative AI (genAI) is what we’re going to talk about in this post.² GenAI is machine learning (ML), where (in basic terms) the program is teaching itself without a human stepping in every time the program runs into something that doesn’t align with the rules baked into its programming. It’s basically making inferences as it plows through data, training itself as it sees patterns and repetitions.

Further, the big leap forward in genAI that we’re talking about is in natural language processing (NLP). You may remember that not too long ago people were losing their minds over computers finally being able to identify pictures of cats. Then cats’ positions (sitting! lying! flushing the toilet!) and then to differentiate between various breeds of cats, and so on. That’s because this whole ML thing started with computer vision, the ability for the program to “see”. I’m going to skip the explanation of how this works (you can look it up) but the main point here is that researchers still had made no progress with NLP. As an analyst, my world was mainly text not imagery, so this really irked me.

But I understood why, because as it turned out, parsing text is extremely complicated. There are components to it, like understanding what words are typically seen together, how much emotion is in a text (sentiment analysis) etc etc. Progress was glacial.

Until OpenAI got the idea to train its program against huge huge data sets. Orders of magnitude larger than what was commonly used, and this was the development that enabled a program to train itself quicker and with a higher rate of success. These large datasets were called Generative Pre-Trained Transformers or GPT. ChatGPT is a chatbot for query. The interface.

Wake up because now we’re getting to the important stuff for writers. OpenAI, developer of ChatGPT, originally started as a non-profit in 2015. It was developing policy, and then doing research in development of AI. This is important because, as it was doing research and not making a product for sale, many institutions with large data sets like Getty Images allowed OpenAI to use its data for training purposes. Since the lawsuits started flying around, however, OpenAI doesn’t talk openly about where it gets its data, but earlier reporting said that for GPT-3 (ChatGPT was trained on GPT-3.5) OpenAI used five gigantic data sets. Three mostly came from the internet and Wikipedia, and two datasets of scanned books, including one that was known to contain pirated work.

OpenAI is now up to GPT-4o, released in May 2024. It’s not only multilingual, it’s also multimodal, processing text, images and audio.

I imagine you have lots of questions.

How good is what ChatGPT comes up with? Well, first, you really mean genAI. There are other programs besides OpenAI’s—all the ginormous tech companies are vying for dominance in the field (Google, Meta, Amazon, Apple). Whoever achieves dominance first will most likely rule the roost, the way Google dominates search. Less than a year from my last professional assessment, it kind of looks like it’s all over and OpenAI is at the top of the heap, but that could change. Everyone’s working on making a better, faster genAI. Senior leadership of OpenAI is bailing left and right because apparently CEO Sam Altman is impossible to work with. Nothing’s certain.

But to get back to the question: when ChatGPT first came out, it was wobbly. It made laughable mistakes. The worst sin was that it had a tendency to make things up if it didn’t see what it expected (back to those inferences). My understanding is that genAI is getting better at not doing this. It’s “success” rates are higher. It’s more reliable. After ChatGPT first came out, everyone wanted to play with it, its user rates were high, and then it dropped off precipitously because it wasn’t doing what people expected. User rates are climbing again because it’s getting better, and because they’re finding better use cases for it.

What about copyright? ChatGPT was trained on copyrighted information, no question. The short answer, however, is that you can’t take the copyrighted data out of the program; that’s not how it works. Now, OpenAI has already developed newer, bigger, better datasets to train its latest genAI models. Do these datasets contain copyrighted material? Don’t expect OpenAI to tell. You’d think they’d be idiots to continue down that path, but fine tuning the models is difficult. It’s like an infinitely complex game of Jenga. You don’t want to inadvertently take away the magic stick.

For more on whether OpenAI can litigate it’s way out of its copyright problems, see this post.

Can genAI create art or is that still the domain of humans? That’s the million-dollar question, isn’t it? It’s hard to say. What genAI does well is parse a whole lot of information and find the patterns quicker—much quicker—than a human. Perhaps it will one day figure out the pattern to making real art. That’s kind of how it works for humans. Michelangelo figured out how to paint his masterpieces, for instance.

Or shoddy programming and complacency could damn it to sifting through piles of what has already been created in order to produce pale (but satisfactory) copies. It will be good enough to write resumes, bland speeches, condolence letters and all the things are don’t require art and just need to be done.

You may be disappointed to find there is no “answer” in this post. It’s still—unbelievably—early days for genAI. Things are still moving fast and it’s hard to keep up with changes. It’s also incredibly hyped³ (meaning lied about) as companies jockey for dominance. It’s hard to know what’s the truth and what OpenAI wants you to believe. They’re not transparent. And the money involved is mind blowing. Don’t expect anyone to play nice.

Which brings me to my last point, the most important point, and that’s that we need government oversight and regulation of AI. These companies are in control right now and they are profit-driven. They are working in the best interests of shareholders, not humanity writ large. Remember, all the disruptive tech of the past two decades that’s been rewarded with huge IPOs and obscene buyouts don’t produce any content (like human creators): they’re platforms for consolidating. Facebook and YouTube don’t create their own content; users do, an endless supply of free content. No wonder Zuckerberg places no value on what humans produce. It can be had for nothing, literally. What matters to those CEOs is that we worthless content producers don’t hold up tech progress.

1. I want to be very clear: this kind of forecasting is far from merely holding an opinion. Analysts in this field have been training in analytic methodologies, both quantitative and qualitative measures, etc. Predictive analysis is extremely difficult, obviously—maybe a topic for a future post.

2. I’ll just add that what most people think of, when they think of AI, is general or generalized AI. Think HAL in 2001: A Space Odyssey or Data on Star Trek: The Next Generation. GenAI is not that.

3. The most hyped emerging technology I’ve seen in 17 years of tech forecasting, and that’s saying a lot.