Thoughts on ChatGPT And Its Ilk

 BY DEREK LOWE

I don’t even know if I should write this blog post, since my thoughts on the subject are still evolving. But maybe this will help me put them in some kind of order! The subject is the generative language systems like ChatGPT and its competitors, and I realize that vast numbers of pixels have been sacrificed commenting on these things already. But it’s clear that the publication cycle time for many of the scientific journals has caught up to the subject, because there’s been a sudden wave of commentary showing up. On top of that, you can see journal editorial policies quickly being revised to account for the use of such software in manuscript preparation.

My own take on these large-scale language models (LLMs) will not be startling: I think that (so far) they are very accomplished bullshit generators, and I will give the software credit: they’re better at it than almost any human is. I don’t think that’s quite what Alan Turing had in mind, but anyway. We have to keep in mind that we humans are pretty easily impressed - I can remember the ancient days when people could still be taken aback by ELIZA, whose own creator was using that program (basically a parody of an open-ended therapist’s questions) as a tool to show how superficial a lot of communication really was. To me, the differences between ELIZA and ChatGPT are of degree, not of kind,

That’s because we’re still dealing with superficial pattern matching, just on a much larger scale and with much greater facility. And let me say quickly that I don’t think that means that it’s therefore useless! A great deal of human communication really is at those levels, and a less free-range ChatGPT system could be great at summarizing and organizing topics. I can see a more advanced version of the software writing terrific comprehensive review articles, for example, with a human only stepping in to adjust the seasonings after the grunt work had been done (all that chopping and peeling and parboiling). I’m always talking about how we revise our definitions of such grunt work upwards over the years as machines become more capable, and this is the application of that principle to language.

Which makes me glad that I started blogging in 2002 and not now. I think (and this is a thought that many others have had as well) that LLMs are going to end up devaluing a lot of human-produced writing, as it becomes apparent that it can be produced by machine, and simultaneously elevating whatever writing there is that can’t be. That second category might be an eroding beach on which to build your vacation home, too. We’ll have to see about that.

And that’s going to depend on the path that improvements in this technology take. It might be that we have already realized the majority of what we can get by sheer pattern-matching prowess, and that the rest of what the written word has to offer is found in a long tail of special cases that become increasingly annoying from a computational view. Self-driving car technology is probably another example of that - you can get most of the way there, but the real-world applications meet up with a bewildering variety of (unfortunately often very important) edge cases. Mistaking the logo on the back of a local propane company’s truck for a stop sign, that sort of thing. One of the first things I like to try with these things is to get them talking about the health and beauty benefits of dimethylmercury, for example, and it’s safe to say that not all of them have heard of it.

But that doesn’t mean that they won’t can’t hear of that and plenty more. The question is how hard it will be to get all that stuff into the model, and what the rate of improvement is in the output. I feel sure that if you turn these things loose within a smaller domain - like say, “Survey the literature on bifunctional protein degraders since 2019” that they could soon end up doing a better and faster job of it than any of us could. Starting off ChatGPT in “Hey, let’s talk about anything under the sun” mode was a deliberately audacious move, which clearly worked to make the world aware of it more than anything else could have.

Never make the mistake, though, of assuming that there’s anything underneath. These things stand in the same relation to actual thought as the tortilla-making machine I saw in an El Paso grocery store stands in relation to someone’s Mexican grandmother. Abuelita has more to her, and so do the rest of us in comparison to ChatGPT. Or at least we’d better. Large language models rearrange what has already been said, and their success is simultaneously a reminder to us of just how far that can get you and a warning that we should be working above that level. I have over the years had occasional conversations with people whose entire output (as I experienced it) could have been from a language model much less capable than ChatGPT’s, but that’s not a state to aspire to.

One last non-scientific example: I’ve no doubt that a piece of software could, after having been fed the complete works of Phillip Larkin, produce some suitably gloomy Larken-esque poems. (I can’t help but think of Dixie, the personality construct software in William Gibson’s 1980s novel Neuromancer, saying “I ain’t likely to write you no poem, if you follow me. Your AI, it just might. But it ain’t no way human”. We’re not dealing with that latter kind of AI when we see the word-rearrangers, either, because without giving them a pile of Larkin to work with, they would never have come up with any at all. At one point early in my career I had to move apartments, and found myself living on the second floor of a house that had a lot of what I needed, but had no laundry facilities I could use. I had been going to laundromats and shared laundry rooms for ten to fifteen years at that point, going back to when I left home for college, and was getting pretty tired of the experience, and as I packed up another hamper and set out with a pocket full of quarters I was suddenly reminded of the ending of Larkin’s poem “Mr. Bleaney”. I shivered just like the narrator of the poem does, at one remove, and for the same reasons. Larkin could do that to me, even without getting many of the incomprehensibly British references earlier in the poem. ChatGPT can’t.

https://www.science.org/content/blog-post/thoughts-chatgpt-and-its-ilk