UPDATE: ZDNet shows some skepticism about GPT-2’s abilities without downplaying its significance.
UPDATE 2: Some sample outputs from the GPT-2 software can be found here. Suffice it to say, novelists’ jobs aren’t in any danger at all, though there are some pretty good examples in there.
UPDATE 3: Scott Alexander of Slate Star Codex shares some insightful thoughts.
UPDATE 4: Hugh Zhang of The Gradient puts this accomplishment into perspective.
As a novelist, it’s easy to get worried about what artificial intelligence, or AI, can do. Everything from movement, to vision, to the game of Go (long regarded as difficult for computers) has been done well by AI and is set to get better and better. This raises the question of what it means for those who do creative work, such as artists and novelists. On the artistic side, a program called StyleGAN creates very good-looking anime faces as well as equally good-looking realistic faces. It’s not good enough to generate whole images in a variety of poses and appearances, but it makes one think what might be possible with time. As for text, that software don’t do so well.
But GPT-2 does.
GPT-2 is an AI-based text generator that creates very convincing tweets and blog posts — so convincing that its creators are keeping the full version under lock and key for fear of misuse. Unlike earlier text generators, this one can stay on topic and avoid spelling, grammar, and punctuation errors with ease. Most notably, it was able to write an article about scientists discovering unicorns — and it was surprisingly readable.
Because this is a text generator, those who write nonfiction will be safe, since good nonfiction requires both research and experience. Fiction authors, on the other hand, simply make things up. Does this technology threaten their profession?
I would say no.
The Verge article linked above shows that while GPT-2 has an excellent command of English, it cannot tell even a short story with any real coherence. The reader will be amazed at the AI’s ability to form sentences, but those sentences will come off as gibberish with good spelling.
Nonetheless, I wanted to know more, so I e-mailed Jack Clark, one of the researchers working on the GPT-2 project. I asked him if the program could replace novelists, and this was his answer (posted with his permission):
Nope! Anecdata, but I occasionally write short stories and used gpt-2 to fool around with some. It can come up with a paragraph of decent stuff some of the time if you condition it with a page of prior stuff. Some of this can be interesting or have merit. I’ve also occasionally seen it generate a page of decent narrative, but you as the writer would have to write around it to stitch together.
It broadly sucks at endings – I think there’s some pretty complex stuff people do to close-out narratives, so I’d be surprised if this kind of thing could be good at that soon, though I’ve learned with AI that ‘soon’ is a relative term.
We’re really excited about opening it up for broader experiments by fiction writers. We released the small model on our GitHub and some people have plugged this into systems that you may want to use to fool around with
https://twitter.com/iglpzfr/status/1096576593374265344?s=19Best,
Jack
However, there’s an important caveat here: the program was trained on 40GB worth of websites, not 40GB worth of novels. And while I suspect that training it on novels will only produce marginally better results with great structure and poor coherence, it could be a powerful engine for coming up with ideas.
(For a much darker take, read this Twitter thread by Tim Soret. My response to such an attitude is summed up in this post.)
I can see AI text generators being used as a tool for authors, to fill in certain scenes. Say, for a fight scene, you could give it the names of the combatants, the styles and weapons used, the area in which they are fighting, and the desired outcome, and the program could generate X number of words describing the action.
But I don’t see it being able to come up with the plot, a reason for the characters to be fighting, what each has at stake in the conflict, and so on.
Kind of how 3d rendering programs can “draw” a picture, but a human operator has to decide what elements to include and the relationships between them.
And I expect that unless used skillfully, the scenes generated by AIs will read fairly flat and repetitive and readers will think, “oh, this is all AI generated”. Kind of like how scenes rendered with DAZ or Poser all tend to look the same unless the artist is really good.
I agree there. A novel written by a program will need so much rewriting that you might as well write it from scratch. The best use I see for this is as a way to come up with ideas; for example, using a different AI, I generated a list of alien-sounding names.