Categories: News

What two years of AI development can tell us about Sora

A screenshot of a video generated by Sora, OpenAI’s generative video model. | Sora/OpenAI CEO Sam Altman

If you want to know the future of OpenAI’s latest tool, take a look at Midjourney and DALL-E 2.

Remember when AI art generators became widely available in 2022 and suddenly the internet was full of uncanny pictures that were very cool but didn’t look quite right on close inspection? Get ready for that to happen again — but this time for video.

Last week, OpenAI released Sora, a generative AI model that produces videos based on a simple prompt. It’s not available to the public yet, but CEO Sam Altman showed off its capabilities by taking requests on X, formerly known as Twitter. Users replied with short prompts: “a monkey playing chess in a park,” or “a bicycle race on ocean with different animals as athletes.” It’s uncanny, mesmerizing, weird, beautiful — and prompting the usual cycle of commentary.

Some people are making strong claims about Sora’s negative effects, expecting a “wave of disinformation” — but while I (and experts) think future powerful AI systems pose really serious risks, claims that a specific model will bring the disinformation wave upon us have not held up so far.

Others are pointing at Sora’s many flaws as representing fundamental limitations of the technology — which was a mistake when people did it with image generator models and which, I suspect, will be a mistake again. As my colleague A.W. Ohlheiser pointed out, “just as DALL-E and ChatGPT improved over time, so could Sora.”

The predictions, both bullish and bearish, may yet pan out — but the conversation around Sora and generative AI would be more productive if people on all sides took into greater account all the ways in which we’ve been proven wrong these last couple of years.

What DALL-E 2 and Midjourney can teach us about Sora

Two years ago, OpenAI announced DALL-E 2, a model that could produce still images from a text prompt. The high-resolution fantastical images it produced were quickly all over social media, as were the takes on what to think of it: Real art? Fake art? A threat to artists? A tool for artists? A disinformation machine? Two years later, it’s worth a bit of a retrospective if we want our takes on Sora to age better.

DALL-E 2’s release was only a few months ahead of Midjourney and Stable Diffusion, two popular competitors. They each had their strengths and weaknesses. DALL-E 2 did more photorealistic pictures and adhered a little better to prompts; Midjourney was “artsier.” Collectively, they made AI art available at the click of a button to millions.

Much of the societal impact of generative AI then didn’t come directly from DALL-E 2, but from the wave of image models it led. Likewise, we might expect that the important question about Sora isn’t just what Sora can do, but what its imitators and competitors will be able to do.

Many people thought that DALL-E and its competitors heralded a flood of deepfake propaganda and scams that’d threaten our democracy. While we may well see an effect like that some day, those calls now seem to have been premature. The effect of deepfakes on our democracy “always seems just around the corner,” analyst Peter Carlyon wrote in December, noting that most propaganda continues to be of a more boring kind — for example, taking remarks out of context, or images of one conflict shared and mislabeled as being from another.

Presumably at some point this will change, but there should be some humility about claims that Sora will be that change. It doesn’t take deepfakes to lie to people, and they remain an expensive way to do it. (AI generations are relatively cheap, but if you’re going for something specific and convincing, that’s much pricier. A tsunami of deepfakes implies a scale that spammers mostly can’t afford at the moment.)

But the place where it seems most crucial to me to remember the last two years of AI history is when I read criticisms of Sora’s images for being clumsy, stilted, inhuman, or obviously flawed. It’s true, they are. Sora “does not accurately model the physics of many basic interactions,” OpenAI’s research release acknowledges, adding that it has trouble with cause and effect, mixing up left and right, and following a trajectory.

Nearly identical criticisms were, of course, made of DALL-E 2 and Midjourney — at least at first. Early coverage of DALL-E 2 highlighted its incompetencies, from creating horrifying monstrosities whenever you asked for multiple characters in a scene to giving people claws instead of hands. AI experts argued that the inability of AI to handle “compositionality” — or instructions about how to compose the elements of a scene — reflected a shortcoming fundamental to the technology.

In practice, though, models got better at fulfilling highly specific prompts and users got better at prompting, and as a result it’s possible today to create images with complex and detailed scenes. Nearly all of the entertaining deficiencies were corrected in DALL-E 3, released last year, and in the latest updates to Midjourney. Today’s image generators can do hands and crowd scenes fine.

In the time between DALL-E 2 and Sora, AI image generation has gone from a party trick to a massive industry. Many of the things DALL-E 2 couldn’t do, DALL-E 3 could. And if DALL-E 3 couldn’t, a competitor often could. That’s a perspective that’s crucial to keep in mind when you read prognosticating on Sora — you’re likely looking at early steps into a major new capability, one that could be used for good or malicious purposes, and while it’s possible to oversell it, it’s also very easy to sell it short.

Instead of overcommitting to any particular perspective on what Sora and its successors will or won’t be able to do, it’s worth admitting some uncertainty about where this is headed. It’s much easier to say, “This technology will keep improving by leaps and bounds” than to guess the specifics of how that will play out.

A version of this story originally appeared in the Future Perfect newsletter. Sign up here!

Vox - Huntsville Tribune

Recent Posts

Why couples are choosing cohabitation over marriage

Naomi Elliott for Vox First comes love, then comes … moving in together. After about…

16 hours ago

What the Supreme Court case on tent encampments could mean for homeless people

A small tent encampment off of Broward Boulevard in Fort Lauderdale, Florida. | Mike Stocker/South…

17 hours ago

Will AI mean the end of liberal democracy?

Journalist Fareed Zakaria speaks during the Ellis Island Medals of Honor ceremony at the Ellis…

2 days ago

Israel and Iran’s conflict enters a new, dangerous phase

A crowd carrying a model of Iran’s first-ever hypersonic missile, Fattah, past a mosque during…

2 days ago

Trump’s jury doesn’t have to like him to be fair to him

Former President Donald Trump appears in Manhattan criminal court on April 19 for jury selection…

2 days ago

Taylor Swift seems sick of being everyone’s best friend

Taylor Swift performs during the Eras tour at the National Stadium in Singapore on March…

3 days ago