ByteDances next-gen AI model can generate clips based on text, images, audio, and video

Big Tech’s race to leapfrog the latest AI models continues with the launch of ByteDance’s next-gen video generator. In a blog post, ByteDance – the China-based company behind TikTok – says Seedance 2.0 supports prompts that combine text, images, video, and audio.

The company claims it “delivers a substantial leap in generation quality,” offering improvements in generating complex scenes with multiple subjects and its ability to follow instructions. Users can refine their text prompts by feeding Seedance 2.0 up to nine images, three video clips, and three audio clips.

The model can generate up to 15-second clips with audio, while taking camera movement, visual effects, and motion into account. It can also reference text-based storyboards, according to ByteDance.

In one example shared by ByteDance, which shows two figure skaters performing a routine together, the company says Seedance 2.0 can “reliably perform a sequence of high-difficulty movements — including synchronized takeoffs, mid-air spins and precise ice landings — while strictly following real-world physical laws.”

Users on social media have already started showing off what the new tool can do, with one person posting an AI-generated video with the likenesses of Brad Pitt and Tom Cruise in a cinematic fight sequence. Deadpool writer Rhett Reese reposted the video with the comment, “I hate to say it. It’s likely over for us.”

For now, Seedance 2.0 is only available through ByteDance’s Dreamina AI platform and through its AI assistant, Doubao. It’s unclear whether it will make its way to TikTok — especially now that the app is under new ownership in the US.