Meta Unveils New AI Model Capable of Generating Video with Sound, Challenging OpenAI

By Katie Paul

NEW YORK – Meta, the parent company of Facebook, announced on Friday the development of a new AI model named Movie Gen, capable of producing realistic video and audio clips in response to user prompts. The company claims that Movie Gen can compete with the offerings from leading media generation startups such as OpenAI and ElevenLabs.

Meta showcased examples of Movie Gen’s capabilities, demonstrating videos featuring animals swimming and surfing, as well as clips that utilize real photos of people engaged in activities like painting on a canvas. Movie Gen can also generate synchronized background music and sound effects, enhancing the overall video experience, and it can edit existing videos.

In one demonstration, the AI inserted pom-poms into the hands of a man running alone in the desert, and in another, it transformed a dry parking lot into a scene with a splashing puddle where a man was skateboarding. The videos created by Movie Gen can be up to 16 seconds long, while the audio can stretch up to 45 seconds. Meta presented data from blind tests indicating that the model performs well compared to rival products from companies like Runway, OpenAI, ElevenLabs, and Kling.

This announcement comes during a pivotal year for Hollywood, which has been navigating how to effectively incorporate generative AI video technology. Earlier this year, Microsoft-backed OpenAI demonstrated its product, Sora, which is capable of generating feature film-like videos from text prompts. While many in the entertainment industry are eager to leverage such tools to streamline filmmaking processes, concerns persist about employing systems that may have been trained on copyrighted materials without authorization.

Lawmakers have also raised alarms about the implications of AI-generated fakes, including deepfakes, particularly regarding their potential misuse in elections worldwide, including in the U.S., Pakistan, India, and Indonesia.

Meta representatives indicated that the company is unlikely to make Movie Gen available for open use by developers, unlike its Llama series of large-language models. The decision regarding each model’s release is approached on a case-by-case basis, though they did not provide specifics about Movie Gen’s future.

Instead, Meta plans to collaborate directly with the entertainment community and content creators to explore the applications of Movie Gen, with intentions to integrate it into its own products next year.

According to the information shared by Meta, the company utilized a combination of licensed and publicly available datasets to develop Movie Gen.

OpenAI has also been engaging with Hollywood executives and agents to discuss potential partnerships regarding Sora, although no deals have been finalized. Concerns about OpenAI’s practices have intensified after actor Scarlett Johansson accused the company of mimicking her voice without consent for its chatbot.

In a notable development, Lions Gate Entertainment, known for franchises such as "The Hunger Games" and "Twilight," announced in September it would grant AI startup Runway access to its film and television library for training purposes, enabling the studio and filmmakers to leverage the AI model to enhance their creative work.