“The Wizard of Oz” may not be the first film shot in color, but many people remember it in that way because of how director Victor Fleming wisely used black and white films for the scenes put in Kansas.
Likewise, “The Wizard of Oz” may not be the first film that has been re -conceptualized with AI, but it may soon be known for that.
For several months, thousands of researchers, programmers, artists, archivists and manufacturers in Google Deepmind, Google Cloud, Sphere Studios, Magnopus, Warner Bros. Discovery and others in the film and technology industry worked to bring the 1939 classic to a very large screen in a very large way.
On August 28, their work on Sphere, the colossal Las Vegas site, is debating new forms of entertainment since it opened in September 2023. Now generative AI will take the center with Dorothy, Toto and more Munchkins than ever fit into a multiplex.
It is appropriate that a job that once broke filming boundaries will do it again. “The Guide to Oz at Sphere” is an equally epic commitment of creativity and technology, where the story will enclose the room’s 17,600-seat spherical spaces to create an immersive sensory experience.
Even a few years ago, such a business would have been almost impossible with conventional CGI. It has really only become possible through the latest progress in generative AI media models, specifically image and VEO, with Gemini, which also plays an important role. Not only does the team need to create an all -inclusive experience, but they also have to do so with only the original material. A line of new dialogue was not added, nor a note of new music was sung to enrich this classic for the sphere.
“We talked about doing it in different ways,” says Jane Rosenthal, Academy and Emmy Award-nominated Manufacturer, a producer at “The Wizard of Oz to sphere.” “We realized that we really needed to do it with AI.”
The wonderful wizards
Not that the team can simply enter a few AI prompts, click on their collective heels and call it one day. Buzz Hays, Global Lead for Entertainment Industry Solutions at Google Cloud and a 37 -year -old manufacturer in Hollywood points out that this is about more than using AI to expand an old movie to a new format.
“We start with the original four-by-three image on a 35mm piece of celluloid-it is actually three separate, grainy films negatives; that’s how they shot technicolor,” says Hays. “Of course, it doesn’t work on a screen that is 160,000 square meters. So we work with Sphere Studios, Magnopus and Visual Effects artists around the world along with our AI models to effectively bring the original characters and environments to a whole new canvas -creating an immersive entertainment experience that still respects the original in all ways.”
When the project was first underway, there were many on the team, including within Google who openly wondered if AI technology was still there to end the work or achieve the group’s collective vision. But because traditional CGI would not do that, at least not without massive expense and many years of wear and tear – and because everyone was happy to break new ground – they came to work.
“The models, they are wildly innovative,” says Dr. Steven Hickson, a Google Deepmind researcher on the project. “We would find something we can’t do, we think it’s impossible, and then a month later we actually like, maybe we do it.”
However, you can see why it seemed impossible.
Enlargement of the original grained images for Sphere’s 16K LED screen – the highest resolution screen in the world – was the first, but far from the only challenge. The team also had to account for all camera rocks in a traditional film that removes characters from parts of certain scenes that would not work in the new theater scale imagined. Conventional CGI may have handled the scaling problem, but there are few ways it can fill the rest of the scenes effectively.
Take the moment when the Feige Lion first throws himself on his soon -to -be companion. The camera panes back and forth between the scarecrow and the tin man, with cut to Dorothy hiding behind a tree in the distance. The experience at Sphere called to keep all these elements together, in hyper -realistic detail.
To achieve this, the team has three major technical obstacles to overcome.
The magic of fine tuning
Using versions of VEO, Imagen and Gemini, specially set for the task, the Google teams and their partners developed an AI-based “Super Resolution” tool to transform the small celluloid frames from 1939 into Ultra-Ultra-high definition images that jump inside the sphere. Then the teams perform AI -wrapping to expand the scope of scenes to both fill the space and fill in the gaps created by camera nit and framing restrictions. Finally, through performance generation, they integrate composites of the famous performances in the expanded environments.
Together, these techniques help achieve the natural movements, staging and fine details that conventional CGI struggles to match.
“When the request came to us, I almost jumped up and down,” says Dr. Irfan Essa, a most important researcher at Google Deepmind and director of its Atlanta Lab. “This is the best opportunity to show the magic we develop using AI.”
Yet for all the powerful new technology that plays, one of the biggest breakthroughs comes from following the cinema’s traditions: having lots of extra material to work with. In addition to old recordings, the team shed archives to build a huge collection of supplementary material, such as recording script, production illustrations, photographs, set plans and scores.
Through a process known as fine tuning, these materials are uploaded to VEO and Gemini so that the models can train on specific details of the original characters, their environments and even elements of the production, such as the camera’s focal length of specific scenes.
With far more source material than just the 102-minute movie to work with, the quality of the outputs improved dramatically. Now Dorothy’s freckles can click in focus and Toto can spoil more seamlessly through several scenes. Each change, notes Hays, was made in close collaboration with Warner Bros. to ensure continuity with the spirit of the original.
Follow the Yellow Wall Code
As the team continues their journey with this truly bigger than life project, many are still in awe of everything they have achieved-and excited about what has still come.
“When you have innovation like this, you don’t always know where it will go,” says Jim Dolan, performing chairman and CEO of Sphere Entertainment. “You have to be able to take a jump of faith. What you want to see in ‘The Wizard of Oz on Sphere’ is clearly a jump of faith.”