We Made an Ad with Generative AI

The Horse in Motion

It all started with a question. Does a horse gallop with all four hooves off the ground? In 1878, English photographer Eadweard Muybridge was tasked to find the answer. Using multiple cameras to create the illusion of motion through a sequence of photographs, the very first film, “The Horse in Motion” was produced.

Since then, film has constantly reinvented itself, pushing creative boundaries and becoming an integral part of human culture and society. From the silent film era that saw the rise of stars like Charlie Chaplin to groundbreaking computer-generated-imagery (CGI) in “Jurassic Park”, film continues to give us new experiences and create visceral emotional responses that we could only dream of a century ago.

Through the history of film, one thing has remained constant – the tension between balancing budget, time, and human resources with the need for creative direction and freedom. While the TV and film industry is often glamorized, any professional filmmaker can tell you that the daily realities of these constraints can be immensely challenging. It’s a fast-paced and high-stakes environment where the cost of mistakes can be extremely high, literally and figuratively. That’s true whether you are an independent filmmaker, part of a major studio, or folks like Visual Designer Brian Townsend and Creative Director Cisco McCarthy from the Windows + Devices Visual Design team at Microsoft.

Townsend and McCarthy have been heavily involved in bringing Microsoft products to life through various visual mediums for more than a decade now. Their work includes producing advertisements, sizzle videos, photography, 3D product renders and motion designs for Windows and Surface. The latest Surface for Business ad that they released a month ago is a testament towards the team’s constant push for innovation with AI around visual product storytelling.

Visualizing Ideas with AI

Earlier this year, the Visual Design team was approached to create an ad for our latest Surface Pro and Surface Laptop for Business Copilot+ PCs. The challenge – it needed to be done within a month. With an immensely tight deadline alongside various resource and budget constraints, the team found themselves asking “How can we create a high-quality creative output, efficiently?” The answer – generative AI.

It is hard to imagine that just slightly more than 10 years ago, producing videos was a slow and cumbersome process. “When we were first creating images of keyboards for the first Surface launch, each keyboard came in five physical layouts, each with 26 different prints for different languages,” says Townsend. The sheer amount of assets one needed to create back then for a single video led Townsend to one of the core ethos for his team – creative efficiency.

In line with this ethos, the Visual Design team has been experimenting with leveraging various generative AI tools as part of their production process since 2022. The evolution of generative AI has enabled filmmakers to create realistic visual effects, lifelike animations, and immersive environments with unprecedented speed and quality. For the Visual Design team, this advancement has been profoundly important in sharing and pitching ideas. “The picture you have in your head is going to be slightly different than the one I have” says McCarthy. “If we can visually express our ideas and get it in the ballpark of what we are each thinking, now everybody’s closer and all on the same page.”

The ability to explore a wide range of creative ideas and land on creative consensus quickly with generative AI, at a fraction of the cost of traditional methods, has been a huge benefit. The team managed to deliver a compelling script, storyboards and a pitch deck that accurately represented how they visualized the ad within a matter of days instead of what usually may have taken weeks. With the foundational elements in place, it now came down to seamlessly integrating generative AI shots into frames within the ad.

Prompt, Prompt, Prompt Again

For those who have ventured into the realm of generative AI, the challenge of translating our mental images into reality is a familiar one. The Visual Design team was no exception. Crafting a prompt that perfectly captured their creative vision demanded both patience and perseverance. Their journey began with using natural language to describe a specific shot to an AI chatbot. They then asked the chatbot to transform these descriptions into a prompt format suitable for specific AI generative tools, marking the start of an iterative dance with AI.

“We probably went through thousands of different prompts, chiseling away at the output little by little until we got what we wanted. There’s never really a one-and-done prompt,” says McCarthy. The team also employed other creative methods, such as taking an image of a composition that aligned with their creative direction and feeding it to an AI chatbot to generate a text prompt that would bring them closer to their goal.

Like carving a masterpiece from a block of marble, each prompt was a careful stroke of the sculptor’s tool that gradually revealed the form within. Through relentless experimentation and countless revisions alongside generative AI, the team eventually conjured a library of stunning art for characters and sets, translating their ideas into captivating visualizations for the ad.  “It comes from being relentless” says McCarthy. “Just keep pushing on the tools and if it truly can’t be done, wait a week.”

Blending AI with the Human Touch

We have come a long way from generative AI producing double elbows in one arm, or a seemingly boneless rubber hand with 7 fingers. Today, AI generative tools do a much better job than they did just merely 6 months ago, and the progress continues to be exponential. However, replicating details such as human motion and likeness continues to be challenging for these tools. When deciding on which shots within the ad were to be AI generated, the team determined that any intricate movement such as closeups of hands typing on keyboards had to be shot live. Shots that were quick cuts or with limited motion, however, were prime for co-creation with generative AI tools.

Part of the creative direction of the ad was to depict the wide range of jobs and situations the Surface devices could be used in. Generative AI allowed the team to quickly produce different real-world scenarios, without the need to travel to a location or reproduce a logistically complex live scenario. The tools offered the team a broader palette of creative options to select from, as well as the agility to pivot towards new ideas when inspired.

However, as with all things created using AI, the occasional AI hallucination would rear its head. To combat this, the team employed a blend of AI and traditional video production tools to meticulously compose frames and correct any unnatural elements. Each frame was carefully curated, removing anything that looked weird and out of place, and ensuring only the most compelling elements remained. Additionally, the team made a conscious effort to use top-down shots to limit hallucinations and maintain visual coherence. From there, the finalized stills were fed into generative AI tools like Hailuo or Kling to add motion for the video.

A New Creative Catalyst

Time has always been one of the most precious and limited resources in any creative endeavor. For the Visual Design team, the production of this ad underscored the transformative benefits generative AI brings to the creative process. “With a small budget and a tiny amount of time, we were able to accomplish in a month what would have traditionally required exponentially more resources. We probably saved 90% of the time and cost it would typically take,” says Townsend.

And it’s not just about saving time but rather transforming how that time is used. The freedom to experiment and explore a vast array of ideas quickly and effortlessly allowed the team to reallocate their time and energy towards more meaningful aspects of the creative process. By automating time-intensive tasks within post-production and digital workflows, the team was able to focus on ideation and creative iteration, fostering a more agile and dynamic workflow. Moreover, the ability to move faster has opened new opportunities to scale their efforts and expand their portfolio to serve a broader range of stakeholders, without compromising quality.

Contrary to the assumption that speed might dilute creativity, Townsend firmly believes the opposite in this new age of AI. “The faster we can make something move, the more creative freedom it gives us.” By reducing the logistical and technical bottlenecks, generative AI has empowered the team to channel their creativity into producing higher-quality outputs and invest their energy in what truly matters: creating meaningful connections with their audience.

Humanizing AI Creativity

Each iteration of film over the decades has continued to unshackle filmmakers over various constraints. Generative AI has revolutionized visual design in filmmaking, democratizing the creative process and enabling creators to push the boundaries of their imagination while significantly reducing time and costs. “It used to be you needed to have a fancy expensive camera and a really powerful computer to be able to move all that footage around, but it’s just not the case anymore,” says McCarthy, holding up his phone. Today, through active collaboration with generative AI, anyone can effectively create their own film. All it takes is imagination and the curiosity to explore the myriad of tools available.

Yet, as we marvel at the capabilities of AI, the Visual Design team recognizes that true magic continues to lie in the application of our humanity. AI can generate beautiful content, but it is our intuition, experiences, and creative instincts that bring authenticity and depth to the stories we tell. It is this human element that ensures our narratives resonate with audiences and inspires action. Ultimately, the most compelling stories will emerge from a harmonious blend of AI’s computational power and the indomitable human spirit, driving us to new heights of innovation and expression.