Skip to content

AI – Visual Design – UX/UI

AI-Powered Creativity with Microsoft Designer

How we made a visual design tool for everybody

  –   The estimated reading time is 9 min.

A tablet and smartphone display a design interface on screens, with a colorful and creative backdrop. The tablet's screen shows design categories and a search bar, while the smartphone displays assorted design templates. Objects around the devices enhance the artistic theme.

“Can you imagine using something like this in real life?” That’s the dreamlike feeling you get when you use Microsoft Designer’s generative AI Image Creator. To type is to draw a boat-swallowing wave in a papercut style that looks like something out of Japan’s wood block printing Edo period. A photorealistic image of a glittery narwhal orbiting the Earth is just a few sentences away.

Image-making with words is about as surreal as painting with sound, but even more surreal has been the rise of generative AI. Since speech recognition became a part of consumer products in the 1980s, people have interacted with artificial intelligence, but generative AI has cracked open incredible new creative potential. For product makers, it’s exciting to think about ways to connect people with that power, especially given how exclusive creativity can often be.

The process of turning a portrait into an avatar.

Developing a new kind of visual design tool

If you’ve ever stopped by a person on the street painting on an easel, you know that it’s like watching an alchemist turn copper into gold. Your first thought is usually, “How did they do that?” Artists have the uncanny ability to visually transcribe the world with hyper creativity, emotional intelligence, and craftsmanship. The years of practice, dedication, and sheer talent it takes to do what they do will never be duplicated. But for other types of creative professionals or anyone wanting to make visual assets, a tool that would empower them to make their ideas a reality is invaluable.

A collage displaying various photo-editing options: framing an image of a woman in nature, applying patterns behind mountains, blurring the background of a young man, removing the background from flamingos, and replacing backgrounds for floral cake designs.
These are just some of the ways you can edit your images in Designer.

Initially, the impetus behind Designer was to further diversify Microsoft 365’s product offerings. Aside from the video editing app Microsoft Clipchamp, “there wasn’t really a great visual design tool,” said Senior Product Designer Vivian Xiao, who contributed to the development of Designer’s Image Creator. While Designer spun out of the technology behind the popular AI-powered PowerPoint Designer, the explosion of LLMs during Designer’s early development offered an opportunity to create a truly AI-first design tool.

“Designer was one of the first products at Microsoft that you could use to generate an image,” said Principal Design Manager Brittany Mederos. From the app’s inception, she co-led the process of adopting AI to build a design tool for anybody to use. Empowering people to have access to visual creation felt right, and it was something we hadn’t yet been able to offer customers. People would now be able to produce professional graphics, edit photos, design layouts, create their own brand, and design social media posts.

This meant that Designer needed to spark inspiration, curiosity, delight, and learning. To do this, we needed to create a user experience that would intuitively teach people how to harness AI to create what they want. Designing a UX that clearly lays out how to write prompts for text generation, like crafting emails or documents, already presented an inherent challenge. Encouraging the use of language to produce visuals is far more complex, especially because many people don’t have a visual vocabulary.

Teaching people how to prompt for visual output

One of the inherent ways that humans learn is through observation. Kids learn from watching their parents. Leonardo da Vinci drew the first helicopter while studying the way birds fly. The difficulty in working with AI, however, is that we don’t yet have a full set of familiar models for how to interact with it. Prompting is still a fresh concept for most of the world.

To show the process of text-to-image prompting, Product Designer Jennifer Jing started by creating an animated video. However, the video didn’t resonate with users. Inspired by Bing Image Creator, now rebranded as Designer, employed visual prompts, Jing suggested the team adopt a similar approach. A visual prompt is an AI-generated image that when hovered over, shows the text prompt that was used to create it. People are unlikely to put much effort into creating or refining prompts if they’re unsure of the outcome or their ability to produce successful outputs. However, once they used the visual prompts, the success rate of keeping and using the outputs was high.

The visual prompt was like a recipe. You see the dish and the ingredients below, with the option of altering the recipe to tastes. But what happens if altered recipe went viral? The team observed the way people used Bing Image Creator on social media, and  made the prompt templates easily sharable, resulting in several viral hits. People turned themselves into Funko Pop figures, or bubbleheads, inspired by Japanese anime. They gave their dogs starring roles in animated movie posters that became trends. The AI-generated posters were a parade of adorable four-legged furry children: a chihuahua and its stickered luggage are about to go on a glorious adventure while a boy cuddling with his golden retriever are “furever friends.” Like proud parents, thousands of people shared their AI-generated dog posters, and they reposted the prompt used to make them.

The way people used the Bing Image Creator would inspire the Designer team and lay the groundwork for developing prompt templates with a Mad Libs-style sentence structure. The template allows people to easily change and tweak the subject and composition of the text. They also have the option of changing any part of the text that doesn’t fit their needs.

An origami Hawaiian bride and groom wearing leis stand on a beautiful beach with paper waves in the background and a bright sun in the sky. The image is vibrant, featuring vivid, bright colors in a papercraft style.
An example of a visual prompt as it's presented along with the revealed prompt.

Another prompting-related challenge was helping people conjure what’s in their mind’s eye when they don’t have the vocabulary to describe what they’re seeing.

“Closing that distance between what people have in their brains and what they actually get on the screen is one of the central challenges with AI,” said Principal Product Design Director Ian Curry, who led the development of the software. To address this challenge, Designer’s visual prompts double as a teaching tool. The templates expose you to art and graphic terminology like 3D claymation, photorealism, minimalism, or papercut.

“Those were the styles that people gravitated towards the most,” said Product Designer Kiana Price, who wrote prompts that defined the styles of Designer’s most recent templates. Alongside Kiana, there’s now a content team that includes several people dedicated entirely to the craft of prompt writing. The art of designing prompts has become a must-have skill.

“There are a ton of new AI features and products popping up every day, with a need for people who understand both the system’s underlying strengths and its limitations in order to properly use it as a collaborator in their daily work processes,” Price said.

Effective prompts connect Designer’s technical capabilities to user interactions. The artistic or design terms used in Designer’s templates become part of the user’s vocabulary. People repurpose them to modify and refine their own prompts. For some customers, talking to a machine is intimidating and that empty prompt box can be just as overwhelming as a blank page. Visual prompts get you started while instinctively conveying intent and eliciting action without explanation.

A birthday card featuring 3D-styled dinosaurs and a "Happy Birthday Carlos!" banner. The background includes clouds, mountains, and trees. A textbox reads, "Wishing Carlos a roaring good time on his birthday!" The left side shows a prompt box for generating text.
Producing a tailor-made birthday card.

Prominently displayed on Designer’s interface, the images of the visual prompts represent a high level of artistry that any creator can easily achieve. They vary in style, color, tone, application, and perspective. Like a greeting card aisle, the prompts display birthdays, holidays, celebrations, sports, travel, leisure, etc. Future iterations of Designer may include UX controls that allow you to edit the composition of your outputs, adjust colors, add localized effects, and have more hands-on manipulation of your images.

Designing with care: understanding exclusion & prioritizing ethics

Designer is unique from most graphic design software, which are historically more expensive and exclusive. They often require an art background or a specific skillset that can only be acquired by taking classes or going to an art school. Learning how to use those programs is a long commitment and a deterrent for some would-be creators who can’t afford the cost.

With the advent of large language models, ensuring ethical AI practices that protect the environment, vulnerable communities, and artists’ intellectual property is critical. While it’s exciting to see the development of these new technological advances, designing responsible AI means incorporating accountability, transparency, fairness, reliability & safety, privacy & security, and inclusivity. Being a product maker is one of the most instrumental roles a person can play in today’s tech landscape. What we create directly impacts billions of our customers, and so designing with AI requires a great deal of awareness and humility.

The team knew that if they crafted an easy and intuitive experience that delighted users in the act of creating, it would boost people’s confidence and unlock their creative potential. The effort people put into their outputs would garner a sense of meaning, especially when designing a birthday card for a loved one or creating an image that acts as an extension of who they are. By outwardly reflecting their inner world, Designer can help reinforce a true sense of value and self-expression.

Read more

To stay in the know with Microsoft Design, follow us on Twitter and Instagram, or join our Windows or Office Insider program. And if you are interested in working with us at Microsoft, head over to aka.ms/DesignCareers.