Syda Productions - stock.adobe.c

Google AI art tool turns 2D creatures into 3D models

These 3D chimera creatures are brought to you by Google's Chimera Painter, an AI-powered tool that turns 2D images into 3D creature models.

An AI tool can only do so much.

But if you're good enough, you can create hideous creatures brought to you by Google's Chimera Painter, a trained machine learning model that automatically fleshes out a monster based on a user-created outline.

The Google AI art tool, released Tuesday, is available free to users in a demo form.

Easy to use, hard to master

Chimera Painter is deceptively simple: It's a paintbrush that enables users to distinguish specific body parts of the outlines of their drawings. Once an outline is complete, the tool will automatically create a 3D model based on it.

In Greek mythology, the Chimera is a fire-breathing monster depicted as a lion with a serpent's tail and the head of a goat protruding from its back. A chimera refers to an organism that contains a blend of genetically distinct tissues.

Yet users need some artistic skills to sketch a creature that will look like a full-fledged monster chimera.

As explained on the Google AI blog post that unveiled Chimera Painter, the tool -- designed to produce video-game-ready creature images -- is powered by generative adversarial networks (GANs) informed by artists.

Competing neural networks

A GAN is a machine learning model in which two convolutional neural networks compete with each other to become more accurate in their predictions. For Chimera Painter, one network created new images, and the other network tried to determine if the pictures were from a sample training data set, which featured artist-created images. 

The outlines characterized the shape and size of the creature and separated out each identifiable body part, including the legs, eyes, ears, head, muzzle, horns and wings. After training the model, it was then used to produce multi-species chimeras based on those outlines, according to Google. Researchers put the best model into Chimera Painter.

To create recognizable creatures, users need to develop recognizable outlines. A badly drawn outline will output a badly textured 3D model.

In training the GANs, Google researchers built a data set of full-color images with single-species outlines of creatures adapted from 3D creature models. The team had to create a semi-automated way to build enough creature models to train the networks, in which artists created or obtained a set of 3D creature models, and then made two sets of textures to overlay on the model using the Unreal Engine, the blog noted.

One set of models contained the full-color texture, while the other had flat colors for each body part. The researchers gave the second set to the GAN during training to ensure that it learned the different sizes, shapes and textures of each body part across different creatures. 

Google researchers compiled more than 10,000 image and segmentation map pairs per 3D creature model using this method, according to the blog.

To create recognizable creatures, users need to develop recognizable outlines. A badly drawn outline will output a badly textured 3D model. Chimera Painter appears to use machine learning models with the capacity to recognize high-quality, intricate outlines.

Google AI art, Chimera Painter
An unskilled artist uses Google's Chimera Painter demo.
Google AI art, Chimera Painter
A skilled artist uses Google's Chimera Painter demo.

Still, the software fails to neatly join outlines that a young child or someone with limited artistic ability might create. (Note the accompanying images -- one created by someone with little artistic ability; the other by a more skilled visual artist.)

The tool could potentially save artists hours spent rendering concept art for a target medium such as video games or card or board games.

If anything, the new tool helps highlight how far AI has come in terms of being able to augment human creativity. While it's hardly the first AI-powered tool to create art, Chimera Painter comes amid a new push in the AI industry to use machine learning to work in concert with and perhaps stimulate human creativity.

Recently, startup AI21 Labs released Wordtune, a tool that can automatically rewrite sentences. A competitor, Grammarly, has also released tools that use AI to improve a person's writing. Meanwhile, numerous startups use AI to help musicians create music for various venues, including video games.

Next Steps

The creative thief: AI tools creating generated art

Dig Deeper on AI technologies

Business Analytics
Data Management