AI-generated art is technology’s latest breakthrough. It has become both an interesting and controversial topic within artist circles and even the business world.
Artificial intelligence is changing the world in ways we never imagined. From self-driving cars to voice assistants like Amazon Alexa, AI has become a part of our daily lives. In recent years, it’s also started making its way into the professional 3D art community.
AI-generated art has been making waves in the professional 3D art community for years now. However, it’s only recently started to become a viable option for professionals and hobbyists alike. While this new technology has many benefits, it also raises some important questions about what it means for artists in the field.
AI-Generated Art: What Is It?
AI-generated art is a type of computer-generated imagery (CGI) that uses artificial intelligence to create original works. It’s often created using generative adversarial networks (GANs), which are algorithms used in machine learning. These algorithms work by creating two separate neural networks that compete against each other. One network attempt to make realistic images while the other tries to spot inconsistencies in its creations.
The networks then learn from each other, creating more realistic images over time. As a result, GANs have become an increasingly popular tool for creating art. While AI-generated art is still fairly new, it’s gaining popularity as more people learn about it and experiment with the technology.
In simpler terms, you input a text description and the algorithm compares the text to its huge compost of captions and alt text for images that would correspond to the text prompt.
Next, the AI generator then applies it to the first representational mapping derived from the text description by reverse engineering from this mush. These extracted features are randomly put back together by the decoder into recognisable patterns.
Finally, it uses extra textual information to determine whether what it has put together is semantically coherent with the initial text prompt.
AI-Generated Art Goes Mainstream
While AI-generated art is still fairly new, it’s gaining popularity as more people learn about it. In February 2019, the Museum of Modern Art in New York City added its first ever AI-generated painting to its collection. The piece called “Portrait of Edmond De Belamy” was created by a French art collective that goes by the name Obvious.
In even more recent news, this year a man won an art competition by entering an AI-generated artwork. Needless to say, these raised thousands of eyebrows and stirred a cornucopia of opinions online.
Jason Allen, the “human” behind the winning AI-generated art titled “Théâtre D’opéra Spatial” had this to say, “I knew this would be controversial. How interesting is it to see how all these people on Twitter who are against AI generated art are the first ones to throw the human under the bus by discrediting the human element! Does this seem hypocritical to you guys?”
A quick Google search will lead you to many similar conversations online while every now and again a new AI art tool crops up in a race to becoming the best.
Current and Upcoming AI Tools for Artists
There is a myriad of AI tools available, spanning years and differing algorithms. In this blog, we will talk about the three most common and most popular AI tools being used within global 3D artist communities to produce AI-generated art.
With the use of these technologies, even complete beginners can now produce intricate, abstract, or lifelike pieces of 3D art by simply entering a few lines into a text field.
DALL-E 2
DALL-E 2 is an AI tool that was developed by OpenAI. Through machine learning, DALL-E 2 has become familiar with how images relate to the language that describes them. It employs a technique known as “diffusion,” which begins with a pattern of random dots and eventually transforms that pattern into an image as it picks up on particular details of that image.
It is used to create realistic images of people and objects. DALL-E 2 can also be used to create portraits, landscapes and other types of scenes with a high degree of success and accuracy.
Interestingly, the name of the software is a combination of the surrealist Spanish artist Salvador Dal and the name of the animated robot character WALL-E from the Pixar film series.
Midjourney
Midjourney is an AI tool used to create realistic images of people and objects. Midjourney can be used to create portraits, landscapes and other types of scenes.
David Holz, the company’s founder, claims that he views artists as clients rather than rivals.
Currently, the only ways to access Midjourney are through a Discord bot on their main Discord, via direct messages, or by inviting the bot to a different server. Users use the /imagine command, similar to other AI art generator applications, and enter a prompt to produce images. The bot then displays a picture.
As a matter of fact, Midjourney was used by The Economist, a British magazine, to create the June 2022 issue cover.
Stable Diffusion
Stable Diffusion is a text-to-image model with deep learning that Stability AI, a firm, released in 2022. Although it can be used for various tasks including inpainting, outpainting, and creating image-to-image translations directed by text prompts, its primary usage is to generate detailed visuals conditioned on text descriptions.
The Stable Diffusion model supports both the ability to redraw existing images that incorporate new elements described within a text prompt and the ability to generate imagery from scratch by use of a text prompt describing elements to be included or omitted from the output.
When combined with a suitable user interface that supports such features, of which there are many different open-source implementations available, the framework also permits the use of prompts to partially edit existing images via inpainting and outpainting.
The Future of the 3D Artist Community
With the rise of AI-generated art, some communities have started labeling the technology as anti-artist, hinting at possibilities of AI replacing 3D artists. At XO3D and as artists ourselves, we beg to differ.
For one, these AI tools make our iteration process quicker.
Case in point: Generating textures and materials
Jayesh, our Junior 3D Visualisation Consultant here at XO3D took on an exploration of generating materials from MidJourney and rendering them in Keyshot.
This is a case study on ceramics. You can see more images and the full process on Behance.
Upon entering the text prompt, “rough teracotta ceramic texture flat,” these were what Midjourney generated:
Taking these AI-generated textures and materials and the addition of the “human element” in creating maps and tileable textures in Photoshop, Jayesh produced this render in Keyshot:
Doing another text prompt, “black and white heightmap of Indo-Japanese traditional texture patterns,” these are what Midjourney generated:
To explain the process briefly, in a 3D space a heightmap can be used in displacement mapping to move the actual geometric position of points over the textured surface, bump mapping to determine where this 3D data would cause shadow in a material, or terrain where the heightmap is transformed into a 3D mesh.
After obtaining these AI-generated heightmaps from Midjourney, they are then taken to Photoshop to transform them into seamless tileable textures. This is a crucial step the “human” artist must take in order to properly materialise any object in a 3D render.
Finally, after careful attention to detail this is the resulting Keyshot render:
Using Midjourney and other similar AI tools sped up Jayesh’s workflow and prototyping process. This allowed him to produce the 3D renders quicker without having to manually source the materials and textures he needed to create a photorealistic result.
Takeaway:
Although the algorithm may produce visually pleasing images, it operates in a socially isolated creative environment. People, places, and politics, on the other hand, provide as inspiration for human artists.
Finally, artificial intelligence tools can add new options for artists and even provide great benefits, but humans will still be the ones to push the boundaries of creativity.
