AI alchemy: DALL·E 2 & the future of generated art


The research company OpenAI introduced the original DALL·E in 2021, but it quickly refined the model as early users experimented with the tool. DALL·E 2 arrived in the spring of 2022, a series of AI systems working in concert. One is OpenAI’s CLIP (Contrastive Language-Image Pre-Training) model, an AI trained to identify hundreds of millions of images and translate them into text descriptions. When a human prompts DALL·E 2 with an image request, CLIP works as a “text encoder,” helping the AI system understand and synthesize elements of the human prompt. 

Next, the AI system combs through hundreds of millions of images classified and compressed in “latent space,” a hidden realm where AI groups millions of images according to their similarities. Latent space is a bit like Hanger 51 at the end of “Raiders of the Lost Ark,” a seemingly endless warehouse with artifacts sorted into crates according to some arcane method that’s too complex to fathom. Finally, the AI system applies an “image decoder” to translate its discoveries into a set of 10 different images corresponding to the original human prompt. Users can even modify these AI-generated images with simple text commands through inpainting and editing functions.

DALL·E 2 is only in preview mode right now, open to a small number of “trusted users” who have already created more than three million images. The company employs text filters and automated analysis to flag images that break the company’s strict content policy that only allows “G-rated imagery.” Users can lose access if they create or distribute images that reflect hate, harassment, violence, adult content, illegal activity, deception, or spam. 

Beyond the content limitations, OpenAI has an even stricter policy about artists profiting on images they create, forbidding the use of DALL·E 2 to create NFTs or other commercial ways to “license, sell, trade, or otherwise transact” the generated images. “I’m personally very excited about NFTs,” said Future when asked about possible scenarios for the future of AI-generated art. “The closer the relationship we can get between creator and audience, the better. I want my subscribers to be economically incentivized and rewarded for just being a supporter. They can be along with my journey.”

For creators looking for more freedom, a community of artists and coders have been working to offer open-source tools for creators to use without such strict usage rules. This community has developed AI tools like CLIP Guided Diffusion, Disco Diffusion, and Centipede Diffusion to utilize and reproduce the functionality of DALL·E 2. This community has pioneered the art of “prompt engineering,” designing optimal text descriptions to make AI create better art. 

 Chris Allen is a digital artist and musician obsessed with this new field. He has spent months working with Disco Diffusion, an open-source tool that accepts text prompts and uses OpenAI’s CLIP-Guided Diffusion neural network to produce custom artwork. While the results can’t quite match DALL·E 2’s performance, thousands of artists have joined the Disco Diffusion community, making digital images, video art, and NFTs.   

“Eventually, there’ll be a continuum,” Allen told me, looking ahead to a future when  DALL·E 2 is available for all artists to use. “There will be corporate-sponsored access with gated communities. For OpenAI and other companies, that will be their business: inventing cool machine learning things and making them available on a subscription or contract basis. Then there will be this long smear of crazy experimental stuff.”

Allen wrote “Zippy’s Disco Diffusion Cheatsheet,” a constantly evolving handbook that introduces artists to this popular open-source AI system. “I love making the art, and I love sharing what I’ve done,” he said. “I love helping other people. It’s a good feeling to know that I’m helping someone achieve their goals, in the same way that someone helped me.”

Allen has created a series of science fiction-themed digital artworks, NFTs, videos, and films. He showed me “Hybridized Crops,” a work of art created with Disco Diffusion, as well as digital editing and animation tools. He has minted 11 editions of this surreal NFT. In the endlessly looping video, the camera moves like a microscope, zooming deeper to reveal microscopic nanobots inhabiting the veins of a plant, organic and inorganic life co-existing at the atomic level. Allen spent hours generating, refining, and stitching together these images on his home computer, tapping into the mighty processing power of Google’s cloud-based computing services. “Lots of digital is very literal. What you put in is exactly what you get out,” concluded Allen. “With AI, you are just steering the horse and hoping it goes the right way. It’s a very liberating experience.” 

Related Posts

Crypto scams fall 65% after gullible noobs exit the market: Chainalysis

Crypto scams fall 65% after gullible noobs exit the market: Chainalysis

Fewer people have fallen victim to cryptocurrency scams in 2022 so far due to falling asset prices and the exit of inexperienced crypto users from the market,…

Mexico Beckons as Patron Brings its Tequila to Decentraland

Mexico Beckons as Patron Brings its Tequila to Decentraland

As the summer season gets into full swing, it was only a matter of time before tequila made another appearance in the metaverse. Now, purveyor of high-quality…

IRS takes out John Doe summons on crypto prime dealer SFOX to find tax cheat customers

IRS takes out John Doe summons on crypto prime dealer SFOX to find tax cheat customers

The Central District of California federal court entered an order Monday to authorize the United States Internal Revenue Service (IRS) to serve a John Doe summons on…

How do Ownership and Copyright Work for NFTs?

How do Ownership and Copyright Work for NFTs?

Answers to critical questions surrounding intellectual property, copyright, and ownership of NFTs Although the popularity of NFT collections recently skyrocketed, the answers to pressing questions surrounding intellectual…

NFT Naruto Museum signs an agreement with The Jackson Family Foundation

NFT Naruto Museum signs an agreement with The Jackson Family Foundation

Being a legend of pop music, an iconic dancer and a truly prominent celebrity of all times, Michael Jackson’s unmatched legacy is eternal. To make it everlasting…

BlockFi tops the Inc. 5000 list with almost 250,000% revenue growth in three years

BlockFi tops the Inc. 5000 list with almost 250,000% revenue growth in three years

Inc. magazine has named BlockFi the fastest growing company in the United States in 2022. At the top of the magazine’s Inc. 5000 2022 list, it experienced…