When Lvmin Zhang was in high school, he says, everyone thought artificial intelligence was science fiction. A machine couldn’t possibly hold a conversation with a person.
That was then. Today Zhang, a graduate student at Stanford University, is at the forefront of AI development, making it easier for artists to create what’s in their heads. Digital art can be painstaking—getting an image or an idea from brain to chip by using software or asking AI to build a vision, layer by layer, without its having any a priori knowledge of what the artist is trying to create.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Christie Hemm Klok
Zhang’s AI, ControlNet, aims to fix that by giving image-creating AI systems what amounts to directions. It provides parameters allowing a user to build on an initial prompt in a way that keeps the parts of the image they want, adjusts to refining prompts and designs the product they have in their brain. Previously this process was hard—the image-creating AI wouldn’t necessarily know, even with the most precise prompts, how to add or shade or place objects in the artwork, for example. ControlNet lets the computerized image creator into the mind of the human creator.
Zhang’s work, developed not long after he started his Ph.D., is becoming popular. He recognizes that the use of AI in art is something that both creators and viewers are still grappling with on a philosophical (Is this art?), if not legal (Is copyright involved?), level, but he believes that AI can be helpful. “The real questions are what it can do, what it can’t, why it’s capable of the things it is and what it should be doing in the first place. That’s the groundwork we need before AI can really help people in ways that are efficient, honest, genuinely engaging and thought-provoking and that, at the end of the day, have some real heart to them.”
This article is part of “The Young American Scientists,” an editorially independent project that was produced with financial support from Regeneron. Special thanks to the Exploratorium in San Francisco, where Zhang's portrait and video interview were captured.