Figuring out the right text prompts to yield the best results with AI systems like OpenAI’s DALL-E 2 has become a science in its own right. Now a startup is looking to let “prompt engineers” cash in with an online marketplace that sells these finely tuned phrases.
PromptBase, launched in June, allows users to sell strings of words that net predictable results with particular systems. Priced at $1.99 — PromptBase takes a 20% cut — the content that the prompts generate range from “viral” headlines to pictures of sports team logos, knitted dolls and animals wearing suits.
At the moment, PromptBase hosts only prompts tested on DALL-E 2 and GPT-3. But according to its founder, Ben Stokes, the plan is to expand the platform to additional systems in the future.
“Our ultimate aim is to build tools in order to help support prompt engineers. It’s early days, so we’re currently just trying to spread the word and find prompt engineers to sign up and start listing their prompts for sale on our marketplace,” Stokes told TechCrunch via email. “We’re already seeing big tech companies build their own systems similar to GPT-3 and DALL-E, and I predict many more to come. Different systems will likely be utilized like tools in a toolbelt, similar to how different programming languages are used today, and we plan to accommodate all of them as they gain popularity.”
Selling prompts isn’t against any AI provider’s terms of service, but it potentially opens a can of ethical and legal worms depending on the nature of the prompts being sold. Moreover, it reveals the fragility — and unpredictability — of even the most capable AI systems available today.
Prompt engineering
Prompt engineering is a concept in AI that looks to embed the description of a task (like generating art of furry creatures) in text. The idea is to provide an AI system “guidelines” or detailed instructions so that it, drawing on its knowledge of the world, reliably accomplishes the thing being asked of it. In general, the results for a prompt like “Film still of a woman drinking coffee, walking to work, telephoto” will be much more consistent than “A woman walking.”
Prompts can be used to teach an image-generating system to distinguish between “an image containing potatoes” and “a collection of potatoes,” for example. They can also act as “filters” of sorts, creating images with the characteristics of a sketch, painting, texture, animation or even a particular illustrator (e.g., Maurice Sendak). And prompts can portray the same subject in different styles, like “a child’s drawing of a koala riding a bike” versus “an old photograph of a koala riding a bike.”
Prompts can be quite nuanced. Owing to the way AI systems make sense of patterns in images and text, not all of them have a predictable — or even sensible — structure. For example, the prompt “A very beautiful painting of a mountain next to a waterfall” returns worse results with DALL-E 2 compared to “A very very very beautiful painting of a mountain next to a waterfall.” The reason? The system attaches an inordinately high value to the word “very.”
It’s worth noting that the “very” example is specific to a particular iteration of DALL-E 2 and most likely wouldn’t work on another. But that’s a major reason prompt engineering can be valuable: discovering edge cases.
Language-conditional models can act a bit like decision transformers, in that you can prompt them with a desired level of "reward".
E.g., want prettier #dalle creations? "Just ask" by adding "[very]^n beautiful":
n=0: "A beautiful painting of a mountain next to a waterfall." pic.twitter.com/vu0NceTxAv
— Phillip Isola (@phillip_isola) June 2, 2022
In a fascinating study out of the University of Texas at Austin, researchers documented an extensive vocabulary of bizarre prompts that can be used to generate images with DALL-E 2. They discovered that the system understands “Apoploe vesrreaitais” — a gibberish phrase — to mean “birds” and “Contarra ccetnxniams luryca tanniounons” to mean “bugs” or “pests” (sometimes). Giving DALL-E 2 the prompt “Apoploe vesrreaitais eating Contarra ccetnxniams luryca tanniounons” yielded pictures of birds eating bugs.
Although these nonsense words probably correspond with some internal logic in the system, that’s why some data scientists have likened prompts to “incantations” or “magic words” — and why prompt engineering has catalyzed an entire field of academic study.
Problematic prompts
A number of researchers and enthusiasts have released free resources containing prompts for popular AI systems, mostly DALL-E 2. PromptBase is one of the first to monetize the exchange — and it already has critics. There’s a long-running debate within the AI community over which research, if any at all, should or can be commercialized; one Reddit user argues that PromptBase is “starting a trend that threatens the openness and accessibility of AI in general.”
But Stokes defends the model, arguing that many of the prompts on PromptBase represent hours of genuine work and insight by engineers.
“Today we have prompts to generate basic text and images, but it’s not too hard to extrapolate years into the future where we’ll have prompts for generating videos, and maybe one day even feature-length films complete with orchestral scores,” Stokes added. “Those people who can craft the quality prompts required guide the AI to do these things will be extremely valuable. It’s unknown how big the market will be, but I can see it being a key tech skill, if not the future of programming.”
Of course, there’s little to prevent a PromptBase customer from publishing a prompt post-purchase. But that could be the least of PromptBase’s problems.
Studies show that language systems trained on vast swaths of public data, like GPT-3, can “leak” personal information, including names and addresses, when fed certain prompts. Some prompts might encourage copyright infringement, like those instructing DALL-E 2 to generate “3D models of Pokémon.” Others could be used to defeat word-level filters to get an image-generating system to output “restricted” images, researchers theorize — like images of violence (e.g., “a horse lying in a puddle of red liquid”).
Exploring Dall-e 2: A thread on its ability to understand different languages, typos, and some other findings.
Prompts in other languages are somewhat reliable, although less reliable than English, obviously. Sometimes it won't output what you intended, but this is explainable pic.twitter.com/UZoii1rVNc
— Igor Baikov (@igor_baikov) June 7, 2022
Stokes said that PromptBase reviews every listing in the marketplace to ensure they don’t violate any “AI generation rules.” But if the business grows, it could become tougher to maintain that level of scrutiny.
Vagrant Gautam, a computational linguist at Saarland University in Germany, agrees that there’s a potential for misuse. However, Gautam — who goes by the pronoun xe — also notes that the prompt marketplace could present an income opportunity for artists and other folks who are creative or skilled at debugging.
“[It points] to the importance of prompt engineering, as well as the importance of the skills involved in doing this — creativity, time, adversarial thinking, etc. A lot of people who’ve been saying that DALL-E 2 is going to make it so easy for them to generate images or art of whatever they want are discovering that there’s an art to doing this and it often takes many tries,” Gautam said.
These tries can become expensive, given systems like DALL-E 2 aren’t exactly free to use. Stokes himself says he paid a “fortune” trying to figure out a prompt for GPT-3 at another of his ventures, Paper Website.
“People are now also complaining about its monetization because they say there’s too few opportunities to tweak your prompt before you have to start paying,” Gautam continued. “I find it very interesting — this trial-and-error, adversarial approach that people have to take to figure out exactly how to prompt generative models to do what they want.”
It’ll be a while before the dust settles in commercialized prompt engineering. But if nothing else, PromptBase will raise — and already has raised — issues around the AI systems that stand to transform countless industries.
Comment