Adobe Firefly

Firefly is the next benchmark in Adobe’s portfolio, an AI tool focusing on generating and interpreting imagery through text and search functions. When used within Photoshop, it's called Generative Fill. Adobe's website cliams groundbreaking and industry-shaking capabilities: "Explore and experiment with ideas in seconds, not hours. Use Generative Fill to create dozens of high-quality concepts quickly with results only a few keystrokes away". Setting aside for a moment the controversy that has arisen within various art communities surrounding its use, I was curious: What can this tool actually do?

Generative Fill, in theory, observes a selected portion of the canvas in Photoshop, resources the Adobe library of imagery, and attempts to fill the selection with your search to match the existing canvas. I wanted to test the waters to see if I can search for key features to add to scenes we typically encounter at Fat Pencil. The goal was not to execute perfect renditions, but to see how detailed the results can be. So, I set out to create scene demonstratives within Photoshop.

The exploration with Generative Fill

The first demonstrative I produced is a scene described by an eyewitness. The intention is to help visualize what they may have seen, including details they recalled about the day and who may have been involved.

I generated the first image by selecting the entire canvas and searching for a street view to establish the environment. Beyond the base image, I selected various sections of the image to illustrate a person looking left towards a fast-driving car. Then, I could search for decals and details to add on top of the generated vehicle. I also attempted to highlight the windshield to place a driver within the car, but with tinted windows, it couldn't place much that was identifiable.

The next scenario illustrates a conflict among multiple people inside a bar or restaurant. The focus was to see if the program would be able to generate multiple people holding objects and allow interaction between them.

This scenario produced a more painterly result instead of accurate forms. Several failed attempts created not-so-believable crowds. What you see above is the most successful generation, with the exception of the aggressor. Attempts to have characters interact were either forbidden by software policies depicting violence, or limited in generation.

The final scenario I tried was of a break-in at a café at night—possibly something a security camera would capture. I attempted to put a suspect behind translucent material like glass to obscure the subject, change lighting from day to night, and see if any additional weather conditions could be applied to the outside view.

This scenario produced by far the most straightforward results of any others I tried. Though Generative Fill didn't quite understand transparency with items like rain and glass, it worked well enough in replacing the shape it was told to overwrite. Lighting and light direction was convincing enough for a demonstration.

Final Thoughts

Generative Fill and Adobe Firefly produced surprising imagery that with additional editing could be quite useful. Just like the prompts given, the work produced seems to stand as more of a suggestion than anything foundational to a visual pipeline. I am under the impression that the tool has a very limited library to select from. Even with the vastness of Adobe Stock and other materials Firefly resources, I can imagine that only a fraction of those images have been processed, tagged, and prepped for Generative Fill. The tools are still in beta, so in due time and with a wider set of generative options to pick from, this tool may become very effective at producing accurate results.

One aspect of Firefly's policy that stood out to me aims to prevent any malicious or false imagery with Generative Fill, but with the right wording, anything is possible.

Sources and links