Adobe’s Upcoming AI Project Will take Image Compositing to a Next Level: Report

Adobe’s Project Clever Composites uses artificial intelligence for automatic image compositing. Specifically,  it predicts an object’s scale, identifies the optimal location to put it in an image, normalizes the object’s colors, estimates the lighting conditions, and generates shadows consistent with the aesthetic of the image. Every year at Adobe Max, Adobe displays what it terms “Sneaks,” which are research and development projects that may or may not become commercial products in the future. This year is no exception, and fortunately, we were offered a glimpse before the actual conference.

Adobe’s Upcoming AI Project Will take Image Compositing to a Next Level: Report

According to Adobe, Image compositing allows you to include yourself to make it appear as though you were present. Or perhaps you want to create a snapshot of yourself camping under a starry sky, but you only have daylight images of the sky and of yourself camping.

Adobe informs us that compositing can be a laborious, time-consuming, and substantially manual procedure. Typically, it entails locating a suitable image of an object or subject, cutting the object or subject out of the image, then adjusting its color, tone, scale, and shadows to fit the appearance of the rest of the scene into which it is being pasted. The Adobe prototype eliminates this process.

In this regard, Adobe researcher Zhifei Zhang told a source,

We developed a more intelligent and automated technique for image object compositing with new compositing-aware search technology. Our compositing-aware search technology uses multiple deep learning models and millions of data points to determine semantic segmentation, compositing-aware search, scale-location prediction for object compositing, color and tone harmonization, lighting estimation, shadow generation, and others.

Zhang claims that each model driving the image-compositing system is trained individually for a particular purpose, such as looking for items that are semantically and geometrically coherent with a given image. The system also makes use of a separate, AI-based auto-compositing pipeline that handles tasks including tone normalization, an estimate of lighting conditions, synthesizing shadows, and estimating an object’s scale and placement for compositing.

According to Zhang, the outcome is a procedure that enables users to assemble items with just a few clicks. Automatic object compositing is difficult to achieve because it requires the composition of multiple different process elements. Our technology acts as the “glue” that binds all of these parts together, Zhang said.

The Sneak system might always be a tech demo, as with all Sneaks. Zhang, though, claims that work is already in progress on an enhanced version that enables compositing 3D objects, not just 2D, and that she thinks it’d be a “wonderful addition” to Photoshop and Lightroom.

Check out? Adobe’s new Feature Content Scheduler Allows you to Directly Post on Social Media