Smart cropping for responsive images
Modern responsive designs require the same image in different places, shapes, and sizes. This makes images probable need to be presented in totally different sizes and aspect ratios when shown on different display devices (art-direction).
A common challenge in many web applications is how to adapt images for a fixed graphic design. Most of the time this means manually crop each image to get a defined size and aspect ratio. But, when we have thousands of images, manual cropping is expensive and time consuming.
A major challenge adapting images is how to crop them preserving the content and the aesthetics of such images.
The obvious solution is automatically crop and resize those images; but each image is different, with different elements, composition, and aesthetics. This makes difficult to define rules to automatically adapt them - there is no way to guess in advance whether to focus on the top, middle, or one side of the pictures -. So, how to crop those images for each of the required without losing the most important content?
Smart cropping for art-directed automatic image optimization
Smart cropping has become a reality, automatically choosing the best image crop according to measures of visual importance - beyond basic face detection - . At Abraia, we have even managed to keep the aesthetic quality in the final composition and choosing a good cut around the important object.
Smart cropping improves art direction and visual experience compared to cropping using fixed shapes and positions and even avoids unnecesary page weight when compared to classic cropping but using the client browser
If Medium were to use smart cropping this is what I would get instead of the awkward crop shown above.
This is what Medium does if I try to feature the image below for this post. It leaves barely any vestige of the original message in the image If Medium happened to use smart cropping I would be able to get this and feature the image I had in mind
This cut is automatic too. The difference is I wouldn't had to look for one more nice picture in pexels :) that's simply well centered, I would be free to feature the image I had originally in mind.
A similar approach allows to drive automatically the overlay of text or logos on images without damaging key elements of the composition.
This is about coping with complexity -not about replacing the human decision-, since models should always play the role of protectors of a master design in which the focus, aesthetics, and key creative variations have already been crafted and validated.
These new tools lay steps contributing to the promise of bridging design and code development, easing the collaboration between designers and developers.
Abraia smart cropping
Abraia's content-aware cropping technology is built on perception models to automatically detect the region of interest every image. Then, based on the most salient parts of the image, aesthetics is used to select the best crop. Finally, the selected crop is resized to fit the size of your site's layout without losing focus on important content.
The algorithm analyze each image to extract the saliency map -as showed in the next picture-, which identifies the elements and features that captures attention -under free viewing conditions-.
Heatmap generated with Abraia's saliency model
Crop candidates are generated based on this map and scored for content preservation and boundary simplicity. This is about the undesirable effect of cutting through an object (it may not only ruins the balance, but also create unnecessary distraction). Finally, the best crop is selected based on aesthetics evaluation for best visual composition, which refers to the placement or arrangement of visual elements in a photo, as shown bellow.
Square smartcrop generated by Abraia technology
Responsive use case
Since every image is unique in its contents, it turns challenging to deliver an optimal image to every layout, especially when you need to scale up to hundreds, thousands or even millions of images.
Suppose that you need to adapt different images for three different layouts on a website. For instance, three versions are demanded: a landscape (3:1), a square image (1:1), and a portrait (2:3).
Abraia provides a set of unique algorithms to smartly crop and resize images delivering the best cropping area to preserve the content and the aesthetics of your images.
Food photos use case
For instance, the design of a food website requires thumbnails of 400x200. Images are uploaded as are (i.e. without editing) and need to be adapted to this fixed aspect ratio (2:1). Each image needs to be cropped and resized in different way, with our smart cropping technology the algorithm chooses the most appealing part of each photo, based on the content of each photo.
To automatically generate those thumbnails of food, the user just needs to specify the final size of the image, and this is manipulated and delivered on-the-fly. Behind of this simple request is a complex algorithm that the image how it is perceived to provide the best human-like results. Replacing the completely infeasible alternative of having a real person sitting 24 hours a day to manually analyze and crop every single one of your images or user uploaded content.
Faces use case
Other common use case is reusing a photography for a small sized profile picture - e.g. for a company overview page -. With the smart cropping for faces we can get a portrait picture from a generic image. This allows directly getting a face crops from general pictures.
It automatically detects the largest face in the image and makes it the focus of the transformation. After that, it automatically selects the best aesthetics crop centered in that face.