In Part 1 of this three-part series, we took a look at the evolving role of images in search results. This second installment explores Google's increasing understanding of image content and its impact on the SERP. In Part 3 we examine case studies showing the impact of images on rank & conversions + how to take advantage of the image landscape for your local SEO.

Increasing Understanding of Image Content

The underpinnings of Google’s photo AI were established as a feature within Google+. Google Photos was spun out of G+ and became a stand-alone product in May, 2015. It clearly demonstrated that Google was increasingly able to understand image content. The Google Cloud Vision API beta, making Google’s sophisticated image ML/AI available to developers was rolled out in late 2015. Google Lens, demonstrating Google’s ability to search using an image, was announced 18 months later.

In late 2020, Google announced that "more than 4 trillion photos are stored in Google Photos, and every week 28 billion new photos and videos are uploaded." With an endless supply of new photos to feed to their ML, increasing use of Lens to understand user queries and an active Cloud Vision AI developer community, Google can continue to train and refine image analysis to parse image content for ever greater detail.

When you examine the results of an image analyzed by the Google Cloud Vision API, you get an idea of the depth of Google’s "understanding"of photos. It can identify prominent natural or human-made landmarks, assess facial expressions, detect objects, identify entities, understand logos, colors and text as well as determine how likely the image is to be "inappropriate." Cloud Vision can not only recognize clothing in an image and label colors, patterns and garment types but evaluate the nuances of a person's clothing style.

It has long been said that an image is worth a thousand words, although there is a growing body of research indicating it can also be more emotive than words. Google seems to get that from an image and more. For Google an image is not only worth a thousand search phrases but also worth several hundred entities, multiple emotions and even a few semantic triples. This is all directed toward improving search results.

Segmenting Images Based on Content

Google is increasingly leveraging AL/ML image analysis to improve its understanding of local entities.

A small example of this is reflected in the classification of images by product type within the Google Business Profile (GBP). This product segmentation allows the user to easily look at images in the category of their choosing. This feature seems to be based on the quantity of categorical level images, not the business's use of a product feed.

Google Matching Images to Intent

Beyond categorization, more importantly, Google is matching images to search intent. While this is true throughout the SERP, it's particularly noticeable in local results.

If you search for engagement rings Williamsville, for example, Google will attempt to show local businesses’ images of engagement rings. If you search for "pearl earring Williamsville NY," Google will swap in earring images if the business or consumers have uploaded them.

What shows infrequently at best, on these high intent searches, is the business profile image. So don't think your profile image will be the main image users see.

Why Is Google Doing This?

Google isn't stupid (ok, maybe a little). But, as noted, the company tests everything and uses what works.

Images are getting bigger, showing in more carousels, showing more frequently in mobile search results and now on the desktop as well. Images are playing an ever-increasing role in search results for a reason.

The reason is the same reason Google always gives: from their point of view, it is better for the user and the search experience. (So you don't think me wildly naive, I would point out it's also better for Google; it provides opportunities to keep users on Google to gather more data, show more ads or capture one more transaction.)

Google's own survey in 2017, of 16,000 adults, found:

  • 90% of purchasers are more likely to use a business that has photos on a search/map results page
  • For 74% of shoppers, seeing photos of the business on the results page indicates a more reputable business

In the final installment, I'll present two case studies, explore the ranking impact of images and "image SEO." I'll offer practical tips for business owners and marketers and discuss the future of visual search.


The Visual Future of Local Search: Read and part 1 and part 3.

Part 1- Images are playing an increasing role in mobile local search Google has increased the space allocated to images over the past 5 years

Part 3- Case studies showing the impact of images on rank & conversions + how to take advantage of the image landscape for your local SEO