zh
English Español Deutsch Français Italiano Português (Brasil) Русский 中文 日本語
Submit post
Go to Blog

Google News Digest: Robots.txt, New Ads Types, GMB Profile Updates, and More

98
Wow-Score
The Wow-Score shows how engaging a blog post is. It is calculated based on the correlation between users’ active reading time, their scrolling speed and the article’s length.

Google News Digest: Robots.txt, New Ads Types, GMB Profile Updates, and More

Judith Lewis
Google News Digest: Robots.txt, New Ads Types, GMB Profile Updates, and More

The last two weeks were relatively calm in the search department, but certain things are going away and many new features were rolled out in Google Ads and Google My Business. Also, the search giant has started several new sites that are supposed to make small business owners' and creative makers' lives easier. So, keep reading to see all the updates, and please share any insights you have in the comments below.


GOOGLE SEARCH NEWS

Google Says Goodbye to Unsupported Rules in Robots.txt

Google announced on Twitter yesterday that they are saying goodbye to "undocumented and unsupported rules in robots.txt." 

In their blog post, they state, "While open-sourcing our parser library, we analyzed the usage of robots.txt rules. In particular, we focused on rules unsupported by the internet draft, such as crawl-delay, nofollow, and noindex. Since these rules were never documented by Google, naturally, their usage in relation to Googlebot is very low."

Google's Gary Illyes chimed in and explained sites are hurting themselves, so this part of the reason the decision was made:

What you should know is if you are using crawl-delay, nofollow, and noindex in your robots.txt file, they will be ignored by Google starting September 1, 2019. If you have relied on this method, Google has provided the following alternative options:

  • Noindex in robots meta tags: Supported both in the HTTP response headers and in HTML, the noindex directive is the most effective way to remove URLs from the index when crawling is allowed.
  • 404 and 410 HTTP status codes: Both status codes mean that the page does not exist, which will drop such URLs from Google's index once they're crawled and processed.
  • Password protection: Unless markup is used to indicate subscription or paywalled content, hiding a page behind a login will generally remove it from Google's index.
  • Disallow in robots.txt: Search engines can only index pages that they know about, so blocking the page from being crawled usually means its content won’t be indexed.  While the search engine may also index a URL based on links from other pages, without seeing the content itself, we aim to make such pages less visible in the future.
  • Search Console Remove URL tool: The tool is a quick and easy method to remove a URL temporarily from Google's search results.

Source: Google Webmasters TwitterA note on unsupported rules in robots.txt

New Search Features Tests

As usual, users spotted (and Barry Schwartz documented) numerous signs of new features tested on SERPs. Here are the most noteworthy of them:

Separate tabs for Google Shopping and other resellers in Google Shopping Box:

New "refine search" dropdowns for mobile search:

Google News Digest: Robots.txt, New Ads Types, GMB Profile Updates, and More. Image 0

New mortgage calculator design:

New local listing tabs, such as Product, Last Models, Video Game, Menu, etc.:

Google News Digest: Robots.txt, New Ads Types, GMB Profile Updates, and More. Image 1

"People also considered" ad carousel:

Sources: New Google User Interface Updates: Refine Layers, Mini-Carousels, Shopping Toggle, Mortgage Calculator & More, Twitter: SEMrushGoogle Testing New Branded Local Pack UI With More Tabs

GOOGLE ADS NEWS

New Ad Types: Live Stream and Swirl

Two new ad types are being tested in Google Display and Video 360 (formerly Doubleclick), Swirl and Live Stream.

Swirl ad is a 3D object that users can interact with, rotating it and zooming in/out. To make creating such models easier, Google plans to add a new editor to its 3D platform Poly

Google News Digest: Robots.txt, New Ads Types, GMB Profile Updates, and More. Image 2

Live Stream ads allow advertisers to include live streams in their display ads. These streams feature regular YouTube controls and can be watched in full screen.

Google News Digest: Robots.txt, New Ads Types, GMB Profile Updates, and More. Image 3

Both new ads types are now in limited beta; it is unknown when they will be available for all advertisers.

Source: Deliver more interactive ad experiences with Display & Video 360

Changes in Portfolio Strategies

Google announced a couple of changes to help simplify portfolio bid strategies. 

The first change is removing the ability to create Enhanced CPC portfolio strategies. They provided no additional benefits compared to individual campaigns and were not widely used, according to Google's statement. All existing Enhanced CPC portfolio strategies will be reverted to individual Enhanced CPC campaigns later this year.

Second, the target spend setting is no longer available for new maximize clicks portfolio. Google said, "Average daily budgets will be the way to determine spend for campaigns using Maximize clicks portfolio. Later this year, our systems will remove legacy target spend settings and instead use your average daily budget to manage spend."

Source: Simplifying portfolio bid strategies

Responsive Ads Creation in Google Ads App

Google Ads mobile app users are now able to create and edit responsive search ads (RSA). The new functionality includes creating and editing a list of negative words and recommendations on conflicting negative keywords. Thus, the Google Ads mobile app is gradually turning from a reporting and analytics instrument into a full-fledged tool for creating advertising campaigns. Now users can launch new promotions while on the go.

Google News Digest: Robots.txt, New Ads Types, GMB Profile Updates, and More. Image 4

Source: Create and edit responsive search ads in the Google Ads mobile app

Shopping Campaigns Data in the Landing Pages Report

You can now review the performance of your landing pages for Shopping campaigns by viewing the “Landing pages” page in Google ads. It provides users with the primary metrics for each specific page, such as clicks, impressions, CTR, conversions, etc. With this report, advertisers will be able to detect issues with their landing pages and fix them quickly.

Google News Digest: Robots.txt, New Ads Types, GMB Profile Updates, and More. Image 5

Source: See landing page performance from Shopping campaigns on the "Landing pages" page

GOOGLE TOOLS NEWS

Preferred Domain Setting is No More

Google has removed the option to choose your preferred domain in GSC. Previously, users could choose whether they wanted Search Console to display data for a WWW or non-WWW version of their site. Now the tool relies on canonical, sitemap, and redirects data to pick the default site version. There are still a few options to make GSC take a specific version of the site into account; you can find them here.

Source: Bye Bye Preferred Domain setting

New GSC Features

A couple of small but handy features appeared in Google Search Console. The first one is Markup search in URL testing tools, such as AMP Test, Rich Results, and URL inspection tool. You can see how it works in this tweet:

Also, you can now check the crawler type for each report (desktop/mobile) and see the annotation to learn when your site was switched to mobile-first indexing.

Sources: Twitter: Google Webmasters

Top Queries Change Alerts for GSC

Google has started to inform site owners of top queries changes. These emails contain information about top queries and how they changed in a week. It can help you track your site's most important keywords performance.

Google News Digest: Robots.txt, New Ads Types, GMB Profile Updates, and More. Image 6

Source: Google Search Console Alerts: Change In Top Queries For Your Site

Google My Business Updates

Several new tools for local optimization were released in the last couple of weeks:

  • Welcome offers allow businesses to reward users who start following them with special rewards, such as discounts, coupons, etc.

Google News Digest: Robots.txt, New Ads Types, GMB Profile Updates, and More. Image 7

  • Short names and URLs can now be claimed by businesses. Soon users will also be able to perform short name searched in Google Maps.
  • Business owners now can manually pick their profile cover photos.
  • Logos: company logo becomes visible at the top-right corner of the business profile. You have to complete core information to make the logo appear;
  • New Dynamic photo module is updated immediately after an image is uploaded. Soon businesses will be able to add captions to their photos.
  • Creating offline materials will now be easier with the help of a special website launched by Google. It allows businesses to quickly create stickers, posters, and other assets.
  • Local Favorite is a new label that will be shown for the top 5% of businesses in a particular category. The goal, according to Google, is "to recognize those businesses that consistently deliver a great experience for people."

Source: Helping businesses capture their identity with Google My Business

Google for Small Business Portal

To make small business owners' lives easier, Google has launched a new website called Google for Small Business. It contains information about various tools for small business.

By answering a few questions about their business, users can get a personalized step-by-step plan with a list of Google products to use to perform better online, attract more customers, and increase work efficiency. "The recommendations will include products to help with all three, but with a special focus on the goal that’s most important to you," said Google spokesperson Kim Spalding. Google for Small Business publishes news about small business tools and service updates and will provide readers with information on finding Grow with Google offline events in their area.

Source: A new way to discover Google tools to grow your small business

A New Site for the Creative Community 

Create with Google is a new site designed to inspire creative makers and to help them find ideas for presentations, speeches, and marketing campaigns.

In the Inspiration section, you can find examples of innovative and imaginative work, and you can "filter by format, vertical and platform, or search by keyword to inspire your creative thinking."

The Audience Connect "transforms creative presentations from broadcast into conversation, by tracking client engagement in response to any video." You can sign up for access here.

YouTube Mockup "lets you upload and showcase work to clients and colleagues in an authentic YouTube environment, simulating your creative across desktop, tablet and mobile."

The site is currently available in English, Spanish, Korean and Japanese.

Source: Create with Google: Inspiration, resources, and tools to fuel your next big idea

Suspicious Site Reporter Extension — Chrome

Two new features were released to help protect users from deceptive websites is now available for Google Chrome. "The Suspicious Site Reporter Extension will improve security for Chrome users by giving power users an easy way to report suspicious sites to Google Safe Browsing." 

Google has also launched a new warning designed to protect users about shady sites and deceptive URLs. They state, "This new warning works by comparing the URL of the page you’re currently on to URLs of pages you’ve recently visited. If the URL looks similar, and might cause you to be confused or deceived, we’ll show a warning that helps you get back to safety." Example: SEMrush.com vs. S3Mrush.com.

Apart from blocking you from visiting suspicious sites, the extension also allows you to send a report about such sites to Google.

Google News Digest: Robots.txt, New Ads Types, GMB Profile Updates, and More. Image 8

Source: New Chrome Protections from Deception


There is a quick recap of the most important recent news from Google. Stay tuned for our next digest in two weeks! 

Judith Lewis
Legend

Getting here is not easy at all!

Judith is a renowned writer, trainer, blogger and a digital and media consultant specialising in applying strategic understanding of digital technologies to help businesses innovate and optimise their effectiveness within the new, networked communications environment.
She is a regular speaker around the world on SEO, content strategy, link building, PPC, and digital strategy, a contributor to the Huffington Post, and has been recognised by her peers as one of the most influential people in the UK digital industry.
Judith judges the MENA, EU, US and UK Search Awards, an industry leading award and has every year since their inception. She has worked with market-leading global businesses including Google, Orange, COI, NatWest/RBS, National Gallery, CIPD, Fidelity, GalaCoral, NBC Universal, Readers Digest, Bayer, Family Search, Amadeus, AMD, AmEx, TotalJobs, Virgin.com, Virgin Startup & more.
She has over 20 years of experience, predating Google in her entry into the market and now runs her own consultancy. She has worked at Centaur Publishing, i-level, & Beyond before setting up her own consultancy. Judith also blogs about chocolate, helps with London Girl Geek Dinners, was a founding Modern Muse, and has contributed to the book “Pimp My Site” and the Econsultancy guide to Paid Search. She is currently working on updating the Econsultancy Best Practice guide to SEO and teaches the advanced SEO course as well as the Paid Search courses.
Share this post
or

Comments

2000
Borislav
Newcomer

Either just recently joined or is too shy to say something.

Thanks for that valuable information. You exceeded my expectations and I learned new things. Where all the other websites are making noise about the robots.txt you went deep into details about all those updates. A lot of my clients are running small businesses and they mainly rely on SEO so I can't wait to try all the new things I've learned about Google business.
Shailendra Vijayvergia
Helper

An experienced member who is always happy to help.

Looks like crash course on Google Updates. Thanks for sharing.
Judith Lewis
Legend

Getting here is not easy at all!

Shailendra Vijayvergia
It most certainly is! It's a lot of work to do but I'm glad it is appreciated. There is a whole team involved.
Caleb Sylvest
Enthusiast

Occasionally takes part in conversations.

Do we need to change anything with our Preferred Domain in GSC?

I still find dealing with http/https, www/non-www, and trailing/non-trailing slash stuff confusing and complicated. Any recommended resources to nail down the and manage the variations?
Judith Lewis
Legend

Getting here is not easy at all!

Caleb Sylvest
I love spreadsheets and .htaccess files because that's how I learned but really you need to establish rules. Important to note that training slash indicates that the file you are looking at is the root file for the folder and not an individual file as indicated by a non-training slash. Check out Wikipedia
Princy Tripathi
Enthusiast

Occasionally takes part in conversations.

Great insight and thanks for sharing!!
can you please elaborate on the thing noindex mentioned in the blog?
Judith Lewis
Legend

Getting here is not easy at all!

Princy Tripathi
I'm not sure I can really elaborate much beyond what was said - it was always undocumented and now it will be unsupported.
Ricci M.
Pro

Asks great questions and provides brilliant answers.

Thanks for sharing Judith. I will enjoy exploring Google for Small Business Portal and Create with Google. They look like they might be useful resources.
Judith Lewis
Legend

Getting here is not easy at all!

Ricci M.
They really both are! You'll enjoy them
Crazy Visitors
Enthusiast

Occasionally takes part in conversations.

WoW 😮 thanks for keeping us well updated with all the updates as of late. I have to give credit to the Twitter post referencing the new Mortgage Calculator & how it’s taking up a lot of “real estate” in the SERPs! Too funny 😁

Now maybe it’s me, as I’ve been practicing SEO for quite some years now, yet fairly new to the industry blogs such as this SEMrush; but it’s been easy to observe all these little updates like these code changes for robots 🤖 - Schema - Rich Results snippets, etc. Which leaves one to ask “Why?” I mean what’s the endgame here Google??

Furthermore, Google has been more talkative on the Social Media front - with its Twitter updates/responses to deny/confirm search algorithm updates like Core. In the past, they didn’t seem so responsive. Now I have taken into account that Corporate could be implementing a new game plan. But if you ask me, I believe they don’t want all this speculation going around with SEO communities etc. & instead want to push all of us into a cohesive, one-direction game plan.

When I combine all of this with Google’s SERPs looking & functioning more and more like a portal - it leaves one itching to ask why?

Thoughts? 💭

Cheers ;)

Chris @ Crazy Visitors 😜
Judith Lewis
Legend

Getting here is not easy at all!

Crazy Visitors
I really think Danny Sullivan has been pushing hard for more communication by Google to everyone - not just SEOs but business owners and the like. I hope that it will continue but as to whether Google is going to pull a Yahoo or AOL - I'm not sure. I don't think so but I wonder how much is to help people avoid phishing and spammy sites.
Nick Samuel
Pro

Asks great questions and provides brilliant answers.

Glad to see Google finally referenced and killed "crawl-delay"; I come across this semi-frequently and always thought it did absolutely nothing! Now I have tangible documentation :-P
Judith Lewis
Legend

Getting here is not easy at all!

Nick Samuel
It's also a relief that they are killing this stuff off - it was just confusing things!
Pratip Biswas
Newcomer

Either just recently joined or is too shy to say something.

Nice Post...Thanks for Sharing.
Judith Lewis
Legend

Getting here is not easy at all!

Pratip Biswas
Thanks for taking the time to read everything and for commenting!

Send feedback

Your feedback must contain at least 3 words (10 characters).

We will only use this email to respond to you on your feedback. Privacy Policy

Thank you for your feedback!