Google Adsense Site Diagnostics

I am never care about how Google Adsense works on my sites. Because the income is small. However, I think why the money are small, except the reason is visiting traffic, but there might be other reasons such as there might be some problem on my site so that Google Adsense can not work well;

OK, let me go to my Google Adsense account and see what help I can find:

First I see there is a menu item named “Site Diagnostics”, interesting, why I did not see it before .

"View detailed diagnostic information about the pages Google crawls to display relevant ads. If our crawler finds any issues with your pages, they'll be listed in the table below. We encourage you to resolve these issues in order to allow us to crawl more of your pages and provide relevant ads.”

I see there are lots of “Blocked URL” records, OMG, I never known it!

I checked the “Reason Blocked”:

Why is my URL showing up as blocked?

Your URL might show up as blocked in the site diagnostics tool for a number of reasons. See below for specific reasons and remedies.

Robots.txt file:
If you have a robots.txt file, the page requesting Google ads may be marked as 'disallow' in your robots.txt file. You can update your robots.txt file to give us access by following these instructions.
If you do not have a robots.txt file, please create one in the root directory of your domain and then update it by following these instructions.
If you don't know whether you have a robots.txt file, please consult your web provider or webhost and ask them to modify the robots.txt file on your domain to comply with these instructions.

I checked my site, I do not have a robots.txt file. So I followed the guide:

1: First I create a robots.txt file in the root directory of my domain name;

2: I need to let my robots.txt file grants access to Google crawler to access my web pages:

add the following 2 lines of code into my robots.txt file top:

User-agent: Mediapartners-Google
Disallow:

  If you would like to grant our crawler access to your pages, you can do so without granting permission to any other bots. Simply add the following two lines of text to the top of your robots.txt file:

User-agent: Mediapartners-Google
Disallow:

This change will allow our crawler to index the content of your site and provide you with the most relevant Google ads for that content.

That is!

Now you have to wait, your setting can not work immediately, and also you can not the Crawl report immediately, you maybe have to wait until the next week because Crawl reports are updated weekly.