Twitter     Facebook
Hosting Discussion
 

Hosting Discussion > HostingDiscussion Community > General Discussion > Google search console page indexing
forgot password?



FORUM SUPPORTERS:

Reply


Old
  Post #1 (permalink)   07-24-2020, 07:55 PM
HD Addict
 
Join Date: Aug 2017
Posts: 106

Status: PeterShene is offline
HI there anybody else notice a big decline in the rate at which google is indexing new websites and web pages? I'm struggling to get a few sites indexed and even waiting weeks for changes on old existing pages.. cant really find anything on this online except the usual Google takes its own time be patient and make sure you content is good stuff. I understand that just in the many years i've done this its at a all time high for slowness. I wonder if there was some kind of algorithm update, or the spider is taxed because of all the new pages popping from businesses looking to get online during the pandemic.
 
 
 


Old
  Post #2 (permalink)   07-25-2020, 09:45 AM
HD Wizard
 
easyhostmedia's Avatar
 
Join Date: Mar 2011
Location: Northumberland, UK
Posts: 6,118

Status: easyhostmedia is online now
Quote:
Originally Posted by PeterShene View Post
HI there anybody else notice a big decline in the rate at which google is indexing new websites and web pages? I'm struggling to get a few sites indexed and even waiting weeks for changes on old existing pages.. cant really find anything on this online except the usual Google takes its own time be patient and make sure you content is good stuff. I understand that just in the many years i've done this its at a all time high for slowness. I wonder if there was some kind of algorithm update, or the spider is taxed because of all the new pages popping from businesses looking to get online during the pandemic.
The hamster wheel broke and i guess people panic bought these, so Google cant get a replacement
__________________
Terry Robertson - CEO Niceday Media Limited
PowerSSL - - We Secure your World
The Scamlist Forum - Fighting against scammers
 
 
 


Old
  Post #3 (permalink)   07-25-2020, 08:53 PM
HD Newbie
 
Join Date: Oct 2008
Posts: 2

Status: moonswamp is offline
same problem but it gonna work when I remove the robots.txt!
Now google indexed all my pages in days!
__________________
Cheap VPS Hosting Reviews
 
 
 


Old
  Post #4 (permalink)   07-26-2020, 02:30 AM
HD Wizard
 
easyhostmedia's Avatar
 
Join Date: Mar 2011
Location: Northumberland, UK
Posts: 6,118

Status: easyhostmedia is online now
Quote:
Originally Posted by moonswamp View Post
same problem but it gonna work when I remove the robots.txt!
Now google indexed all my pages in days!
no need to remove robot.txt.
you just need to allow Googlebot in then robot.txt file
__________________
Terry Robertson - CEO Niceday Media Limited
PowerSSL - - We Secure your World
The Scamlist Forum - Fighting against scammers
 
 
 


Old
  Post #5 (permalink)   08-05-2020, 12:49 PM
HD Addict
 
Join Date: Aug 2017
Posts: 106

Status: PeterShene is offline
So an update on this after much digging around I found that google dedicates crawl index on websites via ip, i have a few websites on shared hosting which has boomed a bit recently and come to think of it the sites that aren't on a shared ip are indexing / updating from crawls much faster. Its a little annoying since in the long run shared ips cant be helped for many people, I also read its a way that google is pushing for more use of its own cloud servers <---- this
 
 
 


Old
  Post #6 (permalink)   08-05-2020, 01:05 PM
HD Wizard
 
easyhostmedia's Avatar
 
Join Date: Mar 2011
Location: Northumberland, UK
Posts: 6,118

Status: easyhostmedia is online now
Quote:
Originally Posted by PeterShene View Post
So an update on this after much digging around I found that google dedicates crawl index on websites via ip, i have a few websites on shared hosting which has boomed a bit recently and come to think of it the sites that aren't on a shared ip are indexing / updating from crawls much faster. Its a little annoying since in the long run shared ips cant be helped for many people, I also read its a way that google is pushing for more use of its own cloud servers <---- this
crawl indexing has always been done using IPs.

why do you think in .htaccess files etc. you allow or deny using IPS
__________________
Terry Robertson - CEO Niceday Media Limited
PowerSSL - - We Secure your World
The Scamlist Forum - Fighting against scammers
 
 
 


Old
  Post #7 (permalink)   08-05-2020, 02:48 PM
HD Community Advisor
 
bigredseo's Avatar
 
Join Date: Aug 2015
Location: Omaha, Nebraska
Posts: 486

Status: bigredseo is offline
For indexing issues, in the search console, click the option to re-submit a sitemap - this will at least get you in the queue.

Also, if you have specific content that you need indexed from a specific page, you can use the Inspect URL, then after processing click the option for Request Indexing (only need to click it once).

We've found that if we use the option of the Inspect URL to request indexing that this usually gives a quicker prod, and often we're seeing a crawl within 24 hours.

I didn't see this particular thread until now, but the pandemic did slow some things at Google, and they are giving more priority to health and wellness sectors - but much of that has returned to normal.

Normal indexing usually can be 7-14 days (we have seen longer in the past) but using the Inspect URL can shorten that down to 24 hours or so.
__________________
Conor Treacy
Big Red SEO - SEO, Web Design, PPC Management & Training
ConorTreacy.com <- That's Me! ;)
I say it so often, I bought the domain! - MakeABackup.com
 
 
 
The Following 2 Users Say Thank You to bigredseo For This Useful Post:
Artashes (08-08-2020), SenseiSteve (08-06-2020)


Old
  Post #8 (permalink)   08-06-2020, 04:02 AM
HD Newbie
 
Join Date: Apr 2005
Location: India
Posts: 23

Status: etechsupport is offline
I'm using sitemap to index my all the pages, and you can also metion sitemap path and enable googlebot in robots.txt that help to index all the pages easily by any search engine
__________________
Technical Support | eTechSupport.net | 24x7 Server Management & Monitoring | Outsourced Web Hosting Support | 600+ Employees Team
 
 
 


Old
  Post #9 (permalink)   08-06-2020, 12:32 PM
HD Community Advisor
 
bigredseo's Avatar
 
Join Date: Aug 2015
Location: Omaha, Nebraska
Posts: 486

Status: bigredseo is offline
rules for googlebot (or any bot) in the robots.txt should only be used to EXCLUDE sections, not include. The only exception to that is that you should have a link to your sitemap XML file in the robots.txt

Keep in mind, robots.txt is only used by honest bots. Places like Yandex or aggressive crawlers ignore any suggestions in robots.txt.
__________________
Conor Treacy
Big Red SEO - SEO, Web Design, PPC Management & Training
ConorTreacy.com <- That's Me! ;)
I say it so often, I bought the domain! - MakeABackup.com
 
 
 
The Following 2 Users Say Thank You to bigredseo For This Useful Post:
bprashanth (09-21-2020), SenseiSteve (08-06-2020)


Old
  Post #10 (permalink)   09-13-2020, 02:54 PM
HD Newbie
 
Join Date: Feb 2018
Posts: 18

Status: rf-harris is offline
Yes, I am also facing indexing issues with most of my backlinks but not with my own website.
 
 
 


Old
  Post #11 (permalink)   09-15-2020, 10:12 AM
HD Newbie
 
Join Date: Sep 2020
Posts: 17

Status: iwinx is offline
There seems to be an issue with it according to google.

https://twitter.com/googlewmc/status...33934373806080
 
 
 


Old
  Post #12 (permalink)   09-16-2020, 08:54 AM
HD Community Advisor
 
bigredseo's Avatar
 
Join Date: Aug 2015
Location: Omaha, Nebraska
Posts: 486

Status: bigredseo is offline
Quote:
Originally Posted by iwinx View Post
There seems to be an issue with it according to google.

https://twitter.com/googlewmc/status...33934373806080
There is a current issue with the Google Search Console reporting system, however this only affects reporting and not actual indexing. If you run a "site:domain.com/link/file.html" from a Google search you should see your page indexed in Google.

Keep in mind that the new issue happening in GSC is only for the past 2 weeks (so, 9/1/2020-9/16/2020). It does not appear to have affected the previous month.

From time to time GSC does have delays in reporting, usually 1-3 days, but this is the longest I've seen in many years (if not the longest ever).

Again, only affects the reporting, not the actual indexing.
__________________
Conor Treacy
Big Red SEO - SEO, Web Design, PPC Management & Training
ConorTreacy.com <- That's Me! ;)
I say it so often, I bought the domain! - MakeABackup.com
 
 
 
The Following 2 Users Say Thank You to bigredseo For This Useful Post:
bprashanth (09-21-2020), iwinx (09-16-2020)


Old
  Post #13 (permalink)   09-20-2020, 12:20 AM
HD Addict
 
Join Date: Aug 2017
Posts: 106

Status: PeterShene is offline
Hmm there are patent documents talking about crawl limits and these guys are talking about tens of thousands f pages ( i assume per ip ). I know im not close to that and the indexing has sped up so maybe it was a covid thing like red said priority being giving to other sites.

Interesting you mention different ways of submission, i have found that something like yoast indexes slower than google xml sitemaps and if you create you own sitemap its faster. Google says the priority in the sitemap is not important, not sure about that not sure if its the format as yoast it like a html sitemap with a link structure, while what i create with screaming frog is just simple straight up.
 
 
 


Old
  Post #14 (permalink)   09-21-2020, 05:22 AM
HD Newbie
 
Join Date: Apr 2011
Posts: 11

Status: bprashanth is offline
Quote:
Originally Posted by bigredseo View Post
...Also, if you have specific content that you need indexed from a specific page, you can use the Inspect URL, then after processing click the option for Request Indexing (only need to click it once).

We've found that if we use the option of the Inspect URL to request indexing that this usually gives a quicker prod, and often we're seeing a crawl within 24 hours....
I can +1 this with confidence.

We have been developing content, albeit slowly, of late. Inspect URL and Reqest Indexing button works faster than any other method. Only thing is that you need to do it separately for every URL that you wish Google to index (a real pain if you want 100s or 1000s done at the same time)
 
 
 
Reply

Thread Tools

New Post New Post   Old Post Old Post
Posting Rules:
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On