Get Paid to Participate - up to $1 per post!     Twitter     Facebook
Hosting Discussion
 

Hosting Discussion > Operating a Web Hosting Business > Promotion and Marketing > How to Resolve 30,000 Crawl Errors in Webmasters?
forgot password?



FORUM SUPPORTERS:

Reply


Old
  Post #1 (permalink)   05-07-2018, 05:23 AM
HD Newbie
 
Join Date: Apr 2018
Location: India
Posts: 19

Status: arihantwebtech is offline
Yesterday I was checking my webmaster and I found 30,000 crawl errors in my webmaster account. Can anyone tell me to solve this issue with any short way. Because manually it been very hard for me.
 
 
 


Old
  Post #2 (permalink)   05-14-2018, 08:47 AM
HD Amateur
 
Join Date: Apr 2018
Posts: 73

Status: Chimpanzee is offline
You can log in the web master tools, in the page of the errors hit mark as solved
There is a page ,it provides free site maps along with page analize, it traves the error in your code and shows it to you.
The if you have a constant mistake you can copy the whole code in word and do search and replace. Hope it helps
 
 
 


Old
  Post #3 (permalink)   05-15-2018, 08:55 AM
HD Newbie
 
Join Date: Apr 2018
Location: Birmingham, AL
Posts: 32

Status: KnownHost is offline
Unfortunately you really need to go over them.

These errors can be caused by a multitude of reasons, could be old pages, bad links, bad direction permissions. Something blocking the bot etc.
All of these will however have the potential to negatively impact your rankings if they are having issues accessing legitimate pages.

It could also be as simple as an automatic sitemap builder including things it shouldnt.
__________________
KnownHostŪ your premium managed hosting provider. Contact us today so that we can handle your hosting needs. Offering Cloud based Shared and Reseller hosting services, OpenVZ and KVM VPS product lines and a full array of Managed Dedicated Servers! Contact us today. https://www.knownhost.com
 
 
 
The Following User Says Thank You to KnownHost For This Useful Post:
20i Richard (05-16-2018)


Old
  Post #4 (permalink)   05-16-2018, 08:45 AM
HD Addict
 
Join Date: Aug 2015
Location: Omaha, Nebraska
Posts: 204

Status: bigredseo is offline
30,000 errors sounds like either you changed the structure in your site, you removed a bunch of pages, or something crazy happened.

It's not uncommon to have a blip of 10-20 errors show up where Google says it can't access a website, but then re-check and it has no problem.

Check the date on the reported issue - is it all from the same date, or two? If so, likely something in the site changed. It should give a reason as to why you're getting the error also.

If it was a page structure change, then you could possibly create a wildcard redirect using regex, but you'll want to research that before implementing anything.

I certainly wouldn't remove them until I know why they're flagged. That heads-up by Google could save your website from losing traffic - so pay attention and get it resolved.
__________________
Conor Treacy
Big Red SEO - Omaha, NE
ConorTreacy.com <- That's Me! ;)
 
 
The Following User Says Thank You to bigredseo For This Useful Post:
Artashes (05-22-2018)


Old
  Post #5 (permalink)   05-22-2018, 12:35 AM
HD Newbie
 
Join Date: Apr 2018
Location: India
Posts: 19

Status: arihantwebtech is offline
Quote:
Originally Posted by bigredseo View Post
30,000 errors sounds like either you changed the structure in your site, you removed a bunch of pages, or something crazy happened.

It's not uncommon to have a blip of 10-20 errors show up where Google says it can't access a website, but then re-check and it has no problem.

Check the date on the reported issue - is it all from the same date, or two? If so, likely something in the site changed. It should give a reason as to why you're getting the error also.

If it was a page structure change, then you could possibly create a wildcard redirect using regex, but you'll want to research that before implementing anything.

I certainly wouldn't remove them until I know why they're flagged. That heads-up by Google could save your website from losing traffic - so pay attention and get it resolved.
As like same happening with me, just I have no loss about ranking yet. But I want to resolve this for my knowledge and SEO understanding purpose. Thanks for Reply!!!

Quote:
Originally Posted by KnownHost View Post
Unfortunately you really need to go over them.

These errors can be caused by a multitude of reasons, could be old pages, bad links, bad direction permissions. Something blocking the bot etc.
All of these will however have the potential to negatively impact your rankings if they are having issues accessing legitimate pages.

It could also be as simple as an automatic sitemap builder including things it shouldnt.
One of my website have 500 above 404 url's and I am trying to them resolve manually. But difficulty is coming over with me.... When I reduced the number from 500 to 400. After two days they again increases and again become 460 from 400. I was done 301 redirections on relevant pages. So, you have any suitable suggestion which I can solve them properly.
 
 
 


Old
  Post #6 (permalink)   05-22-2018, 01:07 PM
HD Addict
 
Join Date: Aug 2015
Location: Omaha, Nebraska
Posts: 204

Status: bigredseo is offline
Are the 404's VALID? Meaning, did a page used to exist that is now deleted? If not, then you could be fielding 404 errors for a robot that's just guessing at words on a site.

Check your backlink profile and see if the places that are linking to that 404 error maybe need a link update, or if they're even legit.

Did you make a massive website structure change? Removing everything from a folder into another?
__________________
Conor Treacy
Big Red SEO - Omaha, NE
ConorTreacy.com <- That's Me! ;)
 
 
 


Old
  Post #7 (permalink)   05-23-2018, 10:24 AM
HD Newbie
 
Join Date: Mar 2018
Posts: 32

Status: AlphaNine_vini is offline
If those urls dont exist any more . Select all errors and mark as resolved.
 
 
 


Old
  Post #8 (permalink)   05-24-2018, 08:57 AM
HD Addict
 
Join Date: Aug 2015
Location: Omaha, Nebraska
Posts: 204

Status: bigredseo is offline
Quote:
Originally Posted by AlphaNine_vini View Post
If those urls dont exist any more . Select all errors and mark as resolved.
NOT a good SEO strategy.

If you've built traffic and potential links, you need to forward those to relevant pages. If nothing relates to the article you removed, then sure, 404 it, or dump the user to a search page - but if there's even a hint of related articles on the new site, you need to 301 redirect the urls and keep that time/money spent marketing working for you.
__________________
Conor Treacy
Big Red SEO - Omaha, NE
ConorTreacy.com <- That's Me! ;)
 
 
 


Old
  Post #9 (permalink)   06-02-2018, 06:16 PM
HD Newbie
 
Join Date: Apr 2018
Posts: 6

Status: AlphaDogHosting is offline
You really have to check what the errors where (404 etc) and then find out why they occurred before you just mark them as resolved. 30,000 is quite high. Also they don't make it very easy to just mark all 30,000 as resolved. Last i checked i think the most rows you could show and clear was 500.
__________________
AlphaDog Hosting
www.alphadoghosting.com
Reliable And Affordable Web Hosting
 
 
 


Old
  Post #10 (permalink)   06-14-2018, 12:31 AM
HD Newbie
 
Join Date: Apr 2018
Location: India
Posts: 19

Status: arihantwebtech is offline
But which code??? Please send me any link where from I can get that code.
 
 
 


Old
  Post #11 (permalink)   06-14-2018, 12:39 AM
HD Newbie
 
Join Date: Apr 2018
Location: India
Posts: 19

Status: arihantwebtech is offline
Quote:
Originally Posted by bigredseo View Post
NOT a good SEO strategy.

If you've built traffic and potential links, you need to forward those to relevant pages. If nothing relates to the article you removed, then sure, 404 it, or dump the user to a search page - but if there's even a hint of related articles on the new site, you need to 301 redirect the urls and keep that time/money spent marketing working for you.
But its is very difficult to do redirection for all url's. Because 301 redirection can be possible by manually. Which very time taken process.
Any automated solution are there??? If yes please let me know>
 
 
 


Old
  Post #12 (permalink)   06-14-2018, 12:16 PM
HD Addict
 
Join Date: Aug 2015
Location: Omaha, Nebraska
Posts: 204

Status: bigredseo is offline
Yes, you can automate it using RegEx, but again, I go all the way back to my previous question;
Did you change the structure of the site, did the pages exist in the past? If so, then you can POSSIBLY use RegEx to redirect entire URL stuctures.

If the pages didn't exist before, then it could just be a bot knocking on the door, but you'll need to see where on the web those links might have existed etc.

One thing I can tell you about SEO - if it's easy, you're not doing it right! Take the time to invest into resolve the issues correctly, and Google, Bing, and users, will reward you with results.
__________________
Conor Treacy
Big Red SEO - Omaha, NE
ConorTreacy.com <- That's Me! ;)
 
 
 
Reply

Thread Tools

New Post New Post   Old Post Old Post
Posting Rules:
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Sponsored By: