pancakes

MicrostockGroup Sponsors


Author Topic: Robots.txt and url server errors  (Read 2085 times)

0 Members and 1 Guest are viewing this topic.

« on: September 02, 2013, 21:19 »
0
Last night i got an email from google saying it couldnt crawl my site because it was missing the robots.txt file. I used fetch as google, and it found it just fine, so i sent it to index. Tonight i have gotten an email saying i have a large number of urls that cant be accessed. 238 to be exact. Any idea what might be causing this? Or how i might fix it?


« Reply #1 on: September 03, 2013, 01:15 »
0
I would be interested to know the answer to this as well as I has similar emails but when I tried to access the folders they were usually there

« Reply #2 on: September 03, 2013, 06:32 »
0
Since it happened the day we were troubleshooting, my best guess would be that the site was down at the time when the googlebot arrived. Then, because of the unusual activity, bluehost throttled my account. Which may have caused pages to load really slow and the bot gave up. These might not have happened all at the same instance but over the 1-2 days. But id like to hear ideas from someone with more knowledge about it than i.

« Reply #3 on: September 03, 2013, 14:34 »
0
Anyone?

« Reply #4 on: September 03, 2013, 15:21 »
0
Didn't you answer the question already?
Crawling errors usually happens when a site has a down time or when links are broken.

« Reply #5 on: September 03, 2013, 15:46 »
0
Didn't you answer the question already?
Crawling errors usually happens when a site has a down time or when links are broken.


Thanks for confirming. I answered with my guess, which isnt the same as knowing.  :)


 

Related Topics

  Subject / Started by Replies Last post
11 Replies
5103 Views
Last post February 22, 2007, 18:05
by epixx
SX Errors

Started by dbvirago StockXpert.com

18 Replies
5049 Views
Last post October 19, 2007, 09:58
by Pixart
3 Replies
4481 Views
Last post May 18, 2008, 15:33
by jorgeinthewater
6 Replies
4418 Views
Last post August 14, 2009, 14:21
by hayaship
2 Replies
3345 Views
Last post September 07, 2013, 06:34
by cathyslife

Sponsors

Mega Bundle of 5,900+ Professional Lightroom Presets

Microstock Poll Results

Sponsors