Graphic Bar
D3xt3r.com
Panel
Graphic Bar
veltx veltix d3xt3r d3x dex dexter BK2 Auto Sales

Thursday December 27th 2012


Not only did the Baidu Bot and BingBot crawl a restricted URL listed in my robots file. But they both acquired the URL address from the same instruction to not index it. It was a "bad bot" trap, and it worked. But surprising from 2 of the internet's top search engines.







Comments


Name

Website

Email (Not public.)
Comment



d3xt3r d3x d3xt3r.net www.d3xt3r.net
Online | 1,538,664 Hits | 9 Users Online | Load 0.028 Secs
Website Design & Implementation By D3xt3r
Powered By X.D.D | W3C Validate: HTML/CSS
veltx veltix d3xt3r d3x dex dexter