Jump to content
Sign in to follow this  
online mcafeeactivate

Is it necessary to use Robots.txt file?

Recommended Posts

Hello friends- Is it necessary to use Robots.txt file? It can be affect on website ranking in search engine.

Share this post


Link to post
Share on other sites

A robots.txt file gives instructions to web robots about the pages the website owner doesn’t wish to be ‘crawled’. For instance, if you didn’t want your images to be listed by Google and other search engines, you’d block them using your robots.txt file.

Share this post


Link to post
Share on other sites

Yes robots.txt is an integral part of your website. It is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.


Cheap VPS | $1 VPS Hosting

Cheap Dedicated Servers | Server starting at $29/MO

Share this post


Link to post
Share on other sites

Yes, absolutely. Because Robosts.txt generator tells the search engines which pages to access and index on your website and which pages not to. But it still depends on the type of website.

Share this post


Link to post
Share on other sites

Is it necessary to use Robots.txt file Because Robosts.txt generator tells the search engines which pages to access and index on your website and which pages not to. But it still depends on the type of website.

Share this post


Link to post
Share on other sites

Yes, your site will work fine. However you will not be able to block search engines from indexing your whole accessible site. This may include login pages, direct URLs to uploaded images, and other undesirable search results.

Share this post


Link to post
Share on other sites
On 2/19/2018 at 3:23 PM, online mcafeeactivate said:

Hello friends- Is it necessary to use Robots.txt file? It can be affect on website ranking in search engine.

No,It is not necessary to use robots.txt file. Suppose if you want to block perticular URL then you will have to use robots.txt file otherwise they can consider by default allows.There is no affect on website ranking..it will depends on you wheather its use or not.. 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×