WebsiteDays.com create your web site free, Hosting, Coding, E-commerce, Free Website Builder. Download free website builder to create your website! Personal, and ECommerce website builders...

WebsiteDays.com for spain coslada

Click on your region below to contact a Websitedays.com designer, or Websitedays.com sales person for info about Online Website builder system.

You can also send an email to

and we can forward your enquiry to respective designers in your area.

Adeje Aguilas Albacete
Albox Alcala de Henares Alcorcon
Alcoy Alcudia Algeciras
Alhaurín el Grande Alicante Almería
Andratx Arboleas Archena
Archidona Arenys de Mar Arenys de Munt
Argentona Arnedo Arona
Arriondas Astigarraga Avila
Aviles Ayamonte Badajoz
Badalona Banyalbufar Barbera del Valles
Barcelona Bejar Benahavis
Benalmadena Benalup Benidorm
Berga Bergara Bilbao
Bossost Brena Alta Bunyola
Burgos Cadiz Cala Sant Vicenc
Calan Porter Calpe Calvia
Campillos Cangas Cangas de Onis
Cantoria Cardedeu Cartagena
Castelldefels Castello d Empuries Castello de la Plana
Cerdanyola Cintruenigo Ciutadella de Menorca
Competa Cordoba Coria
Coslada Cuenca Daganzo
Denia Durcal El Campello
El Puerto de Santa María Elche Elvissa
Empuriabrava Es Cana Esplugas del Llobregat
Estepona Ferrol Figueres
Frigiliana Fuengirola Gandia
Gaucín Getafe Getaria
Gijon Girona Granada
Granollers Grazalema Guarena
Haro Hita Huelva
Huesca Ibiza Igualada
Isla Cristina Jaca Jalon
Javea Jerez de la Frontera L Estartit
La Coruna La Eliana La Escala
La Herradura La Laguna La Rambla
Las Matas Las Regueras Leon
Lerida Llanca Llanes
Llavorsí Lloret de Mar Logrono
Lorca Los Cristianos Lugo
Madrid Magalluf Mahon
Malaga Mallorca Manacor
Manresa Marbella Masquefa
Mataro Mazarron Medina de Rioseco
Menorca Miami Playa Mijas
Mojacar Mondragon Montefrio
Montilla Morella Murcia
Nerja Onda Onteniente
Orihuela Ourense Oviedo
Oyon Padron Palamos
Palma de Mallorca Pamplona Panes
Parets del Vallès Pasajes Paterna
Perelada Periana Platja d Aro
Pontevedra Porto Cristo Pozuelo de Alarcon
Puente Genil Puerto de la Cruz Puerto del Carmen
Puig-reig Quintanilla de Onesimo Requena
Reus Ribadesella Riudellots de La Selva
Riveira Ronda Roquetas de Mar
Rota S Agaro Sabadell
Salamanca Salou San Antonio Abad
San Millan de la Cogolla San Roque San Sebastian Donostia
Sanlucar de Barrameda Sant Boi de Llobregat Sant Cugat del Valles
Sant Fruitos de Bages Santa Cruz Santa Eulalia del Río
Santa Ponsa Santander Santiago de Compostela
Santillana del Mar Sedella Segovia
Seva Sevilla Sitges
Soller Somorrostro Tacoronte
Tarifa Tarragona Tarrega
Terrassa Teruel Toledo
Torre del Mar Torremolinos Torrent
Torrevieja Torrox Trujillo
Tudela Valencia Valladolid
Valldemossa Vejer de la Frontera Vera
Vic Vigo Vilafranca del Penedes
Vitoria Yecla Zamudio
Zaragoza

»   The Proper Way To Use The robot.txt File

Article submitted in category: Web Design And Development
Tags: | disallow | file | user-agent | index | robottxt file | robottxt | spiders | * disallow | user-ag

The Proper Way To Use The robot.txt File by: Jimmy Whisenhunt

When optimizing your web site most webmasters don’t consider using the robot.txt file. This is a very important file for your site. It let the spiders and crawlers know what they can and can not index. This is helpful in keeping them out of folders that you do not want index like the admin or stats folder.

Here is a list of variables that you can include in a robot.txt file and there meaning:

User-agent: In this field you can specify a specific robot to describe access policy for or a “*†for all robots more explained in example.

Disallow: In the field you specify the files and folders not to include in the crawl.

The # is to represent comments

Here are some examples of a robot.txt file

User-agent: *

Disallow:

The above would let all spiders index all content.

Here another

User-agent: *

Disallow: /cgi-bin/

The above would block all spiders from indexing the cgi-bin directory.

User-agent: googlebot

Disallow:

User-agent: *

Disallow: /admin.php

Disallow: /cgi-bin/

Disallow: /admin/

Disallow: /stats/

In the above example googlebot can index everything while all other spiders can not index admin.php, cgi-bin, admin, and stats directory. Notice that you can block single files like admin.php.

About The Author

Jimmy Whisenhunt is the webmaster at VIP Enterprises http://www.vipenterprises.org

vipenter@vipenterprises.org

This article was posted on February 06, 2005