Re: OT - SE robot question

This WebDNA talk-list message is from

2003


It keeps the original formatting.
numero = 49887
interpreted = N
texte = >Does a SE or SE robot index pages in a directory that are not linked to any live pages??? > >Can't seem to find a definitive yes or no out there on the SE help sites. >ThanksGood bots will not see them. Bad bots might see them, but only if they are mentioned in the robots.txt file.Create a robots.txt file using only the ALLOW: tag. Never use the Disallow: tag to hide anything from a bot. Bad bots are drawn to those pages.Make a site map. Do not use forms or graphics in the links on the site map to the pages you want indexed. Use the exact word or phrase in the text of the link that the target page is optimized for.Glenn------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/ Associated Messages, from the most recent to the oldest:

    
  1. Re: OT - SE robot question (Glenn Busbin 2003)
  2. Re: OT - SE robot question (Lester Emo 2003)
  3. Re: OT - SE robot question (Glenn Busbin 2003)
  4. Re: OT - SE robot question (Stuart Tremain 2003)
  5. OT - SE robot question (Lester Emo 2003)
>Does a SE or SE robot index pages in a directory that are not linked to any live pages??? > >Can't seem to find a definitive yes or no out there on the SE help sites. >ThanksGood bots will not see them. Bad bots might see them, but only if they are mentioned in the robots.txt file.Create a robots.txt file using only the ALLOW: tag. Never use the Disallow: tag to hide anything from a bot. Bad bots are drawn to those pages.Make a site map. Do not use forms or graphics in the links on the site map to the pages you want indexed. Use the exact word or phrase in the text of the link that the target page is optimized for.Glenn------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/ Glenn Busbin

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

[WebDNA] Passing a text variable via copyfile (2019) $flushcache causes crash (2000) no emailfolder errors? (2000) Rhapsody? (1997) Extended [ConvertChars] (1997) sort problems....bug or brain fart? (1997) off topic fetch vs PCS photomaster (1997) thankyou.tmpl (1997) [Semi-OT] Eaaaaaasy OSX FTP (2003) WebDNA license logging (2003) [WebDNA] WebDNA programmers for hire? (2010) Replace Statement (1997) Email notification to one of multiple vendors ? (1997) Single Link browsing (1997) A helpful tip to avoid pulling an Aaron (2000) upgrading (1997) Mac: LModelDirector bug fix (1997) Separate SSL Server (1997) Autoreturn Submit? (1997) changing prices in cart (1997)