Pandia Search World

Here are the latest news headlines and articles from Pandia Search World. Go to www.pandia.com/searchworld for the online version of this news service.

The robots.txt file and search engine optimization

On how to tell the search engine spiders and crawlers which directories and files to include, and which to avoid.

Search engines find your web pages and files by sending out robots (also called bots, spiders or crawlers) that follow the links found on your site, read the pages they find and store the content in the search engine databases.

Dan Crow of Google puts it this way: “Usually when the Googlebot finds a page, it reads all the links on that page and then fetches those pages and indexes them. This is the basic process by which Googlebot “crawls” the web.”

But you may have directories and files you would prefer the search engine robots not to index. You may, for instance, have different versions of the same text, and you would like to tell the search engines which is the authoritative one (see: How to avoid duplicate content in search engine promotion).

How do you stop the robots?

the robots.txt file

If you are serious about search engine optimization you should make use of the Robots Exclusion Standard adding a robots.txt file to the root of you domain.

By using the robots.txt file you can tell the search engines what directories and files they should spider and include in their search results, and what directories and files to avoid.

This file must be uploaded to the root accessible directory of your site, not to a sub directory. Hence Pandia’s robots.txt file is found at http://www.pandia.com/robots.txt.

more…

Live Image Search lets you search for faces

Microsoft follows in the footsteps of Google and adds face search to its image search engine.

The Live Search weblog reports that Microsoft’s search image search engine now lets you narrow down image search results to faces, portraits and black and white images.

By including portraits and images Microsoft is going one step further than Google’s undocumented face search feature.

To turn face filtering on in Live Search use the following syntax: jimi hendrix filter:face

For Portrait filtering use the following type of search query: jimi hendrix filter:portrait

Black and white images can be identified by using the following syntax: jimi hendrix filter:bw

more…

The problem of search privacy — and some solutions

Search privacy is a hot issue these days. A recent poll from the people behind the search engine Hakia reveals that 62 percent do not trust their search engine with their information.

The poll has not queried a representative selection of Web searchers. But the opinion of 295 Web searchers who are readers of top-technology Web blogs seems to us to be valuable in regard to a subject like this.

The respondents also disclosed the most important issues search engines can address to win users’ trust. The top two responses were “users did not want search engines to stores any data at all:” (30%); and “users wanted editing permission over the data search engines keep:” (19%).

Solutions to the search privacy problem

Pandia reported in June about metasearch engine Ixquick’s response to the debate about search privacy. Robert E.G. Beens, CEO of Ixquick, promises to delete IP addresses within 48 hours. In comparison Google will not make the data anonymous until after 18 to 24 months.

more…

Search Engine Land launches online community

Welcome Sphinn, Danny Sullivan’s new digg-like search engine discussion site.

Regular readers of Pandia know the story: Search engine guru Danny Sullivan decided to leave Search Engine Watch, the site he founded and edited for many years in order to start a new one: Search Engine Land.

Now he is taking this process one step further, announcing a new discussion forum, a sister site to Search Engine Land.

At its core the new Sphinn site will be a digg-like forum where users can suggest, vote on and discuss search engine relevant articles (cp. John Battelle’s SearchMob).

Readers will also be able to start their own discussions, which in practice makes this new forum a competitor to popular meeting places like Webmaster World and Search Engine Watch forums.

more…

Let Yahoo! write the search query for you

Yahoo! has introduced automatic search query suggestions on its home page.

Quite a few search tools have tried to guess what you are searching for while you are writing. Write “search” and the search tool will give you “people search”, “search engine marketing” or whatever the tool thinks is the most relevant or popular search phrase right now. Hit enter and you are done.

We have found such “help” to be very confusing at best, as we get distracted by the many irrelevant suggestions given by the tools.

We do see the point, though. If the search tool manages to encourage you to use more a more targeted search query, the results will be more relevant. More relevant results mean a happy searchers, and happy searchers are more likely to come back for more.

Yahoo! has now introduced search a feature on the yahoo.com home page. And this is a version of the technology we actually like.

As Kevin Lee of Yahoo! Search says over at the Yahoo! Search Blog:

The nice thing about the suggestions is that they’re not obtrusive, so if you already know what you’re searching for, you can overlook the suggestions without them getting in your way.

The suggestions appear in a drop down menu under the search field, and you can safely ignore it if you don’t like it. You can also disable the service by clicking on the relevant link in the menu.

more…

Google: one million servers and counting

Gartner reckons that Google now make use of more than 1 million servers, spitting out search results, images, videos, emails and ads.

Google’s success is the end result of a complex set of innovation processes. There is the science driven pagerank algorithm, of course., which so far has guaranteed pretty good search results — attracting an obscene number of searchers.

The Adsense text ad program, which is clearly an example of market driven innovation, gives Google a solid revenue stream.

The third cause for success

However, there is another essential factor that is not mentioned equally often: the Google server park. How many computers do you need to handle all that search traffic? And now Google is much more than search.

We are talking about video delivery, email, image and document storage here, and lots of it!

1 million servers, 3 million computers

Peter Hidas of the Gartner Group has an interesting article in the latest issue of Norwegian Computerworld. He refers to Gartner Invest’s attempts at calculating the number of Google computers, and his argument goes like this:

Google reports that it spends some 200 to 250 million US dollars a year on IT equipment. We know that Google make use of a large number of cheap off the shelf servers using open source (and free) LINUX.

If we say that Google spends 900 USD on each machine, and the same sum on storage and peripherals, it is a fair guess that Google uses some 1 million servers in its data centers. Hidas adds:

more…

Pandia Weekend Wrap-up (Week 30/2007)

On US presidential candidates flocking to Google and much more.

Pandia editor in the Koh Samui jungleThe Pandia team is back. We had a wonderful holiday in Thailand, spending one hectic week in Bangkok and two more peaceful ones in Koh Samui. And yes, that is Susanne to the left.

But now we are back in Norway, preparing a new search engine season. Stay prepared for more useful articles on how to cope in the world of web searching.

We will also continue our surveillance of the search engine scene.

We have spent some hours catching up with this week’s news and here are some of the headlines that caught our eyes:

Top 10 Web Analytics Blogs
The most popular web analytics blog at the moment (Occam's Razor by Avinash Kaushik July 27 2007 via SE Guide)

Robots Exclusion Protocol: now with even more flexibility
Dan Crow of Google announces that Google now supports a unavailable_after web page expiration metatag (Google Blog July 27 2007)

Three new features in Live Search Images
Three new features that help you find faces, portraits and black and white images (Live Search Blog July 27 2007)

Microsoft Aiming For A Bigger Share Of Online Advertising
Unofficial SEO Blog July 28 2007.

Google Web History and Search Personalization
Google provides a history of the web sites that you visit (Google Tutor July 23 2007)

Internet leader Google prime stop for presidential hopefuls
Internet giant Google is becoming a prime stop on the 2008 presidential campaign as candidates seek to burnish themselves with net-wise credentials (Physorg July 29 2007)

Google Drops Supplemental Results Query Command
Search Engine Roundtable (July 27 2007)

Digg fires Google for online ads
Digg, the reader-powered news site, fires Google as its online advertising partner in favor of Microsoft (Anchorage July 25 2007)

Google, Yahoo, Microsoft Hosting Extremist Sites
David Utter on western companies hosting Islamist/Jihadi websites (webpronews July 23 2007)

Implementing your Search Strategy for the Latino Market
Search engine marketing for the US Hispanic and Latin American search markets (SE Watch July 24 2007)

Search Engine Optimization Tricks for WordPress
Google's Matt Cutts gives some advice to bloggers (Bruce Clay July 24 2007)

Google 'the most improved brand'
Google is the brand that has gained the most in value over the past year, according a survey of global brands. (BBC July 27 2007)

Google Plans YouTube Antipiracy Tool for September
Google is working "very intensely" on a video recognition technology (PC World July 27 2007)

Who’s Reading the Google Blogs?
The Official Google Blog has over 440.000 readers. (ResearchBuzz July 27 2007)

The Internet Has Crashed
Some weekend humor from Maniac World.

more…

See also Weekend Wrap-ups for week 28 and week 29.


Pandia logo
Pandia Search Central
Search Engine News
Search Engine Detective
SEM Tutorial
Search Tutorial
Resources
Search Tools

RSS web feed

SPONSORS

Page Traffic
Ready To Get New Customers?

YOUR AD HERE!

Adapt Self-Service Search Engine Marketing
Flat Fee PPC Software - Just $499/month

web ceo
Download Free SEO Software

Outsource SEO/PPC - $11 - $15 per hour
Outsource SEO/PPC
$11 - $15 per hour



In depth search engine coverage

Pandia will bring you the latest search engine news updates several times a week.

However, if you want to get an in depth perspective on search tools and search engine marketing, you may also subscribe to Planet Ocean's monthly Search Engine News newsletter.

The Planet Ocean newsletter is part of a package that includes a regularly updated ebook on optimization and the search engine scene.


Search Engine College

Learn Search Engine Marketing Online
Search Engine College is an online training institution offering instructor-led short online courses, online classes and downloadable self-study courses in Search Engine Marketing.
Get some real help now!

Successful Search Engine Copywriting
Is it possible to write text that pleases the readers as much as the search engines? Heather Lloyd-Martin's new book tells you how.
The book shows you how to make sites that rank high and convert visitors to customers.
Whether you are a copywriter, SEO project manager, web designer, PR professional or even a site owner, this book is for you.
Click here to download! Successful Search Engine Copywriting: How to Write Prime Positioned Webpages That Convert Like Crazy


Pandia Search World is delivered by the Pandia Search Central - your online guide to search engines and Internet searching. Go to www.pandia.com/post/ to be added to the list. You may also send an email to subscribe-956602677@ezinedirector.net to subscribe to the weekly Pandia Search World newsletter. Send a mail to subscribe-956602676@ezinedirector.net to get the bimonthly Pandia Post newsletter. Contact the list manager at info@pandia.com . Pandia Search World Archive.