Halloween

Well no doubt the various search providers will update their search pages to reflect the Halloween theme in the near future. Yahoo! seem to be the first to have done so with a link to http://events.yahoo.com/halloween/2009/index.php already listed on the home page.

No doubt the others will follow in the coming hours. Wonder who'll be next?

Cuil, Remember Them?

Hey, finally some news on Cuil, you remember Cuil don't you? The Google killers with the largest database of indexed pages in the world. Oh... not? Don't worry, I'd forgotten about them entirely too. Especially after the recent re-branding of MSN and Live search to Bing (which I'm am quite fond of).

Well it seems that Cuil have decided that it's time to innovate. As reported by Matt McGee on Search Engine Land - it would seem that Cuil have now added Maplines to their search results. This makes for a much more interesting results page as displayed in a search for George Orwell.

While Google keep the monopoly on simple search, could it be that the other providers have decided that if you can't beat them, start a new game? Ask.com brought out their 3D search (which has faded away), Microsoft eventually consolidated everything into Bing which is a lot more interactive, even offering a blurb on sites and now Cuil change it up a little. I can't help but feel sorry for Yahoo! as they really are lagging now (dead duck or just lame I wonder). I don't see people changing their search habits anytime soon, but the internet changes pretty quickly, what will Google do to counter this?

(Google) Bombing the Presidents

Well Google may have diffused that bomb, but looks like bing.com are more than happy to keep it locked and loaded. Danny Sullivan of Search Engine Land pointed out that for some reason the "miserable failure" bomb seems to be live and well (or is that lit and fused?) in recent times.

Google run their algorithms from time to time to locate and diffuse these bombs, but what of the other engines? Yahoo! never seemed to really get rid of it entirely and MSN was pretty much on the same path. At present a search for "miserable failure" on Ask.com reveals George Bush to be the number one contender while surprisingly his right hand man is now Barack Obama (does Ask know something we don't?). Okay, so there is more to this than meets the eye.

The White House implemented 301 redirects a little while ago to send most of that failure link-love back to its intended destination (that of past President GW Bush). It seems that Google have managed to once again find and block this bomb. MSN (now combined with Live as Bing.com) seem to have re-indexed the site and are now reporting the GW Bush bio as a "miserable failure", followed by the Wikipedia report on this event and surprisingly the new (and only one that I know of) bio for B Obama. Yahoo!'s results match those of Ask.com (although they have indexed the new URL). Interestingly enough Ask.com still list the old gwbbio.html file as the URL for GW Bush despite the site having implemented 301 redirects (clicking on the link takes you to /georgewbush/).

Okay, so what does this really tell us? That GW Bush and Obama are miserable failures? Well, that will forever remain a long debate.

From an observation point I'd say that this would suggest that the miserable failures in this case are the search engines. Agreed, they may simply be returning facts based on what the public perceive, however this simply goes to prove how easily these giants can be manipulated to this day.

Google seem to have worked around this one and I have a sneaky suspicion that when Matt Cutts moved from the domain www.mattcutts.com to www.dullest.com he was testing just how well they handle the 301. Perhaps Obama and Bush both owe Matt and his team a thanks on that one. On the other side of this we can gather that most of the search engines still need to work on how they deal with 301's.

Could this leave a door open to spammers to "Google bomb" a page, then simply 301 the link love to another page?

Jeeves Returns

Well at least to Ask in the UK. Okay, so I wasn't ever the biggest fan of the butler, mostly because I thought it was a poor way to promote a search engine. From a marketing perspective, why ask Jeeves if you could simply "Ask it!" Doesn't it make more sense to "ask it" rather than "google it"? Well it would have a good few years back.

Well I guess that's why Ask never got me to do their marketing. Here in South Africa half the searching public don't even know that there ever was a Jeeves, for others Google is the internet (much like that blue icon is the internet). While still no fan of the Jeeves concept I have to admit it's a blast from the past to see an "old friend."

Will Jeeves make a real return? I don't know, but I'd doubt it right now. While he might work for the British public, I don't see him making full time return to the US, or even the rest of the world for that matter. Jeeves was retired for a good few reasons, I don't know what has changed since then and now. But then again, perhaps the public really are wanting to put a real face to search? Would newer users prefer to have a familiar face to credit their answers?

Arrrgh... It's a Treasure, uh, Sitemap

There are two types of sitemaps that you might employ on a website. First the HTML version which is intended to offer the average visitor to the website an overview of your website. The second is a machine readable XML sitemap (commonly referred to as a Google Sitemap) intended to inform a search bot of the pages found on your website. I'd always recommend the use of both - in various forms.

HTML sitemaps are a wonderful resource. If a visitor to the website can't find what they are looking for this is the easiest way to point it out. In addition many fail to realise that his is a prime spot to put text links to pages with your keywords as the anchor text. In addition why not add a little additional information next to the link. Suddenly the whole page becomes a fantastic resource for both human and bot.

XML sitemaps a great for dynamic websites. Many of the websites we have worked on include many products or other kind of items listed. Scripts can be run to dynamically generate this sitemap as listings are changed or updated. These sitemaps can be submitted directly to the search engines informing them that this is a true reflection of your site.

I've tended to break my XML sitemaps down into smaller maps. Often breaking down static pages and separating them from the dynamic pages. On our really large sites I have even broken the dynamic sitemaps into categories. Mostly you don't want to offer a sitemap of thousands of pages to the search engines at a time. A thousand pages or so may be the limit.

Submit your XML sitemaps to the search engines and be sure to link to your HTML sitemap from your homepage, this will ensure that the sitemap is easily crawled by the search engines and will subsequently lead to the other pages being easily indexed.

Too Cuil for You

Well it's been some time since the Google killer Cuil was launched and I've not heard much since. Launched to much fanfare and expectation I think this has to be the largest flop seen in years. I wonder just how much was put into this development in time and money? I wonder if any of the investors would be getting anything back?

As I've not used this search engine (mostly as I found it to be useless at launch) I can't comment too much on the accuracy of the search results but do know that the images displayed still don't quite match up. Nice try though. I think Google, Ask and even Live have better image results blended into their universal search.

Perhaps in time they will be able to make sense of "the largest directory of indexed pages". But for now the results seem to be outdated, irrelevant and at times just wrong. Was this what we expected of the ex-Googlers? Perhaps this is a prime example of why they are Ex Google folk?

SEO Design

There are many aspects to consider when putting a design together, most of which are either second nature to the seasoned developer or overlooked completely by the novice. Although as we all know, what looks good doesn’t necessarily work well and vice versa.

Neatness of Code:
Code should be neat. Simple. No, it doesn't have to conform to W3C standards (Google doesn't even conform and it is estimated that only 3% of all sites actually do - side note that half the sites that do claim to conform don't either). With so few sites conforming to these standards how can Google (or any other engine for that matter) offer decent results if they negate 97% of the internet? Keeping code clean includes keeping all generic information such as style sheets and scripts in separate files.

Robot Tags and Text File:
Often you may not wish for the search engine bots to index certain pages. You can easily add the NoIndex attribute to this page. For whole directories you can simply add them into your robots.txt file located in the root of your website (http://www.website.com/robots.txt). Why is this important for design? Well the crawl rate and indexing is a concern for all departments. Remember I mentioned that all generic information should be kept separate? Well this way you can simply block those directories with the robots.txt file. This way the search engines will be forced to index your actual content pages before attempting to read your styling code.

Content:
Do not replicate content on multiple pages, it's a waste of time, effort and dilutes keyword value. While the duplicate “penalty” is a myth it does confuse the search engine as to which page is more important or even which page was first. Imagine someone giving you the key to a Ferrari and then telling you it was the key to the red Ferrari parked outside? Now imagine there are 10 red Ferrari’s parked outside! Which one does the key fit? If there is only one Ferrari the choice is easy. Usually the page which is indexed first is the one that is credited with being unique. The other pages are simply diluting their keywords and purpose. Personally I've always tried to aim one keyword per page this does lend itself to long tail combinations working on a single page as well.

While it is commonly accepted that the major search engines ignore boilerplate content (such as standard navigation, headers and footers), it has since been suggested that you can point out which sections Google should ignore. This doesn't seem to be in mainstream use just yet and I am sure that this won't make much of a difference as it remains open to abuse - as with so many other on-page factors.

URLS:
URL’s, or URI’s, can make a difference when it comes to ranking. As mentioned before people may link to the page (home or internal) with the actual URL. As mentioned before anchor text is vital for ranking a page so it makes sense then to include keywords in your URL. Long gone are the days when URL’s were dynamic and half the URL’s had strange characters and session ID’s (a massive source of duplicated pages).

www.website.com/page?ID=123
www.website.com/location/
www.website.com/Town-Name/

In addition to duplicating pages session ID’s and multiple variables can also cause a search engine spider to become trapped in deep pages. Once trapped a spider will leave the website this may result in your more important pages not being indexed. We can now specify the URL of a page through the use of a specific tag in the page header. In this instance the search engines (Google & Ask.com) will ignore session variables (or others you may have generated) and only index the page as you specify.

Links:
The easiest way for human and bot to get from page to page is through links. Not all links were created equal. Links hidden in flash, images or scripts may look good to the human but be impossible for the search engine bot to read. Content remains king and while community (social media) has recently been crowned queen but it is the text link that remains everyone’s servant. On your own website you can use desired anchor text to describe the page you are linking to.

From another website, if a link to a website is a vote, then the anchor text tells you what they are voting for. Because so many webmasters, bloggers and publishers link to pages using the URL as the link text it becomes quite clear as to just how valuable it can be to include your desired keywords in your URL. However, no matter how hard you try you will always have broken links to your site. This could be due to a typo or because you've moved the page (or restructured the website) in which case a custom 404 page is crucial. When rebuilding a website and changing your URL structure, it is advisable to 301 (permanent redirect) the old URL to the corresponding new one.

Forms and Restricted Pages:
Don’t hide your great content behind forms and other forms of logins. Robots can’t fill these in and won’t be able to reach these pages. Simply put they won’t know that it exists. There are ways around this, but why make it difficult of the Robots or even Humans who are now becoming more and more reluctant to part with personal info on the web.

Sitemaps:
XML sitemap for robots (often simply referred to as a Google Sitemap). If you have many pages, consider breaking these down into themes. At present I prefer to set up a static XML sitemap for the pages that won't change and a dynamic XML sitemap for listings, products, etc that will change on a regular basis.
HTML or plain text sitemap for humans can be a perfect place to get all those keywords in either the link itself or next to it. This is also an easy way for a visitor to find something listed on the website. Make sure that this page is easily accessible from the homepage.

Summary
It is reported that Google has over 200 criteria points when it comes to ranking a website. Many of those aren’t part of the design. But a few that are include:

• Keep code to the minimal required


• Minimise the use of code that search engines can’t read (hide it when possible)


• Unique content - keep navigation consistent


• Use descriptive URL’s


• Keep unique URL's


• Descriptive internal linking


• Use text links to reach all of your pages


• Custom 404 page


• Don’t hide great content behind forms and login pages


• Use XML Sitemaps for the search engines


• Use a descriptive HTML sitemap

SEO in a Nutshell

SEO in a Nutshell... an optimised website aimed at pleasing the search engines to gain the best possible placing in the SERPS (Search Engine Results Pages)for relevant and often searched words and phrases.

Go on, read that again...

According to Wikipedia:
(SEO) is the process of improving the volume and quality of traffic to a web site from search engines via "natural" ("organic" or "algorithmic") search results. Typically, the higher a site's "page rank" (i.e, the earlier it comes in the search results list), the more visitors it will receive from the search engine.

Okay firstly never confuse that "page rank" with PageRank. Confused already? Don't worry, if you hear of anyone talking of PageRank (named after Larry Page Google co-founder) just ignore them. The only one who knows the true PageRank of any page (yes each page of a website gets its own PageRank) is the Googlebot. What is important is your position in the SERP's, to a degree.

As the search engines continue to try to provide the best possible (read most relevant) search results to each visitor, many searches using the same phrases may actually result in different results. This is usually due to geo-location, after all, if I were looking for my nearest fish and chip shop I would hope the search engine realises that I'm searching for a fish and chip shop in Westville, South Africa. Google have done a fantastic job with this by pretty much offering every country out there their very own version of Google. Often the searcher won't even realise that they are being routed to Google.co.za or Google.co.uk instead of Google.com. All 3 of those will return different results for the same search. It is now possible and quite probable that searches in the same country will return different results.

Enter SEO. While the goal remains to rank well and be found. The difficult part is ranking well for often searched terms that will actually lead to conversion. This means that the website needs to not only rank well but actually generate traffic that will eventually result in a sale, lead, sign up... or what ever the purpose of your website may be. With this in mind 10 converting visitors from 100 in total is better than 1 converting visitor from 1000 visitors. Quality is more important than the quantity of traffic.

Redefined: SEO is the practice of generating quality traffic to a website to increase business.