Blog

This is where you can see what I have been doing recently...

Site Search

21 April 2006

Some of you may have noticed a few changes in the site search function on the home page.

Firstly, I've got rid of the option to search the whole internet (as opposed to just my site) with Google. Basically, I'm not Google, so there's no point in people to come all the way to my site when they could just as easily go to http://www.google.com

Secondly, I've got rid of Google.

Why? Well, I can't let people search my site through them if they do not appear to have indexed any of my pages. All that comes up is a message telling me that my search did not match any documents, which is completely useless. In fact, it's as bad as a broken link.

On the other hand, MSN search has indexed a lot of my pages and even has me ranked quite highly for a few search terms such as ansii series and spirograph spreadsheet. They also allow me to customize what is searched more by allowing me to tell them not just to search my site, but also the forum, and I can even tell them to give more preference to pages from my site.

Unfortunately, there are a few side effects to this update. Firstly, MSN do not provide the capability to change colours and logos etc on the search results page. However, the colours that I did chosen for the Google search results page did not really look very good and to be honest, the results would have looked much better with the default settings. So perhaps it is better not to be able to change much in the way of colours.

Another problem is that once you have made the search, your search terms appear in the box at the top of the search results page, along with with all of the extra information that I included that told MSN where to look for the pages. If you were to use this box yourself to refine your search, you may find it a bit confusing, or, worse still, you may accidentally remove the details about my site site and end up searching the entire internet, instead of just my site.

Google

You may wonder why I even had the Google search box there in the first place, if they did not index my site. Basically, what happened was that when I was first building my site, I had quite a bit of content contained in the downloads section but the homepage was blank apart from the standard header seen all over the website and a 'welcome to my site' message. This didn't exactly make a good first impression.

I therefore added most of what you can see today, such as 'New Features' and 'Latest News' from the BBC's RSS feed. Additionally, I added a Google search box. It did not include the site-search option at that time since Google had not yet discovered my website.

Over time, Google managed to index a few of my pages, starting with the ones linked to from the home page, and working through to a few of the other pages. At this point, I added the option to search my website. It worked quite well.

However, after a while, a strange thing happened. Several of my pages began to drop off their index until there were only about three of my pages listed. Then, these dropped off, until only the homepage was indexed. At this point, I complained to Google, using the 'Dissatisfied? Help us improve' link at the bottom of their page. Unfortunately, the trend continued and even my homepage dropped off their search results.

I assumed that it was just a temporary problem. After all, they are constantly changing their algorithms and small sites like this one are constantly being shuffled around the search results. However, the problem did not go away.

Looking at the server logs for my website, I noticed frequent visits from MSNBot and a number of visitors who were referred to my site by MSN Search. After looking at MSN Search myself, I noticed that my site was indeed indexed quite well by them. I found out how to add a search box to my site and I configured it to search my site and the forum, but giving preference to my site. I also removed the white background, put it in the centre of the page and made it longer, so that hopefully, people will use it to help them find the things on my site.

Permanent link

Tools

•  Feed for this blog
• View all posts