Search-Engine Optimization

Since Google has the most published information regarding their search-engine, it has been possible to study and understand how a general search engine would work like. It is also importance to note that other search-engines might not work exactly alike but it is very likely to work in a very similar manner.

How Google Works

In order for the Google Search-engine to have record of the hundreds of millions of websites and web pages on the WWW, Google has a fast distributed crawling system. Around 600K of data is an estimate of what such crawlers are able to collect per second. Each page on the WWW which is crawled by Google, is than indexed and its data (HTML, URL, etc.) is stored in one of their repositories. The strategy Google uses to find new pages which have not been indexed yet, is to scam through the links from the pages being crawled and URLs are followed. When the links are being crawled, new or updated pages are indexed. Of course, if the crawler tries to follow a broken link, it will negatively indexed.
Apart from having a ‘list’ of all pages on the WWW, Google also needs a way to identify which pages are mostly relevant to the users query. As S. Brin and L. Page explained, “First, it makes use of the link structure of the Web to calculate a quality ranking for each web page. This is called PageRank”. In other words, it focuses its relevancy mostly on page value which is calculated from the links going out from a page and mostly links pointing towards the page (backlinks). The higher value each page links to page A, and the more links page A has, the higher its PageRank will be. PageRank is determined from a number of attributes but backlinks is one of the most affecting its value. The higher the PageRank, the more chance to rank top for a user’s query on a respective topic and keywords.
Another factor Google takes into great consideration is the content of a page. The value within the content must be high in order to improve PageRank. Text syntax, metadata, links and updates, amongst others, help in determining a good value page.