The phrase is most important in the INDEXING STEP of a crawler based search engine. A crawler based search engine is so called because it searches for information on the internet by crawling around like a spider looking for the relevant information based on the search words input. The search engine used the input search words or phrases to retrieve relevant information on the basis of the index that is built into the structure.
