Today’s definition is “Spider (Crawler / Bot)”:

spider (crawler / bot): a software application that is programmed to follow hyperlinks around the World Wide Web, collecting information about the pages it visits for later processing. The most well known spiders are those used by Search Engines to index web pages, but spiders are used for a number of purposes, including unethical ones such as the collection of email addresses for the sending of SPAM.

Read more jargon-busting definitions at our Glossary »