Glossary
Glossary
This section explains the terms that anyone who runs a website or reads about search will come across. The definitions are short and based on search engine documentation and open standards. Where search engines describe their systems only in general terms, we don't go into details that nobody has confirmed either.
Many arguments about search end once it turns out the two sides use the same word for different things: crawling and indexing, PageRank and a third-party domain score.
Search and indexing
- Crawler (search engine bot)
- A search engine program that visits pages on the web by following links and addresses listed in sitemaps.
- Crawling
- The process in which a crawler requests a site's pages. Being crawled does not yet mean a page will appear in search.
- Index
- The database of pages a search engine uses to answer queries. Whether a page gets indexed depends on many conditions that search engines describe only in general terms.
- robots.txt
- A file at the root of a site following the Robots Exclusion Protocol (since 1994; RFC 9309 since 2022). It controls crawling but does not guarantee a page is absent from the index.
- Sitemap
- A file listing a site's addresses according to the Sitemaps protocol (sitemaps.org, since 2006), which is supported by the major search engines.
Links
- Backlink (inbound link)
- A link from another page or another site to your page. It helps a search engine discover the page and is taken into account among many signals.
- Anchor text
- The visible, clickable text of a link. It hints at what the linked page is about.
- nofollow
- The rel="nofollow" link attribute, introduced in 2005. Since 2019 Google treats it as a hint.
- sponsored and ugc
- Attributes added by Google in 2019: sponsored for paid links, ugc for links in user-generated content.
- Link spam
- Under search engine policies, this includes buying and selling links to manipulate rankings. Paid links must be marked sponsored or nofollow.
Metrics
- PageRank
- An algorithm for ranking pages based on links, described by Page and Brin in 1998. The public 0–10 score stopped being updated at the end of 2013 and was removed in 2016.
- TIC (Yandex)
- Yandex's thematic citation index. Replaced by SQI in 2018.
- SQI (Yandex IKS)
- Yandex's site quality index, which replaced TIC in 2018. For details, see Yandex Help for webmasters.
- Third-party domain metrics
- DR, DA and similar domain "authority" scores calculated by individual services. They are not search engine data.
- Webmaster tools
- Google Search Console (Google Webmaster Tools until 2015) and Yandex Webmaster: official services where site owners see the search engine's data about their site.
Materials on these topics
- Practical guides
How to check a website before launch, step by step. Plus a catalogue of upcoming articles.
- Questions and answers
Short answers about PageRank, indexing, links and website metrics.
- Reading lists
Reading paths: your first website, the history of search metrics, links without spam.
- Archived address: former service charts (in Russian)
A note on the old charts address and what to look at instead of a PageRank score.
Related sections
- Questions and answers
Short answers about PageRank, indexing, links and website metrics.
- Reference sections
Map of the reference: PageRank and the history of metrics, indexing, links and their attributes, site-owner tools.
- Practical guides
How to check a website before launch, step by step. Plus a catalogue of upcoming articles.
Sources
- S. Brin, L. Page — "The Anatomy of a Large-Scale Hypertextual Web Search Engine" (1998)
- Google Search Central — documentation and spam policies
- Yandex Help for webmasters
- RFC 9309 (Robots Exclusion Protocol)
- sitemaps.org
Information is current as of the date of preparation. Search engine documentation, policies and webmaster tools change over time, so always check the latest details in the original sources.