Data and methodology
SeoData combines data from licensed data partners with our own crawler, which respects robots.txt. This page explains what each source covers, how often the data changes, what our estimates mean and where they stop being precise.
Two sources, one dataset
Licensed data partners provide search result data, keyword metrics and a large share of the link graph. Our own crawler visits pages directly to confirm links, read on-page signals and run site audits for your projects.
We do not name individual partners. What matters for your work is how fresh the data is and how it is checked, and both are described below.
- Search results and keyword metrics come from licensed data partners and are checked for consistency before they enter the product
- Links are discovered by partners and by our crawler, and our crawler rechecks known links to mark them as live or lost
- Site audits are run only by our crawler, on the projects you set up
- The free domain snapshot on the home page is computed by our crawler at the moment you request it
How often the data updates
| Data | Update cadence |
|---|---|
| Tracked keyword positions | Daily, for every keyword in your projects |
| Link and domain metrics | Refreshed continuously as pages are crawled and rechecked |
| New and lost backlinks | Added as they are found, with first seen and last seen dates |
| Keyword metrics | Refreshed regularly, with the date of the last update shown next to each metric |
| Site audits | On your schedule, daily, weekly or monthly, or on demand |
| Traffic estimates | Recalculated when the underlying positions change |
Every table in the app shows the date the data was last refreshed, so you always know how current it is.
What our estimates mean
Some metrics are measured, and some are estimated. We label them clearly.
Measured values include positions of tracked keywords, HTTP status codes, redirects, response times, links found on a page and audit issues. These are what our crawler or our partners saw at a given moment.
Estimated values include organic traffic, keyword volume, keyword difficulty and domain strength. Organic traffic is estimated from the positions a domain holds, the volume of those keywords and typical click rates by position. Difficulty combines the strength of the pages that rank for a keyword. Domain strength summarizes the quality and number of referring domains. The method behind domain strength is described on the domain authority checker page, and difficulty on the keyword difficulty checker page.
Where estimates stop being precise
Estimates are useful for comparing domains and spotting trends. They are not a copy of a site's analytics.
For a full picture of your own site, compare our estimates with your analytics. Read more in our guide on how to estimate website traffic.
- Traffic estimates cover organic search only, not direct, paid, social or email visits
- Brand searches, long-tail terms and personalized results can make estimates lower or higher than real traffic
- Very new or very small sites may show little data until enough positions are recorded
- Link counts depend on what has been crawled, so a link may exist before we find it
How our crawler behaves
The SeoData crawler identifies itself in the user agent string, follows robots.txt rules and keeps its request rate low so it does not load your server. It does not fill in forms, sign in or crawl pages behind a login. Site audits crawl only the domains you add to your own projects.
How to block the SeoData crawler
Site owners who do not want their pages crawled by SeoData can block the crawler in robots.txt. Add a group for the SeoData crawler user agent with a Disallow rule for the paths you want to exclude, or for the whole site. The crawler reads robots.txt before each visit and follows the change from the next crawl. Blocking our crawler does not remove data that licensed data partners collected independently. For questions about crawling, write to [email protected].
Questions about the data
Answers to common questions are in the FAQ. To see the data on your own domain, run a snapshot from the home page or compare SeoData pricing.