How to Track Google Rankings at Scale Without Building Your Own Scraper

Tracking one Google ranking is easy.

Open a browser, type a keyword, look for your page, write down the position. Done.

Tracking 5,000 keywords every day across countries, cities, languages, and devices is a very different animal. That is when a homemade scraper starts looking less like a clever shortcut and more like a tiny server room full of angry bees.

For SEO teams, rank tracking is not just about “where do we rank today?” The real value is in the trend:

Which keywords moved?
Which pages dropped?
Which competitors appeared?
Did mobile rankings change?
Did rankings differ by city or language?

To answer those questions reliably, you need repeatable SERP data. A SERP API gives you structured search results without maintaining your own browser automation, proxy routing, CAPTCHA handling, and parsing logic.

Quick answer

To track Google rankings at scale, use a SERP API to fetch Google results for each keyword, market, language, and device. Then extract organic results, match your target domain, store the ranking position, and compare daily snapshots over time.

A practical rank tracking workflow looks like this:

Keyword list
→ SERP API request
→ Extract organic results
→ Match target domain
→ Store ranking snapshot
→ Compare changes over time

Why not build your own Google scraper?

You can build a scraper. Many teams try.

The first version usually works. Then the slow drip begins:

For a one-time experiment, a scraper may be fine. For recurring rank tracking, it becomes infrastructure work.

Most SEO teams do not want to run scraping infrastructure. They want ranking data.

What data should a rank tracker collect?

A rank tracker should store more than keyword and position.

Without country, language, device, and timestamp, ranking data loses context.

A position 3 result in the United States desktop SERP is not the same as position 3 in a mobile SERP for London.

Step 1: Prepare keywords and markets

Start with a small test set.

KEYWORDS = [
    "best project management software",
    "crm software for small business",
    "email marketing tools",
]

MARKETS = [
    {
        "country": "us",
        "language": "en",
        "location": "United States",
        "device": "desktop",
    },
    {
        "country": "gb",
        "language": "en",
        "location": "London, England, United Kingdom",
        "device": "mobile",
    },
]

TARGET_DOMAIN = "example.com"

In production, keywords usually come from your SEO database, Google Search Console exports, content plan, or competitor tracking list.

Step 2: Send Google SERP API requests

Keep the request layer separate from the parsing logic.

import os
import requests


SERP_API_KEY = os.getenv("TALORDATA_API_KEY")
SERP_ENDPOINT = os.getenv("TALORDATA_SERP_ENDPOINT")


def call_serp_api(payload):
    if not SERP_API_KEY:
        raise RuntimeError("Missing TALORDATA_API_KEY")

    headers = {
        "Authorization": f"Bearer {SERP_API_KEY}",
        "Content-Type": "application/json",
    }

    response = requests.post(
        SERP_ENDPOINT,
        json=payload,
        headers=headers,
        timeout=30,
    )

    response.raise_for_status()
    return response.json()


def fetch_google_results(keyword, market):
    payload = {
        "engine": "google",
        "q": keyword,
        "gl": market["country"],
        "hl": market["language"],
        "location": market["location"],
        "device": market["device"],
        "num": 10,
        "output": "json",
    }

    return call_serp_api(payload)

A SERP API is useful here because the request already carries the search context: keyword, country, language, location, and device.

Step 3: Extract organic results

Different APIs may use slightly different field names, so keep the parser flexible.

from urllib.parse import urlparse


def clean_text(value):
    if not value:
        return ""
    return " ".join(str(value).split())


def get_domain(url):
    if not url:
        return ""

    parsed = urlparse(url)
    return parsed.netloc.replace("www.", "")


def get_organic_results(serp_json):
    return (
        serp_json.get("organic_results")
        or serp_json.get("organic")
        or serp_json.get("results")
        or []
    )


def normalize_organic_results(serp_json):
    rows = []

    for index, item in enumerate(get_organic_results(serp_json), start=1):
        url = item.get("link") or item.get("url") or ""

        rows.append({
            "position": item.get("position") or item.get("rank") or index,
            "title": clean_text(item.get("title")),
            "url": url,
            "domain": get_domain(url),
            "snippet": clean_text(item.get("snippet") or item.get("description")),
        })

    return rows

This gives you a clean list of organic results that can be matched against your target domain.

Step 4: Match your target domain

Rank tracking usually asks a simple question:

Does my domain appear in this SERP, and where?

def find_target_ranking(results, target_domain):
    target = target_domain.replace("www.", "").lower()

    for item in results:
        domain = item["domain"].lower()

        if domain == target or domain.endswith("." + target):
            return {
                "position": item["position"],
                "matched_url": item["url"],
                "title": item["title"],
                "snippet": item["snippet"],
            }

    return {
        "position": None,
        "matched_url": "",
        "title": "",
        "snippet": "",
    }

Keep None when the domain is not found. Do not force fake positions such as 99 unless your reporting system clearly defines that convention.

Step 5: Save ranking snapshots

For a first version, CSV is enough.

import csv
from datetime import datetime, timezone


CSV_COLUMNS = [
    "keyword",
    "target_domain",
    "country",
    "language",
    "location",
    "device",
    "position",
    "matched_url",
    "title",
    "snippet",
    "collected_at",
]


def export_rankings(rows, filename="google_rankings.csv"):
    with open(filename, mode="w", newline="", encoding="utf-8") as file:
        writer = csv.DictWriter(file, fieldnames=CSV_COLUMNS)
        writer.writeheader()
        writer.writerows(rows)

Then run the workflow:

def run_tracking():
    collected_at = datetime.now(timezone.utc).isoformat()
    output_rows = []

    for keyword in KEYWORDS:
        for market in MARKETS:
            serp_json = fetch_google_results(keyword, market)
            organic_results = normalize_organic_results(serp_json)
            ranking = find_target_ranking(organic_results, TARGET_DOMAIN)

            output_rows.append({
                "keyword": keyword,
                "target_domain": TARGET_DOMAIN,
                "country": market["country"],
                "language": market["language"],
                "location": market["location"],
                "device": market["device"],
                "position": ranking["position"],
                "matched_url": ranking["matched_url"],
                "title": ranking["title"],
                "snippet": ranking["snippet"],
                "collected_at": collected_at,
            })

    export_rankings(output_rows)


if __name__ == "__main__":
    run_tracking()

The output is a ranking snapshot.

Run it daily, and you have rank history.

When CSV is no longer enough

CSV is fine for testing. Once you track rankings every day, move to a database.

A basic table might look like this:

From there, you can build:

• ranking trend charts

• keyword movement alerts

• page drop reports

• competitor visibility reports

• local SEO dashboards

• weekly SEO summaries

Scaling checklist

When your tracker grows, the hard part is no longer the code snippet. It is operations.

Use this checklist:

At scale, rank tracking becomes a data pipeline. Keep it boring. Boring pipelines sleep better.

Best practices

Track the same keyword set consistently. If the list changes every day, trends become harder to trust.

Store location, language, country, device, and timestamp with every row. A ranking without context is a floating number.

Separate organic rankings from ads, local pack, shopping, and maps results. They answer different questions.

Keep raw responses during development. They help you debug missing fields and parser issues.

Do not search every keyword every hour unless the business case is real. More data is not automatically better data.

Review ranking drops with context. A drop may come from a new competitor, a SERP feature change, a location shift, or your page actually losing relevance.

FAQ

What is Google rank tracking?

Google rank tracking is the process of monitoring where a website ranks for specific keywords in Google search results over time. Good rank tracking also stores market, language, device, URL, and timestamp context.

Can I track Google rankings without building a scraper?

Yes. A SERP API can return structured Google search results, so you can extract positions, URLs, titles, snippets, and domains without maintaining your own scraper.

Why is a SERP API better than a homemade scraper for rank tracking?

A homemade scraper requires ongoing work around parsing, blocks, location handling, retries, and layout changes. A SERP API gives you a cleaner data layer for recurring rank tracking.

How often should I track rankings?

Daily tracking is common for active SEO campaigns. Weekly tracking may be enough for slower sites. High-frequency tracking should be reserved for important keywords, launches, or volatile markets.

Should I track desktop and mobile separately?

Yes. Desktop and mobile results can differ. Store device as a separate field so reports do not mix two different SERP contexts.

Can this workflow track local rankings?

Yes. Add location fields such as city or region, then store the location with every ranking snapshot. Local SEO tracking should not rely only on country-level results.