AI agent traffic

A growing share of what fetches your pages is not a person. Assistants read your site to answer someone’s question, search crawlers index it, and training crawlers collect it. That is worth knowing about, and it is emphatically not worth mixing into your visitor counts — so Tinylytics records it separately. ChatGPT fetching /pricing to answer a question shows up as an agent visit, and your traffic chart stays honest.

There is a catch built into how this traffic behaves. Most crawlers request the raw HTML and never execute JavaScript, which means the embed script cannot see them at all. To capture them properly the request has to be reported from your side — from your backend, your middleware, or your edge.

What gets recorded

Tinylytics recognises known user agents and sorts them into four kinds:

AI answers — someone asked an assistant a question and it fetched your page to answer it (ChatGPT-User, Claude-User, Perplexity-User)
Indexing — crawlers building a search index, whether AI-flavoured or classic (OAI-SearchBot, PerplexityBot)
Training — crawlers collecting public pages for model training (GPTBot, ClaudeBot, Google-Extended)
AI crawler — other named AI bots that do not fit the three above

Anonymous scrapers and bots pretending to be Chrome do not make this list, because there is nothing trustworthy to categorise them by. They are still kept out of your human hits.

Where to see it

The AI agents card on a site’s Overview breaks the visits down by agent and by page for whichever date range you have selected. It is hidden by default; switch it on under Settings → Appearance → Stats sections. These visits are never folded into views, unique hits, or public stats. This card is not the Referrers AI assistants chip, which only groups human hits whose Referer host is a known assistant.

Capturing crawlers that skip JavaScript

If a request never reaches us, we cannot record it — so send it yourself:

curl -X POST "https://tinylytics.app/api/v1/sites/SITE_ID/agent_visits" \
  -H "Authorization: Bearer tly-fa-your-api-key" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{
    "path": "/pricing",
    "url": "https://example.com/pricing",
    "user_agent": "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0"
  }'

This needs a full-access API key. Unrecognised browser user agents are ignored on this endpoint, so you can safely fire it for every request and let us decide what counts. If you would rather keep one code path, posting the same payload to POST /sites/:id/hits also works — known crawlers are classified as agent visits rather than becoming human hits.

Here is the same idea as Rails middleware:

class TrackAiCrawlers
  def initialize(app)
    @app = app
  end

  def call(env)
    request = ActionDispatch::Request.new(env)
    track(request) if request.get? || request.head?
    @app.call(env)
  end

  private
    def track(request)
      Thread.new do
        Faraday.post("#{ENV.fetch("TINYLYTICS_API_BASE_URL")}/api/v1/sites/#{ENV.fetch("TINYLYTICS_SITE_ID")}/agent_visits") do |req|
          req.headers["Authorization"] = "Bearer #{ENV.fetch("TINYLYTICS_API_KEY")}"
          req.headers["Content-Type"] = "application/json"
          req.body = {
            path: request.path,
            url: request.original_url,
            user_agent: request.user_agent,
            referrer: request.referer
          }.to_json
        end
      end
    end
end

Whatever you do, do not make your visitors wait on that call. Fire and forget, or push it into a background job — analytics should never be able to slow down a page.

API

GET /api/v1/sites/:id/agent_visits — list visits; a read-only key is enough
POST /api/v1/sites/:id/agent_visits — record one visit; needs a full-access key

The API documentation has the full parameter list.

Navigation