ads.txt "Not found" or "Unauthorized": a technical checklist for redirects, HTTPS, status codes and robots.txt
Key takeaways
- Google starts ads.txt crawling at the root domain: a file at www.domain.com/ads.txt is only crawled if domain.com/ads.txt redirects to it.
- Google crawls ads.txt on both HTTP and HTTPS; if one scheme returns 404, previously crawled entries can be purged even though the other scheme works.
- The ads.txt response must have HTTP status 200 OK; Google ignores a response whose status code says the file was not found, even if a body is returned.
- Google supports multiple redirects that stay within the original root domain and a single redirect to a destination outside it.
- A robots.txt rule such as Disallow: /ads, or one that disallows Googlebot, can make Google's crawler ignore the ads.txt file.

AdSense shows “Not found” when its crawler cannot fetch a valid ads.txt from your root domain, and “Unauthorized” when it fetches one that lacks your publisher ID. Not found is almost always a delivery problem: the bare domain does not serve or redirect to the file, one of HTTP or HTTPS returns 404, the status is not 200, or robots.txt blocks the path. Test each case with curl, not just a browser.
Not found vs Unauthorized: which problem do you have?
| Status | Meaning | Where the fault usually is |
|---|---|---|
| “Not found” | No ads.txt file detected | Server, redirects, DNS, robots.txt, CMS routing |
| “Unauthorized” | File found, your publisher ID missing | The content of the file |
For “Unauthorized”, the fix is in the text: add the line from the “Ads.txt snippet” in AdSense, and check for a ca-pub- prefix or a typo (see ads.txt syntax). The rest of this guide covers “Not found”, where the file often looks fine in a browser but Google’s crawler sees something else.
Which URL does Google actually crawl?
Google’s Ensure your ads.txt files can be crawled page sets three rules:
- Root domain first. “Ads.txt crawling will start at the root domain, and the root domain needs to return from, or redirect to, the ads.txt file.” The root domain is the registrable domain (the IAB spec defines it as the public suffix plus one label), so for
blog.example.co.ukit isexample.co.uk. - Both schemes. “The Google crawler attempts to crawl ads.txt files on both HTTP and HTTPS.” Each must either serve the file or redirect to the scheme that does.
- Status 200. If the status code “indicates the file was not found (e.g., status code 404)”, then “the response will be ignored” and “the file will be considered non-existent”, even if a body is returned.
How do I test it like a crawler?
A browser follows redirects silently, reuses cached responses and may show a page your server returns with an error code. Use curl from a terminal:
curl -sIL http://example.com/ads.txt
curl -sIL https://example.com/ads.txt
curl -sIL http://www.example.com/ads.txt
curl -sIL https://www.example.com/ads.txt
-I shows headers only, -L follows redirects, -s hides the progress bar. For each command, read the chain of HTTP/ status lines and location: headers. A healthy result ends with 200 and a content-type: text/plain header. Then read the body once:
curl -sL https://example.com/ads.txt
You should see your plain-text lines, not HTML.
What redirects are allowed?
The Ads.txt FAQs and the IAB ads.txt 1.1 specification agree:
| Redirect | Allowed? |
|---|---|
example.com/ads.txt to www.example.com/ads.txt |
Yes (within the root domain) |
http:// to https:// on the same host |
Yes |
example.com/ads.txt to sub.example.com/ads.txt |
Yes |
example.com/ads.txt to example.com/page/ads.txt |
Yes (Google’s own example) |
Several hops, all within example.com |
Yes |
One hop to other-domain.com/ads.txt |
Yes, a single hop outside the root domain |
other-domain.com/ads.txt then redirects again |
No: the spec says treat it as an error |
The spec lists 301, 302 and 307 as the redirect codes crawlers should follow. Redirects done with JavaScript or a meta refresh are not HTTP redirects; a crawler reading /ads.txt sees an HTML page instead.
What are the common causes of “Not found”?
The file only exists on www (or only on the bare domain)
If your site lives on www.example.com and the bare domain has no web server, or serves a parking page, Google never reaches the file. Google’s instruction: “If you need to include ‘www’ to access your site, then you need to set up a redirect from domain.com/ads.txt to www.domain.com/ads.txt.” A sitewide bare-to-www redirect works if it preserves the path. A redirect that sends every request to the home page does not.
HTTP or HTTPS returns 404
Google gives this exact scenario: if there is only an ads.txt file for HTTP, the HTTPS crawl returns 404 and “the previously crawled entry will be purged even though an ads.txt file is crawled via HTTP.” Redirect the failing scheme to the working one. On an HTTPS site, the cleanest setup is a permanent redirect from every http:// URL to https://.
The status code is not 200
Two variants:
- Hard 404 with a body. Some setups return a custom error page with status 404. Google ignores it.
- Soft 404. Many CMSs and single-page apps answer every unknown URL with status 200 and the home page or an app shell. The status looks fine, but the content is HTML. The IAB spec says the content type should be
text/plainand that “all other Content-types should be treated as an error and the content ignored”.
The fix in both cases is to make /ads.txt a real static file (or a route that returns plain text) that takes precedence over the CMS fallback. Host-specific steps are in deploying ads.txt on static hosts and CDNs and ads.txt on WordPress without a plugin.
robots.txt blocks it
Google’s examples: with User-agent: * and Disallow: /ads, “the ads.txt file will be ignored by crawlers that respect the robots.txt standard”, because /ads also matches /ads.txt. With User-agent: Googlebot and Disallow: /, it “will be ignored by the Google crawler.” Google’s suggested fixes:
User-agent: *
Allow: /ads.txt
Disallow: /ads
or change Disallow: /ads to Disallow: /ads/. Google adds that robots.txt changes “may not be reflected until the crawlers attempt to visit your site again.”
Firewalls, bot protection and geo-blocking
If a web application firewall, a bot-protection challenge, HTTP authentication or a country block intercepts the request, the crawler receives a challenge page or an error instead of the file. Your browser may pass the check while an automated client does not. The IAB spec says a 401 (restricted) response means the ad system “should seek direct contact with the site”. Exclude /ads.txt from challenges and authentication. The same issues block site review, as described in site down or unavailable.
Wrong location or name
The file must be at the root path, exactly /ads.txt, lowercase. example.com/blog/ads.txt or example.com/ads.txt/ are not the same URL. If your site is on a subdomain, see ads.txt and subdomains.
Too early
Google says changes “may take a few days”, and “up to a month” for sites with few ad requests. Fix delivery first, then use “Check for updates” on the site’s page in AdSense and wait.
Diagnostic table
| curl result | Diagnosis | Fix |
|---|---|---|
200, text/plain, your lines |
Delivery OK | Check content, wait for recrawl |
200, text/html, page markup |
Soft 404 or CMS route | Serve a real static file |
404 on HTTPS only |
Missing on one scheme | Redirect HTTP and HTTPS to the same file |
404 on bare domain, 200 on www |
No root redirect | Redirect example.com/ads.txt to www |
301 to / (home page) |
Path dropped by redirect | Preserve the path in the redirect rule |
403 or 503 with a challenge page |
Firewall or bot protection | Exempt /ads.txt |
401 |
Password protection | Remove auth on /ads.txt |
| Connection error on bare domain | No DNS record or server | Add DNS and a redirect for the apex |
| Two external redirects | Chain leaves the root domain twice | Point directly to the final file |
What if I don’t control the root domain?
If the top-level domain belongs to a platform, the Ads.txt troubleshooter says you can ask the owner to host the file on your behalf, or “to set up a subdomain referral for you”. You cannot fix that from your own subdomain alone.
When curl shows a clean 200 text/plain on all four URL variants and the file contains your exact line, the remaining factor is time. For background on what the alert costs while you wait, see the Earnings at risk alert.
Frequently asked questions
My site uses www. Where should ads.txt be?
It can live at www.example.com/ads.txt, but Google starts crawling at the root domain, so example.com/ads.txt must redirect to it. Google's crawl help says an ads.txt file on www.domain.com/ads.txt will only be crawled if domain.com/ads.txt redirects to it. A sitewide redirect from the bare domain to www usually covers this, as long as it keeps the /ads.txt path.
Can ads.txt redirect to another domain?
Once. Google supports a single HTTP redirect to a destination outside the original root domain, for example example1.com/ads.txt to example2.com/ads.txt. Redirects that stay inside the root domain (to www, to a subdomain or to another path) can be chained. The IAB specification says that if the third-party location redirects again, it should be treated as an error.
What is the difference between Not found and Unauthorized?
"Not found" means Google found no usable ads.txt file on the site. "Unauthorized" means it found a file, but your publisher ID is not in it. The fixes differ: for Not found, check the URL, redirects, status code and robots.txt; for Unauthorized, add or correct the AdSense line, usually a missing line, a typo or a ca-pub- prefix.
Does robots.txt affect ads.txt?
Yes. Google warns that the ads.txt file may be ignored if robots.txt disallows the URL path where it is posted or disallows Google's crawler. A rule like Disallow: /ads also matches /ads.txt because robots.txt rules are prefix matches. Change it to Disallow: /ads/ or add Allow: /ads.txt.
Sources
- Ensure your ads.txt files can be crawled · Google AdSense Help
- Ads.txt FAQs · Google AdSense Help
- Ads.txt guide · Google AdSense Help
- Ads.txt troubleshooter · Google AdSense Help
- Ads.txt Version 1.1 · IAB Tech Lab
Related guides
- How to fix the AdSense "Earnings at risk" ads.txt alert
Why AdSense shows the Earnings at risk ads.txt alert, what Not found and Unauthorized really cost you, and how to fix it and get the warning cleared faster.
- Deploying ads.txt on static hosts and CDNs: public folders, redirects, headers and caching
Where ads.txt goes in Astro, Next.js, Hugo and Eleventy builds, and how Netlify, Vercel, Cloudflare Pages, GitHub Pages and S3 serve, cache and redirect it.
- ads.txt and subdomains: which file the crawler reads and when to use subdomain=
Which ads.txt file governs a subdomain, when you need subdomain= in the root file, and the configurations that silently make a subdomain Unauthorized in AdSense.
- Fix "Site down or unavailable" in AdSense: make your site reachable for Google's crawlers
Why AdSense says your site is down or unreachable when it loads fine for you: DNS, HTTPS, redirects, robots.txt, geo-blocking and Cloudflare-style bot protection.
- Validating ads.txt yourself: curl checks, the IAB Tech Lab open-source crawler and scripted audits
Check your ads.txt without third-party validators: curl tests that mimic a crawler, the IAB Tech Lab open-source crawler and its limits, and a short audit script.