robots.txt or a bot-blocking rule on your server is denying the crawler (facebookexternalhit). It is returned only after Ayrshare’s automatic re-host fallback has also failed, which strongly indicates a host-side block or an unreachable source.
When Ayrshare publishes to Instagram or Threads, it does not blindly hand your URL to Meta and give up. Ayrshare automatically retries the fetch and, if Meta still cannot reach the media, re-hosts a copy on its own CDN and asks Meta to fetch that instead. Code 479 means even that fallback could not get Meta to fetch the media — so retrying the same request will not help until the underlying hosting issue is fixed (retryAvailable: false, HTTP 400).
This page covers failures whose error message or details mention that the social network could not download the media, typically referencing
facebookexternalhit, robots.txt, "Restricted by robots.txt", "HTTP error code 403", or Meta subcode 2207052. For aspect-ratio or format errors on Instagram code 138, see Instagram Media Guidelines or Threads Media Guidelines instead.Symptom
When the crawler is blocked, you’ll see errors like these:Error 479 (primary — dedicated, non-retryable)
Error 440 (transient ingestion — retryable)
Instagram Error 138 (fallback — less specific upstream response)
Threads Error 379
Understanding the split between these codes:
- Code 479 — dedicated media-fetch / crawler-block (non-retryable). This is the code for the failure covered by this page. It is returned with
retryAvailable: falsewhen Meta could not fetch the media (subcode2207052or a media-fetch text pattern) even after Ayrshare re-hosted it on its own CDN. Its message explicitly namesfacebookexternalhit/Facebotandrobots.txt— if you see 479, you’re on the right page. Fix hosting before retrying. - Code 440 — transient ingestion (retryable). Now used for genuinely transient ingestion exhaustion — Meta subcodes
2207032(“download too slow”) /2207003(“create media fail”), or unclassified exhaustion. It carriesretryAvailable: true; retry via/post/retry. - Code 138 — less-specific Instagram fallback. Emitted for the same root cause when the upstream response is less specific. 138 is also used for aspect-ratio / format issues, so the media-fetch variant is identifiable by
"Restricted by robots.txt"or"HTTP error code 403"indetails. - Code 379 — Threads. Does not include a
detailsfield. If Threads fails alongside an Instagram 479 or 138, the root cause is typically the same.
Why This Happens
When you publish to Instagram or Threads via Ayrshare, Meta’s servers fetch your media from the URL you provide. This server-side fetch uses thefacebookexternalhit User-Agent. If your server’s robots.txt disallows this crawler — or a WAF/bot-protection rule blocks it — Meta cannot download the file and the publish fails.
Facebook Page publishing uses a different ingestion path, which is why the same mediaUrl may work for Facebook but fail for Instagram and Threads.
Fix: Update Your robots.txt
Recommended: Allow Meta explicitly, keep others open
Add these rules to yourrobots.txt file:
robots.txt
Advanced: Lock down to social publishers only
If you want to block most crawlers but allow social media platforms:robots.txt
Verify Meta Can Fetch Your URL
Before retrying your post, verify that Meta’s crawler can now access your media. Run this command, replacing$URL with your full media URL:
- Healthy response: HTTP 200 or 206 with binary data in the body.
- Blocked response: HTTP 403 or an empty/HTML error page.
Per Meta’s documentation,
robots.txt changes may take up to 24 hours to propagate through Meta’s crawler cache. If verification succeeds but your post still fails, wait and retry later.If This Doesn’t Fix It
If you’ve updatedrobots.txt and verified with the curl command but still see failures:
- 24-hour propagation delay — Meta caches
robots.txt. Wait up to 24 hours after making changes before retrying. - WAF or bot-fight rules — Cloudflare Bot Fight Mode, AWS WAF managed bot rule groups, and similar services may block Meta’s crawler IP ranges even if
robots.txtallows it. Check your WAF logs and add an exception forfacebookexternalhit. - Hotlink protection / Referer checks — Some CDNs block requests from data-center IPs or without a valid
Refererheader. Whitelist Meta’s crawler or disable hotlink protection for media paths. - Signed-URL / presigned-URL expiry — If your media URL has an expiration timestamp (common with S3 presigned URLs), ensure it doesn’t expire before Meta’s crawler can fetch it. For scheduled posts, generate URLs that remain valid until well after the scheduled time.
- Managed media hosting — If you use a service like Cloudinary, Imgix, or similar where you cannot edit
robots.txt, check their documentation for a Meta/Facebook crawler allow-list setting.
- The failing
postIdfrom the error response - The output of the
curlverification command above - Your
robots.txtcontents
Retry a Failed Post
Retry behavior depends on which code you received:- Code 440 (transient,
retryAvailable: true) — the ingestion simply ran out of time or hit a temporary Meta error. Retry directly using the Retry Post endpoint; no changes to your hosting are needed. - Code 479 (
retryAvailable: false) — Meta could not fetch the media even after Ayrshare re-hosted it on its own CDN, so retrying the unchanged blocked URL will fail again. Fix hosting first: allow Meta’s crawlers (facebookexternalhit/Facebot) inrobots.txtand your WAF, or re-host the media on a Meta-reachable CDN. Verify with thecurlcommand above, then retry via the Retry Post endpoint.