LinkToLooks · Guides · Creator Desk

Creator Desk

Does a storefront page compete with your own posts?

No, and the reason sits in the platforms’ own robots files: your posts and your storefront are not in the same index, and one of them is barred from the answer engines entirely.

Part of: For creators

What the platforms actually allow, read today

A robots file is the only place a platform states, in public and in writing, which crawlers may read it. We fetched three of them this morning with an ordinary browser user agent and, where it mattered, again as Googlebot.

Last verified 4 September 2026 against the live robots.txt of instagram.com, tiktok.com and youtube.com

Instagram is the strict one, and its structure matters more than its headline. The file names roughly two dozen agents individually — Googlebot, Bingbot, DuckDuckBot, Applebot, facebookexternalhit and others — and gives each a short list of paths to avoid, mostly logins, comments and internal endpoints. Then, at the end, it says this to everybody else:

If a crawler is not on the list, it gets nothing. That is a deliberate allow-list, and it means the crawlability of your Instagram content is a decision Meta makes, revisits, and can change without telling you.

TikTok takes the opposite shape and lands somewhere similar. Its wildcard group allows the discovery surfaces and blocks search results, embeds, in-app paths and the shop product pages; profile and video paths are not disallowed, so ordinary search engines may read them. YouTube is the most open of the three — its blocks are almost entirely functional endpoints, and it publishes two sitemaps.

The gap nobody mentions: answer engines are locked out

This is the finding we did not expect to be so one-sided. Both Instagram and TikTok name the AI crawlers explicitly and shut them out of the whole site.

Take the position plainly, because the data supports it: when somebody asks an AI assistant which running shoe to buy, your Reel about that shoe is not in the room. It was excluded by the platform, not by the assistant, and not by anything you did.

A page on a domain you control is in the room by default. That is not a claim that it will be quoted — being crawlable is a long way from being cited, and we have no evidence that any answer engine has ever quoted us. It is a claim about eligibility, which is precisely the part the platform takes away.

The practical version: a storefront page is not competing with your posts for one slot. It is covering a surface your posts are barred from. If you have ever wondered why your product recommendations never surface when somebody asks an assistant for exactly that recommendation, this is a large part of the mechanism.

What a storefront page looks like to a crawler

A storefront page only covers that surface if a crawler can read it, and a page can look complete in a browser while its raw HTML holds almost nothing. If the product list is assembled by JavaScript after the page loads, a crawler that reads only the fetched HTML gets almost none of it.

Google says it processes JavaScript pages in three phases — crawling, rendering and indexing — and that a page can wait in the rendering queue for more than a few seconds. Its JavaScript guidance still calls server-side rendering or pre-rendering a great idea, because it makes a site faster for users and crawlers, and because not all bots can run JavaScript.

This failure is invisible from the inside. If your link-in-bio page or storefront is built by a tool that assembles the product list in the browser, the crawler may be reading a headline and a footer. That page is not competing with your posts. It is not competing with anything.

You can check yours in about a minute, with no tools: open the page, view source rather than inspect element, and search the raw HTML for a product name you can see on screen. Present in the source, the crawler has it. Absent, it does not.

What we cannot show you, and why we are saying so

The strict form of the cannibalisation question is whether a storefront page and a creator’s own posts, both ranking, split the clicks one of them would otherwise have taken. We cannot answer that, and here is exactly why not.

• We have no traffic to divide. Observing a split needs click data from a creator’s posts and their storefront over the same period, and we hold none for anyone.
• We hold no data about anybody else’s storefront. No connected creators, no aggregate panel, no third-party click stream. Where a competitor page would put a chart here, we would be inventing it.
• Robots rules describe permission, not behaviour. A crawler allowed to read a page may still decline to index it. What we measured is the gate, not what came through it.

What survives all three caveats is the structural argument, and it needs no traffic data: different domains, different index surfaces, and one of the two surfaces closed by the platform’s own file. That is enough to retire the worry without pretending to a measurement we do not have.

Who should worry about this, and who should not

Not a problem for you if

• Your storefront lives on a domain you own, or on a subdomain a tool gave you. Different site, different index entry, no split.
• Your storefront page targets a different phrasing from your posts — product questions rather than your handle.
• You are early. Nothing ranks yet, and a competition needs two entrants.

Worth attention if

• You run several pages of your own, on one domain, all targeting the same phrase. That is genuine self-competition, and it is the situation the advice was originally written for.
• Your storefront duplicates its product list at more than one URL — a filtered view, a tag view, a paginated view. Pick one canonical version and point the others at it.
• Your storefront renders in the browser rather than on the server. Fix that first: an unreadable page cannot compete with anything.

Our editorial judgment: creators are handed this worry by people applying ecommerce SEO advice to a situation that does not contain the conditions the advice assumes. The thing actually worth your attention is whether the storefront is readable at all.

Frequently asked

Will a storefront page outrank my Instagram profile for my name?

Almost certainly not, and you would not want it to. A profile on a very high authority domain that exactly matches a handle search is hard to displace. A storefront page competes for product questions, which is a different query set entirely.

Does Google penalise me for having the same product on both?

We have seen no evidence of that and no Google documentation saying so. Duplicate-content handling applies to substantially identical pages, usually within one site. A caption and a storefront entry are neither identical nor on one site.

Should I noindex my storefront to protect my posts?

We would not. You would be removing the only one of the two surfaces you actually control. Keeping a storefront private is a privacy decision rather than a ranking one, and it is a real reason.

Do the AI crawler blocks apply to my own site too?

Only if you add them. A site with no robots.txt, or a permissive one, is readable by all of these crawlers by default. Whether you want that is a separate and genuine question.

Method and scope

Robots files were fetched on 4 September 2026 from instagram.com/robots.txt, tiktok.com/robots.txt and youtube.com/robots.txt. Instagram returns different documents to different user agents: 616,296 bytes of HTML to a Chrome agent and 6,258 bytes of plain text to Googlebot, at the same URL in the same minute. The plain-text version is the one quoted here. Google’s description of how it processes JavaScript comes from its JavaScript SEO basics page, read 29 September 2026. Rules quoted are the rules as published on that date, and platforms change them without notice. We hold no ranking, click or impression data for any creator’s posts or storefront.

How this page was made. Drafted with AI (Claude); not yet independently re-checked. How we work