Custom & Headless Guide

Optimize Custom Stores for AI Agents

Technical developer checklist for custom React, Next.js, Vue, or Laravel backends to optimize indexable API feeds and metadata schema.

1 Dynamic Robots.txt Endpoint

If you are using Next.js (App Router), create a app/robots.ts file to programmatically expose rules allowing crawler bots access to your product dynamic routes:

import { MetadataRoute } from 'next' export default function robots(): MetadataRoute.Robots { return { rules: { userAgent: ['GPTBot', 'PerplexityBot', 'ClaudeBot'], allow: ['/products/', '/api/catalog/'], }, sitemap: 'https://yourstore.com/sitemap.xml', } }

2 Server-Side Rendered (SSR) Schema.org Tags

Ensure schema.org JSON-LD scripts are parsed on the initial HTML server response (rather than client-side React rendering) so bots don't miss them.

// Next.js page metadata setup export async function generateMetadata({ params }) { const product = await fetchProduct(params.id); return { title: product.name, other: { 'product-schema': JSON.stringify({ '@context': 'https://schema.org', '@type': 'Product', 'name': product.name, 'sku': product.sku, 'offers': { '@type': 'Offer', 'price': product.price, 'priceCurrency': 'INR', 'availability': product.inStock ? 'https://schema.org/InStock' : 'https://schema.org/OutOfStock' } }) } } }

3 Expose llms.txt route

For custom backends, place the generated llms.txt file in your application public assets folder (e.g. /public/llms.txt) so it is served statically, or set up a simple path redirect handling it.

4 Connect via PointNXT Developer API

Expose webhook endpoints to listen to inventory events or push products programmatically using PointNXT's lightweight REST APIs.

Register Developer Account
FAQ

Headless & Custom Store FAQ

Common engineering questions about SSR schema markup and AI crawler indexing on custom stacks.

Building a headless commerce stack?

Connect your custom storefront directly to PointNXT's sub-second inventory and order webhooks.

Ask an Architect →
Do AI crawlers execute client-side React or Vue JavaScript?

Most AI crawlers (including GPTBot, ClaudeBot, and PerplexityBot) fetch raw initial HTML responses without executing heavy client-side JavaScript bundles. Always render product metadata and JSON-LD schemas on the server (SSR) or via static generation (ISR).

Can we generate /llms.txt dynamically from our catalog API?

Yes. In Next.js, Nuxt, or Laravel, you can expose a dynamic route at /llms.txt that queries your active categories and top products, caches the markdown output at the CDN edge, and returns Content-Type: text/plain; charset=utf-8.

How does PointNXT integrate with custom headless storefronts?

PointNXT provides REST APIs and real-time webhooks for inventory updates, order creation, and fulfillment tracking—allowing custom storefronts to maintain 99.99% inventory accuracy across D2C and marketplace channels.