Cloudflare announced on Aug 30, 2026 that its AI search custom data service now lets agents and developers query private corpora without leaving the edge network. The extension introduces a Custom Data API, per-account encryption, and a prompt-injection hook callable from Cloudflare Workers.

Cloudflare AI search custom data architecture

  • Edge-native encoder: Runs on Cloudflare’s Raptor-X ASIC (7 nm, 2 TFLOPs per node) delivering sub-50 ms latency for typical knowledge-base queries.
  • Token budget control: Queries are limited to 2,048 tokens with automatic truncation to keep costs predictable.
  • Custom Data API: Developers POST JSON-L or CSV files; the service builds a vector index on-the-fly and stores embeddings in a per-tenant KV store.
  • Security envelope: Embeddings are encrypted with customer-managed keys via Cloudflare Keyless SSL, ensuring zero-knowledge handling.

The design mirrors recent edge-centric LLM services, pushing inference close to data sources to cut round-trip latency and reduce exposure of sensitive payloads.

How developers embed searchable prompts

import { searchCustom } from "@cloudflare/ai";
addEventListener("fetch", event => {
  const query = new URL(event.request.url).searchParams.get("q");
  event.respondWith(searchCustom({
    index: "my‑kb",
    prompt: `Answer using only the facts in my knowledge base: ${query}`
  }));
});

The searchCustom call resolves the query against the encrypted vector store, runs the prompt through the edge encoder, and returns a JSON payload with ranked snippets. Because the pipeline stays on the edge, developers avoid the typical 200-300 ms round-trip to a central cloud provider.

Immediate use cases and business impact

  • Customer-support bots: Index ticket histories and FAQs to keep AI answers within corporate knowledge.
  • Security operations: Upload recent alerts and threat intel for rapid, context-aware triage without exposing logs to third-party LLMs.
  • Internal DevOps assistants: Index runbooks and CI/CD logs, enabling AI-driven suggestions that respect internal policies.

Each scenario benefits from low latency, data isolation, and deterministic token usage, which translates into faster resolution times and reduced operational risk.

Performance benchmarks and cost model

Internal tests show a 3.2× speedup over an AWS Lambda + OpenAI embedding pipeline for 10 KB documents. A single Raptor-X node handled 1,000 queries per second at 42 % CPU utilization. Pricing is tiered: the first 10 GB of vectors are free, then $0.12 per GB-month and $0.0008 per query. These figures position Cloudflare as a cost-effective alternative for enterprises already on its CDN and security stack.

Security, compliance, and risk considerations

Cloudflare guarantees that custom data indexes are not shared across tenants and complies with GDPR, CCPA, and ISO-27001 by default. Customers can enable data-residency flags for US-East, EU-West, or AP-South regions. An audit-log API records every query, token, and snippet hash, aiding SOC 2 audits. However, storing embeddings of personal data at the edge introduces regulatory scrutiny; firms must ensure token-level auditability to satisfy emerging data-protection guidelines.

Market dynamics and competitive landscape

By adding searchable custom data, Cloudflare narrows the gap with Azure Cognitive Search and Amazon Kendra, both of which already support private indexes. Cloudflare’s edge-first approach gives it a latency advantage that could sway latency-sensitive workloads such as real-time fraud detection. Analysts note that "edge-centric AI services are the next frontier for cloud providers seeking to lock in high-value enterprise customers."

What to watch next

  • Model upgrades: A forthcoming 1.2 B-parameter encoder will support multilingual embeddings.
  • Ecosystem expansion: SDKs for Python and Go will let developers manage indexes without writing Workers code.
  • Regulatory trends: Data-protection authorities may issue guidance on edge-stored embeddings, affecting compliance strategies.

Developers interested in the underlying research can explore the open-source benchmark that Cloudflare used, which is listed on code alongside the paper.

Quick start checklist for teams

  1. Create a Cloudflare Workers account and enable the AI Search add-on.
  2. Upload data via the Custom Data API, specifying a unique index name.
  3. Define a per-tenant CMK if you need customer-managed encryption.
  4. Write a Worker that calls searchCustom with a domain-specific prompt.
  5. Monitor query latency and audit logs through the Cloudflare dashboard.

By following these steps, organizations can replace brittle keyword-search systems with a secure, LLM-powered experience that lives at the edge.

This article draws on the original InfoQ report and adds original analysis of performance, security, and market implications.

Related coverage

Explore more on this topic