Copy page

Getting Started

Web Search for AI

Extend Web Search with content enrichment: semantically scored passages (extra_snippets) and/or full page body (full_content). Results are reranked by relevance — built for RAG pipelines, LLM agents, and context-augmented generation.

Code
GET https://api.staan.ai/v2/search/web
POST https://api.staan.ai/v2/search/web

Quick start

Code
curl "https://api.staan.ai/v2/search/web?q=vector+database+comparison&extra_snippets=true&max_snippets=5&min_score=0.2" \
  -H "Authorization: Bearer YOUR_API_KEY"

Code
curl "https://api.staan.ai/v2/search/web?q=vector+database+comparison&full_content=markdown" \
  -H "Authorization: Bearer YOUR_API_KEY"

You can restrict enriched results to trusted sources with domain filtering — either site: operators inside the query, or the include_domains array on POST:

Code
curl "https://api.staan.ai/v2/search/web?q=vector+database+comparison+site:qdrant.tech+OR+site:weaviate.io&extra_snippets=true" \
  -H "Authorization: Bearer YOUR_API_KEY"

Code
curl -X POST "https://api.staan.ai/v2/search/web" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "q": "vector database comparison",
    "market": "en-us",
    "extra_snippets": true,
    "max_snippets": 5,
    "include_domains": ["pinecone.io", "qdrant.tech", "weaviate.io"]
  }'

Code
import requests

response = requests.get(
    "https://api.staan.ai/v2/search/web",
    params={
        "q": "vector database comparison",
        "extra_snippets": "true",
        "max_snippets": 5,
        "min_score": 0.2,
        "full_content": "markdown",
    },
    headers={"Authorization": "Bearer YOUR_API_KEY"},
)

for result in response.json()["web"]["results"]:
    print(result["url"])
    for chunk in result.get("extra_snippets", []):
        print(f"  [{chunk['score']:.2f}] {chunk['chunk'][:120]}...")

Enrichment parameters

Parameter	Type	Default	Description
extra_snippets	boolean	false	Fetch result pages and return semantically scored chunks. Triggers reranking.
full_content	"markdown" | "html"	—	Return the full page body. markdown is recommended for LLM prompts; html for raw markup. Also triggers reranking.
max_snippets	number	3	Max scored chunks per URL (1–10). Only with extra_snippets=true.
min_score	number	0.1	Minimum relevance score (0–1) for a chunk to be included. Only with extra_snippets=true.

How enrichment works

When extra_snippets=true or full_content is set, the API runs a post-search pipeline:

1. SERP — initial ranked results from the search engine
2. Fetch — parallel page downloads (3.5 s per URL, 4 s global timeout)
3. Chunking — content split on Markdown structure (H1–H6); long sections split at ~1,200 characters with overlap, heading context preserved
4. Reranking — all chunks scored against the query; URLs with the best chunks are promoted
5. Filtering — min_score and max_snippets applied per URL; URLs with no qualifying chunks are moved to the end

Enrichment fields

These fields appear on each result when enrichment is active:

Field	When present	Description
extra_snippets	extra_snippets=true	Array of { chunk, score }, ordered by descending relevance score.
full_content	full_content set	{ text, format, length } — full page body.
published_date	Fetch occurred and date resolved	ISO 8601 publication date.

extra_snippets

Code
"extra_snippets": [
  { "chunk": "Most relevant content segment from this page...", "score": 0.87 },
  { "chunk": "Next relevant segment...", "score": 0.72 },
  { "chunk": "A less relevant passage.", "score": 0.31 }
]

• chunk: text segment (~300–1,800 characters) with heading context preserved
• score: relevance in [0, 1], descending

extra_snippets: [] is returned when the page could not be fetched (anti-bot, timeout) or no chunk scored above min_score. Such URLs remain in the response, placed after reranked results.

full_content

Code
"full_content": {
  "text": "# Page title\n\nPage body content...",
  "format": "markdown",
  "length": 12345
}

Always check length > 0 before using text. On fetch failure, the field is present with "text": "" and "length": 0.

published_date

Code
"published_date": "2026-04-24T00:00:00.000Z"

Only present when a fetch occurred and the HTML extractor resolved a date. Check for presence before use.

Full response example

Code
{
  "search_id": "01906c9e-7e3f-7000-8000-abc123def456",
  "query": {
    "q": "vector database comparison",
    "market": "en-us",
    "count": 10,
    "offset": 0
  },
  "web": {
    "results": [
      {
        "title": "Comparing vector databases in 2024",
        "url": "https://www.example.com/vector-dbs",
        "snippet": "A deep dive into Pinecone, Weaviate, Qdrant...",
        "display_url": "www.example.com > ai > vector-dbs",
        "hostname": "www.example.com",
        "published_date": "2024-09-10T00:00:00.000Z",
        "extra_snippets": [
          {
            "chunk": "Pinecone is a fully managed vector database...",
            "score": 0.91
          },
          {
            "chunk": "Qdrant offers a self-hosted alternative with...",
            "score": 0.78
          }
        ],
        "full_content": {
          "text": "# Comparing vector databases\n\nThis guide covers...",
          "format": "markdown",
          "length": 22100
        }
      }
    ]
  }
}

Limits and latency

Limit	Value
Rate limit	20 req/s
Recommended client timeout	8–10 s when enrichment is enabled

Plain search (no enrichment) is significantly faster. Budget for extra latency when extra_snippets or full_content are set.

Best practices

• Use extra_snippets=true for RAG: targeted passages reduce prompt noise and improve answer quality.
• Use domain filtering (include_domains on POST, or site: in q) to ground answers in a curated set of trusted sources.
• Prefer full_content=markdown over html for LLM context unless you need raw markup.
• Cite url (and chunk text) in generated answers for traceability.
• For end-to-end Q&A without managing retrieval yourself, consider Answer.