Application Insights and Azure AI Search

Two Azure services a Python backend meets constantly: the observability stack, and the managed retrieval service that shows up in every Azure RAG design.

Application Insights

Azure’s APM, part of Azure Monitor and backed by Log Analytics. It collects requests, dependencies, exceptions, traces and custom metrics, and correlates them into distributed traces.

The current way to instrument Python is OpenTelemetry, not the legacy SDK:

python

from azure.monitor.opentelemetry import (
    configure_azure_monitor,
)

configure_azure_monitor()   # reads the conn string
FastAPIInstrumentor.instrument_app(app)

That single call wires OTel export to App Insights. The reason it matters beyond convenience: your instrumentation stays vendor-neutral. The same OTel spans can go to Grafana Tempo or Jaeger if the platform moves, which is not true of the old opencensus integration.

The concepts that differ from other APMs

Dependencies are outbound calls - to Postgres, to an HTTP API, to Blob Storage - tracked automatically once instrumented. A slow endpoint is usually diagnosed by looking at its dependency breakdown rather than its own code.

Sampling is on by default. Adaptive sampling drops a proportion of telemetry under load to control cost, which means counts in App Insights are estimates, not exact. Metrics carry an itemCount for reweighting. Reading a sampled request count as a true number is a common and expensive misinterpretation.

KQL is the query language, and being able to write it is a real differentiator:

kusto

requests
| where timestamp > ago(1h)
| summarize p95=percentile(duration, 95) by name
| order by p95 desc

Alerts are KQL queries on a schedule, so the same language covers dashboards, investigation and alerting.

Cost

Billed on ingested volume. The way this goes wrong is verbose logging at INFO across a high-traffic service, and the bill arrives a month later. Set a daily cap, and sample deliberately rather than accepting adaptive defaults. See Sampling Strategies for Tracing.

Azure AI Search

A managed search service - formerly Cognitive Search - offering keyword search, vector search and the hybrid of both. In Azure RAG designs it usually occupies the slot Qdrant or pgvector would elsewhere.

What makes it the default choice on Azure is not the vector index; it is that hybrid retrieval and reranking are built in.

python

results = client.search(
    search_text=query,                    # BM25
    vector_queries=[VectorizedQuery(
        vector=embed(query), k_nearest_neighbors=5,
        fields="contentVector")],         # ANN
    query_type="semantic",                # reranker
)

Three stages in one call: BM25 keyword matching, vector similarity, fused with Reciprocal Rank Fusion, then a semantic reranker over the merged set. Building that yourself means a vector store, a keyword index, a fusion step and a cross-encoder - which is the honest argument for the managed service.

Integrated vectorization goes further, chunking and embedding documents on ingestion via a skillset, so the pipeline is configuration rather than code. That is convenient and it is also lock-in: the chunking strategy becomes the service’s rather than yours, and chunking is one of the highest-leverage choices in RAG - see Chunking.

Where it costs you

Pricing is per search unit - replicas for query throughput and availability, partitions for index size - not per query. It is a provisioned service, so an idle index still bills. Below a few hundred thousand documents, pgvector next to your existing Postgres is usually cheaper and simpler.

Interview angle 5

“How do you instrument a Python service for Application Insights?” - the Azure Monitor OpenTelemetry distro rather than the legacy SDK, so instrumentation stays vendor-neutral and the same spans could be exported elsewhere later.
“Your request count in App Insights looks low. Why?” - adaptive sampling is on by default and drops telemetry under load, so counts are estimates. Reweight using itemCount, or reduce sampling if you need exact numbers, and expect to pay for it.
“How do you find why an endpoint is slow?” - its dependency breakdown, since App Insights tracks outbound calls to databases, HTTP services and storage automatically. Then a KQL percentile query by operation name, rather than reading averages.
“Why use Azure AI Search over a vector database?” - it does hybrid retrieval in one call: BM25 plus vector search fused with RRF, then a semantic reranker. Reproducing that means a vector store, a keyword index, fusion and a cross-encoder. It is provisioned rather than per-query, so it is poor value on a small corpus where pgvector suffices.
“What is the catch with integrated vectorization?” - the service chunks and embeds on ingestion, which is convenient but hands over the chunking strategy, one of the highest-leverage decisions in a RAG system, and makes moving off harder.

Application Insights and Azure AI Search

Application Insights and Azure AI Search