AWS Bedrock: Building AI Applications with Foundation Models

Bedrock is AWS finally getting AI right. I don’t say that lightly. I’ve watched AWS stumble through SageMaker’s complexity, watched teams burn months trying to self-host open-source models on EC2, and watched startups hemorrhage money on OpenAI API calls with zero fallback plan. Bedrock cuts through all of that. You pick a foundation model, call an API, and you’re building. No infrastructure. No GPU capacity planning. No model weight management.

I’ve been running production workloads on Bedrock for over a year now, and the thing that keeps me coming back isn’t any single model — it’s the abstraction. I can swap Claude for Llama for Titan without rewriting my application. I can add guardrails without bolting on a separate content filtering service. I can build RAG pipelines with Knowledge Bases instead of stitching together OpenSearch, embedding models, and chunking logic by hand.

That said, Bedrock isn’t magic. I’ve made expensive mistakes with it. I built a document search system that cost 4x what it should have because I picked the wrong model for the job. I’ve had guardrails block legitimate queries and let problematic ones through. The service is powerful, but only if you understand what you’re actually working with.

Why Bedrock and Not Self-Hosting

I get this question constantly. “Why pay per token when I can run Llama on my own instances?”

You can. And for some workloads you should. But most teams underestimate what self-hosting actually costs. You need GPU instances — p4d.24xlarge at minimum for anything serious, and those run about $32/hour. You need model serving infrastructure. You need to handle scaling, failover, model updates, and monitoring. You need ML engineers who understand inference optimization.

With Bedrock, you get none of that overhead. The pricing is per-token, which means you pay exactly for what you use. For bursty workloads — which describes most AI applications I’ve built — this is dramatically cheaper than keeping GPU instances warm 24/7.

The real killer feature though is model choice. Bedrock gives you access to Claude (Anthropic), Llama (Meta), Titan (Amazon), Mistral, Cohere, and others through a single API. When Anthropic releases a new Claude version, I don’t redeploy anything. I change a model ID string.

# List available foundation models
aws bedrock list-foundation-models \
    --query "modelSummaries[].{id:modelId,name:modelName,provider:providerName}" \
    --output table

If you’re designing scalable systems in AWS, Bedrock fits naturally into serverless architectures. Call it from Lambda, put requests through API Gateway, store results in DynamoDB. No GPU clusters to manage.

Picking the Right Model

This is where most teams go wrong first. They pick the most powerful model available and wonder why their bill is insane. Model selection is a cost-performance tradeoff, and getting it right matters more than almost any other decision you’ll make.

Here’s my honest breakdown after testing all of them in production:

The mistake I made early on was using Sonnet for everything. We had a document classification pipeline that just needed to sort PDFs into 8 categories. Sonnet did it perfectly — and cost us $400/day. Switching to Haiku dropped that to $30/day with identical accuracy. The task was too simple to need a powerful model.

import boto3
import json

bedrock = boto3.client("bedrock-runtime")

def invoke_model(prompt, model_id="anthropic.claude-3-haiku-20240307-v1:0"):
    response = bedrock.invoke_model(
        modelId=model_id,
        body=json.dumps({
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": 1024,
            "messages": [{"role": "user", "content": prompt}]
        })
    )
    return json.loads(response["body"].read())["content"][0]["text"]

Start with the cheapest model that could plausibly work. Test it. Only move up if quality genuinely isn’t there. I’ve found Haiku handles 70% of what teams throw at Sonnet.

The Doc Search System — A RAG War Story

Last year I built an internal documentation search system for a client with about 40,000 technical documents — runbooks, architecture decisions, post-mortems, the works. Engineers were spending 20+ minutes finding relevant docs. The brief was simple: “Let people ask questions in plain English and get answers with sources.”

RAG — Retrieval-Augmented Generation — was the obvious pattern. Embed the documents, store the vectors, retrieve relevant chunks at query time, feed them to an LLM as context. Bedrock Knowledge Bases handles most of this plumbing for you.

My first attempt was a disaster. I embedded everything with Titan Embeddings, dumped it into an OpenSearch Serverless vector store, and pointed Claude Sonnet at the retrieved chunks. It worked. It also cost $2,800/month for a 50-person engineering team. The problem was threefold: I was using Sonnet when Haiku would’ve been fine for most queries, my chunking strategy was terrible (fixed 512-token chunks that split paragraphs mid-sentence), and I was retrieving 10 chunks per query when 4 would’ve been plenty.

Here’s what the fixed version looked like using Bedrock Knowledge Bases:

# Create the knowledge base
aws bedrock-agent create-knowledge-base \
    --name "engineering-docs" \
    --role-arn arn:aws:iam::123456789012:role/bedrock-kb-role \
    --knowledge-base-configuration '{
        "type": "VECTOR",
        "vectorKnowledgeBaseConfiguration": {
            "embeddingModelArn": "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0"
        }
    }' \
    --storage-configuration '{
        "type": "OPENSEARCH_SERVERLESS",
        "opensearchServerlessConfiguration": {
            "collectionArn": "arn:aws:aoss:us-east-1:123456789012:collection/abc123",
            "fieldMapping": {
                "metadataField": "metadata",
                "textField": "text",
                "vectorField": "vector"
            },
            "vectorIndexName": "docs-index"
        }
    }'

The retrieval and generation in Python:

bedrock_agent = boto3.client("bedrock-agent-runtime")

def search_docs(query, kb_id):
    response = bedrock_agent.retrieve_and_generate(
        input={"text": query},
        retrieveAndGenerateConfiguration={
            "type": "KNOWLEDGE_BASE",
            "knowledgeBaseConfiguration": {
                "knowledgeBaseId": kb_id,
                "modelArn": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-haiku-20240307-v1:0",
                "retrievalConfiguration": {
                    "vectorSearchConfiguration": {"numberOfResults": 4}
                }
            }
        }
    )
    return {
        "answer": response["output"]["text"],
        "sources": [c["retrievedReferences"] for c in response.get("citations", [])]
    }

After the fixes — Haiku for generation, semantic chunking, 4 results instead of 10 — the monthly cost dropped to $680. Same quality. Engineers couldn’t tell the difference. That experience taught me more about LLM production deployment than any documentation ever could.

Prompt Engineering That Actually Works

I’m going to skip the “be specific in your prompts” advice you’ve read a hundred times. Here’s what actually moves the needle in production Bedrock applications.

System prompts are your control plane. Every Bedrock model supports them, and they’re where you define behavior, constraints, and output format. Don’t shove this into the user message — it gets diluted by the actual query.

def query_with_system(user_input, system_prompt):
    return bedrock.invoke_model(
        modelId="anthropic.claude-3-haiku-20240307-v1:0",
        body=json.dumps({
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": 1024,
            "system": system_prompt,
            "messages": [{"role": "user", "content": user_input}]
        })
    )

For the doc search system, my system prompt was roughly: “You are a technical documentation assistant. Answer using only the provided context. If the context doesn’t contain the answer, say so. Cite document titles.” That last sentence — “cite document titles” — cut our hallucination rate in half. Models hallucinate less when they know you’re checking their work.

Temperature matters more than people think. For classification and extraction, set it to 0. For creative content, 0.7-1.0. For most production use cases — summarization, Q&A, analysis — I use 0.1. Just enough variation to avoid robotic repetition, not enough to introduce nonsense.

Guardrails — Content Filtering Done Right

Bedrock Guardrails is the feature I didn’t know I needed until a user asked our doc search system to “write a script to delete all production databases” and it happily obliged with a detailed answer sourced from our runbooks. That was a fun incident report.

Guardrails let you define content policies, denied topics, word filters, and PII detection that apply to both inputs and outputs. They run as a layer between your application and the model.

aws bedrock create-guardrail \
    --name "doc-search-guardrail" \
    --blocked-input-messaging "I can't help with that request." \
    --blocked-outputs-messaging "I can't provide that information." \
    --topic-policy-config '{
        "topicsConfig": [{
            "name": "destructive-operations",
            "definition": "Instructions for deleting, destroying, or corrupting production systems or data",
            "type": "DENY"
        }]
    }' \
    --content-policy-config '{
        "filtersConfig": [
            {"type": "VIOLENCE", "inputStrength": "HIGH", "outputStrength": "HIGH"},
            {"type": "PROMPT_ATTACK", "inputStrength": "HIGH", "outputStrength": "NONE"}
        ]
    }' \
    --sensitive-information-policy-config '{
        "piiEntitiesConfig": [
            {"type": "EMAIL", "action": "ANONYMIZE"},
            {"type": "PHONE", "action": "ANONYMIZE"}
        ]
    }'

The prompt attack filter is critical. Without it, users can jailbreak your carefully crafted system prompts with “ignore all previous instructions” style attacks. I’ve seen it happen in production. If you’re thinking about AI governance frameworks, guardrails are your first line of defense.

One gotcha: guardrails add latency. About 100-300ms per invocation in my testing. For real-time chat that’s noticeable. For async processing it’s irrelevant. Plan accordingly.

Streaming and Latency

Nobody wants to stare at a blank screen for 8 seconds while a model generates a response. Streaming is non-negotiable for user-facing applications.

def stream_response(prompt):
    response = bedrock.invoke_model_with_response_stream(
        modelId="anthropic.claude-3-haiku-20240307-v1:0",
        body=json.dumps({
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": 1024,
            "messages": [{"role": "user", "content": prompt}]
        })
    )
    for event in response["body"]:
        chunk = json.loads(event["chunk"]["bytes"])
        if chunk["type"] == "content_block_delta":
            yield chunk["delta"]["text"]

Time-to-first-token with streaming is typically 200-500ms for Haiku, 500-1200ms for Sonnet. That’s the number your users feel. Total generation time matters less because they’re reading as tokens arrive.

If you’re calling Bedrock from Lambda, watch your timeout. A complex Sonnet query can take 30+ seconds to fully generate. I set Lambda timeouts to 60 seconds minimum for any function that calls Bedrock, and I’ve written about why generous timeouts matter in Lambda cold starts.

Cost Management — The Part Nobody Talks About

AI costs sneak up on you. I’ve seen teams go from $200/month in testing to $15,000/month in production because they didn’t think about token economics.

The math is simple but the implications aren’t obvious. Input tokens cost less than output tokens. For Claude Sonnet, input is roughly $3/million tokens and output is $15/million. That 5x difference means you should optimize output length aggressively. If you need a yes/no classification, tell the model to respond with a single word. Don’t let it explain its reasoning unless you need the reasoning.

Provisioned Throughput is Bedrock’s reserved capacity model. You commit to a fixed number of model units for 1 or 6 months and get a lower per-token rate plus guaranteed throughput. It makes sense when you have predictable, high-volume workloads:

aws bedrock create-provisioned-model-throughput \
    --model-units 1 \
    --provisioned-model-name "prod-haiku" \
    --model-id anthropic.claude-3-haiku-20240307-v1:0

For the doc search system, switching to Provisioned Throughput saved us about 35% once we had consistent daily volume. But don’t commit early — use on-demand pricing until you understand your actual usage patterns.

Track your costs with CloudWatch:

aws cloudwatch get-metric-statistics \
    --namespace AWS/Bedrock \
    --metric-name InputTokenCount \
    --dimensions Name=ModelId,Value=anthropic.claude-3-haiku-20240307-v1:0 \
    --start-time 2026-04-01T00:00:00Z \
    --end-time 2026-04-30T23:59:59Z \
    --period 86400 \
    --statistics Sum

Set billing alarms. I’m serious. A runaway loop calling Sonnet can burn through thousands of dollars in hours. I’ve seen it happen. The AI anomaly detection patterns I’ve written about work just as well for detecting anomalous AI spend as they do for detecting anomalous application behavior.

What I’d Do Differently

If I were starting a Bedrock project today, here’s my playbook:

1. Start with Haiku for everything. Only upgrade models where quality genuinely fails.
2. Build guardrails from day one, not after the first incident.
3. Use Knowledge Bases for RAG instead of rolling your own. The managed chunking and embedding pipeline saves weeks.
4. Stream all user-facing responses. No exceptions.
5. Set up cost monitoring before you write a single prompt. CloudWatch alarms on token counts and spend.
6. Test with real data early. Synthetic test prompts tell you nothing about production behavior.

Bedrock isn’t perfect. The model selection is smaller than what you get calling providers directly. Some newer model versions lag behind the direct APIs by weeks. And the documentation, while improving, still has gaps that’ll send you to Stack Overflow.

But for teams building AI applications on AWS, it’s the right starting point. You get model flexibility, managed infrastructure, built-in safety controls, and native integration with the rest of your AWS stack. That combination didn’t exist two years ago.

If you’re already running workloads on AWS and thinking about adding AI capabilities, stop overthinking it. Enable model access in the Bedrock console, write a Lambda function, call the API. You’ll have a working prototype in an afternoon. Making it production-ready — with proper AI governance, cost controls, and scalable architecture — that’s where the real work starts. But at least with Bedrock, you’re building your application instead of building infrastructure.