The Wombelix Post - Cloud

Serverless RAG without monthly costs using AWS Bedrock and S3 Vectors

2025-07-30T00:00:00+02:00

I was curious if it's possible to build a Retrieval-Augmented Generation (RAG) system for an AI chatbot in a pure serverless way without monthly fixed costs. The main goal was to test some ideas without investing much in infrastructure upfront. When AWS launched S3 Vectors on July 15th, it immediately caught my attention because it promised exactly what I was looking for.

Part of the S3 Vectors announcement was that it can be used as a vector store in AWS Bedrock Knowledge Base too, which made it even more interesting. This combination could potentially solve the cost challenge I was facing with traditional vector databases that come with monthly fees regardless of usage.

S3 Vectors is currently in preview and available in five regions: US East (N. Virginia), US East (Ohio), US West (Oregon), EU Central 1 (Frankfurt), and Asia Pacific (Sydney). I picked eu-central-1, which is closest to me, for my tests and pricing calculations. I use around 500 markdown files from the Rancher Manager Documentation as my test dataset. Unlike regular S3 buckets, S3 Vectors doesn't require a globally unique name since you reference it by ARN, which means it only needs to be unique within the same account and region.

Setting up the Bedrock Knowledge Base was overall pretty straightforward. The wizard guides you through all the steps, though I created the S3 bucket and S3 Vectors bucket upfront. I had the Bedrock Knowledge Base documentation and S3 Vectors documentation open in parallel to learn more and make decisions about parsing strategy, chunking, vector dimensions and configuration options. The knowledge base, data source, and vector store all need to be in the same region, which makes sense from performance and traffic costs perspective.

I chose the Amazon Bedrock default parser as my parsing strategy. This parser claims to work well with various file formats including markdown, HTML, PDF, and Office documents. The main advantage is that it doesn't incur additional charges, making it perfect for projects where you want to keep costs low. Since my content was primarily text-based markdown files, the default parser seems more than sufficient for this use case.

For the vector configuration, I used a 1024-dimension index aligned with the Amazon Titan Text Embeddings V2 model. I also went with the default chunking strategy, which splits content into approximately 300-token chunks while preserving sentence boundaries. I want to play around with this another time and see if it would make sense and bring an improvement raising the chunk size. The embedding model can handle up to 8192 tokens.

The data flow is now: From the S3 source bucket through the AWS Bedrock parser, processed by the embedding model, stored in the vector storage. Testing the result and interacting with the synced data is easy using the Chat with your document feature in the Amazon Bedrock console. So, to see first results, you don't have to build out your Chatbot interface yet.

Several things became clear during my test setup. The overall process is simpler as expected, with the wizard handling resource creation and permissions. Since S3 Vectors is in preview, it's not yet available in infrastructure-as-code tools like the AWS Terraform provider. I didn't check if it's available in CloudFormation though. I would love to manage the knowledge base and related resources through IaC next time instead of the click-ops through the console.

The cost aspect, which was my primary motivation, exceeded expectations. With S3 Vectors, we're talking about the cost of a cup of coffee when testing or building a proof of concept. This opens up possibilities for builders who want to experiment with ideas without upfront investment.

One challenge I encountered was with permissions. The Knowledge Base wizard is strict and defaults to claiming ownership of the entire bucket, at least from an IAM role perspective. The managed roles don't cover individual subfolders / prefixes someone might create when uploading data. Manual IAM adjustments work but can cause issues when editing the knowledge base later. There's likely a better way to configure this properly from the start. I have to figure out if the answer is really one source S3 bucket per knowledge base or if there's a better way to leverage prefixes without the clunky IAM role and policy handling.

I'm also curious about further reducing costs for the S3 data source, which could grow over time. S3 Intelligent-Tiering might help reduce ongoing storage costs. The knowledge base index can be recreated from the source data in the S3 bucket, though that will cause costs for using the embedding model again.

Embedding tokens are another cost factor, though not a major one. My test with approximately 5MB of Rancher documentation in markdown format resulted in about 1 million tokens, translating to roughly $0.02 USD with Amazon Titan Text Embeddings V2, which doesn't seem like much. Using batch processing outside of Bedrock could potentially cut costs in half, but batch processing doesn't work in the context of Bedrock Knowledge Base. The knowledge base is convenient to use, but if you build your own stack instead, then batching can become a cost saver, especially at a higher scale beyond simple testing.

I ran into an issue with metadata handling. Using S3 Vectors as vector storage with Bedrock Knowledge Base requires adding the Bedrock-related metadata keys as non-filterable, otherwise the sync fails. By default, all metadata keys in S3 Vectors are considered filterable, but Bedrock's metadata exceeds the 2KB limit for filterable metadata. I found the solution in the S3 Vectors documentation and this AWS re:Post discussion: create the vector index with AMAZON_BEDROCK_TEXT and AMAZON_BEDROCK_METADATA as non-filterable metadata keys.

Additional (slight) costs occur when interacting with the knowledge base, for the input and output tokens of the leveraged LLM through AWS Bedrock. For example, something between $0.05 and $0.10 USD per 1 Million Tokens when using one of the Amazon Nova Models. Based on the S3 Vectors pricing for eu-central-1, storage costs $0.064 per GB per month, PUT requests cost $0.214 per GB, and query requests are $0.0027 per 1,000 requests. For small datasets like my 5MB test, these costs remain minimal.

With my tests I achieved what I wanted. Vector storage was previously the component with higher monthly costs in Bedrock Knowledge Base, but S3 Vectors makes this purely pay-per-use. Now we're talking about costs comparable to a cup of coffee for testing and building MVPs, just perfect for experimentation and idea validation. Even though I didn't run the numbers, I wouldn't be surprised if this is even up to a specific scale extremely interesting and cost efficient.

OpenTofu State and Plan Encryption with AWS KMS

2024-12-22T00:00:00+01:00

Its been a while since OpenTofu 1.7.0 (Archive: [1], [2]) introduced an exciting new feature, the State and Plan Encryption (Archive: [1], [2]) for files at rest, for local storage and remote backends. I always found it challenging to keep .tfstate files secure. Now I can use Amazon Web Servers Key Management Service (AWS KMS) with a customer managed KMS key to encrypt the state before it's uploaded to Amazon Simple Storage Service (Amazon S3).

I decided to use CloudFormation to create a KMS key in eu-central-1, Europe (Frankfurt) and a replica in eu-west-1, Europe (Ireland) for backup purposes. Other AWS resources I've created: IAM User, IAM Roles, IAM Policies, S3 Bucket, DynamoDB Table and AWS Lambda function.. The Lambda function is based on CloudFormation Custom Resource AWS SSM Parameter Store SecureString and is necessary because of AWS CloudFormation and CDK doesn't support AWS SSM Parameter Store SecureString?!. Access and Secret Key for the IAM User are then auto-generated by CloudFormation and stored in AWS SSM Parameter Store.

When the AWS resource are ready, an example config to use state encryption and a S3 based remote backed looks like this:

# SPDX-FileCopyrightText: 2024 Dominik Wombacher <dominik@wombacher.cc>
#
# SPDX-License-Identifier: MIT

terraform {
  required_version = ">= 1.8"
  encryption {
    key_provider "aws_kms" "wombelix-sideprojects" {
      kms_key_id = "arn:${var.aws_partition}:kms:${var.aws_region}:${var.aws_account_id}:key/${var.aws_kms_name}"
      region     = var.aws_region
      key_spec   = "AES_256"
      assume_role = {
        role_arn = "arn:${var.aws_partition}:iam::${var.aws_account_id}:role/OpenTofuStateEncryptionRole"
      }
    }
    method "aes_gcm" "wombelix-sideprojects" {
      keys = key_provider.aws_kms.wombelix-sideprojects
    }
    state {
      method = method.aes_gcm.wombelix-sideprojects
    }
  }
  backend "s3" {
    bucket                  = var.aws_s3_bucket
    key                     = "opentofu-states/${var.project}/terraform.tfstate"
    region                  = var.aws_region
    skip_metadata_api_check = true
    encrypt                 = true
    kms_key_id              = "arn:${var.aws_partition}:kms:${var.aws_region}:${var.aws_account_id}:key/${var.aws_kms_name}"
    dynamodb_table          = "arn:${var.aws_partition}:dynamodb:${var.aws_region}:${var.aws_account_id}:table/iac-opentofu-remote-backend"
    assume_role = {
      role_arn = "arn:${var.aws_partition}:iam::${var.aws_account_id}:role/OpenTofuRemoteBackendRole"
    }
  }
}

For the above example config to interact with AWS, the following Environment variables have to be set:

AWS_REGION
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
TF_VAR_aws_region
TF_VAR_aws_account_id
TF_VAR_aws_kms_name
TF_VAR_aws_s3_bucket
TF_VAR_project

After bootstrapping with CloudFormation, all subsequent IaC can be implemented with OpenTofu. Each project gets its own unique S3 key in the form of opentofu-states/<PROJECT>/terraform.tfstate. That's all the customization it needs, which makes the solution basically maintenance free.

I like that the content of the state file is always encrypted. I also encrypt the S3 Bucket with the the customer managed key. When I use OpenTofu on my local system or in sr.ht builds, I leverage long-term Access and Secret key credentials. But the IAM User has no permissions directly assigned and can only assume a specific Role that allows access to KMS, S3 and DynamoDB. s3:DeleteObject is explicitly set to Deny and versioning enabled on the S3 Bucket. The potential attack surface is therefore very limited. In future I plan to avoid any usage of long-term credentials.

But for now the setup is already pretty decent and secure.

AWS CloudFormation and CDK doesn't support AWS SSM Parameter Store SecureString?!

2024-05-12T00:00:00+02:00

I recently started to set up some resources on AWS for my side projects. For starters an AWS KMS key so I can encrypt data on S3 and in the AWS SSM Parameter Store. To use S3 and DynamoDB as backend and perform end-to-end state encryption for OpenTofu, I also needed an IAM User. So the Idea was to write a CloudFormation template that creates all these resources for me and then use it to deploy other Infrastructure as code via OpenTofu. I'm not a huge fan of IAM Users and access keys, but in this case good enough to get started.

What I wanted: The generated access and secret key are stored in AWS SSM Parameter store. That way I don't have to deal with clear text credentials in CloudFormation.

SSM Parameter Store can save Strings and SecureStrings. As the name implies, a SecureString is encrypted via AWS KMS before put into SSM Parameter Store. But then I learned, neither Cfn nor CDK support it. They can only write clear text Strings to the Parameter Store. What a bummer and pretty unexpected.

So after some research, a Cfn CustomResource is what I need. It's basically a Lambda function that receives a Create/Update/Delete request from Cfn, performs an action and sends the result back to the Stack. It took me a bit to get something together but now it works like a charm.

I'm still a bit disappointed that such a common feature isn't supported. Arguments are mostly that Cfn and CDK are not supposed to deal with secrets. I can understand that, but putting some data that were generated during a Cfn run into the parameter store can't be that unique.

I published my Lambda Function to interact with AWS SSM Parameter Store SecureString under MIT: https://git.sr.ht/~wombelix/cfn-custom-resource-aws-ssm-securestring