LLM Classification Configuration

Astralis' Data Discovery module includes an LLM-powered data classifier that leverages large language models to automatically identify and categorize sensitive data in your systems. This guide explains how to configure AWS Bedrock as the LLM backend for Astralis' data classification capabilities.

Overview

The LLM classifier in Astralis can use a variety of LLM providers to support intelligent, context-aware data categorization.

If you are running Astralis on AWS, you may want to use AWS Bedrock as the LLM provider for Astralis' data classifier.

AWS Bedrock provides access to foundation models from leading AI companies, enabling sophisticated natural language processing for data classification tasks.

Prerequisites

Before configuring AWS Bedrock for Astralis, ensure you have:

• An Astralis deployment running on AWS EKS or AWS ECS (Astralis 2.94.0 or later if you plan to use the optional Gemma 4 31B model through Bedrock Mantle)
• AWS Bedrock enabled in your AWS account with appropriate model access
• Permissions to create and manage IAM roles and policies
• Access to update Astralis configuration settings
• Astralis OAuth access token with config:update scope

Authentication Methods

Astralis supports multiple authentication methods for connecting to AWS Bedrock.

Recommended: IAM Role-Based Authentication

For production deployments, Ethyca recommends using IAM role-based authentication to eliminate the need for long-term credentials:

EKS Deployments: IRSA (IAM Roles for Service Accounts)

If Astralis is deployed on Amazon EKS, use IRSA (see AWS user guide (opens in a new tab)) to enable secure, credential-less authentication.

Step 1: Create an IAM Policy for Bedrock Access

Create an IAM policy that grants the minimum required permissions to invoke Bedrock models:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "InvokeBedrockFoundationModels",
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      "Resource": "arn:aws:bedrock:*:*:foundation-model/*"
    },
    {
      "Sid": "InvokeBedrockInferenceProfiles",
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      "Resource": "arn:aws:bedrock:*:*:inference-profile/*"
    },
    {
      "Sid": "InvokeMantleDefaultProject",
      "Effect": "Allow",
      "Action": [
        "bedrock-mantle:CreateInference"
      ],
      "Resource": "arn:aws:bedrock-mantle:*:ACCOUNT_ID:project/default"
    }
  ]
}

Using the AWS CLI:

aws iam create-policy \
  --policy-name bedrock-invoke-policy \
  --policy-document file://bedrock-policy.json

Step 2: Set Up OIDC Provider and Service Account

Associate an OIDC provider with your EKS cluster (one-time setup):

# Set your cluster details
CLUSTER_NAME="your-cluster-name"
AWS_REGION="us-east-1"
NAMESPACE="fides"
SERVICE_ACCOUNT="fides-bedrock-sa"
 
# Associate OIDC provider with the cluster
eksctl utils associate-iam-oidc-provider \
  --region=$AWS_REGION \
  --cluster=$CLUSTER_NAME \
  --approve

Step 3: Create the IAM Service Account

Create a Kubernetes ServiceAccount linked to an IAM role:

# Replace with your policy ARN from Step 1
POLICY_ARN="arn:aws:iam::ACCOUNT_ID:policy/bedrock-invoke-policy"
 
# Create the service account and IAM role
eksctl create iamserviceaccount \
  --name=$SERVICE_ACCOUNT \
  --namespace=$NAMESPACE \
  --cluster=$CLUSTER_NAME \
  --region=$AWS_REGION \
  --attach-policy-arn=$POLICY_ARN \
  --approve \
  --override-existing-serviceaccounts

Step 4: Update Your Astralis Deployment

Configure your Astralis deployment to use the service account:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fides
  namespace: fides
spec:
  template:
    spec:
      serviceAccountName: fides-bedrock-sa  # Reference the service account
      containers:
      - name: fides
        env:
        - name: AWS_REGION
          value: "us-east-1"  # Set to your AWS region

ECS Deployments: Task IAM Roles

If Astralis is deployed on Amazon ECS, use ECS Task IAM Roles for authentication:

Step 1: Create an IAM Policy

Use the same IAM policy JSON from the EKS section above to create a policy with Bedrock permissions.

Step 2: Create an ECS Task Role

Create an IAM role with a trust policy that allows ECS tasks to assume it:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ecs-tasks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Attach the Bedrock policy to the task role:

aws iam attach-role-policy \
  --role-name fides-ecs-task-role \
  --policy-arn arn:aws:iam::ACCOUNT_ID:policy/bedrock-invoke-policy

Step 3: Update Your ECS Task Definition

Configure your ECS task definition to use the task role:

{
  "family": "fides-task",
  "taskRoleArn": "arn:aws:iam::ACCOUNT_ID:role/fides-ecs-task-role",
  "containerDefinitions": [
    {
      "name": "fides",
      "environment": [
        {
          "name": "AWS_REGION",
          "value": "us-east-1"
        }
      ]
    }
  ]
}

Alternative: Long-Term API Token (Not Recommended for Production)

For development or testing environments, you can use long-term Bedrock API tokens. However, this approach is not recommended for production due to security concerns around credential management and rotation.

To use long-term API tokens, create a long-term Bedrock API token, and then set the following environment variable on your deployed Astralis containers:

AWS_BEARER_TOKEN_BEDROCK="your-bedrock-api-token"

Testing the Bedrock Connection

Once you've configured authentication, verify that Astralis can successfully connect to AWS Bedrock.

Using the Astralis API

Test the connection by making a POST request to the Bedrock test endpoint:

curl -X POST '{{FIDES_URL}}/api/v1/plus/connection/aws-bedrock/test' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer {{FIDES_ACCESS_TOKEN}}' \
  -d '{
    "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0",
    "aws_region": "us-east-1"
  }'

In the above example:

• {{FIDES_URL}} is the URL to your Astralis server
• {{FIDES_ACCESS_TOKEN}} is your Astralis OAuth access token
• model specifies the Bedrock model to test (use any model you have access to; for example bedrock_mantle/google.gemma-4-31b to test the Mantle setup for Gemma 4 31B)
• aws_region is the AWS region where Bedrock is enabled. For Bedrock Mantle models this must be one of the Mantle regions

Successful Response

If the connection is configured correctly, you'll receive a success response:

HTTP/1.1 200 OK
Content-Type: application/json
 
{
  "success": true,
  "message": "AWS Bedrock connection test completed successfully",
  "response_content": "...",
  "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0"
}

Troubleshooting Connection Issues

If the connection test fails, check the following:

Authentication Issues

• Verify that the IAM policy is correctly attached to the role
• Ensure the trust policy allows the service account or ECS task to assume the role
• Check that the service account annotation (EKS) or task role ARN (ECS) is correctly configured

Permission Issues

• Confirm that Bedrock is enabled in your AWS account
• Verify you have requested access to the specific foundation models you're trying to use
• Review AWS CloudTrail logs for AssumeRoleWithWebIdentity (EKS) or IAM role assumption events (ECS)

Network Issues

• Ensure your Astralis deployment has outbound internet access or VPC endpoints for Bedrock
• Verify security groups allow HTTPS (port 443) outbound traffic
• For private deployments, configure VPC endpoints for Bedrock and STS services

Check Astralis Logs

View the Astralis pod or container logs for detailed error messages:

Supported Bedrock Models

Astralis supports any AWS Bedrock foundation model or inference profile that provides text generation capabilities. A Bedrock deployment must set the "Model override" field on every monitor that uses the LLM classifier; a monitor with no override does not use your AWS account's Bedrock models.

Ethyca recommends two models for the LLM classifier on Bedrock. Both are benchmarked by Ethyca at equivalent overall accuracy with batched classification, and both complete a full classification run in a similar amount of time. Claude Haiku 4.5 is the simplest choice and is available in every Bedrock region. If your deployment can use Bedrock Mantle, Gemma 4 31B is worth considering: it costs roughly a tenth as much per token for the same accuracy. Bedrock Mantle is a newer AWS endpoint serving fewer regions, and AWS ramps its throughput gradually for an account that has not used it before, so move one monitor across first and confirm its classification runs before switching others.

Anthropic Claude Haiku 4.5 (recommended)

• Model override: bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
• Requires: model access requested in the Bedrock console. The standard IAM policy above is sufficient.
• Why: the simplest setup, and available in every Bedrock region.

Google Gemma 4 31B (lower cost)

• Model override: bedrock_mantle/google.gemma-4-31b
• Requires: Astralis 2.94.0 or later, the InvokeMantleDefaultProject IAM statement, and a Mantle region. See Using Gemma 4 31B through Bedrock Mantle.
• Why: roughly a tenth of Claude Haiku 4.5's per-token price at equivalent classification accuracy. Gemma 4 is not served through classic Bedrock, so Bedrock Mantle is the only way to use it.

If neither model is available for use in your organization, please contact Ethyca support to determine a viable alternative.

Using Gemma 4 31B through Bedrock Mantle

Gemma 4 models are served only through the Bedrock Mantle endpoint (https://bedrock-mantle.<region>.api.aws/openai/v1), which is a separate IAM service namespace from classic Bedrock. To use Gemma 4 31B:

1. Run Astralis 2.94.0 or later.
2. Include the InvokeMantleDefaultProject statement from the IAM policy above in the role Astralis uses.
3. Deploy in a Mantle region (see below) and set the model override to bedrock_mantle/google.gemma-4-31b.

Bedrock Mantle regions

Mantle serves Gemma 4 in-region only, with no cross-region inference profiles. At the time of writing it is available in us-east-1, us-east-2, us-west-2 and eu-central-1; check the Gemma 4 31B model card (opens in a new tab) for the current list. Astralis selects the region from the AWS_REGION environment variable on its containers (set automatically by IRSA on EKS), so make sure it names a Mantle region.

Batched classification vs. batch inference

"Batch" means two different things in this area, and only one of them is an Astralis feature.

Batched classification is how Astralis composes its prompts. It asks about many of a table's fields in a single request, instead of sending one request per field. Those requests are ordinary real-time inference calls, several of them running at once, and the answers come back while the monitor is still running.

Batch inference is a delivery mode that model providers offer, including AWS Bedrock batch inference (opens in a new tab). You hand the provider a file of independent requests, the provider works through them offline, and you collect the results later — Bedrock reads and writes those files in Amazon S3, with a turnaround measured in hours. Other model providers offer equivalent asynchronous batch APIs.

Astralis does not use provider batch inference APIs for classification, so there is no batch job to configure, no S3 bucket to provision, and no bedrock:CreateModelInvocationJob permission to grant. Enabling batch inference in your AWS account changes nothing about how classification runs, and the provider's batch discount does not apply to it. The savings from batched classification come from sending far fewer, larger requests.

Additional Resources

• AWS Bedrock Documentation (opens in a new tab)
• AWS EKS IRSA Documentation (opens in a new tab)
• AWS ECS Task IAM Roles Documentation (opens in a new tab)
• Astralis Discovery Monitor
• Astralis Configuration Variables