Use secrets manager for api key

2025-02-08 21:36:59 +08:00
parent 48bf360456
commit 4d88731233
4 changed files with 69 additions and 389 deletions
--- a/README.md
+++ b/README.md
@@ -6,24 +6,7 @@ OpenAI-compatible RESTful APIs for Amazon Bedrock

 ## Breaking Changes

-This solution can now **automatically detect** new models supported in Amazon Bedrock. 
-So whenever new models are added to Amazon Bedrock, you can immediately try them without the need to wait for code changes to this repo. 
-
-This is to use the `ListFoundationModels` api and the `ListInferenceProfiles` api by Amazon Bedrock, due to this change, additional IAM permissions are required to your Lambda/Fargate role.
-
-If you are facing error: 'Unsupported model xxx, please use models API to get a list of supported models' even the model ID is correct, 
-please either update your existing stack (**Recommended**) with the new template in the deployment folder or manually add below permissions to the related Lambda/Fargate role.
-
-```json
-{
-   "Action": [
-       "bedrock:ListFoundationModels",
-       "bedrock:ListInferenceProfiles"
-   ],
-   "Resource": "*",
-   "Effect": "Allow"
-}
-```
+This solution now uses Secrets Manager to maintain API Key for security best practice.  You **MUST** create the API Key first in Secrets Manager and rotate it frequently.

 Please raise an GitHub issue if you still have problems.

@@ -74,42 +57,38 @@ Alternatively, you can use Lambda Function URL to replace ALB, see [example](htt

 Please follow the steps below to deploy the Bedrock Proxy APIs into your AWS account. Only supports regions where Amazon Bedrock is available (such as `us-west-2`). The deployment will take approximately **3-5 minutes** 🕒.

-**Step 1: Create your own custom API key (Optional)**
+**Step 1: Create your own API key in Secrets Manager (MUST)**

-#### Store API Key in ParameterStore

-> **Note:** This step is to use any string (without spaces) you like to create a custom API Key (credential) that will be used to access the proxy API later. This key does not have to match your actual OpenAI key, and you don't need to have an OpenAI API key. It is recommended that you take this step and ensure that you keep the key safe and private.
+> **Note:** This step is to use any string (without spaces) you like to create a custom API Key (credential) that will be used to access the proxy API later. This key does not have to match your actual OpenAI key, and you don't need to have an OpenAI API key. please keep the key safe and private.

-1. Open the AWS Management Console and navigate to the Systems Manager service.
-2. In the left-hand navigation pane, click on "Parameter Store".
-3. Click on the "Create parameter" button.
-4. In the "Create parameter" window, select the following options:
-    - Name: Enter a descriptive name for your parameter (e.g., "BedrockProxyAPIKey").
-    - Description: Optionally, provide a description for the parameter.
-    - Tier: Select **Standard**.
-    - Type: Select **SecureString**.
-    - Value: Any string (without spaces).
-5. Click "Create parameter".
-6. Make a note of the parameter name you used (e.g., "BedrockProxyAPIKey"). You'll need this in the next step.
+1. Open the AWS Management Console and navigate to the AWS Secrets Manager service.
+2. Click on "Store a new secret" button. 
+3. In the "Choose secret type" page, select:

-#### Store API Key in ENV variable
+   Secret type: Other type of secret
+   Key/value pairs:
+   - Key: api_key
+   - Value: Enter your API key value
   
-1. Provide an ENV variable to the container named: `API_KEY` with the API key value.
+   Click "Next"
+4. In the "Configure secret" page:
+   Secret name: Enter a name (e.g., "BedrockProxyAPIKey")
+   Description: (Optional) Add a description of your secret
+5. Click "Next" and review all your settings and click "Store"

 **Step 2: Deploy the CloudFormation stack**

 1. Sign in to AWS Management Console, switch to the region to deploy the CloudFormation Stack to.
 2. Click the following button to launch the CloudFormation Stack in that region. Choose one of the following:
-   - **ALB + Lambda**

-      [![Launch Stack](assets/launch-stack.png)](https://console.aws.amazon.com/cloudformation/home#/stacks/create/template?stackName=BedrockProxyAPI&templateURL=https://aws-gcr-solutions.s3.amazonaws.com/bedrock-access-gateway/latest/BedrockProxy.template)
   - **ALB + Fargate**

      [![Launch Stack](assets/launch-stack.png)](https://console.aws.amazon.com/cloudformation/home#/stacks/create/template?stackName=BedrockProxyAPI&templateURL=https://aws-gcr-solutions.s3.amazonaws.com/bedrock-access-gateway/latest/BedrockProxyFargate.template)
 3. Click "Next".
 4. On the "Specify stack details" page, provide the following information:
    - Stack name: Change the stack name if needed.
-    - ApiKeyParam (if you set up an API key in Step 1): Enter the parameter name you used for storing the API key (e.g., `BedrockProxyAPIKey`). If you did not set up an API key, leave this field blank. Click "Next".
+    - ApiKeySecretName: Enter the secret name you used for storing the API key (e.g., `BedrockProxyAPIKey`). Click "Next".
 5. On the "Configure stack options" page, you can leave the default settings or customize them according to your needs.
 6. Click "Next".
 7. On the "Review" page, review the details of the stack you're about to create. Check the "I acknowledge that AWS CloudFormation might create IAM resources" checkbox at the bottom.
--- a/README_CN.md
+++ b/README_CN.md
@@ -6,24 +6,7 @@

 ## 重大变更

-这个方案现在可以**自动检测** Amazon Bedrock 中支持的新模型。
-因此，当 Amazon Bedrock 添加新模型时，您可以立即尝试使用它们，无需等待此代码库的更新。
-
-这是通过使用Amazon Bedrock 的 `ListFoundationModels API` 和 `ListInferenceProfiles` API 实现的。由于这一变更，您需要为 Lambda/Fargate 角色添加额外的 IAM 权限。
-
-如果您遇到错误："Unsupported model xxx, please use models API to get a list of supported models"（即使Model ID 是正确的），
-请使用Deployment 文件夹中的新模板更新您现有的堆栈(**推荐**），或手动为相关的 Lambda/Fargate 角色添加以下权限。
-
-```json
-{
-   "Action": [
-       "bedrock:ListFoundationModels",
-       "bedrock:ListInferenceProfiles"
-   ],
-   "Resource": "*",
-   "Effect": "Allow"
-}
-```
+为了遵循安全最佳实践，本解决方案现使用 Secrets Manager 来管理 API 密钥。您必须先在 Secrets Manager 中创建 API 密钥，并定期轮换该密钥。

 如果依然有问题，请提个GitHub issue。

@@ -76,37 +59,36 @@ OpenAI 的 API 或 SDK 无缝集成并试用 Amazon Bedrock 的模型,而无需

 请按以下步骤将Bedrock代理API部署到您的AWS账户中。仅支持Amazon Bedrock可用的区域(如us-west-2)。 部署预计用时**3-5分钟** 🕒。

-**第一步: 自定义您的API Key (可选)**
+**第一步: 在 Secrets Manager 中创建您的 API 密钥（必须）**

 > 注意:这一步是使用任意字符串（不带空格）创建一个自定义的API Key(凭证),将用于后续访问代理API。此API Key不必与您实际的OpenAI
-> Key一致,您甚至无需拥有OpenAI API Key。建议您执行此步操作并且请确保保管好此API Key。
+> Key一致,您甚至无需拥有OpenAI API Key。请确保保管好此API Key。
+
+1. 打开 AWS 管理控制台并导航至 AWS Secrets Manager 服务。
+2. 点击 "存储新密钥" 按钮。
+3. 在 "选择密钥类型" 页面，选择：
+
+   密钥类型：其他类型的密钥 键/值对：
+   
+   - 键：api_key
+   - 值：输入您的 API 密钥值
+   点击 "下一步"
+4. 在 "配置密钥" 页面： 密钥名称：输入一个名称（例如："BedrockProxyAPIKey"） 描述：（可选）添加密钥的描述
+5. 点击 "下一步"，检查所有设置后点击 "存储"

-1. 打开AWS管理控制台,导航到Systems Manager服务。
-2. 在左侧导航窗格中,单击"参数存储"。
-3. 单击"创建参数"按钮。
-4. 在"创建参数"窗口中,选择以下选项:
-    - 名称:输入参数的描述性名称(例如"BedrockProxyAPIKey")。
-    - 描述:可选,为参数提供描述。
-    - 层级:选择**标准**。
-    - 类型:选择**SecureString**。
-    - 值: 随意字符串（不带空格）。
-5. 单击"创建参数"。
-6. 记录您使用的参数名称(例如"BedrockProxyAPIKey")。您将在下一步中需要它。

 **第二步: 部署CloudFormation堆栈**

 1. 登录AWS管理控制台,切换到要部署CloudFormation堆栈的区域。
 2. 单击以下按钮在该区域启动CloudFormation堆栈，选择一种方式部署。
-   - **ALB + Lambda**

-      [![Launch Stack](assets/launch-stack.png)](https://console.aws.amazon.com/cloudformation/home#/stacks/create/template?stackName=BedrockProxyAPI&templateURL=https://aws-gcr-solutions.s3.amazonaws.com/bedrock-access-gateway/latest/BedrockProxy.template)
   - **ALB + Fargate**

      [![Launch Stack](assets/launch-stack.png)](https://console.aws.amazon.com/cloudformation/home#/stacks/create/template?stackName=BedrockProxyAPI&templateURL=https://aws-gcr-solutions.s3.amazonaws.com/bedrock-access-gateway/latest/BedrockProxyFargate.template)
 3. 单击"下一步"。
 4. 在"指定堆栈详细信息"页面,提供以下信息:
    - 堆栈名称: 可以根据需要更改名称。
-    - ApiKeyParam(如果在步骤1中设置了API Key):输入您用于存储API密钥的参数名称(例如"BedrockProxyAPIKey")，否则,请将此字段留空。
+    - ApiKeySecretName:输入您用于存储API 密钥的名称(例如"BedrockProxyAPIKey")，否则,请将此字段留空。
      单击"下一步"。
 5. 在"配置堆栈选项"页面,您可以保留默认设置或根据需要进行自定义。
 6. 单击"下一步"。
--- a/deployment/BedrockProxy.template
+++ b/deployment/BedrockProxy.template
@@ -1,295 +0,0 @@
-Description: Bedrock Access Gateway - OpenAI-compatible RESTful APIs for Amazon Bedrock
-Transform: AWS::LanguageExtensions
-Parameters:
-  ApiKeyParam:
-    Type: String
-    Default: ""
-    Description: The parameter name in System Manager used to store the API Key, leave blank to use a default key
-Resources:
-  VPCB9E5F0B4:
-    Type: AWS::EC2::VPC
-    Properties:
-      CidrBlock: 10.250.0.0/16
-      EnableDnsHostnames: true
-      EnableDnsSupport: true
-      InstanceTenancy: default
-      Tags:
-        - Key: Name
-          Value: BedrockProxy/VPC
-  VPCPublicSubnet1SubnetB4246D30:
-    Type: AWS::EC2::Subnet
-    Properties:
-      AvailabilityZone:
-        Fn::Select:
-          - 0
-          - Fn::GetAZs: ""
-      CidrBlock: 10.250.0.0/24
-      MapPublicIpOnLaunch: true
-      Tags:
-        - Key: aws-cdk:subnet-name
-          Value: Public
-        - Key: aws-cdk:subnet-type
-          Value: Public
-        - Key: Name
-          Value: BedrockProxy/VPC/PublicSubnet1
-      VpcId:
-        Ref: VPCB9E5F0B4
-  VPCPublicSubnet1RouteTableFEE4B781:
-    Type: AWS::EC2::RouteTable
-    Properties:
-      Tags:
-        - Key: Name
-          Value: BedrockProxy/VPC/PublicSubnet1
-      VpcId:
-        Ref: VPCB9E5F0B4
-  VPCPublicSubnet1RouteTableAssociation0B0896DC:
-    Type: AWS::EC2::SubnetRouteTableAssociation
-    Properties:
-      RouteTableId:
-        Ref: VPCPublicSubnet1RouteTableFEE4B781
-      SubnetId:
-        Ref: VPCPublicSubnet1SubnetB4246D30
-  VPCPublicSubnet1DefaultRoute91CEF279:
-    Type: AWS::EC2::Route
-    Properties:
-      DestinationCidrBlock: 0.0.0.0/0
-      GatewayId:
-        Ref: VPCIGWB7E252D3
-      RouteTableId:
-        Ref: VPCPublicSubnet1RouteTableFEE4B781
-    DependsOn:
-      - VPCVPCGW99B986DC
-  VPCPublicSubnet2Subnet74179F39:
-    Type: AWS::EC2::Subnet
-    Properties:
-      AvailabilityZone:
-        Fn::Select:
-          - 1
-          - Fn::GetAZs: ""
-      CidrBlock: 10.250.1.0/24
-      MapPublicIpOnLaunch: true
-      Tags:
-        - Key: aws-cdk:subnet-name
-          Value: Public
-        - Key: aws-cdk:subnet-type
-          Value: Public
-        - Key: Name
-          Value: BedrockProxy/VPC/PublicSubnet2
-      VpcId:
-        Ref: VPCB9E5F0B4
-  VPCPublicSubnet2RouteTable6F1A15F1:
-    Type: AWS::EC2::RouteTable
-    Properties:
-      Tags:
-        - Key: Name
-          Value: BedrockProxy/VPC/PublicSubnet2
-      VpcId:
-        Ref: VPCB9E5F0B4
-  VPCPublicSubnet2RouteTableAssociation5A808732:
-    Type: AWS::EC2::SubnetRouteTableAssociation
-    Properties:
-      RouteTableId:
-        Ref: VPCPublicSubnet2RouteTable6F1A15F1
-      SubnetId:
-        Ref: VPCPublicSubnet2Subnet74179F39
-  VPCPublicSubnet2DefaultRouteB7481BBA:
-    Type: AWS::EC2::Route
-    Properties:
-      DestinationCidrBlock: 0.0.0.0/0
-      GatewayId:
-        Ref: VPCIGWB7E252D3
-      RouteTableId:
-        Ref: VPCPublicSubnet2RouteTable6F1A15F1
-    DependsOn:
-      - VPCVPCGW99B986DC
-  VPCIGWB7E252D3:
-    Type: AWS::EC2::InternetGateway
-    Properties:
-      Tags:
-        - Key: Name
-          Value: BedrockProxy/VPC
-  VPCVPCGW99B986DC:
-    Type: AWS::EC2::VPCGatewayAttachment
-    Properties:
-      InternetGatewayId:
-        Ref: VPCIGWB7E252D3
-      VpcId:
-        Ref: VPCB9E5F0B4
-  ProxyApiHandlerServiceRoleBE71BFB1:
-    Type: AWS::IAM::Role
-    Properties:
-      AssumeRolePolicyDocument:
-        Statement:
-          - Action: sts:AssumeRole
-            Effect: Allow
-            Principal:
-              Service: lambda.amazonaws.com
-        Version: "2012-10-17"
-      ManagedPolicyArns:
-        - Fn::Join:
-            - ""
-            - - "arn:"
-              - Ref: AWS::Partition
-              - :iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
-  ProxyApiHandlerServiceRoleDefaultPolicy86681202:
-    Type: AWS::IAM::Policy
-    Properties:
-      PolicyDocument:
-        Statement:
-          - Action:
-              - bedrock:ListFoundationModels
-              - bedrock:ListInferenceProfiles
-            Effect: Allow
-            Resource: "*"
-          - Action:
-              - bedrock:InvokeModel
-              - bedrock:InvokeModelWithResponseStream
-            Effect: Allow
-            Resource:
-              - arn:aws:bedrock:*::foundation-model/*
-              - arn:aws:bedrock:*:*:inference-profile/*
-          - Action:
-              - ssm:DescribeParameters
-              - ssm:GetParameters
-              - ssm:GetParameter
-              - ssm:GetParameterHistory
-            Effect: Allow
-            Resource:
-              Fn::Join:
-                - ""
-                - - "arn:"
-                  - Ref: AWS::Partition
-                  - ":ssm:"
-                  - Ref: AWS::Region
-                  - ":"
-                  - Ref: AWS::AccountId
-                  - :parameter/
-                  - Ref: ApiKeyParam
-        Version: "2012-10-17"
-      PolicyName: ProxyApiHandlerServiceRoleDefaultPolicy86681202
-      Roles:
-        - Ref: ProxyApiHandlerServiceRoleBE71BFB1
-  ProxyApiHandlerEC15A492:
-    Type: AWS::Lambda::Function
-    Properties:
-      Architectures:
-        - arm64
-      Code:
-        ImageUri:
-          Fn::Join:
-            - ""
-            - - 366590864501.dkr.ecr.
-              - Ref: AWS::Region
-              - "."
-              - Ref: AWS::URLSuffix
-              - /bedrock-proxy-api:latest
-      Description: Bedrock Proxy API Handler
-      Environment:
-        Variables:
-          API_KEY_PARAM_NAME:
-            Ref: ApiKeyParam
-          DEBUG: "false"
-          DEFAULT_MODEL:
-            Fn::FindInMap:
-              - ProxyRegionTable03E5BEB3
-              - Ref: AWS::Region
-              - model
-              - DefaultValue: anthropic.claude-3-sonnet-20240229-v1:0
-          DEFAULT_EMBEDDING_MODEL: cohere.embed-multilingual-v3
-          ENABLE_CROSS_REGION_INFERENCE: "true"
-      MemorySize: 1024
-      PackageType: Image
-      Role:
-        Fn::GetAtt:
-          - ProxyApiHandlerServiceRoleBE71BFB1
-          - Arn
-      Timeout: 300
-    DependsOn:
-      - ProxyApiHandlerServiceRoleDefaultPolicy86681202
-      - ProxyApiHandlerServiceRoleBE71BFB1
-  ProxyApiHandlerInvoke2UTWxhlfyqbT5FTn5jvgbLgjFfJwzswGk55DU1HYF6C33779:
-    Type: AWS::Lambda::Permission
-    Properties:
-      Action: lambda:InvokeFunction
-      FunctionName:
-        Fn::GetAtt:
-          - ProxyApiHandlerEC15A492
-          - Arn
-      Principal: elasticloadbalancing.amazonaws.com
-  ProxyALB87756780:
-    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
-    Properties:
-      LoadBalancerAttributes:
-        - Key: deletion_protection.enabled
-          Value: "false"
-      Scheme: internet-facing
-      SecurityGroups:
-        - Fn::GetAtt:
-            - ProxyALBSecurityGroup0D6CA3DA
-            - GroupId
-      Subnets:
-        - Ref: VPCPublicSubnet1SubnetB4246D30
-        - Ref: VPCPublicSubnet2Subnet74179F39
-      Type: application
-    DependsOn:
-      - VPCPublicSubnet1DefaultRoute91CEF279
-      - VPCPublicSubnet1RouteTableAssociation0B0896DC
-      - VPCPublicSubnet2DefaultRouteB7481BBA
-      - VPCPublicSubnet2RouteTableAssociation5A808732
-  ProxyALBSecurityGroup0D6CA3DA:
-    Type: AWS::EC2::SecurityGroup
-    Properties:
-      GroupDescription: Automatically created Security Group for ELB BedrockProxyALB1CE4CAD1
-      SecurityGroupEgress:
-        - CidrIp: 255.255.255.255/32
-          Description: Disallow all traffic
-          FromPort: 252
-          IpProtocol: icmp
-          ToPort: 86
-      SecurityGroupIngress:
-        - CidrIp: 0.0.0.0/0
-          Description: Allow from anyone on port 80
-          FromPort: 80
-          IpProtocol: tcp
-          ToPort: 80
-      VpcId:
-        Ref: VPCB9E5F0B4
-  ProxyALBListener933E9515:
-    Type: AWS::ElasticLoadBalancingV2::Listener
-    Properties:
-      DefaultActions:
-        - TargetGroupArn:
-            Ref: ProxyALBListenerTargetsGroup187739FA
-          Type: forward
-      LoadBalancerArn:
-        Ref: ProxyALB87756780
-      Port: 80
-      Protocol: HTTP
-  ProxyALBListenerTargetsGroup187739FA:
-    Type: AWS::ElasticLoadBalancingV2::TargetGroup
-    Properties:
-      HealthCheckEnabled: false
-      TargetType: lambda
-      Targets:
-        - Id:
-            Fn::GetAtt:
-              - ProxyApiHandlerEC15A492
-              - Arn
-    DependsOn:
-      - ProxyApiHandlerInvoke2UTWxhlfyqbT5FTn5jvgbLgjFfJwzswGk55DU1HYF6C33779
-Mappings:
-  ProxyRegionTable03E5BEB3:
-    us-east-1:
-      model: anthropic.claude-3-sonnet-20240229-v1:0
-Outputs:
-  APIBaseUrl:
-    Description: Proxy API Base URL (OPENAI_API_BASE)
-    Value:
-      Fn::Join:
-        - ""
-        - - http://
-          - Fn::GetAtt:
-              - ProxyALB87756780
-              - DNSName
-          - /api/v1
-
--- a/deployment/BedrockProxyFargate.template
+++ b/deployment/BedrockProxyFargate.template
@@ -1,10 +1,10 @@
 Description: Bedrock Access Gateway - OpenAI-compatible RESTful APIs for Amazon Bedrock
 Transform: AWS::LanguageExtensions
 Parameters:
-  ApiKeyParam:
+  ApiKeySecretName:
    Type: String
    Default: ""
-    Description: The parameter name in System Manager used to store the API Key, leave blank to use a default key
+    Description: The secret name in Secrets Manager used to store the API Key
 Resources:
  VPCB9E5F0B4:
    Type: AWS::EC2::VPC
@@ -139,6 +139,22 @@ Resources:
              - logs:PutLogEvents
            Effect: Allow
            Resource: "*"
+          - Action:
+              - secretsmanager:GetSecretValue
+              - secretsmanager:DescribeSecret
+            Effect: Allow
+            Resource:
+              Fn::Join:
+                - ""
+                - - "arn:"
+                  - Ref: AWS::Partition
+                  - ":secretsmanager:"
+                  - Ref: AWS::Region
+                  - ":"
+                  - Ref: AWS::AccountId
+                  - ":secret:"
+                  - Ref: ApiKeySecretName
+                  - -??????
          - Action:
              - ecr:BatchCheckLayerAvailability
              - ecr:GetDownloadUrlForLayer
@@ -184,23 +200,6 @@ Resources:
            Resource:
              - arn:aws:bedrock:*::foundation-model/*
              - arn:aws:bedrock:*:*:inference-profile/*
-          - Action:
-              - ssm:DescribeParameters
-              - ssm:GetParameters
-              - ssm:GetParameter
-              - ssm:GetParameterHistory
-            Effect: Allow
-            Resource:
-              Fn::Join:
-                - ""
-                - - "arn:"
-                  - Ref: AWS::Partition
-                  - ":ssm:"
-                  - Ref: AWS::Region
-                  - ":"
-                  - Ref: AWS::AccountId
-                  - :parameter/
-                  - Ref: ApiKeyParam
        Version: "2012-10-17"
      PolicyName: ProxyTaskRoleDefaultPolicy933321B8
      Roles:
@@ -221,9 +220,6 @@ Resources:
    Properties:
      ContainerDefinitions:
        - Environment:
-            - Name: API_KEY_PARAM_NAME
-              Value:
-                Ref: ApiKeyParam
            - Name: DEBUG
              Value: "false"
            - Name: DEFAULT_MODEL
@@ -251,6 +247,20 @@ Resources:
            - ContainerPort: 80
              HostPort: 80
              Protocol: tcp
+          Secrets:
+            - Name: API_KEY
+              ValueFrom:
+                Fn::Join:
+                  - ""
+                  - - "arn:"
+                    - Ref: AWS::Partition
+                    - ":secretsmanager:"
+                    - Ref: AWS::Region
+                    - ":"
+                    - Ref: AWS::AccountId
+                    - ":secret:"
+                    - Ref: ApiKeySecretName
+                    - ":api_key::"
      Cpu: "1024"
      ExecutionRoleArn:
        Fn::GetAtt:
@@ -345,6 +355,8 @@ Resources:
      LoadBalancerAttributes:
        - Key: deletion_protection.enabled
          Value: "false"
+        - Key: idle_timeout.timeout_seconds
+          Value: "600"
      Scheme: internet-facing
      SecurityGroups:
        - Fn::GetAtt:
@@ -401,7 +413,9 @@ Resources:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      HealthCheckEnabled: true
+      HealthCheckIntervalSeconds: 60
      HealthCheckPath: /health
+      HealthCheckTimeoutSeconds: 30
      Port: 80
      Protocol: HTTP
      TargetGroupAttributes: