AVB/bedrock-access-gateway
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update README

Browse Source
This commit is contained in:
Aiden Dai
2024-05-10 10:21:03 +08:00
parent 9f6b334385
commit d7a26dcf8b
2 changed files with 35 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
README.md
View File

@@ -26,23 +26,15 @@ Please check [Usage Guide](./docs/Usage.md) for more details about how to use th
> **Note:** The legacy [text completion](https://platform.openai.com/docs/api-reference/completions) API is not supported, you should change to use chat completion API. > **Note:** The legacy [text completion](https://platform.openai.com/docs/api-reference/completions) API is not supported, you should change to use chat completion API.
Supported Amazon Bedrock models (Model IDs): Supported Amazon Bedrock models family:
- anthropic.claude-instant-v1 - Anthropic Claude 2 / 3 (Haiku/Sonnet/Opus)
- anthropic.claude-v2:1 - Meta Llama 2 / 3
- anthropic.claude-v2 - Mistral / Mixtral
- anthropic.claude-3-opus-20240229-v1:0 - Cohere Command R / R+
- anthropic.claude-3-sonnet-20240229-v1:0 - Cohere Embedding
- anthropic.claude-3-haiku-20240307-v1:0
- meta.llama2-13b-chat-v1 You can call the `models` API to get the full list of model IDs supported.
- meta.llama2-70b-chat-v1
- meta.llama3-8b-instruct-v1:0
- meta.llama3-70b-instruct-v1:0
- mistral.mistral-7b-instruct-v0:2
- mistral.mixtral-8x7b-instruct-v0:1
- mistral.mistral-large-2402-v1:0
- cohere.embed-multilingual-v3 (embedding)
- cohere.embed-english-v3 (embedding)
> **Note:** The default model is set to `anthropic.claude-3-sonnet-20240229-v1:0` which can be changed via Lambda environment variables (`DEFAULT_MODEL`). > **Note:** The default model is set to `anthropic.claude-3-sonnet-20240229-v1:0` which can be changed via Lambda environment variables (`DEFAULT_MODEL`).
@@ -206,7 +198,7 @@ Short answer is that API Gateway does not support server-sent events (SSE) for s
### Which regions are supported? ### Which regions are supported?
This solution only supports the regions where Amazon Bedrock is available: This solution only supports the regions where Amazon Bedrock is available, as for now, below are the list.
- US East (N. Virginia): us-east-1 - US East (N. Virginia): us-east-1
- US West (Oregon): us-west-2 - US West (Oregon): us-west-2
@@ -216,6 +208,8 @@ This solution only supports the regions where Amazon Bedrock is available:
- Europe (Frankfurt): eu-central-1 - Europe (Frankfurt): eu-central-1
- Europe (Paris): eu-west-3 - Europe (Paris): eu-west-3
Generally speaking, all regions that Amazon Bedrock supports will also be supported, if not, please raise an issue in Github.
Note that not all models are available in those regions. Note that not all models are available in those regions.
### Can I build and use my own ECR image ### Can I build and use my own ECR image
@@ -246,7 +240,13 @@ Fine-tuned models and models with Provisioned Throughput are currently not suppo
### How to upgrade? ### How to upgrade?
If there is no changes on architecture, you can simply deploy the latest image to your Lambda to use the new features (manually) without redeploying the whole CloudFormation stack. To use the latest features, you don't need to redeploy the CloudFormation stack. You simply need to pull the latest image.
To do so, depends on which version you deployed:
- **Lambda version**: Go to AWS Lambda console, find the Lambda function, then find and click the `Deploy new image` button and click save.
- **Fargate version**: Go to ECS console, click the ECS cluster, go the `Tasks` tab, select the only task that is running and simply click `Stop selected` menu. A new task with latest image will start automatically.
## Security ## Security

35
README_CN.md
View File

@@ -28,23 +28,15 @@ OpenAI 的 API 或 SDK 无缝集成并试用 Amazon Bedrock 的模型,而无需
> 注意： 不支持旧的 [text completion](https://platform.openai.com/docs/api-reference/completions) API，请更改为使用Chat Completion API。 > 注意： 不支持旧的 [text completion](https://platform.openai.com/docs/api-reference/completions) API，请更改为使用Chat Completion API。
支持的Amazon Bedrock模型列表（Model IDs）： 支持的Amazon Bedrock模型家族：
- anthropic.claude-instant-v1 - Anthropic Claude 2 / 3 (Haiku/Sonnet/Opus)
- anthropic.claude-v2:1 - Meta Llama 2 / 3
- anthropic.claude-v2 - Mistral / Mixtral
- anthropic.claude-3-opus-20240229-v1:0 - Cohere Command R / R+
- anthropic.claude-3-sonnet-20240229-v1:0 - Cohere Embedding
- anthropic.claude-3-haiku-20240307-v1:0
- meta.llama2-13b-chat-v1 你可以先调用`models` API 获取支持的详细 model ID 列表。
- meta.llama2-70b-chat-v1
- meta.llama3-8b-instruct-v1:0
- meta.llama3-70b-instruct-v1:0
- mistral.mistral-7b-instruct-v0:2
- mistral.mixtral-8x7b-instruct-v0:1
- mistral.mistral-large-2402-v1:0
- cohere.embed-multilingual-v3 (embedding)
- cohere.embed-english-v3 (embedding)
> 注意: 默认模型为 `anthropic.claude-3-sonnet-20240229-v1:0`， 可以通过更改Lambda环境变量进行更改。 > 注意: 默认模型为 `anthropic.claude-3-sonnet-20240229-v1:0`， 可以通过更改Lambda环境变量进行更改。
@@ -211,7 +203,7 @@ print(response)
### 支持哪些区域? ### 支持哪些区域?
只支持Amazon Bedrock可用的区域,即: 只支持Amazon Bedrock可用的区域, 截至当前，包括以下区域:
- 美国东部(弗吉尼亚北部)：us-east-1 - 美国东部(弗吉尼亚北部)：us-east-1
- 美国西部(俄勒冈州)：us-west-2 - 美国西部(俄勒冈州)：us-west-2
@@ -221,6 +213,8 @@ print(response)
- 欧洲(法兰克福)：eu-central-1 - 欧洲(法兰克福)：eu-central-1
- 欧洲(巴黎)：eu-west-3 - 欧洲(巴黎)：eu-west-3
通常来说，所有Amazon Bedrock支持的区域都支持，如果不支持，请提个Github Issue。
注意，并非所有模型都在上面区可用。 注意，并非所有模型都在上面区可用。
### 我可以构建并使用自己的ECR镜像吗? ### 我可以构建并使用自己的ECR镜像吗?
@@ -251,7 +245,12 @@ print(response)
### 如何升级? ### 如何升级?
如果架构没有变化,你可以简单地将最新镜像部署到Lambda中,以使用新功能(手动),而无需重新部署整个CloudFormation堆栈。 要使用最新功能,您无需重新部署CloudFormation堆栈。您只需拉取最新的镜像即可。
具体操作方式取决于您部署的版本:
- **Lambda版本**: 进入AWS Lambda控制台,找到Lambda 函数，然后找到并单击`部署新映像`按钮,然后单击保存。
- **Fargate版本**: 进入ECS控制台,单击ECS集群,转到`任务`选项卡,选择正在运行的唯一任务,然后点击`停止所选`菜单, ECS会自动启动新任务并且使用最新镜像。
## 安全 ## 安全