Add cross-region inference profiles for Llama 3.1 models. (#72)

README.md

@@ -173,6 +173,8 @@ Currently, Bedrock Access Gateway only supports cross-region Inference for the f
 - Claude 3 Opus
 - Claude 3 Sonnet
 - Claude 3.5 Sonnet
+- Meta Llama 3.1 8b Instruct
+- Meta Llama 3.1 70b Instruct
 
 **Prerequisites:**
 - IAM policies must allow cross-region access,Callers need permissions to access models and inference profiles in both regions (added in cloudformation template)

README_CN.md

@@ -175,6 +175,8 @@ Cross-Region Inference 支持跨区域访问的基础模型,即允许用户在
 - Claude 3 Opus
 - Claude 3 Sonnet
 - Claude 3.5 Sonnet
+- Meta Llama 3.1 8b Instruct
+- Meta Llama 3.1 70b Instruct
 
 **使用前提:**
 - IAM Policy 有 inference profiles 相关的权限和调用模型的权限 (cloudformation template 中已添加)

src/api/models/bedrock.py

@@ -139,12 +139,26 @@ class BedrockModel(BaseChatModel):
             "tool_call": False,
             "stream_tool_call": False,
         },
+        # Llama 3.1 8b cross-region inference profile
+        "us.meta.llama3-1-8b-instruct-v1:0": {
+            "system": True,
+            "multimodal": False,
+            "tool_call": False,
+            "stream_tool_call": False,
+        },
         "meta.llama3-1-8b-instruct-v1:0": {
             "system": True,
             "multimodal": False,
             "tool_call": False,
             "stream_tool_call": False,
         },
+        # Llama 3.1 70b cross-region inference profile
+        "us.meta.llama3-1-70b-instruct-v1:0": {
+            "system": True,
+            "multimodal": False,
+            "tool_call": False,
+            "stream_tool_call": False,
+        },
         "meta.llama3-1-70b-instruct-v1:0": {
             "system": True,
             "multimodal": False,