Commit Graph
12 Commits
Author SHA1 Message Date
Mengxin Zhu 9b3da3a5c8 fix(deps): update fastapi and starlette for CVE-2025-62727 (#216)
Update dependencies to fix HIGH severity ReDoS vulnerability:
- fastapi==0.128.0
- starlette==0.49.1

CVE-2025-62727 allows unauthenticated attackers to send crafted HTTP
Range headers that trigger quadratic-time processing in FileResponse
Range parsing, causing CPU exhaustion and DoS.

Fixes #215
2026-01-19 11:57:01 +08:00
Mengxin Zhu b41633b826 feat(apigw): add API Gateway response streaming support (#207)
Replace ALB + Lambda architecture with API Gateway REST API + Lambda
using response streaming for SSE support. This provides:

- No VPC required, reducing complexity and cost
- Native streaming support via API Gateway response streaming
- Pay-per-request pricing model

Changes:
- Add Lambda Web Adapter to Dockerfile for streaming support
- Replace BedrockProxy.template with API Gateway configuration
- Update README with new deployment options and latest models
- Update architecture diagram for API Gateway flow
2025-12-05 10:54:13 +08:00
Kane Zhu 2c518bbd70 fix(docker): add --provenance=false --sbom=false for Lambda compatibility
Docker BuildKit (especially with docker-container driver) may create
OCI image manifests with attestations that AWS Lambda does not support.
Lambda requires Docker V2 Schema 2 format without multi-manifest index.

This fix ensures the build script generates Lambda-compatible images
regardless of the user's Docker/BuildKit configuration.

Fixes #206
2025-11-27 18:54:58 +08:00
Kane Zhu d86e64eed3 refactor(bedrock): unify inference profile metadata handling and cleanup
- Add unified profile_metadata dictionary for both SYSTEM_DEFINED and APPLICATION inference profiles
- Remove unused region prefix functions and defaultdict import
- Add TEMPERATURE_TOPP_CONFLICT_MODELS set for Claude model parameter conflicts
- Improve model ARN parsing and error handling in profile enumeration
- Consolidate profile metadata storage to enable consistent feature detection
2025-10-16 15:24:02 +08:00
Kane Zhu b4800c54a0 feat: add prompt caching support for Claude and Nova models
Add comprehensive prompt caching support with flexible control options:

Features:
- ENV variable control (ENABLE_PROMPT_CACHING, default: false)
- Per-request control via extra_body.prompt_caching
- Pattern-based model detection (Claude, Nova)
- Token limit warnings (Nova 20K limit)
- OpenAI-compatible response format (prompt_tokens_details.cached_tokens)

Supported models:
- Claude 3+ models (anthropic.claude-*)
- Nova models (amazon.nova-*)
- Auto-detection prevents breaking unsupported models

Implementation:
- System prompts caching via extra_body.prompt_caching.system
- Messages caching via extra_body.prompt_caching.messages
- Non-streaming and streaming modes
- Compatible with reasoning, thinking, and tool calls
2025-10-15 11:03:19 +08:00
Mengxin Zhu 371d11d101 chore: cleanup useless files 2025-09-30 16:08:56 +08:00
Mengxin Zhu e3ee9a707f docs: update deployment instructions and enhance ECR push script 2025-09-30 16:06:21 +08:00
Mengxin Zhu 76a3614f17 fix: properly handle tool_use messages in conversation 2025-06-30 00:14:26 +08:00
Meng Xin Zhu 9cc3ea8253 chore: publish templates to s3 in release workflow (#64) 2024-10-28 17:36:35 +08:00
Mengxin Zhu 46fb759137 chore: use correct Dockerfile for building lambda image 2024-10-09 23:39:37 +08:00
Mengxin Zhu 326e566105 chore: use arm64 architecture image for lambda 2024-10-09 23:15:10 +08:00
Meng Xin Zhu c1ee1b4244 chore: add automation script to release images (#58) 2024-10-09 18:20:14 +08:00