AWS Lambda functions now support mounting Amazon S3 buckets as file systems

AWS Lambda has introduced support for Amazon S3 Files, allowing Lambda functions to mount Amazon S3 buckets as file systems. This new feature enables standard file operations to be performed without the need to download data for processing. Built on Amazon EFS, S3 Files offers the performance and simplicity of a file system, combined with the scalability, durability, and cost-effectiveness of S3. Multiple Lambda functions can access the same S3 Files file system at the same time, facilitating data sharing through a common workspace without requiring custom synchronization logic.

The integration with S3 Files simplifies stateful workloads within Lambda by removing the need to download objects, upload results, and manage ephemeral storage limits. This feature is particularly beneficial for AI and machine learning workloads, where agents need to maintain memory and share state across different pipeline steps. Lambda’s durable functions enable these multi-step AI workflows by managing parallel execution with automatic checkpointing. For instance, an orchestrator function can clone a repository to a shared workspace, while multiple agent functions analyze the code concurrently. The durable function manages the checkpointing of execution state, while S3 Files ensures seamless data sharing across all steps.

To utilize S3 Files with Lambda, users can configure their function to mount an S3 bucket through the Lambda console, AWS CLI, AWS SDKs, AWS CloudFormation, or AWS Serverless Application Model (SAM). Further information on using S3 Files with Lambda functions is available in the Lambda developer guide.

S3 Files is available for Lambda functions that are not configured with a capacity provider, across all AWS Regions where both Lambda and S3 Files are supported. There is no additional charge for using S3 Files beyond the standard Lambda and S3 pricing.