1. AWS Lambda Quotas: The official AWS Lambda documentation states the maximum execution duration for a function.
Source: AWS Lambda Developer Guide
"Lambda quotas".
Reference: In the "Function configuration
deployment
and execution quotas" table
the "Timeout" resource has a maximum value of 900 seconds (15 minutes).
2. Invoking ECS Tasks from S3 Events: The AWS documentation outlines the event-driven pattern of using S3 events with EventBridge to start an ECS task
which is ideal for this scenario.
Source: AWS Compute Blog
"Introducing Amazon EventBridge integration for Amazon ECS".
Reference: The blog post details how "Amazon ECS is now a target for Amazon EventBridge
" allowing you to "run Amazon ECS tasks on a schedule
or in response to any EventBridge event." This includes S3 object creation events.
3. AWS Fargate for Long-Running Tasks: Fargate is a serverless compute engine suitable for containerized applications
including batch jobs that run longer than the Lambda timeout.
Source: AWS Fargate FAQs.
Reference: The FAQ page clarifies that Fargate is suitable for "batch workloads... that run for a short period of time
" which can be minutes or hours
thus easily accommodating the 25-minute requirement without the operational overhead of managing servers.
4. Purpose of AWS Glue Crawlers: The AWS Glue documentation specifies the intended use of crawlers
which does not align with triggering application logic.
Source: AWS Glue Developer Guide
"Defining Crawlers".
Reference: The guide states
"You use a crawler to populate the AWS Glue Data Catalog with tables. This is the primary method used by most AWS Glue users." This confirms its purpose is metadata discovery
not event-based invocation.