On Fly Image Transformation Pipeline Using Amazon CloudFront and AWS Lambda

Key Terminology

CloudFront: Amazon's Content Delivery Network (CDN) that distributes content globally with low latency. It caches content at edge locations worldwide for faster delivery to users.

AWS Lambda: A serverless compute service that runs code in response to events without requiring server management. Perfect for image processing tasks that happen on-demand.

S3 Bucket: Amazon's Simple Storage Service - a scalable object storage solution that stores files (like images) securely in the cloud.

System Overview

The architecture presents an elegant solution for on-demand image transformation that optimizes both storage and delivery. When users request images with specific transformations (like resizing or rotation), the system intelligently handles these requests through a series of well-defined steps.

How It Works

1. User Request Handling

Users can request images in two ways:

• Original images: https://cloudfront.net/path/image.jpg

• Transformed images: https://cloudfront.net/t_w-28_h-28_r-5/image.jpg

The transform parameters in the URL specify desired modifications:

• w-28: Width of 28 pixels

• h-28: Height of 28 pixels

• r-5: Rotation of 5 degrees

2. CloudFront Functions Processing

CloudFront Functions act as the first line of processing by:

• Analyzing incoming URLs

• Standardizing transformation parameters

• Ensuring consistent URL patterns for effective caching

For example, if a request comes in with parameters in a different order:

/t_w-28_r-5_h-28/dev.png → /t_w-28_h-28_r-5/dev.png

3. Storage Architecture

The system employs a dual-bucket strategy:

Original S3 Bucket:

• Stores unmodified source images

• Serves as the single source of truth

• Never modified after initial upload

Transformed S3 Bucket:

• Contains all processed variants

• Organized by transformation parameters

• Acts as a persistent cache

4. Transformation Flow

When a transformed image is requested:

1. System checks CloudFront cache

2. If not found, checks Transformed S3 Bucket

3. If still not found:

   • Lambda function activates

   • Retrieves original image

   • Applies requested transformations through API module like (Sharp)

   • Stores result in Transformed S3 Bucket

   • Returns processed image in base64 format to the body

5. Error Handling

The system includes robust error handling:

• Returns 404 if original image isn't found from Original S3 bucket

• Provides fallback options for transformation failures

• Maintains consistent error responses across the platform

Benefits

1. Performance Optimization:

   • Multiple caching layers

   • Edge delivery through CloudFront

   • On-Fly processing only when needed

2. Cost Efficiency:

   • Transforms images only once

   • Caches results for future requests

   • Minimizes unnecessary processing

3. Scalability:

   • Serverless architecture with Lambda

   • Automatic scaling with demand

   • No infrastructure management needed

4. Maintenance:

   • Clear separation of original and transformed images

   • Easy tracking of image variants

   • Simplified troubleshooting

This architecture provides a robust foundation for handling image transformations at scale while maintaining performance and cost efficiency. Building a Scalable Image Transformation System with AWS