Using CloudFront to Access S3 Private Content

by

🌏 閱讀中文版本

Why Use CloudFront to Access S3 Private Content?

Core Value and Three Key Advantages

In AWS cloud architecture, the combination of Amazon S3 and Amazon CloudFront is the gold standard for content delivery. Especially when handling private content (such as premium media, internal documents, protected resources), this architecture simultaneously addresses three critical needs: security, performance, and cost.

1. Global Acceleration and Low Latency

CloudFront has a global network of edge locations. When users request content:

  • Proximity-based routing: Automatically routes to the nearest edge location
  • Caching mechanism: Content is cached at edge locations after the first request, subsequent requests served directly from cache
  • Latency reduction: Compared to direct S3 access, latency can be reduced by 50-80%
  • Bandwidth optimization: Reduces origin requests, lowering S3 egress costs

Real-world example: If your S3 bucket is in US East (us-east-1), Asian users accessing directly experience ~200-300ms latency; through CloudFront’s Tokyo edge, latency drops to 20-50ms.

2. Enhanced Security and Access Control

Making S3 buckets publicly accessible creates major security risks. With CloudFront + OAI/OAC:

  • S3 stays private: Buckets remain completely private, only accessible by CloudFront
  • Single entry point: All requests must go through CloudFront, direct S3 access is blocked
  • Signed URLs: Combine with CloudFront Signed URLs for time-limited access control
  • WAF integration: CloudFront integrates with AWS WAF to block malicious requests and DDoS attacks
  • Geo-restrictions: Configure access limits by country/region

3. Cost Optimization and Traffic Management

  • Reduced S3 request fees: CloudFront caching reduces GET requests to S3
  • Egress cost savings: CloudFront egress rates typically lower than direct S3 transfer
  • Free tier benefits: AWS Free Tier includes 1TB CloudFront egress and 10 million requests monthly
  • Predictable costs: Control cache duration via TTL settings, balancing cost and freshness

Origin Access Identity (OAI) vs Origin Access Control (OAC)

Technology Evolution and Selection Guide

AWS provides two methods for CloudFront to securely access S3: OAI (legacy) and OAC (modern).

Origin Access Identity (OAI) – Legacy Approach

How it works:

  • Create a special CloudFront identity (OAI)
  • Grant this OAI read permissions in S3 Bucket Policy
  • CloudFront uses the OAI identity to request content from S3

Limitations:

  • ❌ No support for S3 server-side encryption (SSE-KMS)
  • ❌ No support for dynamic requests (POST, PUT, DELETE)
  • ❌ Marked as Legacy by AWS

Origin Access Control (OAC) – Modern Approach (Recommended)

Enhanced capabilities:

  • ✅ Full support for S3 SSE-KMS encryption
  • ✅ Supports all HTTP methods (GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS)
  • ✅ Stronger security (uses AWS Signature Version 4)
  • ✅ More granular permission control

Recommendation: Use OAC for new projects; migrate legacy projects from OAI to OAC.

Complete Configuration Steps (Using OAC)

Step 1: Create S3 Bucket

# Create bucket using AWS CLI
aws s3 mb s3://my-private-content-bucket --region us-east-1

# Ensure all public access is blocked
aws s3api put-public-access-block 
  --bucket my-private-content-bucket 
  --public-access-block-configuration 
    "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"

Step 2: Upload Test Content

# Upload test file
echo "This is a private file" > test.txt
aws s3 cp test.txt s3://my-private-content-bucket/test.txt

# Verify direct access is blocked (should return 403)
curl https://my-private-content-bucket.s3.amazonaws.com/test.txt

Step 3: Create CloudFront Distribution with OAC

Navigate to CloudFront Console → Create Distribution:

  1. Origin Settings:
    • Origin Domain: Select your S3 bucket (my-private-content-bucket.s3.us-east-1.amazonaws.com)
    • Origin Access: Choose Origin Access Control Settings (recommended)
    • Click Create New OAC, use default settings
  2. Default Cache Behavior:
    • Viewer Protocol Policy: Redirect HTTP to HTTPS (enforce HTTPS)
    • Allowed HTTP Methods: Choose GET, HEAD (read-only) or GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE (read-write)
    • Cache Policy: CachingOptimized (recommended)
  3. Settings:
    • Price Class: Select geographic coverage (Use All Edge Locations / Use Only North America and Europe)
    • Alternate Domain Names (CNAMEs): Enter custom domain if applicable
    • Custom SSL Certificate: Upload SSL certificate if using custom domain
  4. Click Create Distribution

Step 4: Copy and Apply S3 Bucket Policy

After CloudFront creation, an orange notification banner appears. Click Copy Policy to get JSON similar to:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowCloudFrontServicePrincipalReadOnly",
      "Effect": "Allow",
      "Principal": {
        "Service": "cloudfront.amazonaws.com"
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::my-private-content-bucket/*",
      "Condition": {
        "StringEquals": {
          "AWS:SourceArn": "arn:aws:cloudfront::123456789012:distribution/E1234EXAMPLE"
        }
      }
    }
  ]
}

Navigate to S3 Console → Select bucket → Permissions → Bucket Policy → Paste policy → Save.

Step 5: Verify Access

# Get CloudFront Domain Name (e.g., d123abc456def.cloudfront.net)
CLOUDFRONT_DOMAIN="d123abc456def.cloudfront.net"

# Access via CloudFront (should succeed)
curl https://${CLOUDFRONT_DOMAIN}/test.txt

# Direct S3 access (should fail with 403)
curl https://my-private-content-bucket.s3.amazonaws.com/test.txt

Advanced Configuration: CloudFront Signed URLs

Implementing Time-Limited Access Control

For finer access control (e.g., premium content, time-limited download links), use CloudFront Signed URLs.

Configuration Steps

1. Create CloudFront Key Pair

  1. Log in with AWS root account
  2. Navigate to Security Credentials → CloudFront Key Pairs → Create New Key Pair
  3. Download private key (pk-XXXXX.pem) and note the Access Key ID

2. Set CloudFront Distribution to Restricted

  • Edit Distribution → Behaviors → Select Default Behavior
  • Restrict Viewer Access: Choose Yes
  • Trusted Signers: Select Self (use your AWS account)

3. Generate Signed URL with Python

from datetime import datetime, timedelta
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.backends import default_backend
import base64
import json

def create_signed_url(url, key_pair_id, private_key_path, expiration_minutes=60):
    # Set expiration time
    expiration = datetime.utcnow() + timedelta(minutes=expiration_minutes)
    expiration_timestamp = int(expiration.timestamp())

    # Create policy
    policy = {
        "Statement": [{
            "Resource": url,
            "Condition": {
                "DateLessThan": {
                    "AWS:EpochTime": expiration_timestamp
                }
            }
        }]
    }

    policy_json = json.dumps(policy, separators=(',', ':'))
    policy_b64 = base64.b64encode(policy_json.encode()).decode().replace('+', '-').replace('=', '_').replace('/', '~')

    # Load private key
    with open(private_key_path, 'rb') as key_file:
        private_key = serialization.load_pem_private_key(
            key_file.read(),
            password=None,
            backend=default_backend()
        )

    # Sign
    signature = private_key.sign(policy_json.encode(), padding.PKCS1v15(), hashes.SHA1())
    signature_b64 = base64.b64encode(signature).decode().replace('+', '-').replace('=', '_').replace('/', '~')

    # Construct Signed URL
    signed_url = f"{url}?Policy={policy_b64}&Signature={signature_b64}&Key-Pair-Id={key_pair_id}"
    return signed_url

# Usage example
url = "https://d123abc456def.cloudfront.net/premium/video.mp4"
key_pair_id = "APKAXXXXXXXXXXXXXXXX"
private_key_path = "/path/to/pk-XXXXX.pem"

signed_url = create_signed_url(url, key_pair_id, private_key_path, expiration_minutes=30)
print(f"Signed URL (30-minute validity): {signed_url}")

Frequently Asked Questions (FAQ)

Q1: CloudFront returns 403 Forbidden when accessing S3. How to troubleshoot?

Possible causes and solutions:

  1. Bucket Policy not configured correctly
    • Verify Resource includes /* (e.g., arn:aws:s3:::bucket-name/*)
    • Confirm AWS:SourceArn Distribution ID is correct
  2. S3 Block Public Access settings
    • Ensure settings don’t block CloudFront access (should only block public access, not OAC)
  3. CloudFront Distribution not yet deployed
    • Check Distribution Status is Deployed (initial creation takes 15-20 minutes)
  4. Cache issues
    • Create Invalidation to clear cache: aws cloudfront create-invalidation --distribution-id E1234 --paths "/*"

Q2: How to verify requests truly come from CloudFront?

Verification methods:

  1. Check S3 Access Logs
    • Enable S3 Access Logging
    • Verify Requester ARN contains cloudfront.amazonaws.com
  2. CloudFront Access Logs
    • Enable CloudFront Logging, check x-edge-location, x-edge-result-type fields
  3. Use custom header validation
    • Add custom header in CloudFront Behavior (e.g., X-Custom-Secret: abc123)
    • Validate this header at application layer, only accept requests with correct header

Q3: Can OAI and OAC coexist? How to migrate?

Answer: Yes, they can coexist, but gradual migration is recommended

Migration steps:

  1. Create new Origin in existing Distribution using OAC
  2. Modify Behavior to route certain paths (e.g., /new/*) to new Origin
  3. Gradually test and verify functionality
  4. Update S3 Bucket Policy to authorize both OAI and OAC (transition period)
  5. After confirming all traffic uses OAC, remove OAI-related configurations

Q4: How to set different cache durations for different file types?

Use Cache Behaviors and TTL settings:

  1. Navigate to CloudFront Distribution → Behaviors → Create Behavior
  2. Set Path Pattern (e.g., *.jpg, *.png, *.css)
  3. Configure TTL (Time To Live):
    • Static assets (images, CSS, JS): Minimum TTL = 86400 (1 day), Maximum TTL = 31536000 (1 year)
    • Dynamic content (API responses): Minimum TTL = 0, Maximum TTL = 3600 (1 hour)

Example: Long cache for images

Path Pattern: *.jpg
Cache Policy: CachingOptimized
Minimum TTL: 86400 (1 day)
Maximum TTL: 31536000 (1 year)
Default TTL: 2592000 (30 days)

Q5: How to handle versioning with CloudFront + S3 (Cache Busting)?

Three common approaches:

  1. Filename with version or hash (recommended)
    • Example: style.v2.css, app.a3f2b1c.js
    • Benefit: No cache clearing needed, new versions take effect automatically
  2. Query string versioning
    • Example: style.css?v=2
    • Requires enabling Query String Forwarding in CloudFront settings
  3. Manual Invalidation
    • Use AWS CLI: aws cloudfront create-invalidation --distribution-id E1234 --paths "/style.css"
    • Note: First 1000 invalidations/month free, additional cost $0.005 each

Q6: How to monitor CloudFront costs and traffic?

Use CloudWatch and Cost Explorer:

  1. CloudWatch Metrics
    • Requests: Total request count
    • BytesDownloaded: Download traffic
    • BytesUploaded: Upload traffic (POST, PUT)
    • 4xxErrorRate, 5xxErrorRate: Error rates
  2. Set CloudWatch Alarms 
    aws cloudwatch put-metric-alarm 
  --alarm-name cloudfront-high-error-rate 
  --metric-name 5xxErrorRate 
  --namespace AWS/CloudFront 
  --statistic Average 
  --period 300 
  --threshold 5 
  --comparison-operator GreaterThanThreshold 
  --evaluation-periods 2 
  --alarm-actions arn:aws:sns:us-east-1:123456789012:alerts
  3. Cost Explorer analysis
    • Navigate to AWS Cost Explorer → Select Service = CloudFront
    • Analyze traffic costs by region
    • Identify abnormal traffic spikes

Q7: How to integrate WAF for CloudFront protection?

AWS WAF configuration steps:

  1. Create Web ACL
    • Navigate to AWS WAF Console → Create Web ACL
    • Resource Type: Select CloudFront
  2. Add rules
    • Rate Limiting: Limit same IP to 2000 requests per 5 minutes
    • Geo Blocking: Block specific countries (if needed)
    • SQL Injection / XSS protection: Use AWS Managed Rules
  3. Associate with CloudFront
    • Edit CloudFront Distribution → Security → AWS WAF Web ACL → Select the ACL you created

Example: Rate Limiting rule

{
  "Name": "RateLimitRule",
  "Priority": 1,
  "Statement": {
    "RateBasedStatement": {
      "Limit": 2000,
      "AggregateKeyType": "IP"
    }
  },
  "Action": {
    "Block": {}
  },
  "VisibilityConfig": {
    "SampledRequestsEnabled": true,
    "CloudWatchMetricsEnabled": true,
    "MetricName": "RateLimitRule"
  }
}

Best Practices

Security

  • ✅ Enforce HTTPS: Set Viewer Protocol Policy to “Redirect HTTP to HTTPS”
  • ✅ Enable S3 versioning: Prevent accidental deletion or overwrite of critical files
  • ✅ Enable S3 Object Lock: Protect critical data from deletion (compliance requirement)
  • ✅ Rotate CloudFront Key Pairs regularly: If using Signed URLs, rotate keys annually
  • ✅ Enable CloudTrail: Log all API operations for auditing

Performance Optimization

  • ✅ Enable Gzip/Brotli compression: Enable “Compress Objects Automatically” in CloudFront Behavior
  • ✅ Use HTTP/2 and HTTP/3: CloudFront supports by default, ensure browser compatibility
  • ✅ Choose appropriate Price Class: Select fewer edge locations to reduce cost if users concentrated in specific regions
  • ✅ Set reasonable TTL: Adjust cache duration based on content update frequency

Cost Control

  • ✅ Use S3 Intelligent-Tiering: Automatically move infrequently accessed objects to lower-cost storage tiers
  • ✅ Set S3 Lifecycle Policies: Automatically delete expired temporary files
  • ✅ Monitor Invalidation usage: Consider versioning approach before exceeding free quota
  • ✅ Analyze traffic sources: Identify and block abnormal traffic (e.g., bots, attacks)

Troubleshooting Guide

Issue: CloudFront returns 504 Gateway Timeout

Possible causes:

  • S3 response time too long (large file initial request)
  • Network connectivity issues

Solutions:

  1. Increase CloudFront Origin Response Timeout (default 30s, can adjust to 60s)
  2. Check if S3 bucket is under high load
  3. Consider using S3 Transfer Acceleration

Issue: Low Cache Hit Ratio

Possible causes:

  • Requests with too many Query String parameters or Headers
  • TTL set too short

Solutions:

  1. Review Cache Key Settings, only forward necessary Query Strings and Headers
  2. Adjust Default TTL to extend cache duration
  3. Use CloudFront Cache Policy to centrally manage cache rules

Issue: Some users cannot access (specific regions)

Possible causes:

  • Geo Restriction enabled
  • WAF rules blocking incorrectly

Solutions:

  1. Check CloudFront Distribution → Restrictions → Geographic Restrictions
  2. Review WAF Web ACL Logs to identify false positive rules
  3. Temporarily add problem IP to WAF allowlist for testing

Conclusion

Using CloudFront to access S3 private content is an AWS cloud architecture best practice, achieving:

  • 🚀 Global acceleration: Low-latency, high-bandwidth content delivery
  • 🔒 Enhanced security: S3 fully private, accessible only by CloudFront
  • 💰 Cost optimization: Reduced S3 request fees and egress costs
  • 🛡️ Advanced protection: Integration with WAF, Signed URLs, geo-restrictions

Key takeaways:

  1. Use OAC (Origin Access Control) for new projects instead of legacy OAI
  2. Properly configure S3 Bucket Policy to authorize only specific CloudFront Distribution
  3. Set reasonable TTL based on content type, balancing freshness and cache efficiency
  4. Enable CloudWatch monitoring and CloudTrail auditing to detect anomalies
  5. Regularly review Cost Explorer to optimize cost structure

This architecture suits various scenarios: media streaming, software downloads, premium content, corporate documents, API resources, and more. With proper configuration and continuous optimization, you can build an efficient, secure, and cost-effective cloud content delivery system.

Related Articles

Related posts:

  1. 利用 CloudFront 訪問 S3 的私有內容
  2. 在 AWS 中匯入 SSL/TLS 憑證的完整指南
  3. How to Set Up CloudFront IP Whitelist with AWS WAF
  4. Complete Guide to Importing SSL/TLS Certificates in AWS

Leave a Comment