AWS Cost Reduction ──
Slashing Bandwidth Costs for a VR Video Streaming ServiceSource: S3 + CloudFront / Delivery: Bunny.net × Perma-Cache

1. Breaking Down the Invoice: Pinpointing Three Sources of Waste

It's quickest to just show you the real thing. This is a screenshot of the April 2026 bill (charges by service): $1,305.58 before tax and $1,436.14 including tax (about ¥230,000 at ¥160/$).

Service names alone, though, don't tell you what is being transferred or why the database costs this much. Once we traced what was behind each line item, three sources of waste worth cutting came into sharp focus.

2. Cloning and Upgrading the VR Video Service on EC2 (Amazon Linux 2023)

Rushing into production changes for the sake of cost savings is dangerous. So we left the old production environment completely untouched, cloned the entire VR video service onto a new EC2 instance, and did all the upgrading and testing on the new side. With old and new running side by side, we could fall back to the old one at any time (no downtime, nothing broken). The project is split into three phases, and this section covers Phase 1: cloning and upgrading the platform.

• Phase 1 (this section, platform refresh): Clone the system onto a new EC2 instance (Amazon Linux 2023 / Graviton) and upgrade PHP / CakePHP / the database to eliminate every EOL component.
• Phase 2 (delivery optimization): Offload videos and images to S3 and serve them through a CDN (Section 3 onward).
• Phase 3 (cutover and teardown): Switch production by reassigning the Elastic IP → monitor for a few days → delete the old EC2 / RDS / surplus EBS to lock in the savings.

Building the New Server on Graviton + Amazon Linux 2023

The new server runs Amazon Linux 2023 on Graviton (ARM / t4g), replacing the previous-generation t2.small with a cheaper, more efficient instance. We set up swap and installed nginx / PHP 8.2 / MariaDB 11.4 as middleware. From the OS through the language and framework, we jumped the entire EOL stack straight to currently supported versions.

Replicating the Database from RDS to a Local MariaDB on the Same EC2 Instance

We replicated every database from RDS into a local MariaDB running on the new EC2 instance. After the migration, we compared row counts for every table between old and new to confirm they matched, and upgraded the character set to utf8mb4. We also put daily backups + restore tests in place.

Retiring RDS was possible because the service never used managed HA such as Multi-AZ, and the production database held only a few MB of real data. We were paying Extended Support on an RDS instance whose redundancy features did nothing for us, so we folded it into a local database. The automated backups / PITR we gave up are covered by the in-house backups described above (backups had actually stopped entirely before we started, so this was really a rebuild of the recovery process).

Upgrading the App from CakePHP 3.6 → 4.6 / PHP 7.2 → 8.2

The cloned app went through a major version jump: CakePHP 3.6 → 4.6 and PHP 7.2 → 8.2. We worked through the breaking changes (template structure, authentication, routing, ORM / Query API, error handlers, image processing, and more) step by step. For API authentication (JWT), we replaced an unsupported legacy plugin with firebase/php-jwt plus a custom adapter. The key point here is that we didn't change a single API URL or response, so the VR player clients, which can't be updated, would be unaffected.

Not Touching the URL / API Contract at All

The biggest constraint was that we could not change the …/upload/… URLs and API responses referenced by VR player clients that are already installed and can't be updated. So we route requests to the delivery target inside the server (nginx) and leave the URLs returned by the app as they are. That gives us the ability to swap delivery targets without affecting clients. This decision to keep things loosely coupled is what later let us swap delivery targets (Section 3 onward) without touching the app.

3. Three Delivery Targets: S3 + CloudFront → R2 Rejected → Bunny.net

Delivery cost (egress) was the biggest cost driver, so that's what we went after. This is where the real trial and error began. In the end, the delivery target changed three times.

4. Final Architecture: Bunny × CloudFront Origin × Perma-Cache

Where we landed is a three-tier architecture with clearly separated roles. Putting CloudFront in front as Bunny's origin may look like a detour at first, but it brings two advantages.

[Viewer / VR app]
        │  https://…/upload/… (same URLs as before)
        ▼
[Bunny.net CDN]    … delivery to viewers. Low cost, persisted with Perma-Cache
        │  (fetched only when Bunny doesn't have it)
        ▼
[Amazon CloudFront] … fetch gateway. Nearly $0 within the AWS free tier (1TB/month)
        │
        ▼
[Amazon S3]        … single source of truth for videos/images (the persistent store)

The Role of Each Tier

How It Works (Lookup Order)

Viewer requests are resolved in the following order, top to bottom.

1. On a HIT at the Bunny edge, it's served immediately.
2. On a MISS, it's served from Perma-Cache if the file is there.
3. If it's in neither, Bunny fetches it from S3 via CloudFront and persists it in Perma-Cache. From then on, that file is never fetched from the origin again (essentially one fetch per file).

Why Keep S3 + CloudFront in the Picture?

Delivery itself is left to Bunny. Honestly, running everything on Bunny.net alone would make for a simpler setup. We still keep S3 and CloudFront behind it (as the origin) because we believe holding the master copies of the videos on the AWS side (S3) is the more trustworthy backup. S3 is the only persistent store (the source of truth), and Bunny and CloudFront are strictly disposable caches. With all data anchored in S3 alone, anything lost from a cache can be rebuilt from S3.

That said, we wanted to avoid exposing S3 directly to the internet (to prevent unintended access and unnecessary egress charges). So S3 keeps public access blocked, and CloudFront is the only gateway allowed to fetch from it. Storage (S3) and retrieval (CloudFront) have separate roles, and CloudFront gets to use the AWS free tier (1TB/month). This S3 + CloudFront pair also doubles as a fallback path holding every video: if Bunny.net has an outage, we just switch the nginx configuration.

Why Use CloudFront as the Origin Instead of S3 Directly?

This is the crux of the architecture. Putting CloudFront in as Bunny's origin yields two advantages.

Why No Bulk Migration or Sync Is Needed: Pull, Not Push

This origin setup (making CloudFront Bunny's fetch source) has one more big advantage beyond cost: by design, there is no need to move every video to Bunny up front, nor to sync each new file to Bunny. With the traditional approach of pushing everything to a Bunny Storage Zone, that wouldn't be the case.

• ① No up-front bulk migration: A push approach means transferring 1.7TB in one go (roughly $200 in egress = 1.7TB × $0.114/GB). With pull + Perma-Cache, content is filled in on demand via CloudFront as it gets accessed, each month stays within the 1TB free tier, and the cost is effectively $0. Videos nobody ever watches are never pulled, so they incur neither transfer nor storage.
• ② No syncing of new files: A push approach requires a sync batch or dual writes so that every upload lands in both S3 and Bunny. With pull, once a file is in S3, Bunny fetches it automatically on first access. That's one less moving part to operate, and missed or failed syncs simply can't happen.

Bunny.net Configuration

When you set CloudFront as the origin, the key is to make the Host header match the distribution's domain (if it doesn't match, CloudFront returns a 403).

5. End-to-End Test: Verifying MISS → Pull → HIT with a 760MB Video

With the architecture in place, we ran real data through it end to end. We deliberately picked a demanding file, a 760MB video that had never been fetched, with a Japanese file name containing spaces, so that a single run would exercise every branch of the lookup path.

We confirmed the whole chain in one pass: MISS → pull → 200, Range → 206, HIT on repeat access, and the file actually materializing in Perma-Cache. Partial fetches (206), which seek playback depends on, work, and Japanese file names don't break anything. With that, all the prerequisites for going to production were in place.

6. Cost Estimate: Cutting the Monthly Bill by Compressing Bandwidth Costs

Adding up all the changes so far, the projected monthly cost looks like this. "Before" is the actual April 2026 bill, and "New setup" is an estimate assuming 4TB/month of delivery and 1.7TB of storage (at ¥160/$; actual costs vary with traffic).

The biggest reason costs drop is the difference in unit prices. For delivery egress and storage in particular, just changing where you serve from and where you store things changes the per-GB price by an order of magnitude (all figures approximate).

* The RDS Extended Support fee (roughly $229/month) and the previous-generation EC2 were cut not through lower unit prices but by retiring and replacing them.
* Unit prices are approximate as of June 2026 (CloudFront: first tier, up to 10TB/month; Bunny: Volume Network). Prices are subject to change.

7. Operations: Upload / Update / Delete / Backup

Alongside cost, the other outcome is that day-to-day operations now run smoothly without strain. From upload to deletion, each operation completes automatically.

The purge on deletion has safeguards built in. If credentials aren't configured, it does nothing (no-op), and if a purge fails it just logs the error and never blocks the deletion itself. Since anything missed can be swept up by the cleanup batch, the update and delete flows for delivery won't fall apart in day-to-day operations.

8. Failover and Instant Rollback

• If the CDN (Bunny) goes down: A single server-side setting change instantly switches delivery from Bunny → CloudFront direct. CloudFront + S3 hold all content, so every video keeps streaming without interruption. Switching back is just as instant.
• The failover target is always validated: As mentioned, Bunny fetches through CloudFront even under normal operation, so the failover target is continuously exercised by real daily traffic.
• No risk of data loss: For persistent storage, S3 alone is the source of truth, and CloudFront / Bunny are disposable caches. Whether cached content disappears or is deliberately cleared, it is regenerated from S3.
• Server cutover: Switching production servers over and back is just a matter of reassigning the Elastic IP. No DNS changes are involved, so rollback is instant.

By building no downtime, easy rollback, and no data loss into the architecture itself, rather than just "delivering cheaply," we achieve cost reduction and stability at the same time.

9. Lessons Learned and Pitfalls

• Break the invoice down before you read it: Service-level totals won't show you where to cut. Only by tracing what's behind each line item (what kind of transfer, what the database charges are actually for) does the real picture emerge: transfer, storage, and Extended Support fees.
• Adding a CDN doesn't automatically lower costs: Caching reduces origin round trips, but egress to viewers is still billed by the CDN. Look at the pricing structure and check whether it actually moves the main cost driver.
• Settle terms-of-service gray areas before production: Even with a tempting short-term benefit like free egress, support refusing to give a clear yes or no is a business risk. Stopping on terms-of-service grounds is a valid decision even after testing passes.
• Putting a free tier in front of the origin has two benefits: Making CloudFront Bunny's origin absorbs fetches within the free tier while continuously validating the failover target.
• Loosely coupled URLs make swapping easy: Because the delivery target wasn't hard-coded into the app's URLs and was routed inside the server instead, we could replace the delivery platform without touching the app.
• Long-running batches die when the session drops: For long-running jobs like syncing 1.7TB, the OS may clean up the process the moment the SSH session disconnects. Make it durable by enabling lingering or running it as a systemd unit (if the sync is idempotent, it can resume).
• Prevent cache incidents with a "never overwrite" design: Managing files under fixed, unique paths and giving replacements a new path means updates take effect without purging, which makes cache-related incidents much less likely.