Cloud Bursting for Peak Rendering Workloads
Maximum Scale and VFX Quality with Avere-enabled Rendering on AWS EC2
Preparing to cast off on their most ambitious journey to the cloud, the modern buccaneers at Important Looking Pirates (ILP) had high hopes for a bounty-rich adventure. Affordable access to burst-compute resources for rendering would allow the crew to run more iterations, securing scalability to guarantee peak-production capacity for the studio’s rapidly expanding project pipeline. But like high waves on an open ocean, high latency and network bottlenecks threatened to doom the voyage.
ILP co-founder and VFX Supervisor Niklas Jacobson says that an Avere Systems virtual FXT Edge filer smoothed out the cloud journey. “Avere addressed latency and bandwidth issues, allowing us to present huge volumes of scene data to cloud nodes. By using the cloud to scale render capacity, we can avoid making costly investments in permanent infrastructure simply to cover short-term peak workloads. ILP is passionate about creating the industry’s highest-quality visual effects, and Avere technology lets us viably exploit cloud resources to support that obsession.”
Challenge: Render Power for the Crunch
ILP creates content for all forms of digital productions, including television series, feature films, commercials, games, and more. Nominations for prestigious animago and Visual Effects Society (VES) awards—including for the studio’s work on the SyFy channel miniseries Childhood’s End and the NBC Crossbones and Constantine television shows—give testament to the team’s dedication to quality VFX. Other notable and upcoming projects include the Everest adventure-drama film, the STARZ hit show Black Sails, a new American feature film, and multiple commercials and game projects.
With such success comes the challenge of keeping up with demand for render capacity. Björn Malmgren, ILP’s lead pipeline engineer, explains, “Our projects require increasingly complex effects—a single frame sent to the render farm, for example, can easily hold 2GB of data. During a busy month, we may push more than 20,000 render and simulation jobs through our systems. The majority of the time, we’re able to handle the workload on our in-house infrastructure comprised of a 78-node render farm and a 200TB Dell Compellent storage footprint. But unfortunately, during crunch times when we’re up against production deadlines—often for multiple projects wrapping simultaneously—we may need twice that render capacity. The promise of on-demand access to nearly infinite compute resources led us to investigate cloud services.”
Solution: Avere for High-performance Cloud Rendering
ILP System Administrator, Hans Fagerström, concedes that initial cloud tests were disappointing. “Getting huge volumes of scene data to the cloud render nodes proved to be an insurmountable problem. The only success we had was running very lightweight jobs—but those workloads represent a very small percentage of our pipeline today.”
Fagerström says introducing Avere technology turned the tide on cloud rendering, making burst compute both accessible and affordable. Today, an Avere vFXT Edge filer ensures high-speed data access. The Avere Edge filer operates on Amazon EC2 alongside the studio’s rendering applications, typically Chaos Group V-Ray and Solid Angle Arnold software. The Avere cluster automatically moves scene data to the cloud, providing extremely low-latency access to active data and enabling rendering processes to run at the high performance that ILP production requires.
Benefits: Instant Scale, Savings, and Business Flexibility
Blimey, 2X Render Capacity Overnight!
The Avere vFXT cluster running on EC2 allows ILP to almost instantly scale rendering capacity to accommodate short-term peaks or unanticipated demand. Jacobson says cloud rendering helped ILP meet critical deadlines on the complex effects work for the more than 100 shots produced for the SyFy channel’s Childhood’s End project. “The norm in this industry is uneven workloads, so having on-demand access to cloud-based render nodes ensures we can always run the iterations needed for quality, keep shots in production longer, and work productively right up to deadlines.”
Always the Best Price/Performance
“Accessing cloud resources is nearly transparent,” reports Malmgren. “We needed to be able to very quickly move workloads to the cloud, without spending a lot of time determining which jobs to move, writing scripts, or managing data. Today we don’t differentiate which jobs are run locally and which we move to the cloud. We can utilize cloud bursting for all but a few of our largest simulations. And with minimal administration required, we’re able to focus more resources on developing tools specific to our VFX work.”
Jacobson says that without Avere, the alternative solution for scale would have been to buy more hardware. “It’s very unfortunate if you have to invest in permanent infrastructure for short-term needs. It’s simply not sustainable to continue to purchase hardware for peak workloads only to have that capital sit idle most of the time. On the Childhood’s End project, for example, doubling our infrastructure for a week’s worth of work would not have been feasible. But by taking advantage of Avere-enabled cloud rendering, we were able to deliver extremely high-quality effects while still meeting the show’s aggressive deadlines.
“Avere really was the last piece of the puzzle we needed to take advantage of the cloud. The Avere technology gives us the flexibility to follow the market in terms of price/performance, choosing the most competitive cloud service or even combining cloud services for maximum render power. And the entire process is seamless to artists—they’re not even aware that rendering processes are running in the cloud.”
The Freedom to Swashbuckle!
“It would be sad to not be able to take part in great shows just because we didn’t have sufficient rendering capacity to meet delivery dates,” concludes Jacobson. “By enabling cloud rendering, Avere gives us flexibility and freedom to boldly pursue exciting new opportunities, knowing that we can scale as needed to deliver high-quality VFX on time and on budget.”
About Important Looking Pirates
Important Looking Pirates is a digital animation and effects facility located in the heart of Stockholm, Sweden. Utilizing the industry's latest technology and in-house software development, the studio’s award-winning artists provide high-end digital animation and effects for commercials, broadcast, television series, promos, and feature films. The studio boasts a project portfolio that includes the television series Black Sails, the American movie Everest, the Norwegian Kon-Tiki feature film, as well as commercials for Adidas, Coca-Cola, the Discovery Channel, Fox Sports, IKEA, Samsung, the Swedish Armed Forces, Vodafone, and many more. (www.ilpvfx.com)