ToonBox Gets Non-Disruptive Scaling and Quick Avere ROI
Artist Euphoria Spreads with 18x Render Bump
Every studio encounters the pressures of crunch time, that point in a production where artists and systems are working in overdrive to run final renders, polish shots, and adjust scenes for final reviews. With costly and rippling consequences of even a minor schedule slip, missed deadlines are not an option. IT managers meticulously plan for these peak-workload periods, building infrastructure that will keep workflows moving and artists productive. At ToonBox Entertainment in Toronto, Canada, an essential element of that infrastructure is an Avere Systems FXT Edge filer cluster that delivers high-performance I/O to a 5500-core render farm while maintaining essential responsiveness to artist desktops.
“Implementing the Avere solution allows us to easily and non-disruptively scale storage performance independent of capacity,” says Greg Whynott, manager of Systems and IT at ToonBox. “That means we no longer worry about impacting render-farm workloads or user productivity as a movie progresses through the pipeline or as the studio expands productions. Avere has largely mitigated the storage slowdowns that were the usual culprits during crunch times.”
Challenge: Fix Squirrely Performance
ToonBox implemented Avere Edge filers during production of The Nut Job, the studio’s first feature-length animated film starring the mischievous Surly Squirrel and his rat friend, Buddy. As expected, creation of the highly detailed 3D stereoscopic content for the award-winning movie put heavy performance demand on the studio’s render farm, storage systems and artist workstations. What was not expected was a storage-performance slowdown partway into production.
Working with Scalar, a leading IT solutions integrator and Avere channel partner, ToonBox deployed an Avere FXT Edge filer cluster in front of the studio’s existing storage array. The result? You might say that I/O performance went from squirrely to screaming.
Solution: Avere to Handle All Workflows
“Dropping an Avere cluster into our production environment more than doubled throughput,” Whynott affirms. “We had been implementing QoS procedures on our network switches to manage bandwidth allocation, but with the Avere cluster in place, we were able to stop those processes and simply let jobs run—there was sufficient storage bandwidth for all workflows.”
Whynott reports that the support through the process exceeded expectations. “We called in the morning, the system was racked and the cluster configured over lunchtime on the same day, and by afternoon there was pure euphoria spreading throughout the studio as artists experienced the results.”
“Scalar, Avere’s channel partner, has been an awesome vendor. I think in part because their staff has direct studio experience, so they know the demands of production environments,” continues Whynott.
Today the Avere cluster, configured with 18TB of capacity, front-ends core filers, providing high-speed I/O to the ToonBox 5500-core render farm and responsive access to some 200 Linux-based artist desktops. The infrastructure supports an application environment that includes 3Delight high-performance rendering software, Autodesk Maya 3D animation software, The Foundry NUKE compositing, and other content-creation and editing systems.
Benefits: Performance, Fast ROI, Non-disruptive Scaling
Performance to Avoid the Crunch
“With the Avere solution in place,” continues Whynott, “productivity shot through the ceiling. Jobs that previously took hours to run completed in 20 minutes. And responsiveness to workstations dramatically improved. It’s hard to keep creative juices flowing when a model takes five minutes to load or rotate—Avere remedied that, enabling artists to work faster and benefit from greater interactivity on tasks like video scrubbing and push/pull adjustments.
“The Avere system effectively mitigates crunch time slowdowns, allowing us to deliver responsive I/O to a multitude of processes. When editors are doing video scrubbing, for example, several workstations can be demanding hundreds and hundreds of megabytes of data per second. At the same time, managers may be doing production reviews with clients while other users are running large render jobs in the background—at that point in a production the render farm is typically 100% utilized, 24x7. It’s a huge I/O demand, and Avere gives us the performance to manage it all.”
Whynott says considering administrative savings (high-performance, scalable I/O means IT does not have to play traffic cop) and artist productivity, the Avere solution delivered fast return on investment (ROI). “The Avere system paid for itself within weeks of deployment,” Whynott suggests. “That may sound overstated, but when you consider how quickly costs accelerate if artists can’t work or we miss deadlines, it’s a very real return. There are also the more intangible costs of compromised quality or scene complexity—those impacts are hard to quantify, but we know everyone benefits from not having to make concessions on the director’s vision of the film.”
Non-disruptive Scaling for All Kinds of Mischief
With The Nut Job 2 already in progress, the Avere solution empowers ToonBox artists to create plenty of new mischievous capers for Surly Squirrel and his buddies. Whynott says the ToonBox storage architecture enables fast, cost-effective scaling to meet any production demands. “Avere lets us separate performance and capacity scaling. We’re able to leverage existing storage investments and in the future can use low-cost disk as we need capacity. To boost performance, we simply scale the Avere—and we can do that non-disruptively. Our production schedule really limits maintenance windows, so it’s important to be able to accelerate performance at any time without interrupting mission-critical workflows.”