DUXSoft : Project SPARX

In the last decade or so, the Animation, VFX (Visual Effects) and gaming

arena have seen some rapid progress. Each of these technologies has moved out of

the computing laboratories to become a full-fledged industry in itself. This

progress has also brought with it some new challenges.

Along with the regular management challenges that most organizations face,

these studios have dealt with challenges like Pipelining, Render Management and

Digital Asset Management.

The production process in a studio is usually very serial and iterative in

nature. Due to this, the work flow is referred to as the 'Pipeline' by the

industry. The problem is that, even though Animation, VFX and Gaming have a

common base in 'Computer Graphics', yet each of these industries follow

different production practices. Not just that, unlike other industries, each

project itself may require a different workflow. To complicate things further,

due to the highly artistic nature of the work, the skill set required may also

vary from task to task and not just from project to project. An improper

workflow could derail the project at it crucial stages.

Rendering is the process of creating images of a movie scene and is a highly

compute intensive task. These individual images are sequenced together to form a

movie. One second of a movie scene is made up of 25 very high resolution frames,

while one second of television scene is made of 50—60 normal resolution frames.

Depending on the complexity of the scene, rendering a single frame may take up

to hours.

But fortunately this is a process that can be easily parallelized. If we have

enough resources, each frame can be rendered by different computers. For

example, if one has 100 Computers, 4 seconds or 100 frames of the scene can be

rendered in the same time that it takes to render one frame. Such an array of

computers used for rendering is called a 'Render Farm'. Managing a Render Farm

is a challenge, as one has to deal with multiple jobs (scenes) and multiple

frames being rendered simultaneously.

Has over 14 years of experience in infrastructure development and technical support. Possesses vast knowledge of web app servers and mailing systems on UNIX and MS platforms, and is experienced in planning and management of LAN/WAN, etc. Indivar Nair, VP — Technical Services Q What challenges were you facing before the deployment? SPARX aims to be a one stop solutions to problems one faces in a studio. Along with regular management challenges that most organizations face, studios have to deal with challenges like Pipelining, Render Farm Management and Digital Asset Management. SPARX aims to automate the complete production life cycle of Digital Content creation, right from requirement capture to final delivery and archival. It will help studios dynamically change their production practices without having to undergo a major change management process with every project. Q What according to you is the USP of this project? SPARX Render Management Module uses Sun Grid Engine in the background. Grid Engines are built from ground up to handle thousands of jobs (running on as many nodes) simultaneously. This makes it a very powerful tool for Render Management too.We removed the complexity of using a Grid Engine by giving it a very easy to use, intuitive File Manager type Web Interface.

A huge number of images, scene files, movie clips etc. are generated and used

during a project. Due to intellectual / artistic value attached to it, these are

known as Digital Assets. Storing these assets securely as well ensuring that

they can be retrieved easily when required is another challenge that most

studios face.

This is the problem space that DUX Soft tries to address using SPARX, a

software application that provides end to end production support for studios of

all types and sizes. SPARX aims to automate the complete production life cycle

of Digital Content creation, right from requirement capture to final delivery

and archival. SPARX will help studios dynamically change their production

practices without having to undergo a major Change Management process with every

project. And to enhance the speed of the rendering at the backend of SPARX a

mammoth 100 Core Render farm is sitting.

The Implementation



Right from the start, SPARX has been developed keeping in mind 'ease of use'

for the end user. Therefore the whole system has been designed like a file

system explorer. A graphics artist intuitively knows what to do as soon as he /

she logs in. Over and above this, a context sensitive help screen guides the

user on every step. SPARX web interface has been developed on PHP 5.2.x and uses

various additional plug-in to enhance its features. The web pages have been

written in plain HTML, with very little JavaScripting and AJAX support. It has

been tested to work with most new browsers like IE6, IE7, Firefox and Safari.

SPARX uses Sun Grid Engine 6.1 in the background for its job scheduling and

management activities. The excellent features that Sun Grid Engine provides

allows one to prioritize job queues, pause jobs, manage compute loads, offload

an activity from one compute node to another, have detailed accounting of the

jobs completed, etc. SPARX can be integrated with Microsoft ADS or any other

LDAP service to provide a common authentication and single sign-on to any SPARX

module. At the backend SPARX uses four Dell PowerEdge 2550 servers for ADS,

Storage and SPARX Master Node. For the render farm they have used 25 Dell

PowerEdge 1950 for Render Farm with two dual core processors which essentially

mean 100 cores. The nodes runs OpenSuse 10.3 as the operating system which is

stripped down to an extent that they have minimal OS overhead on the processors.

And as this render farm is connected to the website from where the customers can

initiate process at any time, they needed to keep all these servers running

throughout. But to save power what they have done is that, they have created a

script which turns on all the 25 nodes when any job is initiated. And once the

job is finished all the nodes again goes to shutdown state automatically. This

simple trick helps them to cut down power consumption in a big way.

The Impact



This kind of a outsource workflow management for rendering will essentially

help the small and medium companies which are into this space. Now they can

concentrate on the core job of creating the animation footage instead of wasting

a huge amount of CPU and Human time in rendering the created footage without

even investing a bomb on deploying their own render farm.