Sep 30 2014

About “Mango Pipeline”

What is mango?
Mango is a development project to create a “pipeline in a box” like product, to most this will sound like a very bold claim, and let me assure you that it is. To attempt to live up to this claim i have structure the development road map of mango into different phases. Through out the different phases of mango development road map, we will be tackling the different aspects of the big picture Studio pipeline, mango is currently on phase1 and phase 2 should be released early  2015, with more phases already planned.

What to expect from the Mango Phase 1 Deployment
Phase 1 is the entry point to the essentials building blocks that are need to be setup up and expandable pipeline, While at the same time using a common case scenario for small studios that it hopes to improve at the time of adoption.   The generalist pipeline is a common case scenario that can greatly be improved by automating and tracking their most basic needs.

this I identify as…

  1. Ingesting plates
  2. Making work files
  3. Making 3d elements for compositors
  4. Helping Compositors Build and keep comps up to date
  5. Publishing different outputs from comp (for editorial or DI)
  6. Dailies pipeline, reviewing work and having a running history (even when people don’t want , or can’t use shotgun)

To cover those basic goals, there was core infrastructure that had to be covered like, robust directory structure, and a fast locally hosted database driven versioning system, but i won’t bore you with those details. Instead let’s take a look at some of the more visible components of mango phase 1.

Application Agnostic Pipeline
One of my main initiatives is to write mango apps so that they can run in multiple applications with out much effort. This is done by using an API that translate generic command like file open, close, etc to native command the apps hosting the application can understand, By doing this the Pipeline logic, and UI code of the apps can be share across multiple applications.
This allows me the following…

  1. I can easily integrate new applications with mango as they start trending as long as they have python pyside support
  2. I can handle all future software integrations my self
  3. I can continue to update and enhance the applications with out the need to maintain duplicate code.
  4. Reduces testing and bug fixing freeing me up to work on other aspects of the pipeline

This is all done in the hopes that you the client, doesn’t have to worry about Mango or the big picture of Pipeline.
The current list of application that have been integrated are

  • 3dsmax
  • Maya
  • Nuke
  • Hiero.

With all that said let’s take a look at some of the applications you will be inheriting and the role the play in the Mango Phase 1 the “generalist” pipeline.
Software Launcher.

  • Configures all artist software before launching the app for the artist and allowing to begin work
  • Serves as the primary system to distribute tools, and plugins based on simple text based profiles
  • Sets a repository (for tools and plugins) standard for all applications (3ds Max, Nuke, Maya, Hiero) so that a single person can maintain all packages
  • It’s fully integrated with deadline and the render pipeline so that slaves will all ways have he same set of tools that were used when a job was submitted
  • Minimizes the need for IT to manage and push plugin installations by automating the process via deadline and user interaction with the software launcher
  • New plugins and tools can be deployed sooner and with out affecting other shows by separating this new additions into new profiles

Plate ingestion and proxy creation

  • Quickly ingest image sequences to be tracked in the pipeline via stand alone tool
  • Batch create project, sequence, shot and plate ingestion via csv ingestion tool
  • I’m currently working on writing a Hiero exporter so that plates can be ingested directly from Hiero(More on this at the end)
  • Automatically create jpg proxy versions of the ingested files via (ffmpg)
  • Automate Shotgun project creation and and tasks generation for shots

Version Viewer(Stand alone and software integrated)

  • Quickly find any and all resources made in the mango pipeline
  • Quickly jump to the directory where a selected resource version is stored
  • view resource dependencies (what went into making the currently selected version)
  • Flip current version (Currently using Open source flipbook application Jefecheck, but can be expanded to use other flipbook apps)
  • Submit version for review in shotgun (think of this as submit to dailies)
  • Setup or edit any new or exiting projects sequences or shots
  • Access the plate ingestion tools


Work Flow  Tools

Workflow tools are the apps integrated in the artist application that dictate how we do things (Mostly create resources, do hand offs, and receive new and updated work).
here’s a list of the workflow tools we will be deploying with phase 1

Workspace manger (3ds Max,Nuke,Maya,Hiero)

  • Quick tool for setting your work shot context
  • Create New work files
  • Open any previously create work files,
  • Checkout work files created by other artist
  • Find out if someone made a new version of your file in your absence

Publish Work File (3ds Max,Nuke,Maya,Hiero)

  • Just like it sounds published the currently opened work file to be shared with the rest of the studio

Snapshot Work File (3ds Max,Nuke,Maya,Hiero)

  • This, like the Publish Work File, allows the artist to iterate his current file but without publishing the work file. Think of it a private iteration (stays in the user work area)

Render Pass Manger(3ds Max)

  • Proprietary Render Pass manger for 3ds max
  • Fully tested and developed in production
  • Focus on smaller learning curve (vs RPManager), and optimized for larger scenes
  • Fully integrated into the Mango Versioning System
  • Batch Render and Publish your passes as 3delements
  • Automatically sets Exr optimized settings
  • Fully integrated with deadline (render passes can be batched rendered and published on the farm)
  • Automatically handles dependency tracking

Comp Central(Nuke)

  • This is a shot resource catalog application built for compositors
  • Find, review and import any resources available for comp
  • Find out what “Read” nodes are out of date and update them
  • Rollback “Read” nodes to any previous version at any time

Element Publish (Nuke, and Maya)

  • Render and publish your nuke outputs as any of the following resource types (comp, precomp, paint, roto, plate)
  • Render and publish  your Maya Render Passes as 3delements
  • Fully integrated with mango versioning system,
  • Fully integrated with deadline(render passes can be batched rendered and published on the farm)
  • Elements can also be published locally
  • Automatically handles dependency tracking

All of this tools will be available too you from day one and will be deeply covered in the training time we will have together. Please also keep in mind that as subscription customer, i will be extending Mango, and maintain all the tools and workflows above. Which means that you won’t really have to maintain any of this tools and workflows in house, and the workflows will continue to expand  as other areas of the pipeline are tackled.

Support and Mango Tickets
All support for mango goes through the http://mango1.zendesk.com, here you can make an account and submit problems, bugs, request
as well as track the status of any outstanding tickets (you will receive emails as well).

Shotgun Toolkit vs Mango
Mango is meant to replace the need for Shotgun Tool Kit altogether, and while Mango and STK  seem similar in fashion, it is important to know that they operate on different concepts.
When i worked with the Shotgun Tool Kit (at Pixo) I was not pleased with the over all pipeline vision that shotgun had for the tool kit, which was basically none. I can only guess this is why it is a marketed as  “tool kit” and not a “pipeline”.
Being that the bundled apps, “empty shells” with “hooks”, are written in a generic way so that the user can implement their own vision. I saw no real value in them, at least not in the context of my vision which revolves mostly about user experience. The overcomplicated system that comes along with using the toolkit purely as a deployment tool and directory structure generation tool, was so complicated and cover-some, That’s why I decided to create my on kit. A simpler system that allowed me to develop quickly from scratch and that did not impose road blocks by making me funnel my vision through tools that were created with no  practical cases in mind. What you get from Mango is not a tool kit, it is an end to end pipeline and set of standards and practices that will be developed in several separate pre-planed phases. To me the vision of the product is more important the the code it’s self. This is the main reason why i felt Mango could fill a real need in the industry, “Pipeline in a box”.  Mango those optionally interface with shotgun, but limits it to project management (Planning, Bidding, task tracking, work review, notes, etc), and keeps all the resource tracking to it’s self. Ideally Shotgun in mango is just a module, meaning that if at some point something better, and more affordable comes along, we can switch without changing any other aspect of the pipeline, or you can chose to not use it all.

Hiero Integration
Based on the customer feed back that I’ve received, Hiero still has some serious issues that will keep it from fully replacing smoke and flames in the near future as a finishing tool.  On the other hand it seems to be a great tool for creating project structure and kicking out plates and other elements received from clients via EDL, early in the post production stage.
Because of this, the first Hiero Mango integration project I want to tackle is a Mango Exporter for Hiero.
the idea being that if we get assets (movies or imagess sequences) and an EDL from the client, we can quickly create the entire mango project structure and ingest plates (and create secondary resolutions) for the entire project via Hiero.
I would like to spend some time with you guys and see in what other ways you are using Heiro in the production pipeline, to  see how else we can integrate or workflow and maximize the Hiero Mango integration.
but the Mango Exporter for Hiero is something that i hope to start developing this week.

If you guys are interested you can check an outline of the Mango development road map, and the currently scheduled phases.

Mango Development Road Map

  • Phase 1 (Finished)
    • work files
    •  lighting
    • comp
    • dailies
  • Phase 2 (Hope to finish early next year)
    • Modeling
    • Rigging
    • Animation
    • Look Dev
    • Finalizing Stereo (Nuke, Maya)
  • Phase 3
    • Asset Library
    • Show Archival
  • Phase 4
    • Inter studio remote location work sharing
  • Phase 5
    • Editorial
    • Flame, Smoke
    • Hiero
  • Phase 7
    • Color pipeline
    • show,seq, shot LUTs
    • openColorIO (Hardware profiles)
    • lens Distortion Pipeline

Leave a Reply