Neo4j OEM Customer Success Story
Codex Digital: Boldly Enabling Complicated In Places Film Production Has Not Gone Before
June 24, 2014
Executive Summary
Codex Digital’s Backbone product is doing for film production and video workflows what Enterprise Requirement Planning (ERP) has done for other businesses – provide a customizable, integrated solution, for managing and editing video assets. It’s a first in the film industry and Digital Codex chose the Neo4j graph database from Neo Technology because it saved them roughly 50% of their development costs and time, and provided a flexible, scalable and fast solution that could be easily tailored to match the needs of any film or video project. The “snowflake” problem in the film business is that every project is different. This calls for a high degree of flexibility in the underlying technology. Neo4j’s robust ACID transaction capabilities (a rarity in NoSQL databases) have proven critical to providing industrial strength content and metadata management capabilities.
Codex Digital’s architecture is a modern modular SaaS solution that uses Neo4j as the integrating technology for multiple modules written in a wide variety of languages (C, Javascript, Python). It also integrates simpler data elements such as lists from MySQL, and is currently in use at major film studios.
Background
High-end digital film production is a complicated Big Data problem, for which the Neo4j graph database has proven to be an excellent fit. Codex Digital is an innovator in managing and automating digital video workflow. Its expertise in film is deep and its various technologies have been used on hundreds of productions worldwide, including World War Z, The Host, Iron Man 3, The Wolverine, White House Down, Trance, The Internship and the highly successful HBO multi-year series, Game of Thrones.
To give an idea of the data requirements that need to be managed: one season of HBO’s series Game of Thrones requires 160 TB of on-set storage. Editing and production is always done against huge time and budget constraints. The importance of efficiently filming, editing and managing digital assets can therefore not be underestimated. A robust solution saves producers’ budgets from escalating out of control, and minimizes the occurrence and impact of editing errors, through management of versions, well-managed backup, and reuse of past images from live action, as well as reuse of CGI (computer generated imagery and special effects, green screen images).
Sizing the Problem
Anyone who has attempted to manipulate a single image in a program like Photoshop or Lightroom understands how easy it is to lose an image, a particular version of an image, a back-up, or one of the many variants one needs to spin out, e.g. various image versions, including small images that won’t break an email system’s file limits.
Big budget video production is the same problem multiplied by a million. Literally. A two hour movie is filmed at 24 frames per second. Each final frame may draw upon 25-30 takes. Different versions may be produced, and each frame includes metadata about actors in the frame, equipment and lenses used, location information, frame rate, time of day, product placement, staff on the shoot, and more. Backup status is typically based upon at least three backups. And because mistakes happen, it’s important to enable rollback of metadata and the underlying content for both the central repository and local repositories. Codex Digital Backbone enables complete rollback capability for all versions.
There is also the “snowflake” problem, as it is called, in the film industry. Every film/video project is done differently in the same way that every snowflake is different. As a result, an inflexible and difficult to modify technology won’t work in film production. Complicating the problem further, unique business entities are set up for each video project. So taking data from one entity to another must be easy to be effective. That’s where the Neo4j graph database comes in. It makes it easy to manage this complex metadata, also making it possible to see the structure of the data in a much more intuitive way, says Matt Walker, CTO for the company.
Why Neo4j
Codex Digital’s Walters chose a graph database over other types of databases for its speed, scalability, flexibility and ease of modification. Neo4j’s robust transaction capabilities were particularly important for ensuring consistency and reliable rollback in the database. The open source nature of the Neo4j database appealed to CTO Walker, who had previously used open source MySQL and RedHat Linux. But he selected Neo Technology for the support they offered, ensuring that if a problem occurred, he could get it fixed quickly rather than waiting on someone else’s timetable.
Digital video production often means managing data flows from multiple geographic locations, tracking metadata about how, when and where the video was captured, combining images e.g. with live and green screen integration, managing color spaces, editing lighting and color settings, converting between multiple codecs, preparing videos for different video broadcast standards. A successful solution must support multiple video standards, deal with color spaces, track versions, track checking in and out of video component and audit information around each component of video. It’s very similar to software development in many ways. Versioning, security and audit trails are key.
And because filming takes place in multiple locations, Codex Digital needs to distribute its software to multiple physical locations, along with the multiple versions of content, providing for rollback of video content and metadata when mistakes or miscommunication occurs. Neo4j’s graph database is used both in the Production Server product and also deployed in a cloud-based service for managing distributed assets and new content coming in from other suppliers and other filming locations.
Latency and Scalability
One of the things that has delighted Walters is that his developers are able to pursue an agile strategy. “What Neo4j has done for us is that it allows us to develop each module of our application in the way that is best. We don’t have to worry about performance. Neo4j just keeps delivering the response times we need. We are not trapped in the more common problem of agile developers being held up by the slow turnaround from using a relational database. And of course, we don’t have the problem of a relational database grinding to a halt because we are doing too many table joins [combining information from different relational tables].”
Security
Security is a big issue in the film business and Codex Digital has been careful to integrate security into its application at the API level. Film producers require that tight controls over access and modification of video content and meta information is key requirement for retaining control of versions and avoiding illegal distribution of content.
Extract from the article Crown Jewels: HBO’s ultra-successful, epic fantasy Game of Thrones, using using ARRI Alexa cameras, has utilized Codex tapeless workflows for three complete seasons, Codex Digital Magazine, 2014, Issue 4, p.30. (Reproduced by permission of the company).
“The fourth season of HBO’s hugely popular Game Of Thrones is on the horizon. Based on a series of best-selling novels by George R.R. Martin, and executive-produced for HBO by David Benioff and D.B. Weiss, the show is set on the fictional continents of Westeros and Essos, as a decade-long summer gives way to a harsh winter. It chronicles noble dynasties at war and peace, interweaving plot lines covering social hierarchy, religion, mythical creatures, loyalty, corruption, sexuality, crime and punishment.
Game Of Thrones has earned widespread acclaim and won numerous prestigious accolades, including Primetime Emmy, Golden Globe and Television Critics Association awards. Cinematographer Jonathan Freeman ASC won the 2014 ASC Award, in the One-hour Episodic Television category, for his work on the episode Valor Dohaeris.
Based out of studios in Belfast, Northern Ireland, Game Of Thrones has variously shot on locations in Croatia, Iceland and Morocco, as well as around Northern Ireland. The show’s first season, photographed during 2010, recorded ARRI Alexa’s HD output to tape, but HBO went completely tapeless for season two. The second, third and fourth seasons of Game of Thrones – 30 episodes in total so far – have all used file-based recording systems and near-set technology from Codex.
The Codex workflow was designed by Greg Spence, one of the show’s producers, HBO technology architect Steve Beres, and former associate producer Jonathan Brytus. It includes ARRI’s Alexa cameras and Codex Onboard Recorders and digital lab technology.
“Adopting the Codex workflow was quite a milestone. It removed the time-consuming ingest and transcoding processes that tape involves from the equation, and has saved a huge amount of money,” says producer Greg Spence. “Over the course of the last three years Codex has proven to be consistent, robust and highly mobile. We love it.”
Typically, live action sequences are shot at HD, whilst ARRIRAW is used for greenscreen and VFX work. Camera data is captured by Codex Onboard Recorders with 14 Codex Capture Drives in cycling rotation. Full capture drives are delivered to a dailies colourist (located near set) who applies a dailies colour-timing pass using another Codex Onboard Recorder and Colorfront’s On-Set Live!. The capture drives are then passed on to the dailies team, which employs Codex digital lab technologies to prepare deliverables for editorial, VFX and review purposes, and to archive to LTO-tape. The capture drives are then returned to production and the sequence begins again. HBO has employed near-identical Codex workflows in the production of two TV movies, Hemingway & Gellhorn and Game Change.
A key factor of moving into tapeless production with Codex is that it allowed Game Of Thrones to process its own dailies. The show is shot primarily in Northern Ireland, far removed from the type of post production infrastructure a show of this scale typically requires. With 150TB of storage on-set in Belfast, a whole season can be kept ‘live’, and a small dailies team of three or four people can handle all of the dailies coming from Game Of Thrones’ multiple units. When shooting in Belfast, and in a distant location at the same time, a single Codex lab operator can handle the distant dailies, piping Avid and viewing media back to the rest of the team in Belfast, or on location, helping the production to stay on schedule for dailies and editorial.
The Codex workflow has also provided an efficient way of managing colour correction information throughout the production and post production processes. Colour decisions made on the set by the cinematographer, or by the dailies colourist, are retained and associated with specific camera files, but not “baked in.” The final colourist, and others involved in the post chain, can enjoy the advantages of working with ungraded Alexa source media, while also having immediate access to colour decisions made during production.
This system can retain more than just colour decisions. Virtually any kind of data that a production deems useful can be entered into the system, associated with individual files, and retrieved at later stages of production and post.
“The Codex workflow has simplified many aspects of final post work,” says Spence. “The old days, of sending SR tapes to a post house for transcoding and online editorial, are long gone. With the Codex workflow, the show’s editorial team can perform many of the tasks required for online editorial themselves and simply submit a stack of files, organised in show order for routine assembly.
“Ultimately, the Codex workflow has netted significant time and cost savings to the production without compromising production quality. It has facilitated the capture of rich, high-quality imagery, while eliminating the burdens and costs of tape and processing, along with the physical limitations of tape.”
“CODEX HAS PROVEN TO BE CONSISTENT, ROBUST AND HIGHLY MOBILE. WE LOVE IT.” says
Greg Spence, Producer, Game Of Thrones who has employed the services of several leading cinematographers, including:
- Anette Haellmigk, Jonathan Freeman ASC
- Matthew Jensen ASC, Alik Sakharov ASC
- Martin Kenzie, Robert McLachlan ASC
- Marco Pontecorvo, Kramer Morgenthau ASC
- Chris Seager BSC, Fabian Wagner, P.J. Dillon
- Sam McCurdy BSC, David Katznelson BSC, and
- David Franco