Green Screen vs LED Wall for Virtual Production: A Technical Comparison

Virtual production has revolutionized the way films, TV shows, and commercials are made. It allows filmmakers to create stunning visual effects and realistic virtual environments in real-time, providing unprecedented creative opportunities. The two most used methods of virtual production LED wall virtual production and green screen virtual production.

I realized recently that there’s been quite a bit of talk about LED wall virtual production, but not much being said about green screen virtual production.  This makes sense as LED wall technology has gotten a lot of attention recently with its use on popular shows like The Mandalorian and let’s face it – it’s just cool as hell to see in action. Still, we shouldn’t forget about its humbler counterpart, the green screen. In this blog post, I’ll explore the key differences between these two techniques, including the use of camera tracking and lens encoding in both.

Green Screen Virtual Production

Green screen virtual production, also known as chroma keying, has been a staple in the film and TV industry for decades. In this method, actors perform in front of a green screen, which is later replaced with virtual backgrounds in post-production. The virtual backgrounds are created using computer-generated imagery (CGI) or pre-recorded footage.

There are two ways of incorporating these virtual backgrounds into a green screen shoot and unlike LED wall virtual production, both are accomplished in post.

It used to be that this technique offered no way of creating real world parallax, causing the background to feel flat and at times unnatural. This is especially true for shots that require some type of camera movement.  However, recent technical advances have made it possible to create virtual backgrounds for greenscreen that can incorporate the camera’s parallax and focal distance making green screen a much more powerful tool than it once was!

Matchmoving:

Matchmoving (aka motion tracking or camera solving) is a VFX technique that allows computer graphics and live action footage to be composited while maintaining the correct position, scale and rotation relative to the camera footage.

Using special compositing software such as After Effects or Nuke, compositors identify the most trackable areas in the footage to place digital “markers” which the software then uses to mathematically calculate the camera’s position in the real world, then create an identical virtual camera to reproduce those moves in the virtual world.

This technique is often used to add-in digital assets, effects and even CGI characters overtop of live footage. For the purposes of green screen replacement, tracker markers are placed on the physical greenscreen and then tracked in the software as described above.  Later the footage is chromakeyed and composited over the virtual background.

This method is relatively inexpensive and doesn’t require any additional camera hardware or stand-alone tracking systems.  It does, however, require quite a bit of work in post and doesn’t take into account any lens changes to the camera’s aperture or focus. 

Camera Tracking Systems and Lens Encoders:

Clearly one of the main challenges of green screen virtual production is the need for precise camera tracking and lens encoding. This data is crucial for matching the virtual background with the live-action footage and ensuring accurate perspective and depth.

To achieve this, specialized hardware and software are used. For example, optical or mechanical trackers can be attached to the camera to capture its movement and position, while lens encoders can be used to capture lens data. This data is then fed into the virtual production software, which uses it to accurately composite the virtual background with the live-action footage.

This is a more expensive option then matchmoving but it also provides a more comprehensive and precise composite. 

LED Wall Virtual Production

LED wall virtual production, also known as in-camera VFX or stagecraft, is a more recent technique that has gained popularity due to its real-time capabilities. In this method, instead of using a green screen, a large LED video wall is used as the backdrop, which displays the virtual backgrounds in real-time. The actors perform in front of the LED wall, and the virtual backgrounds are rendered in real-time based on the camera’s movement and position.

One of the main advantages of LED wall virtual production is the elimination of the need for extensive post-production work. Since the virtual backgrounds are rendered in real-time, filmmakers can see the final result on set, allowing for immediate feedback and adjustments. This can save significant time and resources compared to traditional green screen virtual production, where extensive post-production work is required.

In terms of camera tracking and lens encoding, LED wall virtual production also requires specialized hardware and software. The LED wall needs to be precisely calibrated to match the camera’s movement and position, and the virtual backgrounds need to be rendered in real-time based on the camera’s perspective. Lens encoders are also used to capture lens data for accurate depth of field and other optical effects.

Furthermore, not all cameras are capable of working with real-time graphics. You will need  to have a camera with a global shutter and genlock capabilities in order to sync properly with the LED wall. 

KEY DIFFERENCES

Conclusion

Green screen virtual production and LED wall virtual production are two popular methods of virtual production, each with its own advantages and challenges. Green screen virtual production has been widely used in the industry for many years and requires specialized camera tracking hardware and extensive post-production work. On the other hand, LED wall virtual production provides real-time rendering of virtual backgrounds and immediate on-set feedback.

While LED wall virtual production is clearly the most desirable option, it can be cost prohibitive to smaller productions with less of a budget or limited resources.   In that case, greenscreen virtual production is a great option and absolutely worth considering if you have big ideas but a small budget.