There has been a lot of hype recently on rendering in VR environments, mostly due to the upcoming releases of consumer virtual reality headsets. Light fields are an old graphics research topic, explained in great detail at https://graphics.stanford.edu/papers/light/. They have recently become popular because they allow pre-rendered scenes to be viewed from any angle and any position - ideal for VR, where other graphics techniques do not work for stereo display. Here is a recent video that demonstrates their use: https://www.youtube.com/watch?v=Oa7_zz5Vitg&t=56.

How do they work 

Light fields are very simple, they consist of a view of a pre-rendered scene from multiple angles. They are rendered by using the camera angle and position to lookup the closest pre-rendered pixel. Bilinear interpolation blurs the seams between views so the resultant view is smooth.

Demo

You can move around the scene below using W/A/S/D, C/Space and the mouse to look around. The controls are based on typical first-person shooters. A plane is viewed in-front, where a pre-rendered lightfield can be seen. As you move around, the light field will appear 3D.