3D Photos Momentum 360

Facebook’s 3D Photos feature, initially teased in May, is now fully accessible. This innovative feature transforms ordinary pictures into immersive 3D photos, offering a unique visual experience. Research has shed light on the underlying technology, revealing impressive capabilities.

What are 3D Photos?

When you interact with 3D Photos on Facebook, such as scrolling or tilting your phone, the perspective shifts, creating the illusion of looking into a miniature diorama. This effect works effectively with various types of images, including portraits, landscapes, and panoramas.

The Origin of 3D Photos

Interestingly, the concept of 3D photos didn’t stem from enhancing snapshots but from the ambition to democratize virtual reality creation. Previously, it was challenging for an average Facebook user to create a 3D model. While panoramas and 360-degree images on Facebook offered some immersive qualities, they lacked the transformative depth Facebook 3D Photos now provide. The addition of depth to news feed images was a strategic move to enhance user engagement.

How 3D Photos Function

The technology behind 3D Photos involves comparing the foreground and background elements of an image. This data, processed by Facebook’s custom software, creates a dimensional effect as users scroll through their News Feed. The experience is further enhanced when viewed on VR headsets like Oculus Quest, Oculus Go, or HTC Rift, where the visuals dynamically respond to movement. While it might seem like a complex 3D model, it’s essentially a sophisticated illusion of depth.

Evolution of 3D Photo Technology

Early methods of creating 3D effects relied on analyzing parallax—how objects shift at different distances when the camera moves. This process, often CPU-intensive, has been advanced by modern cameras featuring dual lenses, mimicking human eyes. These dual cameras capture simultaneous images, enabling more accurate depth mapping and reducing the computational load. The resulting “depth map” encodes the distance of each element within the frame.

When a phone captures a pair of images using its dual cameras, it calculates a “depth map.” This map is essentially an image that records the distance of everything within the scene, based on the stereo pair of images.

Smartphone Manufacturers’ Role

Many smartphone manufacturers are integrating dual-lens systems, particularly for portrait modes, to generate rough depth maps. While these maps can indicate proximity through color gradients, they often lack a consistent scale, making it difficult to ascertain precise distances and stitch images together effectively.

Major players like Google, Samsung, Huawei, and Apple incorporate features that simulate background blur. However, the depth maps generated by these methods are often not to scale. This means that while colors might represent different distances, there’s no universal standard, complicating the process of creating consistent 3D visuals.

Facebook’s Innovation in 3D Photography

Facebook engineers addressed the scaling issue by developing a system that captures multiple images as the smartphone moves. Each second, a new image and its corresponding depth map are added to a collection. The phone’s motion detection system analyzes these depth maps alongside subtle camera movements to create a unified, dimensional representation. While the exact mathematical processes are proprietary, the outcome is a more robust 3D effect.

Instant 3D Photography Achieved

The process of creating a depth map is remarkably efficient, taking approximately one second per image. This speed has led to the tool being dubbed “Instant 3D Photography.” The collected depth maps are then used to generate a 3D mesh, akin to a digital landscape. This mesh is further refined to correct any visual artifacts or “ripped” edges, ensuring objects appear at their correct depths and move convincingly.

Despite the impressive diorama effect, the foreground elements might appear somewhat flat, as would be expected in a straight-on photograph where side profiles are not visible.

The Power of “Hallucination” in 3D

The final stage involves convolutional neural networks, which “hallucinate” or intelligently fill in the missing portions of the image. Similar to content-aware fill in photo editing software, this process uses surrounding information to accurately recreate textures and predict object shapes, ensuring smooth transitions and natural-looking details, such as hair or skin tones.

This technology allows 3D photos to be viewed as immersive dioramas or in virtual reality, adapting realistically to changing perspectives. The feature is accessible without requiring new software or plugins. The subtle perspective shifts as you scroll through your feed draw attention to these photos, encouraging interaction. While some visual oddities can occur, the overall experience is compelling.

Creating 3D Photos: What You Need

To begin creating 3D photos, a smartphone with two cameras on the rear is generally required. While dual-lens smartphones offer the best experience, other devices and various apps can also be utilized for 3D photo creation.

After capturing images with a dual-lens phone, you can upload them to Facebook. The option to select “3D photos” can be found via the three dots in the upper right corner of a new post, potentially leading you to your device’s Portraits folder. You can add captions before posting. On Android, the depth map option may be available within the camera app. Simulating background blur can help mask artifacts from 3D processing; experimenting with different blur levels is recommended.

Tips and Tricks for Great 3D Snaps

While these are guidelines, experimentation is key. When shooting, focus on finding static objects like trees, flowers, or buildings. 3D photos are excellent for showcasing miniatures, models, and toys. While people and animals can be captured, minimizing movement is crucial for accurate results.

Avoiding Common Pitfalls

Low light conditions can hinder the camera’s ability to capture depth accurately, leading to estimations and incorrect results.

Optimal Depth Calculation Range

The most accurate depth calculations typically occur between 18 inches and 10 feet, though some cameras may extend this range to 30 feet.

Finding the Perfect Angle

When shooting geometric shapes, consider angles that best accentuate the 3D effect.

Handling Fine Details

Intricate details like hair strands or thin cables can be challenging for depth calculations due to the need for sufficient visual volume.

Ensuring Perfect Focus

For landscape shots where depth is key, ensure there’s an object in the foreground, even close to the ground, to establish depth.

Maintaining Stability

Both subjects and cameras should remain still during the capture process to prevent inaccurate depth data and processing errors.

Leveraging Distance and Background

Distinct foreground and background elements enhance the 3D effect. Aim for a clear separation of visual layers, ideally two or three distinct planes of depth.

Dealing with Transparency

Transparent surfaces like glass and water can be problematic for depth cameras, leading to confusion and less accurate results, similar to fine geometric details.

The Current State of 3D Photography

While numerous apps offer filters for 3D effects, Facebook’s 3D Photos provide a level of detail and realism that is currently unmatched. This technology is at the forefront of creating more natural-looking photos and videos. Competitors are also developing similar technologies, which bodes well for consumers who can expect improved products and services in this area.

Currently, creating 3D photos typically requires a dual-camera device, though anyone can view them. The art of capturing memories in 3D is still evolving, and mastering it takes practice. Don’t be discouraged by initial attempts; persistent experimentation with light, composition, and subject matter will lead to success.

Frequently Asked Questions