Live streaming is quite practical for those who are unable to attend various events, but it also has the potential to be an alternative to TV and even change the way people watch shows and events. Compared to VOD, live streams often have a chat function, allowing spectators to interact with one another and the broadcaster. As the internet becomes more and more crucial to modern-day society, TV will eventually be phased out and replaced with online media. As live streaming continues to improve, it is likely people will adapt to watching live shows and events over the internet as opposed to traditional television.
Before diving into 3D live streaming, it’s important to understand the benefits of live streaming video in general. For several years, students, sports fans, music enthusiasts, and video gamers have been using 2D live streaming to watch events from all over the world. One of the most popular uses of live streaming is for gaming, particularly online competitive gaming. Tournaments often feature games with little to no delay between matches, and in some cases, the player not actively in a match is required to watch the other to ensure both are playing fairly. Live streaming has made it possible for players to broadcast matches to each other and watch tournament matches without having to be directly in front of the TV. Meanwhile, sports fans and music enthusiasts often use live streaming to watch concerts, games, and other events that are not readily available on TV.
When it comes to live video streaming, it doesn’t get any better than 3D streaming. Think about it, when regular HD streaming became available, it was highly favored over standard definition, and the same was said for color television over black and white. Human beings always strive for the best, and streaming video is no different. 3D streaming is still a relatively new technology, but many expect it to be the future of streaming video. This article will break down what 3D streaming is and how it’s already being used today.
Benefits of Live Streaming Services
– High stakeholder acceptance: Service providers for online platforms as well as for enterprises define 3D live streaming as a technology that can help to get the actual feel of physical world interactivity while being online. – According to Forbes, 80 percent of audiences would rather watch live video from a brand than read a blog, and 82 percent prefer live video from a brand to social posts. 67 percent of audiences who watched a live stream purchased a ticket to a similar event. – Cost effectiveness: This is beneficial specifically for providers of online training and simulation-based games. 3D environments can be costly to make, and the return on investment in terms of improved learning through further interactivity has been long-winded. Nowadays, costs can be saved due to the reduction of errors made through live streaming rather than updating content and sifting through data for errors made in an offline session.
Evolution of Live Streaming Technology
Live streaming was first introduced in the early 1990s, but it was not very viable until the mid-2000s. This was due to many reasons such as limited internet connectivity, slow internet speed, processing power, and high costs to operate live streaming technology. Early internet streaming depended largely on buffered audio and often used modem connections to broadcast audio data in real time. Audio-only internet broadcasting was also prevalent and was known as streaming audio. This was before telecast data casting was viable for the internet because some programs required special audio and video cards.
The technology of live streaming proceeds to grow at a fast rate right now. This growth has been based on the rise of other internet technologies, and because live streaming has become widely popular, more people would rather watch it on the internet. The latest effort to produce a latest technology live streaming event systems happens to be 3D live streaming. This latest effort offers new and interesting ways of watching live events on the internet and is a great example of how far live streaming technology has come.
The Rise of 3D Live Streaming
3D technology has been around for many years now, but was very slow to take off with the public-at-large due to the high costs associated with the technology which required 3D glasses. Understandably, cost, lack of interest, and the need to wear something on your face meant that live 3D events and products have been relatively slow to come to market. The decision of the Blu-ray Disc Association to stipulate that all Blu-ray players have to be 3D capable has been a real turning point, together with the announcement of 3D channels from Sky and Virgin that are expected to begin broadcasting in the UK in 2010. The backing of some major industry players will potentially see yet another format war reminiscent of the HD-DVD vs. Blu-ray battle, but the outcome would enable anyone with a 3D capable TV to be able to view 3D content without any extra cost. While 3D TV is one thing, the real capabilities of live 3D broadcasting over the internet have the potential to blow 3D TV out of the water.
Understanding Hybrid Live Streaming
With IC’s services, Hybrid Live Streaming can be shown in two of our virtual event solutions. High-end pre-recorded content can be streamed through our Virtual Events platform, 3D content creation from the material can be added for a more immersive experience. Or it can be streamed through SecondLife where a live element can be brought to the audience by using avatars to interact with the environment.
Hybrid events that take advantage of both pre-recorded and real-time elements let organizers take full advantage of the on-demand nature of the internet while fostering a social environment that is traditionally found in live events. Pre-recorded presentations and panel discussions can be streamed live, giving the audience a chance to engage with the speakers through real-time chat or Q&A sessions. These pre-recorded events can later be viewed on demand, usually with better video quality. This is perfect for product presentations or launches, where the product manufacturer can gauge the reaction to the product before deciding on a bigger launch initiative.
When a person hears the word ‘Hybrid’, the first thing that comes up in their mind is ‘a combination of two things’ or ‘something that has two different elements’. That is exactly what Hybrid Live Streaming is. It is a mixture of pre-recorded and live elements. The content is first filmed or recorded in a professional environment, then edited and prepped for the live broadcast. This ‘pre-recorded’ material is shown to the audience with a real-time ‘live’ element overlaid. The event is then archived and made available on demand. With Interactive Connect’s 3D Live Streaming services, Hybrid Live Streaming can be taken to another level – more on this later.
Definition and Concept
A key feature of hybrid live streaming is the ability to integrate the online audience with the on-location event. This can be achieved in many ways. For example, a live Twitter feed can be displayed at the physical event, showing real-time comments from the online audience. On the flip side, comments and posts from the online audience can affect the live event. This was recently demonstrated by AKQA and Unilever at their Cannes “Cannes-Ignite” event. They set up a live streaming control room at the event location, with an operator whose job was to keep an eye on the incoming online audience comments and questions. This interactivity resulted in a last-minute presentation change, in order to address the questions of the online audience.
Hybrid live streaming refers to the use of live streaming technology to combine internet live streams with the physical (or virtual) live events. The concept of hybrid live streaming is to create an ecosystem where audiences who cannot attend the physical event can still participate by watching a live stream of the event. The live stream audience can engage with the event through media-rich interactions and collaborate with the attendees at the physical event.
Advantages of Hybrid Live Streaming
With the push to bring 3D content to the consumer, the professional market will also see an increase in 3D content creation to support the 3D consumer market. This will include 3D television programs, commercials, and movies. These content creators will need a full range of 3D monitoring tools to ensure their 3D content is of the highest quality. The most dominant form of content creation is still done on a computer. This can be from basic video editing to high-end computer graphics using professional workstation applications. The content is then stored as files and streamed into a video encoding application for broadcast. Professional workstation graphics will have a 3D accelerated OpenGL view of the content. All the 3D content will need to be monitored and QC checked for errors in a manner similar to the monitoring of 3D broadcast video. The next generation of 3D monitoring tools will allow a seamless and consistent quality check of 3D content from creation to broadcast.
When monitoring 3D content for quality and errors, it is important to view the video in 3D on a 3D display and check for 3D-specific issues. This will require new 3D monitoring tools to view the video on the display as separate left and right segments or as a combined and frame-compatible 3D video. The monitoring tool must be able to distinguish the 3D video and put it back into an alternate frame format for viewing on a traditional 2D monitoring display. This will allow a simple way to view 3D content and check for errors using a single or dual monitor 2D system. A 3D monitoring GUI will be needed to control the different monitoring modes and view the 3D video on a 3D or traditional 2D display.
There are a few types of 3D television, namely active and passive. Both these types of displays produce 3D by showing the left eye and right eye views at alternating intervals. This requires the encoding system to take both video streams and using some form of time synchronization, package them into a frame-compatible video stream or show the left and right eye video as separate segments. Step one in monitoring 3D content is the ability to view the 3D video and ensure the left and right eye video is synchronized and playing at the same time. This can be done using a dual output 3D-capable device. Connect one output to the left eye video and the other to the right eye. This gives a simple and low-cost solution to view the 3D video and check for synchronization. Any form of anaglyph output can also be used. Dual anaglyph monitors can be expensive, so another option is to use traditional 2D monitoring with the anaglyph glasses. The anaglyph output is the most cost-effective solution but may not provide the most effective way to quality check the 3D video.
Hybrid live streaming system is a powerful addition to traditional video encoding platforms. It offers the ability to mix live and file-based video in the same playlist. The content can be combined into the same streaming video and the event can be streamed live. This gives the system an opportunity to tailor the viewing experience to the end users and the viewing device. This mixed content playlist can also be recorded to disk while the live event is occurring. This recorded content can be time-shifted and streamed as on-demand content at a later time. This gives the content a broader reach and a longer lifespan. With the coming wave of 3D television and other new viewing devices, 3D monitoring tools will be needed to ensure the highest quality experience.
Applications of Hybrid Live Streaming in Singapore
Educational seminars in Singapore can benefit similarly to events, drawing in larger audiences than would be possible with physical-only sessions. 3D live streaming can bring a new level of interactivity to the experience through online Q&A sessions and breakout groups that make the virtual participants feel more involved. Medical conferences, in particular, can use these technologies to produce very realistic virtual experiences. High-quality 3D streaming with VR can simulate the experience of attending a live surgery, offering an invaluable educational experience without risk to actual patients.
Hybrid live streaming can be used across a variety of industries to bring virtual and live content together. Event marketers in Singapore can utilize hybrid streaming to open up their event to global audiences. In an increasingly virtual world, many events are struggling to attract physical attendees. By offering virtual attendance options, event hosts can reach a much wider audience. Additionally, those who are unable to attend physically due to travel or financial constraints can still participate in the event. As a real-life example, the Singapore Symphony Orchestra has begun live streaming their concerts in 3D audio through an online virtual world. This allows virtual attendees to have a unique concert experience, providing true immersion with the event. They hope that through this virtual option, they will be able to reach new audiences both locally and abroad.
Challenges and Considerations for Hybrid Live Streaming
The truth is that on-site live event capture is still a very costly proposition. Capturing on-site video for remote viewers and on-site viewers are two separate activities. Remote viewers only need to see what is happening at the event itself. Major on-site live event captures involve a constant change of venue to catch the activities of the event, requiring multiple videographers focusing on particular activities, interview sessions and sometimes studio spaces to discuss event activities. The cost of trying to replicate this onto video for remote viewers can be so high, that an organisation might still opt to only capture on-demand content, thus defeating the purpose of the initial hybrid live streaming idea.
Hybrid live streaming activities involving single on-site event with remote viewing parties may hit an unexpected cost barrier. Traditional live streaming of an on-site event has always been an expensive proposition. High quality encoding and sufficient bandwidth to reach a large audience has often been financially out of reach for smaller organisations. Yet the growing availability of cost-effective live streaming solutions has led those same organisations to gravitate towards hybrid live streaming, with the assumption that live on-site broadcasting can be a simple extension of remote live broadcasting.
Quality and reliability of video streams is always a consideration for any type of streaming activity, but it may impact hybrid streaming to a greater degree. Viewers compare live broadcasts to traditional broadcasts and their threshold for quality issues may be lower, as live internet broadcasts are often an enhancement in availability over an event that would not be televised at all. Conversely, internet viewers are more forgiving in quality for on-demand events as long as they are free to download. Since the goal is to provide a large portion of on-demand viewing with an on-site event, broadcast quality is very important in this type of hybrid event.
The primary challenge of hybrid streaming is unrealistic expectations. The event may be live, but the show is a condensed replay of the original broadcast in order to accommodate a worldwide audience. This option offers compelling content to the viewer, and it may provide a second revenue stream for the content provider. Unfortunately, viewers typically expect full DVR control for these events. They want to watch any part of the event as if it were fully on demand, yet they expect to finish viewing just as the live event concludes. At that point, they want to watch the recorded content catch up to the live broadcast. This means that viewers want to seamlessly switch between live and on-demand content during the event. Today’s streaming players and platforms were not built to support this use case.
Exploring 3D Live Streaming Services
3D live streaming works similarly to traditional 2D live streaming. It captures video with a 3D camera. The video then goes through a process of video production which involves adding effects, editing, and others. After that, the video is encoded. The only difference is the viewing of the video. When the viewer wears anaglyphic glasses, it allows them to see a 3D video. This differs from someone who has created a 2D live stream to a 3D live stream would need to use a 3D camera. This means that it is possible to alternate between 2D and 3D live streaming without any inconvenience.
The 3D live stream can be viewed with anaglyphic glasses on a monitor that supports the anaglyphic display, a virtual reality HMD, or a stereoscopic display which has a special setting for interlaced video.
3D live streaming is a real-time, 3D computer-generated presentation of an event taking place in the physical world. The virtual scene is a replica of the real-world location and it has its own setup. The user who creates the 3D live stream has control over the 3D camera and can easily move it around the environment.
What is 3D Live Streaming?
Several existing methods of video streaming can be utilized for 3D streaming, as a simple 3D scene can be converted to a plain 2D image stream. However, to make use of the 3D stereoscopic effect, two identical video streams can be sent for the left and right eye from different viewpoints, or the depth information can be sent in an additional stream. Depth information can be used for depth-image based rendering to reconstruct the different viewpoints. A latest method uses multiview video coding (MVC) where video data is separated into multiple video sequences that represent different views, and then each sequence is compressed using conventional 2D video coding techniques. The simplest MVC setup is two-view stereo video. MVC has been standardized as an extension to H.264/AVC. A combination of the left and right view into a single video file can be presented as L + D + 2VD, where L is the base layer containing common information, D is the enhancement layer with depth information, and 2VD is the two-view video sequence. Depth information can also be used to create the 3D effect on regular 2D video, this is called 2D-plus-depth.
Live streaming in 3D refers to the transmission of multimedia data (audio and visual) in a continuous stream across a 3D virtual environment over the internet. 3D live streaming enables users to send live and on-demand streaming media to friends or the general public. Live streaming has a unique capability: unlike recorded video that can be widely distributed, live video is only available at a specific time and must be viewed then. This makes live streaming compelling for certain events, such as sports games, CEO keynotes, and breaking news. 3D live streaming from the real-world environment to a 3D virtual environment like Second Life can have some commercial value. For example, a car company could stream a new car launch from the real world to its Second Life showroom. This draws the crowd in Second Life who are unable to visit the real event. Live streaming video in a 3D virtual environment can also cater to e-learning and simulation usage.
How Does 3D Live Streaming Work?
MVC based 3D video coding requires the transmission of multiple video streams. This increases the complexity of the network resources needed. A method was proposed to convert the right view into the left view, then subtracting this from the original left view. The result was that the data from the right view to left view difference only had to be transmitted once. This made it possible to improve the quality of the 3D video within the limits of network resources, automatically reducing the bit-rate to an appropriate level. This is a huge advantage for 3D live streaming services. An encoder that estimates the viewer’s visual attention to decide the quality of video to code and transmission is a great potential for 3D live streaming on a system with limited resources. With this method, the predicted visual attention is first converted into depth information and then the object based rendering and scalable coding technologies are used to deliver the 3D video. This could make 3D live streaming available on mobile devices no matter the level of network status.
In 3D live streaming services, a right and left image are separately sent to the viewer’s headset, replicating the way that human eyes perceive images. The latency in the 2D image is significantly improved and any additional delay is almost unnoticeable. This makes it easier to cope with latency when maintaining eye contact and make 3D live streaming experience more comfortable. There has been an increase in methods to send only the necessary parts of a 3D scene and the progress in 3D-HEVC standardization.
How does 3D live streaming work?
Benefits of 3D Live Streaming Services
Beyond delivering a richer event experience than traditional live streaming services, 3D live streaming boasts several unique advantages. Requiring no satellite or cable infrastructure, 3D live streaming can often be a more cost-effective option. For events where partaking in person is not possible, 3D live streaming can be a greener solution with a smaller carbon footprint. In some instances, it may be possible for international viewers to “attend” an event via 3D live streaming without incurring the emissions from traveling to the event’s location. As the world becomes increasingly connected through the internet, it is no longer enough to just read or talk about an event after it has occurred. People want to view and experience events for themselves. With 3D live streaming, event attendees will have the opportunity to re-attend an event long after it has occurred. Recorded 3D streams let viewers experience an event at their convenience while still delivering a rich immersive experience. Step-replays and alternative view perspectives will be possible features that allow people to more intimately experience an event.
Use Cases of 3D Live Streaming in Singapore
In Singapore, Singapore Polytechnic has annual graduation ceremonies which they stream live. They could use 3D live streaming to allow parents, especially those located overseas and unable to attend, to have a greater representation of actually being there. Instead of the traditional live streaming, viewers can view the ceremonies in 3D with the graduates and lecturers appearing as 3D models. This would allow the viewers to have a more interactive and immersive experience as compared to 2D live streaming. They would be able to look around the ceremony from the viewpoint of the 3D camera and it would feel as if they were actually there. This is a practical use of 3D live streaming and could be done sooner than we expect due to SP’s strong background in engineering and technology. To the best of our knowledge, this could be the first time 3D live streaming is used for a graduation ceremony.
Implementing 3D Live Streaming Services in Singapore
In a traditional game with a fixed camera, it is often the case that the scene is rendered and then post-processed to add effects. However, in 3D video, it can be necessary to change the viewpoint on the fly if the stream viewer moves the camera during an interactive stream. This means that the post-processing effects may need to be rendered in real-time and vertex and pixel shaders will need to handle possible changes of input and output.
It is noted here that techniques employed for current generation 3D games are not sufficient for the demanding quality and performance required for 3D streaming. This is because games only render a scene once and the user navigates through predefined areas with occlusion providing a significant amount of help in reducing overdraw. In contrast to this, for a 3D video, it is necessary to render the scene multiple times from various viewpoints. This means that a system for 3D streaming must be capable of rendering scenes with a complex depth ordering and minimal overdraw from any viewpoint.
To carry out live 3D streaming successfully, it is necessary to have a properly defined infrastructure to support all aspects of the rendering pipeline.
Technical requirements and infrastructure
Technical Requirements and Infrastructure
The capability of 3D live streaming has already been shown with the live broadcast of the FIFA World Cup in 2010. This was a worldwide event and was viewed by millions of people all over the world. This event showed the use of 3D live streaming technology to move events from a limited audience with small event to allowing the event to be seen by millions around the world. According to an interview with Sony by Steve May, a panel at the Dimension 3 Festival in Paris saw senior figures who worked on the FIFA World Cup production effort, claiming it was a “game changer” for getting 3D known by the public and made them realize its potential for future sporting events. At this current time, 3D streaming in Singapore is mainly focused towards a small group, possibly installed as a test run rather than a full scale event. This is because 3D broadcasting is still a new technology and hardware can be expensive. To hold a full scale 3D event in Singapore, the technology and infrastructure is widely available, but what is more important is the activation of the audience in order to hit critical mass. A test run of this occurred during the 2010 F1 Singapore Motor Grand Prix (SGGP). The event was broadcasted live in 3D at Golden Village Vivocity and a small group of people went to watch. This was a good test run to the future of 3D live events in Singapore, but it is more significant to have an event that can entertain people in the comfort of their own homes. With the current COVID-19 situation, it is difficult to predict when Singapore will get the chance to plan an event to hit critical mass, however the technology and infrastructure is there waiting to be used.
Key Players in the 3D Live Streaming Industry
Also, SingTel and Sony are among the first to conduct live 3D broadcast in Singapore through SingTel mio TV. The live broadcast was a friendly football match between Singapore and Argentina and was a significant milestone for the Singapore ‘LIVE’ 3D TV industry. By broadcasting this friendly, SingTel mio TV became the first to offer ‘LIVE’ 3D content for its viewers in Singapore. The success of this event will likely lead to more ‘LIVE’ 3D broadcast in future. This is a very significant finding as currently most 3D content comes in the form of movies and documentaries where the content is prepared beforehand.
Mitsubishi Electric Research Laboratories (MERL) has recently developed a new algorithm which encodes 3D TV content such that it can be transmitted using current digital TV systems. The approach starts by splitting the original video into background and foreground layers through depth image based rendering. These layers are then converted to 2D images and compressed as separate video streams.
As explained earlier, the technology and consumer readiness are essential factors contributing to the successful introduction of 3D TV. If Singapore decides to implement 3D TV, there are a few key players that are currently doing research in this field. Understanding the current work of these key players may provide useful insights for the successful implementation of 3D TV in Singapore.
Steps to Launch a 3D Live Streaming Service
Common problems with implementation of a 3D live streaming service are the lack of understanding in how 3D live streaming service works, lack of technology, and lack of equipment. That is why the objective of this paper is to deliver understanding to readers on how to implement the 3D live streaming service. It does not refer only to one provider but to all potential providers in the future. With the aim of establishing a service that will provide real-time 3D live streaming for specific events such as sports, cultural events, and medical procedures, for remote viewing in a highly interactive manner, this paper advocates the use of 3D live streaming technology to bring experiences to people who are unable to attend these events in real life. This is able to benefit those who are unable to be at the actual event and will increase the public interactivity and education of 3D viewing technologies.
Future Trends and Opportunities in 3D Live Streaming
The future trends and opportunities in 3D live streaming services are numerous. 3D streaming is currently receiving a very large amount of advertising and interest from the general public. As more and more consumers purchase 3D televisions, there will be a large demand for 3D content. 3D live streaming can be the solution to broadcasting live 3D content for events such as sports, pay per view, and live television series. As discussed previously, 3D streaming has the potential to be very successful for events and PPV due to the fact that it cannot be recorded and viewed later. 3D events such as these are often limited due to the small demographic of 3D TV owners; however, this is expected to change in the near future and 3D live streaming can play a big part. Simulation (video games, animated movies) is another area where 3D streaming can be utilized. In the future, interactive 3D media using avatars will become widely used for services such as education and online meetings. This is an area where traditional 2D video streaming falls short and 3D streaming may open up new possibilities. In the field of telemedicine, doctors may be able to hold live 3D consultations with patients from across the world. 3D streaming has potential future use for mobile gaming. With the advent of 3D mobile devices, cloud gaming services may evolve to stream 3D games from powerful servers in a similar way that Gakai and Onlive currently stream PC games.
