You.i Engine
Video Player

Detailed Description

The video player provides a means of displaying video in an application.

Video Player

Intro

The video player in You.i Engine allows for the playback of streaming video. You.i Engine provides a built-in video player implementation for each of the platforms supported by the engine. CYIAbstractVideoPlayer provides a generic base class to implement a video player. The video player on each platform may vary in interface and functionality but CYIAbstractVideoPlayer simplifies that into a generic interface which can be used to implement a clean cross-platform player screen in any You.i Engine application. This generic base class may be used to implement a custom video player if the provided built-in implementations are not sufficient. See the Custom Implementation section for further details.

States

The CYIAbstractVideoPlayer is governed by a set of states. These states are separated into two categories, media states and playback states, and are managed by the CYIVideoPlayerStateManager. The media states represent the state of the video which is loaded into the player. When a CYIAbstractVideoPlayer is first created it will be in the MEDIA UNLOADED state. The video player must be prepared with a video and in the MEDIA READY state before playback can occur. The playback states indicate whether the current loaded video is playing, paused or buffering. It is the application developer's responsibility to ensure the application's user interface is updated to respect these playback states. The following figure depicts the valid state transitions of the CYIAbstractVideoPlayer:

video-player-states.png

Playback

A video player implementation is supplied for each platform supported by You.i Engine so that the application developer can use CYIAbstractVideoPlayer to get video playback up and running in their application with as few steps as possible. Using the CYIAbstractVideoPlayer is as simple as creating and initializing an instance of the player, connecting to the signal notifications that the application requires and preparing the player with a video. The following code snippet displays this process:

// Create an instance of the built-in player implementation for the current platform.
std::unique_ptr<CYIAbstractVideoPlayer> pPlayer = CYIDefaultVideoPlayerFactory::Create();

// Initialize the video player.
pPlayer->Init();

// Connect the video player signals required by the application.
pPlayer->PlayerStateChanged.Connect(*this, &PlayerScreen::OnPlayerStateChanged);
pPlayer->ErrorOccured.Connect(*this, &PlayerScreen::OnPlayerErrorOccured);

// Prepare the player with an HLS stream URL with the instruction to start playback once the video has been prepared.
m_pPlayer->Prepare(HLS_STREAM_URL, CYIAbstractVideoPlayer::StreamingFormat::HLS, CYIAbstractVideoPlayer::PlaybackState::Playing);

...
...
...

// Optionally clean up the video player. If reset() isn't called, clean-up will happens automatically when pPlayer goes out-of-scope.
pPlayer.reset();
Note
You.i Engine currently only supports one instance of CYIAbstractVideoPlayer when using built-in implementations.

Display

Video

To display the video played by CYIAbstractVideoPlayer the application developer or designer should add a CYIVideoSurfaceView to the view in which the application requires video playback. This CYIVideoSurfaceView contains the CYIVideoSurface where the video is rendered. The developer must set the CYIVideoSurface on the CYIVideoSurfaceView to correspond with the surface of the CYIAbstractVideoPlayer instance which is to be displayed in the view. It should be noted that setting the same CYIVideoSurface on multiple CYIVideoSurfaceView's is not supported. The following code snippet displays how to properly set the video surface on the CYIVideoSurfaceView:

// Find the CYIVideoSurfaceView in the containing view.
CYIVideoSurfaceView *pPlayerSurfaceView = nullptr;
pSceneView->FindNode<CYIVideoSurfaceView>(pPlayerSurfaceView, "VideoSurface", CYISceneView::YI_MANDATORY, "PlayerScreen");
if(pPlayerSurfaceView)
{
    // Set the player's video surface on the video surface view.
    m_pPlayerSurfaceView->SetVideoSurface(m_pPlayer->GetSurface());
}
Note
You.i Engine does not support setting the same CYIVideoSurface on more than one CYIVideoSurfaceView.

Closed Captions

The video player implementations that are provided by You.i Engine use a combination of in-engine and external CC rendering. In-engine rendering makes use of CYIClosedCaptionsDispatcher to dispatch cues to IYIClosedCaptionsRenderer instances. CYIVideoSurfaceView contains an IYIClosedCaptionsRenderer and will automatically register with CYIClosedCaptionsDispatcher if the CYIVideoSurface provided to it contains a reference to one. External CC rendering is accomplished using platform-native controls and cue information is not dispatched internally. A CYIAbstractVideoPlayer implementation which makes use of external CC rendering will have a null CYIClosedCaptionsDispatcher.

In order to support closed captioning, implementers of CYIAbstractVideoPlayer must instantiate a CYIClosedCaptionsDispatcher and dispatch cues appropriately for in-engine CC rendering. The dispatcher must also be assigned to the player's CYIVideoSurface via CYIVideoSurface::SetClosedCaptionsDispatcher. Otherwise, they must ensure the cues are rendered natively by the platform.

Time-based Metadata

If a CYIAbstractVideoPlayer implementation supports emitting any form of time-based metadata it will provide a non-null CYIAbstractVideoPlayer::TimedMetadataInterface via the CYIAbstractVideoPlayer::GetTimedMetadataInterface API. The player will emit CYIAbstractVideoPlayer::TimedMetadataInterface::MetadataAvailable via the interface if it encounters supported timed-metadata within the stream. Currently there are two timed-metadata formats supported by the player; ID3 and DASH Event Message.

ID3

Time-based metadata may be embedded in the individual MPEG-2 TS files of the streaming media. One common standard for this within MPEG-2 TS is ID3. Some player implementations support emitting time-based frames of the ID3 metadata. Currently there are two supported ID3 frames; TXXX and PRIV frames.

DASH Event Message

Time-based metadata may also be embedded in the MPEG-4 segments of DASH-based streaming media. The format used for this type of metadata is called Event Message (EMSG). This type of metadata can be either in-band or out-of-band. In-band event messages are located directly in the MPEG-4 segments and Out-of-band event messages are contained within the DASH manifest.

The following table outlines the current time-based metadata support across platform player implementations:

Metadata Format

Platform ID3 TXXX ID3 PRIV

DASH Event Message

Mobile

iOS Yes Yes

No

Android Yes Yes

Yes

10-Foot

Apple tvOS Yes Yes

No

UWP Yes Yes

Yes

Playstation 4 Yes Yes

Yes

Tizen Yes Yes

No

WebOS No No

No

Development

Windows No No

No

Mac OSX Yes Yes

No

Linux No No No

Thumbnails

CYIPlayerPreviewThumbnailView is provided to display preview thumbnails of a video. This view will display a thumbnail for a given time in the video and supports thumbnails in the BIF format. The following code snippet displays a simple usage of the thumbnail view:

// Find the CYIPlayerPreviewThumbnailView in the containing view.
CYIPlayerPreviewThumbnailView *pThumbnailView = nullptr;
FindNode<CYIPlayerPreviewThumbnailView>(pThumbnailView, "Preview", CYISceneView::YI_MANDATORY, "PlayerScreen");
if(pThumbnailView)
{
    // Set the BIF thumbnail asset from the filesystem.
    pThumbnailView->SetThumbnail(BIF_FILE_PATH);
    // Set the time of the thumbnail to display to 10 seconds into the stream.
    pThumbnailView->SetTime(10000);
    // Animate the popup view in so that it is visible.
    pThumbnailView->ShowPopup();
}

Custom Implementation

In the case that the built-in platform video player implementations do not meet the needs of the application developer a custom implementation can be implemented. When creating a custom video player implementation there are two areas where implementation is required, a CYIAbstractVideoPlayer subclass and a CYIVideoSurface subclass.

Player

The CYIAbstractVideoPlayer is responsible for controlling playback of a video. When implementing a CYIAbstractVideoPlayer subclass there are some key areas which must be implemented over and above pure virtual functions of the base class. This includes directly notifying the CYIAbstractVideoPlayer of various player events as well as notifying CYIAbstractVideoPlayer's CYIVideoPlayerStateManager instance of state transitions in the player. The events which the CYIAbstractVideoPlayer must be directly notified about are as follows:

CYIAbstractVideoPlayer's instance of CYIVideoPlayerStateManager must be notified of internal state transitions in the video player. The state transitions must follow a strict ordering, this order is displayed in the figure in the State section above. Failure to follow the correct ordering of state transitions will result in an assertion in debug builds and a player error in release builds. This strict ordering of states is put in place to prevent differences in player implementations from reaching the CYIAbstractVideoPlayer level which is meant to behave the same regardless of the underlying player implementation. See CYIVideoPlayerStateManager for further detail on valid state transitions.

If the custom player does not do its own rendering of closed captions, You.i Engine's closed captions renderer can be used. To do this, the CYIAbstractVideoPlayer::GetClosedCaptionsDispatcher_() function must be overridden to return a reference to a CYIClosedCaptionsDispatcher object. As the video plays, the custom player should send closed captions drawing requests to the CYIClosedCaptionsDispatcher object.

Playing a video with a custom player implementation will be the same as documented in the Playback section above with the exception that instead of using the CYIDefaultVideoPlayerFactory::Create method the custom implementation must be instanced directly.

Video Surface

The CYIVideoSurface is where the video will be displayed. When implementing a custom video player, it is important to create a CYIVideoSurface subclass. When creating a CYIVideoSurface it is recommended to use a CYIVideoSurfacePlatform or a CYIVideoSurfaceTexture subclass. CYIVideoSurfaceTexture renders the video data provided by the video player into a texture object which the CYIVideoSurfaceView can display. CYIVideoSurfacePlatform takes a screen-space rectangle from the CYIVideoSurfaceView and positions the video into it. If the video player implementation supplies video buffer data it is preferable to render video to texture by subclassing CYIVideoSurfaceTexture as this will provide more capabilities when it comes to transforming the video surface. When implementing a CYIVideoSurfaceTexture a CYIMaterial must be created to render the video, this material includes a shader and texture. The following code snippet displays how to create and initialize the required material as should be done in CYIVideoSurfaceTexture::InitializeMaterial and CYIVideoSurfaceTexture::InitializeTexture:

// Initialize the material.

// Load a shader program from a file.
std::shared_ptr<CYIAssetShaderProgram> pVideoShaderProgram;
pVideoShaderProgram = CYIShaderFactor::CreateAssetFromFile("VERTEX_SHADER_PATH", "FRAGMENT_SHADER_PATH");

std::shared_ptr<CYIMaterial> pVideoMaterial;
// Set the shader program to the shader program loaded from file above.
pVideoMaterial->SetShaderProgram(pVideoShaderProgram);

// Create a new texture and add it to the asset manager.
std::shared_ptr<CYIAssetTexture> pTexture(new CYIAssetTexture);
CYIFramework::GetInstance()->GetAssetManager()->AddAsset(pTexture);

// Initialize the texture.

// Create a bitmap with a black fill to set on the texture.
std::shared_ptr<CYIBitmap> pBitmap(new CYIBitmap);
pBitmap->Fill(CYIColor::Black);
pTexture->SetBitmap(pBitmap);

// Sets the materials texture at slot 0 to the texture created above.
pVideoMaterial->SetTexture(0, pTexture);

If the video player implementation does not supply video buffer data and renders the video to the platform surface directly then CYIVideoSurfacePlatform should be subclassed to set the video's position and size.

Video Surface Capabilities

Each CYIVideoSurface has a set of capabilities which indicate the functions which the video surface is capable of performing. The capabilities may be queried at run time using the CYIVideoSurface::GetCapabilities method. See CYIVideoSurface::Capabilities for more details on each capability. The following table displays the capabilities supported by each default platform player:

Capabilities

Platform Render To Texture Multiple Views Translate Scale Free Transform

Opacity

Mobile

iOS No No Yes Yes No

Yes

Android No No Yes Yes No

Yes

10-Foot

Apple tvOS No No Yes Yes No

Yes

UWP No No Yes Yes No

Yes

Playstation 4 Yes Yes Yes Yes Yes

Yes

Tizen No No No No No

No

Development

Windows Yes Yes Yes Yes Yes

Yes

Mac OS X Yes Yes Yes Yes Yes

Yes

Linux Yes Yes Yes Yes Yes Yes

Classes

class  CYIAbstractVideoPlayer
 An abstract video player that provides a basic interface that all subclasses must implement. More...
 
class  CYIAVPlayer
 Implementation of abstract video player for iOS, tvOs and OSX. More...
 
class  CYIBifThumbnails
 Implementation of thumbnails using BIF file format. More...
 
class  CYIBlueskyVideoPlayer
 
class  CYICastLabsVideoPlayer
 
class  CYIClosedCaptionsDispatcher
 Dispatches closed captioning cues to registered captions renderers. More...
 
class  CYIDefaultVideoPlayerFactory
 Constructs a default concrete CYIAbstractVideoPlayer for this platform. More...
 
class  CYIExoPlayerVideoPlayer
 The ExoPlayer video player object. More...
 
class  CYIFairPlayDRMConfiguration
 A class containing the necessary configurations for playback of FairPlay protected media. More...
 
class  CYIImageMapThumbnails
 Implementation of thumbnails using Image Map file format. More...
 
class  CYILicenseAcquisitionDRMConfiguration
 This base class contains the information required to contact the license aquisition server and obtain the license for playback. More...
 
class  CYIPlayReadyDRMConfiguration
 A class containing the necessary configurations for playback of PlayReady protected media. More...
 
class  IYIThumbnails
 Provides an interface for thumbnail implementations. More...
 
class  CYITizenNaClVideoPlayer
 
class  CYIUWPVideoPlayer
 
class  CYIVestelVideoPlayer
 
class  CYIVideoPlayerStateManager
 A class which manages the various media and playback states of the CYIAbstractVideoPlayer. More...
 
class  CYIVideoSurface
 Representation of a CYIAbstractVideoPlayer's video surface. More...
 
class  CYIVideoSurfacePlatform
 Representation of a CYIAbstractVideoPlayer's video surface, if the player is rendering to a platform-level view outside of You.i Engine. More...
 
class  CYIVideoSurfaceTexture
 Representation of a CYIAbstractVideoPlayer's video surface, if the player is rendering to a texture. More...
 
class  CYIVideoSurfaceView
 View representation of a CYIAbstractVideoPlayer's CYIVideoSurface. More...
 
class  CYIVLCVideoPlayer
 
class  CYIWidevineModularCustomRequestDRMConfiguration
 A class containing the necessary configurations for playback of Widevine Modular protected media, and with the oppurtunity available to handle all network requests required for license acquisition. More...
 
class  CYIWidevineModularDRMConfiguration
 A class containing the necessary configurations for playback of Widevine Modular protected media. More...