Detailed Description

The video player provides a means of displaying video in an application.

Video Player

Intro

The video player in You.i Engine allows for the playback of streaming video. You.i Engine provides a built-in video player implementation for each of the platforms supported by the engine. CYIAbstractVideoPlayer provides a generic base class to implement a video player. The video player on each platform may vary in interface and functionality but CYIAbstractVideoPlayer simplifies that into a generic interface which can be used to implement a clean cross-platform player screen in any You.i Engine application. This generic base class may be used to implement a custom video player if the provided built-in implementations are not sufficient. See the Custom Implementation section for further details.

States

The CYIAbstractVideoPlayer is governed by a set of states. These states are separated into two categories, media states and playback states, and are managed by the CYIVideoPlayerStateManager. The media states represent the state of the video which is loaded into the player. When a CYIAbstractVideoPlayer is first created it will be in the MEDIA UNLOADED state. The video player must be prepared with a video and in the MEDIA READY state before playback can occur. The playback states indicate whether the current loaded video is playing, paused or buffering. It is the application developer's responsibility to ensure the application's user interface is updated to respect these playback states. The following figure depicts the valid state transitions of the CYIAbstractVideoPlayer:

Playback

A video player implementation is supplied for each platform supported by You.i Engine so that the application developer can use CYIAbstractVideoPlayer to get video playback up and running in their application with as few steps as possible. Using the CYIAbstractVideoPlayer is as simple as creating and initializing an instance of the player, connecting to the signal notifications that the application requires and preparing the player with a video. The following code snippet displays this process:

// Create an instance of the built-in player implementation for the current platform.
std::unique_ptr<CYIAbstractVideoPlayer> pPlayer = CYIDefaultVideoPlayerFactory::Create();

// Initialize the video player.
pPlayer->Init();

// Connect the video player signals required by the application.
pPlayer->PlayerStateChanged.Connect(*this, &PlayerScreen::OnPlayerStateChanged);
pPlayer->ErrorOccured.Connect(*this, &PlayerScreen::OnPlayerErrorOccured);

// Prepare the player with an HLS stream URL with the instruction to start playback once the video has been prepared.
m_pPlayer->Prepare(HLS_STREAM_URL, CYIAbstractVideoPlayer::StreamingFormat::HLS, CYIAbstractVideoPlayer::PlaybackState::Playing);

...
...
...

// Optionally clean up the video player. If reset() isn't called, clean-up will happens automatically when pPlayer goes out-of-scope.
pPlayer.reset();

Note: You.i Engine currently only supports one instance of CYIAbstractVideoPlayer when using built-in implementations.

Display

Video

To display the video played by CYIAbstractVideoPlayer the application developer or designer should add a CYIVideoSurfaceView to the view in which the application requires video playback. This CYIVideoSurfaceView contains the CYIVideoSurface where the video is rendered. The developer must set the CYIVideoSurface on the CYIVideoSurfaceView to correspond with the surface of the CYIAbstractVideoPlayer instance which is to be displayed in the view. It should be noted that setting the same CYIVideoSurface on multiple CYIVideoSurfaceView's is not supported. The following code snippet displays how to properly set the video surface on the CYIVideoSurfaceView:

// Find the CYIVideoSurfaceView in the containing view.
CYIVideoSurfaceView *pPlayerSurfaceView = nullptr;
pSceneView->FindNode<CYIVideoSurfaceView>(pPlayerSurfaceView, "VideoSurface", CYISceneView::YI_MANDATORY, "PlayerScreen");
if(pPlayerSurfaceView)
{
    // Set the player's video surface on the video surface view.
    m_pPlayerSurfaceView->SetVideoSurface(m_pPlayer->GetSurface());
}

Note: You.i Engine does not support setting the same CYIVideoSurface on more than one CYIVideoSurfaceView.

Closed Captions

The video player implementations that are provided by You.i Engine use a combination of in-engine and external CC rendering. In-engine rendering makes use of CYIClosedCaptionsDispatcher to dispatch cues to IYIClosedCaptionsRenderer instances. CYIVideoSurfaceView contains an IYIClosedCaptionsRenderer and will automatically register with CYIClosedCaptionsDispatcher if the CYIVideoSurface provided to it contains a reference to one. External CC rendering is accomplished using platform-native controls and cue information is not dispatched internally. A CYIAbstractVideoPlayer implementation which makes use of external CC rendering will have a null CYIClosedCaptionsDispatcher.

In order to support closed captioning, implementers of CYIAbstractVideoPlayer must instantiate a CYIClosedCaptionsDispatcher and dispatch cues appropriately for in-engine CC rendering. The dispatcher must also be assigned to the player's CYIVideoSurface via CYIVideoSurface::SetClosedCaptionsDispatcher. Otherwise, they must ensure the cues are rendered natively by the platform.

Time-based Metadata

If a CYIAbstractVideoPlayer implementation supports emitting any form of time-based metadata it will provide a non-null CYIAbstractVideoPlayer::TimedMetadataInterface via the CYIAbstractVideoPlayer::GetTimedMetadataInterface API. The player will emit CYIAbstractVideoPlayer::TimedMetadataInterface::MetadataAvailable via the interface if it encounters supported timed-metadata within the stream. Currently there are two timed-metadata formats supported by the player; ID3 and DASH Event Message.

ID3

Time-based metadata may be embedded in the individual MPEG-2 TS files of the streaming media. One common standard for this within MPEG-2 TS is ID3. Some player implementations support emitting time-based frames of the ID3 metadata. Currently there are two supported ID3 frames; TXXX and PRIV frames.

DASH Event Message

Time-based metadata may also be embedded in the MPEG-4 segments of DASH-based streaming media. The format used for this type of metadata is called Event Message (EMSG). This type of metadata can be either in-band or out-of-band. In-band event messages are located directly in the MPEG-4 segments and Out-of-band event messages are contained within the DASH manifest.

The following table outlines the current time-based metadata support across platform player implementations:

	Metadata Format
Platform	ID3 TXXX	ID3 PRIV	DASH Event Message
Mobile
iOS	Yes	Yes	No
Android	Yes	Yes	Yes
10-Foot
Apple tvOS	Yes	Yes	No
UWP	Yes	Yes	Yes
Playstation 4	Yes	Yes	Yes
Tizen	Yes	Yes	No
WebOS	No	No	No
Development
Windows	No	No	No
Mac OSX	Yes	Yes	No
Linux	No	No	No

Thumbnails

CYIPlayerPreviewThumbnailView is provided to display preview thumbnails of a video. This view will display a thumbnail for a given time in the video and supports thumbnails in the BIF format. The following code snippet displays a simple usage of the thumbnail view:

// Find the CYIPlayerPreviewThumbnailView in the containing view.
CYIPlayerPreviewThumbnailView *pThumbnailView = nullptr;
FindNode<CYIPlayerPreviewThumbnailView>(pThumbnailView, "Preview", CYISceneView::YI_MANDATORY, "PlayerScreen");
if(pThumbnailView)
{
    // Set the BIF thumbnail asset from the filesystem.
    pThumbnailView->SetThumbnail(BIF_FILE_PATH);
    // Set the time of the thumbnail to display to 10 seconds into the stream.
    pThumbnailView->SetTime(10000);
    // Animate the popup view in so that it is visible.
    pThumbnailView->ShowPopup();
}

Custom Implementation

In the case that the built-in platform video player implementations do not meet the needs of the application developer a custom implementation can be implemented. When creating a custom video player implementation there are two areas where implementation is required, a CYIAbstractVideoPlayer subclass and a CYIVideoSurface subclass.

Player

The CYIAbstractVideoPlayer is responsible for controlling playback of a video. When implementing a CYIAbstractVideoPlayer subclass there are some key areas which must be implemented over and above pure virtual functions of the base class. This includes directly notifying the CYIAbstractVideoPlayer of various player events as well as notifying CYIAbstractVideoPlayer's CYIVideoPlayerStateManager instance of state transitions in the player. The events which the CYIAbstractVideoPlayer must be directly notified about are as follows:

An error has occurred during preparation or playback of the video. [CYIAbstractVideoPlayer::NotifyErrorOccurred]
The duration of the video loaded into the player has changed. This would generally only occur when loading new media into the player. [CYIAbstractVideoPlayer::NotifyDurationChanged]
The video has reached the end of stream and playback has completed. [CYIAbstractVideoPlayer::NotifyPlaybackComplete]

Additional to the above notifications if the underlying player provides an event for the when a seek completes the player should implement CYIAbstractVideoPlayer::HasNativeSeekEventHandling_ to return true. This will override the abstract player's default handling and emit the relevant signal when CYIAbstractVideoPlayer::NotifySeekCompleted is emitted. If the underlying player does not provide this event it should not implement the above method and the abstract player will emit the relevant signals based on the time updating for the seek.

CYIAbstractVideoPlayer's instance of CYIVideoPlayerStateManager must be notified of internal state transitions in the video player. The state transitions must follow a strict ordering, this order is displayed in the figure in the State section above. Failure to follow the correct ordering of state transitions will result in an assertion in debug builds and a player error in release builds. This strict ordering of states is put in place to prevent differences in player implementations from reaching the CYIAbstractVideoPlayer level which is meant to behave the same regardless of the underlying player implementation. See CYIVideoPlayerStateManager for further detail on valid state transitions.

If the custom player does not do its own rendering of closed captions, You.i Engine's closed captions renderer can be used. To do this, the CYIAbstractVideoPlayer::GetClosedCaptionsDispatcher_() function must be overridden to return a reference to a CYIClosedCaptionsDispatcher object. As the video plays, the custom player should send closed captions drawing requests to the CYIClosedCaptionsDispatcher object.

Playing a video with a custom player implementation will be the same as documented in the Playback section above with the exception that instead of using the CYIDefaultVideoPlayerFactory::Create method the custom implementation must be instanced directly.

Video Surface

The CYIVideoSurface is where the video will be displayed. When implementing a custom video player, it is important to create a CYIVideoSurface subclass. When creating a CYIVideoSurface it is recommended to use a CYIVideoSurfacePlatform or a CYIVideoSurfaceTexture subclass. CYIVideoSurfaceTexture renders the video data provided by the video player into a texture object which the CYIVideoSurfaceView can display. CYIVideoSurfacePlatform takes a screen-space rectangle from the CYIVideoSurfaceView and positions the video into it. If the video player implementation supplies video buffer data it is preferable to render video to texture by subclassing CYIVideoSurfaceTexture as this will provide more capabilities when it comes to transforming the video surface. When implementing a CYIVideoSurfaceTexture a CYIMaterial must be created to render the video, this material includes a shader and texture. The following code snippet displays how to create and initialize the required material as should be done in CYIVideoSurfaceTexture::InitializeMaterial and CYIVideoSurfaceTexture::InitializeTexture:

// Initialize the material.

// Load a shader program from a file.
std::shared_ptr<CYIAssetShaderProgram> pVideoShaderProgram;
pVideoShaderProgram = CYIShaderFactor::CreateAssetFromFile("VERTEX_SHADER_PATH", "FRAGMENT_SHADER_PATH");

std::shared_ptr<CYIMaterial> pVideoMaterial;
// Set the shader program to the shader program loaded from file above.
pVideoMaterial->SetShaderProgram(pVideoShaderProgram);

// Create a new texture and add it to the asset manager.
std::shared_ptr<CYIAssetTexture> pTexture(new CYIAssetTexture);
CYIFramework::GetInstance()->GetAssetManager()->AddAsset(pTexture);

// Initialize the texture.

// Create a bitmap with a black fill to set on the texture.
std::shared_ptr<CYIBitmap> pBitmap(new CYIBitmap);
pBitmap->Fill(CYIColor::Black);
pTexture->SetBitmap(pBitmap);

// Sets the materials texture at slot 0 to the texture created above.
pVideoMaterial->SetTexture(0, pTexture);

If the video player implementation does not supply video buffer data and renders the video to the platform surface directly then CYIVideoSurfacePlatform should be subclassed to set the video's position and size.

Video Surface Capabilities

Each CYIVideoSurface has a set of capabilities which indicate the functions which the video surface is capable of performing. The capabilities may be queried at run time using the CYIVideoSurface::GetCapabilities method. See CYIVideoSurface::Capabilities for more details on each capability. The following table displays the capabilities supported by each default platform player:

	Capabilities
Platform	Render To Texture	Multiple Views	Translate	Scale	Free Transform	Opacity
Mobile
iOS	No	No	Yes	Yes	No	Yes
Android	No	No	Yes	Yes	No	Yes
10-Foot
Apple tvOS	No	No	Yes	Yes	No	Yes
UWP	No	No	Yes	Yes	No	Yes
Playstation 4	Yes	Yes	Yes	Yes	Yes	Yes
Tizen	No	No	No	No	No	No
Development
Windows	Yes	Yes	Yes	Yes	Yes	Yes
Mac OS X	Yes	Yes	Yes	Yes	Yes	Yes
Linux	Yes	Yes	Yes	Yes	Yes	Yes

Classes
class	CYIAbstractVideoPlayer
	An abstract video player that provides a basic interface that all subclasses must implement. More...

class	CYIAVPlayer
	Implementation of abstract video player for iOS, tvOs and OSX. More...

class	CYIBlueskyVideoPlayer

class	CYICastLabsVideoPlayer

class	CYIClosedCaptionsDispatcher
	Dispatches closed captioning cues to registered captions renderers. More...

class	CYIClosedCaptionsStyleManager
	A class to manage styles for closed captions. More...

class	CYIDefaultVideoPlayerFactory
	Constructs a default concrete CYIAbstractVideoPlayer for this platform. More...

class	CYIExoPlayerVideoPlayer
	The ExoPlayer video player object. More...

class	CYIFairPlayDRMConfiguration
	A class containing the necessary configurations for playback of FairPlay protected media. More...

class	CYILicenseAcquisitionDRMConfiguration
	This base class contains the information required to contact the license aquisition server and obtain the license for playback. More...

class	CYIPlayReadyDRMConfiguration
	A class containing the necessary configurations for playback of PlayReady protected media. More...

class	IYIThumbnails
	Provides an interface for thumbnail implementations. More...

class	CYITizenAVPlayVideoPlayer

class	CYITizenNaClESVideoPlayer

class	CYIVideoPlayerStateManager
	A class which manages the various media and playback states of the CYIAbstractVideoPlayer. More...

class	CYIVideoSurface
	Representation of a CYIAbstractVideoPlayer's video surface. More...

class	CYIVideoSurfacePlatform
	Representation of a CYIAbstractVideoPlayer's video surface, if the player is rendering to a platform-level view outside of You.i Engine. More...

class	CYIVideoSurfaceTexture
	Representation of a CYIAbstractVideoPlayer's video surface, if the player is rendering to a texture. More...

class	CYIVideoSurfaceView
	View representation of a CYIAbstractVideoPlayer's CYIVideoSurface. More...

class	CYIVLCVideoPlayer

class	CYIWebOSESVideoPlayer

class	CYIWidevineModularCustomRequestDRMConfiguration
	A class containing the necessary configurations for playback of Widevine Modular protected media, and with the oppurtunity available to handle all network requests required for license acquisition. More...

class	CYIWidevineModularDRMConfiguration
	A class containing the necessary configurations for playback of Widevine Modular protected media. More...

class	CYIBifThumbnails
	Implementation of thumbnails using BIF file format. More...

class	CYIImageMapThumbnails
	Implementation of thumbnails using Image Map file format. More...

class	CYIUWPVideoPlayer