You.i Engine
Assets

Detailed Description

The asset system consists of a flexible set of classes that provide functionality for assets to be loaded and managed by applications.

Asset objects represent different types of data that can used in an application. Some asset types have hardware counterparts and may have particular hardware requirements that must be satisfied before they can be used, while others can be manipulated by the application directly after they have been created. There are other types that represent particular assets used by You.i Engine and cannot be loaded or unloaded.

Types

There are several CYIAsset types available for applications to use:

Asset Description Supported Extensions Hardware Dependent
CYIAssetBif Decoded BIF archive which is used to encapsulate a set of still images, or thumbs, that can be used by video playback components or for supporting trick play modes in conjunction with fast-forward or rewind operations. .bif No
CYIAssetBufferObjectData Buffered object content that wraps shared index or vertex buffer object data. N/A No
CYIAssetFramebuffer Frame-buffer asset used in off-screen rendering. N/A Yes
CYIAssetJSON An asset which contains a JSON document.
Note
Due to their being no magic number associated with the JSON file format, detection of JSON files without an extension is based on file contents. Due to this limitation the JSON decoder will be considered last when determining the correct decoder of a file without a file extension to prevent false positives.
.json No
CYIAssetShaderObject Shader data for vertex or fragment shaders. .vert
.frag
No
CYIAssetShaderProgram A series of CYIAssetShaderObjects that constitute a shader program. N/A Yes
CYIAssetSound Contains raw PCM sound data used to play CYISoundInstance in the CYIAudioEngine. .wav
.wave
No
CYIAssetTexture An asset with internal CYIBitmap and IYIGPUObject objects representing image content. .png
.jpg
.jpeg
.9.png
.webp
.gif
.tga
Yes
CYIAssetTextureRaw An asset that contains only raw pixel data instead of relying on CYIBitmap, and IYIGPUObject objects representing image content on the GPU. N/A Yes
CYIAssetTextureFramebuffer An asset containing a frame-buffer, rendered as a texture, used in off-screen rendering. N/A Yes
CYIAssetTimelineSource An asset with a CYITimelineBackingStore object which is used for rendering After Effects based animations or other timeline based operations. .timeline No
CYIAssetTranslation Contains a CYITranslator object for processing language translations. .po No
CYIAssetViewTemplate Contains a CYIViewTemplate object for rendering After Effects based scenes. .layout No

Textures

Textures are one of the most common asset types in a project and are an integral part to any You.i Engine application. Essentially, there are two parts to every texture asset: CYIBitmap and IYIGPUObject. A bitmap contains the raw data for an image and can be modified directly by an application developer. An IYIGPUObject is an interface representing content that can be consumed by 3D rendering hardware.

In many applications, once the bitmap has been loaded and the texture has been made available to the graphics processing unit (GPU), the bitmap can be deleted. This can be done easily by unloading it (see the CYIAsset and CYIAssetTexture classes for more information about loading and unloading). Once unloaded, the memory used by the bitmap will have been reclaimed by the system but the texture will still be available. In some rarer cases, the texture may need to be unloaded instead of the bitmap. For example, in situations where load times are critical but it becomes too expensive to keep both the bitmap and the texture in memory at the same time, the hardware version can be unloaded while retaining the bitmap (see the CYIAssetHardware class for more information about requesting deletion of the GPU asset).

Locating and Loading

Before any asset can be loaded it must be located via the CYIAssetLocator. The locator's job is to find the appropriate asset to load based on the locator's configuration. The asset locator uses configuration objects to filter out content based on screen dimensions, locale, or other custom attributes.

After configuring the locator, the asset can be loaded using the CYIAssetLoader and can be loaded synchronously or asynchronously. Loading assets usually involves decoding the file format and can be a significant performance consideration. To alleviate this problem, asychronous loading can be used to offload the decoding onto another thread.

The following is an example of how to load an asset synchronously with the CYIAssetLoader:

std::shared_ptr<CYIAssetTexture> pImage = YiDynamicCast<CYIAssetTexture>(pLoader->Load(CYIAssetTexture::GetClassTypeInfo(), "myimage.jpg"));

Using asynchronous asset loading is a bit more work since it requires the CYIAssetLoader::DecodeListener interface to be implemented:

class MyDecodeListener : public CYIAssetLoader::DecodeListener
{
void OnDecodeComplete(int32_t jobID, std::shared_ptr<CYIAsset> pAsset, int32_t loadStatus, void *pListenerPrivate)
{
YI_UNUSED(jobID);
YI_UNUSED(pAsset);
YI_UNUSED(loadStatus);
YI_UNUSED(pListenerPrivate);
// The decoded asset "pAsset" is ready to be used
}
void StartAsyncLoad(const CYIString &path)
{
// Load an image using an absolute path
pLoader->LoadAsyncFromPath(CYIAssetTexture::GetClassTypeInfo(), path, nullptr, this, nullptr);
}
};

Life Cycle

Asset objects are typically created by the CYIAssetLoader, which are then used to render a scene, and are subsequently discarded when screens are unloaded (see Screen Management for more details around screen lifecycle behavior).

It's easy for application developers to manage the resources their assets consume. All assets within You.i Engine are reference counted if they are wrapped using a smart pointer, such as std::shared_ptr. They are wrapped automatically if the asset was loaded via the CYIAssetLoader or through the one of the asset factory classes like CYITextureFactory. Reference counting is a means to determine when an object is no longer being used or referenced elsewhere in the application. If the reference count for an asset falls to zero, the resources associated with the asset would be automatically reclaimed since the CYIAsset instance would be deleted.

Caching and Sharing Assets

Assets exist in memory and consume resources for as long as their associated asset object exists. In some cases, the lifetime of those asset objects are tied to other data structures, such as a scene tree. In other scenarios, it may be desirable to employ specialized caching techniques so that they can be retrieved more quickly thus improving the runtime performance of your application in key areas.

Allowing You.i Engine to delete the objects automatically may not be the best solution for every scenario when developing an application, there are other things to consider, such as performance and network bandwidth consumption. As always, those requirements must be balanced against available memory and storage for each platform.

The CYIAssetManager can be used to share assets more easily between components in a decoupled way. It is a weak cache that does not contribute to the reference count of the assets.

Decoding

When an asset is loaded, the content of the file is usually passed through a CYIAssetDecoder. The role of a decoder is to process the content of a file and produce an asset object. If an application requires support for a file format not supported by You.i Engine, custom decoders can be written.

For example, to add support for a new image format, sub-class the CYIAssetDecoderImage type and implement a new CYIImageDecoder:

class MyImageDecoderBMP : public CYIImageDecoder
{
public:
MyImageDecoderBMP();
virtual ~MyImageDecoderBMP();
virtual bool GetImageDimensions(const CYIString &path, uint32_t *pWidth, uint32_t *pHeight) override;
virtual bool IsFormatSupported(const uint8_t *pData, size_t dataSize) override;
protected:
virtual std::unique_ptr<CYIBitmap> DecodeImpl(const CYIString &path, YI_DECODER_BITMAP_PARAMS *pParams, CYITaskBase *pTask) override;
virtual std::unique_ptr<CYIBitmap> DecodeImpl(const uint8_t *pfileData, size_t dataSize, YI_DECODER_BITMAP_PARAMS *pParams, CYITaskBase *pTask) override;
};
class MyAssetDecoderBMP : public CYIAssetDecoderImage
{
public:
MyAssetDecoderBMP()
{
SetDecoder(std::make_unique<MyImageDecoderBMP>());
m_SupportedExtensions = {"bmp", "dib"};
};
virtual ~MyAssetDecoderBMP(){};
};

In this example the asset type is supported (CYIAssetTexture) but the BMP file format is not. Sometimes it is necessary to support a completely custom asset, this is covered in the following section.

Unsupported Asset Types

If an application requires an asset type not supported by You.i Engine, a custom asset type can be created. The following steps show how to do this.

  1. Create a CYIAsset subclass

    This class must contain any data required to represent the custom asset, in whatever format makes the most sense. An RTTI definitions is required as the following steps will rely on it.

    class CustomAsset : public CYIAsset
    {
    public:
    CYIString contents;
    YI_TYPE_BASES(CustomAsset, CYIAsset);
    };
    YI_TYPE_DEF(CustomAsset, CYIAsset);
  2. Create a decoder

    The decoder must parse the asset file and optionally decode the asset from a data stream. The decoder is also responsible for populating an instance of the CYIAsset subclass created in step 1.

    class CustomAssetDecoder : public CYIAssetDecoder
    {
    public:
    CustomAssetDecoder()
    : CYIAssetDecoder(CustomAsset::GetClassTypeInfo())
    {
    m_SupportedExtensions = {"custom"};
    }
    virtual bool IsFormatSupported(const uint8_t *pData, size_t dataSize)
    {
    // Determine if the data stream represents an asset recognized by this format.
    // Many asset formats will include "magic bytes" at the beginning of the data which can be used to identify it.
    if (dataSize < 6)
    {
    return false;
    }
    const CYIString readMagic = CYIString((const char *)pData, 6);
    return readMagic == CYIString("custom");
    }
    virtual bool DecodeAsset(const std::shared_ptr<CYIAsset> &pAsset, CYITaskBase *pTask = nullptr)
    {
    YI_UNUSED(pTask);
    std::shared_ptr<CustomAsset> pCustomAsset = YiDynamicCast<CustomAsset>(pAsset);
    return pCustomAsset && PopulateAsset(pCustomAsset, pAsset->GetPath(), pAsset->GetLoadParameters());
    }
    virtual bool DecodeAsset(const std::shared_ptr<CYIAsset> &pAsset, const uint8_t *pData, size_t dataSize, CYITaskBase *pTask = nullptr)
    {
    YI_UNUSED(pTask);
    std::shared_ptr<CustomAsset> pCustomAsset = YiDynamicCast<CustomAsset>(pAsset);
    return pCustomAsset && PopulateAsset(pCustomAsset, pData, dataSize, pAsset->GetLoadParameters());
    }
    virtual std::shared_ptr<CYIAsset> DecodeAsset(const CYIString &path, const CYIAssetLoadParams *pDecodeParams = nullptr, CYITaskBase *pTask = nullptr)
    {
    YI_UNUSED(pTask);
    std::shared_ptr<CustomAsset> pAsset(new CustomAsset());
    if (!PopulateAsset(pAsset, path, pDecodeParams))
    {
    return std::shared_ptr<CYIAsset>();
    }
    return pAsset;
    }
    virtual std::shared_ptr<CYIAsset> DecodeAsset(const uint8_t *pData, size_t dataSize, const CYIAssetLoadParams *pDecodeParams = nullptr, CYITaskBase *pTask = nullptr)
    {
    YI_UNUSED(pTask);
    std::shared_ptr<CustomAsset> pAsset(new CustomAsset());
    if (!PopulateAsset(pAsset, pData, dataSize, pDecodeParams))
    {
    return std::shared_ptr<CYIAsset>();
    }
    return pAsset;
    }
    bool PopulateAsset(const std::shared_ptr<CustomAsset> &pAsset, const CYIString &path, const CYIAssetLoadParams *pDecodeParams)
    {
    YI_UNUSED(pAsset);
    YI_UNUSED(pDecodeParams);
    std::FILE *pFile = std::fopen(path.GetData(), "r");
    if (pFile)
    {
    // Populate data here.
    std::fclose(pFile);
    return true;
    }
    return false;
    }
    bool PopulateAsset(const std::shared_ptr<CustomAsset> &pAsset, const uint8_t *pData, size_t dataSize, const CYIAssetLoadParams *pDecodeParams)
    {
    YI_UNUSED(pAsset);
    YI_UNUSED(pData);
    YI_UNUSED(dataSize);
    YI_UNUSED(pDecodeParams);
    // Populate asset using pData and dataSize
    return true;
    }
    };
  3. Register the decoder with the CYIAssetLoader

    The CYIAssetLoader will automatically determine the correct CYIAssetDecoder to use when loading an asset.

    pAssetLoader->AddDecoder(std::make_unique<CustomAssetDecoder>());
  4. Associate the custom CYIAsset with an asset type directory

    The CYIAssetLocator will be able to automatically construct a path after the custom asset is associated with a directory.

    CYIAssetLocator assetLocator = pAssetLoader->GetAssetLocator();
    assetLocator.AddAssetTypeDirectory("custom", CustomAsset::GetClassTypeInfo());
    assetLocator.Refresh();
    pAssetLoader->SetAssetLocator(std::move(assetLocator));

Remote Assets

Assets can be remotely hosted on servers to be later downloaded and used by applications at runtime. A manifest is generated, which contains a representation of the folder/file structure for the remote assets. The generated manifest and remote assets folder are both uploaded to the server, which can be downloaded and used by the application.

Requirements

Limitations

RemoteAssetsManifest.png

Usage

Manifests can be generated by making use of the generate_manifest.rb ruby script located under tools/workflow in the You.i Engine SDK package. It will take a relative path to a specified folder as an argument and generate a manifest in that folder, which will contain information about the folder structure and the assets of that folder. This manifest and the specified folder should be uploaded together and placed in the same directory on the server.

For example - using the folder structure above - the specified folder is RemoteAssets. A manifest will be generated inside of the RemoteAssets. The assets folder and manifest should be uploaded to the server together to the same location, or RemoteAssets should be uploaded.

Note
The manifest is a generated file and should not be manually changed.

The manifest can be downloaded and used in an application as follows:

class RemoteAssetCombinedExample : public CYIApp, public CYISignalHandler
{
public:
virtual bool UserInit() override
{
// Start the network so the manifest and assets can be downloaded if it has not already been started.
CYINetworkConfiguration networkConfig;
networkConfig.SetResponseCacheSize(30 * 1024 * 1024);
const CYIUrl manifestUrl("www.example.com/RemoteManifest.json");
// Create the catalog and setup the signal for when the download is done.
m_pCatalog = CYIRemoteAssetCatalog::Create("ExampleCatalog");
m_pCatalog->AssetDownloadCompleted.Connect(*this, &RemoteAssetCombinedExample::AssetDownloadComplete);
// Add the specified layout to the download list and the appropriate CYIAsset type.
// The CYIAsset type will be used by the CYIAssetLocator to resolve the asset path.
m_pCatalog->AddAssetToDownloadList("RemoteAssetExample_MainComp.layout", CYIAssetViewTemplate::GetClassTypeInfo());
// Start the download of the manifest and assets.
m_pCatalog->DownloadManifestAndAssets(manifestUrl);
return true;
}
void AssetDownloadComplete()
{
// Use a relative catalog url from the catalog for the specified layout
// so the CYIAssetLocator will know that the asset is from a remote catalog
// and which catalog it is that the asset can be found within.
const CYIString relativeCatalogUrl = m_pCatalog->GetRelativeCatalogUrlForAsset("RemoteAssetExample_MainComp.layout");
// When the download is complete the scene can be created, added and staged as usual.
GetSceneManager()->AddScene("Main", std::move(pSceneViewMain), 0, CYISceneManager::LayerType::Opaque);
}
private:
std::shared_ptr<CYIRemoteAssetCatalog> m_pCatalog;
};

The assets and the manifest can also be downloaded separately as follows:

class RemoteAssetExample : public CYIApp, public CYISignalHandler
{
public:
virtual bool UserInit() override
{
// Start the network so the manifest and assets can be downloaded if it has not already been started.
CYINetworkConfiguration networkConfig;
networkConfig.SetResponseCacheSize(30 * 1024 * 1024);
const CYIUrl manifestUrl("www.example.com/RemoteManifest.json");
// Create the catalog and setup the signal for when the download is done.
m_pCatalog = CYIRemoteAssetCatalog::Create("ExampleCatalog");
m_pCatalog->ManifestDownloadCompleted.Connect(*this, &RemoteAssetExample::ManifestDownloadComplete);
m_pCatalog->AssetDownloadCompleted.Connect(*this, &RemoteAssetExample::AssetDownloadComplete);
// Start the download of the manifest.
m_pCatalog->DownloadManifest(manifestUrl);
return true;
}
void ManifestDownloadComplete()
{
// Add the specified layout to the download list and the appropriate CYIAsset type.
// The CYIAsset type will be used by the CYIAssetLocator to resolve the asset path.
m_pCatalog->AddAssetToDownloadList("RemoteAssetExample_MainComp.layout", CYIAssetViewTemplate::GetClassTypeInfo());
// Start the download of the assets.
m_pCatalog->DownloadAssets();
}
void AssetDownloadComplete()
{
// Use a relative catalog url from the catalog for the specified layout
// so the CYIAssetLocator will know that the asset is from a remote catalog
// and which catalog it is that the asset can be found within.
const CYIString relativeCatalogUrl = m_pCatalog->GetRelativeCatalogUrlForAsset("RemoteAssetExample_MainComp.layout");
// When the download is complete the scene can be created, added and staged as usual.
GetSceneManager()->AddScene("Main", std::move(pSceneViewMain), 0, CYISceneManager::LayerType::Opaque);
}
private:
std::shared_ptr<CYIRemoteAssetCatalog> m_pCatalog;
};

Classes

class  CYIAbstractAssetConfiguration
 Represents a set of qualifiers and rules for resolving paths to assets, used in conjunction with CYIAssetLocator. More...
 
class  CYIAsset
 Base class for any asset. An asset provides functions for dynamically loading and unloading itself and is used by other classes to help manage device resource consumption. A loaded asset can be created by loading an asset file or by programmatically constructing one at run time. Once loaded, an asset occupies memory and should be unloaded whenever possible based on the resource strategy used by your application. Unloading an asset frees the memory associated with the underlying object, such as a CYIBitmap which is associated with an CYIAssetTexture, but does not remove the asset from the CYIAssetManager if it is being managed. More...
 
class  CYIAssetBif
 Asset representing BIF files. More...
 
class  CYIAssetBufferObjectData
 As asset container which wraps shared index or vertex buffer object data. More...
 
class  CYIAssetCacheStrategy
 Base class for an asset caching strategy. Specializations of this class could apply one of more algorithms based on the information provided through the CYIAsset interface. For example, an algorithm could be created so that the combined resource consumption for managed assets do not exceed a maximum value. More...
 
class  CYIAssetConfigurationDataSource
 Represents a data source for asset configuration. More...
 
class  CYIAssetDecoder
 Base class for all specialized decoders. More...
 
class  CYIAssetDecoderBif
 Asset decoder for view templates from After Effects; these typically have a ".layout" file extension. More...
 
class  CYIAssetDecoderGIF
 Asset decoder for GIF image files. More...
 
class  CYIAssetDecoderImage
 Base class for all specialized raster image decoders. More...
 
class  CYIAssetDecoderJPG
 Asset decoder for JPEG image files. More...
 
class  CYIAssetDecoderJSON
 Parses/decodes JSON data. More...
 
class  CYIAssetDecoderPNG
 Asset decoder for PNG image files. More...
 
class  CYIAssetDecoderScript
 Decodes a script source file. More...
 
class  CYIAssetDecoderShaderObject
 Asset decoder for shader objects; these do not have a file format at present, so this class is purely a container. More...
 
class  CYIAssetDecoderTemplate
 Asset decoder for view templates from After Effects; these typically have a ".layout" file extension. More...
 
class  CYIAssetDecoderTGA
 Asset decoder for TGA image files. More...
 
class  CYIAssetDecoderTimelineSource
 Asset decoder for timeline sources from After Effects; these typically have a ".timeline" file extension. More...
 
class  CYIAssetDecoderTranslation
 Decodes an ".mo" binary translation file. More...
 
class  CYIAssetDecoderVideo
 Asset decoder for video files. More...
 
class  CYIAssetDecoderWAV
 Asset decoder for WAVE sound files; these typically have a ".wav" file extension. More...
 
class  CYIAssetDecoderWEBP
 Asset decoder for WEBP image files. More...
 
class  CYIAssetFramebuffer
 Wraps CYIAssetBuffers and their hardware counterparts, and an internally managed FBO. More...
 
class  CYIAssetHardware
 Hardware-mirrored asset. These assets contain IYIGPUObject counterparts. More...
 
class  CYIAssetJSON
 Represents the contents of a JSON file or string. More...
 
class  CYIAssetLoader
 The asset loader provides an interface for loading asset objects, which in turn wrap resource objects like bitmaps, shaders, etc. More...
 
class  CYIAssetLoadParams
 Base class for decoders which can accept specialized parameter objects. More...
 
class  CYIAssetLocator
 Using a base file path, determines the path to an asset given a file name and an asset type. More...
 
class  CYIAssetManager
 This class provides a cache that users can use to store various types of assets. More...
 
class  CYIAssetManifest
 Stores information for the asset manifest. More...
 
class  CYIAssetScript
 Represents the contents of a script source file. More...
 
class  CYIAssetShaderObject
 Shader object asset. Can either be loaded from a precompiled binary, or loaded as source and compiled at run-time. More...
 
class  CYIAssetShaderProgram
 Shader program asset. Contains a vector of CYIAssetShaderObject types. More...
 
class  CYIAssetSound
 Asset representing sound PCM data. More...
 
class  CYIAssetTexture
 A texture asset representing an image which will be loaded into the GPU. More...
 
class  CYIAssetTextureBase
 This asset represents the base class for POT (Power Of Two) and NPOT (Non-Power Of Two) textures. More...
 
class  CYIAssetTextureFramebuffer
 A texture frame buffer representing an image in the GPU that can be drawn to. More...
 
class  CYIAssetTextureRaw
 A texture asset representing an image which will be loaded into the GPU. More...
 
class  CYIAssetTimelineSource
 TimelineSource asset. Wrapper around a CYITimelineBackingStore. More...
 
class  CYIAssetTranslation
 An asset container which holds a shared translator that refers to a specific localisation resource. More...
 
class  CYIAssetVideo
 Asset representing a video. More...
 
class  CYIAssetViewTemplate
 ViewTemplate asset. Wrapper around a CYIViewTemplate. More...
 
class  CYIBif
 Encapulates and provides services for getting the data in BIF (Base Index Frames) file archives. More...
 
class  CYIImageAssetLoadParams
 Asset loading parameters which are specific to image asset decoding. More...
 
struct  _YI_DECODER_BITMAP_PARAMS
 General container for passing parameters to an image decoder. More...
 
class  CYIImageDecoder
 Raster image decoder base class; all raster image decoders should sub-type this class. More...
 
class  CYIImageDecoderGIF
 Image decoder for the GIF file format. Animated images are not supported at this time. More...
 
class  CYIImageDecoderJPG
 Image decoder for the JPEG file format. More...
 
class  CYIImageDecoderPNG
 
class  CYIImageDecoderTGA
 Image decoder for the TGA file format. More...
 
class  CYIImageDecoderWEBP
 
class  CYIRemoteAssetCatalog
 An asset representing remote asset catalogs. More...
 
class  CYIShaderObjectAssetLoadParams
 Asset loading parameters which are specific to shader object asset decoding. More...
 

Typedefs

typedef struct _YI_DECODER_BITMAP_PARAMS YI_DECODER_BITMAP_PARAMS
 General container for passing parameters to an image decoder. More...
 

Typedef Documentation

General container for passing parameters to an image decoder.