So, you want to convert scriptable objects to data that you can use in your ECS systems – but, you can’t directly access SOs using pure ECS. Today, I’m going to share with you how to convert scriptable objects to Blob Assets and also share a utility class that I created while studying Blob Assets.
First off, let’s quickly define what is a blob asset – it’s an immutable data that can be accessed by your systems across multiple threads. Meaning, you can use these in running parallel jobs, which is really important for creating performant systems using Unity’s DOTS. It’s also important to note that you can only use blittable types in your blob assets, and BlobString or FixedString if you want to use strings.
If you want to learn more about getting started with blob assets, Marnel from Squeaky Wheel wrote a guide to help you out. And Code Monkey created an easy-to-understand video, “What are Blob Assets?”. Also, check Unity’s test scripts regarding Blob Assets, they help a lot in understanding how things work.
Before we start – for context, I’m making an NPC generator as a test project for studying ECS. The code here is stable with the packages – Entities 0.16.0-preview.21 and Hybrid Renderer 0.8.0-preview.19.
Jumping in, the best use case for Blob Assets is converting Scriptable Objects (designer-friendly and can be edited in Unity’s editor) to data that can be used in your jobs with burst.
[EDIT – Sept. 12, 2021]
If you’re encountering this error (I encountered this with the packages – Entities 0.17.0-preview.42 and Hybrid Renderer 0.11.0-preview.44 – other dependent packages should update accordingly):
error ConstructBlobWithRefTypeViolation: You may not build a type TBlobAssetType with Construct as TBlobAssetType is a reference or pointer. Only non-reference types are allowed in Blobs.
Unity is already aware of this issue (see this thread for more info), but the fix is included in Entities 0.18, of which there is no ETA yet as of the time of writing of this edit block. In the meantime you can implement this workaround from this post:
"For now, you can comment out loop on line 149 in BlobAssetSafetyVerifier.cs"
I’ll see if I can find another workaround, until then I’ve implemented the workaround above and the project in this tutorial should still work.
[END OF EDIT BLOCK]
Converting Scriptable Objects
Let’s first create the Scriptable Object that we want to convert. Keeping it simple, let’s add an integer for the maximum number of NPCs we want to generate and, for fun, a maximum number of friends an NPC can have.
[CreateAssetMenu(fileName = "NpcManagerData", menuName = "Game/NpcManagerData")]
public class NpcManagerData : ScriptableObject {
[SerializeField]
private int totalNumberOfNpcs;
[SerializeField]
private int totalFriends;
public int TotalNumberOfNpcs => this.totalNumberOfNpcs;
public int TotalFriends => this.totalFriends;
}
We also need a gameobject that will hold this scriptable object,
public class DataContainer : MonoBehaviour {
[SerializeField]
private NpcManagerData npcManagerData;
// Accessor for the conversion system
public NpcManagerData NpcManagerData => this.npcManagerData;
}
In order to convert this to a blob asset, we need to define the structure of the blob asset. For our purposes, the structure will be pretty similar. You can have computations or a specific conversion logic in the conversion system later, if the need arises.
public struct NpcDataBlobAsset {
public int TotalNumberOfNpcs;
public int TotalFriends ;
}
At this point, it’s also important to note that Scriptable Objects are “scene data” – meaning they exist in the “game object” world of Unity. That said, we need a way to convert these “scene data” to DOTS. We can achieve this by using a GameObjectConversionSystem,
[UpdateInGroup(typeof(GameObjectConversionGroup))]
public class TestGameDataSystem : GameObjectConversionSystem {
// We made this static so that other systems can access the blob asset.
// We'll modify this later to work with job systems.
// For now, let's keep it simple.
public static BlobAssetReference<NpcDataBlobAsset> NpcBlobAssetReference;
protected override void OnCreate() {
base.OnCreate();
// Let's debug here to make sure the system ran
Debug.Log("Prefab entities system created!");
}
protected override void OnUpdate() {
// Access the DataContainer attached to a gameObject here and copy the data to a blob asset
this.Entities.ForEach((DataContainer container) => {
// We use a using block since the BlobBuilder needs to be disposed after using it
using (BlobBuilder blobBuilder = new BlobBuilder(Allocator.Temp)) {
// Take note of the "ref" keywords. Unity will throw an error without them, since we're working with structs.
ref NpcDataBlobAsset npcDataBlobAsset = ref blobBuilder.ConstructRoot<NpcDataBlobAsset>();
// Copy data. We'll work with lists/arrays later.
npcDataBlobAsset.TotalNumberOfNpcs = container.NpcManagerData.TotalNumberOfNpcs;
npcDataBlobAsset.TotalFriends = container.NpcManagerData.TotalFriends;
// Store the created reference to the memory location of the blob asset
NpcBlobAssetReference = blobBuilder.CreateBlobAssetReference<NpcDataBlobAsset>(Allocator.Persistent);
}
});
// Print to check if the conversion was successful.
// Note that we have to access the "Value" of where the reference is pointing to.
Debug.Log($"At prefab entities initialization: total npc count is {NpcBlobAssetReference.Value.TotalNumberOfNpcs.ToString()}");
}
}
If you are not familiar with the code above, check Unity’s talk during Unite Copenhagen 2019 on “Converting scene data to DOTS”. It’s important to note that GameObjectConversionSystems work in the world in between the GameObject/Scene world (where gameobjects exist like Prefabs, Scriptable Objects, etc.) and the Entity or “DOTS world” (where your systems are). I’m probably oversimplifying here since you can have multiple worlds. In that case, the GameObjectConversionSystems still lie between the GameObject/Scene world and your worlds.
Before we test the code, make sure that you have the DataContainer (the one where you put the references to the scriptable object) in a subscene. Because subscenes convert the gameObjects in them to Entites when you “close” them in the Inspector. See Unity’s Unite Copenhagen talk for more info. Here’s what the hierarchy looks like in the editor,
Running the game will print these in the console,
Cool, now we converted the Scriptable Object to a Blob Asset that can be accessed using the static BlobAssetReference in our TestGameDataSystem. Let’s add two more things – an entity with the blob asset reference that can be accessed in our job systems, and the promised blob asset utility class.
Accessing Blob Assets in DOTS/ECS Systems
First, we need a component data that we will attach to the entity,
// This will be used by job systems to access blob asset data,
// since we cannot access static non-readonly fields in jobs
public struct BlobAssetReferences : IComponentData {
public BlobAssetReference<NpcDataBlobAsset> NpcManager;
}
Next, let’s add this component to a new entity which we’ll create right after generating the NpcDataBlobAsset, in our GameObjectConversionSystem earlier,
// ...the rest of the GameObjectConversionSystem earlier
Debug.Log($"At prefab entities initialization: total npc count is {NpcBlobAssetReference.Value.TotalNumberOfNpcs.ToString()}");
// We use the default world here since this is attached to a gameobject in a subscene which is in itself, a World.
// We have 3 worlds at this point: Default, Subscene, and Subscene entity conversion world
EntityManager defaultEntityManager = World.DefaultGameObjectInjectionWorld.EntityManager;
Entity gameDataEntity = defaultEntityManager.CreateEntity();
BlobAssetReferences blobAssetReferences = new BlobAssetReferences {
NpcManager = NpcBlobAssetReference
};
defaultEntityManager.AddComponentData(gameDataEntity, blobAssetReferences);
}
}
Be careful when creating entities in GameObjectConversionSystems since there are 2 EntityManagers there, since we’re working with 2 worlds while in the GameObjectConversionSystems – one that is used in the conversion world, and one that is used in the default world. If you are creating an entity that you want to use in the default world, use the World.DefaultGameObjectInjectionWorld.EntityManager and not the EntityManager in the GameObjectConversionSystems.
Now, let’s create a simple system to see if we can access the blob asset from a job system and run the game,
[UpdateInGroup(typeof(SimulationSystemGroup))]
public class TestSystem : SystemBase {
protected override void OnUpdate() {
if (!TestGameDataSystem.NpcManager.IsCreated) {
// Don't do anything if the NpcManager is not yet created
return;
}
this.Entities.ForEach((Entity entity, int entityInQueryIndex, ref BlobAssetReferences blobAssetReferences) => {
NpcDataBlobAsset npcDataBlobAsset = blobAssetReferences.NpcManager.Value;
for (int i = 0; i < npcDataBlobAsset.TotalNumberOfNpcs; i++) {
// you can now access the NpcDataBlobAsset here
}
}).ScheduleParallel();
}
}
Blob Asset Utility Class
Cool! Next, let’s create another blob asset, but this time let’s refactor the BlobBuilder in our TestGameDataSystem to a utility class, so that we can simplify our code and make it easier to read,
public static class BlobAssetUtils {
private static BlobBuilder BLOB_BUILDER;
// We expose this to the clients to allow them to create BlobArray using BlobBuilderArray
public static BlobBuilder BlobBuilder => BLOB_BUILDER;
// We allow the client to pass an action containing their blob creation logic
public delegate void ActionRef<TBlobAssetType, in TDataType>(ref TBlobAssetType blobAsset, TDataType data);
public static BlobAssetReference<TBlobAssetType> BuildBlobAsset<TBlobAssetType, TDataType>
(TDataType data, ActionRef<TBlobAssetType, TDataType> action) where TBlobAssetType : struct {
BLOB_BUILDER = new BlobBuilder(Allocator.Temp);
// Take note of the "ref" keywords. Unity will throw an error without them, since we're working with structs.
ref TBlobAssetType blobAsset = ref BLOB_BUILDER.ConstructRoot<TBlobAssetType>();
// Invoke the client's blob asset creation logic
action.Invoke(ref blobAsset, data);
// Store the created reference to the memory location of the blob asset, before disposing the builder
BlobAssetReference<TBlobAssetType> blobAssetReference = BLOB_BUILDER.CreateBlobAssetReference<TBlobAssetType>(Allocator.Persistent);
// We're not in a Using block, so we manually dispose the builder
BLOB_BUILDER.Dispose();
// Return the created reference
return blobAssetReference;
}
}
As for our TestGameDataSystem, the OnUpdate function will look like,
protected override void OnUpdate() {
this.Entities.ForEach((DataContainer container) => {
// Use the Utility class here - pass the container data, then the conversion logic as an action
NpcBlobAssetReference = BlobAssetUtils.BuildBlobAsset(container.NpcManagerData, delegate(ref NpcDataBlobAsset blobAsset, NpcManagerData data) {
blobAsset.TotalNumberOfNpcs = data.TotalNumberOfNpcs;
blobAsset.TotalFriends = data.TotalFriends;
});
});
// ...the rest of the code
}
Looking clean already. Now, to give you more idea on the use cases of Blob Assets, let me share a simple library of names (using lists) I created and converted to a blob asset for my NPCs. Starting with the Scriptable Object,
[CreateAssetMenu(fileName = "NamesData", menuName = "Game/NamesData")]
public class NamesData : ScriptableObject {
public List<string> firstNames;
public List<string> lastNames;
}
Now the blob asset,
public struct NamesBlobAsset {
// Blob arrays are memory location offsets from the original Blob Asset Reference.
// At least, that's how I understood the manual.
public BlobArray<FixedString32> FirstNames;
public BlobArray<FixedString32> LastNames;
}
Lastly, the creation logic in the TestGameDataSystem,
protected override void OnUpdate() {
this.Entities.ForEach((DataContainer container) => {
// ...the NpcBlobAssetReference generation code here
NamesLibraryReference = BlobAssetUtils.BuildBlobAsset(container.NamesData, delegate(ref NamesBlobAsset blobAsset, NamesData data) {
// Cache the blob builder from the utility class so we can generate blob arrays
BlobBuilder blobBuilder = BlobAssetUtils.BlobBuilder;
BlobBuilderArray<FixedString32> firstNamesArrayBuilder = blobBuilder.Allocate(ref blobAsset.FirstNames, data.firstNames.Count);
BlobBuilderArray<FixedString32> lastNamesArrayBuilder = blobBuilder.Allocate(ref blobAsset.LastNames, data.lastNames.Count);
for (int i = 0; i < data.firstNames.Count; ++i) {
// Copy the data from the list to the BlobBuilderArray
firstNamesArrayBuilder[i] = new FixedString32(data.firstNames[i]);
}
for (int i = 0; i < data.lastNames.Count; ++i) {
lastNamesArrayBuilder[i] = new FixedString32($" {data.lastNames[i]}");
}
// We don't have to worry about "storing" the created array to the blob asset here
// since that is already handled in the BlobAssetUtils. This is just the creation logic
// that is passed to the utility class.
});
});
// ...the rest of the code
}
And that’s it. Blob Assets are a great way of storing data when working with Unity’s DOTS and there are still a lot to learn. I did stumble into some hurdles and errors here and there while trying to make a simple NPC generator. I’m still working on it but I’ll make sure to write another article/guide when I make any significant progress. See you in the next one!
References:
- Unity’s Scripting API
- Code Monkey’s video “What are Blob Assets?”
- Turbo Makes Games’s video “Converting GameObjects to Entities”
- Unite Copenhagen 2019 Talk on “Converting scene data to DOTS”
- Coffee Bran Games (Marnel)’s “Getting Started with Blob Asset”
Neil Mark Corre 