This article describes an example of the steps that could be taken to create a custom asset using assets or products that users already have consumption access for. This journey is described across a range of distinct personas that may be involved.
Persona 1 - Data Engineer
\uD83D\uDCD8 Writing code
The process may start with a Data engineer who does some work to create some new output data.
Create a new Space and add only the product(s) that need to be customized.
Write the code (in a notebook or the sql in hue) that writes the required output into the publish db.
Publish_db is the default supplied database for outputs you would like to extract from a space following some space activity. When a new asset is created, it will be using the publish_db as its source. For this reason its important to ensure that only the data you wish to extract as custom assets for consumption downstream is written to publish_db.
Run queries to validate the publish db now contains only the data that you want to be in the new customized data product.
Save notebook or query.
\uD83D\uDCD8 Creating a Task
Once the analysis work has been completed in a Space and the output has been written to the publish_db, the user comes out of the spaces portal and returns to the platform where they have the ability to create a Task.
A task enables some code to be run on a repeating basis to continually re-create data outputs. This code is run on updating data so that the output generated is always recent.
Navigate to Code assets via the automation feature found in the main nav bar.
An asset creator role is required for this process.
Create a code asset using the saved code.
Create and run a task using the saved code.
Set trigger to run when source product updates
Add fulfilment team members to ‘share’ task.
Check task runs successfully.
Advise fulfilment team of task name and confirm product ready to be created.
Persona 2 - Data Fulfilment
\uD83D\uDCD8 Creating an Asset and adding it to a product
Now that there is a code asset being run successfully by a task, the task can be used as a source to create a data asset. This asset will update with each scheduled task run. This will ensure that consumers receive updated data feeds when they should.
The user publishing the customized asset needs to have an ‘unenforced subscription’ to the source product. This is the product that was first used in the first section of this guide “Writing code - data engineer”
Create an asset with “Task” as the source.
The user must hold an Asset creator and a Product creator role to do this.
Select the correct Task as the source.
Complete set up by filling in any necessary asset sections.
Release the asset.
Navigate to the product management view.
Create a new product.
In the asset section of the product overview, add the asset you just created.
Select subscription plans for the product.
Set visibility of the product to include the target org.
Add new custom product tag.
Release the product
The user must hold a Product Administrator role to do this.
Persona 3 - Customer
Now that you’ve created a product containing assets that will be updated according to the running of a code asset by a task, it is ready for consumers to subscribe to it and benefit from continually refreshing data. To do this the customer must:
Find the product on exchange.
Subscribe.
Set up export
Use connector and check the “Run this export automatically whenever the underlying product/asset is refreshed” option.
