Integration Best Practices

The platform provides various integration capabilities so that you can connect your work on the platform with your own environment.

To ensure that this integration remains simple and can grow with your needs, we recommend the following best practices:

1. Separate your inputs from your outputs.
   Each endpoint that you define in the platform is connected to one on your cloud storage.

   1. If the endpoint is used for a recurrent publication, the platform will scan the storage on a regular basis to look for new data revisions requests (TNF) and automatically trigger the import.
      By separating your inputs, you can apply the storage policy that works best for your environment and decide if you want to remove the temporary data after import.

   2. If the endpoint is used for a recurrent export, the platform will write a new dataset after each successful export.
      By separating your outputs, you can quickly identify that this data can be re-created by the platform and apply the appropriate storage policy based on your usage of data.

2. Document your platform integration.
   When you create an endpoint, the platform provides you with instructions to update your cloud data storage security to allow data communication.
   Even if you can access all the configuration information in the administration of the platform, we recommend you store the security information in your documentation system and the part of your system that is connected to it.

3. If you are multi-cloud, prioritize exchange from the native platform storage for high traffic.
   Our platform is able to connect with the major cloud providers (AWS, Azure and GCP). As all the cloud providers charge a fee for the outgoing traffic, if you use multi-clouds (exchanging data from several cloud providers) and plan to have high volume exchange with the platform, we recommend using the same storage as our native platform (GCP) to limit those cloud provider fees and ensuring a better transfer speed.

4. If you are not familiar with cloud provider storage, use our documented articles.
   We fully understand that the first cloud exchange setup can be intimidating. Our documentation provides an initial guide on how to use each of the cloud providers and links to resources to get more information.

5. Look at the publishing or export information to understand your cloud traffic.
   You can view the start/end and size of each data operation performed on the platform on the data product or on the export.
   With this information, you can correlate the cloud traffic with your system logs and associate a data size to the time observed to transmit the data.

6. Do not hesitate to contact our support.
   The Help Center can help you configure your endpoints, guide you in your cloud-to-cloud integration and provide more best practices advice based on your specific cloud deployment.

7. Endpoint Transfer Limits

*Users may also decide to use the file upload capability that is offered alongside the endpoint option. Note that files no larger than 7GB can be uploaded using this method.

Related Pages

Network Integration