Blacksmith
Overview
IntegrationsPricing

Appropriate policies

Policies are a really important feature of Blacksmith. They have the effect of adding a reliability layer atop an integration, in a durable and scalable way. Misconfigured policies can lead to an unreliable integration.

A complete policies configuration for an integration shall look like this:

sql:
  - name: "my-sql-store"
    driver: "postgresql"
    policies:
      migration:
        path: "./warehouse/migrations"
        timeout: 45
      operation:
        timeout: 45
      load:
        timeout: 15
        initialInterval: 30
        backoffCoefficient: 1.75
        maximumInterval: 15
        maximumAttempts: 60

Note: The operation and migration policies are only available for sql integrations.

migration policies:

  • path is the relative path to the directory where migrations' files of the integration are versioned. Required to enable migrations.
  • timeout (in seconds) is the maximum time of a single database migration's execution attempt. This timeout should be as short as the longest possible execution of the migration. Required if path is set.

operation policies:

  • timeout (in seconds) is the maximum time of a single database operation's execution attempt. This timeout should be as short as the longest possible execution of the operation. Required.

load policies:

  • timeout (in seconds) is the maximum time of a single data Load execution attempt. This timeout should be as short as the longest possible execution of the Load. Required.
  • initialInterval (in seconds) is the backoff interval for the first retry. Defaults to 5.
  • backoffCoefficient is the coefficient used to calculate the next retry backoff interval. The next retry interval is previous interval multiplied by this coefficient. Must be 1.0 or larger. Defaults to 2.0.
  • maximumInterval (in seconds) is the maximum backoff interval between retries. Exponential backoff leads to interval increase. This value is the cap of the interval. Defaults to 100 * initialInterval.
  • maximumAttempts is the maximum number of attempts of retries. When exceeded the retries stop even if not expired yet.
Is something missing?

If you notice something we've missed or could be improved on, please follow this link and submit a pull request to the repository. Once we merge it, the changes will be reflected on the website the next time it is deployed.

Thank you for your contributions!