dbt transformation

Configuration

dbt Project Repository

First you must define a repository by specifying the URL (ending with GIT) and entering the access credentials if required.

After saving a configuration, click Load branches to select the desired branch. Do not forget to click Save.

Execution Steps

Select the desired execution steps and rearrange them if needed. If there are other steps you require and you don’t see them, please send us a feature suggestion through the help icon.

Execution Parameters

You can define specific run parameters and node selectors. In a nutshell, it is everything after --select. Please refer to the official dbt core documentation.

You can also specify a path, tags, and specific models.

Note: The default thread level for the Keboola dbt transformation is set to 4. You can override this by using the --threads X parameter in the Execution Parameters.

Output Mapping (Keboola Storage Component Only)

This is a specific configuration needed for the Keboola dbt component. Define which tables will be imported within storage. This configuration uses a standard output mapping UI elements with configuration options, such as incremental or full load, filters, etc.

Database Connection (Remote DWH Components Only)

Configuration parameters expose typical dbt connection configurations, adjusted to the specific adapter settings. To validate a connection, run a debug job from the right menu.

Note: You can control threads for the execution as part of these settings.

Run Debug Job

To test whether you are using the correct credentials and the overall project is correct, you can run a debug job. This is the same as running dbt debug from the command prompt.

The Run debug button will create a separate job with standard logging, exposing the results of the dbt debug command:

Manually Triggering dbt Transformation

When you run a dbt transformation manually, a new job is being triggered with standard logging and storing information such as:

person (token) triggered job
start, end and duration of the job
job parameters
component execution log
dbt deps and repository information
full dbt log for all steps defined
storage output (Keboola dbt)
record of producing and storing artifacts

You can also access all configuration jobs from the configuration screen and the Jobs menu section.

Discover

The Discover tab is designed to provide more information about the run. Keboola aims to expand the insights on this tab to provide more information to you. Currently, it provides the timeline designed to visually display the duration of each model build.

dbt Docs

When you press dbt Project documentation, the job will generate the necessary files within artifacts to power documentation. dbt docs are then accessible on the button from the main configuration screen.

The button performs a synchronous action to generate docs in the popup: