Prepare table manipulating configurations

In this section you will prepare component configurations that manipulate tables and buckets in storage. We’ll use those configurations in the following sections to show how branches work.

You can create a configuration of any component in a branch and the following steps are not strictly necessary – you can use branches in a completely empty project. However, the tutorial follows the typical situation when you’re using branches in a non-empty project.

Prepare the production configurations

You’ll start with a data pipeline that pulls data about bitcoin prices and creates a list of top 5 days when the price was the highest. Let’s prepare the production configurations, so that you can try working with branches later.

First you need to pull the bitcoin data. To simplify this, you can download a prepared csv file using HTTP extractor.

Set up HTTP extractor in production

Create new HTTP extractor configuration, fill in Base URL to https://help.keboola.com. Then add a new table to the extractor, named bitcoin_price and fill the Path to /tutorial/branches/bitcoin_price.csv. Table Name should be bitcoin_price.

Prepared HTTP extractor row

Prepared HTTP extractor

Run the extractor and verify that a new table in.c-keboola-ex-http-682373219.bitcoin_price was created.

Note: The number in your bucket name represents configuration ID and will be different from what you see on the screenshot.

Set up transformation

First create a new Snowflake transformation, named Bitcoin.

New snowflake transformation

In the Table Input Mapping section fill in the bitcoin_price table that you created using HTTP extractor.

Snowflake input mapping

In Table Output Mapping add top_prices table that will be created in the transformation.

Snowflake output mapping

Finally, add a new code to Block 1 named Top prices with following query.

CREATE TABLE "top_prices" AS SELECT * FROM "bitcoin_price" ORDER BY PRICE DESC LIMIT 5;

ALTER TABLE "top_prices" DROP COLUMN "_timestamp";

Snowflake output mapping

Finished transformation

Save the transformation and run it. Then verify that there is a new table out.c-bitcoin.top_prices containing 5 values from the source data – dates and amounts when bitcoin had the most value.

New table

Now you have the production set and in the next section you’ll set up a Python transformation using file storage. After that you can give branches a test run.