AWS S3

This writer allows you to write CSV files into a single AWS S3 bucket. After creating a new configuration, select the files you want to write to AWS S3. You also need to set up the proper permissions on AWS. The writer supports additional processor configuration via the JSON editor.

Create New Configuration

Find the AWS S3 writer in the list of writers and create a new configuration. Name it.

Screenshot - Create configuration

In the next step, provide the target S3 bucket and AWS credentials with write permissions to it. We strongly recommend that you create a dedicated user for the writer. To do so, create a new user in AWS IAM and enable programmatic access:

Screenshot - New user

Create a new permission policy with the s3:PutObject and s3:GetBucketLocation permissions to the target bucket.

Screenshot - Permission settings

Or, if you prefer configuration via JSON:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "BucketWrite",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::writer-sample/*",
                "arn:aws:s3:::writer-sample"
            ]
        }
    ]
}

When you finish creating the user, you’ll obtain the Access key ID and Secret access key. Enter them to the extractor configuration together with the destination bucket name:

Screenshot - Configure credentials

Save the credentials.

Configure Tables

To configure tables, click the Add Table button and search for the table you want to upload:

Screenshot - Select table

Then you can modify the table destination path, write the table, or go back to the configuration to add additional tables to the writer. The configuration can write as many tables as you wish. The list is fully searchable, and you can delete or disable each table. In addition, you can explicitly write one table only. The write order of the tables can be changed.