This writer allows you to write CSV files into a single AWS S3 bucket. After creating a new configuration, select the files you want to write to AWS S3. You also need to set up the proper permissions on AWS. The writer supports additional processor configuration via the JSON editor.
We strongly recommend that you create a dedicated user for the writer. To do so, create a new user in AWS IAM and enable programmatic access:
Create a new permission policy with the s3:PutObject
and s3:GetBucketLocation
permissions to the target bucket.
Or, if you prefer configuration via JSON:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BucketWrite",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::writer-sample/*",
"arn:aws:s3:::writer-sample"
]
}
]
}
When you finish creating the user, you’ll obtain the Access key ID and Secret access key.
Create a new configuration of the AWS S3 writer. In the next step, provide the target S3 bucket and AWS credentials with write permissions to it.
Save the credentials and configure tables by clicking the Add Table button and search for the table you want to upload:
Then you can modify the table destination path, write the table, or go back to the configuration to add additional tables to the writer. Configured tables are stored as configuration rows.
© 2023 Keboola