The FTP extractor loads a single, or multiple CSV files from FTP, FTPS and SFTP servers, and stores them in Keboola Connection Storage.
Find the FTP extractor in the list of extractors and create a new configuration. Name it.
To access the files on FTP, you need to set up access credentials.
You can use a private key for the SFTP connection type. If you fill in the private key, the extractor will use it for login instead of the password. You can also change the port for each connection method. The default port for FTP and FTPS is 21, and for SFTP, it is 22.
To create a new table, click the New Table button and assign a name. It will be used to create the destination table name in Storage and can be modified.
The configuration can extract as many tables as you wish. The list is fully searchable, and you can delete or disable each table. In addition, you can explicitly run extraction of only one table. The extraction order of the tables can be changed.
Each table has a different setting, but all tables share the same credentials to the server.
For each table, you have to specify a path. Use the path to a specific file, or
use the glob syntax (for testing the glob path, you can use the Glob tester).
If you want to list inside subdirectories (e.g.,
/dir/**/*.csv), please make sure that your FTP server
supports a recursive list of directories.
Now determine how to save the data in Storage.
There are three options for determining column names:
Primary Key can be used to specify the primary key in Storage. It can be used with Incremental Load and New Files Only to create a configuration that incrementally loads all new files into a table in Storage.
The extractor can optionally add audit columns to the table.
ftp_filename adds the processed file name, and
adds the row number in the source file.
For more features, switch the configuration of each table to the Power User Mode by clicking the Open JSON editor link.
Through editing the full JSON configuration, you can set up
Changing the JSON configuration may render the visual form unable to represent the configuration, and switching back may be disabled. Reverting such changes will re-enable the visual form. But whenever possible, the JSON will translate back to the visual form and vice versa.