Learn about the fields used to create a Flat file connection with ThoughtSpot DataFlow.

Here is a list of the fields for a Flat file connection in ThoughtSpot DataFlow. You need specific information to establish a seamless and secure connection.

Connection properties

Connection name
Name your connection.
Mandatory field.
Example:
FilesConnection
Connection type
Choose the Files connection type.
Mandatory field.
Example:
Files
File location
Specify the base location of the file on the server.
Mandatory field.
Example:
/home/admin/dataflow/data
Files on remote location
Specify If the files on remote server.
Mandatory field.
Default:
No
Other notes:
These are under Advanced configuration
Protocol
Select the required remote server connection
Mandatory field.
For remote location files only.
Example:
FTP
Valid Values:
FTP, SFTP
Default:
FTP
Authentication type
Specify the authentication type for SFTP Protocol
Mandatory field.
For SFTP protocol only
Example:
Password
Valid Values:
Key, Password
Default:
Password
Host
Specify the Hostname or the IP address of the remote server
Mandatory field.
For remote location files only.
Example:
www.example.com
Port
Specify the Port to connect the remote server
Mandatory field.
For remote location files only.
Example:
1234
User
Specify the user to connect to remote server. This user must have data access privileges.
Mandatory field.
For remote location files only.
Example:
userdi
Password
Specify the password.
Optional field.
For remote location files only, when using password authentication.
Example:
pswrd234%!
Key file
Specify the key file and its fully qualified path.
Mandatory field.
For remote location files only, when using key authentication.
Example:
/app/key/sftp.key
Passphrase for key file
Specify the passphrase for the key file.
Optional field.
For remote location files only, when using key authentication.
Example:
!hello

Sync properties

File name
Specify name of the file.
Mandatory field.
Example:
CUSTOMER.csv
Valid Values:
Any string literal
Default:
The name of the file
Other notes:
To specify wildcard, use '*' character in the file name
Column delimiter
Specify the column delimiter character.
Mandatory field.
Example:
,
Valid Values:
Any printable ASCII character or decimal value for ASCII character
Default:
The delimiter specified in sync
Enable archive on success
Specify if system is to archive file after successful sync.
Optional field.
Example:
No
Valid Values:
Yes, No
Default:
No
Delete file on success
Specify if system to delete file after successful sync.
Optional field.
Example:
No
Valid Values:
Yes, No
Default:
No
Compression
Specify if file uses complression, and compression type.
Optional field.
Example:
gzip
Valid Values:
None, gzip
Default:
None
Skip header rows
Specify the number of header rows to skip when loading data.
Optional field.
Example:
5
Valid Values:
Any numeric value
Default:
0
Enclosing character
Specify if the text columns in the source data needs to be enclosed in quotes.
Optional field.
Example:
Single
Valid Values:
Single, Double, Empty
Default:
Empty
Escape character
Specify escape characer if data uses text qualifier is mentioned. This should be the character which escapes the text qualifier character in the source data.
Optional field.
Example:
\\
Valid Values:
Any ASCII character
Default:
Empty
Null value
Specifies the string literal indicates the null value for a column. During the data load, the column value matching this string will be loaded as null in the target.
Optional field.
Example:
NULL
Valid Values:
Any string literal
Default:
NULL
Date style
Specifies how to interpret the date format
Optional field.
Example:
YMD
Valid Values:
YMD, MDY, DMY, DMONY, MONDY, Y2MD, MDY2, DMY2, DMONY2, and MONDY2
Default:
YMD
Other notes:
MDY: 2-digit month, 2-digit day, 4-digit year
DMY: 2-digit month, 2-digit day, 4-digit year
DMONY: 2-digit day, 3-character month name, 4-digit year
MONDY: 3-character month name, 2-digit day, 4-digit year
Y2MD: 2-digit year, 2-digit month, 2-digit day
MDY2: 2-digit month, 2-digit day, 2-digit year
DMY2: 2-digit day, 2-digit month, 2-digit year
DMONY2: 2-digit day, 3-character month name, 2-digit year
MONDY2: 3-character month name, 2-digit day, 2-digit year
Date delimiter
Specifies the separator used in the date format.
Optional field.
Example:
-
Valid Values:
Any printable ASCII character
Default:
-
Time style
Specifies the format of the time portion in the data.
Optional field.
Example:
24HOUR
Valid Values:
12 Hour, 24 Hour
Default:
24HOUR
Time delimiter
Specifies the character used as separate the time components.
Optional field.
Example:
:
Valid Values:
Any printable ASCII character
Default:
:
Statistics
Specify if the file source count needs to be displayed in Monitor statistics.
Optional field.
Example:
ON
Valid Values:
OFF
Default:
ON
Skip trailer rows
Skip trailer row while processing the data
Optional field.
Example:
5
Valid Values:
Any numeric value
Default:
0
Reader encoder
Specify the encoding format of the source system.
Optional field.
Example:
UFT-8
Valid Values:
ISO-8859-1, UTF-16, UTF-16BE, WINDOWS-1252
Default:
UFT-8
TS load options
Specifies the parameters passed with the tsload command, in addition to the commands already included by the application. The format for these parameters is:
--<param_1_name> <optional_param_1_value>
--<param_2_name> <optional_param_2_value>
Optional field.
Example:
--max_ignored_rows 0
Valid Values:
--user "dbuser"
--password "$DIWD"
--target_database "ditest"
--target_schema "falcon_schema"
Default:
--max_ignored_rows 0