--- results/org.apache.nifi/nifi/2.0.0-M4/oss-rebuild/reference/minifi-standard-nar-2.0.0-M4-nar-extension-manifest.xml:extension-manifest.xml 2025-04-04 15:05:48.765425101 +0000 +++ results/org.apache.nifi/nifi/2.0.0-M4/oss-rebuild/rebuild/minifi-standard-nar-2.0.0-M4-nar-extension-manifest.xml:extension-manifest.xml 2025-04-04 15:05:54.517334790 +0000 @@ -39,7 +39,7 @@ ' line terminator is recognized in the behavior of '.', '^', and '$'. Can also be specified via the embedded flag (?d).falsetruetruefalsefalsetruefalsefalseNONEfalsefalseInclude Capture Group 0Include Capture Group 0Indicates that Capture Group 0 should be included as an attribute. Capture Group 0 represents the entirety of the regular expression match, is typically not used, and could have considerable length.truetruetruefalsefalsetruefalsefalseNONEfalsefalseEnable repeating capture groupEnable repeating capture groupIf set to true, every string matching the capture groups will be extracted. Otherwise, if the Regular Expression matches more than once, only the first match will be extracted.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseEnable named group supportEnable named group supportIf set to true, when named groups are present in the regular expression, the name of the group will be used in the attribute name as opposed to the group index. All capturing groups must be named, if the number of groups (not including capture group 0) does not equal the -number of named groups validation will fail.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseA FlowFile attributeA Regular Expression with one or more capturing groupThe first capture group, if any found, will be placed into that attribute name.But all capture groups, including the matching string sequence itself will also be provided at that attribute name with an index value provided.NONEunmatchedFlowFiles are routed to this relationship when no provided Regular Expression matches the content of the FlowFilefalsematchedFlowFiles are routed to this relationship when the Regular Expression is successfully evaluated and the FlowFile is modified as a resultfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.FetchDistributedMapCachePROCESSORComputes cache key(s) from FlowFile attributes, for each incoming FlowFile, and fetches the value(s) from the Distributed Map Cache associated with each key. If configured without a destination attribute, the incoming FlowFile's content is replaced with the binary data received by the Distributed Map Cache. If there is no value stored under that key then the flow file will be routed to 'not-found'. Note that the processor will always attempt to read the entire cached value into memory before placing it in it's destination. This could be potentially problematic if the cached value is very large.mapcachefetchdistributedCache Entry IdentifierCache Entry IdentifierA comma-delimited list of FlowFile attributes, or the results of Attribute Expression Language statements, which will be evaluated against a FlowFile in order to determine the value(s) used to identify duplicates; it is these values that are cached. NOTE: Only a single Cache Entry Identifier is allowed unless Put Cache Value In Attribute is specified. Multiple cache lookups are only supported when the destination is a set of attributes (see the documentation for 'Put Cache Value In Attribute' for more details including naming convention.${hash.value}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseDistributed Cache ServiceDistributed Cache ServiceThe Controller Service that is used to get the cached values.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsePut Cache Value In AttributePut Cache Value In AttributeIf set, the cache value received will be put into an attribute of the FlowFile instead of a the content of theFlowFile. The attribute key to put to is determined by evaluating value of this property. If multiple Cache Entry Identifiers are selected, multiple attributes will be written, using the evaluated value of this property, appended by a period (.) and the name of the cache entry identifier.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax Length To Put In AttributeMax Length To Put In AttributeIf routing the cache value to an attribute of the FlowFile (by setting the "Put Cache Value in attribute" property), the number of characters put to the attribute value will be at most this amount. This is important because attributes are held in memory and large attributes will quickly cause out of memory issues. If the output goes longer than this value, it will be truncated to fit. Consider making this smaller if able.256falsefalsefalseNONEfalsefalseCharacter SetCharacter SetThe Character Set in which the cached value is encoded. This will only be used when routing to an attribute.UTF-8falsefalsefalseNONEfalsefalsesuccessIf the cache was successfully communicated with it will be routed to this relationshipfalsefailureIf unable to communicate with the cache or if the cache entry is evaluated to be blank, the FlowFile will be penalized and routed to this relationshipfalsenot-foundIf a FlowFile's Cache Entry Identifier was not found in the cache, it will be routed to this relationshipfalseuser-definedIf the 'Put Cache Value In Attribute' property is set then whatever it is set to will become the attribute key and the value would be whatever the response was from the Distributed Map Cache. If multiple cache entry identifiers are selected, multiple attributes will be written, using the evaluated value of this property, appended by a period (.) and the name of the cache entry identifier. For example, if the Cache Entry Identifier property is set to 'id,name', and the user-defined property is named 'fetched', then two attributes will be written, fetched.id and fetched.name, containing their respective values.trueINPUT_REQUIREDorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientServiceorg.apache.nifi.distributed.cache.server.map.DistributedMapCacheServerorg.apache.nifi.processors.standard.PutDistributedMapCacheorg.apache.nifi.processors.standard.FetchFTPPROCESSORFetches the content of a file from a remote FTP server and overwrites the contents of an incoming FlowFile with the content of the remote file.ftpgetretrievefilesfetchremoteingestsourceinputHostnameHostnameThe fully-qualified hostname or IP address of the host to fetch the data fromtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port to connect to on the remote host to fetch the data from21truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote FileRemote FileThe fully qualified filename on the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCompletion StrategyCompletion StrategySpecifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred.NoneNoneNoneLeave the file as-isMove FileMove FileMove the file to the directory specified by the <Move Destination Directory> propertyDelete FileDelete FileDeletes the original file from the remote systemtruefalsefalseNONEfalsefalseMove Destination DirectoryMove Destination DirectoryThe directory on the remote server to move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to 'Move File'. The specified directory must already exist on the remote system if 'Create Directory' is disabled, or the rename will fail.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCreate DirectoryCreate DirectoryUsed when 'Completion Strategy' is 'Move File'. Specifies whether or not the remote directory should be created if it does not exist.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseConnection ModeConnection ModeThe FTP Connection ModePassiveActiveActivePassivePassivefalsefalsefalseNONEfalsefalseTransfer ModeTransfer ModeThe FTP Transfer ModeBinaryBinaryBinaryASCIIASCIIfalsefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: SOCKS + AuthN, HTTP + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseInternal Buffer SizeInternal Buffer SizeSet the internal buffer size for buffered data streams16KBfalsefalsefalseNONEfalsefalsefetchfiletransfer-notfound-loglevelLog level when file not foundLog level to use in case the file does not exist when the processor is triggeredERRORTRACETRACEDEBUGDEBUGINFOINFOWARNWARNERRORERRORFATALFATALNONENONEtruefalsefalseNONEfalsefalseftp-use-utf8Use UTF-8 EncodingTells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalsecomms.failureAny FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship.falsenot.foundAny FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship.falsepermission.deniedAny FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationship.falseftp.remote.hostThe hostname or IP address from which the file was pulledftp.remote.portThe port that was used to communicate with the remote FTP serverftp.remote.filenameThe name of the remote file that was pulledfilenameThe filename is updated to point to the filename fo the remote filepathIf the Remote File contains a directory name, that directory name will be added to the FlowFile using the 'path' attributefetch.failure.reasonThe name of the failure relationship applied when routing to any failure relationshipINPUT_REQUIREDRetrieve all files in a directory of an FTP Serverftpfiletransformstateretrievefetchallstreamorg.apache.nifi.processors.standard.ListFTPThe "Hostname" property should be set to the fully qualified hostname of the FTP Server. It's a good idea to parameterize this property by setting it to something like `#{FTP_SERVER}`. +number of named groups validation will fail.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseA FlowFile attributeA Regular Expression with one or more capturing groupThe first capture group, if any found, will be placed into that attribute name.But all capture groups, including the matching string sequence itself will also be provided at that attribute name with an index value provided.NONEunmatchedFlowFiles are routed to this relationship when no provided Regular Expression matches the content of the FlowFilefalsematchedFlowFiles are routed to this relationship when the Regular Expression is successfully evaluated and the FlowFile is modified as a resultfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.FetchDistributedMapCachePROCESSORComputes cache key(s) from FlowFile attributes, for each incoming FlowFile, and fetches the value(s) from the Distributed Map Cache associated with each key. If configured without a destination attribute, the incoming FlowFile's content is replaced with the binary data received by the Distributed Map Cache. If there is no value stored under that key then the flow file will be routed to 'not-found'. Note that the processor will always attempt to read the entire cached value into memory before placing it in it's destination. This could be potentially problematic if the cached value is very large.mapcachefetchdistributedCache Entry IdentifierCache Entry IdentifierA comma-delimited list of FlowFile attributes, or the results of Attribute Expression Language statements, which will be evaluated against a FlowFile in order to determine the value(s) used to identify duplicates; it is these values that are cached. NOTE: Only a single Cache Entry Identifier is allowed unless Put Cache Value In Attribute is specified. Multiple cache lookups are only supported when the destination is a set of attributes (see the documentation for 'Put Cache Value In Attribute' for more details including naming convention.${hash.value}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseDistributed Cache ServiceDistributed Cache ServiceThe Controller Service that is used to get the cached values.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsePut Cache Value In AttributePut Cache Value In AttributeIf set, the cache value received will be put into an attribute of the FlowFile instead of a the content of theFlowFile. The attribute key to put to is determined by evaluating value of this property. If multiple Cache Entry Identifiers are selected, multiple attributes will be written, using the evaluated value of this property, appended by a period (.) and the name of the cache entry identifier.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax Length To Put In AttributeMax Length To Put In AttributeIf routing the cache value to an attribute of the FlowFile (by setting the "Put Cache Value in attribute" property), the number of characters put to the attribute value will be at most this amount. This is important because attributes are held in memory and large attributes will quickly cause out of memory issues. If the output goes longer than this value, it will be truncated to fit. Consider making this smaller if able.256falsefalsefalseNONEfalsefalseCharacter SetCharacter SetThe Character Set in which the cached value is encoded. This will only be used when routing to an attribute.UTF-8falsefalsefalseNONEfalsefalsesuccessIf the cache was successfully communicated with it will be routed to this relationshipfalsefailureIf unable to communicate with the cache or if the cache entry is evaluated to be blank, the FlowFile will be penalized and routed to this relationshipfalsenot-foundIf a FlowFile's Cache Entry Identifier was not found in the cache, it will be routed to this relationshipfalseuser-definedIf the 'Put Cache Value In Attribute' property is set then whatever it is set to will become the attribute key and the value would be whatever the response was from the Distributed Map Cache. If multiple cache entry identifiers are selected, multiple attributes will be written, using the evaluated value of this property, appended by a period (.) and the name of the cache entry identifier. For example, if the Cache Entry Identifier property is set to 'id,name', and the user-defined property is named 'fetched', then two attributes will be written, fetched.id and fetched.name, containing their respective values.trueINPUT_REQUIREDorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientServiceorg.apache.nifi.distributed.cache.server.map.DistributedMapCacheServerorg.apache.nifi.processors.standard.PutDistributedMapCacheorg.apache.nifi.processors.standard.FetchFTPPROCESSORFetches the content of a file from a remote FTP server and overwrites the contents of an incoming FlowFile with the content of the remote file.ftpgetretrievefilesfetchremoteingestsourceinputHostnameHostnameThe fully-qualified hostname or IP address of the host to fetch the data fromtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port to connect to on the remote host to fetch the data from21truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote FileRemote FileThe fully qualified filename on the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCompletion StrategyCompletion StrategySpecifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred.NoneNoneNoneLeave the file as-isMove FileMove FileMove the file to the directory specified by the <Move Destination Directory> propertyDelete FileDelete FileDeletes the original file from the remote systemtruefalsefalseNONEfalsefalseMove Destination DirectoryMove Destination DirectoryThe directory on the remote server to move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to 'Move File'. The specified directory must already exist on the remote system if 'Create Directory' is disabled, or the rename will fail.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCreate DirectoryCreate DirectoryUsed when 'Completion Strategy' is 'Move File'. Specifies whether or not the remote directory should be created if it does not exist.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseConnection ModeConnection ModeThe FTP Connection ModePassiveActiveActivePassivePassivefalsefalsefalseNONEfalsefalseTransfer ModeTransfer ModeThe FTP Transfer ModeBinaryBinaryBinaryASCIIASCIIfalsefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseInternal Buffer SizeInternal Buffer SizeSet the internal buffer size for buffered data streams16KBfalsefalsefalseNONEfalsefalsefetchfiletransfer-notfound-loglevelLog level when file not foundLog level to use in case the file does not exist when the processor is triggeredERRORTRACETRACEDEBUGDEBUGINFOINFOWARNWARNERRORERRORFATALFATALNONENONEtruefalsefalseNONEfalsefalseftp-use-utf8Use UTF-8 EncodingTells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalsecomms.failureAny FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship.falsenot.foundAny FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship.falsepermission.deniedAny FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationship.falseftp.remote.hostThe hostname or IP address from which the file was pulledftp.remote.portThe port that was used to communicate with the remote FTP serverftp.remote.filenameThe name of the remote file that was pulledfilenameThe filename is updated to point to the filename fo the remote filepathIf the Remote File contains a directory name, that directory name will be added to the FlowFile using the 'path' attributefetch.failure.reasonThe name of the failure relationship applied when routing to any failure relationshipINPUT_REQUIREDRetrieve all files in a directory of an FTP Serverftpfiletransformstateretrievefetchallstreamorg.apache.nifi.processors.standard.ListFTPThe "Hostname" property should be set to the fully qualified hostname of the FTP Server. It's a good idea to parameterize this property by setting it to something like `#{FTP_SERVER}`. The "Remote Path" property must be set to the directory on the FTP Server where the files reside. If the flow being built is to be reused elsewhere, it's a good idea to parameterize this property by setting it to something like `#{FTP_REMOTE_PATH}`. Configure the "Username" property to the appropriate username for logging into the FTP Server. It's usually a good idea to parameterize this property by setting it to something like `#{FTP_USERNAME}`. Configure the "Password" property to the appropriate password for the provided username. It's usually a good idea to parameterize this property by setting it to something like `#{FTP_PASSWORD}`. @@ -49,7 +49,7 @@ "Remote File" = "${path}/${filename}" "Username" = "${ftp.listing.user}" "Password" = "#{FTP_PASSWORD}" -org.apache.nifi.processors.standard.GetSFTPorg.apache.nifi.processors.standard.PutSFTPorg.apache.nifi.processors.standard.GetFTPorg.apache.nifi.processors.standard.PutFTPorg.apache.nifi.processors.standard.FetchFilePROCESSORReads the contents of a file from disk and streams it into the contents of an incoming FlowFile. Once this is done, the file is optionally moved elsewhere or deleted to help keep the file system organized.localfilesfilesystemingestingressgetsourceinputfetchFile to FetchFile to FetchThe fully-qualified filename of the file to fetch from the file system${absolute.path}/${filename}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCompletion StrategyCompletion StrategySpecifies what to do with the original file on the file system once it has been pulled into NiFiNoneNoneNoneLeave the file as-isMove FileMove FileMoves the file to the directory specified by the <Move Destination Directory> propertyDelete FileDelete FileDeletes the original file from the file systemtruefalsefalseNONEfalsefalseMove Destination DirectoryMove Destination DirectoryThe directory to the move the original file to once it has been fetched from the file system. This property is ignored unless the Completion Strategy is set to "Move File". If the directory does not exist, it will be created.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMove Conflict StrategyMove Conflict StrategyIf Completion Strategy is set to Move File and a file already exists in the destination directory with the same name, this property specifies how that naming conflict should be resolvedRenameRenameRenameThe existing destination file should remain intact. The newly ingested file should be moved to the destination directory but be renamed to a random filenameReplace FileReplace FileThe newly ingested file should replace the existing file in the Destination DirectoryKeep ExistingKeep ExistingThe existing file should in the Destination Directory should stay intact and the newly ingested file should be deletedFailFailThe existing destination file should remain intact and the incoming FlowFile should be routed to failuretruefalsefalseNONEfalsefalseLog level when file not foundLog level when file not foundLog level to use in case the file does not exist when the processor is triggeredERRORTRACETRACEDEBUGDEBUGINFOINFOWARNWARNERRORERRORFATALFATALNONENONEtruefalsefalseNONEfalsefalseLog level when permission deniedLog level when permission deniedLog level to use in case user exceptionfactory does not have sufficient permissions to read the fileERRORTRACETRACEDEBUGDEBUGINFOINFOWARNWARNERRORERRORFATALFATALNONENONEtruefalsefalseNONEfalsefalsesuccessAny FlowFile that is successfully fetched from the file system will be transferred to this Relationship.falsenot.foundAny FlowFile that could not be fetched from the file system because the file could not be found will be transferred to this Relationship.falsepermission.deniedAny FlowFile that could not be fetched from the file system due to the user running NiFi not having sufficient permissions will be transferred to this Relationship.falsefailureAny FlowFile that could not be fetched from the file system for any reason other than insufficient permissions or the file not existing will be transferred to this Relationship.falseread filesystemProvides operator the ability to read from any file that NiFi has access to.write filesystemProvides operator the ability to delete any file that NiFi has access to.INPUT_REQUIREDIngest all files from a directory into NiFilocalfilesfilesystemingestingressgetsourceinputfetchorg.apache.nifi.processors.standard.ListFileConfigure the "Input Directory" property to point to the directory that you want to ingest files from. +org.apache.nifi.processors.standard.GetSFTPorg.apache.nifi.processors.standard.PutSFTPorg.apache.nifi.processors.standard.GetFTPorg.apache.nifi.processors.standard.PutFTPorg.apache.nifi.processors.standard.FetchFilePROCESSORReads the contents of a file from disk and streams it into the contents of an incoming FlowFile. Once this is done, the file is optionally moved elsewhere or deleted to help keep the file system organized.localfilesfilesystemingestingressgetsourceinputfetchFile to FetchFile to FetchThe fully-qualified filename of the file to fetch from the file system${absolute.path}/${filename}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCompletion StrategyCompletion StrategySpecifies what to do with the original file on the file system once it has been pulled into NiFiNoneNoneNoneLeave the file as-isMove FileMove FileMoves the file to the directory specified by the <Move Destination Directory> propertyDelete FileDelete FileDeletes the original file from the file systemtruefalsefalseNONEfalsefalseMove Destination DirectoryMove Destination DirectoryThe directory to the move the original file to once it has been fetched from the file system. This property is ignored unless the Completion Strategy is set to "Move File". If the directory does not exist, it will be created.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMove Conflict StrategyMove Conflict StrategyIf Completion Strategy is set to Move File and a file already exists in the destination directory with the same name, this property specifies how that naming conflict should be resolvedRenameRenameRenameThe existing destination file should remain intact. The newly ingested file should be moved to the destination directory but be renamed to a random filenameReplace FileReplace FileThe newly ingested file should replace the existing file in the Destination DirectoryKeep ExistingKeep ExistingThe existing file should in the Destination Directory should stay intact and the newly ingested file should be deletedFailFailThe existing destination file should remain intact and the incoming FlowFile should be routed to failuretruefalsefalseNONEfalsefalseLog level when file not foundLog level when file not foundLog level to use in case the file does not exist when the processor is triggeredERRORTRACETRACEDEBUGDEBUGINFOINFOWARNWARNERRORERRORFATALFATALNONENONEtruefalsefalseNONEfalsefalseLog level when permission deniedLog level when permission deniedLog level to use in case user aman does not have sufficient permissions to read the fileERRORTRACETRACEDEBUGDEBUGINFOINFOWARNWARNERRORERRORFATALFATALNONENONEtruefalsefalseNONEfalsefalsesuccessAny FlowFile that is successfully fetched from the file system will be transferred to this Relationship.falsenot.foundAny FlowFile that could not be fetched from the file system because the file could not be found will be transferred to this Relationship.falsepermission.deniedAny FlowFile that could not be fetched from the file system due to the user running NiFi not having sufficient permissions will be transferred to this Relationship.falsefailureAny FlowFile that could not be fetched from the file system for any reason other than insufficient permissions or the file not existing will be transferred to this Relationship.falseread filesystemProvides operator the ability to read from any file that NiFi has access to.write filesystemProvides operator the ability to delete any file that NiFi has access to.INPUT_REQUIREDIngest all files from a directory into NiFilocalfilesfilesystemingestingressgetsourceinputfetchorg.apache.nifi.processors.standard.ListFileConfigure the "Input Directory" property to point to the directory that you want to ingest files from. Set the "Input Directory Location" property to "Local" Optionally, set "Minimum File Age" to a small value such as "1 min" to avoid ingesting files that are still being written to. @@ -64,7 +64,7 @@ Connect the 'success' Relationship to the FetchFile processor. org.apache.nifi.processors.standard.FetchFileSet the "File to Fetch" property to `${absolute.path}/${filename}` Set the "Completion Strategy" property to `None` -org.apache.nifi.processors.standard.GetFileorg.apache.nifi.processors.standard.PutFileorg.apache.nifi.processors.standard.ListFileorg.apache.nifi.processors.standard.FetchSFTPPROCESSORFetches the content of a file from a remote SFTP server and overwrites the contents of an incoming FlowFile with the content of the remote file.sftpgetretrievefilesfetchremoteingestsourceinputHostnameHostnameThe fully-qualified hostname or IP address of the host to fetch the data fromtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port to connect to on the remote host to fetch the data from22truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalsePrivate Key PathPrivate Key PathThe fully qualified path to the Private Key filefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILEPrivate Key PassphrasePrivate Key PassphrasePassword for the private keyfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote FileRemote FileThe fully qualified filename on the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCompletion StrategyCompletion StrategySpecifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred.NoneNoneNoneLeave the file as-isMove FileMove FileMove the file to the directory specified by the <Move Destination Directory> propertyDelete FileDelete FileDeletes the original file from the remote systemtruefalsefalseNONEfalsefalseMove Destination DirectoryMove Destination DirectoryThe directory on the remote server to move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to 'Move File'. The specified directory must already exist on the remote system if 'Create Directory' is disabled, or the rename will fail.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCreate DirectoryCreate DirectoryUsed when 'Completion Strategy' is 'Move File'. Specifies whether or not the remote directory should be created if it does not exist.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseDisable Directory ListingDisable Directory ListingControl how 'Move Destination Directory' is created when 'Completion Strategy' is 'Move File' and 'Create Directory' is enabled. If set to 'true', directory listing is not performed prior to create missing directories. By default, this processor executes a directory listing command to see target directory existence before creating missing directories. However, there are situations that you might need to disable the directory listing such as the following. Directory listing might fail with some permission setups (e.g. chmod 100) on a directory. Also, if any other SFTP client created the directory after this processor performed a listing and before a directory creation request by this processor is finished, then an error is returned because the directory already exists.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseSend Keep Alive On TimeoutSend Keep Alive On TimeoutSend a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.truetruetruefalsefalsetruefalsefalseNONEfalsefalseHost Key FileHost Key FileIf supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be usedfalsefalsefalseNONEfalsefalseSINGLEFILEStrict Host Key CheckingStrict Host Key CheckingIndicates whether or not strict enforcement of hosts keys should be appliedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: SOCKS + AuthN, HTTP + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalsefetchfiletransfer-notfound-loglevelLog level when file not foundLog level to use in case the file does not exist when the processor is triggeredERRORTRACETRACEDEBUGDEBUGINFOINFOWARNWARNERRORERRORFATALFATALNONENONEtruefalsefalseNONEfalsefalseCiphers AllowedCiphers AllowedA comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctrfalsefalsetrueENVIRONMENTfalsefalseKey Algorithms AllowedKey Algorithms AllowedA comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.comfalsefalsetrueENVIRONMENTfalsefalseKey Exchange Algorithms AllowedKey Exchange Algorithms AllowedA comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-cfalsefalsetrueENVIRONMENTfalsefalseMessage Authentication Codes AllowedMessage Authentication Codes AllowedA comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.comfalsefalsetrueENVIRONMENTfalsefalsesuccessAll FlowFiles that are received are routed to successfalsecomms.failureAny FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship.falsenot.foundAny FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship.falsepermission.deniedAny FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationship.falsesftp.remote.hostThe hostname or IP address from which the file was pulledsftp.remote.portThe port that was used to communicate with the remote SFTP serversftp.remote.filenameThe name of the remote file that was pulledfilenameThe filename is updated to point to the filename fo the remote filepathIf the Remote File contains a directory name, that directory name will be added to the FlowFile using the 'path' attributefetch.failure.reasonThe name of the failure relationship applied when routing to any failure relationshipINPUT_REQUIREDRetrieve all files in a directory of an SFTP Serversftpsecurefiletransformstateretrievefetchallstreamorg.apache.nifi.processors.standard.ListSFTPThe "Hostname" property should be set to the fully qualified hostname of the FTP Server. It's a good idea to parameterize this property by setting it to something like `#{SFTP_SERVER}`. +org.apache.nifi.processors.standard.GetFileorg.apache.nifi.processors.standard.PutFileorg.apache.nifi.processors.standard.ListFileorg.apache.nifi.processors.standard.FetchSFTPPROCESSORFetches the content of a file from a remote SFTP server and overwrites the contents of an incoming FlowFile with the content of the remote file.sftpgetretrievefilesfetchremoteingestsourceinputHostnameHostnameThe fully-qualified hostname or IP address of the host to fetch the data fromtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port to connect to on the remote host to fetch the data from22truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalsePrivate Key PathPrivate Key PathThe fully qualified path to the Private Key filefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILEPrivate Key PassphrasePrivate Key PassphrasePassword for the private keyfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote FileRemote FileThe fully qualified filename on the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCompletion StrategyCompletion StrategySpecifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred.NoneNoneNoneLeave the file as-isMove FileMove FileMove the file to the directory specified by the <Move Destination Directory> propertyDelete FileDelete FileDeletes the original file from the remote systemtruefalsefalseNONEfalsefalseMove Destination DirectoryMove Destination DirectoryThe directory on the remote server to move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to 'Move File'. The specified directory must already exist on the remote system if 'Create Directory' is disabled, or the rename will fail.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCreate DirectoryCreate DirectoryUsed when 'Completion Strategy' is 'Move File'. Specifies whether or not the remote directory should be created if it does not exist.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseDisable Directory ListingDisable Directory ListingControl how 'Move Destination Directory' is created when 'Completion Strategy' is 'Move File' and 'Create Directory' is enabled. If set to 'true', directory listing is not performed prior to create missing directories. By default, this processor executes a directory listing command to see target directory existence before creating missing directories. However, there are situations that you might need to disable the directory listing such as the following. Directory listing might fail with some permission setups (e.g. chmod 100) on a directory. Also, if any other SFTP client created the directory after this processor performed a listing and before a directory creation request by this processor is finished, then an error is returned because the directory already exists.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseSend Keep Alive On TimeoutSend Keep Alive On TimeoutSend a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.truetruetruefalsefalsetruefalsefalseNONEfalsefalseHost Key FileHost Key FileIf supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be usedfalsefalsefalseNONEfalsefalseSINGLEFILEStrict Host Key CheckingStrict Host Key CheckingIndicates whether or not strict enforcement of hosts keys should be appliedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalsefetchfiletransfer-notfound-loglevelLog level when file not foundLog level to use in case the file does not exist when the processor is triggeredERRORTRACETRACEDEBUGDEBUGINFOINFOWARNWARNERRORERRORFATALFATALNONENONEtruefalsefalseNONEfalsefalseCiphers AllowedCiphers AllowedA comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctrfalsefalsetrueENVIRONMENTfalsefalseKey Algorithms AllowedKey Algorithms AllowedA comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.comfalsefalsetrueENVIRONMENTfalsefalseKey Exchange Algorithms AllowedKey Exchange Algorithms AllowedA comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-cfalsefalsetrueENVIRONMENTfalsefalseMessage Authentication Codes AllowedMessage Authentication Codes AllowedA comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.comfalsefalsetrueENVIRONMENTfalsefalsesuccessAll FlowFiles that are received are routed to successfalsecomms.failureAny FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship.falsenot.foundAny FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship.falsepermission.deniedAny FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationship.falsesftp.remote.hostThe hostname or IP address from which the file was pulledsftp.remote.portThe port that was used to communicate with the remote SFTP serversftp.remote.filenameThe name of the remote file that was pulledfilenameThe filename is updated to point to the filename fo the remote filepathIf the Remote File contains a directory name, that directory name will be added to the FlowFile using the 'path' attributefetch.failure.reasonThe name of the failure relationship applied when routing to any failure relationshipINPUT_REQUIREDRetrieve all files in a directory of an SFTP Serversftpsecurefiletransformstateretrievefetchallstreamorg.apache.nifi.processors.standard.ListSFTPThe "Hostname" property should be set to the fully qualified hostname of the FTP Server. It's a good idea to parameterize this property by setting it to something like `#{SFTP_SERVER}`. The "Remote Path" property must be set to the directory on the FTP Server where the files reside. If the flow being built is to be reused elsewhere, it's a good idea to parameterize this property by setting it to something like `#{SFTP_REMOTE_PATH}`. Configure the "Username" property to the appropriate username for logging into the FTP Server. It's usually a good idea to parameterize this property by setting it to something like `#{SFTP_USERNAME}`. Configure the "Password" property to the appropriate password for the provided username. It's usually a good idea to parameterize this property by setting it to something like `#{SFTP_PASSWORD}`. @@ -85,7 +85,7 @@ This processor can accept incoming connections; the behavior of the processor is different whether incoming connections are provided: - If no incoming connection(s) are specified, the processor will generate SQL queries on the specified processor schedule. Expression Language is supported for many fields, but no FlowFile attributes are available. However the properties will be evaluated using the Environment/System properties. - If incoming connection(s) are specified and no FlowFile is available to a processor task, no work will be performed. - - If incoming connection(s) are specified and a FlowFile is available to a processor task, the FlowFile's attributes may be used in Expression Language for such fields as Table Name and others. However, the Max-Value Columns and Columns to Return fields must be empty or refer to columns that are available in each specified table.sqlselectjdbcquerydatabasefetchgenerateDatabase Connection Pooling ServiceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-fetch-db-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLtruefalsefalseNONEfalsefalseTable NameTable NameThe name of the database table to be queried.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseColumns to ReturnColumns to ReturnA comma-separated list of column names to be used in the query. If your database requires special treatment of the names (quoting, e.g.), each name should include such treatment. If no column names are supplied, all columns in the specified table will be returned. NOTE: It is important to use consistent column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMaximum-value ColumnsMaximum-value ColumnsA comma-separated list of column names. The processor will keep track of the maximum value for each column that has been returned since the processor started running. Using multiple columns implies an order to the column list, and each column's values are expected to increase more slowly than the previous columns' values. Thus, using multiple columns implies a hierarchical structure of columns, which is usually used for partitioning tables. This processor can be used to retrieve only those rows that have been added/updated since the last retrieval. Note that some JDBC types such as bit/boolean are not conducive to maintaining maximum value, so columns of these types should not be listed in this property, and will result in error(s) during processing. If no columns are provided, all rows from the table will be considered, which could have a performance impact. NOTE: It is important to use consistent max-value column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax Wait TimeMax Wait TimeThe maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsegen-table-fetch-partition-sizePartition SizeThe number of result rows to be fetched by each generated SQL statement. The total number of rows in the table divided by the partition size gives the number of SQL statements (i.e. FlowFiles) generated. A value of zero indicates that a single FlowFile is to be generated whose SQL statement will fetch all rows in the table.10000truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsegen-table-column-for-val-partitioningColumn for Value PartitioningThe name of a column whose values will be used for partitioning. The default behavior is to use row numbers on the result set for partitioning into 'pages' to be fetched from the database, using an offset/limit strategy. However for certain databases, it can be more efficient under the right circumstances to use the column values themselves to define the 'pages'. This property should only be used when the default queries are not performing well, when there is no maximum-value column or a single maximum-value column whose type can be coerced to a long integer (i.e. not date or timestamp), and the column values are evenly distributed and not sparse, for best performance.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-where-clauseAdditional WHERE clauseA custom clause to be added in the WHERE condition when building SQL queries.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsegen-table-custom-orderby-columnCustom ORDER BY ColumnThe name of a column to be used for ordering the results if Max-Value Columns are not provided and partitioning is enabled. This property is ignored if either Max-Value Columns is set or Partition Size = 0. NOTE: If neither Max-Value Columns nor Custom ORDER BY Column is set, then depending on the the database/driver, the processor may report an error and/or the generated SQL may result in missing and/or duplicate rows. This is because without an explicit ordering, fetching each partition is done using an arbitrary ordering.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsegen-table-output-flowfile-on-zero-resultsOutput Empty FlowFile on Zero ResultsDepending on the specified properties, an execution of this processor may not result in any SQL statements generated. When this property is true, an empty FlowFile will be generated (having the parent of the incoming FlowFile if present) and transferred to the 'success' relationship. When this property is false, no output FlowFiles will be generated.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseinitial.maxvalue.<max_value_column>Initial maximum value for the specified columnSpecifies an initial max value for max value columns. Properties should be added in the format `initial.maxvalue.<max_value_column>`. This value is only used the first time the table is accessed (when a Maximum Value Column is specified). In the case of incoming connections, the value is only used the first time for each table specified in the FlowFiles.FLOWFILE_ATTRIBUTESsuccessSuccessfully created FlowFile from SQL query result set.falsefailureThis relationship is only used when SQL query execution (using an incoming FlowFile) failed. The incoming FlowFile will be penalized and routed to this relationship. If no incoming connection(s) are specified, this relationship is unused.falsegeneratetablefetch.sql.errorIf the processor has incoming connections, and processing an incoming FlowFile causes a SQL Exception, the FlowFile is routed to failure and this attribute is set to the exception message.generatetablefetch.tableNameThe name of the database table to be queried.generatetablefetch.columnNamesThe comma-separated list of column names used in the query.generatetablefetch.whereClauseWhere clause used in the query to get the expected rows.generatetablefetch.maxColumnNamesThe comma-separated list of column names used to keep track of data that has been returned since the processor started running.generatetablefetch.limitThe number of result rows to be fetched by the SQL statement.generatetablefetch.offsetOffset to be used to retrieve the corresponding partition.fragment.identifierAll FlowFiles generated from the same query result set will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.fragment.countThis is the total number of FlowFiles produced by a single ResultSet. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming ResultSet.fragment.indexThis is the position of this FlowFile in the list of outgoing FlowFiles that were all generated from the same execution. This can be used in conjunction with the fragment.identifier attribute to know which FlowFiles originated from the same execution and in what order FlowFiles were producedtrueAfter performing a query on the specified table, the maximum values for the specified column(s) will be retained for use in future executions of the query. This allows the Processor to fetch only those records that have max values greater than the retained values. This can be used for incremental fetching, fetching of newly added rows, etc. To clear the maximum values, clear the state of the processor per the State Management documentationCLUSTERINPUT_ALLOWEDorg.apache.nifi.processors.standard.QueryDatabaseTableorg.apache.nifi.processors.standard.ExecuteSQLorg.apache.nifi.processors.standard.ListDatabaseTablesorg.apache.nifi.processors.standard.GetFTPPROCESSORFetches files from an FTP Server and creates FlowFiles from themFTPgetretrievefilesfetchremoteingestsourceinputHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers21truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseConnection ModeConnection ModeThe FTP Connection ModePassiveActiveActivePassivePassivefalsefalsefalseNONEfalsefalseTransfer ModeTransfer ModeThe FTP Transfer ModeBinaryBinaryBinaryASCIIASCIIfalsefalsefalseNONEfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push filesfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFile Filter RegexFile Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetchedfalsefalsefalseNONEfalsefalsePath Filter RegexPath Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scannedfalsefalsefalseNONEfalsefalsePolling IntervalPolling IntervalDetermines how long to wait between fetching the listing for new files60 sectruefalsefalseNONEfalsefalseSearch RecursivelySearch RecursivelyIf true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefollow-symlinkFollow symlinkIf true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseIgnore Dotted FilesIgnore Dotted FilesIf true, files whose names begin with a dot (".") will be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseDelete OriginalDelete OriginalDetermines whether or not the file is deleted from the remote system after it has been successfully transferredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseMax SelectsMax SelectsThe maximum number of files to pull in a single connection100truefalsefalseNONEfalsefalseRemote Poll Batch SizeRemote Poll Batch SizeThe value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.5000truefalsefalseNONEfalsefalseUse Natural OrderingUse Natural OrderingIf true, will pull files in the order in which they are naturally listed; otherwise, the order in which the files will be pulled is not definedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: SOCKS + AuthN, HTTP + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseInternal Buffer SizeInternal Buffer SizeSet the internal buffer size for buffered data streams16KBfalsefalsefalseNONEfalsefalseftp-use-utf8Use UTF-8 EncodingTells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalsefilenameThe filename is set to the name of the file on the remote serverpathThe path is set to the path of the file's directory on the remote server. For example, if the <Remote Path> property is set to /tmp, files picked up from /tmp will have the path attribute set to /tmp. If the <Search Recursively> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to /tmp/abc/1/2/3file.lastModifiedTimeThe date and time that the source file was last modifiedfile.lastAccessTimeThe date and time that the file was last accessed. May not work on all file systemsfile.ownerThe numeric owner id of the source filefile.groupThe numeric group id of the source filefile.permissionsThe read/write/execute permissions of the source fileabsolute.pathThe full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative pathINPUT_FORBIDDENorg.apache.nifi.processors.standard.PutFTPorg.apache.nifi.processors.standard.GetFilePROCESSORCreates FlowFiles from files in a directory. NiFi will ignore files it doesn't have at least read permissions for.localfilesfilesystemingestingressgetsourceinputInput DirectoryInput DirectoryThe input directory from which to pull filestruefalsetrueENVIRONMENTfalsefalseFile FilterFile FilterOnly files whose names match the given regular expression will be picked up[^\.].*truefalsefalseNONEfalsefalsePath FilterPath FilterWhen Recurse Subdirectories is true, then only subdirectories whose path matches the given regular expression will be scannedfalsefalsefalseNONEfalsefalseBatch SizeBatch SizeThe maximum number of files to pull in each iteration10truefalsefalseNONEfalsefalseKeep Source FileKeep Source FileIf true, the file is not deleted after it has been copied to the Content Repository; this causes the file to be picked up continually and is useful for testing purposes. If not keeping original NiFi will need write permissions on the directory it is pulling from otherwise it will ignore the file.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseRecurse SubdirectoriesRecurse SubdirectoriesIndicates whether or not to pull files from subdirectoriestruetruetruefalsefalsetruefalsefalseNONEfalsefalsePolling IntervalPolling IntervalIndicates how long to wait before performing a directory listing0 sectruefalsefalseNONEfalsefalseIgnore Hidden FilesIgnore Hidden FilesIndicates whether or not hidden files should be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseMinimum File AgeMinimum File AgeThe minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored0 sectruefalsefalseNONEfalsefalseMaximum File AgeMaximum File AgeThe maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignoredfalsefalsefalseNONEfalsefalseMinimum File SizeMinimum File SizeThe minimum size that a file must be in order to be pulled0 BtruefalsefalseNONEfalsefalseMaximum File SizeMaximum File SizeThe maximum size that a file can be in order to be pulledfalsefalsefalseNONEfalsefalsesuccessAll files are routed to successfalsefilenameThe filename is set to the name of the file on diskpathThe path is set to the relative path of the file's directory on disk. For example, if the <Input Directory> property is set to /tmp, files picked up from /tmp will have the path attribute set to ./. If the <Recurse Subdirectories> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to abc/1/2/3file.creationTimeThe date and time that the file was created. May not work on all file systemsfile.lastModifiedTimeThe date and time that the file was last modified. May not work on all file systemsfile.lastAccessTimeThe date and time that the file was last accessed. May not work on all file systemsfile.ownerThe owner of the file. May not work on all file systemsfile.groupThe group owner of the file. May not work on all file systemsfile.permissionsThe read/write/execute permissions of the file. May not work on all file systemsabsolute.pathThe full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative pathtrueread filesystemProvides operator the ability to read from any file that NiFi has access to.write filesystemProvides operator the ability to delete any file that NiFi has access to.INPUT_FORBIDDENorg.apache.nifi.processors.standard.PutFileorg.apache.nifi.processors.standard.FetchFileorg.apache.nifi.processors.standard.GetSFTPPROCESSORFetches files from an SFTP Server and creates FlowFiles from themsftpgetretrievefilesfetchremoteingestsourceinputHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers22truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalsePrivate Key PathPrivate Key PathThe fully qualified path to the Private Key filefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILEPrivate Key PassphrasePrivate Key PassphrasePassword for the private keyfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push filesfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFile Filter RegexFile Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetchedfalsefalsefalseNONEfalsefalsePath Filter RegexPath Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scannedfalsefalsefalseNONEfalsefalsePolling IntervalPolling IntervalDetermines how long to wait between fetching the listing for new files60 sectruefalsefalseNONEfalsefalseSearch RecursivelySearch RecursivelyIf true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefollow-symlinkFollow symlinkIf true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseIgnore Dotted FilesIgnore Dotted FilesIf true, files whose names begin with a dot (".") will be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseDelete OriginalDelete OriginalDetermines whether or not the file is deleted from the remote system after it has been successfully transferredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseHost Key FileHost Key FileIf supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be usedfalsefalsefalseNONEfalsefalseSINGLEFILEMax SelectsMax SelectsThe maximum number of files to pull in a single connection100truefalsefalseNONEfalsefalseRemote Poll Batch SizeRemote Poll Batch SizeThe value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.5000truefalsefalseNONEfalsefalseStrict Host Key CheckingStrict Host Key CheckingIndicates whether or not strict enforcement of hosts keys should be appliedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseSend Keep Alive On TimeoutSend Keep Alive On TimeoutSend a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.truetruetruefalsefalsetruefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseUse Natural OrderingUse Natural OrderingIf true, will pull files in the order in which they are naturally listed; otherwise, the order in which the files will be pulled is not definedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: SOCKS + AuthN, HTTP + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseCiphers AllowedCiphers AllowedA comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctrfalsefalsetrueENVIRONMENTfalsefalseKey Algorithms AllowedKey Algorithms AllowedA comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.comfalsefalsetrueENVIRONMENTfalsefalseKey Exchange Algorithms AllowedKey Exchange Algorithms AllowedA comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-cfalsefalsetrueENVIRONMENTfalsefalseMessage Authentication Codes AllowedMessage Authentication Codes AllowedA comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.comfalsefalsetrueENVIRONMENTfalsefalsesuccessAll FlowFiles that are received are routed to successfalsefilenameThe filename is set to the name of the file on the remote serverpathThe path is set to the path of the file's directory on the remote server. For example, if the <Remote Path> property is set to /tmp, files picked up from /tmp will have the path attribute set to /tmp. If the <Search Recursively> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to /tmp/abc/1/2/3file.lastModifiedTimeThe date and time that the source file was last modifiedfile.ownerThe numeric owner id of the source filefile.groupThe numeric group id of the source filefile.permissionsThe read/write/execute permissions of the source fileabsolute.pathThe full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative pathINPUT_FORBIDDENorg.apache.nifi.processors.standard.PutSFTPorg.apache.nifi.processors.standard.HandleHttpRequestPROCESSORStarts an HTTP Server and listens for HTTP Requests. For each request, creates a FlowFile and transfers to 'success'. This Processor is designed to be used in conjunction with the HandleHttpResponse Processor in order to create a Web Service. In case of a multipart request, one FlowFile is generated for each part.httphttpsrequestlisteningressweb serviceListening PortListening PortThe Port to listen on for incoming HTTP requests80truefalsetrueENVIRONMENTfalsefalseHostnameHostnameThe Hostname to bind to. If not specified, will bind to all hostsfalsefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceThe SSL Context Service to use in order to secure the server. If specified, the server will accept only HTTPS requests; otherwise, the server will accept only HTTP requestsorg.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseHTTP ProtocolsHTTP ProtocolsHTTP Protocols supported for Application Layer Protocol Negotiation with TLSHTTP_1_1http/1.1HTTP_1_1HTTP/1.1h2 http/1.1H2_HTTP_1_1HTTP/2 and HTTP/1.1 negotiated based on requested protocolsh2H2HTTP/2truefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceHTTP Context MapHTTP Context MapThe HTTP Context Map Controller Service to use for caching the HTTP Request Informationorg.apache.nifi.http.HttpContextMaporg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseAllowed PathsAllowed PathsA Regular Expression that specifies the valid HTTP Paths that are allowed in the incoming URL Requests. If this value is specified and the path of the HTTP Requests does not match this Regular Expression, the Processor will respond with a 404: NotFoundfalsefalsefalseNONEfalsefalseDefault URL Character SetDefault URL Character SetThe character set to use for decoding URL parameters if the HTTP Request does not supply oneUTF-8truefalsefalseNONEfalsefalseAllow GETAllow GETAllow HTTP GET MethodtruetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow POSTAllow POSTAllow HTTP POST MethodtruetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow PUTAllow PUTAllow HTTP PUT MethodtruetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow DELETEAllow DELETEAllow HTTP DELETE MethodtruetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow HEADAllow HEADAllow HTTP HEAD MethodfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow OPTIONSAllow OPTIONSAllow HTTP OPTIONS MethodfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseMaximum ThreadsMaximum ThreadsThe maximum number of threads that the embedded HTTP server will use for handling requests.200truefalsefalseNONEfalsefalseAdditional HTTP MethodsAdditional HTTP MethodsA comma-separated list of non-standard HTTP Methods that should be allowedfalsefalsefalseNONEfalsefalseClient AuthenticationClient AuthenticationSpecifies whether or not the Processor should authenticate clients. This value is ignored if the <SSL Context Service> Property is not specified or the SSL Context provided uses only a KeyStore and not a TrustStore.No AuthenticationNo AuthenticationNo AuthenticationProcessor will not authenticate clients. Anyone can communicate with this Processor anonymouslyWant AuthenticationWant AuthenticationProcessor will try to verify the client but if unable to verify will allow the client to communicate anonymouslyNeed AuthenticationNeed AuthenticationProcessor will reject communications from any client unless the client provides a certificate that is trusted by the TrustStorespecified in the SSL Context ServicetruefalsefalseNONEfalsefalsecontainer-queue-sizeContainer Queue SizeThe size of the queue for Http Request Containers50truefalsefalseNONEfalsefalsemultipart-request-max-sizeMultipart Request Max SizeThe max size of the request. Only applies for requests with Content-Type: multipart/form-data, and is used to prevent denial of service type of attacks, to prevent filling up the heap or disk space1 MBtruefalsefalseNONEfalsefalsemultipart-read-buffer-sizeMultipart Read Buffer SizeThe threshold size, at which the contents of an incoming file would be written to disk. Only applies for requests with Content-Type: multipart/form-data. It is used to prevent denial of service type of attacks, to prevent filling up the heap or disk space.512 KBtruefalsefalseNONEfalsefalseparameters-to-attributesParameters to Attributes ListA comma-separated list of HTTP parameters or form data to output as attributesfalsefalsefalseNONEfalsefalsesuccessAll content that is received is routed to the 'success' relationshipfalsehttp.context.identifierAn identifier that allows the HandleHttpRequest and HandleHttpResponse to coordinate which FlowFile belongs to which HTTP Request/Response.mime.typeThe MIME Type of the data, according to the HTTP Header "Content-Type"http.servlet.pathThe part of the request URL that is considered the Servlet Pathhttp.context.pathThe part of the request URL that is considered to be the Context Pathhttp.methodThe HTTP Method that was used for the request, such as GET or POSThttp.local.nameIP address/hostname of the serverhttp.server.portListening port of the serverhttp.query.stringThe query string portion of the Request URLhttp.remote.hostThe hostname of the requestorhttp.remote.addrThe hostname:port combination of the requestorhttp.remote.userThe username of the requestorhttp.protocolThe protocol used to communicatehttp.request.uriThe full Request URLhttp.auth.typeThe type of HTTP Authorization usedhttp.principal.nameThe name of the authenticated user making the requesthttp.query.param.XXXEach of query parameters in the request will be added as an attribute, prefixed with "http.query.param."http.param.XXXForm parameters in the request that are configured by "Parameters to Attributes List" will be added as an attribute, prefixed with "http.param.". Putting form parameters of large size is not recommended.http.subject.dnThe Distinguished Name of the requestor. This value will not be populated unless the Processor is configured to use an SSLContext Servicehttp.issuer.dnThe Distinguished Name of the entity that issued the Subject's certificate. This value will not be populated unless the Processor is configured to use an SSLContext Servicehttp.certificate.sans.N.nameX.509 Client Certificate Subject Alternative Name value from mutual TLS authentication. The attribute name has a zero-based index ordered according to the content of Client Certificatehttp.certificate.sans.N.nameTypeX.509 Client Certificate Subject Alternative Name type from mutual TLS authentication. The attribute name has a zero-based index ordered according to the content of Client Certificate. The attribute value is one of the General Names from RFC 3280 Section 4.1.2.7http.headers.XXXEach of the HTTP Headers that is received in the request will be added as an attribute, prefixed with "http.headers." For example, if the request contains an HTTP Header named "x-my-header", then the value will be added to an attribute named "http.headers.x-my-header"http.headers.multipart.XXXEach of the HTTP Headers that is received in the multipart request will be added as an attribute, prefixed with "http.headers.multipart." For example, if the multipart request contains an HTTP Header named "content-disposition", then the value will be added to an attribute named "http.headers.multipart.content-disposition"http.multipart.sizeFor requests with Content-Type "multipart/form-data", the part's content size is recorded into this attributehttp.multipart.content.typeFor requests with Content-Type "multipart/form-data", the part's content type is recorded into this attributehttp.multipart.nameFor requests with Content-Type "multipart/form-data", the part's name is recorded into this attributehttp.multipart.filenameFor requests with Content-Type "multipart/form-data", when the part contains an uploaded file, the name of the file is recorded into this attribute. Files are stored temporarily at the default temporary-file directory specified in "java.io.File" Java Docs)http.multipart.fragments.sequence.numberFor requests with Content-Type "multipart/form-data", the part's index is recorded into this attribute. The index starts with 1.http.multipart.fragments.total.numberFor requests with Content-Type "multipart/form-data", the count of all parts is recorded into this attribute.INPUT_FORBIDDENorg.apache.nifi.processors.standard.HandleHttpResponseorg.apache.nifi.processors.standard.HandleHttpResponsePROCESSORSends an HTTP Response to the Requestor that generated a FlowFile. This Processor is designed to be used in conjunction with the HandleHttpRequest in order to create a web service.httphttpsresponseegressweb serviceHTTP Status CodeHTTP Status CodeThe HTTP Status Code to use when responding to the HTTP Request. See Section 10 of RFC 2616 for more information.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHTTP Context MapHTTP Context MapThe HTTP Context Map Controller Service to use for caching the HTTP Request Informationorg.apache.nifi.http.HttpContextMaporg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseAttributes to add to the HTTP Response (Regex)Attributes to add to the HTTP Response (Regex)Specifies the Regular Expression that determines the names of FlowFile attributes that should be added to the HTTP responsefalsefalsefalseNONEfalsefalseAn HTTP header nameAn HTTP header valueThese HTTPHeaders are set in the HTTP ResponseFLOWFILE_ATTRIBUTESsuccessFlowFiles will be routed to this Relationship after the response has been successfully sent to the requestorfalsefailureFlowFiles will be routed to this Relationship if the Processor is unable to respond to the requestor. This may happen, for instance, if the connection times out or if NiFi is restarted before responding to the HTTP Request.falsehttp.context.identifierThe value of this attribute is used to lookup the HTTP Response so that the proper message can be sent back to the requestor. If this attribute is missing, the FlowFile will be routed to 'failure.'http.request.uriValue of the URI requested by the client. Used for provenance event.http.remote.hostIP address of the client. Used for provenance event.http.local.nameIP address/hostname of the server. Used for provenance event.http.server.portListening port of the server. Used for provenance event.http.subject.dnSSL distinguished name (if any). Used for provenance event.INPUT_REQUIREDorg.apache.nifi.processors.standard.HandleHttpRequestorg.apache.nifi.http.StandardHttpContextMaporg.apache.nifi.processors.standard.IdentifyMimeTypePROCESSORAttempts to identify the MIME Type used for a FlowFile. If the MIME Type can be identified, an attribute with the name 'mime.type' is added with the value being the MIME Type. If the MIME Type cannot be determined, the value will be set to 'application/octet-stream'. In addition, the attribute 'mime.extension' will be set if a common file extension for the MIME Type is known. If the MIME Type detected is of type text/*, attempts to identify the charset used and an attribute with the name 'mime.charset' is added with the value being the charset.compressiongzipbzip2zipMIMEmime.typefileidentifyuse-filename-in-detectionUse Filename In DetectionIf true will pass the filename to Tika to aid in detection.truetruetruefalsefalsetruefalsefalseNONEfalsefalseconfig-strategyConfig StrategySelect the loading strategy for MIME Type configuration to be used.PresetPresetPresetUse default NiFi MIME Types.ReplaceReplaceUse config MIME Types only.MergeMergeUse config together with default NiFi MIME Types.truefalsefalseNONEfalsefalseconfig-bodyConfig BodyBody of MIME type config file. Only one of Config File or Config Body may be used.falsefalsefalseNONEfalsefalseconfig-strategyConfig StrategyReplaceMergeconfig-fileConfig FilePath to MIME type config file. Only one of Config File or Config Body may be used.falsefalsetrueENVIRONMENTfalsefalseconfig-strategyConfig StrategyReplaceMergesuccessAll FlowFiles are routed to successfalsemime.typeThis Processor sets the FlowFile's mime.type attribute to the detected MIME Type. If unable to detect the MIME Type, the attribute's value will be set to application/octet-streammime.extensionThis Processor sets the FlowFile's mime.extension attribute to the file extension associated with the detected MIME Type. If there is no correlated extension, the attribute's value will be emptymime.charsetThis Processor sets the FlowFile's mime.charset attribute to the detected charset. If unable to detect the charset or the detected MIME type is not of type text/*, the attribute will not be settruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.InvokeHTTPPROCESSORAn HTTP client processor which can interact with a configurable HTTP Endpoint. The destination URL and HTTP Method are configurable. When the HTTP Method is PUT, POST or PATCH, the FlowFile contents are included as the body of the request and FlowFile attributes are converted to HTTP headers, optionally, based on configuration properties.httphttpsrestclientHTTP MethodHTTP MethodHTTP request method (GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS). Arbitrary methods are also supported. Methods other than POST, PUT and PATCH will be sent without a message body.GETtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHTTP URLHTTP URLHTTP remote URL including a scheme of http or https, as well as a hostname or IP address with optional port and path elements. Any encoding of the URL must be done by the user.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHTTP/2 DisabledHTTP/2 DisabledDisable negotiation of HTTP/2 protocol. HTTP/2 requires TLS. HTTP/1.1 protocol supported is required when HTTP/2 is disabled.FalseTrueTrueFalseFalsetruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceSSL Context Service provides trusted certificates and client certificates for TLS communication.org.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutMaximum time to wait for initial socket connection to the HTTP URL.5 secstruefalsefalseNONEfalsefalseSocket Read TimeoutSocket Read TimeoutMaximum time to wait for receiving responses from a socket connection to the HTTP URL.15 secstruefalsefalseNONEfalsefalseSocket Write TimeoutSocket Write TimeoutMaximum time to wait for write operations while sending requests from a socket connection to the HTTP URL.15 secstruefalsefalseNONEfalsefalseSocket Idle TimeoutSocket Idle TimeoutMaximum time to wait before closing idle connections to the HTTP URL.5 minstruefalsefalseNONEfalsefalseSocket Idle ConnectionsSocket Idle ConnectionsMaximum number of idle connections to the HTTP URL.5truefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseRequest OAuth2 Access Token ProviderRequest OAuth2 Access Token ProviderEnables managed retrieval of OAuth2 Bearer Token applied to HTTP requests using the Authorization Header.org.apache.nifi.oauth2.OAuth2AccessTokenProviderorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseRequest UsernameRequest UsernameThe username provided for authentication of HTTP requests. Encoded using Base64 for HTTP Basic Authentication as described in RFC 7617.falsefalsefalseNONEfalsefalseRequest PasswordRequest PasswordThe password provided for authentication of HTTP requests. Encoded using Base64 for HTTP Basic Authentication as described in RFC 7617.falsetruefalseNONEfalsefalseRequest Digest Authentication EnabledRequest Digest Authentication EnabledEnable Digest Authentication on HTTP requests with Username and Password credentials as described in RFC 7616.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseRequest UsernameRequest UsernameRequest Failure Penalization EnabledRequest Failure Penalization EnabledEnable penalization of request FlowFiles when receiving HTTP response with a status code between 400 and 499.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseRequest Body EnabledRequest Body EnabledEnable sending HTTP request body for PATCH, POST, or PUT methods.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseHTTP MethodHTTP MethodPOSTPATCHPUTRequest Multipart Form-Data NameRequest Multipart Form-Data NameEnable sending HTTP request body formatted using multipart/form-data and using the form name configured.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseRequest Body EnabledRequest Body EnabledtrueRequest Multipart Form-Data Filename EnabledRequest Multipart Form-Data Filename EnabledEnable sending the FlowFile filename attribute as the filename parameter in the Content-Disposition Header for multipart/form-data HTTP requests.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseRequest Multipart Form-Data NameRequest Multipart Form-Data NameRequest Chunked Transfer-Encoding EnabledRequest Chunked Transfer-Encoding EnabledEnable sending HTTP requests with the Transfer-Encoding Header set to chunked, and disable sending the Content-Length Header. Transfer-Encoding applies to the body in HTTP/1.1 requests as described in RFC 7230 Section 3.3.1falsetruetruefalsefalsetruefalsefalseNONEfalsefalseHTTP MethodHTTP MethodPOSTPATCHPUTRequest Content-EncodingRequest Content-EncodingHTTP Content-Encoding applied to request body during transmission. The receiving server must support the selected encoding to avoid request failures.DISABLEDDISABLEDDISABLEDContent encoding not applied during transmissionGZIPGZIPGzip content encoding and HTTP Content-Encoding header applied during transmissiontruefalsefalseNONEfalsefalseHTTP MethodHTTP MethodPOSTPATCHPUTRequest Content-TypeRequest Content-TypeHTTP Content-Type Header applied to when sending an HTTP request body for PATCH, POST, or PUT methods. The Content-Type defaults to application/octet-stream when not configured.${mime.type}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHTTP MethodHTTP MethodPOSTPATCHPUTRequest Date Header EnabledRequest Date Header EnabledEnable sending HTTP Date Header on HTTP requests as described in RFC 7231 Section 7.1.1.2.TrueTrueTrueFalseFalsetruefalsefalseNONEfalsefalseRequest Header Attributes PatternRequest Header Attributes PatternRegular expression that defines which FlowFile attributes to send as HTTP headers in the request. If not defined, no attributes are sent as headers. Dynamic properties will be always be sent as headers. The dynamic property name will be the header key and the dynamic property value, interpreted as Expression Language, will be the header value. Attributes and their values are limited to ASCII characters due to the requirement of the HTTP protocol.falsefalsefalseNONEfalsefalseRequest User-AgentRequest User-AgentHTTP User-Agent Header applied to requests. RFC 7231 Section 5.5.3 describes recommend formatting.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseResponse Body Attribute NameResponse Body Attribute NameFlowFile attribute name used to write an HTTP response body for FlowFiles transferred to the Original relationship.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseResponse Body Attribute SizeResponse Body Attribute SizeMaximum size in bytes applied when writing an HTTP response body to a FlowFile attribute. Attributes exceeding the maximum will be truncated.256falsefalsefalseNONEfalsefalseResponse Body Attribute NameResponse Body Attribute NameResponse Body IgnoredResponse Body IgnoredDisable writing HTTP response FlowFiles to Response relationshipfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseResponse Cache EnabledResponse Cache EnabledEnable HTTP response caching described in RFC 7234. Caching responses considers ETag and other headers.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseResponse Cache SizeResponse Cache SizeMaximum size of HTTP response cache in bytes. Caching responses considers ETag and other headers.10MBtruefalsefalseNONEfalsefalseResponse Cache EnabledResponse Cache EnabledtrueResponse Cookie StrategyResponse Cookie StrategyStrategy for accepting and persisting HTTP cookies. Accepting cookies enables persistence across multiple requests.DISABLEDDISABLEDDISABLEDACCEPT_ALLACCEPT_ALLtruefalsefalseNONEfalsefalseResponse Generation RequiredResponse Generation RequiredEnable generation and transfer of a FlowFile to the Response relationship regardless of HTTP response received.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseResponse FlowFile Naming StrategyResponse FlowFile Naming StrategyDetermines the strategy used for setting the filename attribute of FlowFiles transferred to the Response relationship.RANDOMRANDOMRANDOMFlowFile filename attribute will be a random value.URL_PATHURL_PATHFlowFile filename attribute will be extracted from the remote URL path. The attribute may contain URL encoded characters. If the path doesn't exist, the attribute will be a random value.truefalsefalseNONEfalsefalseResponse Header Request Attributes EnabledResponse Header Request Attributes EnabledEnable adding HTTP response headers as attributes to FlowFiles transferred to the Original relationship.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseResponse Redirects EnabledResponse Redirects EnabledEnable following HTTP redirects sent with HTTP 300 series responses as described in RFC 7231 Section 6.4.TrueTrueTrueFalseFalsetruefalsefalseNONEfalsefalseHeader NameAttribute Expression LanguageSend request header with a key matching the Dynamic Property Key and a value created by evaluating the Attribute Expression Language set in the value of the Dynamic Property.FLOWFILE_ATTRIBUTESpost:form:<NAME>Attribute Expression LanguageWhen the HTTP Method is POST, dynamic properties with the property name in the form of post:form:<NAME>, where the <NAME> will be the form data name, will be used to fill out the multipart form parts. If send message body is false, the flowfile will not be sent, but any other form data will be.FLOWFILE_ATTRIBUTEStrueOriginalRequest FlowFiles transferred when receiving HTTP responses with a status code between 200 and 299.falseFailureRequest FlowFiles transferred when receiving socket communication errors.falseRetryRequest FlowFiles transferred when receiving HTTP responses with a status code between 500 and 599.falseResponseResponse FlowFiles transferred when receiving HTTP responses with a status code between 200 and 299.falseNo RetryRequest FlowFiles transferred when receiving HTTP responses with a status code between 400 an 499.falseinvokehttp.status.codeThe status code that is returnedinvokehttp.status.messageThe status message that is returnedinvokehttp.response.bodyIn the instance where the status code received is not a success (2xx) then the response body will be put to the 'invokehttp.response.body' attribute of the request FlowFile.invokehttp.request.urlThe original request URLinvokehttp.request.durationDuration (in milliseconds) of the HTTP call to the external endpointinvokehttp.response.urlThe URL that was ultimately requested after any redirects were followedinvokehttp.tx.idThe transaction ID that is returned after reading the responseinvokehttp.remote.dnThe DN of the remote serverinvokehttp.java.exception.classThe Java exception class raised when the processor failsinvokehttp.java.exception.messageThe Java exception message raised when the processor failsuser-definedIf the 'Put Response Body In Attribute' property is set then whatever it is set to will become the attribute key and the value would be the body of the HTTP response.trueINPUT_ALLOWEDorg.apache.nifi.processors.standard.JoinEnrichmentPROCESSORJoins together Records from two different FlowFiles where one FlowFile, the 'original' contains arbitrary records and the second FlowFile, the 'enrichment' contains additional data that should be used to enrich the first. See Additional Details for more information on how to configure this processor and the different use cases that it aims to accomplish.forkjoinenrichmentrecordsqlwraprecordpathmergecombinestreamsOriginal Record ReaderOriginal Record ReaderThe Record Reader for reading the 'original' FlowFileorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseEnrichment Record ReaderEnrichment Record ReaderThe Record Reader for reading the 'enrichment' FlowFileorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseRecord WriterRecord WriterThe Record Writer to use for writing the results. If the Record Writer is configured to inherit the schema from the Record, the schema that it will inherit will be the result of merging both the 'original' record schema and the 'enrichment' record schema.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseJoin StrategyJoin StrategySpecifies how to join the two FlowFiles into a single FlowFileWrapperWrapperWrapperThe output is a Record that contains two fields: (1) 'original', containing the Record from the original FlowFile and (2) 'enrichment' containing the corresponding Record from the enrichment FlowFile. Records will be correlated based on their index in the FlowFile. If one FlowFile has more Records than the other, a null value will be used.SQLSQLThe output is derived by evaluating a SQL SELECT statement that allows for two tables: 'original' and 'enrichment'. This allows for SQL JOIN statements to be used in order to correlate the Records of the two FlowFiles, so the index in which the Record is encountered in the FlowFile does not matter.Insert Enrichment FieldsInsert Enrichment FieldsThe enrichment is joined together with the original FlowFile by placing all fields of the enrichment Record into the corresponding Record from the original FlowFile. Records will be correlated based on their index in the FlowFile.truefalsefalseNONEfalsefalseSQLSQLThe SQL SELECT statement to evaluate. Expression Language may be provided, but doing so may result in poorer performance. Because this Processor is dealing with two FlowFiles at a time, it's also important to understand how attributes will be referenced. If both FlowFiles have an attribute with the same name but different values, the Expression Language will resolve to the value provided by the 'enrichment' FlowFile.SELECT original.*, enrichment.* + - If incoming connection(s) are specified and a FlowFile is available to a processor task, the FlowFile's attributes may be used in Expression Language for such fields as Table Name and others. However, the Max-Value Columns and Columns to Return fields must be empty or refer to columns that are available in each specified table.sqlselectjdbcquerydatabasefetchgenerateDatabase Connection Pooling ServiceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-fetch-db-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLtruefalsefalseNONEfalsefalseTable NameTable NameThe name of the database table to be queried.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseColumns to ReturnColumns to ReturnA comma-separated list of column names to be used in the query. If your database requires special treatment of the names (quoting, e.g.), each name should include such treatment. If no column names are supplied, all columns in the specified table will be returned. NOTE: It is important to use consistent column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMaximum-value ColumnsMaximum-value ColumnsA comma-separated list of column names. The processor will keep track of the maximum value for each column that has been returned since the processor started running. Using multiple columns implies an order to the column list, and each column's values are expected to increase more slowly than the previous columns' values. Thus, using multiple columns implies a hierarchical structure of columns, which is usually used for partitioning tables. This processor can be used to retrieve only those rows that have been added/updated since the last retrieval. Note that some JDBC types such as bit/boolean are not conducive to maintaining maximum value, so columns of these types should not be listed in this property, and will result in error(s) during processing. If no columns are provided, all rows from the table will be considered, which could have a performance impact. NOTE: It is important to use consistent max-value column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax Wait TimeMax Wait TimeThe maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsegen-table-fetch-partition-sizePartition SizeThe number of result rows to be fetched by each generated SQL statement. The total number of rows in the table divided by the partition size gives the number of SQL statements (i.e. FlowFiles) generated. A value of zero indicates that a single FlowFile is to be generated whose SQL statement will fetch all rows in the table.10000truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsegen-table-column-for-val-partitioningColumn for Value PartitioningThe name of a column whose values will be used for partitioning. The default behavior is to use row numbers on the result set for partitioning into 'pages' to be fetched from the database, using an offset/limit strategy. However for certain databases, it can be more efficient under the right circumstances to use the column values themselves to define the 'pages'. This property should only be used when the default queries are not performing well, when there is no maximum-value column or a single maximum-value column whose type can be coerced to a long integer (i.e. not date or timestamp), and the column values are evenly distributed and not sparse, for best performance.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-where-clauseAdditional WHERE clauseA custom clause to be added in the WHERE condition when building SQL queries.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsegen-table-custom-orderby-columnCustom ORDER BY ColumnThe name of a column to be used for ordering the results if Max-Value Columns are not provided and partitioning is enabled. This property is ignored if either Max-Value Columns is set or Partition Size = 0. NOTE: If neither Max-Value Columns nor Custom ORDER BY Column is set, then depending on the the database/driver, the processor may report an error and/or the generated SQL may result in missing and/or duplicate rows. This is because without an explicit ordering, fetching each partition is done using an arbitrary ordering.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsegen-table-output-flowfile-on-zero-resultsOutput Empty FlowFile on Zero ResultsDepending on the specified properties, an execution of this processor may not result in any SQL statements generated. When this property is true, an empty FlowFile will be generated (having the parent of the incoming FlowFile if present) and transferred to the 'success' relationship. When this property is false, no output FlowFiles will be generated.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseinitial.maxvalue.<max_value_column>Initial maximum value for the specified columnSpecifies an initial max value for max value columns. Properties should be added in the format `initial.maxvalue.<max_value_column>`. This value is only used the first time the table is accessed (when a Maximum Value Column is specified). In the case of incoming connections, the value is only used the first time for each table specified in the FlowFiles.FLOWFILE_ATTRIBUTESsuccessSuccessfully created FlowFile from SQL query result set.falsefailureThis relationship is only used when SQL query execution (using an incoming FlowFile) failed. The incoming FlowFile will be penalized and routed to this relationship. If no incoming connection(s) are specified, this relationship is unused.falsegeneratetablefetch.sql.errorIf the processor has incoming connections, and processing an incoming FlowFile causes a SQL Exception, the FlowFile is routed to failure and this attribute is set to the exception message.generatetablefetch.tableNameThe name of the database table to be queried.generatetablefetch.columnNamesThe comma-separated list of column names used in the query.generatetablefetch.whereClauseWhere clause used in the query to get the expected rows.generatetablefetch.maxColumnNamesThe comma-separated list of column names used to keep track of data that has been returned since the processor started running.generatetablefetch.limitThe number of result rows to be fetched by the SQL statement.generatetablefetch.offsetOffset to be used to retrieve the corresponding partition.fragment.identifierAll FlowFiles generated from the same query result set will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.fragment.countThis is the total number of FlowFiles produced by a single ResultSet. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming ResultSet.fragment.indexThis is the position of this FlowFile in the list of outgoing FlowFiles that were all generated from the same execution. This can be used in conjunction with the fragment.identifier attribute to know which FlowFiles originated from the same execution and in what order FlowFiles were producedtrueAfter performing a query on the specified table, the maximum values for the specified column(s) will be retained for use in future executions of the query. This allows the Processor to fetch only those records that have max values greater than the retained values. This can be used for incremental fetching, fetching of newly added rows, etc. To clear the maximum values, clear the state of the processor per the State Management documentationCLUSTERINPUT_ALLOWEDorg.apache.nifi.processors.standard.QueryDatabaseTableorg.apache.nifi.processors.standard.ExecuteSQLorg.apache.nifi.processors.standard.ListDatabaseTablesorg.apache.nifi.processors.standard.GetFTPPROCESSORFetches files from an FTP Server and creates FlowFiles from themFTPgetretrievefilesfetchremoteingestsourceinputHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers21truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseConnection ModeConnection ModeThe FTP Connection ModePassiveActiveActivePassivePassivefalsefalsefalseNONEfalsefalseTransfer ModeTransfer ModeThe FTP Transfer ModeBinaryBinaryBinaryASCIIASCIIfalsefalsefalseNONEfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push filesfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFile Filter RegexFile Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetchedfalsefalsefalseNONEfalsefalsePath Filter RegexPath Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scannedfalsefalsefalseNONEfalsefalsePolling IntervalPolling IntervalDetermines how long to wait between fetching the listing for new files60 sectruefalsefalseNONEfalsefalseSearch RecursivelySearch RecursivelyIf true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefollow-symlinkFollow symlinkIf true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseIgnore Dotted FilesIgnore Dotted FilesIf true, files whose names begin with a dot (".") will be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseDelete OriginalDelete OriginalDetermines whether or not the file is deleted from the remote system after it has been successfully transferredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseMax SelectsMax SelectsThe maximum number of files to pull in a single connection100truefalsefalseNONEfalsefalseRemote Poll Batch SizeRemote Poll Batch SizeThe value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.5000truefalsefalseNONEfalsefalseUse Natural OrderingUse Natural OrderingIf true, will pull files in the order in which they are naturally listed; otherwise, the order in which the files will be pulled is not definedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseInternal Buffer SizeInternal Buffer SizeSet the internal buffer size for buffered data streams16KBfalsefalsefalseNONEfalsefalseftp-use-utf8Use UTF-8 EncodingTells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalsefilenameThe filename is set to the name of the file on the remote serverpathThe path is set to the path of the file's directory on the remote server. For example, if the <Remote Path> property is set to /tmp, files picked up from /tmp will have the path attribute set to /tmp. If the <Search Recursively> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to /tmp/abc/1/2/3file.lastModifiedTimeThe date and time that the source file was last modifiedfile.lastAccessTimeThe date and time that the file was last accessed. May not work on all file systemsfile.ownerThe numeric owner id of the source filefile.groupThe numeric group id of the source filefile.permissionsThe read/write/execute permissions of the source fileabsolute.pathThe full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative pathINPUT_FORBIDDENorg.apache.nifi.processors.standard.PutFTPorg.apache.nifi.processors.standard.GetFilePROCESSORCreates FlowFiles from files in a directory. NiFi will ignore files it doesn't have at least read permissions for.localfilesfilesystemingestingressgetsourceinputInput DirectoryInput DirectoryThe input directory from which to pull filestruefalsetrueENVIRONMENTfalsefalseFile FilterFile FilterOnly files whose names match the given regular expression will be picked up[^\.].*truefalsefalseNONEfalsefalsePath FilterPath FilterWhen Recurse Subdirectories is true, then only subdirectories whose path matches the given regular expression will be scannedfalsefalsefalseNONEfalsefalseBatch SizeBatch SizeThe maximum number of files to pull in each iteration10truefalsefalseNONEfalsefalseKeep Source FileKeep Source FileIf true, the file is not deleted after it has been copied to the Content Repository; this causes the file to be picked up continually and is useful for testing purposes. If not keeping original NiFi will need write permissions on the directory it is pulling from otherwise it will ignore the file.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseRecurse SubdirectoriesRecurse SubdirectoriesIndicates whether or not to pull files from subdirectoriestruetruetruefalsefalsetruefalsefalseNONEfalsefalsePolling IntervalPolling IntervalIndicates how long to wait before performing a directory listing0 sectruefalsefalseNONEfalsefalseIgnore Hidden FilesIgnore Hidden FilesIndicates whether or not hidden files should be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseMinimum File AgeMinimum File AgeThe minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored0 sectruefalsefalseNONEfalsefalseMaximum File AgeMaximum File AgeThe maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignoredfalsefalsefalseNONEfalsefalseMinimum File SizeMinimum File SizeThe minimum size that a file must be in order to be pulled0 BtruefalsefalseNONEfalsefalseMaximum File SizeMaximum File SizeThe maximum size that a file can be in order to be pulledfalsefalsefalseNONEfalsefalsesuccessAll files are routed to successfalsefilenameThe filename is set to the name of the file on diskpathThe path is set to the relative path of the file's directory on disk. For example, if the <Input Directory> property is set to /tmp, files picked up from /tmp will have the path attribute set to ./. If the <Recurse Subdirectories> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to abc/1/2/3file.creationTimeThe date and time that the file was created. May not work on all file systemsfile.lastModifiedTimeThe date and time that the file was last modified. May not work on all file systemsfile.lastAccessTimeThe date and time that the file was last accessed. May not work on all file systemsfile.ownerThe owner of the file. May not work on all file systemsfile.groupThe group owner of the file. May not work on all file systemsfile.permissionsThe read/write/execute permissions of the file. May not work on all file systemsabsolute.pathThe full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative pathtrueread filesystemProvides operator the ability to read from any file that NiFi has access to.write filesystemProvides operator the ability to delete any file that NiFi has access to.INPUT_FORBIDDENorg.apache.nifi.processors.standard.PutFileorg.apache.nifi.processors.standard.FetchFileorg.apache.nifi.processors.standard.GetSFTPPROCESSORFetches files from an SFTP Server and creates FlowFiles from themsftpgetretrievefilesfetchremoteingestsourceinputHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers22truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalsePrivate Key PathPrivate Key PathThe fully qualified path to the Private Key filefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILEPrivate Key PassphrasePrivate Key PassphrasePassword for the private keyfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push filesfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFile Filter RegexFile Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetchedfalsefalsefalseNONEfalsefalsePath Filter RegexPath Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scannedfalsefalsefalseNONEfalsefalsePolling IntervalPolling IntervalDetermines how long to wait between fetching the listing for new files60 sectruefalsefalseNONEfalsefalseSearch RecursivelySearch RecursivelyIf true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefollow-symlinkFollow symlinkIf true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseIgnore Dotted FilesIgnore Dotted FilesIf true, files whose names begin with a dot (".") will be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseDelete OriginalDelete OriginalDetermines whether or not the file is deleted from the remote system after it has been successfully transferredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseHost Key FileHost Key FileIf supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be usedfalsefalsefalseNONEfalsefalseSINGLEFILEMax SelectsMax SelectsThe maximum number of files to pull in a single connection100truefalsefalseNONEfalsefalseRemote Poll Batch SizeRemote Poll Batch SizeThe value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.5000truefalsefalseNONEfalsefalseStrict Host Key CheckingStrict Host Key CheckingIndicates whether or not strict enforcement of hosts keys should be appliedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseSend Keep Alive On TimeoutSend Keep Alive On TimeoutSend a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.truetruetruefalsefalsetruefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseUse Natural OrderingUse Natural OrderingIf true, will pull files in the order in which they are naturally listed; otherwise, the order in which the files will be pulled is not definedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseCiphers AllowedCiphers AllowedA comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctrfalsefalsetrueENVIRONMENTfalsefalseKey Algorithms AllowedKey Algorithms AllowedA comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.comfalsefalsetrueENVIRONMENTfalsefalseKey Exchange Algorithms AllowedKey Exchange Algorithms AllowedA comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-cfalsefalsetrueENVIRONMENTfalsefalseMessage Authentication Codes AllowedMessage Authentication Codes AllowedA comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.comfalsefalsetrueENVIRONMENTfalsefalsesuccessAll FlowFiles that are received are routed to successfalsefilenameThe filename is set to the name of the file on the remote serverpathThe path is set to the path of the file's directory on the remote server. For example, if the <Remote Path> property is set to /tmp, files picked up from /tmp will have the path attribute set to /tmp. If the <Search Recursively> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to /tmp/abc/1/2/3file.lastModifiedTimeThe date and time that the source file was last modifiedfile.ownerThe numeric owner id of the source filefile.groupThe numeric group id of the source filefile.permissionsThe read/write/execute permissions of the source fileabsolute.pathThe full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative pathINPUT_FORBIDDENorg.apache.nifi.processors.standard.PutSFTPorg.apache.nifi.processors.standard.HandleHttpRequestPROCESSORStarts an HTTP Server and listens for HTTP Requests. For each request, creates a FlowFile and transfers to 'success'. This Processor is designed to be used in conjunction with the HandleHttpResponse Processor in order to create a Web Service. In case of a multipart request, one FlowFile is generated for each part.httphttpsrequestlisteningressweb serviceListening PortListening PortThe Port to listen on for incoming HTTP requests80truefalsetrueENVIRONMENTfalsefalseHostnameHostnameThe Hostname to bind to. If not specified, will bind to all hostsfalsefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceThe SSL Context Service to use in order to secure the server. If specified, the server will accept only HTTPS requests; otherwise, the server will accept only HTTP requestsorg.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseHTTP ProtocolsHTTP ProtocolsHTTP Protocols supported for Application Layer Protocol Negotiation with TLSHTTP_1_1http/1.1HTTP_1_1HTTP/1.1h2 http/1.1H2_HTTP_1_1HTTP/2 and HTTP/1.1 negotiated based on requested protocolsh2H2HTTP/2truefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceHTTP Context MapHTTP Context MapThe HTTP Context Map Controller Service to use for caching the HTTP Request Informationorg.apache.nifi.http.HttpContextMaporg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseAllowed PathsAllowed PathsA Regular Expression that specifies the valid HTTP Paths that are allowed in the incoming URL Requests. If this value is specified and the path of the HTTP Requests does not match this Regular Expression, the Processor will respond with a 404: NotFoundfalsefalsefalseNONEfalsefalseDefault URL Character SetDefault URL Character SetThe character set to use for decoding URL parameters if the HTTP Request does not supply oneUTF-8truefalsefalseNONEfalsefalseAllow GETAllow GETAllow HTTP GET MethodtruetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow POSTAllow POSTAllow HTTP POST MethodtruetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow PUTAllow PUTAllow HTTP PUT MethodtruetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow DELETEAllow DELETEAllow HTTP DELETE MethodtruetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow HEADAllow HEADAllow HTTP HEAD MethodfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow OPTIONSAllow OPTIONSAllow HTTP OPTIONS MethodfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseMaximum ThreadsMaximum ThreadsThe maximum number of threads that the embedded HTTP server will use for handling requests.200truefalsefalseNONEfalsefalseAdditional HTTP MethodsAdditional HTTP MethodsA comma-separated list of non-standard HTTP Methods that should be allowedfalsefalsefalseNONEfalsefalseClient AuthenticationClient AuthenticationSpecifies whether or not the Processor should authenticate clients. This value is ignored if the <SSL Context Service> Property is not specified or the SSL Context provided uses only a KeyStore and not a TrustStore.No AuthenticationNo AuthenticationNo AuthenticationProcessor will not authenticate clients. Anyone can communicate with this Processor anonymouslyWant AuthenticationWant AuthenticationProcessor will try to verify the client but if unable to verify will allow the client to communicate anonymouslyNeed AuthenticationNeed AuthenticationProcessor will reject communications from any client unless the client provides a certificate that is trusted by the TrustStorespecified in the SSL Context ServicetruefalsefalseNONEfalsefalsecontainer-queue-sizeContainer Queue SizeThe size of the queue for Http Request Containers50truefalsefalseNONEfalsefalsemultipart-request-max-sizeMultipart Request Max SizeThe max size of the request. Only applies for requests with Content-Type: multipart/form-data, and is used to prevent denial of service type of attacks, to prevent filling up the heap or disk space1 MBtruefalsefalseNONEfalsefalsemultipart-read-buffer-sizeMultipart Read Buffer SizeThe threshold size, at which the contents of an incoming file would be written to disk. Only applies for requests with Content-Type: multipart/form-data. It is used to prevent denial of service type of attacks, to prevent filling up the heap or disk space.512 KBtruefalsefalseNONEfalsefalseparameters-to-attributesParameters to Attributes ListA comma-separated list of HTTP parameters or form data to output as attributesfalsefalsefalseNONEfalsefalsesuccessAll content that is received is routed to the 'success' relationshipfalsehttp.context.identifierAn identifier that allows the HandleHttpRequest and HandleHttpResponse to coordinate which FlowFile belongs to which HTTP Request/Response.mime.typeThe MIME Type of the data, according to the HTTP Header "Content-Type"http.servlet.pathThe part of the request URL that is considered the Servlet Pathhttp.context.pathThe part of the request URL that is considered to be the Context Pathhttp.methodThe HTTP Method that was used for the request, such as GET or POSThttp.local.nameIP address/hostname of the serverhttp.server.portListening port of the serverhttp.query.stringThe query string portion of the Request URLhttp.remote.hostThe hostname of the requestorhttp.remote.addrThe hostname:port combination of the requestorhttp.remote.userThe username of the requestorhttp.protocolThe protocol used to communicatehttp.request.uriThe full Request URLhttp.auth.typeThe type of HTTP Authorization usedhttp.principal.nameThe name of the authenticated user making the requesthttp.query.param.XXXEach of query parameters in the request will be added as an attribute, prefixed with "http.query.param."http.param.XXXForm parameters in the request that are configured by "Parameters to Attributes List" will be added as an attribute, prefixed with "http.param.". Putting form parameters of large size is not recommended.http.subject.dnThe Distinguished Name of the requestor. This value will not be populated unless the Processor is configured to use an SSLContext Servicehttp.issuer.dnThe Distinguished Name of the entity that issued the Subject's certificate. This value will not be populated unless the Processor is configured to use an SSLContext Servicehttp.certificate.sans.N.nameX.509 Client Certificate Subject Alternative Name value from mutual TLS authentication. The attribute name has a zero-based index ordered according to the content of Client Certificatehttp.certificate.sans.N.nameTypeX.509 Client Certificate Subject Alternative Name type from mutual TLS authentication. The attribute name has a zero-based index ordered according to the content of Client Certificate. The attribute value is one of the General Names from RFC 3280 Section 4.1.2.7http.headers.XXXEach of the HTTP Headers that is received in the request will be added as an attribute, prefixed with "http.headers." For example, if the request contains an HTTP Header named "x-my-header", then the value will be added to an attribute named "http.headers.x-my-header"http.headers.multipart.XXXEach of the HTTP Headers that is received in the multipart request will be added as an attribute, prefixed with "http.headers.multipart." For example, if the multipart request contains an HTTP Header named "content-disposition", then the value will be added to an attribute named "http.headers.multipart.content-disposition"http.multipart.sizeFor requests with Content-Type "multipart/form-data", the part's content size is recorded into this attributehttp.multipart.content.typeFor requests with Content-Type "multipart/form-data", the part's content type is recorded into this attributehttp.multipart.nameFor requests with Content-Type "multipart/form-data", the part's name is recorded into this attributehttp.multipart.filenameFor requests with Content-Type "multipart/form-data", when the part contains an uploaded file, the name of the file is recorded into this attribute. Files are stored temporarily at the default temporary-file directory specified in "java.io.File" Java Docs)http.multipart.fragments.sequence.numberFor requests with Content-Type "multipart/form-data", the part's index is recorded into this attribute. The index starts with 1.http.multipart.fragments.total.numberFor requests with Content-Type "multipart/form-data", the count of all parts is recorded into this attribute.INPUT_FORBIDDENorg.apache.nifi.processors.standard.HandleHttpResponseorg.apache.nifi.processors.standard.HandleHttpResponsePROCESSORSends an HTTP Response to the Requestor that generated a FlowFile. This Processor is designed to be used in conjunction with the HandleHttpRequest in order to create a web service.httphttpsresponseegressweb serviceHTTP Status CodeHTTP Status CodeThe HTTP Status Code to use when responding to the HTTP Request. See Section 10 of RFC 2616 for more information.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHTTP Context MapHTTP Context MapThe HTTP Context Map Controller Service to use for caching the HTTP Request Informationorg.apache.nifi.http.HttpContextMaporg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseAttributes to add to the HTTP Response (Regex)Attributes to add to the HTTP Response (Regex)Specifies the Regular Expression that determines the names of FlowFile attributes that should be added to the HTTP responsefalsefalsefalseNONEfalsefalseAn HTTP header nameAn HTTP header valueThese HTTPHeaders are set in the HTTP ResponseFLOWFILE_ATTRIBUTESsuccessFlowFiles will be routed to this Relationship after the response has been successfully sent to the requestorfalsefailureFlowFiles will be routed to this Relationship if the Processor is unable to respond to the requestor. This may happen, for instance, if the connection times out or if NiFi is restarted before responding to the HTTP Request.falsehttp.context.identifierThe value of this attribute is used to lookup the HTTP Response so that the proper message can be sent back to the requestor. If this attribute is missing, the FlowFile will be routed to 'failure.'http.request.uriValue of the URI requested by the client. Used for provenance event.http.remote.hostIP address of the client. Used for provenance event.http.local.nameIP address/hostname of the server. Used for provenance event.http.server.portListening port of the server. Used for provenance event.http.subject.dnSSL distinguished name (if any). Used for provenance event.INPUT_REQUIREDorg.apache.nifi.processors.standard.HandleHttpRequestorg.apache.nifi.http.StandardHttpContextMaporg.apache.nifi.processors.standard.IdentifyMimeTypePROCESSORAttempts to identify the MIME Type used for a FlowFile. If the MIME Type can be identified, an attribute with the name 'mime.type' is added with the value being the MIME Type. If the MIME Type cannot be determined, the value will be set to 'application/octet-stream'. In addition, the attribute 'mime.extension' will be set if a common file extension for the MIME Type is known. If the MIME Type detected is of type text/*, attempts to identify the charset used and an attribute with the name 'mime.charset' is added with the value being the charset.compressiongzipbzip2zipMIMEmime.typefileidentifyuse-filename-in-detectionUse Filename In DetectionIf true will pass the filename to Tika to aid in detection.truetruetruefalsefalsetruefalsefalseNONEfalsefalseconfig-strategyConfig StrategySelect the loading strategy for MIME Type configuration to be used.PresetPresetPresetUse default NiFi MIME Types.ReplaceReplaceUse config MIME Types only.MergeMergeUse config together with default NiFi MIME Types.truefalsefalseNONEfalsefalseconfig-bodyConfig BodyBody of MIME type config file. Only one of Config File or Config Body may be used.falsefalsefalseNONEfalsefalseconfig-strategyConfig StrategyReplaceMergeconfig-fileConfig FilePath to MIME type config file. Only one of Config File or Config Body may be used.falsefalsetrueENVIRONMENTfalsefalseconfig-strategyConfig StrategyReplaceMergesuccessAll FlowFiles are routed to successfalsemime.typeThis Processor sets the FlowFile's mime.type attribute to the detected MIME Type. If unable to detect the MIME Type, the attribute's value will be set to application/octet-streammime.extensionThis Processor sets the FlowFile's mime.extension attribute to the file extension associated with the detected MIME Type. If there is no correlated extension, the attribute's value will be emptymime.charsetThis Processor sets the FlowFile's mime.charset attribute to the detected charset. If unable to detect the charset or the detected MIME type is not of type text/*, the attribute will not be settruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.InvokeHTTPPROCESSORAn HTTP client processor which can interact with a configurable HTTP Endpoint. The destination URL and HTTP Method are configurable. When the HTTP Method is PUT, POST or PATCH, the FlowFile contents are included as the body of the request and FlowFile attributes are converted to HTTP headers, optionally, based on configuration properties.httphttpsrestclientHTTP MethodHTTP MethodHTTP request method (GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS). Arbitrary methods are also supported. Methods other than POST, PUT and PATCH will be sent without a message body.GETtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHTTP URLHTTP URLHTTP remote URL including a scheme of http or https, as well as a hostname or IP address with optional port and path elements. Any encoding of the URL must be done by the user.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHTTP/2 DisabledHTTP/2 DisabledDisable negotiation of HTTP/2 protocol. HTTP/2 requires TLS. HTTP/1.1 protocol supported is required when HTTP/2 is disabled.FalseTrueTrueFalseFalsetruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceSSL Context Service provides trusted certificates and client certificates for TLS communication.org.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutMaximum time to wait for initial socket connection to the HTTP URL.5 secstruefalsefalseNONEfalsefalseSocket Read TimeoutSocket Read TimeoutMaximum time to wait for receiving responses from a socket connection to the HTTP URL.15 secstruefalsefalseNONEfalsefalseSocket Write TimeoutSocket Write TimeoutMaximum time to wait for write operations while sending requests from a socket connection to the HTTP URL.15 secstruefalsefalseNONEfalsefalseSocket Idle TimeoutSocket Idle TimeoutMaximum time to wait before closing idle connections to the HTTP URL.5 minstruefalsefalseNONEfalsefalseSocket Idle ConnectionsSocket Idle ConnectionsMaximum number of idle connections to the HTTP URL.5truefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseRequest OAuth2 Access Token ProviderRequest OAuth2 Access Token ProviderEnables managed retrieval of OAuth2 Bearer Token applied to HTTP requests using the Authorization Header.org.apache.nifi.oauth2.OAuth2AccessTokenProviderorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseRequest UsernameRequest UsernameThe username provided for authentication of HTTP requests. Encoded using Base64 for HTTP Basic Authentication as described in RFC 7617.falsefalsefalseNONEfalsefalseRequest PasswordRequest PasswordThe password provided for authentication of HTTP requests. Encoded using Base64 for HTTP Basic Authentication as described in RFC 7617.falsetruefalseNONEfalsefalseRequest Digest Authentication EnabledRequest Digest Authentication EnabledEnable Digest Authentication on HTTP requests with Username and Password credentials as described in RFC 7616.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseRequest UsernameRequest UsernameRequest Failure Penalization EnabledRequest Failure Penalization EnabledEnable penalization of request FlowFiles when receiving HTTP response with a status code between 400 and 499.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseRequest Body EnabledRequest Body EnabledEnable sending HTTP request body for PATCH, POST, or PUT methods.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseHTTP MethodHTTP MethodPOSTPATCHPUTRequest Multipart Form-Data NameRequest Multipart Form-Data NameEnable sending HTTP request body formatted using multipart/form-data and using the form name configured.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseRequest Body EnabledRequest Body EnabledtrueRequest Multipart Form-Data Filename EnabledRequest Multipart Form-Data Filename EnabledEnable sending the FlowFile filename attribute as the filename parameter in the Content-Disposition Header for multipart/form-data HTTP requests.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseRequest Multipart Form-Data NameRequest Multipart Form-Data NameRequest Chunked Transfer-Encoding EnabledRequest Chunked Transfer-Encoding EnabledEnable sending HTTP requests with the Transfer-Encoding Header set to chunked, and disable sending the Content-Length Header. Transfer-Encoding applies to the body in HTTP/1.1 requests as described in RFC 7230 Section 3.3.1falsetruetruefalsefalsetruefalsefalseNONEfalsefalseHTTP MethodHTTP MethodPOSTPATCHPUTRequest Content-EncodingRequest Content-EncodingHTTP Content-Encoding applied to request body during transmission. The receiving server must support the selected encoding to avoid request failures.DISABLEDDISABLEDDISABLEDContent encoding not applied during transmissionGZIPGZIPGzip content encoding and HTTP Content-Encoding header applied during transmissiontruefalsefalseNONEfalsefalseHTTP MethodHTTP MethodPOSTPATCHPUTRequest Content-TypeRequest Content-TypeHTTP Content-Type Header applied to when sending an HTTP request body for PATCH, POST, or PUT methods. The Content-Type defaults to application/octet-stream when not configured.${mime.type}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHTTP MethodHTTP MethodPOSTPATCHPUTRequest Date Header EnabledRequest Date Header EnabledEnable sending HTTP Date Header on HTTP requests as described in RFC 7231 Section 7.1.1.2.TrueTrueTrueFalseFalsetruefalsefalseNONEfalsefalseRequest Header Attributes PatternRequest Header Attributes PatternRegular expression that defines which FlowFile attributes to send as HTTP headers in the request. If not defined, no attributes are sent as headers. Dynamic properties will be always be sent as headers. The dynamic property name will be the header key and the dynamic property value, interpreted as Expression Language, will be the header value. Attributes and their values are limited to ASCII characters due to the requirement of the HTTP protocol.falsefalsefalseNONEfalsefalseRequest User-AgentRequest User-AgentHTTP User-Agent Header applied to requests. RFC 7231 Section 5.5.3 describes recommend formatting.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseResponse Body Attribute NameResponse Body Attribute NameFlowFile attribute name used to write an HTTP response body for FlowFiles transferred to the Original relationship.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseResponse Body Attribute SizeResponse Body Attribute SizeMaximum size in bytes applied when writing an HTTP response body to a FlowFile attribute. Attributes exceeding the maximum will be truncated.256falsefalsefalseNONEfalsefalseResponse Body Attribute NameResponse Body Attribute NameResponse Body IgnoredResponse Body IgnoredDisable writing HTTP response FlowFiles to Response relationshipfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseResponse Cache EnabledResponse Cache EnabledEnable HTTP response caching described in RFC 7234. Caching responses considers ETag and other headers.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseResponse Cache SizeResponse Cache SizeMaximum size of HTTP response cache in bytes. Caching responses considers ETag and other headers.10MBtruefalsefalseNONEfalsefalseResponse Cache EnabledResponse Cache EnabledtrueResponse Cookie StrategyResponse Cookie StrategyStrategy for accepting and persisting HTTP cookies. Accepting cookies enables persistence across multiple requests.DISABLEDDISABLEDDISABLEDACCEPT_ALLACCEPT_ALLtruefalsefalseNONEfalsefalseResponse Generation RequiredResponse Generation RequiredEnable generation and transfer of a FlowFile to the Response relationship regardless of HTTP response received.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseResponse FlowFile Naming StrategyResponse FlowFile Naming StrategyDetermines the strategy used for setting the filename attribute of FlowFiles transferred to the Response relationship.RANDOMRANDOMRANDOMFlowFile filename attribute will be a random value.URL_PATHURL_PATHFlowFile filename attribute will be extracted from the remote URL path. The attribute may contain URL encoded characters. If the path doesn't exist, the attribute will be a random value.truefalsefalseNONEfalsefalseResponse Header Request Attributes EnabledResponse Header Request Attributes EnabledEnable adding HTTP response headers as attributes to FlowFiles transferred to the Original relationship.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseResponse Redirects EnabledResponse Redirects EnabledEnable following HTTP redirects sent with HTTP 300 series responses as described in RFC 7231 Section 6.4.TrueTrueTrueFalseFalsetruefalsefalseNONEfalsefalseHeader NameAttribute Expression LanguageSend request header with a key matching the Dynamic Property Key and a value created by evaluating the Attribute Expression Language set in the value of the Dynamic Property.FLOWFILE_ATTRIBUTESpost:form:<NAME>Attribute Expression LanguageWhen the HTTP Method is POST, dynamic properties with the property name in the form of post:form:<NAME>, where the <NAME> will be the form data name, will be used to fill out the multipart form parts. If send message body is false, the flowfile will not be sent, but any other form data will be.FLOWFILE_ATTRIBUTEStrueOriginalRequest FlowFiles transferred when receiving HTTP responses with a status code between 200 and 299.falseFailureRequest FlowFiles transferred when receiving socket communication errors.falseRetryRequest FlowFiles transferred when receiving HTTP responses with a status code between 500 and 599.falseResponseResponse FlowFiles transferred when receiving HTTP responses with a status code between 200 and 299.falseNo RetryRequest FlowFiles transferred when receiving HTTP responses with a status code between 400 an 499.falseinvokehttp.status.codeThe status code that is returnedinvokehttp.status.messageThe status message that is returnedinvokehttp.response.bodyIn the instance where the status code received is not a success (2xx) then the response body will be put to the 'invokehttp.response.body' attribute of the request FlowFile.invokehttp.request.urlThe original request URLinvokehttp.request.durationDuration (in milliseconds) of the HTTP call to the external endpointinvokehttp.response.urlThe URL that was ultimately requested after any redirects were followedinvokehttp.tx.idThe transaction ID that is returned after reading the responseinvokehttp.remote.dnThe DN of the remote serverinvokehttp.java.exception.classThe Java exception class raised when the processor failsinvokehttp.java.exception.messageThe Java exception message raised when the processor failsuser-definedIf the 'Put Response Body In Attribute' property is set then whatever it is set to will become the attribute key and the value would be the body of the HTTP response.trueINPUT_ALLOWEDorg.apache.nifi.processors.standard.JoinEnrichmentPROCESSORJoins together Records from two different FlowFiles where one FlowFile, the 'original' contains arbitrary records and the second FlowFile, the 'enrichment' contains additional data that should be used to enrich the first. See Additional Details for more information on how to configure this processor and the different use cases that it aims to accomplish.forkjoinenrichmentrecordsqlwraprecordpathmergecombinestreamsOriginal Record ReaderOriginal Record ReaderThe Record Reader for reading the 'original' FlowFileorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseEnrichment Record ReaderEnrichment Record ReaderThe Record Reader for reading the 'enrichment' FlowFileorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseRecord WriterRecord WriterThe Record Writer to use for writing the results. If the Record Writer is configured to inherit the schema from the Record, the schema that it will inherit will be the result of merging both the 'original' record schema and the 'enrichment' record schema.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseJoin StrategyJoin StrategySpecifies how to join the two FlowFiles into a single FlowFileWrapperWrapperWrapperThe output is a Record that contains two fields: (1) 'original', containing the Record from the original FlowFile and (2) 'enrichment' containing the corresponding Record from the enrichment FlowFile. Records will be correlated based on their index in the FlowFile. If one FlowFile has more Records than the other, a null value will be used.SQLSQLThe output is derived by evaluating a SQL SELECT statement that allows for two tables: 'original' and 'enrichment'. This allows for SQL JOIN statements to be used in order to correlate the Records of the two FlowFiles, so the index in which the Record is encountered in the FlowFile does not matter.Insert Enrichment FieldsInsert Enrichment FieldsThe enrichment is joined together with the original FlowFile by placing all fields of the enrichment Record into the corresponding Record from the original FlowFile. Records will be correlated based on their index in the FlowFile.truefalsefalseNONEfalsefalseSQLSQLThe SQL SELECT statement to evaluate. Expression Language may be provided, but doing so may result in poorer performance. Because this Processor is dealing with two FlowFiles at a time, it's also important to understand how attributes will be referenced. If both FlowFiles have an attribute with the same name but different values, the Expression Language will resolve to the value provided by the 'enrichment' FlowFile.SELECT original.*, enrichment.* FROM original LEFT OUTER JOIN enrichment ON original.id = enrichment.idtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseJoin StrategyJoin StrategySQLdbf-default-precisionDefault Decimal PrecisionWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'precision' denoting number of available digits is required. Generally, precision is defined by column data type definition or database engines default. However undefined precision (0) can be returned from some database engines. 'Default Decimal Precision' is used when writing those undefined precision numbers.10falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseJoin StrategyJoin StrategySQLdbf-default-scaleDefault Decimal ScaleWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'scale' denoting number of available decimal digits is required. Generally, scale is defined by column data type definition or database engines default. However when undefined precision (0) is returned, scale can also be uncertain with some database engines. 'Default Decimal Scale' is used when writing those undefined numbers. If a value has more decimals than specified scale, then the value will be rounded-up, e.g. 1.53 becomes 2 with scale 0, and 1.5 with scale 1.0falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseJoin StrategyJoin StrategySQLInsertion Record PathInsertion Record PathSpecifies where in the 'original' Record the 'enrichment' Record's fields should be inserted. Note that if the RecordPath does not point to any existing field in the original Record, the enrichment will not be inserted./truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseJoin StrategyJoin StrategyInsert Enrichment FieldsMaximum number of BinsMaximum number of BinsSpecifies the maximum number of bins that can be held in memory at any one time10000truefalsefalseNONEfalsefalseTimeoutTimeoutSpecifies the maximum amount of time to wait for the second FlowFile once the first arrives at the processor, after which point the first FlowFile will be routed to the 'timeout' relationship.10 mintruefalsefalseNONEfalsefalsetimeoutIf one of the incoming FlowFiles (i.e., the 'original' FlowFile or the 'enrichment' FlowFile) arrives to this Processor but the other does not arrive within the configured Timeout period, the FlowFile that did arrive is routed to this relationship.falsejoinedThe resultant FlowFile with Records joined together from both the original and enrichment FlowFiles will be routed to this relationshipfalsefailureIf both the 'original' and 'enrichment' FlowFiles arrive at the processor but there was a failure in joining the records, both of those FlowFiles will be routed to this relationship.falseoriginalBoth of the incoming FlowFiles ('original' and 'enrichment') will be routed to this Relationship. I.e., this is the 'original' version of both of these FlowFiles.truemime.typeSets the mime.type attribute to the MIME Type specified by the Record Writerrecord.countThe number of records in the FlowFiletruetrueINPUT_REQUIREDMEMORYThis Processor will load into heap all FlowFiles that are on its incoming queues. While it loads the FlowFiles themselves, and not their content, the FlowFile attributes can be very memory intensive. Additionally, if the Join Strategy is set to SQL, the SQL engine may require buffering the entire contents of the enrichment FlowFile for each concurrent task. See Processor's Additional Details for more details and for steps on how to mitigate these concerns.org.apache.nifi.processors.standard.ForkEnrichmentorg.apache.nifi.processors.standard.ListDatabaseTablesPROCESSORGenerates a set of flow files, each containing attributes corresponding to metadata about a table from a database connection. Once metadata about a table has been fetched, it will not be fetched again until the Refresh Interval (if set) has elapsed, or until state has been manually cleared.sqllistjdbctabledatabaselist-db-tables-db-connectionDatabase Connection Pooling ServiceThe Controller Service that is used to obtain connection to databaseorg.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalselist-db-tables-catalogCatalogThe name of a catalog from which to list database tables. The name must match the catalog name as it is stored in the database. If the property is not set, the catalog name will not be used to narrow the search for tables. If the property is set to an empty string, tables without a catalog will be listed.falsefalsefalseNONEfalsefalselist-db-tables-schema-patternSchema PatternA pattern for matching schemas in the database. Within a pattern, "%" means match any substring of 0 or more characters, and "_" means match any one character. The pattern must match the schema name as it is stored in the database. If the property is not set, the schema name will not be used to narrow the search for tables. If the property is set to an empty string, tables without a schema will be listed.falsefalsefalseNONEfalsefalselist-db-tables-name-patternTable Name PatternA pattern for matching tables in the database. Within a pattern, "%" means match any substring of 0 or more characters, and "_" means match any one character. The pattern must match the table name as it is stored in the database. If the property is not set, all tables will be retrieved.falsefalsefalseNONEfalsefalselist-db-tables-typesTable TypesA comma-separated list of table types to include. For example, some databases support TABLE and VIEW types. If the property is not set, tables of all types will be returned.TABLEfalsefalsefalseNONEfalsefalselist-db-include-countInclude CountWhether to include the table's row count as a flow file attribute. This affects performance as a database query will be generated for each table in the retrieved list.falsetruetruefalsefalsetruefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalselist-db-refresh-intervalRefresh IntervalThe amount of time to elapse before resetting the processor state, thereby causing all current tables to be listed. During this interval, the processor may continue to run, but tables that have already been listed will not be re-listed. However new/added tables will be listed as the processor runs. A value of zero means the state will never be automatically reset, the user must Clear State manually.0 sectruefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalsedb.table.nameContains the name of a database table from the connectiondb.table.catalogContains the name of the catalog to which the table belongs (may be null)db.table.schemaContains the name of the schema to which the table belongs (may be null)db.table.fullnameContains the fully-qualifed table name (possibly including catalog, schema, etc.)db.table.typeContains the type of the database table from the connection. Typical types are "TABLE", "VIEW", "SYSTEM TABLE", "GLOBAL TEMPORARY", "LOCAL TEMPORARY", "ALIAS", "SYNONYM"db.table.remarksContains the name of a database table from the connectiondb.table.countContains the number of rows in the tabletruetrueAfter performing a listing of tables, the timestamp of the query is stored. This allows the Processor to not re-list tables the next time that the Processor is run. Specifying the refresh interval in the processor properties will indicate that when the processor detects the interval has elapsed, the state will be reset and tables will be re-listed as a result. This processor is meant to be run on the primary node only.CLUSTERINPUT_FORBIDDENPerform a full load of a database, retrieving all rows from all tables, or a specific set of tables.full loadrdbmsjdbcdatabaseorg.apache.nifi.processors.standard.ListDatabaseTablesConfigure the "Database Connection Pooling Service" property to specify a Connection Pool that is applicable for interacting with your database. @@ -107,7 +107,7 @@ Leave the "SQL select query" unset. Connect the "success" relationship to the next Processor in the flow. -TIMER_DRIVEN1 min1org.apache.nifi.processors.standard.ListFTPPROCESSORPerforms a listing of the files residing on an FTP server. For each file that is found on the remote server, a new FlowFile will be created with the filename attribute set to the name of the file on the remote server. This can then be used in conjunction with FetchFTP in order to fetch those files.listftpremoteingestsourceinputfileslisting-strategyListing StrategySpecify how to determine new/updated entities. See each strategy descriptions for detail.timestampsTracking TimestampstimestampsThis strategy tracks the latest timestamp of listed entity to determine new/updated entities. Since it only tracks few timestamps, it can manage listing state efficiently. However, any newly added, or updated entity having timestamp older than the tracked latest timestamp can not be picked by this strategy. For example, such situation can happen in a file system if a file with old timestamp is copied or moved into the target directory without its last modified timestamp being updated. Also may miss files when multiple subdirectories are being written at the same time while listing is running.Tracking EntitiesentitiesThis strategy tracks information of all the listed entities within the latest 'Entity Tracking Time Window' to determine new/updated entities. This strategy can pick entities having old timestamp that can be missed with 'Tracking Timestamps'. Works even when multiple subdirectories are being written at the same time while listing is running. However additional DistributedMapCache controller service is required and more JVM heap memory is used. See the description of 'Entity Tracking Time Window' property for further details on how it works.No TrackingnoneThis strategy lists an entity without any tracking. The same entity will be listed each time on executing this processor. It is recommended to change the default run schedule value. Any property that related to the persisting state will be disregarded.Time Windowtime-windowThis strategy uses a sliding time window. The window starts where the previous window ended and ends with the 'current time'. One cycle will list files with modification time falling within the time window. Works even when multiple subdirectories are being written at the same time while listing is running. IMPORTANT: This strategy works properly only if the time on both the system hosting NiFi and the one hosting the files are accurate.truefalsefalseNONEfalsefalseHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueENVIRONMENTfalsefalsePortPortThe port to connect to on the remote host to fetch the data from21truefalsetrueENVIRONMENTfalsefalseUsernameUsernameUsernametruefalsetrueENVIRONMENTfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push files.falsefalsetrueENVIRONMENTfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseDistributed Cache ServiceDistributed Cache ServiceNOTE: This property is used merely for migration from old NiFi version before state management was introduced at version 0.5.0. The stored value in the cache service will be migrated into the state when this processor is started at the first time. The specified Controller Service was used to maintain state about what had been pulled from the remote server so that if a new node begins pulling data, it won't duplicate all of the work that has been done. If not specified, the information was not shared across the cluster. This property did not need to be set for standalone instances of NiFi but was supposed to be configured if NiFi had been running within a cluster.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseSearch RecursivelySearch RecursivelyIf true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefollow-symlinkFollow symlinkIf true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseFile Filter RegexFile Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetchedfalsefalsefalseNONEfalsefalsePath Filter RegexPath Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scannedfalsefalsefalseNONEfalsefalseIgnore Dotted FilesIgnore Dotted FilesIf true, files whose names begin with a dot (".") will be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseRemote Poll Batch SizeRemote Poll Batch SizeThe value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.5000truefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseConnection ModeConnection ModeThe FTP Connection ModePassiveActiveActivePassivePassivefalsefalsefalseNONEfalsefalseTransfer ModeTransfer ModeThe FTP Transfer ModeBinaryBinaryBinaryASCIIASCIIfalsefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: SOCKS + AuthN, HTTP + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseInternal Buffer SizeInternal Buffer SizeSet the internal buffer size for buffered data streams16KBfalsefalsefalseNONEfalsefalsetarget-system-timestamp-precisionTarget System Timestamp PrecisionSpecify timestamp precision at the target system. Since this processor uses timestamp of entities to decide which should be listed, it is crucial to use the right timestamp precision.auto-detectAuto Detectauto-detectAutomatically detect time unit deterministically based on candidate entries timestamp. Please note that this option may take longer to list entities unnecessarily, if none of entries has a precise precision timestamp. E.g. even if a target system supports millis, if all entries only have timestamps without millis, such as '2017-06-16 09:06:34.000', then its precision is determined as 'seconds'.MillisecondsmillisThis option provides the minimum latency for an entry from being available to being listed if target system supports millis, if not, use other options.SecondssecondsFor a target system that does not have millis precision, but has in seconds.MinutesminutesFor a target system that only supports precision in minutes.truefalsefalseNONEfalsefalseet-state-cacheEntity Tracking State CacheListed entities are stored in the specified cache storage so that this processor can resume listing across NiFi restart or in case of primary node change. 'Tracking Entities' strategy require tracking information of all listed entities within the last 'Tracking Time Window'. To support large number of entities, the strategy uses DistributedMapCache instead of managed state. Cache key format is 'ListedEntities::{processorId}(::{nodeId})'. If it tracks per node listed entities, then the optional '::{nodeId}' part is added to manage state separately. E.g. cluster wide cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b', per node cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b::nifi-node3' The stored cache content is Gzipped JSON string. The cache key will be deleted when target listing configuration is changed. Used by 'Tracking Entities' strategy.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseet-time-windowEntity Tracking Time WindowSpecify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile is emitted if one of following condition meets: 1. does not exist in the already-listed entities, 2. has newer timestamp than the cached entity, 3. has different size than the cached entity. If a cached entity's timestamp becomes older than specified time window, that entity will be removed from the cached already-listed entities. Used by 'Tracking Entities' strategy.3 hoursfalsefalsetrueENVIRONMENTfalsefalseet-initial-listing-targetEntity Tracking Initial Listing TargetSpecify how initial listing should be handled. Used by 'Tracking Entities' strategy.allTracking Time WindowwindowIgnore entities having timestamp older than the specified 'Tracking Time Window' at the initial listing activity.All AvailableallRegardless of entities timestamp, all existing entities will be listed at the initial listing activity.falsefalsefalseNONEfalsefalseftp-use-utf8Use UTF-8 EncodingTells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalseftp.remote.hostThe hostname of the FTP Serverftp.remote.portThe port that was connected to on the FTP Serverftp.listing.userThe username of the user that performed the FTP Listingfile.ownerThe numeric owner id of the source filefile.groupThe numeric group id of the source filefile.permissionsThe read/write/execute permissions of the source filefile.sizeThe number of bytes in the source filefile.lastModifiedTimeThe timestamp of when the file in the filesystem waslast modified as 'yyyy-MM-dd'T'HH:mm:ssZ'filenameThe name of the file on the FTP ServerpathThe fully qualified name of the directory on the FTP Server from which the file was pulledtruetrueAfter performing a listing of files, the timestamp of the newest file is stored. This allows the Processor to list only files that have been added or modified after this date the next time that the Processor is run. State is stored across the cluster so that this Processor can be run on Primary Node only and if a new Primary Node is selected, the new node will not duplicate the data that was listed by the previous Primary Node.CLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.standard.FetchFTPorg.apache.nifi.processors.standard.GetFTPorg.apache.nifi.processors.standard.PutFTPTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.ListFilePROCESSORRetrieves a listing of files from the input directory. For each file listed, creates a FlowFile that represents the file so that it can be fetched in conjunction with FetchFile. This Processor is designed to run on Primary Node only in a cluster when 'Input Directory Location' is set to 'Remote'. If the primary node changes, the new Primary Node will pick up where the previous node left off without duplicating all the data. When 'Input Directory Location' is 'Local', the 'Execution' mode can be anything, and synchronization won't happen. Unlike GetFile, this Processor does not delete any data from the local filesystem.filegetlistingestsourcefilesystemInput DirectoryInput DirectoryThe input directory from which files to pull filestruefalsetrueENVIRONMENTfalsefalselisting-strategyListing StrategySpecify how to determine new/updated entities. See each strategy descriptions for detail.timestampsTracking TimestampstimestampsThis strategy tracks the latest timestamp of listed entity to determine new/updated entities. Since it only tracks few timestamps, it can manage listing state efficiently. However, any newly added, or updated entity having timestamp older than the tracked latest timestamp can not be picked by this strategy. For example, such situation can happen in a file system if a file with old timestamp is copied or moved into the target directory without its last modified timestamp being updated. Also may miss files when multiple subdirectories are being written at the same time while listing is running.Tracking EntitiesentitiesThis strategy tracks information of all the listed entities within the latest 'Entity Tracking Time Window' to determine new/updated entities. This strategy can pick entities having old timestamp that can be missed with 'Tracking Timestamps'. Works even when multiple subdirectories are being written at the same time while listing is running. However additional DistributedMapCache controller service is required and more JVM heap memory is used. See the description of 'Entity Tracking Time Window' property for further details on how it works.No TrackingnoneThis strategy lists an entity without any tracking. The same entity will be listed each time on executing this processor. It is recommended to change the default run schedule value. Any property that related to the persisting state will be disregarded.truefalsefalseNONEfalsefalseRecurse SubdirectoriesRecurse SubdirectoriesIndicates whether to list files from subdirectories of the directorytruetruetruefalsefalsetruefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseInput Directory LocationInput Directory LocationSpecifies where the Input Directory is located. This is used to determine whether state should be stored locally or across the cluster.LocalLocalLocalInput Directory is located on a local disk. State will be stored locally on each node in the cluster.RemoteRemoteInput Directory is located on a remote system. State will be stored across the cluster so that the listing can be performed on Primary Node Only and another node can pick up where the last node left off, if the Primary Node changestruefalsefalseNONEfalsefalseFile FilterFile FilterOnly files whose names match the given regular expression will be picked up[^\.].*truefalsefalseNONEfalsefalsePath FilterPath FilterWhen Recurse Subdirectories is true, then only subdirectories whose path matches the given regular expression will be scannedfalsefalsefalseNONEfalsefalseInclude File AttributesInclude File AttributesWhether or not to include information such as the file's Last Modified Time and Owner as FlowFile Attributes. Depending on the File System being used, gathering this information can be expensive and as a result should be disabled. This is especially true of remote file shares.truetruetruefalsefalsetruefalsefalseNONEfalsefalseMinimum File AgeMinimum File AgeThe minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored0 sectruefalsefalseNONEfalsefalseMaximum File AgeMaximum File AgeThe maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignoredfalsefalsefalseNONEfalsefalseMinimum File SizeMinimum File SizeThe minimum size that a file must be in order to be pulled0 BtruefalsefalseNONEfalsefalseMaximum File SizeMaximum File SizeThe maximum size that a file can be in order to be pulledfalsefalsefalseNONEfalsefalseIgnore Hidden FilesIgnore Hidden FilesIndicates whether or not hidden files should be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalsetarget-system-timestamp-precisionTarget System Timestamp PrecisionSpecify timestamp precision at the target system. Since this processor uses timestamp of entities to decide which should be listed, it is crucial to use the right timestamp precision.auto-detectAuto Detectauto-detectAutomatically detect time unit deterministically based on candidate entries timestamp. Please note that this option may take longer to list entities unnecessarily, if none of entries has a precise precision timestamp. E.g. even if a target system supports millis, if all entries only have timestamps without millis, such as '2017-06-16 09:06:34.000', then its precision is determined as 'seconds'.MillisecondsmillisThis option provides the minimum latency for an entry from being available to being listed if target system supports millis, if not, use other options.SecondssecondsFor a target system that does not have millis precision, but has in seconds.MinutesminutesFor a target system that only supports precision in minutes.truefalsefalseNONEfalsefalseet-state-cacheEntity Tracking State CacheListed entities are stored in the specified cache storage so that this processor can resume listing across NiFi restart or in case of primary node change. 'Tracking Entities' strategy require tracking information of all listed entities within the last 'Tracking Time Window'. To support large number of entities, the strategy uses DistributedMapCache instead of managed state. Cache key format is 'ListedEntities::{processorId}(::{nodeId})'. If it tracks per node listed entities, then the optional '::{nodeId}' part is added to manage state separately. E.g. cluster wide cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b', per node cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b::nifi-node3' The stored cache content is Gzipped JSON string. The cache key will be deleted when target listing configuration is changed. Used by 'Tracking Entities' strategy.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseet-time-windowEntity Tracking Time WindowSpecify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile is emitted if one of following condition meets: 1. does not exist in the already-listed entities, 2. has newer timestamp than the cached entity, 3. has different size than the cached entity. If a cached entity's timestamp becomes older than specified time window, that entity will be removed from the cached already-listed entities. Used by 'Tracking Entities' strategy.3 hoursfalsefalsetrueENVIRONMENTfalsefalseet-initial-listing-targetEntity Tracking Initial Listing TargetSpecify how initial listing should be handled. Used by 'Tracking Entities' strategy.allTracking Time WindowwindowIgnore entities having timestamp older than the specified 'Tracking Time Window' at the initial listing activity.All AvailableallRegardless of entities timestamp, all existing entities will be listed at the initial listing activity.falsefalsefalseNONEfalsefalseet-node-identifierEntity Tracking Node IdentifierThe configured value will be appended to the cache key so that listing state can be tracked per NiFi node rather than cluster wide when tracking state is scoped to LOCAL. Used by 'Tracking Entities' strategy.${hostname()}falsefalsetrueENVIRONMENTfalsefalsetrack-performanceTrack PerformanceWhether or not the Processor should track the performance of disk access operations. If true, all accesses to disk will be recorded, including the file being accessed, the information being obtained, and how long it takes. This is then logged periodically at a DEBUG level. While the amount of data will be capped, this option may still consume a significant amount of heap (controlled by the 'Maximum Number of Files to Track' property), but it can be very useful for troubleshooting purposes if performance is poor is degraded.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsemax-performance-metricsMaximum Number of Files to TrackIf the 'Track Performance' property is set to 'true', this property indicates the maximum number of files whose performance metrics should be held onto. A smaller value for this property will result in less heap utilization, while a larger value may provide more accurate insights into how the disk access operations are performing100000truefalsetrueENVIRONMENTfalsefalsemax-operation-timeMax Disk Operation TimeThe maximum amount of time that any single disk operation is expected to take. If any disk operation takes longer than this amount of time, a warning bulletin will be generated for each operation that exceeds this amount of time.10 secsfalsefalsetrueENVIRONMENTfalsefalsemax-listing-timeMax Directory Listing TimeThe maximum amount of time that listing any single directory is expected to take. If the listing for the directory specified by the 'Input Directory' property, or the listing of any subdirectory (if 'Recurse' is set to true) takes longer than this amount of time, a warning bulletin will be generated for each directory listing that exceeds this amount of time.3 minsfalsefalsetrueENVIRONMENTfalsefalsesuccessAll FlowFiles that are received are routed to successfalsefilenameThe name of the file that was read from filesystem.pathThe path is set to the relative path of the file's directory on filesystem compared to the Input Directory property. For example, if Input Directory is set to /tmp, then files picked up from /tmp will have the path attribute set to "/". If the Recurse Subdirectories property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to "abc/1/2/3/".absolute.pathThe absolute.path is set to the absolute path of the file's directory on filesystem. For example, if the Input Directory property is set to /tmp, then files picked up from /tmp will have the path attribute set to "/tmp/". If the Recurse Subdirectories property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to "/tmp/abc/1/2/3/".file.ownerThe user that owns the file in filesystemfile.groupThe group that owns the file in filesystemfile.sizeThe number of bytes in the file in filesystemfile.permissionsThe permissions for the file in filesystem. This is formatted as 3 characters for the owner, 3 for the group, and 3 for other users. For example rw-rw-r--file.lastModifiedTimeThe timestamp of when the file in filesystem was last modified as 'yyyy-MM-dd'T'HH:mm:ssZ'file.lastAccessTimeThe timestamp of when the file in filesystem was last accessed as 'yyyy-MM-dd'T'HH:mm:ssZ'file.creationTimeThe timestamp of when the file in filesystem was created as 'yyyy-MM-dd'T'HH:mm:ssZ'trueAfter performing a listing of files, the timestamp of the newest file is stored. This allows the Processor to list only files that have been added or modified after this date the next time that the Processor is run. Whether the state is stored with a Local or Cluster scope depends on the value of the <Input Directory Location> property.LOCALCLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.standard.GetFileorg.apache.nifi.processors.standard.PutFileorg.apache.nifi.processors.standard.FetchFileTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.ListSFTPPROCESSORPerforms a listing of the files residing on an SFTP server. For each file that is found on the remote server, a new FlowFile will be created with the filename attribute set to the name of the file on the remote server. This can then be used in conjunction with FetchSFTP in order to fetch those files.listsftpremoteingestsourceinputfileslisting-strategyListing StrategySpecify how to determine new/updated entities. See each strategy descriptions for detail.timestampsTracking TimestampstimestampsThis strategy tracks the latest timestamp of listed entity to determine new/updated entities. Since it only tracks few timestamps, it can manage listing state efficiently. However, any newly added, or updated entity having timestamp older than the tracked latest timestamp can not be picked by this strategy. For example, such situation can happen in a file system if a file with old timestamp is copied or moved into the target directory without its last modified timestamp being updated. Also may miss files when multiple subdirectories are being written at the same time while listing is running.Tracking EntitiesentitiesThis strategy tracks information of all the listed entities within the latest 'Entity Tracking Time Window' to determine new/updated entities. This strategy can pick entities having old timestamp that can be missed with 'Tracking Timestamps'. Works even when multiple subdirectories are being written at the same time while listing is running. However additional DistributedMapCache controller service is required and more JVM heap memory is used. See the description of 'Entity Tracking Time Window' property for further details on how it works.No TrackingnoneThis strategy lists an entity without any tracking. The same entity will be listed each time on executing this processor. It is recommended to change the default run schedule value. Any property that related to the persisting state will be disregarded.Time Windowtime-windowThis strategy uses a sliding time window. The window starts where the previous window ended and ends with the 'current time'. One cycle will list files with modification time falling within the time window. Works even when multiple subdirectories are being written at the same time while listing is running. IMPORTANT: This strategy works properly only if the time on both the system hosting NiFi and the one hosting the files are accurate.truefalsefalseNONEfalsefalseHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers22truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalsePrivate Key PathPrivate Key PathThe fully qualified path to the Private Key filefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILEPrivate Key PassphrasePrivate Key PassphrasePassword for the private keyfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push files.falsefalsetrueENVIRONMENTfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseDistributed Cache ServiceDistributed Cache ServiceNOTE: This property is used merely for migration from old NiFi version before state management was introduced at version 0.5.0. The stored value in the cache service will be migrated into the state when this processor is started at the first time. The specified Controller Service was used to maintain state about what had been pulled from the remote server so that if a new node begins pulling data, it won't duplicate all of the work that has been done. If not specified, the information was not shared across the cluster. This property did not need to be set for standalone instances of NiFi but was supposed to be configured if NiFi had been running within a cluster.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseSearch RecursivelySearch RecursivelyIf true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefollow-symlinkFollow symlinkIf true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseFile Filter RegexFile Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetchedfalsefalsefalseNONEfalsefalsePath Filter RegexPath Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scannedfalsefalsefalseNONEfalsefalseIgnore Dotted FilesIgnore Dotted FilesIf true, files whose names begin with a dot (".") will be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseRemote Poll Batch SizeRemote Poll Batch SizeThe value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.5000truefalsefalseNONEfalsefalseStrict Host Key CheckingStrict Host Key CheckingIndicates whether or not strict enforcement of hosts keys should be appliedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseHost Key FileHost Key FileIf supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be usedfalsefalsefalseNONEfalsefalseSINGLEFILEConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseSend Keep Alive On TimeoutSend Keep Alive On TimeoutSend a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.truetruetruefalsefalsetruefalsefalseNONEfalsefalsetarget-system-timestamp-precisionTarget System Timestamp PrecisionSpecify timestamp precision at the target system. Since this processor uses timestamp of entities to decide which should be listed, it is crucial to use the right timestamp precision.auto-detectAuto Detectauto-detectAutomatically detect time unit deterministically based on candidate entries timestamp. Please note that this option may take longer to list entities unnecessarily, if none of entries has a precise precision timestamp. E.g. even if a target system supports millis, if all entries only have timestamps without millis, such as '2017-06-16 09:06:34.000', then its precision is determined as 'seconds'.MillisecondsmillisThis option provides the minimum latency for an entry from being available to being listed if target system supports millis, if not, use other options.SecondssecondsFor a target system that does not have millis precision, but has in seconds.MinutesminutesFor a target system that only supports precision in minutes.truefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: SOCKS + AuthN, HTTP + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseet-state-cacheEntity Tracking State CacheListed entities are stored in the specified cache storage so that this processor can resume listing across NiFi restart or in case of primary node change. 'Tracking Entities' strategy require tracking information of all listed entities within the last 'Tracking Time Window'. To support large number of entities, the strategy uses DistributedMapCache instead of managed state. Cache key format is 'ListedEntities::{processorId}(::{nodeId})'. If it tracks per node listed entities, then the optional '::{nodeId}' part is added to manage state separately. E.g. cluster wide cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b', per node cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b::nifi-node3' The stored cache content is Gzipped JSON string. The cache key will be deleted when target listing configuration is changed. Used by 'Tracking Entities' strategy.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseet-time-windowEntity Tracking Time WindowSpecify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile is emitted if one of following condition meets: 1. does not exist in the already-listed entities, 2. has newer timestamp than the cached entity, 3. has different size than the cached entity. If a cached entity's timestamp becomes older than specified time window, that entity will be removed from the cached already-listed entities. Used by 'Tracking Entities' strategy.3 hoursfalsefalsetrueENVIRONMENTfalsefalseet-initial-listing-targetEntity Tracking Initial Listing TargetSpecify how initial listing should be handled. Used by 'Tracking Entities' strategy.allTracking Time WindowwindowIgnore entities having timestamp older than the specified 'Tracking Time Window' at the initial listing activity.All AvailableallRegardless of entities timestamp, all existing entities will be listed at the initial listing activity.falsefalsefalseNONEfalsefalseMinimum File AgeMinimum File AgeThe minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored0 sectruefalsefalseNONEfalsefalseMaximum File AgeMaximum File AgeThe maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignoredfalsefalsefalseNONEfalsefalseMinimum File SizeMinimum File SizeThe minimum size that a file must be in order to be pulled0 BtruefalsefalseNONEfalsefalseMaximum File SizeMaximum File SizeThe maximum size that a file can be in order to be pulledfalsefalsefalseNONEfalsefalseCiphers AllowedCiphers AllowedA comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctrfalsefalsetrueENVIRONMENTfalsefalseKey Algorithms AllowedKey Algorithms AllowedA comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.comfalsefalsetrueENVIRONMENTfalsefalseKey Exchange Algorithms AllowedKey Exchange Algorithms AllowedA comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-cfalsefalsetrueENVIRONMENTfalsefalseMessage Authentication Codes AllowedMessage Authentication Codes AllowedA comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.comfalsefalsetrueENVIRONMENTfalsefalsesuccessAll FlowFiles that are received are routed to successfalsesftp.remote.hostThe hostname of the SFTP Serversftp.remote.portThe port that was connected to on the SFTP Serversftp.listing.userThe username of the user that performed the SFTP Listingfile.ownerThe numeric owner id of the source filefile.groupThe numeric group id of the source filefile.permissionsThe read/write/execute permissions of the source filefile.sizeThe number of bytes in the source filefile.lastModifiedTimeThe timestamp of when the file in the filesystem waslast modified as 'yyyy-MM-dd'T'HH:mm:ssZ'filenameThe name of the file on the SFTP ServerpathThe fully qualified name of the directory on the SFTP Server from which the file was pulledmime.typeThe MIME Type that is provided by the configured Record WritertruetrueAfter performing a listing of files, the timestamp of the newest file is stored. This allows the Processor to list only files that have been added or modified after this date the next time that the Processor is run. State is stored across the cluster so that this Processor can be run on Primary Node only and if a new Primary Node is selected, the new node will not duplicate the data that was listed by the previous Primary Node.CLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.standard.FetchSFTPorg.apache.nifi.processors.standard.GetSFTPorg.apache.nifi.processors.standard.PutSFTPTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.ListenFTPPROCESSORStarts an FTP server that listens on the specified port and transforms incoming files into FlowFiles. The URI of the service will be ftp://{hostname}:{port}. The default port is 2221.ingestFTPFTPSlistenbind-addressBind AddressThe address the FTP server should be bound to. If not set (or set to 0.0.0.0), the server binds to all available addresses (i.e. all network interfaces of the host machine).falsefalsetrueENVIRONMENTfalsefalselistening-portListening PortThe Port to listen on for incoming connections. On Linux, root privileges are required to use port numbers below 1024.2221truefalsetrueENVIRONMENTfalsefalseusernameUsernameThe name of the user that is allowed to log in to the FTP server. If a username is provided, a password must also be provided. If no username is specified, anonymous connections will be permitted.falsefalsetrueENVIRONMENTfalsefalsepasswordPasswordIf the Username is set, then a password must also be specified. The password provided by the client trying to log in to the FTP server will be checked against this password.falsetruetrueENVIRONMENTfalsefalsessl-context-serviceSSL Context ServiceSpecifies the SSL Context Service that can be used to create secure connections. If an SSL Context Service is selected, then a keystore file must also be specified in the SSL Context Service. Without a keystore file, the processor cannot be started successfully.Specifying a truststore file is optional. If a truststore file is specified, client authentication is required (the client needs to send a certificate to the server).Regardless of the selected TLS protocol, the highest available protocol is used for the connection. For example if NiFi is running on Java 11 and TLSv1.2 is selected in the controller service as the preferred TLS Protocol, TLSv1.3 will be used (regardless of TLSv1.2 being selected) because Java 11 supports TLSv1.3.org.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalsesuccessRelationship for successfully received files.falsefilenameThe name of the file received via the FTP/FTPS connection.pathThe path pointing to the file's target directory. E.g.: file.txt is uploaded to /Folder1/SubFolder, then the value of the path attribute will be "/Folder1/SubFolder/" (note that it ends with a separator character).INPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenHTTPPROCESSORStarts an HTTP Server and listens on a given base path to transform incoming requests into FlowFiles. The default URI of the Service will be http://{hostname}:{port}/contentListener. Only HEAD and POST requests are supported. GET, PUT, DELETE, OPTIONS and TRACE will result in an error and the HTTP response status code 405; CONNECT will also result in an error and the HTTP response status code 400. GET is supported on <service_URI>/healthcheck. If the service is available, it returns "200 OK" with the content "OK". The health check functionality can be configured to be accessible via a different port. For details see the documentation of the "Listening Port for health check requests" property.A Record Reader and Record Writer property can be enabled on the processor to process incoming requests as records. Record processing is not allowed for multipart requests and request in FlowFileV3 format (minifi).ingesthttphttpsrestlistenBase PathBase PathBase path for incoming connectionscontentListenertruefalsetrueENVIRONMENTfalsefalseListening PortListening PortThe Port to listen on for incoming connectionstruefalsetrueENVIRONMENTfalsefalsehealth-check-portListening Port for Health Check RequestsThe port to listen on for incoming health check requests. If set, it must be different from the Listening Port. Configure this port if the processor is set to use two-way SSL and a load balancer that does not support client authentication for health check requests is used. Only /<base_path>/healthcheck service is available via this port and only GET and HEAD requests are supported. If the processor is set not to use SSL, SSL will not be used on this port, either. If the processor is set to use one-way SSL, one-way SSL will be used on this port. If the processor is set to use two-way SSL, one-way SSL will be used on this port (client authentication not required).falsefalsetrueENVIRONMENTfalsefalseMax Data to Receive per SecondMax Data to Receive per SecondThe maximum amount of data to receive per second; this allows the bandwidth to be throttled to a specified data rate; if not specified, the data rate is not throttledfalsefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceSSL Context Service enables support for HTTPSorg.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseHTTP ProtocolsHTTP ProtocolsHTTP Protocols supported for Application Layer Protocol Negotiation with TLSHTTP_1_1http/1.1HTTP_1_1HTTP/1.1h2 http/1.1H2_HTTP_1_1HTTP/2 and HTTP/1.1 negotiated based on requested protocolsh2H2HTTP/2truefalsefalseNONEfalsefalseSSL Context ServiceSSL Context Serviceclient-authenticationClient AuthenticationClient Authentication policy for TLS connections. Required when SSL Context Service configured.AUTOAUTOAUTOInferred based on SSL Context Service properties. The presence of Trust Store properties implies REQUIRED, otherwise NONE is configured.WANTWANTRequests the client certificate on handshake and validates if present but does not require itREQUIREDREQUIREDRequests the client certificate on handshake and rejects the connection if it is not present and validNONENONEDoes not request the client certificate on handshakefalsefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceAuthorized DN PatternAuthorized Subject DN PatternA Regular Expression to apply against the Subject's Distinguished Name of incoming connections. If the Pattern does not match the Subject DN, the the processor will respond with a status of HTTP 403 Forbidden..*truefalsefalseNONEfalsefalseauthorized-issuer-dn-patternAuthorized Issuer DN PatternA Regular Expression to apply against the Issuer's Distinguished Name of incoming connections. If the Pattern does not match the Issuer DN, the processor will respond with a status of HTTP 403 Forbidden..*falsefalsefalseNONEfalsefalseMax Unconfirmed Flowfile TimeMax Unconfirmed Flowfile TimeThe maximum amount of time to wait for a FlowFile to be confirmed before it is removed from the cache60 secstruefalsefalseNONEfalsefalseHTTP Headers to receive as Attributes (Regex)HTTP Headers to receive as Attributes (Regex)Specifies the Regular Expression that determines the names of HTTP Headers that should be passed along as FlowFile attributesfalsefalsefalseNONEfalsefalseReturn CodeReturn CodeThe HTTP return code returned after every HTTP call200falsefalsefalseNONEfalsefalsemultipart-request-max-sizeMultipart Request Max SizeThe max size of the request. Only applies for requests with Content-Type: multipart/form-data, and is used to prevent denial of service type of attacks, to prevent filling up the heap or disk space1 MBtruefalsefalseNONEfalsefalsemultipart-read-buffer-sizeMultipart Read Buffer SizeThe threshold size, at which the contents of an incoming file would be written to disk. Only applies for requests with Content-Type: multipart/form-data. It is used to prevent denial of service type of attacks, to prevent filling up the heap or disk space.512 KBtruefalsefalseNONEfalsefalsemax-thread-pool-sizeMaximum Thread Pool SizeThe maximum number of threads to be used by the embedded Jetty server. The value can be set between 8 and 1000. The value of this property affects the performance of the flows and the operating system, therefore the default value should only be changed in justified cases. A value that is less than the default value may be suitable if only a small number of HTTP clients connect to the server. A greater value may be suitable if a large number of HTTP clients are expected to make requests to the server simultaneously.200truefalsefalseNONEfalsefalserecord-readerRecord ReaderThe Record Reader to use parsing the incoming FlowFile into Recordsorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalserecord-writerRecord WriterThe Record Writer to use for serializing Records after they have been transformedorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-readerRecord ReadersuccessRelationship for successfully received FlowFilesfalseINPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenRELPPROCESSORListens for RELP messages being sent to a given port over TCP. Each message will be acknowledged after successfully writing the message to a FlowFile. Each FlowFile will contain data portion of one or more RELP frames. In the case where the RELP frames contain syslog messages, the output of this processor can be sent to a ParseSyslog processor for further processing.listenrelptcplogsLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsePortPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalseReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor during these surges.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the received data.UTF-8truefalsefalseNONEfalsefalseMax Number of TCP ConnectionsMax Number of Worker ThreadsThe maximum number of worker threads available for servicing TCP connections.2truefalsefalseNONEfalsefalseMax Batch SizeMax Batch SizeThe maximum number of messages to add to a single FlowFile. If multiple messages are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages1truefalsefalseNONEfalsefalseMessage DelimiterBatching Message DelimiterSpecifies the delimiter to place between messages when multiple messages are bundled together (see <Max Batch Size> property).\ntruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection.org.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseClient AuthClient AuthThe client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.REQUIREDWANTWANTREQUIREDREQUIREDNONENONEfalsefalsefalseNONEfalsefalsesuccessMessages received successfully will be sent out this relationship.falserelp.commandThe command of the RELP frames.relp.senderThe sending host of the messages.relp.portThe sending port the messages were received over.relp.txnrThe transaction number of the message. Only included if <Batch Size> is 1.mime.typeThe mime.type of the content which is text/plainINPUT_FORBIDDENorg.apache.nifi.processors.standard.ParseSyslogorg.apache.nifi.processors.standard.ListenSyslogPROCESSORListens for Syslog messages being sent to a given port over TCP or UDP. Incoming messages are checked against regular expressions for RFC5424 and RFC3164 formatted messages. The format of each message is: (<PRIORITY>)(VERSION )(TIMESTAMP) (HOSTNAME) (BODY) where version is optional. The timestamp can be an RFC5424 timestamp with a format of "yyyy-MM-dd'T'HH:mm:ss.SZ" or "yyyy-MM-dd'T'HH:mm:ss.S+hh:mm", or it can be an RFC3164 timestamp with a format of "MMM d HH:mm:ss". If an incoming messages matches one of these patterns, the message will be parsed and the individual pieces will be placed in FlowFile attributes, with the original message in the content of the FlowFile. If an incoming message does not match one of these patterns it will not be parsed and the syslog.valid attribute will be set to false with the original message in the content of the FlowFile. Valid messages will be transferred on the success relationship, and invalid messages will be transferred on the invalid relationship.sysloglistenudptcplogsProtocolProtocolThe protocol for Syslog communication.UDPTCPTCPUDPUDPtruefalsefalseNONEfalsefalsePortPortThe port for Syslog communication. Note that Expression language is not evaluated per FlowFile.truefalsetrueENVIRONMENTfalsefalseLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsesocket-keep-aliveSocket Keep AliveWhether or not to have TCP socket keep alive turned on. Timing details depend on operating system properties.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseProtocolProtocolTCPSSL Context ServiceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, syslog messages will be received over a secure connection.org.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProtocolProtocolTCPClient AuthClient AuthThe client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.REQUIREDWANTWANTREQUIREDREQUIREDNONENONEfalsefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive Syslog messages. Adjust this value appropriately based on the expected size of the incoming Syslog messages. When UDP is selected each buffer will hold one Syslog message. When TCP is selected messages are read from an incoming connection until the buffer is full, or the connection is closed. 65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseProtocolProtocolTCPMax Number of TCP ConnectionsMax Number of TCP ConnectionsThe maximum number of concurrent connections to accept Syslog messages in TCP mode.2truefalsefalseNONEfalsefalseProtocolProtocolTCPMax Batch SizeMax Batch SizeThe maximum number of Syslog events to add to a single FlowFile. If multiple events are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages1truefalsefalseNONEfalsefalseMessage DelimiterMessage DelimiterSpecifies the delimiter to place between Syslog messages when multiple messages are bundled together (see <Max Batch Size> property).\ntruefalsefalseNONEfalsefalseParse MessagesParse MessagesIndicates if the processor should parse the Syslog messages. If set to false, each outgoing FlowFile will only contain the sender, protocol, and port, and no additional attributes.truetruetruefalsefalsetruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the Syslog messages. Note that Expression language is not evaluated per FlowFile.UTF-8truefalsetrueENVIRONMENTfalsefalsesuccessSyslog messages that match one of the expected formats will be sent out this relationship as a FlowFile per message.falseinvalidSyslog messages that do not match one of the expected formats will be sent out this relationship as a FlowFile per message.falsesyslog.priorityThe priority of the Syslog message.syslog.severityThe severity of the Syslog message derived from the priority.syslog.facilityThe facility of the Syslog message derived from the priority.syslog.versionThe optional version from the Syslog message.syslog.timestampThe timestamp of the Syslog message.syslog.hostnameThe hostname or IP address of the Syslog message.syslog.senderThe hostname of the Syslog server that sent the message.syslog.bodyThe body of the Syslog message, everything after the hostname.syslog.validAn indicator of whether this message matched the expected formats. If this value is false, the other attributes will be empty and only the original message will be available in the content.syslog.protocolThe protocol over which the Syslog message was received.syslog.portThe port over which the Syslog message was received.mime.typeThe mime.type of the FlowFile which will be text/plain for Syslog messages.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.PutSyslogorg.apache.nifi.processors.standard.ParseSyslogorg.apache.nifi.processors.standard.ListenTCPPROCESSORListens for incoming TCP connections and reads data from each connection using a line separator as the message demarcator. The default behavior is for each message to produce a single FlowFile, however this can be controlled by increasing the Batch Size to a larger value for higher throughput. The Receive Buffer Size must be set as large as the largest messages expected to be received, meaning if every 100kb there is a line separator, then the Receive Buffer Size must be greater than 100kb. The processor can be configured to use an SSL Context Service to only allow secure connections. When connected clients present certificates for mutual TLS authentication, the Distinguished Names of the client certificate's issuer and subject are added to the outgoing FlowFiles as attributes. The processor does not perform authorization based on Distinguished Name values, but since these values are attached to the outgoing FlowFiles, authorization can be implemented based on these attributes.listentcptlssslLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsePortPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalseReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor during these surges.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the received data.UTF-8truefalsefalseNONEfalsefalseMax Number of TCP ConnectionsMax Number of Worker ThreadsThe maximum number of worker threads available for servicing TCP connections.2truefalsefalseNONEfalsefalseMax Batch SizeMax Batch SizeThe maximum number of messages to add to a single FlowFile. If multiple messages are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages1truefalsefalseNONEfalsefalseMessage DelimiterBatching Message DelimiterSpecifies the delimiter to place between messages when multiple messages are bundled together (see <Max Batch Size> property).\ntruefalsefalseNONEfalsefalseidle-timeoutIdle Connection TimeoutThe amount of time a client's connection will remain open if no data is received. The default of 0 seconds will leave connections open until they are closed by the client.0 secondstruefalsefalseNONEfalsefalsemax-receiving-threadsMax Number of Receiving Message Handler ThreadsThis property is deprecated and no longer used.falsefalsefalseNONEfalsefalsepool-receive-buffersPool Receive BuffersEnable or disable pooling of buffers that the processor uses for handling bytes received on socket connections. The framework allocates buffers as needed during processing.TrueTrueTrueFalseFalsetruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection.org.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseClient AuthClient AuthThe client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.REQUIREDWANTWANTREQUIREDREQUIREDNONENONEfalsefalsefalseNONEfalsefalsesuccessMessages received successfully will be sent out this relationship.falsetcp.senderThe sending host of the messages.tcp.portThe sending port the messages were received.client.certificate.issuer.dnFor connections using mutual TLS, the Distinguished Name of the Certificate Authority that issued the client's certificate is attached to the FlowFile.client.certificate.subject.dnFor connections using mutual TLS, the Distinguished Name of the client certificate's owner (subject) is attached to the FlowFile.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenTCPRecordPROCESSORListens for incoming TCP connections and reads data from each connection using a configured record reader, and writes the records to a flow file using a configured record writer. The type of record reader selected will determine how clients are expected to send data. For example, when using a Grok reader to read logs, a client can keep an open connection and continuously stream data, but when using an JSON reader, the client cannot send an array of JSON documents and then send another array on the same connection, as the reader would be in a bad state at that point. Records will be read from the connection in blocking mode, and will timeout according to the Read Timeout specified in the processor. If the read times out, or if any other error is encountered when reading, the connection will be closed, and any records read up to that point will be handled according to the configured Read Error Strategy (Discard or Transfer). In cases where clients are keeping a connection open, the concurrent tasks for the processor should be adjusted to match the Max Number of TCP Connections allowed, so that there is a task processing each connection. The processor can be configured to use an SSL Context Service to only allow secure connections. When connected clients present certificates for mutual TLS authentication, the Distinguished Names of the client certificate's issuer and subject are added to the outgoing FlowFiles as attributes. The processor does not perform authorization based on Distinguished Name values, but since these values are attached to the outgoing FlowFiles, authorization can be implemented based on these attributes.listentcprecordtlssslLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalseportPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalsemax-size-socket-bufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalsemax-number-tcp-connectionsMax Number of TCP ConnectionsThe maximum number of concurrent TCP connections to accept. In cases where clients are keeping a connection open, the concurrent tasks for the processor should be adjusted to match the Max Number of TCP Connections allowed, so that there is a task processing each connection.2truefalsefalseNONEfalsefalseread-timeoutRead TimeoutThe amount of time to wait before timing out when reading from a connection.10 secondstruefalsefalseNONEfalsefalserecord-readerRecord ReaderThe Record Reader to use for incoming FlowFilesorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterThe Record Writer to use in order to serialize the data before writing to a FlowFileorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsereader-error-handling-strategyRead Error StrategyIndicates how to deal with an error while reading the next record from a connection, when previous records have already been read from the connection.TransferTransferTransferTransfers any records already received and closes the connection.DiscardDiscardDiscards any records already received and closes the connection.truefalsefalseNONEfalsefalserecord-batch-sizeRecord Batch SizeThe maximum number of records to write to a single FlowFile.1000truefalsefalseNONEfalsefalsessl-context-serviceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection.org.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseclient-authClient AuthThe client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.REQUIREDWANTWANTREQUIREDREQUIREDNONENONEfalsefalsefalseNONEfalsefalsesuccessMessages received successfully will be sent out this relationship.falsetcp.senderThe host that sent the data.tcp.portThe port that the processor accepted the connection on.record.countThe number of records written to the flow file.mime.typeThe mime-type of the writer used to write the records to the flow file.client.certificate.issuer.dnFor connections using mutual TLS, the Distinguished Name of the Certificate Authority that issued the client's certificate is attached to the FlowFile.client.certificate.subject.dnFor connections using mutual TLS, the Distinguished Name of the client certificate's owner (subject) is attached to the FlowFile.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenUDPPROCESSORListens for Datagram Packets on a given port. The default behavior produces a FlowFile per datagram, however for higher throughput the Max Batch Size property may be increased to specify the number of datagrams to batch together in a single FlowFile. This processor can be restricted to listening for datagrams from a specific remote host and port by specifying the Sending Host and Sending Host Port properties, otherwise it will listen for datagrams from all hosts and ports.ingestudplistensourceLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsePortPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalseReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the received data.UTF-8truefalsefalseNONEfalsefalseMax Batch SizeMax Batch SizeThe maximum number of messages to add to a single FlowFile. If multiple messages are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages1truefalsefalseNONEfalsefalseMessage DelimiterBatching Message DelimiterSpecifies the delimiter to place between messages when multiple messages are bundled together (see <Max Batch Size> property).\ntruefalsefalseNONEfalsefalseSending HostSending HostIP, or name, of a remote host. Only Datagrams from the specified Sending Host Port and this host will be accepted. Improves Performance. May be a system property or an environment variable.falsefalsetrueENVIRONMENTfalsefalseSending Host PortSending Host PortPort being used by remote host to send Datagrams. Only Datagrams from the specified Sending Host and this port will be accepted. Improves Performance. May be a system property or an environment variable.falsefalsetrueENVIRONMENTfalsefalsesuccessMessages received successfully will be sent out this relationship.falseudp.senderThe sending host of the messages.udp.portThe sending port the messages were received.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenUDPRecordPROCESSORListens for Datagram Packets on a given port and reads the content of each datagram using the configured Record Reader. Each record will then be written to a flow file using the configured Record Writer. This processor can be restricted to listening for datagrams from a specific remote host and port by specifying the Sending Host and Sending Host Port properties, otherwise it will listen for datagrams from all hosts and ports.ingestudplistensourcerecordLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsePortPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalseReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the received data.UTF-8truefalsefalseNONEfalsefalsepoll-timeoutPoll TimeoutThe amount of time to wait when polling the internal queue for more datagrams. If no datagrams are found after waiting for the configured timeout, then the processor will emit whatever records have been obtained up to that point.50 mstruefalsefalseNONEfalsefalsebatch-sizeBatch SizeThe maximum number of datagrams to write as records to a single FlowFile. The Batch Size will only be reached when data is coming in more frequently than the Poll Timeout.1000truefalsefalseNONEfalsefalserecord-readerRecord ReaderThe Record Reader to use for reading the content of incoming datagrams.org.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterThe Record Writer to use in order to serialize the data before writing to a flow file.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsesending-hostSending HostIP, or name, of a remote host. Only Datagrams from the specified Sending Host Port and this host will be accepted. Improves Performance. May be a system property or an environment variable.falsefalsetrueENVIRONMENTfalsefalsesending-host-portSending Host PortPort being used by remote host to send Datagrams. Only Datagrams from the specified Sending Host and this port will be accepted. Improves Performance. May be a system property or an environment variable.falsefalsetrueENVIRONMENTfalsefalsesuccessMessages received successfully will be sent out this relationship.falseparse.failureIf a datagram cannot be parsed using the configured Record Reader, the contents of the message will be routed to this Relationship as its own individual FlowFile.falseudp.senderThe sending host of the messages.udp.portThe sending port the messages were received.record.countThe number of records written to the flow file.mime.typeThe mime-type of the writer used to write the records to the flow file.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.LogAttributePROCESSOREmits attributes of the FlowFile at the specified log levelattributesloggingLog LevelLog LevelThe Log Level to use when logging the AttributesinfotracetracedebugdebuginfoinfowarnwarnerrorerrortruefalsefalseNONEfalsefalseLog PayloadLog PayloadIf true, the FlowFile's payload will be logged, in addition to its attributes; otherwise, just the Attributes will be logged.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseAttributes to LogAttributes to LogA comma-separated list of Attributes to Log. If not specified, all attributes will be logged unless `Attributes to Log by Regular Expression` is modified. There's an AND relationship between the two properties.falsefalsefalseNONEfalsefalseattributes-to-log-regexAttributes to Log by Regular ExpressionA regular expression indicating the Attributes to Log. If not specified, all attributes will be logged unless `Attributes to Log` is modified. There's an AND relationship between the two properties..*falsefalsefalseNONEfalsefalseAttributes to IgnoreAttributes to IgnoreA comma-separated list of Attributes to ignore. If not specified, no attributes will be ignored unless `Attributes to Ignore by Regular Expression` is modified. There's an OR relationship between the two properties.falsefalsefalseNONEfalsefalseattributes-to-ignore-regexAttributes to Ignore by Regular ExpressionA regular expression indicating the Attributes to Ignore. If not specified, no attributes will be ignored unless `Attributes to Ignore` is modified. There's an OR relationship between the two properties.falsefalsefalseNONEfalsefalseLog FlowFile PropertiesLog FlowFile PropertiesSpecifies whether or not to log FlowFile "properties", such as Entry Date, Lineage Start Date, and content sizetruetruetruefalsefalsetruefalsefalseNONEfalsefalseOutput FormatOutput FormatSpecifies the format to use for logging FlowFile attributesLine per AttributeLine per AttributeLine per AttributeEach FlowFile attribute will be logged using a single line for the attribute name and another line for the attribute value. This format is often most advantageous when looking at the attributes of a single FlowFile.Single LineSingle LineAll FlowFile attribute names and values will be logged on a single line. This format is often most advantageous when comparing logs from multiple FlowFiles.truefalsefalseNONEfalsefalseLog prefixLog prefixLog prefix appended to the log lines. It helps to distinguish the output of multiple LogAttribute processors.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsecharacter-setCharacter SetThe name of the CharacterSet to useUTF-8truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessAll FlowFiles are routed to this relationshipfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.LogMessagePROCESSOREmits a log message at the specified log levelattributeslogginglog-levelLog LevelThe Log Level to use when logging the message: [trace, debug, info, warn, error]infotruefalsetrueFLOWFILE_ATTRIBUTESfalsefalselog-prefixLog prefixLog prefix appended to the log lines. It helps to distinguish the output of multiple LogMessage processors.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalselog-messageLog messageThe log message to emitfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessAll FlowFiles are routed to this relationshipfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.LookupAttributePROCESSORLookup attributes from a lookup servicelookupcacheenrichjoinattributesAttribute Expression Languagelookup-serviceLookup ServiceThe lookup service to use for attribute lookupsorg.apache.nifi.lookup.StringLookupServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseinclude-empty-valuesInclude Empty ValuesInclude null or blank values for keys that are null or blanktruetruetruefalsefalsetruefalsefalseNONEfalsefalseThe name of the attribute to add to the FlowFileThe name of the key or property to retrieve from the lookup serviceAdds a FlowFile attribute specified by the dynamic property's key with the value found in the lookup service using the the dynamic property's valueFLOWFILE_ATTRIBUTESfailureFlowFiles with failing lookups are routed to this relationshipfalseunmatchedFlowFiles with missing lookups are routed to this relationshipfalsematchedFlowFiles with matching lookups are routed to this relationshipfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.LookupRecordPROCESSORExtracts one or more fields from a Record and looks up a value for those fields in a LookupService. If a result is returned by the LookupService, that result is optionally added to the Record. In this case, the processor functions as an Enrichment processor. Regardless, the Record is then routed to either the 'matched' relationship or 'unmatched' relationship (if the 'Routing Strategy' property is configured to do so), indicating whether or not a result was returned by the LookupService, allowing the processor to also function as a Routing processor. The "coordinates" to use for looking up a value in the Lookup Service are defined by adding a user-defined property. Each property that is added will have an entry added to a Map, where the name of the property becomes the Map Key and the value returned by the RecordPath becomes the value for that key. If multiple values are returned by the RecordPath, then the Record will be routed to the 'unmatched' relationship (or 'success', depending on the 'Routing Strategy' property's configuration). If one or more fields match the Result RecordPath, all fields that match will be updated. If there is no match in the configured LookupService, then no fields will be updated. I.e., it will not overwrite an existing value in the Record with a null value. Please note, however, that if the results returned by the LookupService are not accounted for in your schema (specifically, the schema that is configured for your Record Writer) then the fields will not be written out to the FlowFile.lookupenrichmentrouterecordcsvjsonavrodatabasedblogsconvertfilterrecord-readerRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalselookup-serviceLookup ServiceThe Lookup Service to use in order to lookup a value in each Recordorg.apache.nifi.lookup.LookupServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseRoot Record PathRoot Record PathA RecordPath that points to a child Record within each of the top-level Records in the FlowFile. If specified, the additional RecordPath properties will be evaluated against this child Record instead of the top-level Record. This allows for performing enrichment against multiple child Records within a single top-level Record.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalserouting-strategyRouting StrategySpecifies how to route records after a Lookup has completedroute-to-successRoute to 'success'route-to-successRecords will be routed to a 'success' Relationship regardless of whether or not there is a match in the configured Lookup ServiceRoute to 'matched' or 'unmatched'route-to-matched-unmatchedRecords will be routed to either a 'matched' or an 'unmatched' Relationship depending on whether or not there was a match in the configured Lookup Service. A single input FlowFile may result in two different output FlowFiles. If the given Record Paths evaluate such that multiple sub-records are evaluated, the parent Record will be routed to 'unmatched' unless all sub-records match.truefalsefalseNONEfalsefalseresult-contentsRecord Result ContentsWhen a result is obtained that contains a Record, this property determines whether the Record itself is inserted at the configured path or if the contents of the Record (i.e., the sub-fields) will be inserted at the configured path.insert-entire-recordInsert Entire Recordinsert-entire-recordThe entire Record that is retrieved from the Lookup Service will be inserted into the destination path.Insert Record Fieldsrecord-fieldsAll of the fields in the Record that is retrieved from the Lookup Service will be inserted into the destination path.truefalsefalseNONEfalsefalserecord-update-strategyRecord Update StrategyThis property defines the strategy to use when updating the record with the value returned by the Lookup Service.use-propertyReplace Existing Valuesreplace-existing-valuesThe "Result RecordPath" property will be ignored and the lookup service must be a single simple key lookup service. Every dynamic property value should be a record path. For each dynamic property, the value contained in the field corresponding to the record path will be used as the key in the Lookup Service and the value returned by the Lookup Service will be used to replace the existing value. It is possible to configure multiple dynamic properties to replace multiple values in one execution. This strategy only supports simple types replacements (strings, integers, etc).Use "Result RecordPath" Propertyuse-propertyThe "Result RecordPath" property will be used to determine which part of the record should be updated with the value returned by the Lookup ServicetruefalsefalseNONEfalsefalseresult-record-pathResult RecordPathA RecordPath that points to the field whose value should be updated with whatever value is returned from the Lookup Service. If not specified, the value that is returned from the Lookup Service will be ignored, except for determining whether the FlowFile should be routed to the 'matched' or 'unmatched' Relationship.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalserecord-update-strategyRecord Update Strategyuse-propertyrecord-path-lookup-miss-result-cache-sizeCache SizeSpecifies how many lookup values/records should be cached.Setting this property to zero means no caching will be done and the table will be queried for each lookup value in each record. If the lookup table changes often or the most recent data must be retrieved, do not use the cache.0truefalsetrueENVIRONMENTfalsefalseValue To LookupValid Record PathA RecordPath that points to the field whose value will be looked up in the configured Lookup ServiceFLOWFILE_ATTRIBUTESsuccessAll records will be sent to this Relationship if configured to do so, unless a failure occursfalsefailureIf a FlowFile cannot be enriched, the unchanged FlowFile will be routed to this relationshipfalsemime.typeSets the mime.type attribute to the MIME Type specified by the Record Writerrecord.countThe number of records in the FlowFiletruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ConvertRecordorg.apache.nifi.processors.standard.SplitRecordorg.apache.nifi.lookup.SimpleKeyValueLookupServiceorg.apache.nifi.lookup.maxmind.IPLookupServiceorg.apache.nifi.lookup.db.DatabaseRecordLookupServiceorg.apache.nifi.processors.standard.MergeContentPROCESSORMerges a Group of FlowFiles together based on a user-defined strategy and packages them into a single FlowFile. It is recommended that the Processor be configured with only a single incoming connection, as Group of FlowFiles will not be created from FlowFiles in different connections. This processor updates the mime.type attribute as appropriate. NOTE: this processor should NOT be configured with Cron Driven for the Scheduling Strategy.mergecontentcorrelationtarzipstreamconcatenationarchiveflowfile-streamflowfile-stream-v3Merge StrategyMerge StrategySpecifies the algorithm used to merge content. The 'Defragment' algorithm combines fragments that are associated by attributes back into a single cohesive FlowFile. The 'Bin-Packing Algorithm' generates a FlowFile populated by arbitrarily chosen FlowFilesBin-Packing AlgorithmBin-Packing AlgorithmBin-Packing AlgorithmGenerates 'bins' of FlowFiles and fills each bin as full as possible. FlowFiles are placed into a bin based on their size and optionally their attributes (if the <Correlation Attribute> property is set)DefragmentDefragmentCombines fragments that are associated by attributes back into a single cohesive FlowFile. If using this strategy, all FlowFiles must have the attributes <fragment.identifier>, <fragment.count>, and <fragment.index>. All FlowFiles with the same value for "fragment.identifier" will be grouped together. All FlowFiles in this group must have the same value for the "fragment.count" attribute. All FlowFiles in this group must have a unique value for the "fragment.index" attribute between 0 and the value of the "fragment.count" attribute.truefalsefalseNONEfalsefalseMerge FormatMerge FormatDetermines the format that will be used to merge the content.Binary ConcatenationTARTARA bin of FlowFiles will be combined into a single TAR file. The FlowFiles' <path> attribute will be used to create a directory in the TAR file if the <Keep Paths> property is set to true; otherwise, all FlowFiles will be added at the root of the TAR file. If a FlowFile has an attribute named <tar.permissions> that is 3 characters, each between 0-7, that attribute will be used as the TAR entry's 'mode'.ZIPZIPA bin of FlowFiles will be combined into a single ZIP file. The FlowFiles' <path> attribute will be used to create a directory in the ZIP file if the <Keep Paths> property is set to true; otherwise, all FlowFiles will be added at the root of the ZIP file. The <Compression Level> property indicates the ZIP compression to use.FlowFile Stream, v3FlowFile Stream, v3A bin of FlowFiles will be combined into a single Version 3 FlowFile StreamFlowFile Stream, v2FlowFile Stream, v2A bin of FlowFiles will be combined into a single Version 2 FlowFile StreamFlowFile Tar, v1FlowFile Tar, v1A bin of FlowFiles will be combined into a single Version 1 FlowFile PackageBinary ConcatenationBinary ConcatenationThe contents of all FlowFiles will be concatenated together into a single FlowFileAvroAvroThe Avro contents of all FlowFiles will be concatenated together into a single FlowFiletruefalsefalseNONEfalsefalseAttribute StrategyAttribute StrategyDetermines which FlowFile attributes should be added to the bundle. If 'Keep All Unique Attributes' is selected, any attribute on any FlowFile that gets bundled will be kept unless its value conflicts with the value from another FlowFile. If 'Keep Only Common Attributes' is selected, only the attributes that exist on all FlowFiles in the bundle, with the same value, will be preserved.Keep Only Common AttributesKeep Only Common AttributesKeep Only Common AttributesAny attribute that is not the same on all FlowFiles in a bin will be dropped. Those that are the same across all FlowFiles will be retained.Keep All Unique AttributesKeep All Unique AttributesAny attribute that has the same value for all FlowFiles in a bin, or has no value for a FlowFile, will be kept. For example, if a bin consists of 3 FlowFiles and 2 of them have a value of 'hello' for the 'greeting' attribute and the third FlowFile has no 'greeting' attribute then the outbound FlowFile will get a 'greeting' attribute with the value 'hello'.truefalsefalseNONEfalsefalseCorrelation Attribute NameCorrelation Attribute NameIf specified, like FlowFiles will be binned together, where 'like FlowFiles' means FlowFiles that have the same value for this Attribute. If not specified, FlowFiles are bundled by the order in which they are pulled from the queue.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMerge StrategyMerge StrategyBin-Packing Algorithmmergecontent-metadata-strategyMetadata StrategyFor FlowFiles whose input format supports metadata (Avro, e.g.), this property determines which metadata should be added to the bundle. If 'Use First Metadata' is selected, the metadata keys/values from the first FlowFile to be bundled will be used. If 'Keep Only Common Metadata' is selected, only the metadata that exists on all FlowFiles in the bundle, with the same value, will be preserved. If 'Ignore Metadata' is selected, no metadata is transferred to the outgoing bundled FlowFile. If 'Do Not Merge Uncommon Metadata' is selected, any FlowFile whose metadata values do not match those of the first bundled FlowFile will not be merged.Do Not Merge Uncommon MetadataUse First MetadataUse First MetadataFor any input format that supports metadata (Avro, e.g.), the metadata for the first FlowFile in the bin will be set on the output FlowFile.Keep Only Common MetadataKeep Only Common MetadataFor any input format that supports metadata (Avro, e.g.), any FlowFile whose metadata values match those of the first FlowFile, any additional metadata will be dropped but the FlowFile will be merged. Any FlowFile whose metadata values do not match those of the first FlowFile in the bin will not be merged.Do Not Merge Uncommon MetadataDo Not Merge Uncommon MetadataFor any input format that supports metadata (Avro, e.g.), any FlowFile whose metadata values do not match those of the first FlowFile in the bin will not be merged.Ignore MetadataIgnore MetadataIgnores (does not transfer, compare, etc.) any metadata from a FlowFile whose content supports embedded metadata.truefalsefalseNONEfalsefalseMerge FormatMerge FormatAvroMinimum Number of EntriesMinimum Number of EntriesThe minimum number of files to include in a bundle1truefalsefalseNONEfalsefalseMerge StrategyMerge StrategyBin-Packing AlgorithmMaximum Number of EntriesMaximum Number of EntriesThe maximum number of files to include in a bundle1000truefalsefalseNONEfalsefalseMerge StrategyMerge StrategyBin-Packing AlgorithmMinimum Group SizeMinimum Group SizeThe minimum size for the bundle0 BtruefalsefalseNONEfalsefalseMerge StrategyMerge StrategyBin-Packing AlgorithmMaximum Group SizeMaximum Group SizeThe maximum size for the bundle. If not specified, there is no maximum.falsefalsefalseNONEfalsefalseMerge StrategyMerge StrategyBin-Packing AlgorithmMax Bin AgeMax Bin AgeThe maximum age of a Bin that will trigger a Bin to be complete. Expected format is <duration> <time unit> where <duration> is a positive integer and time unit is one of seconds, minutes, hoursfalsefalsefalseNONEfalsefalseMaximum number of BinsMaximum number of BinsSpecifies the maximum number of bins that can be held in memory at any one time5truefalsefalseNONEfalsefalseDelimiter StrategyDelimiter StrategyDetermines if Header, Footer, and Demarcator should point to files containing the respective content, or if the values of the properties should be used as the content.Do Not Use DelimitersDo Not Use DelimitersDo Not Use DelimitersNo Header, Footer, or Demarcator will be usedFilenameFilenameThe values of Header, Footer, and Demarcator will be retrieved from the contents of a fileTextTextThe values of Header, Footer, and Demarcator will be specified as property valuestruefalsefalseNONEfalsefalseMerge FormatMerge FormatBinary ConcatenationHeader FileHeaderFilename or text specifying the header to use. If not specified, no header is supplied.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLETEXTFILEMerge FormatMerge FormatBinary ConcatenationDelimiter StrategyDelimiter StrategyFilenameTextFooter FileFooterFilename or text specifying the footer to use. If not specified, no footer is supplied.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLETEXTFILEMerge FormatMerge FormatBinary ConcatenationDelimiter StrategyDelimiter StrategyFilenameTextDemarcator FileDemarcatorFilename or text specifying the demarcator to use. If not specified, no demarcator is supplied.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLETEXTFILEMerge FormatMerge FormatBinary ConcatenationDelimiter StrategyDelimiter StrategyFilenameTextCompression LevelCompression LevelSpecifies the compression level to use when using the Zip Merge Format; if not using the Zip Merge Format, this value is ignored100112233445566778899truefalsefalseNONEfalsefalseMerge FormatMerge FormatZIPKeep PathKeep PathIf using the Zip or Tar Merge Format, specifies whether or not the FlowFiles' paths should be included in their entry names.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseMerge FormatMerge FormatZIPTARTar Modified TimeTar Modified TimeIf using the Tar Merge Format, specifies if the Tar entry should store the modified timestamp either by expression (e.g. ${file.lastModifiedTime} or static value, both of which must match the ISO8601 format 'yyyy-MM-dd'T'HH:mm:ssZ'.${file.lastModifiedTime}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMerge FormatMerge FormatTARfailureIf the bundle cannot be created, all FlowFiles that would have been used to created the bundle will be transferred to failurefalseoriginalThe FlowFiles that were used to create the bundlefalsemergedThe FlowFile containing the merged contentfalsefragment.identifierApplicable only if the <Merge Strategy> property is set to Defragment. All FlowFiles with the same value for this attribute will be bundled together.fragment.indexApplicable only if the <Merge Strategy> property is set to Defragment. This attribute indicates the order in which the fragments should be assembled. This attribute must be present on all FlowFiles when using the Defragment Merge Strategy and must be a unique (i.e., unique across all FlowFiles that have the same value for the "fragment.identifier" attribute) integer between 0 and the value of the fragment.count attribute. If two or more FlowFiles have the same value for the "fragment.identifier" attribute and the same value for the "fragment.index" attribute, the first FlowFile processed will be accepted and subsequent FlowFiles will not be accepted into the Bin.fragment.countApplicable only if the <Merge Strategy> property is set to Defragment. This attribute indicates how many FlowFiles should be expected in the given bundle. At least one FlowFile must have this attribute in the bundle. If multiple FlowFiles contain the "fragment.count" attribute in a given bundle, all must have the same value.segment.original.filenameApplicable only if the <Merge Strategy> property is set to Defragment. This attribute must be present on all FlowFiles with the same value for the fragment.identifier attribute. All FlowFiles in the same bundle must have the same value for this attribute. The value of this attribute will be used for the filename of the completed merged FlowFile.tar.permissionsApplicable only if the <Merge Format> property is set to TAR. The value of this attribute must be 3 characters; each character must be in the range 0 to 7 (inclusive) and indicates the file permissions that should be used for the FlowFile's TAR entry. If this attribute is missing or has an invalid value, the default value of 644 will be usedfilenameWhen more than 1 file is merged, the filename comes from the segment.original.filename attribute. If that attribute does not exist in the source FlowFiles, then the filename is set to the number of nanoseconds matching system time. Then a filename extension may be applied:if Merge Format is TAR, then the filename will be appended with .tar, if Merge Format is ZIP, then the filename will be appended with .zip, if Merge Format is FlowFileStream, then the filename will be appended with .pkgmerge.countThe number of FlowFiles that were merged into this bundlemerge.bin.ageThe age of the bin, in milliseconds, when it was merged and output. Effectively this is the greatest amount of time that any FlowFile in this bundle remained waiting in this processor before it was outputmerge.uuidUUID of the merged flow file that will be added to the original flow files attributes.merge.reasonThis processor allows for several thresholds to be configured for merging FlowFiles. This attribute indicates which of the Thresholds resulted in the FlowFiles being merged. For an explanation of each of the possible values and their meanings, see the Processor's Usage / documentation and see the 'Additional Details' page.truetrueINPUT_REQUIREDMEMORYWhile content is not stored in memory, the FlowFiles' attributes are. The configuration of MergeContent (maximum bin size, maximum group size, maximum bin age, max number of entries) will influence how much memory is used. If merging together many small FlowFiles, a two-stage approach may be necessary in order to avoid excessive use of memory.Concatenate FlowFiles with textual content together in order to create fewer, larger FlowFiles.concatenatebundleaggregatebinmergecombinesmashINPUT_ALLOWED"Merge Strategy" = "Bin Packing Algorithm" +TIMER_DRIVEN1 min1org.apache.nifi.processors.standard.ListFTPPROCESSORPerforms a listing of the files residing on an FTP server. For each file that is found on the remote server, a new FlowFile will be created with the filename attribute set to the name of the file on the remote server. This can then be used in conjunction with FetchFTP in order to fetch those files.listftpremoteingestsourceinputfileslisting-strategyListing StrategySpecify how to determine new/updated entities. See each strategy descriptions for detail.timestampsTracking TimestampstimestampsThis strategy tracks the latest timestamp of listed entity to determine new/updated entities. Since it only tracks few timestamps, it can manage listing state efficiently. However, any newly added, or updated entity having timestamp older than the tracked latest timestamp can not be picked by this strategy. For example, such situation can happen in a file system if a file with old timestamp is copied or moved into the target directory without its last modified timestamp being updated. Also may miss files when multiple subdirectories are being written at the same time while listing is running.Tracking EntitiesentitiesThis strategy tracks information of all the listed entities within the latest 'Entity Tracking Time Window' to determine new/updated entities. This strategy can pick entities having old timestamp that can be missed with 'Tracking Timestamps'. Works even when multiple subdirectories are being written at the same time while listing is running. However additional DistributedMapCache controller service is required and more JVM heap memory is used. See the description of 'Entity Tracking Time Window' property for further details on how it works.No TrackingnoneThis strategy lists an entity without any tracking. The same entity will be listed each time on executing this processor. It is recommended to change the default run schedule value. Any property that related to the persisting state will be disregarded.Time Windowtime-windowThis strategy uses a sliding time window. The window starts where the previous window ended and ends with the 'current time'. One cycle will list files with modification time falling within the time window. Works even when multiple subdirectories are being written at the same time while listing is running. IMPORTANT: This strategy works properly only if the time on both the system hosting NiFi and the one hosting the files are accurate.truefalsefalseNONEfalsefalseHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueENVIRONMENTfalsefalsePortPortThe port to connect to on the remote host to fetch the data from21truefalsetrueENVIRONMENTfalsefalseUsernameUsernameUsernametruefalsetrueENVIRONMENTfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push files.falsefalsetrueENVIRONMENTfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseDistributed Cache ServiceDistributed Cache ServiceNOTE: This property is used merely for migration from old NiFi version before state management was introduced at version 0.5.0. The stored value in the cache service will be migrated into the state when this processor is started at the first time. The specified Controller Service was used to maintain state about what had been pulled from the remote server so that if a new node begins pulling data, it won't duplicate all of the work that has been done. If not specified, the information was not shared across the cluster. This property did not need to be set for standalone instances of NiFi but was supposed to be configured if NiFi had been running within a cluster.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseSearch RecursivelySearch RecursivelyIf true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefollow-symlinkFollow symlinkIf true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseFile Filter RegexFile Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetchedfalsefalsefalseNONEfalsefalsePath Filter RegexPath Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scannedfalsefalsefalseNONEfalsefalseIgnore Dotted FilesIgnore Dotted FilesIf true, files whose names begin with a dot (".") will be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseRemote Poll Batch SizeRemote Poll Batch SizeThe value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.5000truefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseConnection ModeConnection ModeThe FTP Connection ModePassiveActiveActivePassivePassivefalsefalsefalseNONEfalsefalseTransfer ModeTransfer ModeThe FTP Transfer ModeBinaryBinaryBinaryASCIIASCIIfalsefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseInternal Buffer SizeInternal Buffer SizeSet the internal buffer size for buffered data streams16KBfalsefalsefalseNONEfalsefalsetarget-system-timestamp-precisionTarget System Timestamp PrecisionSpecify timestamp precision at the target system. Since this processor uses timestamp of entities to decide which should be listed, it is crucial to use the right timestamp precision.auto-detectAuto Detectauto-detectAutomatically detect time unit deterministically based on candidate entries timestamp. Please note that this option may take longer to list entities unnecessarily, if none of entries has a precise precision timestamp. E.g. even if a target system supports millis, if all entries only have timestamps without millis, such as '2017-06-16 09:06:34.000', then its precision is determined as 'seconds'.MillisecondsmillisThis option provides the minimum latency for an entry from being available to being listed if target system supports millis, if not, use other options.SecondssecondsFor a target system that does not have millis precision, but has in seconds.MinutesminutesFor a target system that only supports precision in minutes.truefalsefalseNONEfalsefalseet-state-cacheEntity Tracking State CacheListed entities are stored in the specified cache storage so that this processor can resume listing across NiFi restart or in case of primary node change. 'Tracking Entities' strategy require tracking information of all listed entities within the last 'Tracking Time Window'. To support large number of entities, the strategy uses DistributedMapCache instead of managed state. Cache key format is 'ListedEntities::{processorId}(::{nodeId})'. If it tracks per node listed entities, then the optional '::{nodeId}' part is added to manage state separately. E.g. cluster wide cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b', per node cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b::nifi-node3' The stored cache content is Gzipped JSON string. The cache key will be deleted when target listing configuration is changed. Used by 'Tracking Entities' strategy.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseet-time-windowEntity Tracking Time WindowSpecify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile is emitted if one of following condition meets: 1. does not exist in the already-listed entities, 2. has newer timestamp than the cached entity, 3. has different size than the cached entity. If a cached entity's timestamp becomes older than specified time window, that entity will be removed from the cached already-listed entities. Used by 'Tracking Entities' strategy.3 hoursfalsefalsetrueENVIRONMENTfalsefalseet-initial-listing-targetEntity Tracking Initial Listing TargetSpecify how initial listing should be handled. Used by 'Tracking Entities' strategy.allTracking Time WindowwindowIgnore entities having timestamp older than the specified 'Tracking Time Window' at the initial listing activity.All AvailableallRegardless of entities timestamp, all existing entities will be listed at the initial listing activity.falsefalsefalseNONEfalsefalseftp-use-utf8Use UTF-8 EncodingTells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalseftp.remote.hostThe hostname of the FTP Serverftp.remote.portThe port that was connected to on the FTP Serverftp.listing.userThe username of the user that performed the FTP Listingfile.ownerThe numeric owner id of the source filefile.groupThe numeric group id of the source filefile.permissionsThe read/write/execute permissions of the source filefile.sizeThe number of bytes in the source filefile.lastModifiedTimeThe timestamp of when the file in the filesystem waslast modified as 'yyyy-MM-dd'T'HH:mm:ssZ'filenameThe name of the file on the FTP ServerpathThe fully qualified name of the directory on the FTP Server from which the file was pulledtruetrueAfter performing a listing of files, the timestamp of the newest file is stored. This allows the Processor to list only files that have been added or modified after this date the next time that the Processor is run. State is stored across the cluster so that this Processor can be run on Primary Node only and if a new Primary Node is selected, the new node will not duplicate the data that was listed by the previous Primary Node.CLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.standard.FetchFTPorg.apache.nifi.processors.standard.GetFTPorg.apache.nifi.processors.standard.PutFTPTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.ListFilePROCESSORRetrieves a listing of files from the input directory. For each file listed, creates a FlowFile that represents the file so that it can be fetched in conjunction with FetchFile. This Processor is designed to run on Primary Node only in a cluster when 'Input Directory Location' is set to 'Remote'. If the primary node changes, the new Primary Node will pick up where the previous node left off without duplicating all the data. When 'Input Directory Location' is 'Local', the 'Execution' mode can be anything, and synchronization won't happen. Unlike GetFile, this Processor does not delete any data from the local filesystem.filegetlistingestsourcefilesystemInput DirectoryInput DirectoryThe input directory from which files to pull filestruefalsetrueENVIRONMENTfalsefalselisting-strategyListing StrategySpecify how to determine new/updated entities. See each strategy descriptions for detail.timestampsTracking TimestampstimestampsThis strategy tracks the latest timestamp of listed entity to determine new/updated entities. Since it only tracks few timestamps, it can manage listing state efficiently. However, any newly added, or updated entity having timestamp older than the tracked latest timestamp can not be picked by this strategy. For example, such situation can happen in a file system if a file with old timestamp is copied or moved into the target directory without its last modified timestamp being updated. Also may miss files when multiple subdirectories are being written at the same time while listing is running.Tracking EntitiesentitiesThis strategy tracks information of all the listed entities within the latest 'Entity Tracking Time Window' to determine new/updated entities. This strategy can pick entities having old timestamp that can be missed with 'Tracking Timestamps'. Works even when multiple subdirectories are being written at the same time while listing is running. However additional DistributedMapCache controller service is required and more JVM heap memory is used. See the description of 'Entity Tracking Time Window' property for further details on how it works.No TrackingnoneThis strategy lists an entity without any tracking. The same entity will be listed each time on executing this processor. It is recommended to change the default run schedule value. Any property that related to the persisting state will be disregarded.truefalsefalseNONEfalsefalseRecurse SubdirectoriesRecurse SubdirectoriesIndicates whether to list files from subdirectories of the directorytruetruetruefalsefalsetruefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseInput Directory LocationInput Directory LocationSpecifies where the Input Directory is located. This is used to determine whether state should be stored locally or across the cluster.LocalLocalLocalInput Directory is located on a local disk. State will be stored locally on each node in the cluster.RemoteRemoteInput Directory is located on a remote system. State will be stored across the cluster so that the listing can be performed on Primary Node Only and another node can pick up where the last node left off, if the Primary Node changestruefalsefalseNONEfalsefalseFile FilterFile FilterOnly files whose names match the given regular expression will be picked up[^\.].*truefalsefalseNONEfalsefalsePath FilterPath FilterWhen Recurse Subdirectories is true, then only subdirectories whose path matches the given regular expression will be scannedfalsefalsefalseNONEfalsefalseInclude File AttributesInclude File AttributesWhether or not to include information such as the file's Last Modified Time and Owner as FlowFile Attributes. Depending on the File System being used, gathering this information can be expensive and as a result should be disabled. This is especially true of remote file shares.truetruetruefalsefalsetruefalsefalseNONEfalsefalseMinimum File AgeMinimum File AgeThe minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored0 sectruefalsefalseNONEfalsefalseMaximum File AgeMaximum File AgeThe maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignoredfalsefalsefalseNONEfalsefalseMinimum File SizeMinimum File SizeThe minimum size that a file must be in order to be pulled0 BtruefalsefalseNONEfalsefalseMaximum File SizeMaximum File SizeThe maximum size that a file can be in order to be pulledfalsefalsefalseNONEfalsefalseIgnore Hidden FilesIgnore Hidden FilesIndicates whether or not hidden files should be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalsetarget-system-timestamp-precisionTarget System Timestamp PrecisionSpecify timestamp precision at the target system. Since this processor uses timestamp of entities to decide which should be listed, it is crucial to use the right timestamp precision.auto-detectAuto Detectauto-detectAutomatically detect time unit deterministically based on candidate entries timestamp. Please note that this option may take longer to list entities unnecessarily, if none of entries has a precise precision timestamp. E.g. even if a target system supports millis, if all entries only have timestamps without millis, such as '2017-06-16 09:06:34.000', then its precision is determined as 'seconds'.MillisecondsmillisThis option provides the minimum latency for an entry from being available to being listed if target system supports millis, if not, use other options.SecondssecondsFor a target system that does not have millis precision, but has in seconds.MinutesminutesFor a target system that only supports precision in minutes.truefalsefalseNONEfalsefalseet-state-cacheEntity Tracking State CacheListed entities are stored in the specified cache storage so that this processor can resume listing across NiFi restart or in case of primary node change. 'Tracking Entities' strategy require tracking information of all listed entities within the last 'Tracking Time Window'. To support large number of entities, the strategy uses DistributedMapCache instead of managed state. Cache key format is 'ListedEntities::{processorId}(::{nodeId})'. If it tracks per node listed entities, then the optional '::{nodeId}' part is added to manage state separately. E.g. cluster wide cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b', per node cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b::nifi-node3' The stored cache content is Gzipped JSON string. The cache key will be deleted when target listing configuration is changed. Used by 'Tracking Entities' strategy.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseet-time-windowEntity Tracking Time WindowSpecify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile is emitted if one of following condition meets: 1. does not exist in the already-listed entities, 2. has newer timestamp than the cached entity, 3. has different size than the cached entity. If a cached entity's timestamp becomes older than specified time window, that entity will be removed from the cached already-listed entities. Used by 'Tracking Entities' strategy.3 hoursfalsefalsetrueENVIRONMENTfalsefalseet-initial-listing-targetEntity Tracking Initial Listing TargetSpecify how initial listing should be handled. Used by 'Tracking Entities' strategy.allTracking Time WindowwindowIgnore entities having timestamp older than the specified 'Tracking Time Window' at the initial listing activity.All AvailableallRegardless of entities timestamp, all existing entities will be listed at the initial listing activity.falsefalsefalseNONEfalsefalseet-node-identifierEntity Tracking Node IdentifierThe configured value will be appended to the cache key so that listing state can be tracked per NiFi node rather than cluster wide when tracking state is scoped to LOCAL. Used by 'Tracking Entities' strategy.${hostname()}falsefalsetrueENVIRONMENTfalsefalsetrack-performanceTrack PerformanceWhether or not the Processor should track the performance of disk access operations. If true, all accesses to disk will be recorded, including the file being accessed, the information being obtained, and how long it takes. This is then logged periodically at a DEBUG level. While the amount of data will be capped, this option may still consume a significant amount of heap (controlled by the 'Maximum Number of Files to Track' property), but it can be very useful for troubleshooting purposes if performance is poor is degraded.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsemax-performance-metricsMaximum Number of Files to TrackIf the 'Track Performance' property is set to 'true', this property indicates the maximum number of files whose performance metrics should be held onto. A smaller value for this property will result in less heap utilization, while a larger value may provide more accurate insights into how the disk access operations are performing100000truefalsetrueENVIRONMENTfalsefalsemax-operation-timeMax Disk Operation TimeThe maximum amount of time that any single disk operation is expected to take. If any disk operation takes longer than this amount of time, a warning bulletin will be generated for each operation that exceeds this amount of time.10 secsfalsefalsetrueENVIRONMENTfalsefalsemax-listing-timeMax Directory Listing TimeThe maximum amount of time that listing any single directory is expected to take. If the listing for the directory specified by the 'Input Directory' property, or the listing of any subdirectory (if 'Recurse' is set to true) takes longer than this amount of time, a warning bulletin will be generated for each directory listing that exceeds this amount of time.3 minsfalsefalsetrueENVIRONMENTfalsefalsesuccessAll FlowFiles that are received are routed to successfalsefilenameThe name of the file that was read from filesystem.pathThe path is set to the relative path of the file's directory on filesystem compared to the Input Directory property. For example, if Input Directory is set to /tmp, then files picked up from /tmp will have the path attribute set to "/". If the Recurse Subdirectories property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to "abc/1/2/3/".absolute.pathThe absolute.path is set to the absolute path of the file's directory on filesystem. For example, if the Input Directory property is set to /tmp, then files picked up from /tmp will have the path attribute set to "/tmp/". If the Recurse Subdirectories property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to "/tmp/abc/1/2/3/".file.ownerThe user that owns the file in filesystemfile.groupThe group that owns the file in filesystemfile.sizeThe number of bytes in the file in filesystemfile.permissionsThe permissions for the file in filesystem. This is formatted as 3 characters for the owner, 3 for the group, and 3 for other users. For example rw-rw-r--file.lastModifiedTimeThe timestamp of when the file in filesystem was last modified as 'yyyy-MM-dd'T'HH:mm:ssZ'file.lastAccessTimeThe timestamp of when the file in filesystem was last accessed as 'yyyy-MM-dd'T'HH:mm:ssZ'file.creationTimeThe timestamp of when the file in filesystem was created as 'yyyy-MM-dd'T'HH:mm:ssZ'trueAfter performing a listing of files, the timestamp of the newest file is stored. This allows the Processor to list only files that have been added or modified after this date the next time that the Processor is run. Whether the state is stored with a Local or Cluster scope depends on the value of the <Input Directory Location> property.LOCALCLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.standard.GetFileorg.apache.nifi.processors.standard.PutFileorg.apache.nifi.processors.standard.FetchFileTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.ListSFTPPROCESSORPerforms a listing of the files residing on an SFTP server. For each file that is found on the remote server, a new FlowFile will be created with the filename attribute set to the name of the file on the remote server. This can then be used in conjunction with FetchSFTP in order to fetch those files.listsftpremoteingestsourceinputfileslisting-strategyListing StrategySpecify how to determine new/updated entities. See each strategy descriptions for detail.timestampsTracking TimestampstimestampsThis strategy tracks the latest timestamp of listed entity to determine new/updated entities. Since it only tracks few timestamps, it can manage listing state efficiently. However, any newly added, or updated entity having timestamp older than the tracked latest timestamp can not be picked by this strategy. For example, such situation can happen in a file system if a file with old timestamp is copied or moved into the target directory without its last modified timestamp being updated. Also may miss files when multiple subdirectories are being written at the same time while listing is running.Tracking EntitiesentitiesThis strategy tracks information of all the listed entities within the latest 'Entity Tracking Time Window' to determine new/updated entities. This strategy can pick entities having old timestamp that can be missed with 'Tracking Timestamps'. Works even when multiple subdirectories are being written at the same time while listing is running. However additional DistributedMapCache controller service is required and more JVM heap memory is used. See the description of 'Entity Tracking Time Window' property for further details on how it works.No TrackingnoneThis strategy lists an entity without any tracking. The same entity will be listed each time on executing this processor. It is recommended to change the default run schedule value. Any property that related to the persisting state will be disregarded.Time Windowtime-windowThis strategy uses a sliding time window. The window starts where the previous window ended and ends with the 'current time'. One cycle will list files with modification time falling within the time window. Works even when multiple subdirectories are being written at the same time while listing is running. IMPORTANT: This strategy works properly only if the time on both the system hosting NiFi and the one hosting the files are accurate.truefalsefalseNONEfalsefalseHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers22truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalsePrivate Key PathPrivate Key PathThe fully qualified path to the Private Key filefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILEPrivate Key PassphrasePrivate Key PassphrasePassword for the private keyfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push files.falsefalsetrueENVIRONMENTfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseDistributed Cache ServiceDistributed Cache ServiceNOTE: This property is used merely for migration from old NiFi version before state management was introduced at version 0.5.0. The stored value in the cache service will be migrated into the state when this processor is started at the first time. The specified Controller Service was used to maintain state about what had been pulled from the remote server so that if a new node begins pulling data, it won't duplicate all of the work that has been done. If not specified, the information was not shared across the cluster. This property did not need to be set for standalone instances of NiFi but was supposed to be configured if NiFi had been running within a cluster.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseSearch RecursivelySearch RecursivelyIf true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefollow-symlinkFollow symlinkIf true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseFile Filter RegexFile Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetchedfalsefalsefalseNONEfalsefalsePath Filter RegexPath Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scannedfalsefalsefalseNONEfalsefalseIgnore Dotted FilesIgnore Dotted FilesIf true, files whose names begin with a dot (".") will be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseRemote Poll Batch SizeRemote Poll Batch SizeThe value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.5000truefalsefalseNONEfalsefalseStrict Host Key CheckingStrict Host Key CheckingIndicates whether or not strict enforcement of hosts keys should be appliedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseHost Key FileHost Key FileIf supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be usedfalsefalsefalseNONEfalsefalseSINGLEFILEConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseSend Keep Alive On TimeoutSend Keep Alive On TimeoutSend a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.truetruetruefalsefalsetruefalsefalseNONEfalsefalsetarget-system-timestamp-precisionTarget System Timestamp PrecisionSpecify timestamp precision at the target system. Since this processor uses timestamp of entities to decide which should be listed, it is crucial to use the right timestamp precision.auto-detectAuto Detectauto-detectAutomatically detect time unit deterministically based on candidate entries timestamp. Please note that this option may take longer to list entities unnecessarily, if none of entries has a precise precision timestamp. E.g. even if a target system supports millis, if all entries only have timestamps without millis, such as '2017-06-16 09:06:34.000', then its precision is determined as 'seconds'.MillisecondsmillisThis option provides the minimum latency for an entry from being available to being listed if target system supports millis, if not, use other options.SecondssecondsFor a target system that does not have millis precision, but has in seconds.MinutesminutesFor a target system that only supports precision in minutes.truefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseet-state-cacheEntity Tracking State CacheListed entities are stored in the specified cache storage so that this processor can resume listing across NiFi restart or in case of primary node change. 'Tracking Entities' strategy require tracking information of all listed entities within the last 'Tracking Time Window'. To support large number of entities, the strategy uses DistributedMapCache instead of managed state. Cache key format is 'ListedEntities::{processorId}(::{nodeId})'. If it tracks per node listed entities, then the optional '::{nodeId}' part is added to manage state separately. E.g. cluster wide cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b', per node cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b::nifi-node3' The stored cache content is Gzipped JSON string. The cache key will be deleted when target listing configuration is changed. Used by 'Tracking Entities' strategy.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseet-time-windowEntity Tracking Time WindowSpecify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile is emitted if one of following condition meets: 1. does not exist in the already-listed entities, 2. has newer timestamp than the cached entity, 3. has different size than the cached entity. If a cached entity's timestamp becomes older than specified time window, that entity will be removed from the cached already-listed entities. Used by 'Tracking Entities' strategy.3 hoursfalsefalsetrueENVIRONMENTfalsefalseet-initial-listing-targetEntity Tracking Initial Listing TargetSpecify how initial listing should be handled. Used by 'Tracking Entities' strategy.allTracking Time WindowwindowIgnore entities having timestamp older than the specified 'Tracking Time Window' at the initial listing activity.All AvailableallRegardless of entities timestamp, all existing entities will be listed at the initial listing activity.falsefalsefalseNONEfalsefalseMinimum File AgeMinimum File AgeThe minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored0 sectruefalsefalseNONEfalsefalseMaximum File AgeMaximum File AgeThe maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignoredfalsefalsefalseNONEfalsefalseMinimum File SizeMinimum File SizeThe minimum size that a file must be in order to be pulled0 BtruefalsefalseNONEfalsefalseMaximum File SizeMaximum File SizeThe maximum size that a file can be in order to be pulledfalsefalsefalseNONEfalsefalseCiphers AllowedCiphers AllowedA comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctrfalsefalsetrueENVIRONMENTfalsefalseKey Algorithms AllowedKey Algorithms AllowedA comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.comfalsefalsetrueENVIRONMENTfalsefalseKey Exchange Algorithms AllowedKey Exchange Algorithms AllowedA comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-cfalsefalsetrueENVIRONMENTfalsefalseMessage Authentication Codes AllowedMessage Authentication Codes AllowedA comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.comfalsefalsetrueENVIRONMENTfalsefalsesuccessAll FlowFiles that are received are routed to successfalsesftp.remote.hostThe hostname of the SFTP Serversftp.remote.portThe port that was connected to on the SFTP Serversftp.listing.userThe username of the user that performed the SFTP Listingfile.ownerThe numeric owner id of the source filefile.groupThe numeric group id of the source filefile.permissionsThe read/write/execute permissions of the source filefile.sizeThe number of bytes in the source filefile.lastModifiedTimeThe timestamp of when the file in the filesystem waslast modified as 'yyyy-MM-dd'T'HH:mm:ssZ'filenameThe name of the file on the SFTP ServerpathThe fully qualified name of the directory on the SFTP Server from which the file was pulledmime.typeThe MIME Type that is provided by the configured Record WritertruetrueAfter performing a listing of files, the timestamp of the newest file is stored. This allows the Processor to list only files that have been added or modified after this date the next time that the Processor is run. State is stored across the cluster so that this Processor can be run on Primary Node only and if a new Primary Node is selected, the new node will not duplicate the data that was listed by the previous Primary Node.CLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.standard.FetchSFTPorg.apache.nifi.processors.standard.GetSFTPorg.apache.nifi.processors.standard.PutSFTPTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.ListenFTPPROCESSORStarts an FTP server that listens on the specified port and transforms incoming files into FlowFiles. The URI of the service will be ftp://{hostname}:{port}. The default port is 2221.ingestFTPFTPSlistenbind-addressBind AddressThe address the FTP server should be bound to. If not set (or set to 0.0.0.0), the server binds to all available addresses (i.e. all network interfaces of the host machine).falsefalsetrueENVIRONMENTfalsefalselistening-portListening PortThe Port to listen on for incoming connections. On Linux, root privileges are required to use port numbers below 1024.2221truefalsetrueENVIRONMENTfalsefalseusernameUsernameThe name of the user that is allowed to log in to the FTP server. If a username is provided, a password must also be provided. If no username is specified, anonymous connections will be permitted.falsefalsetrueENVIRONMENTfalsefalsepasswordPasswordIf the Username is set, then a password must also be specified. The password provided by the client trying to log in to the FTP server will be checked against this password.falsetruetrueENVIRONMENTfalsefalsessl-context-serviceSSL Context ServiceSpecifies the SSL Context Service that can be used to create secure connections. If an SSL Context Service is selected, then a keystore file must also be specified in the SSL Context Service. Without a keystore file, the processor cannot be started successfully.Specifying a truststore file is optional. If a truststore file is specified, client authentication is required (the client needs to send a certificate to the server).Regardless of the selected TLS protocol, the highest available protocol is used for the connection. For example if NiFi is running on Java 11 and TLSv1.2 is selected in the controller service as the preferred TLS Protocol, TLSv1.3 will be used (regardless of TLSv1.2 being selected) because Java 11 supports TLSv1.3.org.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalsesuccessRelationship for successfully received files.falsefilenameThe name of the file received via the FTP/FTPS connection.pathThe path pointing to the file's target directory. E.g.: file.txt is uploaded to /Folder1/SubFolder, then the value of the path attribute will be "/Folder1/SubFolder/" (note that it ends with a separator character).INPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenHTTPPROCESSORStarts an HTTP Server and listens on a given base path to transform incoming requests into FlowFiles. The default URI of the Service will be http://{hostname}:{port}/contentListener. Only HEAD and POST requests are supported. GET, PUT, DELETE, OPTIONS and TRACE will result in an error and the HTTP response status code 405; CONNECT will also result in an error and the HTTP response status code 400. GET is supported on <service_URI>/healthcheck. If the service is available, it returns "200 OK" with the content "OK". The health check functionality can be configured to be accessible via a different port. For details see the documentation of the "Listening Port for health check requests" property.A Record Reader and Record Writer property can be enabled on the processor to process incoming requests as records. Record processing is not allowed for multipart requests and request in FlowFileV3 format (minifi).ingesthttphttpsrestlistenBase PathBase PathBase path for incoming connectionscontentListenertruefalsetrueENVIRONMENTfalsefalseListening PortListening PortThe Port to listen on for incoming connectionstruefalsetrueENVIRONMENTfalsefalsehealth-check-portListening Port for Health Check RequestsThe port to listen on for incoming health check requests. If set, it must be different from the Listening Port. Configure this port if the processor is set to use two-way SSL and a load balancer that does not support client authentication for health check requests is used. Only /<base_path>/healthcheck service is available via this port and only GET and HEAD requests are supported. If the processor is set not to use SSL, SSL will not be used on this port, either. If the processor is set to use one-way SSL, one-way SSL will be used on this port. If the processor is set to use two-way SSL, one-way SSL will be used on this port (client authentication not required).falsefalsetrueENVIRONMENTfalsefalseMax Data to Receive per SecondMax Data to Receive per SecondThe maximum amount of data to receive per second; this allows the bandwidth to be throttled to a specified data rate; if not specified, the data rate is not throttledfalsefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceSSL Context Service enables support for HTTPSorg.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseHTTP ProtocolsHTTP ProtocolsHTTP Protocols supported for Application Layer Protocol Negotiation with TLSHTTP_1_1http/1.1HTTP_1_1HTTP/1.1h2 http/1.1H2_HTTP_1_1HTTP/2 and HTTP/1.1 negotiated based on requested protocolsh2H2HTTP/2truefalsefalseNONEfalsefalseSSL Context ServiceSSL Context Serviceclient-authenticationClient AuthenticationClient Authentication policy for TLS connections. Required when SSL Context Service configured.AUTOAUTOAUTOInferred based on SSL Context Service properties. The presence of Trust Store properties implies REQUIRED, otherwise NONE is configured.WANTWANTRequests the client certificate on handshake and validates if present but does not require itREQUIREDREQUIREDRequests the client certificate on handshake and rejects the connection if it is not present and validNONENONEDoes not request the client certificate on handshakefalsefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceAuthorized DN PatternAuthorized Subject DN PatternA Regular Expression to apply against the Subject's Distinguished Name of incoming connections. If the Pattern does not match the Subject DN, the the processor will respond with a status of HTTP 403 Forbidden..*truefalsefalseNONEfalsefalseauthorized-issuer-dn-patternAuthorized Issuer DN PatternA Regular Expression to apply against the Issuer's Distinguished Name of incoming connections. If the Pattern does not match the Issuer DN, the processor will respond with a status of HTTP 403 Forbidden..*falsefalsefalseNONEfalsefalseMax Unconfirmed Flowfile TimeMax Unconfirmed Flowfile TimeThe maximum amount of time to wait for a FlowFile to be confirmed before it is removed from the cache60 secstruefalsefalseNONEfalsefalseHTTP Headers to receive as Attributes (Regex)HTTP Headers to receive as Attributes (Regex)Specifies the Regular Expression that determines the names of HTTP Headers that should be passed along as FlowFile attributesfalsefalsefalseNONEfalsefalseReturn CodeReturn CodeThe HTTP return code returned after every HTTP call200falsefalsefalseNONEfalsefalsemultipart-request-max-sizeMultipart Request Max SizeThe max size of the request. Only applies for requests with Content-Type: multipart/form-data, and is used to prevent denial of service type of attacks, to prevent filling up the heap or disk space1 MBtruefalsefalseNONEfalsefalsemultipart-read-buffer-sizeMultipart Read Buffer SizeThe threshold size, at which the contents of an incoming file would be written to disk. Only applies for requests with Content-Type: multipart/form-data. It is used to prevent denial of service type of attacks, to prevent filling up the heap or disk space.512 KBtruefalsefalseNONEfalsefalsemax-thread-pool-sizeMaximum Thread Pool SizeThe maximum number of threads to be used by the embedded Jetty server. The value can be set between 8 and 1000. The value of this property affects the performance of the flows and the operating system, therefore the default value should only be changed in justified cases. A value that is less than the default value may be suitable if only a small number of HTTP clients connect to the server. A greater value may be suitable if a large number of HTTP clients are expected to make requests to the server simultaneously.200truefalsefalseNONEfalsefalserecord-readerRecord ReaderThe Record Reader to use parsing the incoming FlowFile into Recordsorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalserecord-writerRecord WriterThe Record Writer to use for serializing Records after they have been transformedorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-readerRecord ReadersuccessRelationship for successfully received FlowFilesfalseINPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenRELPPROCESSORListens for RELP messages being sent to a given port over TCP. Each message will be acknowledged after successfully writing the message to a FlowFile. Each FlowFile will contain data portion of one or more RELP frames. In the case where the RELP frames contain syslog messages, the output of this processor can be sent to a ParseSyslog processor for further processing.listenrelptcplogsLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsePortPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalseReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor during these surges.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the received data.UTF-8truefalsefalseNONEfalsefalseMax Number of TCP ConnectionsMax Number of Worker ThreadsThe maximum number of worker threads available for servicing TCP connections.2truefalsefalseNONEfalsefalseMax Batch SizeMax Batch SizeThe maximum number of messages to add to a single FlowFile. If multiple messages are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages1truefalsefalseNONEfalsefalseMessage DelimiterBatching Message DelimiterSpecifies the delimiter to place between messages when multiple messages are bundled together (see <Max Batch Size> property).\ntruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection.org.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseClient AuthClient AuthThe client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.REQUIREDWANTWANTREQUIREDREQUIREDNONENONEfalsefalsefalseNONEfalsefalsesuccessMessages received successfully will be sent out this relationship.falserelp.commandThe command of the RELP frames.relp.senderThe sending host of the messages.relp.portThe sending port the messages were received over.relp.txnrThe transaction number of the message. Only included if <Batch Size> is 1.mime.typeThe mime.type of the content which is text/plainINPUT_FORBIDDENorg.apache.nifi.processors.standard.ParseSyslogorg.apache.nifi.processors.standard.ListenSyslogPROCESSORListens for Syslog messages being sent to a given port over TCP or UDP. Incoming messages are checked against regular expressions for RFC5424 and RFC3164 formatted messages. The format of each message is: (<PRIORITY>)(VERSION )(TIMESTAMP) (HOSTNAME) (BODY) where version is optional. The timestamp can be an RFC5424 timestamp with a format of "yyyy-MM-dd'T'HH:mm:ss.SZ" or "yyyy-MM-dd'T'HH:mm:ss.S+hh:mm", or it can be an RFC3164 timestamp with a format of "MMM d HH:mm:ss". If an incoming messages matches one of these patterns, the message will be parsed and the individual pieces will be placed in FlowFile attributes, with the original message in the content of the FlowFile. If an incoming message does not match one of these patterns it will not be parsed and the syslog.valid attribute will be set to false with the original message in the content of the FlowFile. Valid messages will be transferred on the success relationship, and invalid messages will be transferred on the invalid relationship.sysloglistenudptcplogsProtocolProtocolThe protocol for Syslog communication.UDPTCPTCPUDPUDPtruefalsefalseNONEfalsefalsePortPortThe port for Syslog communication. Note that Expression language is not evaluated per FlowFile.truefalsetrueENVIRONMENTfalsefalseLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsesocket-keep-aliveSocket Keep AliveWhether or not to have TCP socket keep alive turned on. Timing details depend on operating system properties.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseProtocolProtocolTCPSSL Context ServiceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, syslog messages will be received over a secure connection.org.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProtocolProtocolTCPClient AuthClient AuthThe client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.REQUIREDWANTWANTREQUIREDREQUIREDNONENONEfalsefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive Syslog messages. Adjust this value appropriately based on the expected size of the incoming Syslog messages. When UDP is selected each buffer will hold one Syslog message. When TCP is selected messages are read from an incoming connection until the buffer is full, or the connection is closed. 65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseProtocolProtocolTCPMax Number of TCP ConnectionsMax Number of TCP ConnectionsThe maximum number of concurrent connections to accept Syslog messages in TCP mode.2truefalsefalseNONEfalsefalseProtocolProtocolTCPMax Batch SizeMax Batch SizeThe maximum number of Syslog events to add to a single FlowFile. If multiple events are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages1truefalsefalseNONEfalsefalseMessage DelimiterMessage DelimiterSpecifies the delimiter to place between Syslog messages when multiple messages are bundled together (see <Max Batch Size> property).\ntruefalsefalseNONEfalsefalseParse MessagesParse MessagesIndicates if the processor should parse the Syslog messages. If set to false, each outgoing FlowFile will only contain the sender, protocol, and port, and no additional attributes.truetruetruefalsefalsetruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the Syslog messages. Note that Expression language is not evaluated per FlowFile.UTF-8truefalsetrueENVIRONMENTfalsefalsesuccessSyslog messages that match one of the expected formats will be sent out this relationship as a FlowFile per message.falseinvalidSyslog messages that do not match one of the expected formats will be sent out this relationship as a FlowFile per message.falsesyslog.priorityThe priority of the Syslog message.syslog.severityThe severity of the Syslog message derived from the priority.syslog.facilityThe facility of the Syslog message derived from the priority.syslog.versionThe optional version from the Syslog message.syslog.timestampThe timestamp of the Syslog message.syslog.hostnameThe hostname or IP address of the Syslog message.syslog.senderThe hostname of the Syslog server that sent the message.syslog.bodyThe body of the Syslog message, everything after the hostname.syslog.validAn indicator of whether this message matched the expected formats. If this value is false, the other attributes will be empty and only the original message will be available in the content.syslog.protocolThe protocol over which the Syslog message was received.syslog.portThe port over which the Syslog message was received.mime.typeThe mime.type of the FlowFile which will be text/plain for Syslog messages.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.PutSyslogorg.apache.nifi.processors.standard.ParseSyslogorg.apache.nifi.processors.standard.ListenTCPPROCESSORListens for incoming TCP connections and reads data from each connection using a line separator as the message demarcator. The default behavior is for each message to produce a single FlowFile, however this can be controlled by increasing the Batch Size to a larger value for higher throughput. The Receive Buffer Size must be set as large as the largest messages expected to be received, meaning if every 100kb there is a line separator, then the Receive Buffer Size must be greater than 100kb. The processor can be configured to use an SSL Context Service to only allow secure connections. When connected clients present certificates for mutual TLS authentication, the Distinguished Names of the client certificate's issuer and subject are added to the outgoing FlowFiles as attributes. The processor does not perform authorization based on Distinguished Name values, but since these values are attached to the outgoing FlowFiles, authorization can be implemented based on these attributes.listentcptlssslLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsePortPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalseReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor during these surges.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the received data.UTF-8truefalsefalseNONEfalsefalseMax Number of TCP ConnectionsMax Number of Worker ThreadsThe maximum number of worker threads available for servicing TCP connections.2truefalsefalseNONEfalsefalseMax Batch SizeMax Batch SizeThe maximum number of messages to add to a single FlowFile. If multiple messages are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages1truefalsefalseNONEfalsefalseMessage DelimiterBatching Message DelimiterSpecifies the delimiter to place between messages when multiple messages are bundled together (see <Max Batch Size> property).\ntruefalsefalseNONEfalsefalseidle-timeoutIdle Connection TimeoutThe amount of time a client's connection will remain open if no data is received. The default of 0 seconds will leave connections open until they are closed by the client.0 secondstruefalsefalseNONEfalsefalsemax-receiving-threadsMax Number of Receiving Message Handler ThreadsThis property is deprecated and no longer used.falsefalsefalseNONEfalsefalsepool-receive-buffersPool Receive BuffersEnable or disable pooling of buffers that the processor uses for handling bytes received on socket connections. The framework allocates buffers as needed during processing.TrueTrueTrueFalseFalsetruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection.org.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseClient AuthClient AuthThe client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.REQUIREDWANTWANTREQUIREDREQUIREDNONENONEfalsefalsefalseNONEfalsefalsesuccessMessages received successfully will be sent out this relationship.falsetcp.senderThe sending host of the messages.tcp.portThe sending port the messages were received.client.certificate.issuer.dnFor connections using mutual TLS, the Distinguished Name of the Certificate Authority that issued the client's certificate is attached to the FlowFile.client.certificate.subject.dnFor connections using mutual TLS, the Distinguished Name of the client certificate's owner (subject) is attached to the FlowFile.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenTCPRecordPROCESSORListens for incoming TCP connections and reads data from each connection using a configured record reader, and writes the records to a flow file using a configured record writer. The type of record reader selected will determine how clients are expected to send data. For example, when using a Grok reader to read logs, a client can keep an open connection and continuously stream data, but when using an JSON reader, the client cannot send an array of JSON documents and then send another array on the same connection, as the reader would be in a bad state at that point. Records will be read from the connection in blocking mode, and will timeout according to the Read Timeout specified in the processor. If the read times out, or if any other error is encountered when reading, the connection will be closed, and any records read up to that point will be handled according to the configured Read Error Strategy (Discard or Transfer). In cases where clients are keeping a connection open, the concurrent tasks for the processor should be adjusted to match the Max Number of TCP Connections allowed, so that there is a task processing each connection. The processor can be configured to use an SSL Context Service to only allow secure connections. When connected clients present certificates for mutual TLS authentication, the Distinguished Names of the client certificate's issuer and subject are added to the outgoing FlowFiles as attributes. The processor does not perform authorization based on Distinguished Name values, but since these values are attached to the outgoing FlowFiles, authorization can be implemented based on these attributes.listentcprecordtlssslLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalseportPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalsemax-size-socket-bufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalsemax-number-tcp-connectionsMax Number of TCP ConnectionsThe maximum number of concurrent TCP connections to accept. In cases where clients are keeping a connection open, the concurrent tasks for the processor should be adjusted to match the Max Number of TCP Connections allowed, so that there is a task processing each connection.2truefalsefalseNONEfalsefalseread-timeoutRead TimeoutThe amount of time to wait before timing out when reading from a connection.10 secondstruefalsefalseNONEfalsefalserecord-readerRecord ReaderThe Record Reader to use for incoming FlowFilesorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterThe Record Writer to use in order to serialize the data before writing to a FlowFileorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsereader-error-handling-strategyRead Error StrategyIndicates how to deal with an error while reading the next record from a connection, when previous records have already been read from the connection.TransferTransferTransferTransfers any records already received and closes the connection.DiscardDiscardDiscards any records already received and closes the connection.truefalsefalseNONEfalsefalserecord-batch-sizeRecord Batch SizeThe maximum number of records to write to a single FlowFile.1000truefalsefalseNONEfalsefalsessl-context-serviceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection.org.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseclient-authClient AuthThe client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.REQUIREDWANTWANTREQUIREDREQUIREDNONENONEfalsefalsefalseNONEfalsefalsesuccessMessages received successfully will be sent out this relationship.falsetcp.senderThe host that sent the data.tcp.portThe port that the processor accepted the connection on.record.countThe number of records written to the flow file.mime.typeThe mime-type of the writer used to write the records to the flow file.client.certificate.issuer.dnFor connections using mutual TLS, the Distinguished Name of the Certificate Authority that issued the client's certificate is attached to the FlowFile.client.certificate.subject.dnFor connections using mutual TLS, the Distinguished Name of the client certificate's owner (subject) is attached to the FlowFile.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenUDPPROCESSORListens for Datagram Packets on a given port. The default behavior produces a FlowFile per datagram, however for higher throughput the Max Batch Size property may be increased to specify the number of datagrams to batch together in a single FlowFile. This processor can be restricted to listening for datagrams from a specific remote host and port by specifying the Sending Host and Sending Host Port properties, otherwise it will listen for datagrams from all hosts and ports.ingestudplistensourceLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsePortPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalseReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the received data.UTF-8truefalsefalseNONEfalsefalseMax Batch SizeMax Batch SizeThe maximum number of messages to add to a single FlowFile. If multiple messages are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages1truefalsefalseNONEfalsefalseMessage DelimiterBatching Message DelimiterSpecifies the delimiter to place between messages when multiple messages are bundled together (see <Max Batch Size> property).\ntruefalsefalseNONEfalsefalseSending HostSending HostIP, or name, of a remote host. Only Datagrams from the specified Sending Host Port and this host will be accepted. Improves Performance. May be a system property or an environment variable.falsefalsetrueENVIRONMENTfalsefalseSending Host PortSending Host PortPort being used by remote host to send Datagrams. Only Datagrams from the specified Sending Host and this port will be accepted. Improves Performance. May be a system property or an environment variable.falsefalsetrueENVIRONMENTfalsefalsesuccessMessages received successfully will be sent out this relationship.falseudp.senderThe sending host of the messages.udp.portThe sending port the messages were received.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenUDPRecordPROCESSORListens for Datagram Packets on a given port and reads the content of each datagram using the configured Record Reader. Each record will then be written to a flow file using the configured Record Writer. This processor can be restricted to listening for datagrams from a specific remote host and port by specifying the Sending Host and Sending Host Port properties, otherwise it will listen for datagrams from all hosts and ports.ingestudplistensourcerecordLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsePortPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalseReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the received data.UTF-8truefalsefalseNONEfalsefalsepoll-timeoutPoll TimeoutThe amount of time to wait when polling the internal queue for more datagrams. If no datagrams are found after waiting for the configured timeout, then the processor will emit whatever records have been obtained up to that point.50 mstruefalsefalseNONEfalsefalsebatch-sizeBatch SizeThe maximum number of datagrams to write as records to a single FlowFile. The Batch Size will only be reached when data is coming in more frequently than the Poll Timeout.1000truefalsefalseNONEfalsefalserecord-readerRecord ReaderThe Record Reader to use for reading the content of incoming datagrams.org.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterThe Record Writer to use in order to serialize the data before writing to a flow file.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsesending-hostSending HostIP, or name, of a remote host. Only Datagrams from the specified Sending Host Port and this host will be accepted. Improves Performance. May be a system property or an environment variable.falsefalsetrueENVIRONMENTfalsefalsesending-host-portSending Host PortPort being used by remote host to send Datagrams. Only Datagrams from the specified Sending Host and this port will be accepted. Improves Performance. May be a system property or an environment variable.falsefalsetrueENVIRONMENTfalsefalsesuccessMessages received successfully will be sent out this relationship.falseparse.failureIf a datagram cannot be parsed using the configured Record Reader, the contents of the message will be routed to this Relationship as its own individual FlowFile.falseudp.senderThe sending host of the messages.udp.portThe sending port the messages were received.record.countThe number of records written to the flow file.mime.typeThe mime-type of the writer used to write the records to the flow file.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.LogAttributePROCESSOREmits attributes of the FlowFile at the specified log levelattributesloggingLog LevelLog LevelThe Log Level to use when logging the AttributesinfotracetracedebugdebuginfoinfowarnwarnerrorerrortruefalsefalseNONEfalsefalseLog PayloadLog PayloadIf true, the FlowFile's payload will be logged, in addition to its attributes; otherwise, just the Attributes will be logged.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseAttributes to LogAttributes to LogA comma-separated list of Attributes to Log. If not specified, all attributes will be logged unless `Attributes to Log by Regular Expression` is modified. There's an AND relationship between the two properties.falsefalsefalseNONEfalsefalseattributes-to-log-regexAttributes to Log by Regular ExpressionA regular expression indicating the Attributes to Log. If not specified, all attributes will be logged unless `Attributes to Log` is modified. There's an AND relationship between the two properties..*falsefalsefalseNONEfalsefalseAttributes to IgnoreAttributes to IgnoreA comma-separated list of Attributes to ignore. If not specified, no attributes will be ignored unless `Attributes to Ignore by Regular Expression` is modified. There's an OR relationship between the two properties.falsefalsefalseNONEfalsefalseattributes-to-ignore-regexAttributes to Ignore by Regular ExpressionA regular expression indicating the Attributes to Ignore. If not specified, no attributes will be ignored unless `Attributes to Ignore` is modified. There's an OR relationship between the two properties.falsefalsefalseNONEfalsefalseLog FlowFile PropertiesLog FlowFile PropertiesSpecifies whether or not to log FlowFile "properties", such as Entry Date, Lineage Start Date, and content sizetruetruetruefalsefalsetruefalsefalseNONEfalsefalseOutput FormatOutput FormatSpecifies the format to use for logging FlowFile attributesLine per AttributeLine per AttributeLine per AttributeEach FlowFile attribute will be logged using a single line for the attribute name and another line for the attribute value. This format is often most advantageous when looking at the attributes of a single FlowFile.Single LineSingle LineAll FlowFile attribute names and values will be logged on a single line. This format is often most advantageous when comparing logs from multiple FlowFiles.truefalsefalseNONEfalsefalseLog prefixLog prefixLog prefix appended to the log lines. It helps to distinguish the output of multiple LogAttribute processors.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsecharacter-setCharacter SetThe name of the CharacterSet to useUTF-8truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessAll FlowFiles are routed to this relationshipfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.LogMessagePROCESSOREmits a log message at the specified log levelattributeslogginglog-levelLog LevelThe Log Level to use when logging the message: [trace, debug, info, warn, error]infotruefalsetrueFLOWFILE_ATTRIBUTESfalsefalselog-prefixLog prefixLog prefix appended to the log lines. It helps to distinguish the output of multiple LogMessage processors.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalselog-messageLog messageThe log message to emitfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessAll FlowFiles are routed to this relationshipfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.LookupAttributePROCESSORLookup attributes from a lookup servicelookupcacheenrichjoinattributesAttribute Expression Languagelookup-serviceLookup ServiceThe lookup service to use for attribute lookupsorg.apache.nifi.lookup.StringLookupServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseinclude-empty-valuesInclude Empty ValuesInclude null or blank values for keys that are null or blanktruetruetruefalsefalsetruefalsefalseNONEfalsefalseThe name of the attribute to add to the FlowFileThe name of the key or property to retrieve from the lookup serviceAdds a FlowFile attribute specified by the dynamic property's key with the value found in the lookup service using the the dynamic property's valueFLOWFILE_ATTRIBUTESfailureFlowFiles with failing lookups are routed to this relationshipfalseunmatchedFlowFiles with missing lookups are routed to this relationshipfalsematchedFlowFiles with matching lookups are routed to this relationshipfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.LookupRecordPROCESSORExtracts one or more fields from a Record and looks up a value for those fields in a LookupService. If a result is returned by the LookupService, that result is optionally added to the Record. In this case, the processor functions as an Enrichment processor. Regardless, the Record is then routed to either the 'matched' relationship or 'unmatched' relationship (if the 'Routing Strategy' property is configured to do so), indicating whether or not a result was returned by the LookupService, allowing the processor to also function as a Routing processor. The "coordinates" to use for looking up a value in the Lookup Service are defined by adding a user-defined property. Each property that is added will have an entry added to a Map, where the name of the property becomes the Map Key and the value returned by the RecordPath becomes the value for that key. If multiple values are returned by the RecordPath, then the Record will be routed to the 'unmatched' relationship (or 'success', depending on the 'Routing Strategy' property's configuration). If one or more fields match the Result RecordPath, all fields that match will be updated. If there is no match in the configured LookupService, then no fields will be updated. I.e., it will not overwrite an existing value in the Record with a null value. Please note, however, that if the results returned by the LookupService are not accounted for in your schema (specifically, the schema that is configured for your Record Writer) then the fields will not be written out to the FlowFile.lookupenrichmentrouterecordcsvjsonavrodatabasedblogsconvertfilterrecord-readerRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalselookup-serviceLookup ServiceThe Lookup Service to use in order to lookup a value in each Recordorg.apache.nifi.lookup.LookupServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseRoot Record PathRoot Record PathA RecordPath that points to a child Record within each of the top-level Records in the FlowFile. If specified, the additional RecordPath properties will be evaluated against this child Record instead of the top-level Record. This allows for performing enrichment against multiple child Records within a single top-level Record.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalserouting-strategyRouting StrategySpecifies how to route records after a Lookup has completedroute-to-successRoute to 'success'route-to-successRecords will be routed to a 'success' Relationship regardless of whether or not there is a match in the configured Lookup ServiceRoute to 'matched' or 'unmatched'route-to-matched-unmatchedRecords will be routed to either a 'matched' or an 'unmatched' Relationship depending on whether or not there was a match in the configured Lookup Service. A single input FlowFile may result in two different output FlowFiles. If the given Record Paths evaluate such that multiple sub-records are evaluated, the parent Record will be routed to 'unmatched' unless all sub-records match.truefalsefalseNONEfalsefalseresult-contentsRecord Result ContentsWhen a result is obtained that contains a Record, this property determines whether the Record itself is inserted at the configured path or if the contents of the Record (i.e., the sub-fields) will be inserted at the configured path.insert-entire-recordInsert Entire Recordinsert-entire-recordThe entire Record that is retrieved from the Lookup Service will be inserted into the destination path.Insert Record Fieldsrecord-fieldsAll of the fields in the Record that is retrieved from the Lookup Service will be inserted into the destination path.truefalsefalseNONEfalsefalserecord-update-strategyRecord Update StrategyThis property defines the strategy to use when updating the record with the value returned by the Lookup Service.use-propertyReplace Existing Valuesreplace-existing-valuesThe "Result RecordPath" property will be ignored and the lookup service must be a single simple key lookup service. Every dynamic property value should be a record path. For each dynamic property, the value contained in the field corresponding to the record path will be used as the key in the Lookup Service and the value returned by the Lookup Service will be used to replace the existing value. It is possible to configure multiple dynamic properties to replace multiple values in one execution. This strategy only supports simple types replacements (strings, integers, etc).Use "Result RecordPath" Propertyuse-propertyThe "Result RecordPath" property will be used to determine which part of the record should be updated with the value returned by the Lookup ServicetruefalsefalseNONEfalsefalseresult-record-pathResult RecordPathA RecordPath that points to the field whose value should be updated with whatever value is returned from the Lookup Service. If not specified, the value that is returned from the Lookup Service will be ignored, except for determining whether the FlowFile should be routed to the 'matched' or 'unmatched' Relationship.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalserecord-update-strategyRecord Update Strategyuse-propertyrecord-path-lookup-miss-result-cache-sizeCache SizeSpecifies how many lookup values/records should be cached.Setting this property to zero means no caching will be done and the table will be queried for each lookup value in each record. If the lookup table changes often or the most recent data must be retrieved, do not use the cache.0truefalsetrueENVIRONMENTfalsefalseValue To LookupValid Record PathA RecordPath that points to the field whose value will be looked up in the configured Lookup ServiceFLOWFILE_ATTRIBUTESsuccessAll records will be sent to this Relationship if configured to do so, unless a failure occursfalsefailureIf a FlowFile cannot be enriched, the unchanged FlowFile will be routed to this relationshipfalsemime.typeSets the mime.type attribute to the MIME Type specified by the Record Writerrecord.countThe number of records in the FlowFiletruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ConvertRecordorg.apache.nifi.processors.standard.SplitRecordorg.apache.nifi.lookup.SimpleKeyValueLookupServiceorg.apache.nifi.lookup.maxmind.IPLookupServiceorg.apache.nifi.lookup.db.DatabaseRecordLookupServiceorg.apache.nifi.processors.standard.MergeContentPROCESSORMerges a Group of FlowFiles together based on a user-defined strategy and packages them into a single FlowFile. It is recommended that the Processor be configured with only a single incoming connection, as Group of FlowFiles will not be created from FlowFiles in different connections. This processor updates the mime.type attribute as appropriate. NOTE: this processor should NOT be configured with Cron Driven for the Scheduling Strategy.mergecontentcorrelationtarzipstreamconcatenationarchiveflowfile-streamflowfile-stream-v3Merge StrategyMerge StrategySpecifies the algorithm used to merge content. The 'Defragment' algorithm combines fragments that are associated by attributes back into a single cohesive FlowFile. The 'Bin-Packing Algorithm' generates a FlowFile populated by arbitrarily chosen FlowFilesBin-Packing AlgorithmBin-Packing AlgorithmBin-Packing AlgorithmGenerates 'bins' of FlowFiles and fills each bin as full as possible. FlowFiles are placed into a bin based on their size and optionally their attributes (if the <Correlation Attribute> property is set)DefragmentDefragmentCombines fragments that are associated by attributes back into a single cohesive FlowFile. If using this strategy, all FlowFiles must have the attributes <fragment.identifier>, <fragment.count>, and <fragment.index>. All FlowFiles with the same value for "fragment.identifier" will be grouped together. All FlowFiles in this group must have the same value for the "fragment.count" attribute. All FlowFiles in this group must have a unique value for the "fragment.index" attribute between 0 and the value of the "fragment.count" attribute.truefalsefalseNONEfalsefalseMerge FormatMerge FormatDetermines the format that will be used to merge the content.Binary ConcatenationTARTARA bin of FlowFiles will be combined into a single TAR file. The FlowFiles' <path> attribute will be used to create a directory in the TAR file if the <Keep Paths> property is set to true; otherwise, all FlowFiles will be added at the root of the TAR file. If a FlowFile has an attribute named <tar.permissions> that is 3 characters, each between 0-7, that attribute will be used as the TAR entry's 'mode'.ZIPZIPA bin of FlowFiles will be combined into a single ZIP file. The FlowFiles' <path> attribute will be used to create a directory in the ZIP file if the <Keep Paths> property is set to true; otherwise, all FlowFiles will be added at the root of the ZIP file. The <Compression Level> property indicates the ZIP compression to use.FlowFile Stream, v3FlowFile Stream, v3A bin of FlowFiles will be combined into a single Version 3 FlowFile StreamFlowFile Stream, v2FlowFile Stream, v2A bin of FlowFiles will be combined into a single Version 2 FlowFile StreamFlowFile Tar, v1FlowFile Tar, v1A bin of FlowFiles will be combined into a single Version 1 FlowFile PackageBinary ConcatenationBinary ConcatenationThe contents of all FlowFiles will be concatenated together into a single FlowFileAvroAvroThe Avro contents of all FlowFiles will be concatenated together into a single FlowFiletruefalsefalseNONEfalsefalseAttribute StrategyAttribute StrategyDetermines which FlowFile attributes should be added to the bundle. If 'Keep All Unique Attributes' is selected, any attribute on any FlowFile that gets bundled will be kept unless its value conflicts with the value from another FlowFile. If 'Keep Only Common Attributes' is selected, only the attributes that exist on all FlowFiles in the bundle, with the same value, will be preserved.Keep Only Common AttributesKeep Only Common AttributesKeep Only Common AttributesAny attribute that is not the same on all FlowFiles in a bin will be dropped. Those that are the same across all FlowFiles will be retained.Keep All Unique AttributesKeep All Unique AttributesAny attribute that has the same value for all FlowFiles in a bin, or has no value for a FlowFile, will be kept. For example, if a bin consists of 3 FlowFiles and 2 of them have a value of 'hello' for the 'greeting' attribute and the third FlowFile has no 'greeting' attribute then the outbound FlowFile will get a 'greeting' attribute with the value 'hello'.truefalsefalseNONEfalsefalseCorrelation Attribute NameCorrelation Attribute NameIf specified, like FlowFiles will be binned together, where 'like FlowFiles' means FlowFiles that have the same value for this Attribute. If not specified, FlowFiles are bundled by the order in which they are pulled from the queue.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMerge StrategyMerge StrategyBin-Packing Algorithmmergecontent-metadata-strategyMetadata StrategyFor FlowFiles whose input format supports metadata (Avro, e.g.), this property determines which metadata should be added to the bundle. If 'Use First Metadata' is selected, the metadata keys/values from the first FlowFile to be bundled will be used. If 'Keep Only Common Metadata' is selected, only the metadata that exists on all FlowFiles in the bundle, with the same value, will be preserved. If 'Ignore Metadata' is selected, no metadata is transferred to the outgoing bundled FlowFile. If 'Do Not Merge Uncommon Metadata' is selected, any FlowFile whose metadata values do not match those of the first bundled FlowFile will not be merged.Do Not Merge Uncommon MetadataUse First MetadataUse First MetadataFor any input format that supports metadata (Avro, e.g.), the metadata for the first FlowFile in the bin will be set on the output FlowFile.Keep Only Common MetadataKeep Only Common MetadataFor any input format that supports metadata (Avro, e.g.), any FlowFile whose metadata values match those of the first FlowFile, any additional metadata will be dropped but the FlowFile will be merged. Any FlowFile whose metadata values do not match those of the first FlowFile in the bin will not be merged.Do Not Merge Uncommon MetadataDo Not Merge Uncommon MetadataFor any input format that supports metadata (Avro, e.g.), any FlowFile whose metadata values do not match those of the first FlowFile in the bin will not be merged.Ignore MetadataIgnore MetadataIgnores (does not transfer, compare, etc.) any metadata from a FlowFile whose content supports embedded metadata.truefalsefalseNONEfalsefalseMerge FormatMerge FormatAvroMinimum Number of EntriesMinimum Number of EntriesThe minimum number of files to include in a bundle1truefalsefalseNONEfalsefalseMerge StrategyMerge StrategyBin-Packing AlgorithmMaximum Number of EntriesMaximum Number of EntriesThe maximum number of files to include in a bundle1000truefalsefalseNONEfalsefalseMerge StrategyMerge StrategyBin-Packing AlgorithmMinimum Group SizeMinimum Group SizeThe minimum size for the bundle0 BtruefalsefalseNONEfalsefalseMerge StrategyMerge StrategyBin-Packing AlgorithmMaximum Group SizeMaximum Group SizeThe maximum size for the bundle. If not specified, there is no maximum.falsefalsefalseNONEfalsefalseMerge StrategyMerge StrategyBin-Packing AlgorithmMax Bin AgeMax Bin AgeThe maximum age of a Bin that will trigger a Bin to be complete. Expected format is <duration> <time unit> where <duration> is a positive integer and time unit is one of seconds, minutes, hoursfalsefalsefalseNONEfalsefalseMaximum number of BinsMaximum number of BinsSpecifies the maximum number of bins that can be held in memory at any one time5truefalsefalseNONEfalsefalseDelimiter StrategyDelimiter StrategyDetermines if Header, Footer, and Demarcator should point to files containing the respective content, or if the values of the properties should be used as the content.Do Not Use DelimitersDo Not Use DelimitersDo Not Use DelimitersNo Header, Footer, or Demarcator will be usedFilenameFilenameThe values of Header, Footer, and Demarcator will be retrieved from the contents of a fileTextTextThe values of Header, Footer, and Demarcator will be specified as property valuestruefalsefalseNONEfalsefalseMerge FormatMerge FormatBinary ConcatenationHeader FileHeaderFilename or text specifying the header to use. If not specified, no header is supplied.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLETEXTFILEMerge FormatMerge FormatBinary ConcatenationDelimiter StrategyDelimiter StrategyFilenameTextFooter FileFooterFilename or text specifying the footer to use. If not specified, no footer is supplied.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLETEXTFILEMerge FormatMerge FormatBinary ConcatenationDelimiter StrategyDelimiter StrategyFilenameTextDemarcator FileDemarcatorFilename or text specifying the demarcator to use. If not specified, no demarcator is supplied.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLETEXTFILEMerge FormatMerge FormatBinary ConcatenationDelimiter StrategyDelimiter StrategyFilenameTextCompression LevelCompression LevelSpecifies the compression level to use when using the Zip Merge Format; if not using the Zip Merge Format, this value is ignored100112233445566778899truefalsefalseNONEfalsefalseMerge FormatMerge FormatZIPKeep PathKeep PathIf using the Zip or Tar Merge Format, specifies whether or not the FlowFiles' paths should be included in their entry names.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseMerge FormatMerge FormatZIPTARTar Modified TimeTar Modified TimeIf using the Tar Merge Format, specifies if the Tar entry should store the modified timestamp either by expression (e.g. ${file.lastModifiedTime} or static value, both of which must match the ISO8601 format 'yyyy-MM-dd'T'HH:mm:ssZ'.${file.lastModifiedTime}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMerge FormatMerge FormatTARfailureIf the bundle cannot be created, all FlowFiles that would have been used to created the bundle will be transferred to failurefalseoriginalThe FlowFiles that were used to create the bundlefalsemergedThe FlowFile containing the merged contentfalsefragment.identifierApplicable only if the <Merge Strategy> property is set to Defragment. All FlowFiles with the same value for this attribute will be bundled together.fragment.indexApplicable only if the <Merge Strategy> property is set to Defragment. This attribute indicates the order in which the fragments should be assembled. This attribute must be present on all FlowFiles when using the Defragment Merge Strategy and must be a unique (i.e., unique across all FlowFiles that have the same value for the "fragment.identifier" attribute) integer between 0 and the value of the fragment.count attribute. If two or more FlowFiles have the same value for the "fragment.identifier" attribute and the same value for the "fragment.index" attribute, the first FlowFile processed will be accepted and subsequent FlowFiles will not be accepted into the Bin.fragment.countApplicable only if the <Merge Strategy> property is set to Defragment. This attribute indicates how many FlowFiles should be expected in the given bundle. At least one FlowFile must have this attribute in the bundle. If multiple FlowFiles contain the "fragment.count" attribute in a given bundle, all must have the same value.segment.original.filenameApplicable only if the <Merge Strategy> property is set to Defragment. This attribute must be present on all FlowFiles with the same value for the fragment.identifier attribute. All FlowFiles in the same bundle must have the same value for this attribute. The value of this attribute will be used for the filename of the completed merged FlowFile.tar.permissionsApplicable only if the <Merge Format> property is set to TAR. The value of this attribute must be 3 characters; each character must be in the range 0 to 7 (inclusive) and indicates the file permissions that should be used for the FlowFile's TAR entry. If this attribute is missing or has an invalid value, the default value of 644 will be usedfilenameWhen more than 1 file is merged, the filename comes from the segment.original.filename attribute. If that attribute does not exist in the source FlowFiles, then the filename is set to the number of nanoseconds matching system time. Then a filename extension may be applied:if Merge Format is TAR, then the filename will be appended with .tar, if Merge Format is ZIP, then the filename will be appended with .zip, if Merge Format is FlowFileStream, then the filename will be appended with .pkgmerge.countThe number of FlowFiles that were merged into this bundlemerge.bin.ageThe age of the bin, in milliseconds, when it was merged and output. Effectively this is the greatest amount of time that any FlowFile in this bundle remained waiting in this processor before it was outputmerge.uuidUUID of the merged flow file that will be added to the original flow files attributes.merge.reasonThis processor allows for several thresholds to be configured for merging FlowFiles. This attribute indicates which of the Thresholds resulted in the FlowFiles being merged. For an explanation of each of the possible values and their meanings, see the Processor's Usage / documentation and see the 'Additional Details' page.truetrueINPUT_REQUIREDMEMORYWhile content is not stored in memory, the FlowFiles' attributes are. The configuration of MergeContent (maximum bin size, maximum group size, maximum bin age, max number of entries) will influence how much memory is used. If merging together many small FlowFiles, a two-stage approach may be necessary in order to avoid excessive use of memory.Concatenate FlowFiles with textual content together in order to create fewer, larger FlowFiles.concatenatebundleaggregatebinmergecombinesmashINPUT_ALLOWED"Merge Strategy" = "Bin Packing Algorithm" "Merge Format" = "Binary Concatenation" "Delimiter Strategy" = "Text" "Demarcator" = "\n" (a newline can be inserted by pressing Shift + Enter) @@ -204,7 +204,7 @@ Add a single additional property. The name of the property should describe the criteria. The property's value should be a RecordPath that returns `true` if the Record meets the criteria or `false` otherwise. For example, if we want to separate records based on whether or not they have a transaction total of more than $1,000 we could add a new property named `largeTransaction` with a value of `/transaction/total > 1000`. This will create two FlowFiles. In the first, all records will have a total over `1000`. In the second, all records will have a transaction less than or equal to 1000. Each FlowFile will have an attribute named `largeTransaction` with a value of `true` or `false`. -org.apache.nifi.processors.standard.ConvertRecordorg.apache.nifi.processors.standard.SplitRecordorg.apache.nifi.processors.standard.UpdateRecordorg.apache.nifi.processors.standard.QueryRecordorg.apache.nifi.processors.standard.PutDatabaseRecordPROCESSORThe PutDatabaseRecord processor uses a specified RecordReader to input (possibly multiple) records from an incoming flow file. These records are translated to SQL statements and executed as a single transaction. If any errors occur, the flow file is routed to failure or retry, and if the records are transmitted successfully, the incoming flow file is routed to success. The type of statement executed by the processor is specified via the Statement Type property, which accepts some hard-coded values such as INSERT, UPDATE, and DELETE, as well as 'Use statement.type Attribute', which causes the processor to get the statement type from a flow file attribute. IMPORTANT: If the Statement Type is UPDATE, then the incoming records must not alter the value(s) of the primary keys (or user-specified Update Keys). If such records are encountered, the UPDATE statement issued to the database may do nothing (if no existing records with the new primary key values are found), or could inadvertently corrupt the existing data (by changing records for which the new values of the primary keys exist).sqlrecordjdbcputdatabaseupdateinsertdeleteput-db-record-record-readerRecord ReaderSpecifies the Controller Service to use for parsing incoming data and determining the data's schema.org.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLfalsefalsefalseNONEfalsefalseput-db-record-statement-typeStatement TypeSpecifies the type of SQL Statement to generate. Please refer to the database documentation for a description of the behavior of each operation. Please note that some Database Types may not support certain Statement Types. If 'Use statement.type Attribute' is chosen, then the value is taken from the statement.type attribute in the FlowFile. The 'Use statement.type Attribute' option is the only one that allows the 'SQL' statement type. If 'SQL' is specified, the value of the field specified by the 'Field Containing SQL' property is expected to be a valid SQL statement on the target database, and will be executed as-is.UPDATEUPDATEINSERTINSERTUPSERTUPSERTINSERT_IGNOREINSERT_IGNOREDELETEDELETEUse statement.type AttributeUse statement.type AttributeUse Record PathUse Record PathtruefalsefalseNONEfalsefalseStatement Type Record PathStatement Type Record PathSpecifies a RecordPath to evaluate against each Record in order to determine the Statement Type. The RecordPath should equate to either INSERT, UPDATE, UPSERT, or DELETE. (Debezium style operation types are also supported: "r" and "c" for INSERT, "u" for UPDATE, and "d" for DELETE)truefalsefalseNONEfalsefalseput-db-record-statement-typeStatement TypeUse Record PathData Record PathData Record PathIf specified, this property denotes a RecordPath that will be evaluated against each incoming Record and the Record that results from evaluating the RecordPath will be sent to the database instead of sending the entire incoming Record. If not specified, the entire incoming Record will be published to the database.falsefalsefalseNONEfalsefalseput-db-record-dcbp-serviceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database for sending records.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseput-db-record-catalog-nameCatalog NameThe name of the catalog that the statement should update. This may not apply for the database that you are updating. In this case, leave the field empty. Note that if the property is set and the database is case-sensitive, the catalog name must match the database's catalog name exactly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-schema-nameSchema NameThe name of the schema that the table belongs to. This may not apply for the database that you are updating. In this case, leave the field empty. Note that if the property is set and the database is case-sensitive, the schema name must match the database's schema name exactly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-table-nameTable NameThe name of the table that the statement should affect. Note that if the database is case-sensitive, the table name must match the database's table name exactly.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-binary-formatBinary String FormatThe format to be applied when decoding string values to binary.UTF-8UTF-8UTF-8String values for binary columns contain the original value as text via UTF-8 character encodingHexadecimalHexadecimalString values for binary columns contain the original value in hexadecimal formatBase64Base64String values for binary columns contain the original value in Base64 encoded formattruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-translate-field-namesTranslate Field NamesIf true, the Processor will attempt to translate field names into the appropriate column names for the table specified. If false, the field names must match the column names exactly, or the column will not be updatedtruetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-db-record-unmatched-field-behaviorUnmatched Field BehaviorIf an incoming record has a field that does not map to any of the database table's columns, this property specifies how to handle the situationIgnore Unmatched FieldsIgnore Unmatched FieldsIgnore Unmatched FieldsAny field in the document that cannot be mapped to a column in the database is ignoredFail on Unmatched FieldsFail on Unmatched FieldsIf the document has any field that cannot be mapped to a column in the database, the FlowFile will be routed to the failure relationshipfalsefalsefalseNONEfalsefalseput-db-record-unmatched-column-behaviorUnmatched Column BehaviorIf an incoming record does not have a field mapping for all of the database table's columns, this property specifies how to handle the situationFail on Unmatched ColumnsIgnore Unmatched ColumnsIgnore Unmatched ColumnsAny column in the database that does not have a field in the document will be assumed to not be required. No notification will be loggedWarn on Unmatched ColumnsWarn on Unmatched ColumnsAny column in the database that does not have a field in the document will be assumed to not be required. A warning will be loggedFail on Unmatched ColumnsFail on Unmatched ColumnsA flow will fail if any column in the database that does not have a field in the document. An error will be loggedfalsefalsefalseNONEfalsefalseput-db-record-update-keysUpdate KeysA comma-separated list of column names that uniquely identifies a row in the database for UPDATE statements. If the Statement Type is UPDATE and this property is not set, the table's Primary Keys are used. In this case, if no Primary Key exists, the conversion to SQL will fail if Unmatched Column Behaviour is set to FAIL. This property is ignored if the Statement Type is INSERTfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-statement-typeStatement TypeUse statement.type AttributeUse Record PathUPSERTUPDATESQLput-db-record-field-containing-sqlField Containing SQLIf the Statement Type is 'SQL' (as set in the statement.type attribute), this field indicates which field in the record(s) contains the SQL statement to execute. The value of the field must be a single SQL statement. If the Statement Type is not 'SQL', this field is ignored.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-statement-typeStatement TypeUse statement.type AttributeUse Record Pathput-db-record-allow-multiple-statementsAllow Multiple SQL StatementsIf the Statement Type is 'SQL' (as set in the statement.type attribute), this field indicates whether to split the field value by a semicolon and execute each statement separately. If any statement causes an error, the entire set of statements will be rolled back. If the Statement Type is not 'SQL', this field is ignored.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseput-db-record-statement-typeStatement TypeUse statement.type AttributeUse Record Pathput-db-record-quoted-identifiersQuote Column IdentifiersEnabling this option will cause all column names to be quoted, allowing you to use reserved words as column names in your tables.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-db-record-quoted-table-identifiersQuote Table IdentifiersEnabling this option will cause the table name to be quoted to support the use of special characters in the table name.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-db-record-query-timeoutMax Wait TimeThe maximum amount of time allowed for a running SQL statement , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueENVIRONMENTfalsefalserollback-on-failureRollback On FailureSpecify how to handle error. By default (false), if an error occurs while processing a FlowFile, the FlowFile will be routed to 'failure' or 'retry' relationship based on error type, and processor can continue with next FlowFile. Instead, you may want to rollback currently processed FlowFiles and stop further processing immediately. In that case, you can do so by enabling this 'Rollback On Failure' property. If enabled, failed FlowFiles will stay in the input relationship without penalizing it and being processed repeatedly until it gets processed successfully or removed by other means. It is important to set adequate 'Yield Duration' to avoid retrying too frequently.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsetable-schema-cache-sizeTable Schema Cache SizeSpecifies how many Table Schemas should be cached100truefalsefalseNONEfalsefalseput-db-record-max-batch-sizeMaximum Batch SizeSpecifies maximum number of sql statements to be included in each batch sent to the database. Zero means the batch size is not limited, and all statements are put into a single batch which can cause high memory usage issues for a very large number of statements.1000falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedatabase-session-autocommitDatabase Session AutoCommitThe autocommit mode to set on the database connection being used. If set to false, the operation(s) will be explicitly committed or rolled back (based on success or failure respectively). If set to true, the driver/database automatically handles the commit/rollback.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseretryA FlowFile is routed to this relationship if the database cannot be updated but attempting the operation again may succeedfalsesuccessSuccessfully created FlowFile from SQL query result set.falsefailureA FlowFile is routed to this relationship if the database cannot be updated and retrying the operation will also fail, such as an invalid query or an integrity constraint violationfalsestatement.typeIf 'Use statement.type Attribute' is selected for the Statement Type property, the value of this attribute will be used to determine the type of statement (INSERT, UPDATE, DELETE, SQL, etc.) to generate and execute.putdatabaserecord.errorIf an error occurs during processing, the flow file will be routed to failure or retry, and this attribute will be populated with the cause of the error.INPUT_REQUIREDInsert records into a databaseINPUT_ALLOWEDorg.apache.nifi.processors.standard.PutDistributedMapCachePROCESSORGets the content of a FlowFile and puts it to a distributed map cache, using a cache key computed from FlowFile attributes. If the cache already contains the entry and the cache update strategy is 'keep original' the entry is not replaced.'mapcacheputdistributedCache Entry IdentifierCache Entry IdentifierA FlowFile attribute, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the cache keytruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseDistributed Cache ServiceDistributed Cache ServiceThe Controller Service that is used to cache flow filesorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseCache update strategyCache update strategyDetermines how the cache is updated if the cache already contains the entryreplaceReplace if presentreplaceAdds the specified entry to the cache, replacing any value that is currently set.Keep originalkeeporiginalAdds the specified entry to the cache, if the key does not exist.truefalsefalseNONEfalsefalseMax cache entry sizeMax cache entry sizeThe maximum amount of data to put into cache1 MBfalsefalsefalseNONEfalsefalsesuccessAny FlowFile that is successfully inserted into cache will be routed to this relationshipfalsefailureAny FlowFile that cannot be inserted into the cache will be routed to this relationshipfalsecachedAll FlowFiles will have an attribute 'cached'. The value of this attribute is true, is the FlowFile is cached, otherwise false.trueINPUT_REQUIREDorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientServiceorg.apache.nifi.distributed.cache.server.map.DistributedMapCacheServerorg.apache.nifi.processors.standard.FetchDistributedMapCacheorg.apache.nifi.processors.standard.PutEmailPROCESSORSends an e-mail to configured recipients for each incoming FlowFileemailputnotifysmtpSMTP HostnameSMTP HostnameThe hostname of the SMTP hosttruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP PortSMTP PortThe Port used for SMTP communications25truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseauthorization-modeAuthorization ModeHow to authorize sending email on the user's behalf.password-based-authorization-modeUse Passwordpassword-based-authorization-modeUse passwordUse OAuth2oauth-based-authorization-modeUse OAuth2 to acquire access tokentruefalsefalseNONEfalsefalseoauth2-access-token-providerOAuth2 Access Token ProviderOAuth2 service that can provide access tokens.org.apache.nifi.oauth2.OAuth2AccessTokenProviderorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseauthorization-modeAuthorization Modeoauth-based-authorization-modeSMTP UsernameSMTP UsernameUsername for the SMTP accountfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP PasswordSMTP PasswordPassword for the SMTP accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseauthorization-modeAuthorization Modepassword-based-authorization-modeSMTP AuthSMTP AuthFlag indicating whether authentication should be usedtruetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP TLSSMTP STARTTLSFlag indicating whether Opportunistic TLS should be enabled using STARTTLS commandfalsetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP Socket FactorySMTP Socket FactorySocket Factory to use for SMTP Connectionjavax.net.ssl.SSLSocketFactorytruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP X-Mailer HeaderSMTP X-Mailer HeaderX-Mailer used in the header of the outgoing emailNiFitruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseattribute-name-regexAttributes to Send as Headers (Regex)A Regular Expression that is matched against all FlowFile attribute names. Any attribute whose name matches the regex will be added to the Email messages as a Header. If not specified, no FlowFile attributes will be added as headers.falsefalsefalseNONEfalsefalseContent TypeContent TypeMime Type used to interpret the contents of the email, such as text/plain or text/htmltext/plaintruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFromFromSpecifies the Email address to use as the sender. Comma separated sequence of addresses following RFC822 syntax.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseToToThe recipients to include in the To-Line of the email. Comma separated sequence of addresses following RFC822 syntax.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCCCCThe recipients to include in the CC-Line of the email. Comma separated sequence of addresses following RFC822 syntax.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseBCCBCCThe recipients to include in the BCC-Line of the email. Comma separated sequence of addresses following RFC822 syntax.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSubjectSubjectThe email subjectMessage from NiFitruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessageMessageThe body of the email messagefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseemail-ff-content-as-messageFlow file content as messageSpecifies whether or not the FlowFile content should be the message of the email. If true, the 'Message' property is ignored.falsetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseinput-character-setInput Character SetSpecifies the character set of the FlowFile contents for reading input FlowFile contents to generate the message body or as an attachment to the message. If not set, UTF-8 will be the default value.UTF-8truefalsefalseNONEfalsefalseAttach FileAttach FileSpecifies whether or not the FlowFile content should be attached to the emailfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseInclude All Attributes In MessageInclude All Attributes In MessageSpecifies whether or not all FlowFile attributes should be recorded in the body of the email messagefalsetruetruefalsefalsetruefalsefalseNONEfalsefalsemail.propertyNameValue for a specific property to be set in the JavaMail Session objectDynamic property names that will be passed to the Mail session. Possible properties can be found in: https://javaee.github.io/javamail/docs/api/com/sun/mail/smtp/package-summary.html.FLOWFILE_ATTRIBUTEStruesuccessFlowFiles that are successfully sent will be routed to this relationshipfalsefailureFlowFiles that fail to send will be routed to this relationshipfalsetrueINPUT_REQUIREDMEMORYThe entirety of the FlowFile's content (as a String object) will be read into memory in case the property to use the flow file content as the email body is set to true.org.apache.nifi.processors.standard.PutFTPPROCESSORSends FlowFiles to an FTP ServerremotecopyegressputftparchivefilesHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers21truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push filesfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCreate DirectoryCreate DirectorySpecifies whether or not the remote directory should be created if it does not exist.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseBatch SizeBatch SizeThe maximum number of FlowFiles to send in a single connection500truefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseConflict ResolutionConflict ResolutionDetermines how to handle the problem of filename collisionsNONEREPLACEREPLACERemote file is replaced with new file, FlowFile goes to successIGNOREIGNOREFile is not transferred, FlowFile goes to successRENAMERENAMENew file is renamed with a one-up number at the beginning, FlowFile goes to successREJECTREJECTFile is not transferred, FlowFile goes to rejectFAILFAILFile is not transferred, FlowFile goes to failureNONENONEDo not check for conflict before transfer, FlowFile goes to success or failuretruefalsefalseNONEfalsefalseDot RenameDot RenameIf true, then the filename of the sent file is prepended with a "." and then renamed back to the original once the file is completely sent. Otherwise, there is no rename. This property is ignored if the Temporary Filename property is set.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseTemporary FilenameTemporary FilenameIf set, the filename of the sent file will be equal to the value specified during the transfer and after successful completion will be renamed to the original filename. If this value is set, the Dot Rename property is ignored.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseTransfer ModeTransfer ModeThe FTP Transfer ModeBinaryBinaryBinaryASCIIASCIIfalsefalsefalseNONEfalsefalseConnection ModeConnection ModeThe FTP Connection ModePassiveActiveActivePassivePassivefalsefalsefalseNONEfalsefalseReject Zero-Byte FilesReject Zero-Byte FilesDetermines whether or not Zero-byte files should be rejected without attempting to transfertruetruetruefalsefalsefalsefalsefalseNONEfalsefalseLast Modified TimeLast Modified TimeThe lastModifiedTime to assign to the file after transferring it. If not set, the lastModifiedTime will not be changed. Format must be yyyy-MM-dd'T'HH:mm:ssZ. You may also use expression language such as ${file.lastModifiedTime}. If the value is invalid, the processor will not be invalid but will fail to change lastModifiedTime of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePermissionsPermissionsThe permissions to assign to the file after transferring it. Format must be either UNIX rwxrwxrwx with a - in place of denied permissions (e.g. rw-r--r--) or an octal number (e.g. 644). If not set, the permissions will not be changed. You may also use expression language such as ${file.permissions}. If the value is invalid, the processor will not be invalid but will fail to change permissions of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: SOCKS + AuthN, HTTP + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseInternal Buffer SizeInternal Buffer SizeSet the internal buffer size for buffered data streams16KBfalsefalsefalseNONEfalsefalseftp-use-utf8Use UTF-8 EncodingTells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsepre.cmd._____Not usedThe command specified in the key will be executed before doing a put. You may add these optional properties to send any commands to the FTP server before the file is actually transferred (before the put command). This option is only available for the PutFTP processor, as only FTP has this functionality. This is essentially the same as sending quote commands to an FTP server from the command line. While this is the same as sending a quote command, it is very important that you leave off the .NONEpost.cmd._____Not usedThe command specified in the key will be executed after doing a put. You may add these optional properties to send any commands to the FTP server before the file is actually transferred (before the put command). This option is only available for the PutFTP processor, as only FTP has this functionality. This is essentially the same as sending quote commands to an FTP server from the command line. While this is the same as sending a quote command, it is very important that you leave off the .NONEsuccessFlowFiles that are successfully sent will be routed to successfalserejectFlowFiles that were rejected by the destination systemfalsefailureFlowFiles that failed to send to the remote system; failure is usually looped back to this processorfalsetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.GetFTPorg.apache.nifi.processors.standard.PutFilePROCESSORWrites the contents of a FlowFile to the local file systemputlocalcopyarchivefilesfilesystemDirectoryDirectoryThe directory to which files should be written. You may use expression language such as /aa/bb/${path}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseConflict Resolution StrategyConflict Resolution StrategyIndicates what should happen when a file with the same name already exists in the output directoryfailreplacereplaceignoreignorefailfailtruefalsefalseNONEfalsefalseCreate Missing DirectoriesCreate Missing DirectoriesIf true, then missing destination directories will be created. If false, flowfiles are penalized and sent to failure.truetruetruefalsefalsetruefalsefalseNONEfalsefalseMaximum File CountMaximum File CountSpecifies the maximum number of files that can exist in the output directoryfalsefalsefalseNONEfalsefalseLast Modified TimeLast Modified TimeSets the lastModifiedTime on the output file to the value of this attribute. Format must be yyyy-MM-dd'T'HH:mm:ssZ. You may also use expression language such as ${file.lastModifiedTime}.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePermissionsPermissionsSets the permissions on the output file to the value of this attribute. Format must be either UNIX rwxrwxrwx with a - in place of denied permissions (e.g. rw-r--r--) or an octal number (e.g. 644). You may also use expression language such as ${file.permissions}.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseOwnerOwnerSets the owner on the output file to the value of this attribute. You may also use expression language such as ${file.owner}. Note on many operating systems Nifi must be running as a super-user to have the permissions to set the file owner.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseGroupGroupSets the group on the output file to the value of this attribute. You may also use expression language such as ${file.group}.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessFiles that have been successfully written to the output directory are transferred to this relationshipfalsefailureFiles that could not be written to the output directory for some reason are transferred to this relationshipfalsefilenameThe filename to use when writing the FlowFile to disk.truewrite filesystemProvides operator the ability to write to any file that NiFi has access to.INPUT_REQUIREDorg.apache.nifi.processors.standard.FetchFileorg.apache.nifi.processors.standard.GetFileorg.apache.nifi.processors.standard.PutRecordPROCESSORThe PutRecord processor uses a specified RecordReader to input (possibly multiple) records from an incoming flow file, and sends them to a destination specified by a Record Destination Service (i.e. record sink).recordputsinkput-record-readerRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseput-record-sinkRecord Destination ServiceSpecifies the Controller Service to use for writing out the query result records to some destination.org.apache.nifi.record.sink.RecordSinkServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseput-record-include-zero-record-resultsInclude Zero Record ResultsIf no records are read from the incoming FlowFile, this property specifies whether or not an empty record set will be transmitted. The original FlowFile will still be routed to success, but if no transmission occurs, no provenance SEND event will be generated.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseretryThe original FlowFile is routed to this relationship if the records could not be transmitted but attempting the operation again may succeedfalsesuccessThe original FlowFile will be routed to this relationship if the records were transmitted successfullyfalsefailureA FlowFile is routed to this relationship if the records could not be transmitted and retrying the operation will also failfalseINPUT_REQUIREDorg.apache.nifi.processors.standard.PutSFTPPROCESSORSends FlowFiles to an SFTP ServerremotecopyegressputsftparchivefilesHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers22truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalsePrivate Key PathPrivate Key PathThe fully qualified path to the Private Key filefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILEPrivate Key PassphrasePrivate Key PassphrasePassword for the private keyfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push filesfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCreate DirectoryCreate DirectorySpecifies whether or not the remote directory should be created if it does not exist.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseDisable Directory ListingDisable Directory ListingIf set to 'true', directory listing is not performed prior to create missing directories. By default, this processor executes a directory listing command to see target directory existence before creating missing directories. However, there are situations that you might need to disable the directory listing such as the following. Directory listing might fail with some permission setups (e.g. chmod 100) on a directory. Also, if any other SFTP client created the directory after this processor performed a listing and before a directory creation request by this processor is finished, then an error is returned because the directory already exists.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseBatch SizeBatch SizeThe maximum number of FlowFiles to send in a single connection500truefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseConflict ResolutionConflict ResolutionDetermines how to handle the problem of filename collisionsNONEREPLACEREPLACERemote file is replaced with new file, FlowFile goes to successIGNOREIGNOREFile is not transferred, FlowFile goes to successRENAMERENAMENew file is renamed with a one-up number at the beginning, FlowFile goes to successREJECTREJECTFile is not transferred, FlowFile goes to rejectFAILFAILFile is not transferred, FlowFile goes to failureNONENONEDo not check for conflict before transfer, FlowFile goes to success or failuretruefalsefalseNONEfalsefalseReject Zero-Byte FilesReject Zero-Byte FilesDetermines whether or not Zero-byte files should be rejected without attempting to transfertruetruetruefalsefalsefalsefalsefalseNONEfalsefalseDot RenameDot RenameIf true, then the filename of the sent file is prepended with a "." and then renamed back to the original once the file is completely sent. Otherwise, there is no rename. This property is ignored if the Temporary Filename property is set.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseTemporary FilenameTemporary FilenameIf set, the filename of the sent file will be equal to the value specified during the transfer and after successful completion will be renamed to the original filename. If this value is set, the Dot Rename property is ignored.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHost Key FileHost Key FileIf supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be usedfalsefalsefalseNONEfalsefalseSINGLEFILELast Modified TimeLast Modified TimeThe lastModifiedTime to assign to the file after transferring it. If not set, the lastModifiedTime will not be changed. Format must be yyyy-MM-dd'T'HH:mm:ssZ. You may also use expression language such as ${file.lastModifiedTime}. If the value is invalid, the processor will not be invalid but will fail to change lastModifiedTime of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePermissionsPermissionsThe permissions to assign to the file after transferring it. Format must be either UNIX rwxrwxrwx with a - in place of denied permissions (e.g. rw-r--r--) or an octal number (e.g. 644). If not set, the permissions will not be changed. You may also use expression language such as ${file.permissions}. If the value is invalid, the processor will not be invalid but will fail to change permissions of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseRemote OwnerRemote OwnerInteger value representing the User ID to set on the file after transferring it. If not set, the owner will not be set. You may also use expression language such as ${file.owner}. If the value is invalid, the processor will not be invalid but will fail to change the owner of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseRemote GroupRemote GroupInteger value representing the Group ID to set on the file after transferring it. If not set, the group will not be set. You may also use expression language such as ${file.group}. If the value is invalid, the processor will not be invalid but will fail to change the group of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseStrict Host Key CheckingStrict Host Key CheckingIndicates whether or not strict enforcement of hosts keys should be appliedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseSend Keep Alive On TimeoutSend Keep Alive On TimeoutSend a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.truetruetruefalsefalsetruefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: SOCKS + AuthN, HTTP + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseCiphers AllowedCiphers AllowedA comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctrfalsefalsetrueENVIRONMENTfalsefalseKey Algorithms AllowedKey Algorithms AllowedA comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.comfalsefalsetrueENVIRONMENTfalsefalseKey Exchange Algorithms AllowedKey Exchange Algorithms AllowedA comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-cfalsefalsetrueENVIRONMENTfalsefalseMessage Authentication Codes AllowedMessage Authentication Codes AllowedA comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.comfalsefalsetrueENVIRONMENTfalsefalsesuccessFlowFiles that are successfully sent will be routed to successfalserejectFlowFiles that were rejected by the destination systemfalsefailureFlowFiles that failed to send to the remote system; failure is usually looped back to this processorfalsetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.GetSFTPorg.apache.nifi.processors.standard.PutSQLPROCESSORExecutes a SQL UPDATE or INSERT command. The content of an incoming FlowFile is expected to be the SQL command to execute. The SQL command may use the ? to escape parameters. In this case, the parameters to use must exist as FlowFile attributes with the naming convention sql.args.N.type and sql.args.N.value, where N is a positive integer. The sql.args.N.type is expected to be a number indicating the JDBC Type. The content of the FlowFile is expected to be in UTF-8 format.sqlputrdbmsdatabaseupdateinsertrelationalJDBC Connection PoolJDBC Connection PoolSpecifies the JDBC Connection Pool to use in order to convert the JSON message to a SQL statement. The Connection Pool is necessary in order to determine the appropriate database column types.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseputsql-sql-statementSQL StatementThe SQL statement to execute. The statement can be empty, a constant value, or built from attributes using Expression Language. If this property is specified, it will be used regardless of the content of incoming FlowFiles. If this property is empty, the content of the incoming FlowFile is expected to contain a valid SQL statement, to be issued by the processor to the database.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSupport Fragmented TransactionsSupport Fragmented TransactionsIf true, when a FlowFile is consumed by this Processor, the Processor will first check the fragment.identifier and fragment.count attributes of that FlowFile. If the fragment.count value is greater than 1, the Processor will not process any FlowFile with that fragment.identifier until all are available; at that point, it will process all FlowFiles with that fragment.identifier as a single transaction, in the order specified by the FlowFiles' fragment.index attributes. This Provides atomicity of those SQL statements. Once any statement of this transaction throws exception when executing, this transaction will be rolled back. When transaction rollback happened, none of these FlowFiles would be routed to 'success'. If the <Rollback On Failure> is set true, these FlowFiles will stay in the input relationship. When the <Rollback On Failure> is set false,, if any of these FlowFiles will be routed to 'retry', all of these FlowFiles will be routed to 'retry'.Otherwise, they will be routed to 'failure'. If this value is false, these attributes will be ignored and the updates will occur independent of one another.truetruetruefalsefalsefalsefalsefalseNONEfalsefalsedatabase-session-autocommitDatabase Session AutoCommitThe autocommit mode to set on the database connection being used. If set to false, the operation(s) will be explicitly committed or rolled back (based on success or failure respectively), if set to true the driver/database handles the commit/rollback.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseTransaction TimeoutTransaction TimeoutIf the <Support Fragmented Transactions> property is set to true, specifies how long to wait for all FlowFiles for a particular fragment.identifier attribute to arrive before just transferring all of the FlowFiles with that identifier to the 'failure' relationshipfalsefalsefalseNONEfalsefalseBatch SizeBatch SizeThe preferred number of FlowFiles to put to the database in a single transaction100truefalsefalseNONEfalsefalseObtain Generated KeysObtain Generated KeysIf true, any key that is automatically generated by the database will be added to the FlowFile that generated it using the sql.generate.key attribute. This may result in slightly slower performance and is not supported by all databases.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalserollback-on-failureRollback On FailureSpecify how to handle error. By default (false), if an error occurs while processing a FlowFile, the FlowFile will be routed to 'failure' or 'retry' relationship based on error type, and processor can continue with next FlowFile. Instead, you may want to rollback currently processed FlowFiles and stop further processing immediately. In that case, you can do so by enabling this 'Rollback On Failure' property. If enabled, failed FlowFiles will stay in the input relationship without penalizing it and being processed repeatedly until it gets processed successfully or removed by other means. It is important to set adequate 'Yield Duration' to avoid retrying too frequently.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseretryA FlowFile is routed to this relationship if the database cannot be updated but attempting the operation again may succeedfalsesuccessA FlowFile is routed to this relationship after the database is successfully updatedfalsefailureA FlowFile is routed to this relationship if the database cannot be updated and retrying the operation will also fail, such as an invalid query or an integrity constraint violationfalsefragment.identifierIf the <Support Fragment Transactions> property is true, this attribute is used to determine whether or not two FlowFiles belong to the same transaction.fragment.countIf the <Support Fragment Transactions> property is true, this attribute is used to determine how many FlowFiles are needed to complete the transaction.fragment.indexIf the <Support Fragment Transactions> property is true, this attribute is used to determine the order that the FlowFiles in a transaction should be evaluated.sql.args.N.typeIncoming FlowFiles are expected to be parametrized SQL statements. The type of each Parameter is specified as an integer that represents the JDBC Type of the parameter.sql.args.N.valueIncoming FlowFiles are expected to be parametrized SQL statements. The value of the Parameters are specified as sql.args.1.value, sql.args.2.value, sql.args.3.value, and so on. The type of the sql.args.1.value Parameter is specified by the sql.args.1.type attribute.sql.args.N.formatThis attribute is always optional, but default options may not always work for your data. Incoming FlowFiles are expected to be parametrized SQL statements. In some cases a format option needs to be specified, currently this is only applicable for binary data types, dates, times and timestamps. Binary Data Types (defaults to 'ascii') - ascii: each string character in your attribute value represents a single byte. This is the format provided by Avro Processors. base64: the string is a Base64 encoded string that can be decoded to bytes. hex: the string is hex encoded with all letters in upper case and no '0x' at the beginning. Dates/Times/Timestamps - Date, Time and Timestamp formats all support both custom formats or named format ('yyyy-MM-dd','ISO_OFFSET_DATE_TIME') as specified according to java.time.format.DateTimeFormatter. If not specified, a long value input is expected to be an unix epoch (milli seconds from 1970/1/1), or a string value in 'yyyy-MM-dd' format for Date, 'HH:mm:ss.SSS' for Time (some database engines e.g. Derby or MySQL do not support milliseconds and will truncate milliseconds), 'yyyy-MM-dd HH:mm:ss.SSS' for Timestamp is used.sql.generated.keyIf the database generated a key for an INSERT statement and the Obtain Generated Keys property is set to true, this attribute will be added to indicate the generated key, if possible. This feature is not supported by all database vendors.trueINPUT_REQUIREDorg.apache.nifi.processors.standard.ConvertJSONToSQLorg.apache.nifi.processors.standard.PutSyslogPROCESSORSends Syslog messages to a given host and port over TCP or UDP. Messages are constructed from the "Message ___" properties of the processor which can use expression language to generate messages from incoming FlowFiles. The properties are used to construct messages of the form: (<PRIORITY>)(VERSION )(TIMESTAMP) (HOSTNAME) (BODY) where version is optional. The constructed messages are checked against regular expressions for RFC5424 and RFC3164 formatted messages. The timestamp can be an RFC5424 timestamp with a format of "yyyy-MM-dd'T'HH:mm:ss.SZ" or "yyyy-MM-dd'T'HH:mm:ss.S+hh:mm", or it can be an RFC3164 timestamp with a format of "MMM d HH:mm:ss". If a message is constructed that does not form a valid Syslog message according to the above description, then it is routed to the invalid relationship. Valid messages are sent to the Syslog server and successes are routed to the success relationship, failures routed to the failure relationship.syslogputudptcplogsHostnameHostnameThe IP address or hostname of the Syslog server.localhosttruefalsetrueENVIRONMENTfalsefalseProtocolProtocolThe protocol for Syslog communication.UDPTCPTCPUDPUDPtruefalsefalseNONEfalsefalsePortPortThe port for Syslog communication. Note that Expression language is not evaluated per FlowFile.truefalsetrueENVIRONMENTfalsefalseMax Size of Socket Send BufferMax Size of Socket Send BufferThe maximum size of the socket send buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsetrueENVIRONMENTfalsefalseSSL Context ServiceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, syslog messages will be sent over a secure connection.org.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProtocolProtocolTCPIdle Connection ExpirationIdle Connection ExpirationThe amount of time a connection should be held open without being used before closing the connection.5 secondstruefalsetrueENVIRONMENTfalsefalseTimeoutTimeoutThe timeout for connecting to and communicating with the syslog server. Does not apply to UDP. Note that Expression language is not evaluated per FlowFile.10 secondsfalsefalsetrueENVIRONMENTfalsefalseBatch SizeBatch SizeThe number of incoming FlowFiles to process in a single execution of this processor.25truefalsetrueENVIRONMENTfalsefalseCharacter SetCharacter SetSpecifies the character set of the Syslog messages. Note that Expression language is not evaluated per FlowFile.UTF-8truefalsetrueENVIRONMENTfalsefalseMessage PriorityMessage PriorityThe priority for the Syslog messages, excluding < >.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessage VersionMessage VersionThe version for the Syslog messages.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessage TimestampMessage TimestampThe timestamp for the Syslog messages. The timestamp can be an RFC5424 timestamp with a format of "yyyy-MM-dd'T'HH:mm:ss.SZ" or "yyyy-MM-dd'T'HH:mm:ss.S+hh:mm", " or it can be an RFC3164 timestamp with a format of "MMM d HH:mm:ss".${now():format('MMM d HH:mm:ss')}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessage HostnameMessage HostnameThe hostname for the Syslog messages.${hostname(true)}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessage BodyMessage BodyThe body for the Syslog messages.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessFlowFiles that are sent successfully to Syslog are sent out this relationship.falsefailureFlowFiles that failed to send to Syslog are sent out this relationship.falseinvalidFlowFiles that do not form a valid Syslog message are sent out this relationship.falsetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ListenSyslogorg.apache.nifi.processors.standard.ParseSyslogorg.apache.nifi.processors.standard.PutTCPPROCESSORSends serialized FlowFiles or Records over TCP to a configurable destination with optional support for TLSremoteegressputtcpHostnameHostnameDestination hostname or IP addresslocalhosttruefalsetrueENVIRONMENTfalsefalsePortPortDestination port numbertruefalsetrueENVIRONMENTfalsefalseMax Size of Socket Send BufferMax Size of Socket Send BufferThe maximum size of the socket send buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseIdle Connection ExpirationIdle Connection ExpirationThe amount of time a connection should be held open without being used before closing the connection. A value of 0 seconds will disable this feature.15 secondstruefalsetrueENVIRONMENTfalsefalseTimeoutTimeoutThe timeout for connecting to and communicating with the destination. Does not apply to UDP10 secondsfalsefalsetrueENVIRONMENTfalsefalseConnection Per FlowFileConnection Per FlowFileSpecifies whether to send each FlowFile's content on an individual connection.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceSpecifies the SSL Context Service to enable TLS socket communicationorg.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseTransmission StrategyTransmission StrategySpecifies the strategy used for reading input FlowFiles and transmitting messages to the destination socket addressFLOWFILE_ORIENTEDFlowFile-orientedFLOWFILE_ORIENTEDSend FlowFile content as a single streamRecord-orientedRECORD_ORIENTEDRead Records from input FlowFiles and send serialized Records as individual messagestruefalsefalseNONEfalsefalseOutgoing Message DelimiterOutgoing Message DelimiterSpecifies the delimiter to use when sending messages out over the same TCP stream. The delimiter is appended to each FlowFile message that is transmitted over the stream so that the receiver can determine when one message ends and the next message begins. Users should ensure that the FlowFile content does not contain the delimiter character to avoid errors. In order to use a new line character you can enter '\n'. For a tab character use '\t'. Finally for a carriage return use '\r'.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseTransmission StrategyTransmission StrategyFLOWFILE_ORIENTEDCharacter SetCharacter SetSpecifies the character set of the data being sent.UTF-8truefalsetrueENVIRONMENTfalsefalseTransmission StrategyTransmission StrategyFLOWFILE_ORIENTEDRecord ReaderRecord ReaderSpecifies the Controller Service to use for reading Records from input FlowFilesorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseTransmission StrategyTransmission StrategyRECORD_ORIENTEDRecord WriterRecord WriterSpecifies the Controller Service to use for writing Records to the configured socket addressorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseTransmission StrategyTransmission StrategyRECORD_ORIENTEDsuccessFlowFiles that are sent successfully to the destination are sent out this relationship.falsefailureFlowFiles that failed to send to the destination are sent out this relationship.falserecord.count.transmittedCount of records transmitted to configured destination addresstrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ListenTCPorg.apache.nifi.processors.standard.PutUDPorg.apache.nifi.processors.standard.PutUDPPROCESSORThe PutUDP processor receives a FlowFile and packages the FlowFile content into a single UDP datagram packet which is then transmitted to the configured UDP server. The user must ensure that the FlowFile content being fed to this processor is not larger than the maximum size for the underlying UDP transport. The maximum transport size will vary based on the platform setup but is generally just under 64KB. FlowFiles will be marked as failed if their content is larger than the maximum transport size.remoteegressputudpHostnameHostnameDestination hostname or IP addresslocalhosttruefalsetrueENVIRONMENTfalsefalsePortPortDestination port numbertruefalsetrueENVIRONMENTfalsefalseMax Size of Socket Send BufferMax Size of Socket Send BufferThe maximum size of the socket send buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseIdle Connection ExpirationIdle Connection ExpirationThe amount of time a connection should be held open without being used before closing the connection. A value of 0 seconds will disable this feature.15 secondstruefalsetrueENVIRONMENTfalsefalseTimeoutTimeoutThe timeout for connecting to and communicating with the destination. Does not apply to UDP10 secondsfalsefalsetrueENVIRONMENTfalsefalsesuccessFlowFiles that are sent successfully to the destination are sent out this relationship.falsefailureFlowFiles that failed to send to the destination are sent out this relationship.falsetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ListenUDPorg.apache.nifi.processors.standard.PutTCPorg.apache.nifi.processors.standard.QueryDatabaseTablePROCESSORGenerates a SQL select query, or uses a provided statement, and executes it to fetch all rows whose values in the specified Maximum Value column(s) are larger than the previously-seen maxima. Query result will be converted to Avro format. Expression Language is supported for several properties, but no incoming connections are permitted. The Environment/System properties may be used to provide values for any property containing Expression Language. If it is desired to leverage flow file attributes to perform these queries, the GenerateTableFetch and/or ExecuteSQL processors can be used for this purpose. Streaming is used so arbitrarily large result sets are supported. This processor can be scheduled to run on a timer or cron expression, using the standard scheduling methods. This processor is intended to be run on the Primary Node only. FlowFile attribute 'querydbtable.row.count' indicates how many rows were selected.sqlselectjdbcquerydatabaseDatabase Connection Pooling ServiceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-fetch-db-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLtruefalsefalseNONEfalsefalseTable NameTable NameThe name of the database table to be queried. When a custom query is used, this property is used to alias the query and appears as an attribute on the FlowFile.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseColumns to ReturnColumns to ReturnA comma-separated list of column names to be used in the query. If your database requires special treatment of the names (quoting, e.g.), each name should include such treatment. If no column names are supplied, all columns in the specified table will be returned. NOTE: It is important to use consistent column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-where-clauseAdditional WHERE clauseA custom clause to be added in the WHERE condition when building SQL queries.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-sql-queryCustom QueryA custom SQL query used to retrieve data. Instead of building a SQL query from other properties, this query will be wrapped as a sub-query. Query must have no ORDER BY statement.falsefalsetrueENVIRONMENTfalsefalseMaximum-value ColumnsMaximum-value ColumnsA comma-separated list of column names. The processor will keep track of the maximum value for each column that has been returned since the processor started running. Using multiple columns implies an order to the column list, and each column's values are expected to increase more slowly than the previous columns' values. Thus, using multiple columns implies a hierarchical structure of columns, which is usually used for partitioning tables. This processor can be used to retrieve only those rows that have been added/updated since the last retrieval. Note that some JDBC types such as bit/boolean are not conducive to maintaining maximum value, so columns of these types should not be listed in this property, and will result in error(s) during processing. If no columns are provided, all rows from the table will be considered, which could have a performance impact. NOTE: It is important to use consistent max-value column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseinitial-load-strategyInitial Load StrategyHow to handle existing rows in the database table when the processor is started for the first time (or its state has been cleared). The property will be ignored, if any 'initial.maxvalue.*' dynamic property has also been configured.Start at BeginningStart at BeginningStart at BeginningLoads all existing rows from the database table.Start at Current Maximum ValuesStart at Current Maximum ValuesLoads only the newly inserted or updated rows based on the maximum value(s) of the column(s) configured in the 'Maximum-value Columns' property.truefalsefalseNONEfalsefalseMax Wait TimeMax Wait TimeThe maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFetch SizeFetch SizeThe number of result rows to be fetched from the result set at a time. This is a hint to the database driver and may not be honored and/or exact. If the value specified is zero, then the hint is ignored. If using PostgreSQL, then 'Set Auto Commit' must be equal to 'false' to cause 'Fetch Size' to take effect.0truefalsetrueENVIRONMENTfalsefalseSet Auto CommitSet Auto CommitAllows enabling or disabling the auto commit functionality of the DB connection. Default value is 'No value set'. 'No value set' will leave the db connection's auto commit mode unchanged. For some JDBC drivers such as PostgreSQL driver, it is required to disable the auto commit functionality to get the 'Fetch Size' setting to take effect. When auto commit is enabled, PostgreSQL driver ignores 'Fetch Size' setting and loads all rows of the result set to memory at once. This could lead for a large amount of memory usage when executing queries which fetch large data sets. More Details of this behaviour in PostgreSQL driver can be found in https://jdbc.postgresql.org//documentation/head/query.html.truetruefalsefalsefalsefalsetrueENVIRONMENTfalsefalseqdbt-max-rowsMax Rows Per Flow FileThe maximum number of result rows that will be included in a single FlowFile. This will allow you to break up very large result sets into multiple FlowFiles. If the value specified is zero, then all rows are returned in a single FlowFile.0truefalsetrueENVIRONMENTfalsefalseqdbt-output-batch-sizeOutput Batch SizeThe number of output FlowFiles to queue before committing the process session. When set to zero, the session will be committed when all result set rows have been processed and the output FlowFiles are ready for transfer to the downstream relationship. For large result sets, this can cause a large burst of FlowFiles to be transferred at the end of processor execution. If this property is set, then when the specified number of FlowFiles are ready for transfer, then the session will be committed, thus releasing the FlowFiles to the downstream relationship. NOTE: The maxvalue.* and fragment.count attributes will not be set on FlowFiles when this property is set.0truefalsetrueENVIRONMENTfalsefalseqdbt-max-fragsMaximum Number of FragmentsThe maximum number of fragments. If the value specified is zero, then all fragments are returned. This prevents OutOfMemoryError when this processor ingests huge table. NOTE: Setting this property can result in data loss, as the incoming results are not ordered, and fragments may end at arbitrary boundaries where rows are not included in the result set.0truefalsetrueENVIRONMENTfalsefalsedbf-normalizeNormalize Table/Column NamesWhether to change non-Avro-compatible characters in column names to Avro-compatible characters. For example, colons and periods will be changed to underscores in order to build a valid Avro record.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsetransaction-isolation-levelTransaction Isolation LevelThis setting will set the transaction isolation level for the database connection for drivers that support this settingTRANSACTION_NONE0TRANSACTION_READ_COMMITTED2TRANSACTION_READ_UNCOMMITTED1TRANSACTION_REPEATABLE_READ4TRANSACTION_SERIALIZABLE8falsefalsefalseNONEfalsefalsedbf-user-logical-typesUse Avro Logical TypesWhether to use Avro Logical Types for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP columns. If disabled, written as string. If enabled, Logical types are used and written as its underlying type, specifically, DECIMAL/NUMBER as logical 'decimal': written as bytes with additional precision and scale meta data, DATE as logical 'date-millis': written as int denoting days since Unix epoch (1970-01-01), TIME as logical 'time-millis': written as int denoting milliseconds since Unix epoch, and TIMESTAMP as logical 'timestamp-millis': written as long denoting milliseconds since Unix epoch. If a reader of written Avro records also knows these logical types, then these values can be deserialized with more context depending on reader implementation.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-default-precisionDefault Decimal PrecisionWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'precision' denoting number of available digits is required. Generally, precision is defined by column data type definition or database engines default. However undefined precision (0) can be returned from some database engines. 'Default Decimal Precision' is used when writing those undefined precision numbers.10truefalsetrueENVIRONMENTfalsefalsedbf-default-scaleDefault Decimal ScaleWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'scale' denoting number of available decimal digits is required. Generally, scale is defined by column data type definition or database engines default. However when undefined precision (0) is returned, scale can also be uncertain with some database engines. 'Default Decimal Scale' is used when writing those undefined numbers. If a value has more decimals than specified scale, then the value will be rounded-up, e.g. 1.53 becomes 2 with scale 0, and 1.5 with scale 1.0truefalsetrueENVIRONMENTfalsefalseinitial.maxvalue.<max_value_column>Initial maximum value for the specified columnSpecifies an initial max value for max value column(s). Properties should be added in the format `initial.maxvalue.<max_value_column>`. This value is only used the first time the table is accessed (when a Maximum Value Column is specified).ENVIRONMENTsuccessSuccessfully created FlowFile from SQL query result set.falsetablenameName of the table being queriedquerydbtable.row.countThe number of rows selected by the queryfragment.identifierIf 'Max Rows Per Flow File' is set then all FlowFiles from the same query result set will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.fragment.countIf 'Max Rows Per Flow File' is set then this is the total number of FlowFiles produced by a single ResultSet. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming ResultSet. If Output Batch Size is set, then this attribute will not be populated.fragment.indexIf 'Max Rows Per Flow File' is set then the position of this FlowFile in the list of outgoing FlowFiles that were all derived from the same result set FlowFile. This can be used in conjunction with the fragment.identifier attribute to know which FlowFiles originated from the same query result set and in what order FlowFiles were producedmaxvalue.*Each attribute contains the observed maximum value of a specified 'Maximum-value Column'. The suffix of the attribute is the name of the column. If Output Batch Size is set, then this attribute will not be populated.truetrueAfter performing a query on the specified table, the maximum values for the specified column(s) will be retained for use in future executions of the query. This allows the Processor to fetch only those records that have max values greater than the retained values. This can be used for incremental fetching, fetching of newly added rows, etc. To clear the maximum values, clear the state of the processor per the State Management documentationCLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.standard.GenerateTableFetchorg.apache.nifi.processors.standard.ExecuteSQLTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.QueryDatabaseTableRecordPROCESSORGenerates a SQL select query, or uses a provided statement, and executes it to fetch all rows whose values in the specified Maximum Value column(s) are larger than the previously-seen maxima. Query result will be converted to the format specified by the record writer. Expression Language is supported for several properties, but no incoming connections are permitted. The Environment/System properties may be used to provide values for any property containing Expression Language. If it is desired to leverage flow file attributes to perform these queries, the GenerateTableFetch and/or ExecuteSQL processors can be used for this purpose. Streaming is used so arbitrarily large result sets are supported. This processor can be scheduled to run on a timer or cron expression, using the standard scheduling methods. This processor is intended to be run on the Primary Node only. FlowFile attribute 'querydbtable.row.count' indicates how many rows were selected.sqlselectjdbcquerydatabaserecordDatabase Connection Pooling ServiceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-fetch-db-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLtruefalsefalseNONEfalsefalseTable NameTable NameThe name of the database table to be queried. When a custom query is used, this property is used to alias the query and appears as an attribute on the FlowFile.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseColumns to ReturnColumns to ReturnA comma-separated list of column names to be used in the query. If your database requires special treatment of the names (quoting, e.g.), each name should include such treatment. If no column names are supplied, all columns in the specified table will be returned. NOTE: It is important to use consistent column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-where-clauseAdditional WHERE clauseA custom clause to be added in the WHERE condition when building SQL queries.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-sql-queryCustom QueryA custom SQL query used to retrieve data. Instead of building a SQL query from other properties, this query will be wrapped as a sub-query. Query must have no ORDER BY statement.falsefalsetrueENVIRONMENTfalsefalseqdbtr-record-writerRecord WriterSpecifies the Controller Service to use for writing results to a FlowFile. The Record Writer may use Inherit Schema to emulate the inferred schema behavior, i.e. an explicit schema need not be defined in the writer, and will be supplied by the same logic used to infer the schema from the column types.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseMaximum-value ColumnsMaximum-value ColumnsA comma-separated list of column names. The processor will keep track of the maximum value for each column that has been returned since the processor started running. Using multiple columns implies an order to the column list, and each column's values are expected to increase more slowly than the previous columns' values. Thus, using multiple columns implies a hierarchical structure of columns, which is usually used for partitioning tables. This processor can be used to retrieve only those rows that have been added/updated since the last retrieval. Note that some JDBC types such as bit/boolean are not conducive to maintaining maximum value, so columns of these types should not be listed in this property, and will result in error(s) during processing. If no columns are provided, all rows from the table will be considered, which could have a performance impact. NOTE: It is important to use consistent max-value column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseinitial-load-strategyInitial Load StrategyHow to handle existing rows in the database table when the processor is started for the first time (or its state has been cleared). The property will be ignored, if any 'initial.maxvalue.*' dynamic property has also been configured.Start at BeginningStart at BeginningStart at BeginningLoads all existing rows from the database table.Start at Current Maximum ValuesStart at Current Maximum ValuesLoads only the newly inserted or updated rows based on the maximum value(s) of the column(s) configured in the 'Maximum-value Columns' property.truefalsefalseNONEfalsefalseMax Wait TimeMax Wait TimeThe maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFetch SizeFetch SizeThe number of result rows to be fetched from the result set at a time. This is a hint to the database driver and may not be honored and/or exact. If the value specified is zero, then the hint is ignored. If using PostgreSQL, then 'Set Auto Commit' must be equal to 'false' to cause 'Fetch Size' to take effect.0truefalsetrueENVIRONMENTfalsefalseSet Auto CommitSet Auto CommitAllows enabling or disabling the auto commit functionality of the DB connection. Default value is 'No value set'. 'No value set' will leave the db connection's auto commit mode unchanged. For some JDBC drivers such as PostgreSQL driver, it is required to disable the auto commit functionality to get the 'Fetch Size' setting to take effect. When auto commit is enabled, PostgreSQL driver ignores 'Fetch Size' setting and loads all rows of the result set to memory at once. This could lead for a large amount of memory usage when executing queries which fetch large data sets. More Details of this behaviour in PostgreSQL driver can be found in https://jdbc.postgresql.org//documentation/head/query.html.truetruefalsefalsefalsefalsetrueENVIRONMENTfalsefalseqdbt-max-rowsMax Rows Per Flow FileThe maximum number of result rows that will be included in a single FlowFile. This will allow you to break up very large result sets into multiple FlowFiles. If the value specified is zero, then all rows are returned in a single FlowFile.0truefalsetrueENVIRONMENTfalsefalseqdbt-output-batch-sizeOutput Batch SizeThe number of output FlowFiles to queue before committing the process session. When set to zero, the session will be committed when all result set rows have been processed and the output FlowFiles are ready for transfer to the downstream relationship. For large result sets, this can cause a large burst of FlowFiles to be transferred at the end of processor execution. If this property is set, then when the specified number of FlowFiles are ready for transfer, then the session will be committed, thus releasing the FlowFiles to the downstream relationship. NOTE: The maxvalue.* and fragment.count attributes will not be set on FlowFiles when this property is set.0truefalsetrueENVIRONMENTfalsefalseqdbt-max-fragsMaximum Number of FragmentsThe maximum number of fragments. If the value specified is zero, then all fragments are returned. This prevents OutOfMemoryError when this processor ingests huge table. NOTE: Setting this property can result in data loss, as the incoming results are not ordered, and fragments may end at arbitrary boundaries where rows are not included in the result set.0truefalsetrueENVIRONMENTfalsefalseqdbtr-normalizeNormalize Table/Column NamesWhether to change characters in column names when creating the output schema. For example, colons and periods will be changed to underscores.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-user-logical-typesUse Avro Logical TypesWhether to use Avro Logical Types for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP columns. If disabled, written as string. If enabled, Logical types are used and written as its underlying type, specifically, DECIMAL/NUMBER as logical 'decimal': written as bytes with additional precision and scale meta data, DATE as logical 'date-millis': written as int denoting days since Unix epoch (1970-01-01), TIME as logical 'time-millis': written as int denoting milliseconds since Unix epoch, and TIMESTAMP as logical 'timestamp-millis': written as long denoting milliseconds since Unix epoch. If a reader of written Avro records also knows these logical types, then these values can be deserialized with more context depending on reader implementation.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-default-precisionDefault Decimal PrecisionWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'precision' denoting number of available digits is required. Generally, precision is defined by column data type definition or database engines default. However undefined precision (0) can be returned from some database engines. 'Default Decimal Precision' is used when writing those undefined precision numbers.10truefalsetrueENVIRONMENTfalsefalsedbf-default-scaleDefault Decimal ScaleWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'scale' denoting number of available decimal digits is required. Generally, scale is defined by column data type definition or database engines default. However when undefined precision (0) is returned, scale can also be uncertain with some database engines. 'Default Decimal Scale' is used when writing those undefined numbers. If a value has more decimals than specified scale, then the value will be rounded-up, e.g. 1.53 becomes 2 with scale 0, and 1.5 with scale 1.0truefalsetrueENVIRONMENTfalsefalseinitial.maxvalue.<max_value_column>Initial maximum value for the specified columnSpecifies an initial max value for max value column(s). Properties should be added in the format `initial.maxvalue.<max_value_column>`. This value is only used the first time the table is accessed (when a Maximum Value Column is specified).ENVIRONMENTsuccessSuccessfully created FlowFile from SQL query result set.falsetablenameName of the table being queriedquerydbtable.row.countThe number of rows selected by the queryfragment.identifierIf 'Max Rows Per Flow File' is set then all FlowFiles from the same query result set will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.fragment.countIf 'Max Rows Per Flow File' is set then this is the total number of FlowFiles produced by a single ResultSet. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming ResultSet. If Output Batch Size is set, then this attribute will not be populated.fragment.indexIf 'Max Rows Per Flow File' is set then the position of this FlowFile in the list of outgoing FlowFiles that were all derived from the same result set FlowFile. This can be used in conjunction with the fragment.identifier attribute to know which FlowFiles originated from the same query result set and in what order FlowFiles were producedmaxvalue.*Each attribute contains the observed maximum value of a specified 'Maximum-value Column'. The suffix of the attribute is the name of the column. If Output Batch Size is set, then this attribute will not be populated.mime.typeSets the mime.type attribute to the MIME Type specified by the Record Writer.record.countThe number of records output by the Record Writer.truetrueAfter performing a query on the specified table, the maximum values for the specified column(s) will be retained for use in future executions of the query. This allows the Processor to fetch only those records that have max values greater than the retained values. This can be used for incremental fetching, fetching of newly added rows, etc. To clear the maximum values, clear the state of the processor per the State Management documentationCLUSTERINPUT_FORBIDDENRetrieve all rows from a database table.jdbcrdbmscdcdatabasetablestreamINPUT_ALLOWEDConfigure the "Database Connection Pooling Service" to specify a Connection Pooling Service so that the Processor knows how to connect to the database. +org.apache.nifi.processors.standard.ConvertRecordorg.apache.nifi.processors.standard.SplitRecordorg.apache.nifi.processors.standard.UpdateRecordorg.apache.nifi.processors.standard.QueryRecordorg.apache.nifi.processors.standard.PutDatabaseRecordPROCESSORThe PutDatabaseRecord processor uses a specified RecordReader to input (possibly multiple) records from an incoming flow file. These records are translated to SQL statements and executed as a single transaction. If any errors occur, the flow file is routed to failure or retry, and if the records are transmitted successfully, the incoming flow file is routed to success. The type of statement executed by the processor is specified via the Statement Type property, which accepts some hard-coded values such as INSERT, UPDATE, and DELETE, as well as 'Use statement.type Attribute', which causes the processor to get the statement type from a flow file attribute. IMPORTANT: If the Statement Type is UPDATE, then the incoming records must not alter the value(s) of the primary keys (or user-specified Update Keys). If such records are encountered, the UPDATE statement issued to the database may do nothing (if no existing records with the new primary key values are found), or could inadvertently corrupt the existing data (by changing records for which the new values of the primary keys exist).sqlrecordjdbcputdatabaseupdateinsertdeleteput-db-record-record-readerRecord ReaderSpecifies the Controller Service to use for parsing incoming data and determining the data's schema.org.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLfalsefalsefalseNONEfalsefalseput-db-record-statement-typeStatement TypeSpecifies the type of SQL Statement to generate. Please refer to the database documentation for a description of the behavior of each operation. Please note that some Database Types may not support certain Statement Types. If 'Use statement.type Attribute' is chosen, then the value is taken from the statement.type attribute in the FlowFile. The 'Use statement.type Attribute' option is the only one that allows the 'SQL' statement type. If 'SQL' is specified, the value of the field specified by the 'Field Containing SQL' property is expected to be a valid SQL statement on the target database, and will be executed as-is.UPDATEUPDATEINSERTINSERTUPSERTUPSERTINSERT_IGNOREINSERT_IGNOREDELETEDELETEUse statement.type AttributeUse statement.type AttributeUse Record PathUse Record PathtruefalsefalseNONEfalsefalseStatement Type Record PathStatement Type Record PathSpecifies a RecordPath to evaluate against each Record in order to determine the Statement Type. The RecordPath should equate to either INSERT, UPDATE, UPSERT, or DELETE. (Debezium style operation types are also supported: "r" and "c" for INSERT, "u" for UPDATE, and "d" for DELETE)truefalsefalseNONEfalsefalseput-db-record-statement-typeStatement TypeUse Record PathData Record PathData Record PathIf specified, this property denotes a RecordPath that will be evaluated against each incoming Record and the Record that results from evaluating the RecordPath will be sent to the database instead of sending the entire incoming Record. If not specified, the entire incoming Record will be published to the database.falsefalsefalseNONEfalsefalseput-db-record-dcbp-serviceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database for sending records.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseput-db-record-catalog-nameCatalog NameThe name of the catalog that the statement should update. This may not apply for the database that you are updating. In this case, leave the field empty. Note that if the property is set and the database is case-sensitive, the catalog name must match the database's catalog name exactly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-schema-nameSchema NameThe name of the schema that the table belongs to. This may not apply for the database that you are updating. In this case, leave the field empty. Note that if the property is set and the database is case-sensitive, the schema name must match the database's schema name exactly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-table-nameTable NameThe name of the table that the statement should affect. Note that if the database is case-sensitive, the table name must match the database's table name exactly.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-binary-formatBinary String FormatThe format to be applied when decoding string values to binary.UTF-8UTF-8UTF-8String values for binary columns contain the original value as text via UTF-8 character encodingHexadecimalHexadecimalString values for binary columns contain the original value in hexadecimal formatBase64Base64String values for binary columns contain the original value in Base64 encoded formattruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-translate-field-namesTranslate Field NamesIf true, the Processor will attempt to translate field names into the appropriate column names for the table specified. If false, the field names must match the column names exactly, or the column will not be updatedtruetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-db-record-unmatched-field-behaviorUnmatched Field BehaviorIf an incoming record has a field that does not map to any of the database table's columns, this property specifies how to handle the situationIgnore Unmatched FieldsIgnore Unmatched FieldsIgnore Unmatched FieldsAny field in the document that cannot be mapped to a column in the database is ignoredFail on Unmatched FieldsFail on Unmatched FieldsIf the document has any field that cannot be mapped to a column in the database, the FlowFile will be routed to the failure relationshipfalsefalsefalseNONEfalsefalseput-db-record-unmatched-column-behaviorUnmatched Column BehaviorIf an incoming record does not have a field mapping for all of the database table's columns, this property specifies how to handle the situationFail on Unmatched ColumnsIgnore Unmatched ColumnsIgnore Unmatched ColumnsAny column in the database that does not have a field in the document will be assumed to not be required. No notification will be loggedWarn on Unmatched ColumnsWarn on Unmatched ColumnsAny column in the database that does not have a field in the document will be assumed to not be required. A warning will be loggedFail on Unmatched ColumnsFail on Unmatched ColumnsA flow will fail if any column in the database that does not have a field in the document. An error will be loggedfalsefalsefalseNONEfalsefalseput-db-record-update-keysUpdate KeysA comma-separated list of column names that uniquely identifies a row in the database for UPDATE statements. If the Statement Type is UPDATE and this property is not set, the table's Primary Keys are used. In this case, if no Primary Key exists, the conversion to SQL will fail if Unmatched Column Behaviour is set to FAIL. This property is ignored if the Statement Type is INSERTfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-statement-typeStatement TypeUse statement.type AttributeUse Record PathUPSERTUPDATESQLput-db-record-field-containing-sqlField Containing SQLIf the Statement Type is 'SQL' (as set in the statement.type attribute), this field indicates which field in the record(s) contains the SQL statement to execute. The value of the field must be a single SQL statement. If the Statement Type is not 'SQL', this field is ignored.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-statement-typeStatement TypeUse statement.type AttributeUse Record Pathput-db-record-allow-multiple-statementsAllow Multiple SQL StatementsIf the Statement Type is 'SQL' (as set in the statement.type attribute), this field indicates whether to split the field value by a semicolon and execute each statement separately. If any statement causes an error, the entire set of statements will be rolled back. If the Statement Type is not 'SQL', this field is ignored.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseput-db-record-statement-typeStatement TypeUse statement.type AttributeUse Record Pathput-db-record-quoted-identifiersQuote Column IdentifiersEnabling this option will cause all column names to be quoted, allowing you to use reserved words as column names in your tables.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-db-record-quoted-table-identifiersQuote Table IdentifiersEnabling this option will cause the table name to be quoted to support the use of special characters in the table name.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-db-record-query-timeoutMax Wait TimeThe maximum amount of time allowed for a running SQL statement , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueENVIRONMENTfalsefalserollback-on-failureRollback On FailureSpecify how to handle error. By default (false), if an error occurs while processing a FlowFile, the FlowFile will be routed to 'failure' or 'retry' relationship based on error type, and processor can continue with next FlowFile. Instead, you may want to rollback currently processed FlowFiles and stop further processing immediately. In that case, you can do so by enabling this 'Rollback On Failure' property. If enabled, failed FlowFiles will stay in the input relationship without penalizing it and being processed repeatedly until it gets processed successfully or removed by other means. It is important to set adequate 'Yield Duration' to avoid retrying too frequently.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsetable-schema-cache-sizeTable Schema Cache SizeSpecifies how many Table Schemas should be cached100truefalsefalseNONEfalsefalseput-db-record-max-batch-sizeMaximum Batch SizeSpecifies maximum number of sql statements to be included in each batch sent to the database. Zero means the batch size is not limited, and all statements are put into a single batch which can cause high memory usage issues for a very large number of statements.1000falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedatabase-session-autocommitDatabase Session AutoCommitThe autocommit mode to set on the database connection being used. If set to false, the operation(s) will be explicitly committed or rolled back (based on success or failure respectively). If set to true, the driver/database automatically handles the commit/rollback.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseretryA FlowFile is routed to this relationship if the database cannot be updated but attempting the operation again may succeedfalsesuccessSuccessfully created FlowFile from SQL query result set.falsefailureA FlowFile is routed to this relationship if the database cannot be updated and retrying the operation will also fail, such as an invalid query or an integrity constraint violationfalsestatement.typeIf 'Use statement.type Attribute' is selected for the Statement Type property, the value of this attribute will be used to determine the type of statement (INSERT, UPDATE, DELETE, SQL, etc.) to generate and execute.putdatabaserecord.errorIf an error occurs during processing, the flow file will be routed to failure or retry, and this attribute will be populated with the cause of the error.INPUT_REQUIREDInsert records into a databaseINPUT_ALLOWEDorg.apache.nifi.processors.standard.PutDistributedMapCachePROCESSORGets the content of a FlowFile and puts it to a distributed map cache, using a cache key computed from FlowFile attributes. If the cache already contains the entry and the cache update strategy is 'keep original' the entry is not replaced.'mapcacheputdistributedCache Entry IdentifierCache Entry IdentifierA FlowFile attribute, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the cache keytruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseDistributed Cache ServiceDistributed Cache ServiceThe Controller Service that is used to cache flow filesorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseCache update strategyCache update strategyDetermines how the cache is updated if the cache already contains the entryreplaceReplace if presentreplaceAdds the specified entry to the cache, replacing any value that is currently set.Keep originalkeeporiginalAdds the specified entry to the cache, if the key does not exist.truefalsefalseNONEfalsefalseMax cache entry sizeMax cache entry sizeThe maximum amount of data to put into cache1 MBfalsefalsefalseNONEfalsefalsesuccessAny FlowFile that is successfully inserted into cache will be routed to this relationshipfalsefailureAny FlowFile that cannot be inserted into the cache will be routed to this relationshipfalsecachedAll FlowFiles will have an attribute 'cached'. The value of this attribute is true, is the FlowFile is cached, otherwise false.trueINPUT_REQUIREDorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientServiceorg.apache.nifi.distributed.cache.server.map.DistributedMapCacheServerorg.apache.nifi.processors.standard.FetchDistributedMapCacheorg.apache.nifi.processors.standard.PutEmailPROCESSORSends an e-mail to configured recipients for each incoming FlowFileemailputnotifysmtpSMTP HostnameSMTP HostnameThe hostname of the SMTP hosttruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP PortSMTP PortThe Port used for SMTP communications25truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseauthorization-modeAuthorization ModeHow to authorize sending email on the user's behalf.password-based-authorization-modeUse Passwordpassword-based-authorization-modeUse passwordUse OAuth2oauth-based-authorization-modeUse OAuth2 to acquire access tokentruefalsefalseNONEfalsefalseoauth2-access-token-providerOAuth2 Access Token ProviderOAuth2 service that can provide access tokens.org.apache.nifi.oauth2.OAuth2AccessTokenProviderorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseauthorization-modeAuthorization Modeoauth-based-authorization-modeSMTP UsernameSMTP UsernameUsername for the SMTP accountfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP PasswordSMTP PasswordPassword for the SMTP accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseauthorization-modeAuthorization Modepassword-based-authorization-modeSMTP AuthSMTP AuthFlag indicating whether authentication should be usedtruetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP TLSSMTP STARTTLSFlag indicating whether Opportunistic TLS should be enabled using STARTTLS commandfalsetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP Socket FactorySMTP Socket FactorySocket Factory to use for SMTP Connectionjavax.net.ssl.SSLSocketFactorytruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP X-Mailer HeaderSMTP X-Mailer HeaderX-Mailer used in the header of the outgoing emailNiFitruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseattribute-name-regexAttributes to Send as Headers (Regex)A Regular Expression that is matched against all FlowFile attribute names. Any attribute whose name matches the regex will be added to the Email messages as a Header. If not specified, no FlowFile attributes will be added as headers.falsefalsefalseNONEfalsefalseContent TypeContent TypeMime Type used to interpret the contents of the email, such as text/plain or text/htmltext/plaintruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFromFromSpecifies the Email address to use as the sender. Comma separated sequence of addresses following RFC822 syntax.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseToToThe recipients to include in the To-Line of the email. Comma separated sequence of addresses following RFC822 syntax.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCCCCThe recipients to include in the CC-Line of the email. Comma separated sequence of addresses following RFC822 syntax.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseBCCBCCThe recipients to include in the BCC-Line of the email. Comma separated sequence of addresses following RFC822 syntax.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSubjectSubjectThe email subjectMessage from NiFitruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessageMessageThe body of the email messagefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseemail-ff-content-as-messageFlow file content as messageSpecifies whether or not the FlowFile content should be the message of the email. If true, the 'Message' property is ignored.falsetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseinput-character-setInput Character SetSpecifies the character set of the FlowFile contents for reading input FlowFile contents to generate the message body or as an attachment to the message. If not set, UTF-8 will be the default value.UTF-8truefalsefalseNONEfalsefalseAttach FileAttach FileSpecifies whether or not the FlowFile content should be attached to the emailfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseInclude All Attributes In MessageInclude All Attributes In MessageSpecifies whether or not all FlowFile attributes should be recorded in the body of the email messagefalsetruetruefalsefalsetruefalsefalseNONEfalsefalsemail.propertyNameValue for a specific property to be set in the JavaMail Session objectDynamic property names that will be passed to the Mail session. Possible properties can be found in: https://javaee.github.io/javamail/docs/api/com/sun/mail/smtp/package-summary.html.FLOWFILE_ATTRIBUTEStruesuccessFlowFiles that are successfully sent will be routed to this relationshipfalsefailureFlowFiles that fail to send will be routed to this relationshipfalsetrueINPUT_REQUIREDMEMORYThe entirety of the FlowFile's content (as a String object) will be read into memory in case the property to use the flow file content as the email body is set to true.org.apache.nifi.processors.standard.PutFTPPROCESSORSends FlowFiles to an FTP ServerremotecopyegressputftparchivefilesHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers21truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push filesfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCreate DirectoryCreate DirectorySpecifies whether or not the remote directory should be created if it does not exist.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseBatch SizeBatch SizeThe maximum number of FlowFiles to send in a single connection500truefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseConflict ResolutionConflict ResolutionDetermines how to handle the problem of filename collisionsNONEREPLACEREPLACERemote file is replaced with new file, FlowFile goes to successIGNOREIGNOREFile is not transferred, FlowFile goes to successRENAMERENAMENew file is renamed with a one-up number at the beginning, FlowFile goes to successREJECTREJECTFile is not transferred, FlowFile goes to rejectFAILFAILFile is not transferred, FlowFile goes to failureNONENONEDo not check for conflict before transfer, FlowFile goes to success or failuretruefalsefalseNONEfalsefalseDot RenameDot RenameIf true, then the filename of the sent file is prepended with a "." and then renamed back to the original once the file is completely sent. Otherwise, there is no rename. This property is ignored if the Temporary Filename property is set.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseTemporary FilenameTemporary FilenameIf set, the filename of the sent file will be equal to the value specified during the transfer and after successful completion will be renamed to the original filename. If this value is set, the Dot Rename property is ignored.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseTransfer ModeTransfer ModeThe FTP Transfer ModeBinaryBinaryBinaryASCIIASCIIfalsefalsefalseNONEfalsefalseConnection ModeConnection ModeThe FTP Connection ModePassiveActiveActivePassivePassivefalsefalsefalseNONEfalsefalseReject Zero-Byte FilesReject Zero-Byte FilesDetermines whether or not Zero-byte files should be rejected without attempting to transfertruetruetruefalsefalsefalsefalsefalseNONEfalsefalseLast Modified TimeLast Modified TimeThe lastModifiedTime to assign to the file after transferring it. If not set, the lastModifiedTime will not be changed. Format must be yyyy-MM-dd'T'HH:mm:ssZ. You may also use expression language such as ${file.lastModifiedTime}. If the value is invalid, the processor will not be invalid but will fail to change lastModifiedTime of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePermissionsPermissionsThe permissions to assign to the file after transferring it. Format must be either UNIX rwxrwxrwx with a - in place of denied permissions (e.g. rw-r--r--) or an octal number (e.g. 644). If not set, the permissions will not be changed. You may also use expression language such as ${file.permissions}. If the value is invalid, the processor will not be invalid but will fail to change permissions of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseInternal Buffer SizeInternal Buffer SizeSet the internal buffer size for buffered data streams16KBfalsefalsefalseNONEfalsefalseftp-use-utf8Use UTF-8 EncodingTells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsepre.cmd._____Not usedThe command specified in the key will be executed before doing a put. You may add these optional properties to send any commands to the FTP server before the file is actually transferred (before the put command). This option is only available for the PutFTP processor, as only FTP has this functionality. This is essentially the same as sending quote commands to an FTP server from the command line. While this is the same as sending a quote command, it is very important that you leave off the .NONEpost.cmd._____Not usedThe command specified in the key will be executed after doing a put. You may add these optional properties to send any commands to the FTP server before the file is actually transferred (before the put command). This option is only available for the PutFTP processor, as only FTP has this functionality. This is essentially the same as sending quote commands to an FTP server from the command line. While this is the same as sending a quote command, it is very important that you leave off the .NONEsuccessFlowFiles that are successfully sent will be routed to successfalserejectFlowFiles that were rejected by the destination systemfalsefailureFlowFiles that failed to send to the remote system; failure is usually looped back to this processorfalsetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.GetFTPorg.apache.nifi.processors.standard.PutFilePROCESSORWrites the contents of a FlowFile to the local file systemputlocalcopyarchivefilesfilesystemDirectoryDirectoryThe directory to which files should be written. You may use expression language such as /aa/bb/${path}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseConflict Resolution StrategyConflict Resolution StrategyIndicates what should happen when a file with the same name already exists in the output directoryfailreplacereplaceignoreignorefailfailtruefalsefalseNONEfalsefalseCreate Missing DirectoriesCreate Missing DirectoriesIf true, then missing destination directories will be created. If false, flowfiles are penalized and sent to failure.truetruetruefalsefalsetruefalsefalseNONEfalsefalseMaximum File CountMaximum File CountSpecifies the maximum number of files that can exist in the output directoryfalsefalsefalseNONEfalsefalseLast Modified TimeLast Modified TimeSets the lastModifiedTime on the output file to the value of this attribute. Format must be yyyy-MM-dd'T'HH:mm:ssZ. You may also use expression language such as ${file.lastModifiedTime}.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePermissionsPermissionsSets the permissions on the output file to the value of this attribute. Format must be either UNIX rwxrwxrwx with a - in place of denied permissions (e.g. rw-r--r--) or an octal number (e.g. 644). You may also use expression language such as ${file.permissions}.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseOwnerOwnerSets the owner on the output file to the value of this attribute. You may also use expression language such as ${file.owner}. Note on many operating systems Nifi must be running as a super-user to have the permissions to set the file owner.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseGroupGroupSets the group on the output file to the value of this attribute. You may also use expression language such as ${file.group}.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessFiles that have been successfully written to the output directory are transferred to this relationshipfalsefailureFiles that could not be written to the output directory for some reason are transferred to this relationshipfalsefilenameThe filename to use when writing the FlowFile to disk.truewrite filesystemProvides operator the ability to write to any file that NiFi has access to.INPUT_REQUIREDorg.apache.nifi.processors.standard.FetchFileorg.apache.nifi.processors.standard.GetFileorg.apache.nifi.processors.standard.PutRecordPROCESSORThe PutRecord processor uses a specified RecordReader to input (possibly multiple) records from an incoming flow file, and sends them to a destination specified by a Record Destination Service (i.e. record sink).recordputsinkput-record-readerRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseput-record-sinkRecord Destination ServiceSpecifies the Controller Service to use for writing out the query result records to some destination.org.apache.nifi.record.sink.RecordSinkServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseput-record-include-zero-record-resultsInclude Zero Record ResultsIf no records are read from the incoming FlowFile, this property specifies whether or not an empty record set will be transmitted. The original FlowFile will still be routed to success, but if no transmission occurs, no provenance SEND event will be generated.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseretryThe original FlowFile is routed to this relationship if the records could not be transmitted but attempting the operation again may succeedfalsesuccessThe original FlowFile will be routed to this relationship if the records were transmitted successfullyfalsefailureA FlowFile is routed to this relationship if the records could not be transmitted and retrying the operation will also failfalseINPUT_REQUIREDorg.apache.nifi.processors.standard.PutSFTPPROCESSORSends FlowFiles to an SFTP ServerremotecopyegressputsftparchivefilesHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers22truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalsePrivate Key PathPrivate Key PathThe fully qualified path to the Private Key filefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILEPrivate Key PassphrasePrivate Key PassphrasePassword for the private keyfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push filesfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCreate DirectoryCreate DirectorySpecifies whether or not the remote directory should be created if it does not exist.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseDisable Directory ListingDisable Directory ListingIf set to 'true', directory listing is not performed prior to create missing directories. By default, this processor executes a directory listing command to see target directory existence before creating missing directories. However, there are situations that you might need to disable the directory listing such as the following. Directory listing might fail with some permission setups (e.g. chmod 100) on a directory. Also, if any other SFTP client created the directory after this processor performed a listing and before a directory creation request by this processor is finished, then an error is returned because the directory already exists.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseBatch SizeBatch SizeThe maximum number of FlowFiles to send in a single connection500truefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseConflict ResolutionConflict ResolutionDetermines how to handle the problem of filename collisionsNONEREPLACEREPLACERemote file is replaced with new file, FlowFile goes to successIGNOREIGNOREFile is not transferred, FlowFile goes to successRENAMERENAMENew file is renamed with a one-up number at the beginning, FlowFile goes to successREJECTREJECTFile is not transferred, FlowFile goes to rejectFAILFAILFile is not transferred, FlowFile goes to failureNONENONEDo not check for conflict before transfer, FlowFile goes to success or failuretruefalsefalseNONEfalsefalseReject Zero-Byte FilesReject Zero-Byte FilesDetermines whether or not Zero-byte files should be rejected without attempting to transfertruetruetruefalsefalsefalsefalsefalseNONEfalsefalseDot RenameDot RenameIf true, then the filename of the sent file is prepended with a "." and then renamed back to the original once the file is completely sent. Otherwise, there is no rename. This property is ignored if the Temporary Filename property is set.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseTemporary FilenameTemporary FilenameIf set, the filename of the sent file will be equal to the value specified during the transfer and after successful completion will be renamed to the original filename. If this value is set, the Dot Rename property is ignored.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHost Key FileHost Key FileIf supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be usedfalsefalsefalseNONEfalsefalseSINGLEFILELast Modified TimeLast Modified TimeThe lastModifiedTime to assign to the file after transferring it. If not set, the lastModifiedTime will not be changed. Format must be yyyy-MM-dd'T'HH:mm:ssZ. You may also use expression language such as ${file.lastModifiedTime}. If the value is invalid, the processor will not be invalid but will fail to change lastModifiedTime of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePermissionsPermissionsThe permissions to assign to the file after transferring it. Format must be either UNIX rwxrwxrwx with a - in place of denied permissions (e.g. rw-r--r--) or an octal number (e.g. 644). If not set, the permissions will not be changed. You may also use expression language such as ${file.permissions}. If the value is invalid, the processor will not be invalid but will fail to change permissions of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseRemote OwnerRemote OwnerInteger value representing the User ID to set on the file after transferring it. If not set, the owner will not be set. You may also use expression language such as ${file.owner}. If the value is invalid, the processor will not be invalid but will fail to change the owner of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseRemote GroupRemote GroupInteger value representing the Group ID to set on the file after transferring it. If not set, the group will not be set. You may also use expression language such as ${file.group}. If the value is invalid, the processor will not be invalid but will fail to change the group of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseStrict Host Key CheckingStrict Host Key CheckingIndicates whether or not strict enforcement of hosts keys should be appliedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseSend Keep Alive On TimeoutSend Keep Alive On TimeoutSend a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.truetruetruefalsefalsetruefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseCiphers AllowedCiphers AllowedA comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctrfalsefalsetrueENVIRONMENTfalsefalseKey Algorithms AllowedKey Algorithms AllowedA comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.comfalsefalsetrueENVIRONMENTfalsefalseKey Exchange Algorithms AllowedKey Exchange Algorithms AllowedA comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-cfalsefalsetrueENVIRONMENTfalsefalseMessage Authentication Codes AllowedMessage Authentication Codes AllowedA comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.comfalsefalsetrueENVIRONMENTfalsefalsesuccessFlowFiles that are successfully sent will be routed to successfalserejectFlowFiles that were rejected by the destination systemfalsefailureFlowFiles that failed to send to the remote system; failure is usually looped back to this processorfalsetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.GetSFTPorg.apache.nifi.processors.standard.PutSQLPROCESSORExecutes a SQL UPDATE or INSERT command. The content of an incoming FlowFile is expected to be the SQL command to execute. The SQL command may use the ? to escape parameters. In this case, the parameters to use must exist as FlowFile attributes with the naming convention sql.args.N.type and sql.args.N.value, where N is a positive integer. The sql.args.N.type is expected to be a number indicating the JDBC Type. The content of the FlowFile is expected to be in UTF-8 format.sqlputrdbmsdatabaseupdateinsertrelationalJDBC Connection PoolJDBC Connection PoolSpecifies the JDBC Connection Pool to use in order to convert the JSON message to a SQL statement. The Connection Pool is necessary in order to determine the appropriate database column types.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseputsql-sql-statementSQL StatementThe SQL statement to execute. The statement can be empty, a constant value, or built from attributes using Expression Language. If this property is specified, it will be used regardless of the content of incoming FlowFiles. If this property is empty, the content of the incoming FlowFile is expected to contain a valid SQL statement, to be issued by the processor to the database.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSupport Fragmented TransactionsSupport Fragmented TransactionsIf true, when a FlowFile is consumed by this Processor, the Processor will first check the fragment.identifier and fragment.count attributes of that FlowFile. If the fragment.count value is greater than 1, the Processor will not process any FlowFile with that fragment.identifier until all are available; at that point, it will process all FlowFiles with that fragment.identifier as a single transaction, in the order specified by the FlowFiles' fragment.index attributes. This Provides atomicity of those SQL statements. Once any statement of this transaction throws exception when executing, this transaction will be rolled back. When transaction rollback happened, none of these FlowFiles would be routed to 'success'. If the <Rollback On Failure> is set true, these FlowFiles will stay in the input relationship. When the <Rollback On Failure> is set false,, if any of these FlowFiles will be routed to 'retry', all of these FlowFiles will be routed to 'retry'.Otherwise, they will be routed to 'failure'. If this value is false, these attributes will be ignored and the updates will occur independent of one another.truetruetruefalsefalsefalsefalsefalseNONEfalsefalsedatabase-session-autocommitDatabase Session AutoCommitThe autocommit mode to set on the database connection being used. If set to false, the operation(s) will be explicitly committed or rolled back (based on success or failure respectively), if set to true the driver/database handles the commit/rollback.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseTransaction TimeoutTransaction TimeoutIf the <Support Fragmented Transactions> property is set to true, specifies how long to wait for all FlowFiles for a particular fragment.identifier attribute to arrive before just transferring all of the FlowFiles with that identifier to the 'failure' relationshipfalsefalsefalseNONEfalsefalseBatch SizeBatch SizeThe preferred number of FlowFiles to put to the database in a single transaction100truefalsefalseNONEfalsefalseObtain Generated KeysObtain Generated KeysIf true, any key that is automatically generated by the database will be added to the FlowFile that generated it using the sql.generate.key attribute. This may result in slightly slower performance and is not supported by all databases.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalserollback-on-failureRollback On FailureSpecify how to handle error. By default (false), if an error occurs while processing a FlowFile, the FlowFile will be routed to 'failure' or 'retry' relationship based on error type, and processor can continue with next FlowFile. Instead, you may want to rollback currently processed FlowFiles and stop further processing immediately. In that case, you can do so by enabling this 'Rollback On Failure' property. If enabled, failed FlowFiles will stay in the input relationship without penalizing it and being processed repeatedly until it gets processed successfully or removed by other means. It is important to set adequate 'Yield Duration' to avoid retrying too frequently.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseretryA FlowFile is routed to this relationship if the database cannot be updated but attempting the operation again may succeedfalsesuccessA FlowFile is routed to this relationship after the database is successfully updatedfalsefailureA FlowFile is routed to this relationship if the database cannot be updated and retrying the operation will also fail, such as an invalid query or an integrity constraint violationfalsefragment.identifierIf the <Support Fragment Transactions> property is true, this attribute is used to determine whether or not two FlowFiles belong to the same transaction.fragment.countIf the <Support Fragment Transactions> property is true, this attribute is used to determine how many FlowFiles are needed to complete the transaction.fragment.indexIf the <Support Fragment Transactions> property is true, this attribute is used to determine the order that the FlowFiles in a transaction should be evaluated.sql.args.N.typeIncoming FlowFiles are expected to be parametrized SQL statements. The type of each Parameter is specified as an integer that represents the JDBC Type of the parameter.sql.args.N.valueIncoming FlowFiles are expected to be parametrized SQL statements. The value of the Parameters are specified as sql.args.1.value, sql.args.2.value, sql.args.3.value, and so on. The type of the sql.args.1.value Parameter is specified by the sql.args.1.type attribute.sql.args.N.formatThis attribute is always optional, but default options may not always work for your data. Incoming FlowFiles are expected to be parametrized SQL statements. In some cases a format option needs to be specified, currently this is only applicable for binary data types, dates, times and timestamps. Binary Data Types (defaults to 'ascii') - ascii: each string character in your attribute value represents a single byte. This is the format provided by Avro Processors. base64: the string is a Base64 encoded string that can be decoded to bytes. hex: the string is hex encoded with all letters in upper case and no '0x' at the beginning. Dates/Times/Timestamps - Date, Time and Timestamp formats all support both custom formats or named format ('yyyy-MM-dd','ISO_OFFSET_DATE_TIME') as specified according to java.time.format.DateTimeFormatter. If not specified, a long value input is expected to be an unix epoch (milli seconds from 1970/1/1), or a string value in 'yyyy-MM-dd' format for Date, 'HH:mm:ss.SSS' for Time (some database engines e.g. Derby or MySQL do not support milliseconds and will truncate milliseconds), 'yyyy-MM-dd HH:mm:ss.SSS' for Timestamp is used.sql.generated.keyIf the database generated a key for an INSERT statement and the Obtain Generated Keys property is set to true, this attribute will be added to indicate the generated key, if possible. This feature is not supported by all database vendors.trueINPUT_REQUIREDorg.apache.nifi.processors.standard.ConvertJSONToSQLorg.apache.nifi.processors.standard.PutSyslogPROCESSORSends Syslog messages to a given host and port over TCP or UDP. Messages are constructed from the "Message ___" properties of the processor which can use expression language to generate messages from incoming FlowFiles. The properties are used to construct messages of the form: (<PRIORITY>)(VERSION )(TIMESTAMP) (HOSTNAME) (BODY) where version is optional. The constructed messages are checked against regular expressions for RFC5424 and RFC3164 formatted messages. The timestamp can be an RFC5424 timestamp with a format of "yyyy-MM-dd'T'HH:mm:ss.SZ" or "yyyy-MM-dd'T'HH:mm:ss.S+hh:mm", or it can be an RFC3164 timestamp with a format of "MMM d HH:mm:ss". If a message is constructed that does not form a valid Syslog message according to the above description, then it is routed to the invalid relationship. Valid messages are sent to the Syslog server and successes are routed to the success relationship, failures routed to the failure relationship.syslogputudptcplogsHostnameHostnameThe IP address or hostname of the Syslog server.localhosttruefalsetrueENVIRONMENTfalsefalseProtocolProtocolThe protocol for Syslog communication.UDPTCPTCPUDPUDPtruefalsefalseNONEfalsefalsePortPortThe port for Syslog communication. Note that Expression language is not evaluated per FlowFile.truefalsetrueENVIRONMENTfalsefalseMax Size of Socket Send BufferMax Size of Socket Send BufferThe maximum size of the socket send buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsetrueENVIRONMENTfalsefalseSSL Context ServiceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, syslog messages will be sent over a secure connection.org.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProtocolProtocolTCPIdle Connection ExpirationIdle Connection ExpirationThe amount of time a connection should be held open without being used before closing the connection.5 secondstruefalsetrueENVIRONMENTfalsefalseTimeoutTimeoutThe timeout for connecting to and communicating with the syslog server. Does not apply to UDP. Note that Expression language is not evaluated per FlowFile.10 secondsfalsefalsetrueENVIRONMENTfalsefalseBatch SizeBatch SizeThe number of incoming FlowFiles to process in a single execution of this processor.25truefalsetrueENVIRONMENTfalsefalseCharacter SetCharacter SetSpecifies the character set of the Syslog messages. Note that Expression language is not evaluated per FlowFile.UTF-8truefalsetrueENVIRONMENTfalsefalseMessage PriorityMessage PriorityThe priority for the Syslog messages, excluding < >.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessage VersionMessage VersionThe version for the Syslog messages.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessage TimestampMessage TimestampThe timestamp for the Syslog messages. The timestamp can be an RFC5424 timestamp with a format of "yyyy-MM-dd'T'HH:mm:ss.SZ" or "yyyy-MM-dd'T'HH:mm:ss.S+hh:mm", " or it can be an RFC3164 timestamp with a format of "MMM d HH:mm:ss".${now():format('MMM d HH:mm:ss')}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessage HostnameMessage HostnameThe hostname for the Syslog messages.${hostname(true)}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessage BodyMessage BodyThe body for the Syslog messages.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessFlowFiles that are sent successfully to Syslog are sent out this relationship.falsefailureFlowFiles that failed to send to Syslog are sent out this relationship.falseinvalidFlowFiles that do not form a valid Syslog message are sent out this relationship.falsetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ListenSyslogorg.apache.nifi.processors.standard.ParseSyslogorg.apache.nifi.processors.standard.PutTCPPROCESSORSends serialized FlowFiles or Records over TCP to a configurable destination with optional support for TLSremoteegressputtcpHostnameHostnameDestination hostname or IP addresslocalhosttruefalsetrueENVIRONMENTfalsefalsePortPortDestination port numbertruefalsetrueENVIRONMENTfalsefalseMax Size of Socket Send BufferMax Size of Socket Send BufferThe maximum size of the socket send buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseIdle Connection ExpirationIdle Connection ExpirationThe amount of time a connection should be held open without being used before closing the connection. A value of 0 seconds will disable this feature.15 secondstruefalsetrueENVIRONMENTfalsefalseTimeoutTimeoutThe timeout for connecting to and communicating with the destination. Does not apply to UDP10 secondsfalsefalsetrueENVIRONMENTfalsefalseConnection Per FlowFileConnection Per FlowFileSpecifies whether to send each FlowFile's content on an individual connection.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceSpecifies the SSL Context Service to enable TLS socket communicationorg.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseTransmission StrategyTransmission StrategySpecifies the strategy used for reading input FlowFiles and transmitting messages to the destination socket addressFLOWFILE_ORIENTEDFlowFile-orientedFLOWFILE_ORIENTEDSend FlowFile content as a single streamRecord-orientedRECORD_ORIENTEDRead Records from input FlowFiles and send serialized Records as individual messagestruefalsefalseNONEfalsefalseOutgoing Message DelimiterOutgoing Message DelimiterSpecifies the delimiter to use when sending messages out over the same TCP stream. The delimiter is appended to each FlowFile message that is transmitted over the stream so that the receiver can determine when one message ends and the next message begins. Users should ensure that the FlowFile content does not contain the delimiter character to avoid errors. In order to use a new line character you can enter '\n'. For a tab character use '\t'. Finally for a carriage return use '\r'.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseTransmission StrategyTransmission StrategyFLOWFILE_ORIENTEDCharacter SetCharacter SetSpecifies the character set of the data being sent.UTF-8truefalsetrueENVIRONMENTfalsefalseTransmission StrategyTransmission StrategyFLOWFILE_ORIENTEDRecord ReaderRecord ReaderSpecifies the Controller Service to use for reading Records from input FlowFilesorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseTransmission StrategyTransmission StrategyRECORD_ORIENTEDRecord WriterRecord WriterSpecifies the Controller Service to use for writing Records to the configured socket addressorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseTransmission StrategyTransmission StrategyRECORD_ORIENTEDsuccessFlowFiles that are sent successfully to the destination are sent out this relationship.falsefailureFlowFiles that failed to send to the destination are sent out this relationship.falserecord.count.transmittedCount of records transmitted to configured destination addresstrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ListenTCPorg.apache.nifi.processors.standard.PutUDPorg.apache.nifi.processors.standard.PutUDPPROCESSORThe PutUDP processor receives a FlowFile and packages the FlowFile content into a single UDP datagram packet which is then transmitted to the configured UDP server. The user must ensure that the FlowFile content being fed to this processor is not larger than the maximum size for the underlying UDP transport. The maximum transport size will vary based on the platform setup but is generally just under 64KB. FlowFiles will be marked as failed if their content is larger than the maximum transport size.remoteegressputudpHostnameHostnameDestination hostname or IP addresslocalhosttruefalsetrueENVIRONMENTfalsefalsePortPortDestination port numbertruefalsetrueENVIRONMENTfalsefalseMax Size of Socket Send BufferMax Size of Socket Send BufferThe maximum size of the socket send buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseIdle Connection ExpirationIdle Connection ExpirationThe amount of time a connection should be held open without being used before closing the connection. A value of 0 seconds will disable this feature.15 secondstruefalsetrueENVIRONMENTfalsefalseTimeoutTimeoutThe timeout for connecting to and communicating with the destination. Does not apply to UDP10 secondsfalsefalsetrueENVIRONMENTfalsefalsesuccessFlowFiles that are sent successfully to the destination are sent out this relationship.falsefailureFlowFiles that failed to send to the destination are sent out this relationship.falsetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ListenUDPorg.apache.nifi.processors.standard.PutTCPorg.apache.nifi.processors.standard.QueryDatabaseTablePROCESSORGenerates a SQL select query, or uses a provided statement, and executes it to fetch all rows whose values in the specified Maximum Value column(s) are larger than the previously-seen maxima. Query result will be converted to Avro format. Expression Language is supported for several properties, but no incoming connections are permitted. The Environment/System properties may be used to provide values for any property containing Expression Language. If it is desired to leverage flow file attributes to perform these queries, the GenerateTableFetch and/or ExecuteSQL processors can be used for this purpose. Streaming is used so arbitrarily large result sets are supported. This processor can be scheduled to run on a timer or cron expression, using the standard scheduling methods. This processor is intended to be run on the Primary Node only. FlowFile attribute 'querydbtable.row.count' indicates how many rows were selected.sqlselectjdbcquerydatabaseDatabase Connection Pooling ServiceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-fetch-db-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLtruefalsefalseNONEfalsefalseTable NameTable NameThe name of the database table to be queried. When a custom query is used, this property is used to alias the query and appears as an attribute on the FlowFile.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseColumns to ReturnColumns to ReturnA comma-separated list of column names to be used in the query. If your database requires special treatment of the names (quoting, e.g.), each name should include such treatment. If no column names are supplied, all columns in the specified table will be returned. NOTE: It is important to use consistent column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-where-clauseAdditional WHERE clauseA custom clause to be added in the WHERE condition when building SQL queries.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-sql-queryCustom QueryA custom SQL query used to retrieve data. Instead of building a SQL query from other properties, this query will be wrapped as a sub-query. Query must have no ORDER BY statement.falsefalsetrueENVIRONMENTfalsefalseMaximum-value ColumnsMaximum-value ColumnsA comma-separated list of column names. The processor will keep track of the maximum value for each column that has been returned since the processor started running. Using multiple columns implies an order to the column list, and each column's values are expected to increase more slowly than the previous columns' values. Thus, using multiple columns implies a hierarchical structure of columns, which is usually used for partitioning tables. This processor can be used to retrieve only those rows that have been added/updated since the last retrieval. Note that some JDBC types such as bit/boolean are not conducive to maintaining maximum value, so columns of these types should not be listed in this property, and will result in error(s) during processing. If no columns are provided, all rows from the table will be considered, which could have a performance impact. NOTE: It is important to use consistent max-value column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseinitial-load-strategyInitial Load StrategyHow to handle existing rows in the database table when the processor is started for the first time (or its state has been cleared). The property will be ignored, if any 'initial.maxvalue.*' dynamic property has also been configured.Start at BeginningStart at BeginningStart at BeginningLoads all existing rows from the database table.Start at Current Maximum ValuesStart at Current Maximum ValuesLoads only the newly inserted or updated rows based on the maximum value(s) of the column(s) configured in the 'Maximum-value Columns' property.truefalsefalseNONEfalsefalseMax Wait TimeMax Wait TimeThe maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFetch SizeFetch SizeThe number of result rows to be fetched from the result set at a time. This is a hint to the database driver and may not be honored and/or exact. If the value specified is zero, then the hint is ignored. If using PostgreSQL, then 'Set Auto Commit' must be equal to 'false' to cause 'Fetch Size' to take effect.0truefalsetrueENVIRONMENTfalsefalseSet Auto CommitSet Auto CommitAllows enabling or disabling the auto commit functionality of the DB connection. Default value is 'No value set'. 'No value set' will leave the db connection's auto commit mode unchanged. For some JDBC drivers such as PostgreSQL driver, it is required to disable the auto commit functionality to get the 'Fetch Size' setting to take effect. When auto commit is enabled, PostgreSQL driver ignores 'Fetch Size' setting and loads all rows of the result set to memory at once. This could lead for a large amount of memory usage when executing queries which fetch large data sets. More Details of this behaviour in PostgreSQL driver can be found in https://jdbc.postgresql.org//documentation/head/query.html.truetruefalsefalsefalsefalsetrueENVIRONMENTfalsefalseqdbt-max-rowsMax Rows Per Flow FileThe maximum number of result rows that will be included in a single FlowFile. This will allow you to break up very large result sets into multiple FlowFiles. If the value specified is zero, then all rows are returned in a single FlowFile.0truefalsetrueENVIRONMENTfalsefalseqdbt-output-batch-sizeOutput Batch SizeThe number of output FlowFiles to queue before committing the process session. When set to zero, the session will be committed when all result set rows have been processed and the output FlowFiles are ready for transfer to the downstream relationship. For large result sets, this can cause a large burst of FlowFiles to be transferred at the end of processor execution. If this property is set, then when the specified number of FlowFiles are ready for transfer, then the session will be committed, thus releasing the FlowFiles to the downstream relationship. NOTE: The maxvalue.* and fragment.count attributes will not be set on FlowFiles when this property is set.0truefalsetrueENVIRONMENTfalsefalseqdbt-max-fragsMaximum Number of FragmentsThe maximum number of fragments. If the value specified is zero, then all fragments are returned. This prevents OutOfMemoryError when this processor ingests huge table. NOTE: Setting this property can result in data loss, as the incoming results are not ordered, and fragments may end at arbitrary boundaries where rows are not included in the result set.0truefalsetrueENVIRONMENTfalsefalsedbf-normalizeNormalize Table/Column NamesWhether to change non-Avro-compatible characters in column names to Avro-compatible characters. For example, colons and periods will be changed to underscores in order to build a valid Avro record.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsetransaction-isolation-levelTransaction Isolation LevelThis setting will set the transaction isolation level for the database connection for drivers that support this settingTRANSACTION_NONE0TRANSACTION_READ_COMMITTED2TRANSACTION_READ_UNCOMMITTED1TRANSACTION_REPEATABLE_READ4TRANSACTION_SERIALIZABLE8falsefalsefalseNONEfalsefalsedbf-user-logical-typesUse Avro Logical TypesWhether to use Avro Logical Types for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP columns. If disabled, written as string. If enabled, Logical types are used and written as its underlying type, specifically, DECIMAL/NUMBER as logical 'decimal': written as bytes with additional precision and scale meta data, DATE as logical 'date-millis': written as int denoting days since Unix epoch (1970-01-01), TIME as logical 'time-millis': written as int denoting milliseconds since Unix epoch, and TIMESTAMP as logical 'timestamp-millis': written as long denoting milliseconds since Unix epoch. If a reader of written Avro records also knows these logical types, then these values can be deserialized with more context depending on reader implementation.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-default-precisionDefault Decimal PrecisionWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'precision' denoting number of available digits is required. Generally, precision is defined by column data type definition or database engines default. However undefined precision (0) can be returned from some database engines. 'Default Decimal Precision' is used when writing those undefined precision numbers.10truefalsetrueENVIRONMENTfalsefalsedbf-default-scaleDefault Decimal ScaleWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'scale' denoting number of available decimal digits is required. Generally, scale is defined by column data type definition or database engines default. However when undefined precision (0) is returned, scale can also be uncertain with some database engines. 'Default Decimal Scale' is used when writing those undefined numbers. If a value has more decimals than specified scale, then the value will be rounded-up, e.g. 1.53 becomes 2 with scale 0, and 1.5 with scale 1.0truefalsetrueENVIRONMENTfalsefalseinitial.maxvalue.<max_value_column>Initial maximum value for the specified columnSpecifies an initial max value for max value column(s). Properties should be added in the format `initial.maxvalue.<max_value_column>`. This value is only used the first time the table is accessed (when a Maximum Value Column is specified).ENVIRONMENTsuccessSuccessfully created FlowFile from SQL query result set.falsetablenameName of the table being queriedquerydbtable.row.countThe number of rows selected by the queryfragment.identifierIf 'Max Rows Per Flow File' is set then all FlowFiles from the same query result set will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.fragment.countIf 'Max Rows Per Flow File' is set then this is the total number of FlowFiles produced by a single ResultSet. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming ResultSet. If Output Batch Size is set, then this attribute will not be populated.fragment.indexIf 'Max Rows Per Flow File' is set then the position of this FlowFile in the list of outgoing FlowFiles that were all derived from the same result set FlowFile. This can be used in conjunction with the fragment.identifier attribute to know which FlowFiles originated from the same query result set and in what order FlowFiles were producedmaxvalue.*Each attribute contains the observed maximum value of a specified 'Maximum-value Column'. The suffix of the attribute is the name of the column. If Output Batch Size is set, then this attribute will not be populated.truetrueAfter performing a query on the specified table, the maximum values for the specified column(s) will be retained for use in future executions of the query. This allows the Processor to fetch only those records that have max values greater than the retained values. This can be used for incremental fetching, fetching of newly added rows, etc. To clear the maximum values, clear the state of the processor per the State Management documentationCLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.standard.GenerateTableFetchorg.apache.nifi.processors.standard.ExecuteSQLTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.QueryDatabaseTableRecordPROCESSORGenerates a SQL select query, or uses a provided statement, and executes it to fetch all rows whose values in the specified Maximum Value column(s) are larger than the previously-seen maxima. Query result will be converted to the format specified by the record writer. Expression Language is supported for several properties, but no incoming connections are permitted. The Environment/System properties may be used to provide values for any property containing Expression Language. If it is desired to leverage flow file attributes to perform these queries, the GenerateTableFetch and/or ExecuteSQL processors can be used for this purpose. Streaming is used so arbitrarily large result sets are supported. This processor can be scheduled to run on a timer or cron expression, using the standard scheduling methods. This processor is intended to be run on the Primary Node only. FlowFile attribute 'querydbtable.row.count' indicates how many rows were selected.sqlselectjdbcquerydatabaserecordDatabase Connection Pooling ServiceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-fetch-db-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLtruefalsefalseNONEfalsefalseTable NameTable NameThe name of the database table to be queried. When a custom query is used, this property is used to alias the query and appears as an attribute on the FlowFile.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseColumns to ReturnColumns to ReturnA comma-separated list of column names to be used in the query. If your database requires special treatment of the names (quoting, e.g.), each name should include such treatment. If no column names are supplied, all columns in the specified table will be returned. NOTE: It is important to use consistent column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-where-clauseAdditional WHERE clauseA custom clause to be added in the WHERE condition when building SQL queries.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-sql-queryCustom QueryA custom SQL query used to retrieve data. Instead of building a SQL query from other properties, this query will be wrapped as a sub-query. Query must have no ORDER BY statement.falsefalsetrueENVIRONMENTfalsefalseqdbtr-record-writerRecord WriterSpecifies the Controller Service to use for writing results to a FlowFile. The Record Writer may use Inherit Schema to emulate the inferred schema behavior, i.e. an explicit schema need not be defined in the writer, and will be supplied by the same logic used to infer the schema from the column types.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseMaximum-value ColumnsMaximum-value ColumnsA comma-separated list of column names. The processor will keep track of the maximum value for each column that has been returned since the processor started running. Using multiple columns implies an order to the column list, and each column's values are expected to increase more slowly than the previous columns' values. Thus, using multiple columns implies a hierarchical structure of columns, which is usually used for partitioning tables. This processor can be used to retrieve only those rows that have been added/updated since the last retrieval. Note that some JDBC types such as bit/boolean are not conducive to maintaining maximum value, so columns of these types should not be listed in this property, and will result in error(s) during processing. If no columns are provided, all rows from the table will be considered, which could have a performance impact. NOTE: It is important to use consistent max-value column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseinitial-load-strategyInitial Load StrategyHow to handle existing rows in the database table when the processor is started for the first time (or its state has been cleared). The property will be ignored, if any 'initial.maxvalue.*' dynamic property has also been configured.Start at BeginningStart at BeginningStart at BeginningLoads all existing rows from the database table.Start at Current Maximum ValuesStart at Current Maximum ValuesLoads only the newly inserted or updated rows based on the maximum value(s) of the column(s) configured in the 'Maximum-value Columns' property.truefalsefalseNONEfalsefalseMax Wait TimeMax Wait TimeThe maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFetch SizeFetch SizeThe number of result rows to be fetched from the result set at a time. This is a hint to the database driver and may not be honored and/or exact. If the value specified is zero, then the hint is ignored. If using PostgreSQL, then 'Set Auto Commit' must be equal to 'false' to cause 'Fetch Size' to take effect.0truefalsetrueENVIRONMENTfalsefalseSet Auto CommitSet Auto CommitAllows enabling or disabling the auto commit functionality of the DB connection. Default value is 'No value set'. 'No value set' will leave the db connection's auto commit mode unchanged. For some JDBC drivers such as PostgreSQL driver, it is required to disable the auto commit functionality to get the 'Fetch Size' setting to take effect. When auto commit is enabled, PostgreSQL driver ignores 'Fetch Size' setting and loads all rows of the result set to memory at once. This could lead for a large amount of memory usage when executing queries which fetch large data sets. More Details of this behaviour in PostgreSQL driver can be found in https://jdbc.postgresql.org//documentation/head/query.html.truetruefalsefalsefalsefalsetrueENVIRONMENTfalsefalseqdbt-max-rowsMax Rows Per Flow FileThe maximum number of result rows that will be included in a single FlowFile. This will allow you to break up very large result sets into multiple FlowFiles. If the value specified is zero, then all rows are returned in a single FlowFile.0truefalsetrueENVIRONMENTfalsefalseqdbt-output-batch-sizeOutput Batch SizeThe number of output FlowFiles to queue before committing the process session. When set to zero, the session will be committed when all result set rows have been processed and the output FlowFiles are ready for transfer to the downstream relationship. For large result sets, this can cause a large burst of FlowFiles to be transferred at the end of processor execution. If this property is set, then when the specified number of FlowFiles are ready for transfer, then the session will be committed, thus releasing the FlowFiles to the downstream relationship. NOTE: The maxvalue.* and fragment.count attributes will not be set on FlowFiles when this property is set.0truefalsetrueENVIRONMENTfalsefalseqdbt-max-fragsMaximum Number of FragmentsThe maximum number of fragments. If the value specified is zero, then all fragments are returned. This prevents OutOfMemoryError when this processor ingests huge table. NOTE: Setting this property can result in data loss, as the incoming results are not ordered, and fragments may end at arbitrary boundaries where rows are not included in the result set.0truefalsetrueENVIRONMENTfalsefalseqdbtr-normalizeNormalize Table/Column NamesWhether to change characters in column names when creating the output schema. For example, colons and periods will be changed to underscores.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-user-logical-typesUse Avro Logical TypesWhether to use Avro Logical Types for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP columns. If disabled, written as string. If enabled, Logical types are used and written as its underlying type, specifically, DECIMAL/NUMBER as logical 'decimal': written as bytes with additional precision and scale meta data, DATE as logical 'date-millis': written as int denoting days since Unix epoch (1970-01-01), TIME as logical 'time-millis': written as int denoting milliseconds since Unix epoch, and TIMESTAMP as logical 'timestamp-millis': written as long denoting milliseconds since Unix epoch. If a reader of written Avro records also knows these logical types, then these values can be deserialized with more context depending on reader implementation.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-default-precisionDefault Decimal PrecisionWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'precision' denoting number of available digits is required. Generally, precision is defined by column data type definition or database engines default. However undefined precision (0) can be returned from some database engines. 'Default Decimal Precision' is used when writing those undefined precision numbers.10truefalsetrueENVIRONMENTfalsefalsedbf-default-scaleDefault Decimal ScaleWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'scale' denoting number of available decimal digits is required. Generally, scale is defined by column data type definition or database engines default. However when undefined precision (0) is returned, scale can also be uncertain with some database engines. 'Default Decimal Scale' is used when writing those undefined numbers. If a value has more decimals than specified scale, then the value will be rounded-up, e.g. 1.53 becomes 2 with scale 0, and 1.5 with scale 1.0truefalsetrueENVIRONMENTfalsefalseinitial.maxvalue.<max_value_column>Initial maximum value for the specified columnSpecifies an initial max value for max value column(s). Properties should be added in the format `initial.maxvalue.<max_value_column>`. This value is only used the first time the table is accessed (when a Maximum Value Column is specified).ENVIRONMENTsuccessSuccessfully created FlowFile from SQL query result set.falsetablenameName of the table being queriedquerydbtable.row.countThe number of rows selected by the queryfragment.identifierIf 'Max Rows Per Flow File' is set then all FlowFiles from the same query result set will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.fragment.countIf 'Max Rows Per Flow File' is set then this is the total number of FlowFiles produced by a single ResultSet. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming ResultSet. If Output Batch Size is set, then this attribute will not be populated.fragment.indexIf 'Max Rows Per Flow File' is set then the position of this FlowFile in the list of outgoing FlowFiles that were all derived from the same result set FlowFile. This can be used in conjunction with the fragment.identifier attribute to know which FlowFiles originated from the same query result set and in what order FlowFiles were producedmaxvalue.*Each attribute contains the observed maximum value of a specified 'Maximum-value Column'. The suffix of the attribute is the name of the column. If Output Batch Size is set, then this attribute will not be populated.mime.typeSets the mime.type attribute to the MIME Type specified by the Record Writer.record.countThe number of records output by the Record Writer.truetrueAfter performing a query on the specified table, the maximum values for the specified column(s) will be retained for use in future executions of the query. This allows the Processor to fetch only those records that have max values greater than the retained values. This can be used for incremental fetching, fetching of newly added rows, etc. To clear the maximum values, clear the state of the processor per the State Management documentationCLUSTERINPUT_FORBIDDENRetrieve all rows from a database table.jdbcrdbmscdcdatabasetablestreamINPUT_ALLOWEDConfigure the "Database Connection Pooling Service" to specify a Connection Pooling Service so that the Processor knows how to connect to the database. Set the "Database Type" property to the type of database to query, or "Generic" if the database vendor is not listed. Set the "Table Name" property to the name of the table to retrieve records from. Configure the "Record Writer" to specify a Record Writer that is appropriate for the desired output format.