org.apache.nifinifi-standard-nar2.0.0-M4org.apache.nifinifi-standard-shared-nar2.0.0-M42.0.0-M4nifi-2.0.0-M4-RC1UNKNOWN19c5be0org.apache.nifi.processors.standard.AttributesToCSVPROCESSORGenerates a CSV representation of the input FlowFile Attributes. The resulting CSV can be written to either a newly generated attribute named 'CSVAttributes' or written to the FlowFile as content. If the attribute value contains a comma, newline or double quote, then the attribute value will be escaped with double quotes. Any double quote characters in the attribute value are escaped with another double quote.csvattributesflowfileattribute-listAttribute ListComma separated list of attributes to be included in the resulting CSV. If this value is left empty then all existing Attributes will be included. This list of attributes is case sensitive and supports attribute names that contain commas. If an attribute specified in the list is not found it will be emitted to the resulting CSV with an empty string or null depending on the 'Null Value' property. If a core attribute is specified in this list and the 'Include Core Attributes' property is false, the core attribute will be included. The attribute list ALWAYS wins.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseattributes-regexAttributes Regular ExpressionRegular expression that will be evaluated against the flow file attributes to select the matching attributes. This property can be used in combination with the attributes list property. The final output will contain a combination of matches found in the ATTRIBUTE_LIST and ATTRIBUTE_REGEX.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedestinationDestinationControl if CSV value is written as a new flowfile attribute 'CSVData' or written in the flowfile content.flowfile-attributeflowfile-attributeflowfile-attributeThe resulting CSV string will be placed into a new flowfile attribute named 'CSVData'. The content of the flowfile will not be changed.flowfile-contentflowfile-contentThe resulting CSV string will be placed into the content of the flowfile.Existing flowfile context will be overwritten. 'CSVData' will not be written to at all (neither null nor empty string).truefalsefalseNONEfalsefalseinclude-core-attributesInclude Core AttributesDetermines if the FlowFile org.apache.nifi.flowfile.attributes.CoreAttributes, which are contained in every FlowFile, should be included in the final CSV value generated. Core attributes will be added to the end of the CSVData and CSVSchema strings. The Attribute List property overrides this setting.truetruetruefalsefalsetruefalsefalseNONEfalsefalsenull-valueNull ValueIf true a non existing or empty attribute will be 'null' in the resulting CSV. If false an empty string will be placed in the CSVfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseinclude-schemaInclude SchemaIf true the schema (attribute names) will also be converted to a CSV string which will either be applied to a new attribute named 'CSVSchema' or applied at the first row in the content depending on the DESTINATION property setting.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessSuccessfully converted attributes to CSVfalsefailureFailed to convert attributes to CSVfalseCSVSchemaCSV representation of the SchemaCSVDataCSV representation of AttributestruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.AttributesToJSONPROCESSORGenerates a JSON representation of the input FlowFile Attributes. The resulting JSON can be written to either a new Attribute 'JSONAttributes' or written to the FlowFile as content. Attributes which contain nested JSON objects can either be handled as JSON or as escaped JSON depending on the strategy chosen.jsonattributesflowfileAttributes ListAttributes ListComma separated list of attributes to be included in the resulting JSON. If this value is left empty then all existing Attributes will be included. This list of attributes is case sensitive. If an attribute specified in the list is not found it will be be emitted to the resulting JSON with an empty string or NULL value.falsefalsefalseNONEfalsefalseattributes-to-json-regexAttributes Regular ExpressionRegular expression that will be evaluated against the flow file attributes to select the matching attributes. This property can be used in combination with the attributes list property.falsefalsetrueENVIRONMENTfalsefalseDestinationDestinationControl if JSON value is written as a new flowfile attribute 'JSONAttributes' or written in the flowfile content. Writing to flowfile content will overwrite any existing flowfile content.flowfile-attributeflowfile-attributeflowfile-attributeflowfile-contentflowfile-contenttruefalsefalseNONEfalsefalseInclude Core AttributesInclude Core AttributesDetermines if the FlowFile org.apache.nifi.flowfile.attributes.CoreAttributes which are contained in every FlowFile should be included in the final JSON value generated.truetruetruefalsefalsetruefalsefalseNONEfalsefalseNull ValueNull ValueIf true a non existing selected attribute will be NULL in the resulting JSON. If false an empty string will be placed in the JSONfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseJSON Handling StrategyJSON Handling StrategyStrategy to use for handling attributes which contain nested JSON.ESCAPEDEscapedESCAPEDEscapes JSON attribute values to stringsNestedNESTEDHandles JSON attribute values as nested structured objects or arraystruefalsefalseNONEfalsefalsePretty PrintPretty PrintApply pretty print formatting to the output.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseDestinationDestinationflowfile-contentsuccessSuccessfully converted attributes to JSONfalsefailureFailed to convert attributes to JSONfalseJSONAttributesJSON representation of AttributestruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.CalculateRecordStatsPROCESSORA processor that can count the number of items in a record set, as well as provide counts based on user-defined criteria on subsets of the record set.recordstatsmetricsrecord-stats-readerRecord ReaderA record reader to use for reading the records.org.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-stats-limitrecord-stats-limitLimit the number of individual stats that are returned for each record path to the top N results.10truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseRecord Path propertyThe Record Path valueA Record Path value, pointing to a field to be countedFLOWFILE_ATTRIBUTESsuccessIf a flowfile is successfully processed, it goes here.falsefailureIf a flowfile fails to be processed, it goes here.falserecord.countA count of the records in the record set in the FlowFile.recordStats.<User Defined Property Name>.countA count of the records that contain a value for the user defined property.recordStats.<User Defined Property Name>.<value>.countEach value discovered for the user defined property will have its own count attribute. Total number of top N value counts to be added is defined by the limit configuration.INPUT_REQUIREDorg.apache.nifi.processors.standard.CompressContentPROCESSORCompresses or decompresses the contents of FlowFiles using a user-specified compression algorithm and updates the mime.type attribute as appropriate. A common idiom is to precede CompressContent with IdentifyMimeType and configure Mode='decompress' AND Compression Format='use mime.type attribute'. When used in this manner, the MIME type is automatically detected and the data is decompressed, if necessary. If decompression is unnecessary, the data is passed through to the 'success' relationship. This processor operates in a very memory efficient way so very large objects well beyond the heap size are generally fine to process.contentcompressdecompressgzipbzip2lzmaxz-lzma2snappysnappy-hadoopsnappy framedlz4-frameddeflatezstdbrotliModeModeIndicates whether the processor should compress content or decompress content. Must be either 'compress' or 'decompress'compresscompresscompressdecompressdecompresstruefalsefalseNONEfalsefalseCompression FormatCompression FormatThe compression format to use. Valid values are: GZIP, Deflate, ZSTD, BZIP2, XZ-LZMA2, LZMA, Brotli, Snappy, Snappy Hadoop, Snappy Framed, and LZ4-Frameduse mime.type attributeuse mime.type attributeuse mime.type attributegzipgzipdeflatedeflatebzip2bzip2xz-lzma2xz-lzma2lzmalzmasnappysnappysnappy-hadoopsnappy-hadoopsnappy framedsnappy framedlz4-framedlz4-framedzstdzstdbrotlibrotlitruefalsefalseNONEfalsefalseCompression LevelCompression LevelThe compression level to use; this is valid only when using gzip, deflate or xz-lzma2 compression. A lower value results in faster processing but less compression; a value of 0 indicates no (that is, simple archiving) for gzip or minimal for xz-lzma2 compression. Higher levels can mean much larger memory usage such as the case with levels 7-9 for xz-lzma/2 so be careful relative to heap size.100112233445566778899truefalsefalseNONEfalsefalseModeModecompressCompression FormatCompression Formatzstduse mime.type attributedeflatebrotligzipxz-lzma2Update FilenameUpdate FilenameIf true, will remove the filename extension when decompressing data (only if the extension indicates the appropriate compression format) and add the appropriate extension when compressing datafalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefailureFlowFiles will be transferred to the failure relationship if they fail to compress/decompressfalsesuccessFlowFiles will be transferred to the success relationship after successfully being compressed or decompressedfalsemime.typeIf the Compression Format is set to use mime.type attribute, this attribute is used to determine the compression type. Otherwise, this attribute is ignored.mime.typeIf the Mode property is set to compress, the appropriate MIME Type is set. If the Mode property is set to decompress and the file is successfully decompressed, this attribute is removed, as the MIME Type is no longer known.truetrueINPUT_REQUIREDCPUAn instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.MEMORYAn instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.Compress the contents of a FlowFileINPUT_ALLOWED"Mode" = "compress" "Compression Format" should be set to whichever compression algorithm should be used.Decompress the contents of a FlowFileINPUT_ALLOWED"Mode" = "decompress" "Compression Format" should be set to whichever compression algorithm was used to compress the data previously.Check whether or not a FlowFile is compressed and if so, decompress it.If IdentifyMimeType determines that the content is not compressed, CompressContent will pass the FlowFile along to the 'success' relationship without attempting to decompress it.autodetectmime typecompressdecompressgzipbzip2org.apache.nifi.processors.standard.IdentifyMimeTypeDefault property values are sufficient. Connect the 'success' relationship to CompressContent. org.apache.nifi.processors.standard.CompressContent"Mode" = "decompress" "Compression Format" = "use mime.type attribute" org.apache.nifi.processors.standard.ControlRatePROCESSORControls the rate at which data is transferred to follow-on processors. If you configure a very small Time Duration, then the accuracy of the throttle gets worse. You can improve this accuracy by decreasing the Yield Duration, at the expense of more Tasks given to the processor.rate controlthrottleratethroughputRate Control CriteriaRate Control CriteriaIndicates the criteria that is used to control the throughput rate. Changing this value resets the rate counters.data ratedata ratedata rateRate is controlled by counting bytes transferred per time duration.flowfile countflowfile countRate is controlled by counting FlowFiles transferred per time durationattribute valueattribute valueRate is controlled by accumulating the value of a specified attribute that is transferred per time durationdata rate or flowfile countdata rate or flowfile countRate is controlled by counting bytes and FlowFiles transferred per time duration; if either threshold is met, throttling is enforcedtruefalsefalseNONEfalsefalseTime DurationTime DurationThe amount of time to which the Maximum Rate pertains. Changing this value resets the rate counters.1 mintruefalsefalseNONEfalsefalseMaximum RateMaximum RateThe maximum rate at which data should pass through this processor. The format of this property is expected to be a positive integer, or a Data Size (such as '1 MB') if Rate Control Criteria is set to 'data rate'.falsefalsefalseNONEfalsefalseRate Control CriteriaRate Control Criteriadata rateattribute valueflowfile countMaximum Data RateMaximum Data RateThe maximum rate at which data should pass through this processor. The format of this property is expected to be a Data Size (such as '1 MB') representing bytes per Time Duration.falsefalsefalseNONEfalsefalseRate Control CriteriaRate Control Criteriadata rate or flowfile countMaximum FlowFile RateMaximum FlowFile RateThe maximum rate at which FlowFiles should pass through this processor. The format of this property is expected to be a positive integer representing FlowFiles count per Time DurationfalsefalsefalseNONEfalsefalseRate Control CriteriaRate Control Criteriadata rate or flowfile countRate Exceeded StrategyRate Exceeded StrategySpecifies how to handle an incoming FlowFile when the maximum data rate has been exceeded.Hold FlowFileHold FlowFileHold FlowFileThe FlowFile will be held in its input queue until the rate of data has fallen below the configured maximum and will then be allowed through.Route to 'rate exceeded'Route to 'rate exceeded'The FlowFile will be routed to the 'rate exceeded' Relationship.truefalsefalseNONEfalsefalseRate Controlled AttributeRate Controlled AttributeThe name of an attribute whose values build toward the rate limit if Rate Control Criteria is set to 'attribute value'. The value of the attribute referenced by this property must be a positive long, or the FlowFile will be routed to failure. This value is ignored if Rate Control Criteria is not set to 'attribute value'. Changing this value resets the rate counters.falsefalsefalseNONEfalsefalseRate Control CriteriaRate Control Criteriaattribute valueGrouping AttributeGrouping AttributeBy default, a single "throttle" is used for all FlowFiles. If this value is specified, a separate throttle is used for each value specified by the attribute with this name. Changing this value resets the rate counters.falsefalsefalseNONEfalsefalsefailureFlowFiles will be routed to this relationship if they are missing a necessary Rate Controlled Attribute or the attribute is not in the expected formatfalsesuccessFlowFiles are transferred to this relationship under normal conditionsfalsetruetrueINPUT_REQUIREDLimit the rate at which data is sent to a downstream system with little to no burststhrottlelimitslow downdata rateINPUT_ALLOWEDSet the "Rate Control Criteria" to `data rate`. Set the "Time Duration" property to `1 sec`. Configure the "Maximum Rate" property to specify how much data should be allowed through each second. For example, to allow through 8 MB per second, set "Maximum Rate" to `8 MB`. Limit the rate at which FlowFiles are sent to a downstream system with little to no burststhrottlelimitslow downflowfile rateINPUT_ALLOWEDSet the "Rate Control Criteria" to `flowfile count`. Set the "Time Duration" property to `1 sec`. Configure the "Maximum Rate" property to specify how many FlowFiles should be allowed through each second. For example, to allow through 100 FlowFiles per second, set "Maximum Rate" to `100`. Reject requests that exceed a specific rate with little to no burststhrottlelimitslow downrequest rateINPUT_ALLOWEDSet the "Rate Control Criteria" to `flowfile count`. Set the "Time Duration" property to `1 sec`. Set the "Rate Exceeded Strategy" property to `Route to 'rate exceeded'`. Configure the "Maximum Rate" property to specify how many requests should be allowed through each second. For example, to allow through 100 requests per second, set "Maximum Rate" to `100`. If more than 100 requests come in during any one second, the additional requests will be routed to `rate exceeded` instead of `success`. Reject requests that exceed a specific rate, allowing for burststhrottlelimitslow downrequest rateINPUT_ALLOWEDSet the "Rate Control Criteria" to `flowfile count`. Set the "Time Duration" property to `1 min`. Set the "Rate Exceeded Strategy" property to `Route to 'rate exceeded'`. Configure the "Maximum Rate" property to specify how many requests should be allowed through each minute. For example, to allow through 100 requests per second, set "Maximum Rate" to `6000`. This will allow through 6,000 FlowFiles per minute, which averages to 100 FlowFiles per second. However, those 6,000 FlowFiles may come all within the first couple of seconds, or they may come in over a period of 60 seconds. As a result, this gives us an average rate of 100 FlowFiles per second but allows for bursts of data. If more than 6,000 requests come in during any one minute, the additional requests will be routed to `rate exceeded` instead of `success`. org.apache.nifi.processors.standard.ConvertCharacterSetPROCESSORConverts a FlowFile's content from one character set to anothertextconvertcharactersetcharacter setInput Character SetInput Character SetThe name of the CharacterSet to expect for InputtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseOutput Character SetOutput Character SetThe name of the CharacterSet to convert totruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ConvertJSONToSQLPROCESSORConverts a JSON-formatted FlowFile into an UPDATE, INSERT, or DELETE SQL statement. The incoming FlowFile is expected to be "flat" JSON message, meaning that it consists of a single JSON element and each field maps to a simple type. If a field maps to a JSON object, that JSON object will be interpreted as Text. If the input is an array of JSON elements, each element in the array is output as a separate FlowFile to the 'sql' relationship. Upon successful conversion, the original FlowFile is routed to the 'original' relationship and the SQL is routed to the 'sql' relationship.jsonsqldatabaserdbmsinsertupdatedeleterelationalflatJDBC Connection PoolJDBC Connection PoolSpecifies the JDBC Connection Pool to use in order to convert the JSON message to a SQL statement. The Connection Pool is necessary in order to determine the appropriate database column types.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseStatement TypeStatement TypeSpecifies the type of SQL Statement to generateUPDATEUPDATEINSERTINSERTDELETEDELETEUse statement.type AttributeUse statement.type AttributetruefalsefalseNONEfalsefalseTable NameTable NameThe name of the table that the statement should updatetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCatalog NameCatalog NameThe name of the catalog that the statement should update. This may not apply for the database that you are updating. In this case, leave the field emptyfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSchema NameSchema NameThe name of the schema that the table belongs to. This may not apply for the database that you are updating. In this case, leave the field emptyfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseTranslate Field NamesTranslate Field NamesIf true, the Processor will attempt to translate JSON field names into the appropriate column names for the table specified. If false, the JSON field names must match the column names exactly, or the column will not be updatedtruetruetruefalsefalsefalsefalsefalseNONEfalsefalseUnmatched Field BehaviorUnmatched Field BehaviorIf an incoming JSON element has a field that does not map to any of the database table's columns, this property specifies how to handle the situationIgnore Unmatched FieldsIgnore Unmatched FieldsIgnore Unmatched FieldsAny field in the JSON document that cannot be mapped to a column in the database is ignoredFailFailIf the JSON document has any field that cannot be mapped to a column in the database, the FlowFile will be routed to the failure relationshipfalsefalsefalseNONEfalsefalseUnmatched Column BehaviorUnmatched Column BehaviorIf an incoming JSON element does not have a field mapping for all of the database table's columns, this property specifies how to handle the situationFail on Unmatched ColumnsIgnore Unmatched ColumnsIgnore Unmatched ColumnsAny column in the database that does not have a field in the JSON document will be assumed to not be required. No notification will be loggedWarn on Unmatched ColumnsWarn on Unmatched ColumnsAny column in the database that does not have a field in the JSON document will be assumed to not be required. A warning will be loggedFail on Unmatched ColumnsFail on Unmatched ColumnsA flow will fail if any column in the database that does not have a field in the JSON document. An error will be loggedfalsefalsefalseNONEfalsefalseUpdate KeysUpdate KeysA comma-separated list of column names that uniquely identifies a row in the database for UPDATE statements. If the Statement Type is UPDATE and this property is not set, the table's Primary Keys are used. In this case, if no Primary Key exists, the conversion to SQL will fail if Unmatched Column Behaviour is set to FAIL. This property is ignored if the Statement Type is INSERTfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsejts-quoted-identifiersQuote Column IdentifiersEnabling this option will cause all column names to be quoted, allowing you to use reserved words as column names in your tables.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalsejts-quoted-table-identifiersQuote Table IdentifiersEnabling this option will cause the table name to be quoted to support the use of special characters in the table namefalsetruetruefalsefalsefalsefalsefalseNONEfalsefalsejts-sql-param-attr-prefixSQL Parameter Attribute PrefixThe string to be prepended to the outgoing flow file attributes, such as <sql>.args.1.value, where <sql> is replaced with the specified valuesqltruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsetable-schema-cache-sizeTable Schema Cache SizeSpecifies how many Table Schemas should be cached100truefalsefalseNONEfalsefalsesqlA FlowFile is routed to this relationship when its contents have successfully been converted into a SQL statementfalsefailureA FlowFile is routed to this relationship if it cannot be converted into a SQL statement. Common causes include invalid JSON content or the JSON content missing a required field (if using an INSERT statement type).falseoriginalWhen a FlowFile is converted to SQL, the original JSON FlowFile is routed to this relationshipfalsemime.typeSets mime.type of FlowFile that is routed to 'sql' to 'text/plain'.<sql>.tableSets the <sql>.table attribute of FlowFile that is routed to 'sql' to the name of the table that is updated by the SQL statement. The prefix for this attribute ('sql', e.g.) is determined by the SQL Parameter Attribute Prefix property.<sql>.catalogIf the Catalog name is set for this database, specifies the name of the catalog that the SQL statement will update. If no catalog is used, this attribute will not be added. The prefix for this attribute ('sql', e.g.) is determined by the SQL Parameter Attribute Prefix property.fragment.identifierAll FlowFiles routed to the 'sql' relationship for the same incoming FlowFile (multiple will be output for the same incoming FlowFile if the incoming FlowFile is a JSON Array) will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.fragment.countThe number of SQL FlowFiles that were produced for same incoming FlowFile. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming FlowFile.fragment.indexThe position of this FlowFile in the list of outgoing FlowFiles that were all derived from the same incoming FlowFile. This can be used in conjunction with the fragment.identifier and fragment.count attributes to know which FlowFiles originated from the same incoming FlowFile and in what order the SQL FlowFiles were produced<sql>.args.N.typeThe output SQL statements are parametrized in order to avoid SQL Injection Attacks. The types of the Parameters to use are stored in attributes named <sql>.args.1.type, <sql>.args.2.type, <sql>.args.3.type, and so on. The type is a number representing a JDBC Type constant. Generally, this is useful only for software to read and interpret but is added so that a processor such as PutSQL can understand how to interpret the values. The prefix for this attribute ('sql', e.g.) is determined by the SQL Parameter Attribute Prefix property.<sql>.args.N.valueThe output SQL statements are parametrized in order to avoid SQL Injection Attacks. The values of the Parameters to use are stored in the attributes named sql.args.1.value, sql.args.2.value, sql.args.3.value, and so on. Each of these attributes has a corresponding <sql>.args.N.type attribute that indicates how the value should be interpreted when inserting it into the database.The prefix for this attribute ('sql', e.g.) is determined by the SQL Parameter Attribute Prefix property.truetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.PutSQLorg.apache.nifi.processors.standard.PutDatabaseRecordorg.apache.nifi.processors.standard.ConvertRecordPROCESSORConverts records from one data format to another using configured Record Reader and Record Write Controller Services. The Reader and Writer must be configured with "matching" schemas. By this, we mean the schemas must have the same field names. The types of the fields do not have to be the same if a field value can be coerced from one type to another. For instance, if the input schema has a field named "balance" of type double, the output schema can have a field named "balance" with a type of string, double, or float. If any field is present in the input that is not present in the output, the field will be left out of the output. If any field is specified in the output schema but is not present in the input data/schema, then the field will not be present in the output or will have a null value, depending on the writer.convertrecordgenericschemajsoncsvavrologlogsfreeformtextRecord ReaderRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseRecord WriterRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseInclude Zero Record FlowFilesInclude Zero Record FlowFilesWhen converting an incoming FlowFile, if the conversion results in no data, this property specifies whether or not a FlowFile will be sent to the corresponding relationshiptruetruetruefalsefalsetruefalsefalseNONEfalsefalsefailureIf a FlowFile cannot be transformed from the configured input format to the configured output format, the unchanged FlowFile will be routed to this relationshipfalsesuccessFlowFiles that are successfully transformed will be routed to this relationshipfalsemime.typeSets the mime.type attribute to the MIME Type specified by the Record Writerrecord.countThe number of records in the FlowFilerecord.error.messageThis attribute provides on failure the error message encountered by the Reader or Writer.truetrueINPUT_REQUIREDConvert data from one record-oriented format to anotherINPUT_ALLOWEDThe Record Reader should be configured according to the incoming data format. The Record Writer should be configured according to the desired output format.org.apache.nifi.processors.standard.CountTextPROCESSORCounts various metrics on incoming text. The requested results will be recorded as attributes. The resulting flowfile will not have its content modified.counttextlinewordcharactertext-line-countCount LinesIf enabled, will count the number of lines present in the incoming text.truetruetruefalsefalsetruefalsefalseNONEfalsefalsetext-line-nonempty-countCount Non-Empty LinesIf enabled, will count the number of lines that contain a non-whitespace character present in the incoming text.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsetext-word-countCount WordsIf enabled, will count the number of words (alphanumeric character groups bounded by whitespace) present in the incoming text. Common logical delimiters [_-.] do not bound a word unless 'Split Words on Symbols' is true.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsetext-character-countCount CharactersIf enabled, will count the number of characters (including whitespace and symbols, but not including newlines and carriage returns) present in the incoming text.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesplit-words-on-symbolsSplit Words on SymbolsIf enabled, the word count will identify strings separated by common logical delimiters [ _ - . ] as independent words (ex. split-words-on-symbols = 4 words).falsetruetruefalsefalsetruefalsefalseNONEfalsefalsecharacter-encodingCharacter EncodingSpecifies a character encoding to use.UTF-8ISO-8859-1ISO-8859-1UTF-8UTF-8UTF-16UTF-16UTF-16LEUTF-16LEUTF-16BEUTF-16BEUS-ASCIIUS-ASCIItruefalsefalseNONEfalsefalseajust-immediatelyCall Immediate AdjustmentIf true, the counter will be updated immediately, without regard to whether the ProcessSession is commit or rolled back;otherwise, the counter will be incremented only if and when the ProcessSession is committed.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessThe flowfile contains the original content with one or more attributes added containing the respective countsfalsefailureIf the flowfile text cannot be counted for some reason, the original file will be routed to this destination and nothing will be routed elsewherefalsetext.line.countThe number of lines of text present in the FlowFile contenttext.line.nonempty.countThe number of lines of text (with at least one non-whitespace character) present in the original FlowFiletext.word.countThe number of words present in the original FlowFiletext.character.countThe number of characters (given the specified character encoding) present in the original FlowFiletruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.SplitTextorg.apache.nifi.processors.standard.CryptographicHashContentPROCESSORCalculates a cryptographic hash value for the flowfile content using the given algorithm and writes it to an output attribute. Please refer to https://csrc.nist.gov/Projects/Hash-Functions/NIST-Policy-on-Hash-Functions for help to decide which algorithm to use.contenthashshablake2md5cryptographyfail_when_emptyFail if the content is emptyRoute to failure if the content is empty. While hashing an empty value is valid, some flows may want to detect empty input.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsehash_algorithmHash AlgorithmThe hash algorithm to use. Note that not all of the algorithms available are recommended for use (some are provided for legacy compatibility). There are many things to consider when picking an algorithm; it is recommended to use the most secure algorithm possible.SHA-256MD2MD2MD2 (16 byte output) [WARNING -- Cryptographically broken] Cryptographically broken due to collisionsMD5MD5MD5 (16 byte output) [WARNING -- Cryptographically broken] Cryptographically broken due to collisionsSHA-1SHA-1SHA-1 (20 byte output) [WARNING -- Cryptographically broken] Cryptographically broken due to collisionsSHA-224SHA-224SHA-224 (28 byte output) SHA-2 familySHA-256SHA-256SHA-256 (32 byte output) SHA-2 familySHA-384SHA-384SHA-384 (48 byte output) SHA-2 familySHA-512SHA-512SHA-512 (64 byte output) SHA-2 familySHA-512/224SHA-512/224SHA-512/224 (28 byte output) SHA-2 using SHA-512 with truncated outputSHA-512/256SHA-512/256SHA-512/256 (32 byte output) SHA-2 using SHA-512 with truncated outputSHA3-224SHA3-224SHA3-224 (28 byte output) Keccak-based SHA3 familySHA3-256SHA3-256SHA3-256 (32 byte output) Keccak-based SHA3 familySHA3-384SHA3-384SHA3-384 (48 byte output) Keccak-based SHA3 familySHA3-512SHA3-512SHA3-512 (64 byte output) Keccak-based SHA3 familyBLAKE2-160BLAKE2-160BLAKE2-160 (20 byte output) Also known as Blake2bBLAKE2-256BLAKE2-256BLAKE2-256 (32 byte output) Also known as Blake2bBLAKE2-384BLAKE2-384BLAKE2-384 (48 byte output) Also known as Blake2bBLAKE2-512BLAKE2-512BLAKE2-512 (64 byte output) Also known as Blake2btruefalsefalseNONEfalsefalsesuccessUsed for flowfiles that have a hash value addedfalsefailureUsed for flowfiles that have no content if the 'fail on empty' setting is enabledfalsecontent_<algorithm>This processor adds an attribute whose value is the result of hashing the flowfile content. The name of this attribute is specified by the value of the algorithm, e.g. 'content_SHA-256'.truetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.DebugFlowPROCESSORThe DebugFlow processor aids testing and debugging the FlowFile framework by allowing various responses to be explicitly triggered in response to the receipt of a FlowFile or a timer event without a FlowFile if using timer or cron based scheduling. It can force responses needed to exercise or test various failure modes that can occur when a processor runs.testdebugprocessorutilityflowFlowFileFlowFile Success IterationsFlowFile Success IterationsNumber of FlowFiles to forward to success relationship.1truefalsefalseNONEfalsefalseFlowFile Failure IterationsFlowFile Failure IterationsNumber of FlowFiles to forward to failure relationship.0truefalsefalseNONEfalsefalseFlowFile Rollback IterationsFlowFile Rollback IterationsNumber of FlowFiles to roll back (without penalty).0truefalsefalseNONEfalsefalseFlowFile Rollback Yield IterationsFlowFile Rollback Yield IterationsNumber of FlowFiles to roll back and yield.0truefalsefalseNONEfalsefalseFlowFile Rollback Penalty IterationsFlowFile Rollback Penalty IterationsNumber of FlowFiles to roll back with penalty.0truefalsefalseNONEfalsefalseFlowFile Exception IterationsFlowFile Exception IterationsNumber of FlowFiles to throw exception.0truefalsefalseNONEfalsefalseFlowFile Exception ClassFlowFile Exception ClassException class to be thrown (must extend java.lang.RuntimeException).java.lang.RuntimeExceptiontruefalsefalseNONEfalsefalseNo FlowFile Skip IterationsNo FlowFile Skip IterationsNumber of times to skip onTrigger if no FlowFile.1truefalsefalseNONEfalsefalseNo FlowFile Exception IterationsNo FlowFile Exception IterationsNumber of times to throw NPE exception if no FlowFile.0truefalsefalseNONEfalsefalseNo FlowFile Yield IterationsNo FlowFile Yield IterationsNumber of times to yield if no FlowFile.0truefalsefalseNONEfalsefalseNo FlowFile Exception ClassNo FlowFile Exception ClassException class to be thrown if no FlowFile (must extend java.lang.RuntimeException).java.lang.RuntimeExceptiontruefalsefalseNONEfalsefalseWrite IterationsWrite IterationsNumber of times to write to the FlowFile0truefalsefalseNONEfalsefalseContent SizeContent SizeThe number of bytes to write each time that the FlowFile is written to1 KBtruefalsefalseNONEfalsefalse@OnScheduled Pause Time@OnScheduled Pause TimeSpecifies how long the processor should sleep in the @OnScheduled method, so that the processor can be forced to take a long time to start up0 sectruefalsefalseNONEfalsefalseFail When @OnScheduled calledFail When @OnScheduled calledSpecifies whether or not the Processor should throw an Exception when the methods annotated with @OnScheduled are calledfalsetruetruefalsefalsetruefalsefalseNONEfalsefalse@OnUnscheduled Pause Time@OnUnscheduled Pause TimeSpecifies how long the processor should sleep in the @OnUnscheduled method, so that the processor can be forced to take a long time to respond when user clicks stop0 sectruefalsefalseNONEfalsefalseFail When @OnUnscheduled calledFail When @OnUnscheduled calledSpecifies whether or not the Processor should throw an Exception when the methods annotated with @OnUnscheduled are calledfalsetruetruefalsefalsetruefalsefalseNONEfalsefalse@OnStopped Pause Time@OnStopped Pause TimeSpecifies how long the processor should sleep in the @OnStopped method, so that the processor can be forced to take a long time to shutdown0 sectruefalsetrueENVIRONMENTfalsefalseFail When @OnStopped calledFail When @OnStopped calledSpecifies whether or not the Processor should throw an Exception when the methods annotated with @OnStopped are calledfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseOnTrigger Pause TimeOnTrigger Pause TimeSpecifies how long the processor should sleep in the onTrigger() method, so that the processor can be forced to take a long time to perform its task0 sectruefalsefalseNONEfalsefalseCustomValidate Pause TimeCustomValidate Pause TimeSpecifies how long the processor should sleep in the customValidate() method0 sectruefalsefalseNONEfalsefalseIgnore Interrupts When PausedIgnore Interrupts When PausedIf the Processor's thread(s) are sleeping (due to one of the "Pause Time" properties above), and the thread is interrupted, this indicates whether the Processor should ignore the interrupt and continue sleeping or if it should allow itself to be interrupted.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessFlowFiles processed successfully.falsefailureFlowFiles that failed to process.falseorg.apache.nifi.processors.standard.DeduplicateRecordPROCESSORThis processor de-duplicates individual records within a record set. It can operate on a per-file basis using an in-memory hashset or bloom filter. When configured with a distributed map cache, it de-duplicates records across multiple files.textrecordupdatechangereplacemodifydistinctuniquefilterhashdupeduplicatededupededuplication-strategyDeduplication StrategyThe strategy to use for detecting and routing duplicate records. The option for detecting duplicates across a single FlowFile operates in-memory, whereas detection spanning multiple FlowFiles utilises a distributed map cache.singleSingle FilesingleMultiple FilesmultipletruefalsefalseNONEfalsefalsedistributed-map-cacheDistributed Map Cache clientThis property is required when the deduplication strategy is set to 'multiple files.' The map cache will for each record, atomically check whether the cache key exists and if not, set it.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalsededuplication-strategyDeduplication Strategymultiplecache-identifierCache IdentifierAn optional expression language field that overrides the record's computed cache key. This field has an additional attribute available: ${record.hash.value}, which contains the cache key derived from dynamic properties (if set) or record fields.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsededuplication-strategyDeduplication Strategymultipleput-cache-identifierCache the Entry IdentifierFor each record, check whether the cache identifier exists in the distributed map cache. If it doesn't exist and this property is true, put the identifier to the cache.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedistributed-map-cacheDistributed Map Cache clientrecord-readerRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseinclude-zero-record-flowfilesInclude Zero Record FlowFilesIf a FlowFile sent to either the duplicate or non-duplicate relationships contains no records, a value of `false` in this property causes the FlowFile to be dropped. Otherwise, the empty FlowFile is emitted.truetruetruefalsefalsetruefalsefalseNONEfalsefalserecord-hashing-algorithmRecord Hashing AlgorithmThe algorithm used to hash the cache key.SHA-256NonenoneDo not use a hashing algorithm. The value of resolved RecordPaths will be combined with a delimiter (~) to form the unique cache key. This may use significantly more storage depending on the size and shape or your data.SHA-256SHA-256SHA-256 cryptographic hashing algorithm.SHA-512SHA-512SHA-512 cryptographic hashing algorithm.truefalsefalseNONEfalsefalsefilter-typeFilter TypeThe filter used to determine whether a record has been seen before based on the matching RecordPath criteria. If hash set is selected, a Java HashSet object will be used to deduplicate all encountered records. If the bloom filter option is selected, a bloom filter will be used. The bloom filter option is less memory intensive, but has a chance of having false positives.hash-setHashSethash-setExactly matches records seen before with 100% accuracy at the expense of more storage usage. Stores the filter data in a single cache entry in the distributed cache, and is loaded entirely into memory during duplicate detection. This filter is preferred for small to medium data sets and offers high performance, being loaded into memory when this processor is running.BloomFilterbloom-filterSpace-efficient data structure ideal for large data sets using probability to determine if a record was seen previously. False positive matches are possible, but false negatives are not – in other words, a query returns either "possibly in the set" or "definitely not in the set". You should use this option if the FlowFile content is large and you can tolerate some duplication in the data. Uses constant storage space regardless of the record set size.truefalsefalseNONEfalsefalsededuplication-strategyDeduplication Strategysinglefilter-capacity-hintFilter Capacity HintAn estimation of the total number of unique records to be processed. The more accurate this number is will lead to fewer false negatives on a BloomFilter.25000truefalsefalseNONEfalsefalsefilter-typeFilter Typebloom-filterbloom-filter-certaintyBloom Filter CertaintyThe desired false positive probability when using the BloomFilter type. Using a value of .05 for example, guarantees a five-percent probability that the result is a false positive. The closer to 1 this value is set, the more precise the result at the expense of more storage space utilization.0.10falsefalsefalseNONEfalsefalseName of the property.A valid RecordPath to the record field to be included in the cache key used for deduplication.A record's cache key is generated by combining the name of each dynamic property with its evaluated record value (as specified by the corresponding RecordPath).NONEduplicateRecords detected as duplicates are routed to this relationship.falsenon-duplicateRecords not found in the cache are routed to this relationship.falsefailureIf unable to communicate with the cache, the FlowFile will be penalized and routed to this relationshipfalseoriginalThe original input FlowFile is sent to this relationship unless a fatal error occurs.falserecord.countNumber of records written to the destination FlowFile.trueINPUT_REQUIREDMEMORYThe HashSet filter type will grow memory space proportionate to the number of unique records processed. The BloomFilter type will use constant memory regardless of the number of records processed.CPUIf a more advanced hash algorithm is chosen, the amount of time required to hash any particular record could increase substantially.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientServiceorg.apache.nifi.distributed.cache.server.map.DistributedMapCacheServerorg.apache.nifi.processors.standard.DetectDuplicateorg.apache.nifi.processors.standard.DeleteFilePROCESSORDeletes a file from the filesystem.fileremovedeletelocalfilesfilesystemDirectory PathDirectory PathThe path to the directory the file to delete is located in.${absolute.path}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFilenameFilenameThe name of the file to delete.${filename}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsenot foundAll FlowFiles, for which the file to delete did not exist, are routed to this relationshipfalsesuccessAll FlowFiles, for which an existing file has been deleted, are routed to this relationshipfalsefailureAll FlowFiles, for which an existing file could not be deleted, are routed to this relationshipfalseDeleteFile.failure.reasonHuman-readable reason of failure. Only available if FlowFile is routed to relationship 'failure'.DeleteFile.failure.exception.classThe class name of the exception thrown during processor execution. Only available if an exception caused the FlowFile to be routed to relationship 'failure'.DeleteFile.failure.exception.messageThe message of the exception thrown during processor execution. Only available if an exception caused the FlowFile to be routed to relationship 'failure'.trueread filesystemProvides operator the ability to read from any file that NiFi has access to.write filesystemProvides operator the ability to delete any file that NiFi has access to.INPUT_REQUIREDDelete source file only after its processing completedINPUT_ALLOWEDRetrieve a file from the filesystem, e.g. using 'ListFile' and 'FetchFile'. Process the file using any combination of processors. Store the resulting file to a destination, e.g. using 'PutSFTP'. Using 'DeleteFile', delete the file from the filesystem only after the result has been stored. org.apache.nifi.processors.standard.DetectDuplicatePROCESSORCaches a value, computed from FlowFile attributes, for each incoming FlowFile and determines if the cached value has already been seen. If so, routes the FlowFile to 'duplicate' with an attribute named 'original.identifier' that specifies the original FlowFile's "description", which is specified in the <FlowFile Description> property. If the FlowFile is not determined to be a duplicate, the Processor routes the FlowFile to 'non-duplicate'hashdupeduplicatededupeCache Entry IdentifierCache Entry IdentifierA FlowFile attribute, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the value used to identify duplicates; it is this value that is cached${hash.value}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFlowFile DescriptionFlowFile DescriptionWhen a FlowFile is added to the cache, this value is stored along with it so that if a duplicate is found, this description of the original FlowFile will be added to the duplicate's "original.flowfile.description" attributetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseAge Off DurationAge Off DurationTime interval to age off cached FlowFilesfalsefalsefalseNONEfalsefalseDistributed Cache ServiceDistributed Cache ServiceThe Controller Service that is used to cache unique identifiers, used to determine duplicatesorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseCache The Entry IdentifierCache The Entry IdentifierWhen true this cause the processor to check for duplicates and cache the Entry Identifier. When false, the processor would only check for duplicates and not cache the Entry Identifier, requiring another processor to add identifiers to the distributed cache.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseduplicateIf a FlowFile has been detected to be a duplicate, it will be routed to this relationshipfalsenon-duplicateIf a FlowFile's Cache Entry Identifier was not found in the cache, it will be routed to this relationshipfalsefailureIf unable to communicate with the cache, the FlowFile will be penalized and routed to this relationshipfalseoriginal.flowfile.descriptionAll FlowFiles routed to the duplicate relationship will have an attribute added named original.flowfile.description. The value of this attribute is determined by the attributes of the original copy of the data and by the FlowFile Description property.trueINPUT_REQUIREDorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientServiceorg.apache.nifi.distributed.cache.server.map.DistributedMapCacheServerorg.apache.nifi.processors.standard.DistributeLoadPROCESSORDistributes FlowFiles to downstream processors based on a Distribution Strategy. If using the Round Robin strategy, the default is to assign each destination a weighting of 1 (evenly distributed). However, optional properties can be added to the change this; adding a property with the name '5' and value '10' means that the relationship with name '5' will be receive 10 FlowFiles in each iteration instead of 1.distributeload balancerouteround robinweightedNumber of RelationshipsNumber of RelationshipsDetermines the number of Relationships to which the load should be distributed1truefalsefalseNONEfalsefalseDistribution StrategyDistribution StrategyDetermines how the load will be distributed. Relationship weight is in numeric order where '1' has the greatest weight.round robinround robinround robinRelationship selection is evenly distributed in a round robin fashion; all relationships must be available.next availablenext availableRelationship selection is distributed across all available relationships in order of their weight; at least one relationship must be available.overflowoverflowRelationship selection is the first available relationship without further distribution among all relationships; at least one relationship must be available.truefalsefalseNONEfalsefalseThe relationship name (positive number)The relationship Weight (positive number)Adding a property with the name '5' and value '10' means that the relationship with name '5' will receive 10 FlowFiles in each iteration instead of 1.NONE1Where to route flowfiles for this relationship indexfalseA number 1..<Number Of Relationships>FlowFiles are sent to this relationship per the <Distribution Strategy>distribute.load.relationshipThe name of the specific relationship the FlowFile has been routed throughtruetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.DuplicateFlowFilePROCESSORIntended for load testing, this processor will create the configured number of copies of each incoming FlowFile. The original FlowFile as well as all generated copies are sent to the 'success' relationship. In addition, each FlowFile gets an attribute 'copy.index' set to the copy number, where the original FlowFile gets a value of zero, and all copies receive incremented integer values.testloadduplicateNumber of CopiesNumber of CopiesSpecifies how many copies of each incoming FlowFile will be made100truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessThe original FlowFile and all copies will be sent to this relationshipfalsecopy.indexA zero-based incrementing integer value based on which copy the FlowFile is.trueINPUT_REQUIREDorg.apache.nifi.processors.standard.EncodeContentPROCESSOREncode or decode the contents of a FlowFile using Base64, Base32, or hex encoding schemesencodedecodebase64base32hexModeModeSpecifies whether the content should be encoded or decoded.EncodeEncodeEncodeTransform original input to encoded representationDecodeDecodeTransform encoded input to original representationtruefalsefalseNONEfalsefalseEncodingEncodingSpecifies the type of encoding used.base64Base64base64Encode or decode using Base64 set of charactersBase32base32Encode or decode using Base32 set of charactersHexadecimalhexEncode or decode using hexadecimal set of characterstruefalsefalseNONEfalsefalseLine Output ModeLine Output ModeControls the line formatting for encoded content based on selected property values.SINGLE_LINESingle LineSINGLE_LINEThe encoded content will be written as a single line.Multiple LinesMULTIPLE_LINESThe encoded content will be written as multiple lines.truefalsefalseNONEfalsefalseEncodingEncodingbase32base64ModeModeEncodeEncoded Line LengthEncoded Line LengthEach line of encoded data will contain up to the configured number of characters, rounded down to the nearest multiple of 4.76truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseEncodingEncodingbase32base64ModeModeEncodeLine Output ModeLine Output ModeMULTIPLE_LINESfailureAny FlowFile that cannot be encoded or decoded will be routed to failurefalsesuccessAny FlowFile that is successfully encoded or decoded will be routed to successfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.EnforceOrderPROCESSOREnforces expected ordering of FlowFiles that belong to the same data group within a single node. Although PriorityAttributePrioritizer can be used on a connection to ensure that flow files going through that connection are in priority order, depending on error-handling, branching, and other flow designs, it is possible for FlowFiles to get out-of-order. EnforceOrder can be used to enforce original ordering for those FlowFiles. [IMPORTANT] In order to take effect of EnforceOrder, FirstInFirstOutPrioritizer should be used at EVERY downstream relationship UNTIL the order of FlowFiles physically get FIXED by operation such as MergeContent or being stored to the final destination.sortordergroup-idGroup IdentifierEnforceOrder is capable of multiple ordering groups. 'Group Identifier' is used to determine which group a FlowFile belongs to. This property will be evaluated with each incoming FlowFile. If evaluated result is empty, the FlowFile will be routed to failure.${filename}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseorder-attributeOrder AttributeA name of FlowFile attribute whose value will be used to enforce order of FlowFiles within a group. If a FlowFile does not have this attribute, or its value is not an integer, the FlowFile will be routed to failure.truefalsefalseNONEfalsefalseinitial-orderInitial OrderWhen the first FlowFile of a group arrives, initial target order will be computed and stored in the managed state. After that, target order will start being tracked by EnforceOrder and stored in the state management store. If Expression Language is used but evaluated result was not an integer, then the FlowFile will be routed to failure, and initial order will be left unknown until consecutive FlowFiles provide a valid initial order.0truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsemaximum-orderMaximum OrderIf specified, any FlowFiles that have larger order will be routed to failure. This property is computed only once for a given group. After a maximum order is computed, it will be persisted in the state management store and used for other FlowFiles belonging to the same group. If Expression Language is used but evaluated result was not an integer, then the FlowFile will be routed to failure, and maximum order will be left unknown until consecutive FlowFiles provide a valid maximum order.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsebatch-countBatch CountThe maximum number of FlowFiles that EnforceOrder can process at an execution.1000truefalsefalseNONEfalsefalsewait-timeoutWait TimeoutIndicates the duration after which waiting FlowFiles will be routed to the 'overtook' relationship.10 mintruefalsefalseNONEfalsefalseinactive-timeoutInactive TimeoutIndicates the duration after which state for an inactive group will be cleared from managed state. Group is determined as inactive if any new incoming FlowFile has not seen for a group for specified duration. Inactive Timeout must be longer than Wait Timeout. If a FlowFile arrives late after its group is already cleared, it will be treated as a brand new group, but will never match the order since expected preceding FlowFiles are already gone. The FlowFile will eventually timeout for waiting and routed to 'overtook'. To avoid this, group states should be kept long enough, however, shorter duration would be helpful for reusing the same group identifier again.30 mintruefalsefalseNONEfalsefalseovertookA FlowFile that waited for preceding FlowFiles longer than Wait Timeout and overtook those FlowFiles, will be routed to this relationship.falseskippedA FlowFile that has an order younger than current, which means arrived too late and skipped, will be routed to this relationship.falsesuccessA FlowFile with a matching order number will be routed to this relationship.falsewaitA FlowFile with non matching order will be routed to this relationshipfalsefailureA FlowFiles which does not have required attributes, or fails to compute those will be routed to this relationshipfalseEnforceOrder.startedAtAll FlowFiles going through this processor will have this attribute. This value is used to determine wait timeout.EnforceOrder.resultAll FlowFiles going through this processor will have this attribute denoting which relationship it was routed to.EnforceOrder.detailFlowFiles routed to 'failure' or 'skipped' relationship will have this attribute describing details.EnforceOrder.expectedOrderFlowFiles routed to 'wait' or 'skipped' relationship will have this attribute denoting expected order when the FlowFile was processed.trueEnforceOrder uses following states per ordering group: '<groupId>.target' is a order number which is being waited to arrive next. When a FlowFile with a matching order arrives, or a FlowFile overtakes the FlowFile being waited for because of wait timeout, target order will be updated to (FlowFile.order + 1). '<groupId>.max is the maximum order number for a group. '<groupId>.updatedAt' is a timestamp when the order of a group was updated last time. These managed states will be removed automatically once a group is determined as inactive, see 'Inactive Timeout' for detail.LOCALINPUT_REQUIREDorg.apache.nifi.processors.standard.EvaluateJsonPathPROCESSOREvaluates one or more JsonPath expressions against the content of a FlowFile. The results of those expressions are assigned to FlowFile Attributes or are written to the content of the FlowFile itself, depending on configuration of the Processor. JsonPaths are entered by adding user-defined properties; the name of the property maps to the Attribute Name into which the result will be placed (if the Destination is flowfile-attribute; otherwise, the property name is ignored). The value of the property must be a valid JsonPath expression. A Return Type of 'auto-detect' will make a determination based off the configured destination. When 'Destination' is set to 'flowfile-attribute,' a return type of 'scalar' will be used. When 'Destination' is set to 'flowfile-content,' a return type of 'JSON' will be used.If the JsonPath evaluates to a JSON array or JSON object and the Return Type is set to 'scalar' the FlowFile will be unmodified and will be routed to failure. A Return Type of JSON can return scalar values if the provided JsonPath evaluates to the specified value and will be routed as a match.If Destination is 'flowfile-content' and the JsonPath does not evaluate to a defined path, the FlowFile will be routed to 'unmatched' without having its contents modified. If Destination is 'flowfile-attribute' and the expression matches nothing, attributes will be created with empty strings as the value unless 'Path Not Found Behaviour' is set to 'skip', and the FlowFile will always be routed to 'matched.'JSONevaluateJsonPathDestinationDestinationIndicates whether the results of the JsonPath evaluation are written to the FlowFile content or a FlowFile attribute; if using attribute, must specify the Attribute Name property. If set to flowfile-content, only one JsonPath may be specified, and the property name is ignored.flowfile-contentflowfile-contentflowfile-contentflowfile-attributeflowfile-attributetruefalsefalseNONEfalsefalseReturn TypeReturn TypeIndicates the desired return type of the JSON Path expressions. Selecting 'auto-detect' will set the return type to 'json' for a Destination of 'flowfile-content', and 'scalar' for a Destination of 'flowfile-attribute'.auto-detectauto-detectauto-detectjsonjsonscalarscalartruefalsefalseNONEfalsefalsePath Not Found BehaviorPath Not Found BehaviorIndicates how to handle missing JSON path expressions when destination is set to 'flowfile-attribute'. Selecting 'warn' will generate a warning when a JSON path expression is not found. Selecting 'skip' will omit attributes for any unmatched JSON path expressions.ignorewarnwarnignoreignoreskipskiptruefalsefalseNONEfalsefalseDestinationDestinationflowfile-attributeNull Value RepresentationNull Value RepresentationIndicates the desired representation of JSON Path expressions resulting in a null value.empty stringempty stringempty stringthe string 'null'the string 'null'truefalsefalseNONEfalsefalseMax String LengthMax String LengthThe maximum allowed length of a string value when parsing the JSON document20 MBtruefalsefalseNONEfalsefalseA FlowFile attribute(if <Destination> is set to 'flowfile-attribute')A JsonPath expressionIf <Destination>='flowfile-attribute' then that FlowFile attribute will be set to any JSON objects that match the JsonPath. If <Destination>='flowfile-content' then the FlowFile content will be updated to any JSON objects that match the JsonPath.NONEfailureFlowFiles are routed to this relationship when the JsonPath cannot be evaluated against the content of the FlowFile; for instance, if the FlowFile is not valid JSONfalseunmatchedFlowFiles are routed to this relationship when the JsonPath does not match the content of the FlowFile and the Destination is set to flowfile-contentfalsematchedFlowFiles are routed to this relationship when the JsonPath is successfully evaluated and the FlowFile is modified as a resultfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.EvaluateXPathPROCESSOREvaluates one or more XPaths against the content of a FlowFile. The results of those XPaths are assigned to FlowFile Attributes or are written to the content of the FlowFile itself, depending on configuration of the Processor. XPaths are entered by adding user-defined properties; the name of the property maps to the Attribute Name into which the result will be placed (if the Destination is flowfile-attribute; otherwise, the property name is ignored). The value of the property must be a valid XPath expression. If the XPath evaluates to more than one node and the Return Type is set to 'nodeset' (either directly, or via 'auto-detect' with a Destination of 'flowfile-content'), the FlowFile will be unmodified and will be routed to failure. If the XPath does not evaluate to a Node, the FlowFile will be routed to 'unmatched' without having its contents modified. If Destination is flowfile-attribute and the expression matches nothing, attributes will be created with empty strings as the value, and the FlowFile will always be routed to 'matched'XMLevaluateXPathDestinationDestinationIndicates whether the results of the XPath evaluation are written to the FlowFile content or a FlowFile attribute; if using attribute, must specify the Attribute Name property. If set to flowfile-content, only one XPath may be specified, and the property name is ignored.flowfile-contentflowfile-contentflowfile-contentflowfile-attributeflowfile-attributetruefalsefalseNONEfalsefalseReturn TypeReturn TypeIndicates the desired return type of the Xpath expressions. Selecting 'auto-detect' will set the return type to 'nodeset' for a Destination of 'flowfile-content', and 'string' for a Destination of 'flowfile-attribute'.auto-detectauto-detectauto-detectnodesetnodesetstringstringtruefalsefalseNONEfalsefalseValidate DTDAllow DTDAllow embedded Document Type Declaration in XML. This feature should be disabled to avoid XML entity expansion vulnerabilities.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseA FlowFile attribute(if <Destination> is set to 'flowfile-attribute'An XPath expressionIf <Destination>='flowfile-attribute' then the FlowFile attribute is set to the result of the XPath Expression. If <Destination>='flowfile-content' then the FlowFile content is set to the result of the XPath Expression.NONEfailureFlowFiles are routed to this relationship when the XPath cannot be evaluated against the content of the FlowFile; for instance, if the FlowFile is not valid XML, or if the Return Type is 'nodeset' and the XPath evaluates to multiple nodesfalseunmatchedFlowFiles are routed to this relationship when the XPath does not match the content of the FlowFile and the Destination is set to flowfile-contentfalsematchedFlowFiles are routed to this relationship when the XPath is successfully evaluated and the FlowFile is modified as a resultfalseuser-definedThis processor adds user-defined attributes if the <Destination> property is set to flowfile-attribute.truetrueINPUT_REQUIREDMEMORYProcessing requires reading the entire FlowFile into memoryorg.apache.nifi.processors.standard.EvaluateXQueryPROCESSOREvaluates one or more XQueries against the content of a FlowFile. The results of those XQueries are assigned to FlowFile Attributes or are written to the content of the FlowFile itself, depending on configuration of the Processor. XQueries are entered by adding user-defined properties; the name of the property maps to the Attribute Name into which the result will be placed (if the Destination is 'flowfile-attribute'; otherwise, the property name is ignored). The value of the property must be a valid XQuery. If the XQuery returns more than one result, new attributes or FlowFiles (for Destinations of 'flowfile-attribute' or 'flowfile-content' respectively) will be created for each result (attributes will have a '.n' one-up number appended to the specified attribute name). If any provided XQuery returns a result, the FlowFile(s) will be routed to 'matched'. If no provided XQuery returns a result, the FlowFile will be routed to 'unmatched'. If the Destination is 'flowfile-attribute' and the XQueries matche nothing, no attributes will be applied to the FlowFile.XMLevaluateXPathXQueryDestinationDestinationIndicates whether the results of the XQuery evaluation are written to the FlowFile content or a FlowFile attribute. If set to <flowfile-content>, only one XQuery may be specified and the property name is ignored. If set to <flowfile-attribute> and the XQuery returns more than one result, multiple attributes will be added to theFlowFile, each named with a '.n' one-up number appended to the specified attribute nameflowfile-contentflowfile-contentflowfile-contentflowfile-attributeflowfile-attributetruefalsefalseNONEfalsefalseOutput: MethodOutput: MethodIdentifies the overall method that should be used for outputting a result tree.xmlxmlxmlhtmlhtmltexttexttruefalsefalseNONEfalsefalseOutput: Omit XML DeclarationOutput: Omit XML DeclarationSpecifies whether the processor should output an XML declaration when transforming a result tree.falsetruefalsefalseNONEfalsefalseOutput: IndentOutput: IndentSpecifies whether the processor may add additional whitespace when outputting a result tree.falsetruefalsefalseNONEfalsefalseValidate DTDAllow DTDAllow embedded Document Type Declaration in XML. This feature should be disabled to avoid XML entity expansion vulnerabilities.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseA FlowFile attribute(if <Destination> is set to 'flowfile-attribute'An XQueryIf <Destination>='flowfile-attribute' then the FlowFile attribute is set to the result of the XQuery. If <Destination>='flowfile-content' then the FlowFile content is set to the result of the XQuery.NONEfailureFlowFiles are routed to this relationship when the XQuery cannot be evaluated against the content of the FlowFile.falseunmatchedFlowFiles are routed to this relationship when the XQuery does not match the content of the FlowFile and the Destination is set to flowfile-contentfalsematchedFlowFiles are routed to this relationship when the XQuery is successfully evaluated and the FlowFile is modified as a resultfalseuser-definedThis processor adds user-defined attributes if the <Destination> property is set to flowfile-attribute .truetrueINPUT_REQUIREDMEMORYProcessing requires reading the entire FlowFile into memoryorg.apache.nifi.processors.standard.ExecuteProcessPROCESSORRuns an operating system command specified by the user and writes the output of that command to a FlowFile. If the command is expected to be long-running, the Processor can output the partial data on a specified interval. When this option is used, the output is expected to be in textual format, as it typically does not make sense to split binary data on arbitrary time-based intervals.commandprocesssourceexternalinvokescriptCommandCommandSpecifies the command to be executed; if just the name of an executable is provided, it must be in the user's environment PATH.truefalsetrueENVIRONMENTfalsefalseCommand ArgumentsCommand ArgumentsThe arguments to supply to the executable delimited by white space. White space can be escaped by enclosing it in double-quotes.falsefalsetrueENVIRONMENTfalsefalseBatch DurationBatch DurationIf the process is expected to be long-running and produce textual output, a batch duration can be specified so that the output will be captured for this amount of time and a FlowFile will then be sent out with the results and a new FlowFile will be started, rather than waiting for the process to finish before sending out the resultsfalsefalsefalseNONEfalsefalseRedirect Error StreamRedirect Error StreamIf true will redirect any error stream output of the process to the output stream. This is particularly helpful for processes which write extensively to the error stream or for troubleshooting.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseWorking DirectoryWorking DirectoryThe directory to use as the current working directory when executing the commandfalsefalsetrueENVIRONMENTfalsefalseArgument DelimiterArgument DelimiterDelimiter to use to separate arguments for a command [default: space]. Must be a single character. truefalsefalseNONEfalsefalseOutput MIME typeOutput MIME TypeSpecifies the value to set for the "mime.type" attribute. This property is ignored if 'Batch Duration' is set.falsefalsefalseNONEfalsefalseAn environment variable nameAn environment variable valueThese environment variables are passed to the process spawned by this ProcessorNONEsuccessAll created FlowFiles are routed to this relationshipfalsecommandExecuted commandcommand.argumentsArguments of the commandmime.typeSets the MIME type of the output if the 'Output MIME Type' property is set and 'Batch Duration' is not setexecute codeProvides operator the ability to execute arbitrary code assuming all permissions that NiFi has.INPUT_FORBIDDENorg.apache.nifi.processors.standard.ExecuteSQLPROCESSORExecutes provided SQL select query. Query result will be converted to Avro format. Streaming is used so arbitrarily large result sets are supported. This processor can be scheduled to run on a timer, or cron expression, using the standard scheduling methods, or it can be triggered by an incoming FlowFile. If it is triggered by an incoming FlowFile, then attributes of that FlowFile will be available when evaluating the select query, and the query may use the ? to escape parameters. In this case, the parameters to use must exist as FlowFile attributes with the naming convention sql.args.N.type and sql.args.N.value, where N is a positive integer. The sql.args.N.type is expected to be a number indicating the JDBC Type. The content of the FlowFile is expected to be in UTF-8 format. FlowFile attribute 'executesql.row.count' indicates how many rows were selected.sqlselectjdbcquerydatabaseDatabase Connection Pooling ServiceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain connection to databaseorg.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsesql-pre-querySQL Pre-QueryA semicolon-delimited list of queries executed before the main SQL query is executed. For example, set session properties before main query. It's possible to include semicolons in the statements themselves by escaping them with a backslash ('\;'). Results/outputs from these queries will be suppressed if there are no errors.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSQL select querySQL select queryThe SQL select query to execute. The query can be empty, a constant value, or built from attributes using Expression Language. If this property is specified, it will be used regardless of the content of incoming flowfiles. If this property is empty, the content of the incoming flow file is expected to contain a valid SQL select query, to be issued by the processor to the database. Note that Expression Language is not evaluated for flow file contents.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesql-post-querySQL Post-QueryA semicolon-delimited list of queries executed after the main SQL query is executed. Example like setting session properties after main query. It's possible to include semicolons in the statements themselves by escaping them with a backslash ('\;'). Results/outputs from these queries will be suppressed if there are no errors.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax Wait TimeMax Wait TimeThe maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedbf-normalizeNormalize Table/Column NamesWhether to change non-Avro-compatible characters in column names to Avro-compatible characters. For example, colons and periods will be changed to underscores in order to build a valid Avro record.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-user-logical-typesUse Avro Logical TypesWhether to use Avro Logical Types for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP columns. If disabled, written as string. If enabled, Logical types are used and written as its underlying type, specifically, DECIMAL/NUMBER as logical 'decimal': written as bytes with additional precision and scale meta data, DATE as logical 'date-millis': written as int denoting days since Unix epoch (1970-01-01), TIME as logical 'time-millis': written as int denoting milliseconds since Unix epoch, and TIMESTAMP as logical 'timestamp-millis': written as long denoting milliseconds since Unix epoch. If a reader of written Avro records also knows these logical types, then these values can be deserialized with more context depending on reader implementation.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsecompression-formatCompression FormatCompression type to use when writing Avro files. Default is None.NONEBZIP2BZIP2DEFLATEDEFLATENONENONESNAPPYSNAPPYLZOLZOtruefalsefalseNONEfalsefalsedbf-default-precisionDefault Decimal PrecisionWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'precision' denoting number of available digits is required. Generally, precision is defined by column data type definition or database engines default. However undefined precision (0) can be returned from some database engines. 'Default Decimal Precision' is used when writing those undefined precision numbers.10truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedbf-default-scaleDefault Decimal ScaleWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'scale' denoting number of available decimal digits is required. Generally, scale is defined by column data type definition or database engines default. However when undefined precision (0) is returned, scale can also be uncertain with some database engines. 'Default Decimal Scale' is used when writing those undefined numbers. If a value has more decimals than specified scale, then the value will be rounded-up, e.g. 1.53 becomes 2 with scale 0, and 1.5 with scale 1.0truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseesql-max-rowsMax Rows Per Flow FileThe maximum number of result rows that will be included in a single FlowFile. This will allow you to break up very large result sets into multiple FlowFiles. If the value specified is zero, then all rows are returned in a single FlowFile.0truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseesql-output-batch-sizeOutput Batch SizeThe number of output FlowFiles to queue before committing the process session. When set to zero, the session will be committed when all result set rows have been processed and the output FlowFiles are ready for transfer to the downstream relationship. For large result sets, this can cause a large burst of FlowFiles to be transferred at the end of processor execution. If this property is set, then when the specified number of FlowFiles are ready for transfer, then the session will be committed, thus releasing the FlowFiles to the downstream relationship. NOTE: The fragment.count attribute will not be set on FlowFiles when this property is set.0truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseesql-fetch-sizeFetch SizeThe number of result rows to be fetched from the result set at a time. This is a hint to the database driver and may not be honored and/or exact. If the value specified is zero, then the hint is ignored.0truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseesql-auto-commitSet Auto CommitEnables or disables the auto commit functionality of the DB connection. Default value is 'true'. The default value can be used with most of the JDBC drivers and this functionality doesn't have any impact in most of the cases since this processor is used to read data. However, for some JDBC drivers such as PostgreSQL driver, it is required to disable the auto committing functionality to limit the number of result rows fetching at a time. When auto commit is enabled, postgreSQL driver loads whole result set to memory at once. This could lead for a large amount of memory usage when executing queries which fetch large data sets. More Details of this behaviour in PostgreSQL driver can be found in https://jdbc.postgresql.org//documentation/head/query.html. truetruetruefalsefalsetruefalsefalseNONEfalsefalsesql.args.N.typeSQL type argument to be suppliedIncoming FlowFiles are expected to be parametrized SQL statements. The type of each Parameter is specified as an integer that represents the JDBC Type of the parameter. The following types are accepted: [LONGNVARCHAR: -16], [BIT: -7], [BOOLEAN: 16], [TINYINT: -6], [BIGINT: -5], [LONGVARBINARY: -4], [VARBINARY: -3], [BINARY: -2], [LONGVARCHAR: -1], [CHAR: 1], [NUMERIC: 2], [DECIMAL: 3], [INTEGER: 4], [SMALLINT: 5] [FLOAT: 6], [REAL: 7], [DOUBLE: 8], [VARCHAR: 12], [DATE: 91], [TIME: 92], [TIMESTAMP: 93], [VARCHAR: 12], [CLOB: 2005], [NCLOB: 2011]NONEsql.args.N.valueArgument to be suppliedIncoming FlowFiles are expected to be parametrized SQL statements. The value of the Parameters are specified as sql.args.1.value, sql.args.2.value, sql.args.3.value, and so on. The type of the sql.args.1.value Parameter is specified by the sql.args.1.type attribute.NONEsql.args.N.formatSQL format argument to be suppliedThis attribute is always optional, but default options may not always work for your data. Incoming FlowFiles are expected to be parametrized SQL statements. In some cases a format option needs to be specified, currently this is only applicable for binary data types, dates, times and timestamps. Binary Data Types (defaults to 'ascii') - ascii: each string character in your attribute value represents a single byte. This is the format provided by Avro Processors. base64: the string is a Base64 encoded string that can be decoded to bytes. hex: the string is hex encoded with all letters in upper case and no '0x' at the beginning. Dates/Times/Timestamps - Date, Time and Timestamp formats all support both custom formats or named format ('yyyy-MM-dd','ISO_OFFSET_DATE_TIME') as specified according to java.time.format.DateTimeFormatter. If not specified, a long value input is expected to be an unix epoch (milli seconds from 1970/1/1), or a string value in 'yyyy-MM-dd' format for Date, 'HH:mm:ss.SSS' for Time (some database engines e.g. Derby or MySQL do not support milliseconds and will truncate milliseconds), 'yyyy-MM-dd HH:mm:ss.SSS' for Timestamp is used.NONEtruesuccessSuccessfully created FlowFile from SQL query result set.falsefailureSQL query execution failed. Incoming FlowFile will be penalized and routed to this relationshipfalsesql.args.N.typeIncoming FlowFiles are expected to be parametrized SQL statements. The type of each Parameter is specified as an integer that represents the JDBC Type of the parameter. The following types are accepted: [LONGNVARCHAR: -16], [BIT: -7], [BOOLEAN: 16], [TINYINT: -6], [BIGINT: -5], [LONGVARBINARY: -4], [VARBINARY: -3], [BINARY: -2], [LONGVARCHAR: -1], [CHAR: 1], [NUMERIC: 2], [DECIMAL: 3], [INTEGER: 4], [SMALLINT: 5] [FLOAT: 6], [REAL: 7], [DOUBLE: 8], [VARCHAR: 12], [DATE: 91], [TIME: 92], [TIMESTAMP: 93], [VARCHAR: 12], [CLOB: 2005], [NCLOB: 2011]sql.args.N.valueIncoming FlowFiles are expected to be parametrized SQL statements. The value of the Parameters are specified as sql.args.1.value, sql.args.2.value, sql.args.3.value, and so on. The type of the sql.args.1.value Parameter is specified by the sql.args.1.type attribute.sql.args.N.formatThis attribute is always optional, but default options may not always work for your data. Incoming FlowFiles are expected to be parametrized SQL statements. In some cases a format option needs to be specified, currently this is only applicable for binary data types, dates, times and timestamps. Binary Data Types (defaults to 'ascii') - ascii: each string character in your attribute value represents a single byte. This is the format provided by Avro Processors. base64: the string is a Base64 encoded string that can be decoded to bytes. hex: the string is hex encoded with all letters in upper case and no '0x' at the beginning. Dates/Times/Timestamps - Date, Time and Timestamp formats all support both custom formats or named format ('yyyy-MM-dd','ISO_OFFSET_DATE_TIME') as specified according to java.time.format.DateTimeFormatter. If not specified, a long value input is expected to be an unix epoch (milli seconds from 1970/1/1), or a string value in 'yyyy-MM-dd' format for Date, 'HH:mm:ss.SSS' for Time (some database engines e.g. Derby or MySQL do not support milliseconds and will truncate milliseconds), 'yyyy-MM-dd HH:mm:ss.SSS' for Timestamp is used.executesql.row.countContains the number of rows returned by the query. If 'Max Rows Per Flow File' is set, then this number will reflect the number of rows in the Flow File instead of the entire result set.executesql.query.durationCombined duration of the query execution time and fetch time in milliseconds. If 'Max Rows Per Flow File' is set, then this number will reflect only the fetch time for the rows in the Flow File instead of the entire result set.executesql.query.executiontimeDuration of the query execution time in milliseconds. This number will reflect the query execution time regardless of the 'Max Rows Per Flow File' setting.executesql.query.fetchtimeDuration of the result set fetch time in milliseconds. If 'Max Rows Per Flow File' is set, then this number will reflect only the fetch time for the rows in the Flow File instead of the entire result set.executesql.resultset.indexAssuming multiple result sets are returned, the zero based index of this result set.executesql.error.messageIf processing an incoming flow file causes an Exception, the Flow File is routed to failure and this attribute is set to the exception message.fragment.identifierIf 'Max Rows Per Flow File' is set then all FlowFiles from the same query result set will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.fragment.countIf 'Max Rows Per Flow File' is set then this is the total number of FlowFiles produced by a single ResultSet. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming ResultSet. If Output Batch Size is set, then this attribute will not be populated.fragment.indexIf 'Max Rows Per Flow File' is set then the position of this FlowFile in the list of outgoing FlowFiles that were all derived from the same result set FlowFile. This can be used in conjunction with the fragment.identifier attribute to know which FlowFiles originated from the same query result set and in what order FlowFiles were producedinput.flowfile.uuidIf the processor has an incoming connection, outgoing FlowFiles will have this attribute set to the value of the input FlowFile's UUID. If there is no incoming connection, the attribute will not be added.INPUT_ALLOWEDorg.apache.nifi.processors.standard.ExecuteSQLRecordPROCESSORExecutes provided SQL select query. Query result will be converted to the format specified by a Record Writer. Streaming is used so arbitrarily large result sets are supported. This processor can be scheduled to run on a timer, or cron expression, using the standard scheduling methods, or it can be triggered by an incoming FlowFile. If it is triggered by an incoming FlowFile, then attributes of that FlowFile will be available when evaluating the select query, and the query may use the ? to escape parameters. In this case, the parameters to use must exist as FlowFile attributes with the naming convention sql.args.N.type and sql.args.N.value, where N is a positive integer. The sql.args.N.type is expected to be a number indicating the JDBC Type. The content of the FlowFile is expected to be in UTF-8 format. FlowFile attribute 'executesql.row.count' indicates how many rows were selected.sqlselectjdbcquerydatabaserecordDatabase Connection Pooling ServiceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain connection to databaseorg.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsesql-pre-querySQL Pre-QueryA semicolon-delimited list of queries executed before the main SQL query is executed. For example, set session properties before main query. It's possible to include semicolons in the statements themselves by escaping them with a backslash ('\;'). Results/outputs from these queries will be suppressed if there are no errors.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSQL select querySQL select queryThe SQL select query to execute. The query can be empty, a constant value, or built from attributes using Expression Language. If this property is specified, it will be used regardless of the content of incoming flowfiles. If this property is empty, the content of the incoming flow file is expected to contain a valid SQL select query, to be issued by the processor to the database. Note that Expression Language is not evaluated for flow file contents.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesql-post-querySQL Post-QueryA semicolon-delimited list of queries executed after the main SQL query is executed. Example like setting session properties after main query. It's possible to include semicolons in the statements themselves by escaping them with a backslash ('\;'). Results/outputs from these queries will be suppressed if there are no errors.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax Wait TimeMax Wait TimeThe maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseesqlrecord-record-writerRecord WriterSpecifies the Controller Service to use for writing results to a FlowFile. The Record Writer may use Inherit Schema to emulate the inferred schema behavior, i.e. an explicit schema need not be defined in the writer, and will be supplied by the same logic used to infer the schema from the column types.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseesqlrecord-normalizeNormalize Table/Column NamesWhether to change characters in column names. For example, colons and periods will be changed to underscores.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-user-logical-typesUse Avro Logical TypesWhether to use Avro Logical Types for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP columns. If disabled, written as string. If enabled, Logical types are used and written as its underlying type, specifically, DECIMAL/NUMBER as logical 'decimal': written as bytes with additional precision and scale meta data, DATE as logical 'date-millis': written as int denoting days since Unix epoch (1970-01-01), TIME as logical 'time-millis': written as int denoting milliseconds since Unix epoch, and TIMESTAMP as logical 'timestamp-millis': written as long denoting milliseconds since Unix epoch. If a reader of written Avro records also knows these logical types, then these values can be deserialized with more context depending on reader implementation.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-default-precisionDefault Decimal PrecisionWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'precision' denoting number of available digits is required. Generally, precision is defined by column data type definition or database engines default. However undefined precision (0) can be returned from some database engines. 'Default Decimal Precision' is used when writing those undefined precision numbers.10truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedbf-default-scaleDefault Decimal ScaleWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'scale' denoting number of available decimal digits is required. Generally, scale is defined by column data type definition or database engines default. However when undefined precision (0) is returned, scale can also be uncertain with some database engines. 'Default Decimal Scale' is used when writing those undefined numbers. If a value has more decimals than specified scale, then the value will be rounded-up, e.g. 1.53 becomes 2 with scale 0, and 1.5 with scale 1.0truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseesql-max-rowsMax Rows Per Flow FileThe maximum number of result rows that will be included in a single FlowFile. This will allow you to break up very large result sets into multiple FlowFiles. If the value specified is zero, then all rows are returned in a single FlowFile.0truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseesql-output-batch-sizeOutput Batch SizeThe number of output FlowFiles to queue before committing the process session. When set to zero, the session will be committed when all result set rows have been processed and the output FlowFiles are ready for transfer to the downstream relationship. For large result sets, this can cause a large burst of FlowFiles to be transferred at the end of processor execution. If this property is set, then when the specified number of FlowFiles are ready for transfer, then the session will be committed, thus releasing the FlowFiles to the downstream relationship. NOTE: The fragment.count attribute will not be set on FlowFiles when this property is set.0truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseesql-fetch-sizeFetch SizeThe number of result rows to be fetched from the result set at a time. This is a hint to the database driver and may not be honored and/or exact. If the value specified is zero, then the hint is ignored.0truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseesql-auto-commitSet Auto CommitEnables or disables the auto commit functionality of the DB connection. Default value is 'true'. The default value can be used with most of the JDBC drivers and this functionality doesn't have any impact in most of the cases since this processor is used to read data. However, for some JDBC drivers such as PostgreSQL driver, it is required to disable the auto committing functionality to limit the number of result rows fetching at a time. When auto commit is enabled, postgreSQL driver loads whole result set to memory at once. This could lead for a large amount of memory usage when executing queries which fetch large data sets. More Details of this behaviour in PostgreSQL driver can be found in https://jdbc.postgresql.org//documentation/head/query.html. truetruetruefalsefalsetruefalsefalseNONEfalsefalsesql.args.N.typeSQL type argument to be suppliedIncoming FlowFiles are expected to be parametrized SQL statements. The type of each Parameter is specified as an integer that represents the JDBC Type of the parameter. The following types are accepted: [LONGNVARCHAR: -16], [BIT: -7], [BOOLEAN: 16], [TINYINT: -6], [BIGINT: -5], [LONGVARBINARY: -4], [VARBINARY: -3], [BINARY: -2], [LONGVARCHAR: -1], [CHAR: 1], [NUMERIC: 2], [DECIMAL: 3], [INTEGER: 4], [SMALLINT: 5] [FLOAT: 6], [REAL: 7], [DOUBLE: 8], [VARCHAR: 12], [DATE: 91], [TIME: 92], [TIMESTAMP: 93], [VARCHAR: 12], [CLOB: 2005], [NCLOB: 2011]NONEsql.args.N.valueArgument to be suppliedIncoming FlowFiles are expected to be parametrized SQL statements. The value of the Parameters are specified as sql.args.1.value, sql.args.2.value, sql.args.3.value, and so on. The type of the sql.args.1.value Parameter is specified by the sql.args.1.type attribute.NONEsql.args.N.formatSQL format argument to be suppliedThis attribute is always optional, but default options may not always work for your data. Incoming FlowFiles are expected to be parametrized SQL statements. In some cases a format option needs to be specified, currently this is only applicable for binary data types, dates, times and timestamps. Binary Data Types (defaults to 'ascii') - ascii: each string character in your attribute value represents a single byte. This is the format provided by Avro Processors. base64: the string is a Base64 encoded string that can be decoded to bytes. hex: the string is hex encoded with all letters in upper case and no '0x' at the beginning. Dates/Times/Timestamps - Date, Time and Timestamp formats all support both custom formats or named format ('yyyy-MM-dd','ISO_OFFSET_DATE_TIME') as specified according to java.time.format.DateTimeFormatter. If not specified, a long value input is expected to be an unix epoch (milli seconds from 1970/1/1), or a string value in 'yyyy-MM-dd' format for Date, 'HH:mm:ss.SSS' for Time (some database engines e.g. Derby or MySQL do not support milliseconds and will truncate milliseconds), 'yyyy-MM-dd HH:mm:ss.SSS' for Timestamp is used.NONEtruesuccessSuccessfully created FlowFile from SQL query result set.falsefailureSQL query execution failed. Incoming FlowFile will be penalized and routed to this relationshipfalsesql.args.N.typeIncoming FlowFiles are expected to be parametrized SQL statements. The type of each Parameter is specified as an integer that represents the JDBC Type of the parameter. The following types are accepted: [LONGNVARCHAR: -16], [BIT: -7], [BOOLEAN: 16], [TINYINT: -6], [BIGINT: -5], [LONGVARBINARY: -4], [VARBINARY: -3], [BINARY: -2], [LONGVARCHAR: -1], [CHAR: 1], [NUMERIC: 2], [DECIMAL: 3], [INTEGER: 4], [SMALLINT: 5] [FLOAT: 6], [REAL: 7], [DOUBLE: 8], [VARCHAR: 12], [DATE: 91], [TIME: 92], [TIMESTAMP: 93], [VARCHAR: 12], [CLOB: 2005], [NCLOB: 2011]sql.args.N.valueIncoming FlowFiles are expected to be parametrized SQL statements. The value of the Parameters are specified as sql.args.1.value, sql.args.2.value, sql.args.3.value, and so on. The type of the sql.args.1.value Parameter is specified by the sql.args.1.type attribute.sql.args.N.formatThis attribute is always optional, but default options may not always work for your data. Incoming FlowFiles are expected to be parametrized SQL statements. In some cases a format option needs to be specified, currently this is only applicable for binary data types, dates, times and timestamps. Binary Data Types (defaults to 'ascii') - ascii: each string character in your attribute value represents a single byte. This is the format provided by Avro Processors. base64: the string is a Base64 encoded string that can be decoded to bytes. hex: the string is hex encoded with all letters in upper case and no '0x' at the beginning. Dates/Times/Timestamps - Date, Time and Timestamp formats all support both custom formats or named format ('yyyy-MM-dd','ISO_OFFSET_DATE_TIME') as specified according to java.time.format.DateTimeFormatter. If not specified, a long value input is expected to be an unix epoch (milli seconds from 1970/1/1), or a string value in 'yyyy-MM-dd' format for Date, 'HH:mm:ss.SSS' for Time (some database engines e.g. Derby or MySQL do not support milliseconds and will truncate milliseconds), 'yyyy-MM-dd HH:mm:ss.SSS' for Timestamp is used.executesql.row.countContains the number of rows returned in the select queryexecutesql.query.durationCombined duration of the query execution time and fetch time in millisecondsexecutesql.query.executiontimeDuration of the query execution time in millisecondsexecutesql.query.fetchtimeDuration of the result set fetch time in millisecondsexecutesql.resultset.indexAssuming multiple result sets are returned, the zero based index of this result set.executesql.error.messageIf processing an incoming flow file causes an Exception, the Flow File is routed to failure and this attribute is set to the exception message.fragment.identifierIf 'Max Rows Per Flow File' is set then all FlowFiles from the same query result set will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.fragment.countIf 'Max Rows Per Flow File' is set then this is the total number of FlowFiles produced by a single ResultSet. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming ResultSet. If Output Batch Size is set, then this attribute will not be populated.fragment.indexIf 'Max Rows Per Flow File' is set then the position of this FlowFile in the list of outgoing FlowFiles that were all derived from the same result set FlowFile. This can be used in conjunction with the fragment.identifier attribute to know which FlowFiles originated from the same query result set and in what order FlowFiles were producedinput.flowfile.uuidIf the processor has an incoming connection, outgoing FlowFiles will have this attribute set to the value of the input FlowFile's UUID. If there is no incoming connection, the attribute will not be added.mime.typeSets the mime.type attribute to the MIME Type specified by the Record Writer.record.countThe number of records output by the Record Writer.INPUT_ALLOWEDorg.apache.nifi.processors.standard.ExecuteStreamCommandPROCESSORThe ExecuteStreamCommand processor provides a flexible way to integrate external commands and scripts into NiFi data flows. ExecuteStreamCommand can pass the incoming FlowFile's content to the command that it executes similarly how piping works.command executioncommandstreamexecuteWorking DirectoryWorking DirectoryThe directory to use as the current working directory when executing the commandfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCommand PathCommand PathSpecifies the command to be executed; if just the name of an executable is provided, it must be in the user's environment PATH.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseargumentsStrategyCommand Arguments StrategyStrategy for configuring arguments to be supplied to the command.Command Arguments PropertyCommand Arguments PropertyCommand Arguments PropertyArguments to be supplied to the executable are taken from the Command Arguments propertyDynamic Property ArgumentsDynamic Property ArgumentsArguments to be supplied to the executable are taken from dynamic properties with pattern of 'command.argument.<commandIndex>'falsefalsefalseNONEfalsefalseCommand ArgumentsCommand ArgumentsThe arguments to supply to the executable delimited by the ';' character.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseargumentsStrategyCommand Arguments StrategyCommand Arguments PropertyArgument DelimiterArgument DelimiterDelimiter to use to separate arguments for a command [default: ;]. Must be a single character;truefalsefalseNONEfalsefalseargumentsStrategyCommand Arguments StrategyCommand Arguments PropertyIgnore STDINIgnore STDINIf true, the contents of the incoming flowfile will not be passed to the executing commandfalsetruetruefalsefalsefalsefalsefalseNONEfalsefalseOutput Destination AttributeOutput Destination AttributeIf set, the output of the stream command will be put into an attribute of the original FlowFile instead of a separate FlowFile. There will no longer be a relationship for 'output stream' or 'nonzero status'. The value of this property will be the key for the output attribute.falsefalsefalseNONEfalsefalseMax Attribute LengthMax Attribute LengthIf routing the output of the stream command to an attribute, the number of characters put to the attribute value will be at most this amount. This is important because attributes are held in memory and large attributes will quickly cause out of memory issues. If the output goes longer than this value, it will truncated to fit. Consider making this smaller if able.256falsefalsefalseNONEfalsefalseOutput MIME TypeOutput MIME TypeSpecifies the value to set for the "mime.type" attribute. This property is ignored if 'Output Destination Attribute' is set.falsefalsefalseNONEfalsefalseAn environment variable nameAn environment variable valueThese environment variables are passed to the process spawned by this ProcessorNONEcommand.argument.<commandIndex>Argument to be supplied to the commandThese arguments are supplied to the process spawned by this Processor when using the Command Arguments Strategy : Dynamic Property Arguments. <commandIndex> is a number and it will determine the order.NONEtrueoutput streamThe destination path for the flow file created from the command's output, if the returned status code is zero.falsenonzero statusThe destination path for the flow file created from the command's output, if the returned status code is non-zero. All flow files routed to this relationship will be penalized.falseoriginalThe original FlowFile will be routed. It will have new attributes detailing the result of the script execution.falseexecution.commandThe name of the command executedexecution.command.argsThe semi-colon delimited list of arguments. Sensitive properties will be maskedexecution.statusThe exit status code returned from executing the commandexecution.errorAny error messages returned from executing the commandmime.typeSets the MIME type of the output if the 'Output MIME Type' property is set and 'Output Destination Attribute' is not settrueexecute codeProvides operator the ability to execute arbitrary code assuming all permissions that NiFi has.INPUT_REQUIREDorg.apache.nifi.processors.standard.ExtractGrokPROCESSOREvaluates one or more Grok Expressions against the content of a FlowFile, adding the results as attributes or replacing the content of the FlowFile with a JSON notation of the matched contentgroklogtextparsedelimitextractGrok ExpressionGrok ExpressionGrok expression. If other Grok expressions are referenced in this expression, they must be provided in the Grok Pattern File if set or exist in the default Grok patternstruefalsefalseNONEfalsefalseGrok Pattern fileGrok PatternsCustom Grok pattern definitions. These definitions will be loaded after the default Grok patterns. The Grok Parser will use the default Grok patterns when this property is not configured.falsefalsefalseNONEfalsefalseSINGLEURLFILETEXTDestinationDestinationControl if Grok output value is written as a new flowfile attributes, in this case each of the Grok identifier that is matched in the flowfile will be added as an attribute, prefixed with "grok." or written in the flowfile content. Writing to flowfile content will overwrite any existing flowfile content.flowfile-attributeflowfile-attributeflowfile-attributeflowfile-contentflowfile-contenttruefalsefalseNONEfalsefalseCharacter SetCharacter SetThe Character Set in which the file is encodedUTF-8truefalsefalseNONEfalsefalseMaximum Buffer SizeMaximum Buffer SizeSpecifies the maximum amount of data to buffer (per file) in order to apply the Grok expressions. Files larger than the specified maximum will not be fully evaluated.1 MBtruefalsefalseNONEfalsefalseNamed captures onlyNamed captures onlyOnly store named captures from grokfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseKeep Empty CapturesKeep Empty CapturesIf true, then empty capture values will be included in the returned capture map.truetruetruefalsefalsetruefalsefalseNONEfalsefalseunmatchedFlowFiles are routed to this relationship when no provided Grok Expression matches the content of the FlowFilefalsematchedFlowFiles are routed to this relationship when the Grok Expression is successfully evaluated and the FlowFile is modified as a resultfalsegrok.XXXWhen operating in flowfile-attribute mode, each of the Grok identifier that is matched in the flowfile will be added as an attribute, prefixed with "grok." For example,if the grok identifier "timestamp" is matched, then the value will be added to an attribute named "grok.timestamp"truetruereference remote resourcesPatterns can reference resources over HTTPINPUT_REQUIREDorg.apache.nifi.processors.standard.ExtractRecordSchemaPROCESSORExtracts the record schema from the FlowFile using the supplied Record Reader and writes it to the `avro.schema` attribute.recordgenericschemajsoncsvavrofreeformtextxmlrecord-readerRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsecache-sizeSchema Cache SizeSpecifies the number of schemas to cache. This value should reflect the expected number of different schemas that may be in the incoming FlowFiles. This ensures more efficient retrieval of the schemas and thus the processor performance.10truefalsefalseNONEfalsefalsesuccessFlowFiles whose record schemas are successfully extracted will be routed to this relationshipfalsefailureIf a FlowFile's record schema cannot be extracted from the configured input format, the FlowFile will be routed to this relationshipfalserecord.error.messageThis attribute provides on failure the error message encountered by the Reader.avro.schemaThis attribute provides the schema extracted from the input FlowFile using the provided RecordReader.truetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ExtractTextPROCESSOREvaluates one or more Regular Expressions against the content of a FlowFile. The results of those Regular Expressions are assigned to FlowFile Attributes. Regular Expressions are entered by adding user-defined properties; the name of the property maps to the Attribute Name into which the result will be placed. The attributes are generated differently based on the enabling of named capture groups. If named capture groups are not enabled: The first capture group, if any found, will be placed into that attribute name.But all capture groups, including the matching string sequence itself will also be provided at that attribute name with an index value provided, with the exception of a capturing group that is optional and does not match - for example, given the attribute name "regex" and expression "abc(def)?(g)" we would add an attribute "regex.1" with a value of "def" if the "def" matched. If the "def" did not match, no attribute named "regex.1" would be added but an attribute named "regex.2" with a value of "g" will be added regardless.If named capture groups are enabled: Each named capture group, if found will be placed into the attributes name with the name provided. If enabled the matching string sequence itself will be placed into the attribute name. If multiple matches are enabled, and index will be applied after the first set of matches. The exception is a capturing group that is optional and does not match For example, given the attribute name "regex" and expression "abc(?<NAMED>def)?(?<NAMED-TWO>g)" we would add an attribute "regex.NAMED" with the value of "def" if the "def" matched. We would add an attribute "regex.NAMED-TWO" with the value of "g" if the "g" matched regardless. The value of the property must be a valid Regular Expressions with one or more capturing groups. If named capture groups are enabled, all capture groups must be named. If they are not, then the processor configuration will fail validation. If the Regular Expression matches more than once, only the first match will be used unless the property enabling repeating capture group is set to true. If any provided Regular Expression matches, the FlowFile(s) will be routed to 'matched'. If no provided Regular Expression matches, the FlowFile will be routed to 'unmatched' and no attributes will be applied to the FlowFile.evaluateextractTextRegular ExpressionregexCharacter SetCharacter SetThe Character Set in which the file is encodedUTF-8truefalsefalseNONEfalsefalseMaximum Buffer SizeMaximum Buffer SizeSpecifies the maximum amount of data to buffer (per FlowFile) in order to apply the regular expressions. FlowFiles larger than the specified maximum will not be fully evaluated.1 MBtruefalsefalseNONEfalsefalseMaximum Capture Group LengthMaximum Capture Group LengthSpecifies the maximum number of characters a given capture group value can have. Any characters beyond the max will be truncated.1024falsefalsefalseNONEfalsefalseEnable Canonical EquivalenceEnable Canonical EquivalenceIndicates that two characters match only when their full canonical decompositions match.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseEnable Case-insensitive MatchingEnable Case-insensitive MatchingIndicates that two characters match even if they are in a different case. Can also be specified via the embedded flag (?i).falsetruetruefalsefalsetruefalsefalseNONEfalsefalsePermit Whitespace and Comments in PatternPermit Whitespace and Comments in PatternIn this mode, whitespace is ignored, and embedded comments starting with # are ignored until the end of a line. Can also be specified via the embedded flag (?x).falsetruetruefalsefalsetruefalsefalseNONEfalsefalseEnable DOTALL ModeEnable DOTALL ModeIndicates that the expression '.' should match any character, including a line terminator. Can also be specified via the embedded flag (?s).falsetruetruefalsefalsetruefalsefalseNONEfalsefalseEnable Literal Parsing of the PatternEnable Literal Parsing of the PatternIndicates that Metacharacters and escape characters should be given no special meaning.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseEnable Multiline ModeEnable Multiline ModeIndicates that '^' and '$' should match just after and just before a line terminator or end of sequence, instead of only the beginning or end of the entire input. Can also be specified via the embeded flag (?m).falsetruetruefalsefalsetruefalsefalseNONEfalsefalseEnable Unicode-aware Case FoldingEnable Unicode-aware Case FoldingWhen used with 'Enable Case-insensitive Matching', matches in a manner consistent with the Unicode Standard. Can also be specified via the embedded flag (?u).falsetruetruefalsefalsetruefalsefalseNONEfalsefalseEnable Unicode Predefined Character ClassesEnable Unicode Predefined Character ClassesSpecifies conformance with the Unicode Technical Standard #18: Unicode Regular Expression Annex C: Compatibility Properties. Can also be specified via the embedded flag (?U).falsetruetruefalsefalsetruefalsefalseNONEfalsefalseEnable Unix Lines ModeEnable Unix Lines ModeIndicates that only the ' ' line terminator is recognized in the behavior of '.', '^', and '$'. Can also be specified via the embedded flag (?d).falsetruetruefalsefalsetruefalsefalseNONEfalsefalseInclude Capture Group 0Include Capture Group 0Indicates that Capture Group 0 should be included as an attribute. Capture Group 0 represents the entirety of the regular expression match, is typically not used, and could have considerable length.truetruetruefalsefalsetruefalsefalseNONEfalsefalseEnable repeating capture groupEnable repeating capture groupIf set to true, every string matching the capture groups will be extracted. Otherwise, if the Regular Expression matches more than once, only the first match will be extracted.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseEnable named group supportEnable named group supportIf set to true, when named groups are present in the regular expression, the name of the group will be used in the attribute name as opposed to the group index. All capturing groups must be named, if the number of groups (not including capture group 0) does not equal the number of named groups validation will fail.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseA FlowFile attributeA Regular Expression with one or more capturing groupThe first capture group, if any found, will be placed into that attribute name.But all capture groups, including the matching string sequence itself will also be provided at that attribute name with an index value provided.NONEunmatchedFlowFiles are routed to this relationship when no provided Regular Expression matches the content of the FlowFilefalsematchedFlowFiles are routed to this relationship when the Regular Expression is successfully evaluated and the FlowFile is modified as a resultfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.FetchDistributedMapCachePROCESSORComputes cache key(s) from FlowFile attributes, for each incoming FlowFile, and fetches the value(s) from the Distributed Map Cache associated with each key. If configured without a destination attribute, the incoming FlowFile's content is replaced with the binary data received by the Distributed Map Cache. If there is no value stored under that key then the flow file will be routed to 'not-found'. Note that the processor will always attempt to read the entire cached value into memory before placing it in it's destination. This could be potentially problematic if the cached value is very large.mapcachefetchdistributedCache Entry IdentifierCache Entry IdentifierA comma-delimited list of FlowFile attributes, or the results of Attribute Expression Language statements, which will be evaluated against a FlowFile in order to determine the value(s) used to identify duplicates; it is these values that are cached. NOTE: Only a single Cache Entry Identifier is allowed unless Put Cache Value In Attribute is specified. Multiple cache lookups are only supported when the destination is a set of attributes (see the documentation for 'Put Cache Value In Attribute' for more details including naming convention.${hash.value}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseDistributed Cache ServiceDistributed Cache ServiceThe Controller Service that is used to get the cached values.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsePut Cache Value In AttributePut Cache Value In AttributeIf set, the cache value received will be put into an attribute of the FlowFile instead of a the content of theFlowFile. The attribute key to put to is determined by evaluating value of this property. If multiple Cache Entry Identifiers are selected, multiple attributes will be written, using the evaluated value of this property, appended by a period (.) and the name of the cache entry identifier.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax Length To Put In AttributeMax Length To Put In AttributeIf routing the cache value to an attribute of the FlowFile (by setting the "Put Cache Value in attribute" property), the number of characters put to the attribute value will be at most this amount. This is important because attributes are held in memory and large attributes will quickly cause out of memory issues. If the output goes longer than this value, it will be truncated to fit. Consider making this smaller if able.256falsefalsefalseNONEfalsefalseCharacter SetCharacter SetThe Character Set in which the cached value is encoded. This will only be used when routing to an attribute.UTF-8falsefalsefalseNONEfalsefalsesuccessIf the cache was successfully communicated with it will be routed to this relationshipfalsefailureIf unable to communicate with the cache or if the cache entry is evaluated to be blank, the FlowFile will be penalized and routed to this relationshipfalsenot-foundIf a FlowFile's Cache Entry Identifier was not found in the cache, it will be routed to this relationshipfalseuser-definedIf the 'Put Cache Value In Attribute' property is set then whatever it is set to will become the attribute key and the value would be whatever the response was from the Distributed Map Cache. If multiple cache entry identifiers are selected, multiple attributes will be written, using the evaluated value of this property, appended by a period (.) and the name of the cache entry identifier. For example, if the Cache Entry Identifier property is set to 'id,name', and the user-defined property is named 'fetched', then two attributes will be written, fetched.id and fetched.name, containing their respective values.trueINPUT_REQUIREDorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientServiceorg.apache.nifi.distributed.cache.server.map.DistributedMapCacheServerorg.apache.nifi.processors.standard.PutDistributedMapCacheorg.apache.nifi.processors.standard.FetchFTPPROCESSORFetches the content of a file from a remote FTP server and overwrites the contents of an incoming FlowFile with the content of the remote file.ftpgetretrievefilesfetchremoteingestsourceinputHostnameHostnameThe fully-qualified hostname or IP address of the host to fetch the data fromtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port to connect to on the remote host to fetch the data from21truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote FileRemote FileThe fully qualified filename on the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCompletion StrategyCompletion StrategySpecifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred.NoneNoneNoneLeave the file as-isMove FileMove FileMove the file to the directory specified by the <Move Destination Directory> propertyDelete FileDelete FileDeletes the original file from the remote systemtruefalsefalseNONEfalsefalseMove Destination DirectoryMove Destination DirectoryThe directory on the remote server to move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to 'Move File'. The specified directory must already exist on the remote system if 'Create Directory' is disabled, or the rename will fail.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCreate DirectoryCreate DirectoryUsed when 'Completion Strategy' is 'Move File'. Specifies whether or not the remote directory should be created if it does not exist.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseConnection ModeConnection ModeThe FTP Connection ModePassiveActiveActivePassivePassivefalsefalsefalseNONEfalsefalseTransfer ModeTransfer ModeThe FTP Transfer ModeBinaryBinaryBinaryASCIIASCIIfalsefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseInternal Buffer SizeInternal Buffer SizeSet the internal buffer size for buffered data streams16KBfalsefalsefalseNONEfalsefalsefetchfiletransfer-notfound-loglevelLog level when file not foundLog level to use in case the file does not exist when the processor is triggeredERRORTRACETRACEDEBUGDEBUGINFOINFOWARNWARNERRORERRORFATALFATALNONENONEtruefalsefalseNONEfalsefalseftp-use-utf8Use UTF-8 EncodingTells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalsecomms.failureAny FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship.falsenot.foundAny FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship.falsepermission.deniedAny FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationship.falseftp.remote.hostThe hostname or IP address from which the file was pulledftp.remote.portThe port that was used to communicate with the remote FTP serverftp.remote.filenameThe name of the remote file that was pulledfilenameThe filename is updated to point to the filename fo the remote filepathIf the Remote File contains a directory name, that directory name will be added to the FlowFile using the 'path' attributefetch.failure.reasonThe name of the failure relationship applied when routing to any failure relationshipINPUT_REQUIREDRetrieve all files in a directory of an FTP Serverftpfiletransformstateretrievefetchallstreamorg.apache.nifi.processors.standard.ListFTPThe "Hostname" property should be set to the fully qualified hostname of the FTP Server. It's a good idea to parameterize this property by setting it to something like `#{FTP_SERVER}`. The "Remote Path" property must be set to the directory on the FTP Server where the files reside. If the flow being built is to be reused elsewhere, it's a good idea to parameterize this property by setting it to something like `#{FTP_REMOTE_PATH}`. Configure the "Username" property to the appropriate username for logging into the FTP Server. It's usually a good idea to parameterize this property by setting it to something like `#{FTP_USERNAME}`. Configure the "Password" property to the appropriate password for the provided username. It's usually a good idea to parameterize this property by setting it to something like `#{FTP_PASSWORD}`. The 'success' Relationship of this Processor is then connected to FetchFTP. org.apache.nifi.processors.standard.FetchFTP"Hostname" = "${ftp.remote.host}" "Remote File" = "${path}/${filename}" "Username" = "${ftp.listing.user}" "Password" = "#{FTP_PASSWORD}" org.apache.nifi.processors.standard.GetSFTPorg.apache.nifi.processors.standard.PutSFTPorg.apache.nifi.processors.standard.GetFTPorg.apache.nifi.processors.standard.PutFTPorg.apache.nifi.processors.standard.FetchFilePROCESSORReads the contents of a file from disk and streams it into the contents of an incoming FlowFile. Once this is done, the file is optionally moved elsewhere or deleted to help keep the file system organized.localfilesfilesystemingestingressgetsourceinputfetchFile to FetchFile to FetchThe fully-qualified filename of the file to fetch from the file system${absolute.path}/${filename}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCompletion StrategyCompletion StrategySpecifies what to do with the original file on the file system once it has been pulled into NiFiNoneNoneNoneLeave the file as-isMove FileMove FileMoves the file to the directory specified by the <Move Destination Directory> propertyDelete FileDelete FileDeletes the original file from the file systemtruefalsefalseNONEfalsefalseMove Destination DirectoryMove Destination DirectoryThe directory to the move the original file to once it has been fetched from the file system. This property is ignored unless the Completion Strategy is set to "Move File". If the directory does not exist, it will be created.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMove Conflict StrategyMove Conflict StrategyIf Completion Strategy is set to Move File and a file already exists in the destination directory with the same name, this property specifies how that naming conflict should be resolvedRenameRenameRenameThe existing destination file should remain intact. The newly ingested file should be moved to the destination directory but be renamed to a random filenameReplace FileReplace FileThe newly ingested file should replace the existing file in the Destination DirectoryKeep ExistingKeep ExistingThe existing file should in the Destination Directory should stay intact and the newly ingested file should be deletedFailFailThe existing destination file should remain intact and the incoming FlowFile should be routed to failuretruefalsefalseNONEfalsefalseLog level when file not foundLog level when file not foundLog level to use in case the file does not exist when the processor is triggeredERRORTRACETRACEDEBUGDEBUGINFOINFOWARNWARNERRORERRORFATALFATALNONENONEtruefalsefalseNONEfalsefalseLog level when permission deniedLog level when permission deniedLog level to use in case user exceptionfactory does not have sufficient permissions to read the fileERRORTRACETRACEDEBUGDEBUGINFOINFOWARNWARNERRORERRORFATALFATALNONENONEtruefalsefalseNONEfalsefalsesuccessAny FlowFile that is successfully fetched from the file system will be transferred to this Relationship.falsenot.foundAny FlowFile that could not be fetched from the file system because the file could not be found will be transferred to this Relationship.falsepermission.deniedAny FlowFile that could not be fetched from the file system due to the user running NiFi not having sufficient permissions will be transferred to this Relationship.falsefailureAny FlowFile that could not be fetched from the file system for any reason other than insufficient permissions or the file not existing will be transferred to this Relationship.falseread filesystemProvides operator the ability to read from any file that NiFi has access to.write filesystemProvides operator the ability to delete any file that NiFi has access to.INPUT_REQUIREDIngest all files from a directory into NiFilocalfilesfilesystemingestingressgetsourceinputfetchorg.apache.nifi.processors.standard.ListFileConfigure the "Input Directory" property to point to the directory that you want to ingest files from. Set the "Input Directory Location" property to "Local" Optionally, set "Minimum File Age" to a small value such as "1 min" to avoid ingesting files that are still being written to. Connect the 'success' Relationship to the FetchFile processor. org.apache.nifi.processors.standard.FetchFileSet the "File to Fetch" property to `${absolute.path}/${filename}` Set the "Completion Strategy" property to `None` Ingest specific files from a directory into NiFi, filtering on filenamelocalfilesfilesystemingestingressgetsourceinputfetchfilterorg.apache.nifi.processors.standard.ListFileConfigure the "Input Directory" property to point to the directory that you want to ingest files from. Set the "Input Directory Location" property to "Local" Set the "File Filter" property to a Regular Expression that matches the filename (without path) of the files that you want to ingest. For example, to ingest all .jpg files, set the value to `.*\.jpg` Optionally, set "Minimum File Age" to a small value such as "1 min" to avoid ingesting files that are still being written to. Connect the 'success' Relationship to the FetchFile processor. org.apache.nifi.processors.standard.FetchFileSet the "File to Fetch" property to `${absolute.path}/${filename}` Set the "Completion Strategy" property to `None` org.apache.nifi.processors.standard.GetFileorg.apache.nifi.processors.standard.PutFileorg.apache.nifi.processors.standard.ListFileorg.apache.nifi.processors.standard.FetchSFTPPROCESSORFetches the content of a file from a remote SFTP server and overwrites the contents of an incoming FlowFile with the content of the remote file.sftpgetretrievefilesfetchremoteingestsourceinputHostnameHostnameThe fully-qualified hostname or IP address of the host to fetch the data fromtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port to connect to on the remote host to fetch the data from22truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalsePrivate Key PathPrivate Key PathThe fully qualified path to the Private Key filefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILEPrivate Key PassphrasePrivate Key PassphrasePassword for the private keyfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote FileRemote FileThe fully qualified filename on the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCompletion StrategyCompletion StrategySpecifies what to do with the original file on the server once it has been pulled into NiFi. If the Completion Strategy fails, a warning will be logged but the data will still be transferred.NoneNoneNoneLeave the file as-isMove FileMove FileMove the file to the directory specified by the <Move Destination Directory> propertyDelete FileDelete FileDeletes the original file from the remote systemtruefalsefalseNONEfalsefalseMove Destination DirectoryMove Destination DirectoryThe directory on the remote server to move the original file to once it has been ingested into NiFi. This property is ignored unless the Completion Strategy is set to 'Move File'. The specified directory must already exist on the remote system if 'Create Directory' is disabled, or the rename will fail.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCreate DirectoryCreate DirectoryUsed when 'Completion Strategy' is 'Move File'. Specifies whether or not the remote directory should be created if it does not exist.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseDisable Directory ListingDisable Directory ListingControl how 'Move Destination Directory' is created when 'Completion Strategy' is 'Move File' and 'Create Directory' is enabled. If set to 'true', directory listing is not performed prior to create missing directories. By default, this processor executes a directory listing command to see target directory existence before creating missing directories. However, there are situations that you might need to disable the directory listing such as the following. Directory listing might fail with some permission setups (e.g. chmod 100) on a directory. Also, if any other SFTP client created the directory after this processor performed a listing and before a directory creation request by this processor is finished, then an error is returned because the directory already exists.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseSend Keep Alive On TimeoutSend Keep Alive On TimeoutSend a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.truetruetruefalsefalsetruefalsefalseNONEfalsefalseHost Key FileHost Key FileIf supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be usedfalsefalsefalseNONEfalsefalseSINGLEFILEStrict Host Key CheckingStrict Host Key CheckingIndicates whether or not strict enforcement of hosts keys should be appliedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalsefetchfiletransfer-notfound-loglevelLog level when file not foundLog level to use in case the file does not exist when the processor is triggeredERRORTRACETRACEDEBUGDEBUGINFOINFOWARNWARNERRORERRORFATALFATALNONENONEtruefalsefalseNONEfalsefalseCiphers AllowedCiphers AllowedA comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctrfalsefalsetrueENVIRONMENTfalsefalseKey Algorithms AllowedKey Algorithms AllowedA comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.comfalsefalsetrueENVIRONMENTfalsefalseKey Exchange Algorithms AllowedKey Exchange Algorithms AllowedA comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-cfalsefalsetrueENVIRONMENTfalsefalseMessage Authentication Codes AllowedMessage Authentication Codes AllowedA comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.comfalsefalsetrueENVIRONMENTfalsefalsesuccessAll FlowFiles that are received are routed to successfalsecomms.failureAny FlowFile that could not be fetched from the remote server due to a communications failure will be transferred to this Relationship.falsenot.foundAny FlowFile for which we receive a 'Not Found' message from the remote server will be transferred to this Relationship.falsepermission.deniedAny FlowFile that could not be fetched from the remote server due to insufficient permissions will be transferred to this Relationship.falsesftp.remote.hostThe hostname or IP address from which the file was pulledsftp.remote.portThe port that was used to communicate with the remote SFTP serversftp.remote.filenameThe name of the remote file that was pulledfilenameThe filename is updated to point to the filename fo the remote filepathIf the Remote File contains a directory name, that directory name will be added to the FlowFile using the 'path' attributefetch.failure.reasonThe name of the failure relationship applied when routing to any failure relationshipINPUT_REQUIREDRetrieve all files in a directory of an SFTP Serversftpsecurefiletransformstateretrievefetchallstreamorg.apache.nifi.processors.standard.ListSFTPThe "Hostname" property should be set to the fully qualified hostname of the FTP Server. It's a good idea to parameterize this property by setting it to something like `#{SFTP_SERVER}`. The "Remote Path" property must be set to the directory on the FTP Server where the files reside. If the flow being built is to be reused elsewhere, it's a good idea to parameterize this property by setting it to something like `#{SFTP_REMOTE_PATH}`. Configure the "Username" property to the appropriate username for logging into the FTP Server. It's usually a good idea to parameterize this property by setting it to something like `#{SFTP_USERNAME}`. Configure the "Password" property to the appropriate password for the provided username. It's usually a good idea to parameterize this property by setting it to something like `#{SFTP_PASSWORD}`. The 'success' Relationship of this Processor is then connected to FetchSFTP. org.apache.nifi.processors.standard.FetchSFTP"Hostname" = "${sftp.remote.host}" "Remote File" = "${path}/${filename}" "Username" = "${sftp.listing.user}" "Password" = "#{SFTP_PASSWORD}" org.apache.nifi.processors.standard.GetSFTPorg.apache.nifi.processors.standard.PutSFTPorg.apache.nifi.processors.standard.GetFTPorg.apache.nifi.processors.standard.PutFTPorg.apache.nifi.processors.standard.FilterAttributePROCESSORFilters the attributes of a FlowFile by retaining specified attributes and removing the rest or by removing specified attributes and retaining the rest.attributesmodificationfilterretainremovedeleteregexregular expressionAttribute Expression LanguageFilter ModeFilter ModeSpecifies the strategy to apply on filtered attributes. Either 'Remove' or 'Retain' only the matching attributes.RetainRetainRetainRetains only the attributes matching the filter, all other attributes are removed.RemoveRemoveRemoves the attributes matching the filter, all other attributes are retained.truefalsefalseNONEfalsefalseAttribute Matching StrategyAttribute Matching StrategySpecifies the strategy to filter attributes by.Enumerate attributesEnumerate attributesEnumerate attributesProvides a set of attribute keys to filter for, separated by a comma delimiter ','.Use regular expressionUse regular expressionProvides a regular expression to match keys of attributes to filter for.truefalsefalseNONEfalsefalseFiltered AttributesFiltered AttributesA set of attribute names to filter from FlowFiles. Each attribute name is separated by the comma delimiter ','.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseAttribute Matching StrategyAttribute Matching StrategyEnumerate attributesFiltered Attributes PatternFiltered Attributes PatternA regular expression to match names of attributes to filter from FlowFiles.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseAttribute Matching StrategyAttribute Matching StrategyUse regular expressionsuccessAll successful FlowFiles are routed to this relationshipfalsetruetrueINPUT_REQUIREDRetain all FlowFile attributes matching a regular expressionINPUT_ALLOWEDSet "Filter Mode" to "Retain". Set "Attribute Matching Strategy" to "Use regular expression". Specify the "Filtered Attributes Pattern", e.g. "my-property|a-prefix[.].*". Remove only a specified set of FlowFile attributesINPUT_ALLOWEDSet "Filter Mode" to "Remove". Set "Attribute Matching Strategy" to "Enumerate attributes". Specify the set of "Filtered Attributes" using the delimiter comma ',', e.g. "my-property,other,filename". org.apache.nifi.processors.standard.FlattenJsonPROCESSORProvides the user with the ability to take a nested JSON document and flatten it into a simple key/value pair document. The keys are combined at each level with a user-defined separator that defaults to '.'. This Processor also allows to unflatten back the flattened json. It supports four kinds of flatten mode such as normal, keep-arrays, dot notation for MongoDB query and keep-primitive-arrays. Default flatten mode is 'keep-arrays'.jsonflattenunflattenflatten-json-separatorSeparatorThe separator character used for joining keys. Must be a JSON-legal character..falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseflatten-modeFlatten ModeSpecifies how json should be flattened/unflattenedkeep arraysnormalnormalFlattens every objects into a single level jsonkeep arrayskeep arraysFlattens every objects and keep arrays formatdot notationdot notationConforms to MongoDB dot notation to update also nested documentskeep primitive arrayskeep primitive arraysFlattens every objects except arrays which contain only primitive types (strings, numbers, booleans and null)truefalsefalseNONEfalsefalseignore-reserved-charactersIgnore Reserved CharactersIf true, reserved characters in keys will be ignoredfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseflatten-json-return-typeReturn TypeSpecifies the desired return type of json such as flatten/unflattenflattenflattenflattenunflattenunflattentruefalsefalseNONEfalsefalseflatten-json-character-setCharacter SetThe Character Set in which file is encodedUTF-8truefalsefalseNONEfalsefalseflatten-json-pretty-print-jsonPretty Print JSONSpecifies whether or not resulted json should be pretty printedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessSuccessfully flattened/unflattened files go to this relationship.falsefailureFiles that cannot be flattened/unflattened go to this relationship.falsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ForkEnrichmentPROCESSORUsed in conjunction with the JoinEnrichment processor, this processor is responsible for adding the attributes that are necessary for the JoinEnrichment processor to perform its function. Each incoming FlowFile will be cloned. The original FlowFile will have appropriate attributes added and then be transferred to the 'original' relationship. The clone will have appropriate attributes added and then be routed to the 'enrichment' relationship. See the documentation for the JoinEnrichment processor (and especially its Additional Details) for more information on how these Processors work together and how to perform enrichment tasks in NiFi by using these Processors.forkjoinenrichrecordenrichmentA clone of the incoming FlowFile will be routed to this relationship, after adding appropriate attributes.falseoriginalThe incoming FlowFile will be routed to this relationship, after adding appropriate attributes.falseenrichment.group.idThe Group ID to use in order to correlate the 'original' FlowFile with the 'enrichment' FlowFile.enrichment.roleThe role to use for enrichment. This will either be ORIGINAL or ENRICHMENT.truetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.JoinEnrichmentorg.apache.nifi.processors.standard.ForkRecordPROCESSORThis processor allows the user to fork a record into multiple records. The user must specify at least one Record Path, as a dynamic property, pointing to a field of type ARRAY containing RECORD objects. The processor accepts two modes: 'split' and 'extract'. In both modes, there is one record generated per element contained in the designated array. In the 'split' mode, each generated record will preserve the same schema as given in the input but the array will contain only one element. In the 'extract' mode, the element of the array must be of record type and will be the generated record. Additionally, in the 'extract' mode, it is possible to specify if each generated record should contain all the fields of the parent records from the root level to the extracted record. This assumes that the fields to add in the record are defined in the schema of the Record Writer controller service. See examples in the additional details documentation of this processor.forkrecordcontentarraystreameventrecord-readerRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsefork-modeModeSpecifies the forking mode of the processorsplitExtractextractGenerated records will be the elements of the arraySplitsplitGenerated records will preserve the input schema and will contain a one-element arraytruefalsefalseNONEfalsefalseinclude-parent-fieldsInclude Parent FieldsThis parameter is only valid with the 'extract' mode. If set to true, all the fields from the root level to the given array will be added as fields of each element of the array to fork.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseRecord Path propertyThe Record Path valueA Record Path value, pointing to a field of type ARRAY containing RECORD objectsFLOWFILE_ATTRIBUTESfailureIn case a FlowFile generates an error during the fork operation, it will be routed to this relationshipfalseoriginalThe original FlowFiles will be routed to this relationshipfalseforkThe FlowFiles containing the forked records will be routed to this relationshipfalserecord.countThe generated FlowFile will have a 'record.count' attribute indicating the number of records that were written to the FlowFile.mime.typeThe MIME Type indicated by the Record Writer<Attributes from Record Writer>Any Attribute that the configured Record Writer returns will be added to the FlowFile.truetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.GenerateFlowFilePROCESSORThis processor creates FlowFiles with random data or custom content. GenerateFlowFile is useful for load testing, configuration, and simulation. Also see DuplicateFlowFile for additional load testing.testrandomgenerateloadFile SizeFile SizeThe size of the file that will be used0BtruefalsefalseNONEfalsefalseBatch SizeBatch SizeThe number of FlowFiles to be transferred in each invocation1truefalsefalseNONEfalsefalseData FormatData FormatSpecifies whether the data should be Text or BinaryTextBinaryBinaryTextTexttruefalsefalseNONEfalsefalseUnique FlowFilesUnique FlowFilesIf true, each FlowFile that is generated will be unique. If false, a random value will be generated and all FlowFiles will get the same content but this offers much higher throughputfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsegenerate-ff-custom-textCustom TextIf Data Format is text and if Unique FlowFiles is false, then this custom text will be used as content of the generated FlowFiles and the File Size will be ignored. Finally, if Expression Language is used, evaluation will be performed only once per batch of generated FlowFilesfalsefalsetrueENVIRONMENTfalsefalsecharacter-setCharacter SetSpecifies the character set to use when writing the bytes of Custom Text to a flow file.UTF-8truefalsefalseNONEfalsefalsemime-typeMime TypeSpecifies the value to set for the "mime.type" attribute.falsefalsefalseNONEfalsefalseGenerated FlowFile attribute nameGenerated FlowFile attribute valueSpecifies an attribute on generated FlowFiles defined by the Dynamic Property's key and value. If Expression Language is used, evaluation will be performed only once per batch of generated FlowFiles.ENVIRONMENTsuccessfalsemime.typeSets the MIME type of the output if the 'Mime Type' property is settrueINPUT_FORBIDDENTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.GenerateRecordPROCESSORThis processor creates FlowFiles with records having random value for the specified fields. GenerateRecord is useful for testing, configuration, and simulation. It uses either user-defined properties to define a record schema or a provided schema and generates the specified number of records using random data for the fields in the schema.testrandomgeneratefakerecord-writerRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsenumber-of-recordsNumber of RecordsSpecifies how many records will be generated for each outgoing FlowFile.100truefalsetrueENVIRONMENTfalsefalsenullable-fieldsNullable FieldsWhether the generated fields will be nullable. Note that this property is ignored if Schema Text is set. Also it only affects the schema of the generated data, not whether any values will be null. If this property is true, see 'Null Value Percentage' to set the probability that any generated field will be null.truetruetruefalsefalsetruefalsefalseNONEfalsefalsenull-percentageNull Value PercentageThe percent probability (0-100%) that a generated value for any nullable field will be null. Set this property to zero to have no null values, or 100 to have all null values.0truefalsetrueENVIRONMENTfalsefalsenullable-fieldsNullable Fieldstrueschema-textSchema TextThe text of an Avro-formatted Schema used to generate record data. If this property is set, any user-defined properties are ignored.falsefalsetrueENVIRONMENTfalsefalseField name in generated recordFaker category for generated record valuesCustom properties define the generated record schema using configured field names and value data types in absence of the Schema Text propertyNONEsuccessFlowFiles that are successfully created will be routed to this relationshipfalsemime.typeSets the mime.type attribute to the MIME Type specified by the Record Writerrecord.countThe number of records in the FlowFiletrueINPUT_FORBIDDENorg.apache.nifi.processors.standard.GenerateTableFetchPROCESSORGenerates SQL select queries that fetch "pages" of rows from a table. The partition size property, along with the table's row count, determine the size and number of pages and generated FlowFiles. In addition, incremental fetching can be achieved by setting Maximum-Value Columns, which causes the processor to track the columns' maximum values, thus only fetching rows whose columns' values exceed the observed maximums. This processor is intended to be run on the Primary Node only. This processor can accept incoming connections; the behavior of the processor is different whether incoming connections are provided: - If no incoming connection(s) are specified, the processor will generate SQL queries on the specified processor schedule. Expression Language is supported for many fields, but no FlowFile attributes are available. However the properties will be evaluated using the Environment/System properties. - If incoming connection(s) are specified and no FlowFile is available to a processor task, no work will be performed. - If incoming connection(s) are specified and a FlowFile is available to a processor task, the FlowFile's attributes may be used in Expression Language for such fields as Table Name and others. However, the Max-Value Columns and Columns to Return fields must be empty or refer to columns that are available in each specified table.sqlselectjdbcquerydatabasefetchgenerateDatabase Connection Pooling ServiceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-fetch-db-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLtruefalsefalseNONEfalsefalseTable NameTable NameThe name of the database table to be queried.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseColumns to ReturnColumns to ReturnA comma-separated list of column names to be used in the query. If your database requires special treatment of the names (quoting, e.g.), each name should include such treatment. If no column names are supplied, all columns in the specified table will be returned. NOTE: It is important to use consistent column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMaximum-value ColumnsMaximum-value ColumnsA comma-separated list of column names. The processor will keep track of the maximum value for each column that has been returned since the processor started running. Using multiple columns implies an order to the column list, and each column's values are expected to increase more slowly than the previous columns' values. Thus, using multiple columns implies a hierarchical structure of columns, which is usually used for partitioning tables. This processor can be used to retrieve only those rows that have been added/updated since the last retrieval. Note that some JDBC types such as bit/boolean are not conducive to maintaining maximum value, so columns of these types should not be listed in this property, and will result in error(s) during processing. If no columns are provided, all rows from the table will be considered, which could have a performance impact. NOTE: It is important to use consistent max-value column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMax Wait TimeMax Wait TimeThe maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsegen-table-fetch-partition-sizePartition SizeThe number of result rows to be fetched by each generated SQL statement. The total number of rows in the table divided by the partition size gives the number of SQL statements (i.e. FlowFiles) generated. A value of zero indicates that a single FlowFile is to be generated whose SQL statement will fetch all rows in the table.10000truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsegen-table-column-for-val-partitioningColumn for Value PartitioningThe name of a column whose values will be used for partitioning. The default behavior is to use row numbers on the result set for partitioning into 'pages' to be fetched from the database, using an offset/limit strategy. However for certain databases, it can be more efficient under the right circumstances to use the column values themselves to define the 'pages'. This property should only be used when the default queries are not performing well, when there is no maximum-value column or a single maximum-value column whose type can be coerced to a long integer (i.e. not date or timestamp), and the column values are evenly distributed and not sparse, for best performance.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-where-clauseAdditional WHERE clauseA custom clause to be added in the WHERE condition when building SQL queries.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsegen-table-custom-orderby-columnCustom ORDER BY ColumnThe name of a column to be used for ordering the results if Max-Value Columns are not provided and partitioning is enabled. This property is ignored if either Max-Value Columns is set or Partition Size = 0. NOTE: If neither Max-Value Columns nor Custom ORDER BY Column is set, then depending on the the database/driver, the processor may report an error and/or the generated SQL may result in missing and/or duplicate rows. This is because without an explicit ordering, fetching each partition is done using an arbitrary ordering.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsegen-table-output-flowfile-on-zero-resultsOutput Empty FlowFile on Zero ResultsDepending on the specified properties, an execution of this processor may not result in any SQL statements generated. When this property is true, an empty FlowFile will be generated (having the parent of the incoming FlowFile if present) and transferred to the 'success' relationship. When this property is false, no output FlowFiles will be generated.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseinitial.maxvalue.<max_value_column>Initial maximum value for the specified columnSpecifies an initial max value for max value columns. Properties should be added in the format `initial.maxvalue.<max_value_column>`. This value is only used the first time the table is accessed (when a Maximum Value Column is specified). In the case of incoming connections, the value is only used the first time for each table specified in the FlowFiles.FLOWFILE_ATTRIBUTESsuccessSuccessfully created FlowFile from SQL query result set.falsefailureThis relationship is only used when SQL query execution (using an incoming FlowFile) failed. The incoming FlowFile will be penalized and routed to this relationship. If no incoming connection(s) are specified, this relationship is unused.falsegeneratetablefetch.sql.errorIf the processor has incoming connections, and processing an incoming FlowFile causes a SQL Exception, the FlowFile is routed to failure and this attribute is set to the exception message.generatetablefetch.tableNameThe name of the database table to be queried.generatetablefetch.columnNamesThe comma-separated list of column names used in the query.generatetablefetch.whereClauseWhere clause used in the query to get the expected rows.generatetablefetch.maxColumnNamesThe comma-separated list of column names used to keep track of data that has been returned since the processor started running.generatetablefetch.limitThe number of result rows to be fetched by the SQL statement.generatetablefetch.offsetOffset to be used to retrieve the corresponding partition.fragment.identifierAll FlowFiles generated from the same query result set will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.fragment.countThis is the total number of FlowFiles produced by a single ResultSet. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming ResultSet.fragment.indexThis is the position of this FlowFile in the list of outgoing FlowFiles that were all generated from the same execution. This can be used in conjunction with the fragment.identifier attribute to know which FlowFiles originated from the same execution and in what order FlowFiles were producedtrueAfter performing a query on the specified table, the maximum values for the specified column(s) will be retained for use in future executions of the query. This allows the Processor to fetch only those records that have max values greater than the retained values. This can be used for incremental fetching, fetching of newly added rows, etc. To clear the maximum values, clear the state of the processor per the State Management documentationCLUSTERINPUT_ALLOWEDorg.apache.nifi.processors.standard.QueryDatabaseTableorg.apache.nifi.processors.standard.ExecuteSQLorg.apache.nifi.processors.standard.ListDatabaseTablesorg.apache.nifi.processors.standard.GetFTPPROCESSORFetches files from an FTP Server and creates FlowFiles from themFTPgetretrievefilesfetchremoteingestsourceinputHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers21truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseConnection ModeConnection ModeThe FTP Connection ModePassiveActiveActivePassivePassivefalsefalsefalseNONEfalsefalseTransfer ModeTransfer ModeThe FTP Transfer ModeBinaryBinaryBinaryASCIIASCIIfalsefalsefalseNONEfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push filesfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFile Filter RegexFile Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetchedfalsefalsefalseNONEfalsefalsePath Filter RegexPath Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scannedfalsefalsefalseNONEfalsefalsePolling IntervalPolling IntervalDetermines how long to wait between fetching the listing for new files60 sectruefalsefalseNONEfalsefalseSearch RecursivelySearch RecursivelyIf true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefollow-symlinkFollow symlinkIf true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseIgnore Dotted FilesIgnore Dotted FilesIf true, files whose names begin with a dot (".") will be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseDelete OriginalDelete OriginalDetermines whether or not the file is deleted from the remote system after it has been successfully transferredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseMax SelectsMax SelectsThe maximum number of files to pull in a single connection100truefalsefalseNONEfalsefalseRemote Poll Batch SizeRemote Poll Batch SizeThe value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.5000truefalsefalseNONEfalsefalseUse Natural OrderingUse Natural OrderingIf true, will pull files in the order in which they are naturally listed; otherwise, the order in which the files will be pulled is not definedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseInternal Buffer SizeInternal Buffer SizeSet the internal buffer size for buffered data streams16KBfalsefalsefalseNONEfalsefalseftp-use-utf8Use UTF-8 EncodingTells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalsefilenameThe filename is set to the name of the file on the remote serverpathThe path is set to the path of the file's directory on the remote server. For example, if the <Remote Path> property is set to /tmp, files picked up from /tmp will have the path attribute set to /tmp. If the <Search Recursively> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to /tmp/abc/1/2/3file.lastModifiedTimeThe date and time that the source file was last modifiedfile.lastAccessTimeThe date and time that the file was last accessed. May not work on all file systemsfile.ownerThe numeric owner id of the source filefile.groupThe numeric group id of the source filefile.permissionsThe read/write/execute permissions of the source fileabsolute.pathThe full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative pathINPUT_FORBIDDENorg.apache.nifi.processors.standard.PutFTPorg.apache.nifi.processors.standard.GetFilePROCESSORCreates FlowFiles from files in a directory. NiFi will ignore files it doesn't have at least read permissions for.localfilesfilesystemingestingressgetsourceinputInput DirectoryInput DirectoryThe input directory from which to pull filestruefalsetrueENVIRONMENTfalsefalseFile FilterFile FilterOnly files whose names match the given regular expression will be picked up[^\.].*truefalsefalseNONEfalsefalsePath FilterPath FilterWhen Recurse Subdirectories is true, then only subdirectories whose path matches the given regular expression will be scannedfalsefalsefalseNONEfalsefalseBatch SizeBatch SizeThe maximum number of files to pull in each iteration10truefalsefalseNONEfalsefalseKeep Source FileKeep Source FileIf true, the file is not deleted after it has been copied to the Content Repository; this causes the file to be picked up continually and is useful for testing purposes. If not keeping original NiFi will need write permissions on the directory it is pulling from otherwise it will ignore the file.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseRecurse SubdirectoriesRecurse SubdirectoriesIndicates whether or not to pull files from subdirectoriestruetruetruefalsefalsetruefalsefalseNONEfalsefalsePolling IntervalPolling IntervalIndicates how long to wait before performing a directory listing0 sectruefalsefalseNONEfalsefalseIgnore Hidden FilesIgnore Hidden FilesIndicates whether or not hidden files should be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseMinimum File AgeMinimum File AgeThe minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored0 sectruefalsefalseNONEfalsefalseMaximum File AgeMaximum File AgeThe maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignoredfalsefalsefalseNONEfalsefalseMinimum File SizeMinimum File SizeThe minimum size that a file must be in order to be pulled0 BtruefalsefalseNONEfalsefalseMaximum File SizeMaximum File SizeThe maximum size that a file can be in order to be pulledfalsefalsefalseNONEfalsefalsesuccessAll files are routed to successfalsefilenameThe filename is set to the name of the file on diskpathThe path is set to the relative path of the file's directory on disk. For example, if the <Input Directory> property is set to /tmp, files picked up from /tmp will have the path attribute set to ./. If the <Recurse Subdirectories> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to abc/1/2/3file.creationTimeThe date and time that the file was created. May not work on all file systemsfile.lastModifiedTimeThe date and time that the file was last modified. May not work on all file systemsfile.lastAccessTimeThe date and time that the file was last accessed. May not work on all file systemsfile.ownerThe owner of the file. May not work on all file systemsfile.groupThe group owner of the file. May not work on all file systemsfile.permissionsThe read/write/execute permissions of the file. May not work on all file systemsabsolute.pathThe full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative pathtrueread filesystemProvides operator the ability to read from any file that NiFi has access to.write filesystemProvides operator the ability to delete any file that NiFi has access to.INPUT_FORBIDDENorg.apache.nifi.processors.standard.PutFileorg.apache.nifi.processors.standard.FetchFileorg.apache.nifi.processors.standard.GetSFTPPROCESSORFetches files from an SFTP Server and creates FlowFiles from themsftpgetretrievefilesfetchremoteingestsourceinputHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers22truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalsePrivate Key PathPrivate Key PathThe fully qualified path to the Private Key filefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILEPrivate Key PassphrasePrivate Key PassphrasePassword for the private keyfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push filesfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFile Filter RegexFile Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetchedfalsefalsefalseNONEfalsefalsePath Filter RegexPath Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scannedfalsefalsefalseNONEfalsefalsePolling IntervalPolling IntervalDetermines how long to wait between fetching the listing for new files60 sectruefalsefalseNONEfalsefalseSearch RecursivelySearch RecursivelyIf true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefollow-symlinkFollow symlinkIf true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseIgnore Dotted FilesIgnore Dotted FilesIf true, files whose names begin with a dot (".") will be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseDelete OriginalDelete OriginalDetermines whether or not the file is deleted from the remote system after it has been successfully transferredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseHost Key FileHost Key FileIf supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be usedfalsefalsefalseNONEfalsefalseSINGLEFILEMax SelectsMax SelectsThe maximum number of files to pull in a single connection100truefalsefalseNONEfalsefalseRemote Poll Batch SizeRemote Poll Batch SizeThe value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.5000truefalsefalseNONEfalsefalseStrict Host Key CheckingStrict Host Key CheckingIndicates whether or not strict enforcement of hosts keys should be appliedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseSend Keep Alive On TimeoutSend Keep Alive On TimeoutSend a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.truetruetruefalsefalsetruefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseUse Natural OrderingUse Natural OrderingIf true, will pull files in the order in which they are naturally listed; otherwise, the order in which the files will be pulled is not definedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseCiphers AllowedCiphers AllowedA comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctrfalsefalsetrueENVIRONMENTfalsefalseKey Algorithms AllowedKey Algorithms AllowedA comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.comfalsefalsetrueENVIRONMENTfalsefalseKey Exchange Algorithms AllowedKey Exchange Algorithms AllowedA comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-cfalsefalsetrueENVIRONMENTfalsefalseMessage Authentication Codes AllowedMessage Authentication Codes AllowedA comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.comfalsefalsetrueENVIRONMENTfalsefalsesuccessAll FlowFiles that are received are routed to successfalsefilenameThe filename is set to the name of the file on the remote serverpathThe path is set to the path of the file's directory on the remote server. For example, if the <Remote Path> property is set to /tmp, files picked up from /tmp will have the path attribute set to /tmp. If the <Search Recursively> property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to /tmp/abc/1/2/3file.lastModifiedTimeThe date and time that the source file was last modifiedfile.ownerThe numeric owner id of the source filefile.groupThe numeric group id of the source filefile.permissionsThe read/write/execute permissions of the source fileabsolute.pathThe full/absolute path from where a file was picked up. The current 'path' attribute is still populated, but may be a relative pathINPUT_FORBIDDENorg.apache.nifi.processors.standard.PutSFTPorg.apache.nifi.processors.standard.HandleHttpRequestPROCESSORStarts an HTTP Server and listens for HTTP Requests. For each request, creates a FlowFile and transfers to 'success'. This Processor is designed to be used in conjunction with the HandleHttpResponse Processor in order to create a Web Service. In case of a multipart request, one FlowFile is generated for each part.httphttpsrequestlisteningressweb serviceListening PortListening PortThe Port to listen on for incoming HTTP requests80truefalsetrueENVIRONMENTfalsefalseHostnameHostnameThe Hostname to bind to. If not specified, will bind to all hostsfalsefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceThe SSL Context Service to use in order to secure the server. If specified, the server will accept only HTTPS requests; otherwise, the server will accept only HTTP requestsorg.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseHTTP ProtocolsHTTP ProtocolsHTTP Protocols supported for Application Layer Protocol Negotiation with TLSHTTP_1_1http/1.1HTTP_1_1HTTP/1.1h2 http/1.1H2_HTTP_1_1HTTP/2 and HTTP/1.1 negotiated based on requested protocolsh2H2HTTP/2truefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceHTTP Context MapHTTP Context MapThe HTTP Context Map Controller Service to use for caching the HTTP Request Informationorg.apache.nifi.http.HttpContextMaporg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseAllowed PathsAllowed PathsA Regular Expression that specifies the valid HTTP Paths that are allowed in the incoming URL Requests. If this value is specified and the path of the HTTP Requests does not match this Regular Expression, the Processor will respond with a 404: NotFoundfalsefalsefalseNONEfalsefalseDefault URL Character SetDefault URL Character SetThe character set to use for decoding URL parameters if the HTTP Request does not supply oneUTF-8truefalsefalseNONEfalsefalseAllow GETAllow GETAllow HTTP GET MethodtruetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow POSTAllow POSTAllow HTTP POST MethodtruetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow PUTAllow PUTAllow HTTP PUT MethodtruetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow DELETEAllow DELETEAllow HTTP DELETE MethodtruetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow HEADAllow HEADAllow HTTP HEAD MethodfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseAllow OPTIONSAllow OPTIONSAllow HTTP OPTIONS MethodfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseMaximum ThreadsMaximum ThreadsThe maximum number of threads that the embedded HTTP server will use for handling requests.200truefalsefalseNONEfalsefalseAdditional HTTP MethodsAdditional HTTP MethodsA comma-separated list of non-standard HTTP Methods that should be allowedfalsefalsefalseNONEfalsefalseClient AuthenticationClient AuthenticationSpecifies whether or not the Processor should authenticate clients. This value is ignored if the <SSL Context Service> Property is not specified or the SSL Context provided uses only a KeyStore and not a TrustStore.No AuthenticationNo AuthenticationNo AuthenticationProcessor will not authenticate clients. Anyone can communicate with this Processor anonymouslyWant AuthenticationWant AuthenticationProcessor will try to verify the client but if unable to verify will allow the client to communicate anonymouslyNeed AuthenticationNeed AuthenticationProcessor will reject communications from any client unless the client provides a certificate that is trusted by the TrustStorespecified in the SSL Context ServicetruefalsefalseNONEfalsefalsecontainer-queue-sizeContainer Queue SizeThe size of the queue for Http Request Containers50truefalsefalseNONEfalsefalsemultipart-request-max-sizeMultipart Request Max SizeThe max size of the request. Only applies for requests with Content-Type: multipart/form-data, and is used to prevent denial of service type of attacks, to prevent filling up the heap or disk space1 MBtruefalsefalseNONEfalsefalsemultipart-read-buffer-sizeMultipart Read Buffer SizeThe threshold size, at which the contents of an incoming file would be written to disk. Only applies for requests with Content-Type: multipart/form-data. It is used to prevent denial of service type of attacks, to prevent filling up the heap or disk space.512 KBtruefalsefalseNONEfalsefalseparameters-to-attributesParameters to Attributes ListA comma-separated list of HTTP parameters or form data to output as attributesfalsefalsefalseNONEfalsefalsesuccessAll content that is received is routed to the 'success' relationshipfalsehttp.context.identifierAn identifier that allows the HandleHttpRequest and HandleHttpResponse to coordinate which FlowFile belongs to which HTTP Request/Response.mime.typeThe MIME Type of the data, according to the HTTP Header "Content-Type"http.servlet.pathThe part of the request URL that is considered the Servlet Pathhttp.context.pathThe part of the request URL that is considered to be the Context Pathhttp.methodThe HTTP Method that was used for the request, such as GET or POSThttp.local.nameIP address/hostname of the serverhttp.server.portListening port of the serverhttp.query.stringThe query string portion of the Request URLhttp.remote.hostThe hostname of the requestorhttp.remote.addrThe hostname:port combination of the requestorhttp.remote.userThe username of the requestorhttp.protocolThe protocol used to communicatehttp.request.uriThe full Request URLhttp.auth.typeThe type of HTTP Authorization usedhttp.principal.nameThe name of the authenticated user making the requesthttp.query.param.XXXEach of query parameters in the request will be added as an attribute, prefixed with "http.query.param."http.param.XXXForm parameters in the request that are configured by "Parameters to Attributes List" will be added as an attribute, prefixed with "http.param.". Putting form parameters of large size is not recommended.http.subject.dnThe Distinguished Name of the requestor. This value will not be populated unless the Processor is configured to use an SSLContext Servicehttp.issuer.dnThe Distinguished Name of the entity that issued the Subject's certificate. This value will not be populated unless the Processor is configured to use an SSLContext Servicehttp.certificate.sans.N.nameX.509 Client Certificate Subject Alternative Name value from mutual TLS authentication. The attribute name has a zero-based index ordered according to the content of Client Certificatehttp.certificate.sans.N.nameTypeX.509 Client Certificate Subject Alternative Name type from mutual TLS authentication. The attribute name has a zero-based index ordered according to the content of Client Certificate. The attribute value is one of the General Names from RFC 3280 Section 4.1.2.7http.headers.XXXEach of the HTTP Headers that is received in the request will be added as an attribute, prefixed with "http.headers." For example, if the request contains an HTTP Header named "x-my-header", then the value will be added to an attribute named "http.headers.x-my-header"http.headers.multipart.XXXEach of the HTTP Headers that is received in the multipart request will be added as an attribute, prefixed with "http.headers.multipart." For example, if the multipart request contains an HTTP Header named "content-disposition", then the value will be added to an attribute named "http.headers.multipart.content-disposition"http.multipart.sizeFor requests with Content-Type "multipart/form-data", the part's content size is recorded into this attributehttp.multipart.content.typeFor requests with Content-Type "multipart/form-data", the part's content type is recorded into this attributehttp.multipart.nameFor requests with Content-Type "multipart/form-data", the part's name is recorded into this attributehttp.multipart.filenameFor requests with Content-Type "multipart/form-data", when the part contains an uploaded file, the name of the file is recorded into this attribute. Files are stored temporarily at the default temporary-file directory specified in "java.io.File" Java Docs)http.multipart.fragments.sequence.numberFor requests with Content-Type "multipart/form-data", the part's index is recorded into this attribute. The index starts with 1.http.multipart.fragments.total.numberFor requests with Content-Type "multipart/form-data", the count of all parts is recorded into this attribute.INPUT_FORBIDDENorg.apache.nifi.processors.standard.HandleHttpResponseorg.apache.nifi.processors.standard.HandleHttpResponsePROCESSORSends an HTTP Response to the Requestor that generated a FlowFile. This Processor is designed to be used in conjunction with the HandleHttpRequest in order to create a web service.httphttpsresponseegressweb serviceHTTP Status CodeHTTP Status CodeThe HTTP Status Code to use when responding to the HTTP Request. See Section 10 of RFC 2616 for more information.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHTTP Context MapHTTP Context MapThe HTTP Context Map Controller Service to use for caching the HTTP Request Informationorg.apache.nifi.http.HttpContextMaporg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseAttributes to add to the HTTP Response (Regex)Attributes to add to the HTTP Response (Regex)Specifies the Regular Expression that determines the names of FlowFile attributes that should be added to the HTTP responsefalsefalsefalseNONEfalsefalseAn HTTP header nameAn HTTP header valueThese HTTPHeaders are set in the HTTP ResponseFLOWFILE_ATTRIBUTESsuccessFlowFiles will be routed to this Relationship after the response has been successfully sent to the requestorfalsefailureFlowFiles will be routed to this Relationship if the Processor is unable to respond to the requestor. This may happen, for instance, if the connection times out or if NiFi is restarted before responding to the HTTP Request.falsehttp.context.identifierThe value of this attribute is used to lookup the HTTP Response so that the proper message can be sent back to the requestor. If this attribute is missing, the FlowFile will be routed to 'failure.'http.request.uriValue of the URI requested by the client. Used for provenance event.http.remote.hostIP address of the client. Used for provenance event.http.local.nameIP address/hostname of the server. Used for provenance event.http.server.portListening port of the server. Used for provenance event.http.subject.dnSSL distinguished name (if any). Used for provenance event.INPUT_REQUIREDorg.apache.nifi.processors.standard.HandleHttpRequestorg.apache.nifi.http.StandardHttpContextMaporg.apache.nifi.processors.standard.IdentifyMimeTypePROCESSORAttempts to identify the MIME Type used for a FlowFile. If the MIME Type can be identified, an attribute with the name 'mime.type' is added with the value being the MIME Type. If the MIME Type cannot be determined, the value will be set to 'application/octet-stream'. In addition, the attribute 'mime.extension' will be set if a common file extension for the MIME Type is known. If the MIME Type detected is of type text/*, attempts to identify the charset used and an attribute with the name 'mime.charset' is added with the value being the charset.compressiongzipbzip2zipMIMEmime.typefileidentifyuse-filename-in-detectionUse Filename In DetectionIf true will pass the filename to Tika to aid in detection.truetruetruefalsefalsetruefalsefalseNONEfalsefalseconfig-strategyConfig StrategySelect the loading strategy for MIME Type configuration to be used.PresetPresetPresetUse default NiFi MIME Types.ReplaceReplaceUse config MIME Types only.MergeMergeUse config together with default NiFi MIME Types.truefalsefalseNONEfalsefalseconfig-bodyConfig BodyBody of MIME type config file. Only one of Config File or Config Body may be used.falsefalsefalseNONEfalsefalseconfig-strategyConfig StrategyReplaceMergeconfig-fileConfig FilePath to MIME type config file. Only one of Config File or Config Body may be used.falsefalsetrueENVIRONMENTfalsefalseconfig-strategyConfig StrategyReplaceMergesuccessAll FlowFiles are routed to successfalsemime.typeThis Processor sets the FlowFile's mime.type attribute to the detected MIME Type. If unable to detect the MIME Type, the attribute's value will be set to application/octet-streammime.extensionThis Processor sets the FlowFile's mime.extension attribute to the file extension associated with the detected MIME Type. If there is no correlated extension, the attribute's value will be emptymime.charsetThis Processor sets the FlowFile's mime.charset attribute to the detected charset. If unable to detect the charset or the detected MIME type is not of type text/*, the attribute will not be settruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.InvokeHTTPPROCESSORAn HTTP client processor which can interact with a configurable HTTP Endpoint. The destination URL and HTTP Method are configurable. When the HTTP Method is PUT, POST or PATCH, the FlowFile contents are included as the body of the request and FlowFile attributes are converted to HTTP headers, optionally, based on configuration properties.httphttpsrestclientHTTP MethodHTTP MethodHTTP request method (GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS). Arbitrary methods are also supported. Methods other than POST, PUT and PATCH will be sent without a message body.GETtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHTTP URLHTTP URLHTTP remote URL including a scheme of http or https, as well as a hostname or IP address with optional port and path elements. Any encoding of the URL must be done by the user.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHTTP/2 DisabledHTTP/2 DisabledDisable negotiation of HTTP/2 protocol. HTTP/2 requires TLS. HTTP/1.1 protocol supported is required when HTTP/2 is disabled.FalseTrueTrueFalseFalsetruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceSSL Context Service provides trusted certificates and client certificates for TLS communication.org.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutMaximum time to wait for initial socket connection to the HTTP URL.5 secstruefalsefalseNONEfalsefalseSocket Read TimeoutSocket Read TimeoutMaximum time to wait for receiving responses from a socket connection to the HTTP URL.15 secstruefalsefalseNONEfalsefalseSocket Write TimeoutSocket Write TimeoutMaximum time to wait for write operations while sending requests from a socket connection to the HTTP URL.15 secstruefalsefalseNONEfalsefalseSocket Idle TimeoutSocket Idle TimeoutMaximum time to wait before closing idle connections to the HTTP URL.5 minstruefalsefalseNONEfalsefalseSocket Idle ConnectionsSocket Idle ConnectionsMaximum number of idle connections to the HTTP URL.5truefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.org.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseRequest OAuth2 Access Token ProviderRequest OAuth2 Access Token ProviderEnables managed retrieval of OAuth2 Bearer Token applied to HTTP requests using the Authorization Header.org.apache.nifi.oauth2.OAuth2AccessTokenProviderorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseRequest UsernameRequest UsernameThe username provided for authentication of HTTP requests. Encoded using Base64 for HTTP Basic Authentication as described in RFC 7617.falsefalsefalseNONEfalsefalseRequest PasswordRequest PasswordThe password provided for authentication of HTTP requests. Encoded using Base64 for HTTP Basic Authentication as described in RFC 7617.falsetruefalseNONEfalsefalseRequest Digest Authentication EnabledRequest Digest Authentication EnabledEnable Digest Authentication on HTTP requests with Username and Password credentials as described in RFC 7616.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseRequest UsernameRequest UsernameRequest Failure Penalization EnabledRequest Failure Penalization EnabledEnable penalization of request FlowFiles when receiving HTTP response with a status code between 400 and 499.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseRequest Body EnabledRequest Body EnabledEnable sending HTTP request body for PATCH, POST, or PUT methods.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseHTTP MethodHTTP MethodPOSTPATCHPUTRequest Multipart Form-Data NameRequest Multipart Form-Data NameEnable sending HTTP request body formatted using multipart/form-data and using the form name configured.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseRequest Body EnabledRequest Body EnabledtrueRequest Multipart Form-Data Filename EnabledRequest Multipart Form-Data Filename EnabledEnable sending the FlowFile filename attribute as the filename parameter in the Content-Disposition Header for multipart/form-data HTTP requests.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseRequest Multipart Form-Data NameRequest Multipart Form-Data NameRequest Chunked Transfer-Encoding EnabledRequest Chunked Transfer-Encoding EnabledEnable sending HTTP requests with the Transfer-Encoding Header set to chunked, and disable sending the Content-Length Header. Transfer-Encoding applies to the body in HTTP/1.1 requests as described in RFC 7230 Section 3.3.1falsetruetruefalsefalsetruefalsefalseNONEfalsefalseHTTP MethodHTTP MethodPOSTPATCHPUTRequest Content-EncodingRequest Content-EncodingHTTP Content-Encoding applied to request body during transmission. The receiving server must support the selected encoding to avoid request failures.DISABLEDDISABLEDDISABLEDContent encoding not applied during transmissionGZIPGZIPGzip content encoding and HTTP Content-Encoding header applied during transmissiontruefalsefalseNONEfalsefalseHTTP MethodHTTP MethodPOSTPATCHPUTRequest Content-TypeRequest Content-TypeHTTP Content-Type Header applied to when sending an HTTP request body for PATCH, POST, or PUT methods. The Content-Type defaults to application/octet-stream when not configured.${mime.type}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHTTP MethodHTTP MethodPOSTPATCHPUTRequest Date Header EnabledRequest Date Header EnabledEnable sending HTTP Date Header on HTTP requests as described in RFC 7231 Section 7.1.1.2.TrueTrueTrueFalseFalsetruefalsefalseNONEfalsefalseRequest Header Attributes PatternRequest Header Attributes PatternRegular expression that defines which FlowFile attributes to send as HTTP headers in the request. If not defined, no attributes are sent as headers. Dynamic properties will be always be sent as headers. The dynamic property name will be the header key and the dynamic property value, interpreted as Expression Language, will be the header value. Attributes and their values are limited to ASCII characters due to the requirement of the HTTP protocol.falsefalsefalseNONEfalsefalseRequest User-AgentRequest User-AgentHTTP User-Agent Header applied to requests. RFC 7231 Section 5.5.3 describes recommend formatting.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseResponse Body Attribute NameResponse Body Attribute NameFlowFile attribute name used to write an HTTP response body for FlowFiles transferred to the Original relationship.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseResponse Body Attribute SizeResponse Body Attribute SizeMaximum size in bytes applied when writing an HTTP response body to a FlowFile attribute. Attributes exceeding the maximum will be truncated.256falsefalsefalseNONEfalsefalseResponse Body Attribute NameResponse Body Attribute NameResponse Body IgnoredResponse Body IgnoredDisable writing HTTP response FlowFiles to Response relationshipfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseResponse Cache EnabledResponse Cache EnabledEnable HTTP response caching described in RFC 7234. Caching responses considers ETag and other headers.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseResponse Cache SizeResponse Cache SizeMaximum size of HTTP response cache in bytes. Caching responses considers ETag and other headers.10MBtruefalsefalseNONEfalsefalseResponse Cache EnabledResponse Cache EnabledtrueResponse Cookie StrategyResponse Cookie StrategyStrategy for accepting and persisting HTTP cookies. Accepting cookies enables persistence across multiple requests.DISABLEDDISABLEDDISABLEDACCEPT_ALLACCEPT_ALLtruefalsefalseNONEfalsefalseResponse Generation RequiredResponse Generation RequiredEnable generation and transfer of a FlowFile to the Response relationship regardless of HTTP response received.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseResponse FlowFile Naming StrategyResponse FlowFile Naming StrategyDetermines the strategy used for setting the filename attribute of FlowFiles transferred to the Response relationship.RANDOMRANDOMRANDOMFlowFile filename attribute will be a random value.URL_PATHURL_PATHFlowFile filename attribute will be extracted from the remote URL path. The attribute may contain URL encoded characters. If the path doesn't exist, the attribute will be a random value.truefalsefalseNONEfalsefalseResponse Header Request Attributes EnabledResponse Header Request Attributes EnabledEnable adding HTTP response headers as attributes to FlowFiles transferred to the Original relationship.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseResponse Redirects EnabledResponse Redirects EnabledEnable following HTTP redirects sent with HTTP 300 series responses as described in RFC 7231 Section 6.4.TrueTrueTrueFalseFalsetruefalsefalseNONEfalsefalseHeader NameAttribute Expression LanguageSend request header with a key matching the Dynamic Property Key and a value created by evaluating the Attribute Expression Language set in the value of the Dynamic Property.FLOWFILE_ATTRIBUTESpost:form:<NAME>Attribute Expression LanguageWhen the HTTP Method is POST, dynamic properties with the property name in the form of post:form:<NAME>, where the <NAME> will be the form data name, will be used to fill out the multipart form parts. If send message body is false, the flowfile will not be sent, but any other form data will be.FLOWFILE_ATTRIBUTEStrueOriginalRequest FlowFiles transferred when receiving HTTP responses with a status code between 200 and 299.falseFailureRequest FlowFiles transferred when receiving socket communication errors.falseRetryRequest FlowFiles transferred when receiving HTTP responses with a status code between 500 and 599.falseResponseResponse FlowFiles transferred when receiving HTTP responses with a status code between 200 and 299.falseNo RetryRequest FlowFiles transferred when receiving HTTP responses with a status code between 400 an 499.falseinvokehttp.status.codeThe status code that is returnedinvokehttp.status.messageThe status message that is returnedinvokehttp.response.bodyIn the instance where the status code received is not a success (2xx) then the response body will be put to the 'invokehttp.response.body' attribute of the request FlowFile.invokehttp.request.urlThe original request URLinvokehttp.request.durationDuration (in milliseconds) of the HTTP call to the external endpointinvokehttp.response.urlThe URL that was ultimately requested after any redirects were followedinvokehttp.tx.idThe transaction ID that is returned after reading the responseinvokehttp.remote.dnThe DN of the remote serverinvokehttp.java.exception.classThe Java exception class raised when the processor failsinvokehttp.java.exception.messageThe Java exception message raised when the processor failsuser-definedIf the 'Put Response Body In Attribute' property is set then whatever it is set to will become the attribute key and the value would be the body of the HTTP response.trueINPUT_ALLOWEDorg.apache.nifi.processors.standard.JoinEnrichmentPROCESSORJoins together Records from two different FlowFiles where one FlowFile, the 'original' contains arbitrary records and the second FlowFile, the 'enrichment' contains additional data that should be used to enrich the first. See Additional Details for more information on how to configure this processor and the different use cases that it aims to accomplish.forkjoinenrichmentrecordsqlwraprecordpathmergecombinestreamsOriginal Record ReaderOriginal Record ReaderThe Record Reader for reading the 'original' FlowFileorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseEnrichment Record ReaderEnrichment Record ReaderThe Record Reader for reading the 'enrichment' FlowFileorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseRecord WriterRecord WriterThe Record Writer to use for writing the results. If the Record Writer is configured to inherit the schema from the Record, the schema that it will inherit will be the result of merging both the 'original' record schema and the 'enrichment' record schema.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseJoin StrategyJoin StrategySpecifies how to join the two FlowFiles into a single FlowFileWrapperWrapperWrapperThe output is a Record that contains two fields: (1) 'original', containing the Record from the original FlowFile and (2) 'enrichment' containing the corresponding Record from the enrichment FlowFile. Records will be correlated based on their index in the FlowFile. If one FlowFile has more Records than the other, a null value will be used.SQLSQLThe output is derived by evaluating a SQL SELECT statement that allows for two tables: 'original' and 'enrichment'. This allows for SQL JOIN statements to be used in order to correlate the Records of the two FlowFiles, so the index in which the Record is encountered in the FlowFile does not matter.Insert Enrichment FieldsInsert Enrichment FieldsThe enrichment is joined together with the original FlowFile by placing all fields of the enrichment Record into the corresponding Record from the original FlowFile. Records will be correlated based on their index in the FlowFile.truefalsefalseNONEfalsefalseSQLSQLThe SQL SELECT statement to evaluate. Expression Language may be provided, but doing so may result in poorer performance. Because this Processor is dealing with two FlowFiles at a time, it's also important to understand how attributes will be referenced. If both FlowFiles have an attribute with the same name but different values, the Expression Language will resolve to the value provided by the 'enrichment' FlowFile.SELECT original.*, enrichment.* FROM original LEFT OUTER JOIN enrichment ON original.id = enrichment.idtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseJoin StrategyJoin StrategySQLdbf-default-precisionDefault Decimal PrecisionWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'precision' denoting number of available digits is required. Generally, precision is defined by column data type definition or database engines default. However undefined precision (0) can be returned from some database engines. 'Default Decimal Precision' is used when writing those undefined precision numbers.10falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseJoin StrategyJoin StrategySQLdbf-default-scaleDefault Decimal ScaleWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'scale' denoting number of available decimal digits is required. Generally, scale is defined by column data type definition or database engines default. However when undefined precision (0) is returned, scale can also be uncertain with some database engines. 'Default Decimal Scale' is used when writing those undefined numbers. If a value has more decimals than specified scale, then the value will be rounded-up, e.g. 1.53 becomes 2 with scale 0, and 1.5 with scale 1.0falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseJoin StrategyJoin StrategySQLInsertion Record PathInsertion Record PathSpecifies where in the 'original' Record the 'enrichment' Record's fields should be inserted. Note that if the RecordPath does not point to any existing field in the original Record, the enrichment will not be inserted./truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseJoin StrategyJoin StrategyInsert Enrichment FieldsMaximum number of BinsMaximum number of BinsSpecifies the maximum number of bins that can be held in memory at any one time10000truefalsefalseNONEfalsefalseTimeoutTimeoutSpecifies the maximum amount of time to wait for the second FlowFile once the first arrives at the processor, after which point the first FlowFile will be routed to the 'timeout' relationship.10 mintruefalsefalseNONEfalsefalsetimeoutIf one of the incoming FlowFiles (i.e., the 'original' FlowFile or the 'enrichment' FlowFile) arrives to this Processor but the other does not arrive within the configured Timeout period, the FlowFile that did arrive is routed to this relationship.falsejoinedThe resultant FlowFile with Records joined together from both the original and enrichment FlowFiles will be routed to this relationshipfalsefailureIf both the 'original' and 'enrichment' FlowFiles arrive at the processor but there was a failure in joining the records, both of those FlowFiles will be routed to this relationship.falseoriginalBoth of the incoming FlowFiles ('original' and 'enrichment') will be routed to this Relationship. I.e., this is the 'original' version of both of these FlowFiles.truemime.typeSets the mime.type attribute to the MIME Type specified by the Record Writerrecord.countThe number of records in the FlowFiletruetrueINPUT_REQUIREDMEMORYThis Processor will load into heap all FlowFiles that are on its incoming queues. While it loads the FlowFiles themselves, and not their content, the FlowFile attributes can be very memory intensive. Additionally, if the Join Strategy is set to SQL, the SQL engine may require buffering the entire contents of the enrichment FlowFile for each concurrent task. See Processor's Additional Details for more details and for steps on how to mitigate these concerns.org.apache.nifi.processors.standard.ForkEnrichmentorg.apache.nifi.processors.standard.ListDatabaseTablesPROCESSORGenerates a set of flow files, each containing attributes corresponding to metadata about a table from a database connection. Once metadata about a table has been fetched, it will not be fetched again until the Refresh Interval (if set) has elapsed, or until state has been manually cleared.sqllistjdbctabledatabaselist-db-tables-db-connectionDatabase Connection Pooling ServiceThe Controller Service that is used to obtain connection to databaseorg.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalselist-db-tables-catalogCatalogThe name of a catalog from which to list database tables. The name must match the catalog name as it is stored in the database. If the property is not set, the catalog name will not be used to narrow the search for tables. If the property is set to an empty string, tables without a catalog will be listed.falsefalsefalseNONEfalsefalselist-db-tables-schema-patternSchema PatternA pattern for matching schemas in the database. Within a pattern, "%" means match any substring of 0 or more characters, and "_" means match any one character. The pattern must match the schema name as it is stored in the database. If the property is not set, the schema name will not be used to narrow the search for tables. If the property is set to an empty string, tables without a schema will be listed.falsefalsefalseNONEfalsefalselist-db-tables-name-patternTable Name PatternA pattern for matching tables in the database. Within a pattern, "%" means match any substring of 0 or more characters, and "_" means match any one character. The pattern must match the table name as it is stored in the database. If the property is not set, all tables will be retrieved.falsefalsefalseNONEfalsefalselist-db-tables-typesTable TypesA comma-separated list of table types to include. For example, some databases support TABLE and VIEW types. If the property is not set, tables of all types will be returned.TABLEfalsefalsefalseNONEfalsefalselist-db-include-countInclude CountWhether to include the table's row count as a flow file attribute. This affects performance as a database query will be generated for each table in the retrieved list.falsetruetruefalsefalsetruefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalselist-db-refresh-intervalRefresh IntervalThe amount of time to elapse before resetting the processor state, thereby causing all current tables to be listed. During this interval, the processor may continue to run, but tables that have already been listed will not be re-listed. However new/added tables will be listed as the processor runs. A value of zero means the state will never be automatically reset, the user must Clear State manually.0 sectruefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalsedb.table.nameContains the name of a database table from the connectiondb.table.catalogContains the name of the catalog to which the table belongs (may be null)db.table.schemaContains the name of the schema to which the table belongs (may be null)db.table.fullnameContains the fully-qualifed table name (possibly including catalog, schema, etc.)db.table.typeContains the type of the database table from the connection. Typical types are "TABLE", "VIEW", "SYSTEM TABLE", "GLOBAL TEMPORARY", "LOCAL TEMPORARY", "ALIAS", "SYNONYM"db.table.remarksContains the name of a database table from the connectiondb.table.countContains the number of rows in the tabletruetrueAfter performing a listing of tables, the timestamp of the query is stored. This allows the Processor to not re-list tables the next time that the Processor is run. Specifying the refresh interval in the processor properties will indicate that when the processor detects the interval has elapsed, the state will be reset and tables will be re-listed as a result. This processor is meant to be run on the primary node only.CLUSTERINPUT_FORBIDDENPerform a full load of a database, retrieving all rows from all tables, or a specific set of tables.full loadrdbmsjdbcdatabaseorg.apache.nifi.processors.standard.ListDatabaseTablesConfigure the "Database Connection Pooling Service" property to specify a Connection Pool that is applicable for interacting with your database. Leave the RecordWriter property unset. Set the "Catalog" property to the name of the database Catalog; leave it empty to include all catalogs. Set the "Schema Pattern" property to a Java Regular Expression that matches all database Schemas that should be included; leave it empty to include all Schemas. Set the "Table Name Pattern" property to a Java Regular Expression that matches the names of all tables that should be included; leave it empty to include all Tables. Connect the "success" relationship to GenerateTableFetch. org.apache.nifi.processors.standard.GenerateTableFetchConfigure the "Database Connection Pooling Service" property to specify the same Connection Pool that was used in ListDatabaseTables. Set the "Database Type" property to match the appropriate value for your RDBMS vendor. Set "Table Name" to `${db.table.fullname}` Leave the RecordWriter property unset. Connect the "success" relationship to ExecuteSQLRecord. org.apache.nifi.processors.standard.ExecuteSQLRecordConfigure the "Database Connection Pooling Service" property to specify the same Connection Pool that was used in ListDatabaseTables. Configure the "Record Writer" property to specify a Record Writer that is appropriate for the desired output data type. Leave the "SQL select query" unset. Connect the "success" relationship to the next Processor in the flow. TIMER_DRIVEN1 min1org.apache.nifi.processors.standard.ListFTPPROCESSORPerforms a listing of the files residing on an FTP server. For each file that is found on the remote server, a new FlowFile will be created with the filename attribute set to the name of the file on the remote server. This can then be used in conjunction with FetchFTP in order to fetch those files.listftpremoteingestsourceinputfileslisting-strategyListing StrategySpecify how to determine new/updated entities. See each strategy descriptions for detail.timestampsTracking TimestampstimestampsThis strategy tracks the latest timestamp of listed entity to determine new/updated entities. Since it only tracks few timestamps, it can manage listing state efficiently. However, any newly added, or updated entity having timestamp older than the tracked latest timestamp can not be picked by this strategy. For example, such situation can happen in a file system if a file with old timestamp is copied or moved into the target directory without its last modified timestamp being updated. Also may miss files when multiple subdirectories are being written at the same time while listing is running.Tracking EntitiesentitiesThis strategy tracks information of all the listed entities within the latest 'Entity Tracking Time Window' to determine new/updated entities. This strategy can pick entities having old timestamp that can be missed with 'Tracking Timestamps'. Works even when multiple subdirectories are being written at the same time while listing is running. However additional DistributedMapCache controller service is required and more JVM heap memory is used. See the description of 'Entity Tracking Time Window' property for further details on how it works.No TrackingnoneThis strategy lists an entity without any tracking. The same entity will be listed each time on executing this processor. It is recommended to change the default run schedule value. Any property that related to the persisting state will be disregarded.Time Windowtime-windowThis strategy uses a sliding time window. The window starts where the previous window ended and ends with the 'current time'. One cycle will list files with modification time falling within the time window. Works even when multiple subdirectories are being written at the same time while listing is running. IMPORTANT: This strategy works properly only if the time on both the system hosting NiFi and the one hosting the files are accurate.truefalsefalseNONEfalsefalseHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueENVIRONMENTfalsefalsePortPortThe port to connect to on the remote host to fetch the data from21truefalsetrueENVIRONMENTfalsefalseUsernameUsernameUsernametruefalsetrueENVIRONMENTfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push files.falsefalsetrueENVIRONMENTfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseDistributed Cache ServiceDistributed Cache ServiceNOTE: This property is used merely for migration from old NiFi version before state management was introduced at version 0.5.0. The stored value in the cache service will be migrated into the state when this processor is started at the first time. The specified Controller Service was used to maintain state about what had been pulled from the remote server so that if a new node begins pulling data, it won't duplicate all of the work that has been done. If not specified, the information was not shared across the cluster. This property did not need to be set for standalone instances of NiFi but was supposed to be configured if NiFi had been running within a cluster.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseSearch RecursivelySearch RecursivelyIf true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefollow-symlinkFollow symlinkIf true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseFile Filter RegexFile Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetchedfalsefalsefalseNONEfalsefalsePath Filter RegexPath Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scannedfalsefalsefalseNONEfalsefalseIgnore Dotted FilesIgnore Dotted FilesIf true, files whose names begin with a dot (".") will be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseRemote Poll Batch SizeRemote Poll Batch SizeThe value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.5000truefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseConnection ModeConnection ModeThe FTP Connection ModePassiveActiveActivePassivePassivefalsefalsefalseNONEfalsefalseTransfer ModeTransfer ModeThe FTP Transfer ModeBinaryBinaryBinaryASCIIASCIIfalsefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseInternal Buffer SizeInternal Buffer SizeSet the internal buffer size for buffered data streams16KBfalsefalsefalseNONEfalsefalsetarget-system-timestamp-precisionTarget System Timestamp PrecisionSpecify timestamp precision at the target system. Since this processor uses timestamp of entities to decide which should be listed, it is crucial to use the right timestamp precision.auto-detectAuto Detectauto-detectAutomatically detect time unit deterministically based on candidate entries timestamp. Please note that this option may take longer to list entities unnecessarily, if none of entries has a precise precision timestamp. E.g. even if a target system supports millis, if all entries only have timestamps without millis, such as '2017-06-16 09:06:34.000', then its precision is determined as 'seconds'.MillisecondsmillisThis option provides the minimum latency for an entry from being available to being listed if target system supports millis, if not, use other options.SecondssecondsFor a target system that does not have millis precision, but has in seconds.MinutesminutesFor a target system that only supports precision in minutes.truefalsefalseNONEfalsefalseet-state-cacheEntity Tracking State CacheListed entities are stored in the specified cache storage so that this processor can resume listing across NiFi restart or in case of primary node change. 'Tracking Entities' strategy require tracking information of all listed entities within the last 'Tracking Time Window'. To support large number of entities, the strategy uses DistributedMapCache instead of managed state. Cache key format is 'ListedEntities::{processorId}(::{nodeId})'. If it tracks per node listed entities, then the optional '::{nodeId}' part is added to manage state separately. E.g. cluster wide cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b', per node cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b::nifi-node3' The stored cache content is Gzipped JSON string. The cache key will be deleted when target listing configuration is changed. Used by 'Tracking Entities' strategy.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseet-time-windowEntity Tracking Time WindowSpecify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile is emitted if one of following condition meets: 1. does not exist in the already-listed entities, 2. has newer timestamp than the cached entity, 3. has different size than the cached entity. If a cached entity's timestamp becomes older than specified time window, that entity will be removed from the cached already-listed entities. Used by 'Tracking Entities' strategy.3 hoursfalsefalsetrueENVIRONMENTfalsefalseet-initial-listing-targetEntity Tracking Initial Listing TargetSpecify how initial listing should be handled. Used by 'Tracking Entities' strategy.allTracking Time WindowwindowIgnore entities having timestamp older than the specified 'Tracking Time Window' at the initial listing activity.All AvailableallRegardless of entities timestamp, all existing entities will be listed at the initial listing activity.falsefalsefalseNONEfalsefalseftp-use-utf8Use UTF-8 EncodingTells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessAll FlowFiles that are received are routed to successfalseftp.remote.hostThe hostname of the FTP Serverftp.remote.portThe port that was connected to on the FTP Serverftp.listing.userThe username of the user that performed the FTP Listingfile.ownerThe numeric owner id of the source filefile.groupThe numeric group id of the source filefile.permissionsThe read/write/execute permissions of the source filefile.sizeThe number of bytes in the source filefile.lastModifiedTimeThe timestamp of when the file in the filesystem waslast modified as 'yyyy-MM-dd'T'HH:mm:ssZ'filenameThe name of the file on the FTP ServerpathThe fully qualified name of the directory on the FTP Server from which the file was pulledtruetrueAfter performing a listing of files, the timestamp of the newest file is stored. This allows the Processor to list only files that have been added or modified after this date the next time that the Processor is run. State is stored across the cluster so that this Processor can be run on Primary Node only and if a new Primary Node is selected, the new node will not duplicate the data that was listed by the previous Primary Node.CLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.standard.FetchFTPorg.apache.nifi.processors.standard.GetFTPorg.apache.nifi.processors.standard.PutFTPTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.ListFilePROCESSORRetrieves a listing of files from the input directory. For each file listed, creates a FlowFile that represents the file so that it can be fetched in conjunction with FetchFile. This Processor is designed to run on Primary Node only in a cluster when 'Input Directory Location' is set to 'Remote'. If the primary node changes, the new Primary Node will pick up where the previous node left off without duplicating all the data. When 'Input Directory Location' is 'Local', the 'Execution' mode can be anything, and synchronization won't happen. Unlike GetFile, this Processor does not delete any data from the local filesystem.filegetlistingestsourcefilesystemInput DirectoryInput DirectoryThe input directory from which files to pull filestruefalsetrueENVIRONMENTfalsefalselisting-strategyListing StrategySpecify how to determine new/updated entities. See each strategy descriptions for detail.timestampsTracking TimestampstimestampsThis strategy tracks the latest timestamp of listed entity to determine new/updated entities. Since it only tracks few timestamps, it can manage listing state efficiently. However, any newly added, or updated entity having timestamp older than the tracked latest timestamp can not be picked by this strategy. For example, such situation can happen in a file system if a file with old timestamp is copied or moved into the target directory without its last modified timestamp being updated. Also may miss files when multiple subdirectories are being written at the same time while listing is running.Tracking EntitiesentitiesThis strategy tracks information of all the listed entities within the latest 'Entity Tracking Time Window' to determine new/updated entities. This strategy can pick entities having old timestamp that can be missed with 'Tracking Timestamps'. Works even when multiple subdirectories are being written at the same time while listing is running. However additional DistributedMapCache controller service is required and more JVM heap memory is used. See the description of 'Entity Tracking Time Window' property for further details on how it works.No TrackingnoneThis strategy lists an entity without any tracking. The same entity will be listed each time on executing this processor. It is recommended to change the default run schedule value. Any property that related to the persisting state will be disregarded.truefalsefalseNONEfalsefalseRecurse SubdirectoriesRecurse SubdirectoriesIndicates whether to list files from subdirectories of the directorytruetruetruefalsefalsetruefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseInput Directory LocationInput Directory LocationSpecifies where the Input Directory is located. This is used to determine whether state should be stored locally or across the cluster.LocalLocalLocalInput Directory is located on a local disk. State will be stored locally on each node in the cluster.RemoteRemoteInput Directory is located on a remote system. State will be stored across the cluster so that the listing can be performed on Primary Node Only and another node can pick up where the last node left off, if the Primary Node changestruefalsefalseNONEfalsefalseFile FilterFile FilterOnly files whose names match the given regular expression will be picked up[^\.].*truefalsefalseNONEfalsefalsePath FilterPath FilterWhen Recurse Subdirectories is true, then only subdirectories whose path matches the given regular expression will be scannedfalsefalsefalseNONEfalsefalseInclude File AttributesInclude File AttributesWhether or not to include information such as the file's Last Modified Time and Owner as FlowFile Attributes. Depending on the File System being used, gathering this information can be expensive and as a result should be disabled. This is especially true of remote file shares.truetruetruefalsefalsetruefalsefalseNONEfalsefalseMinimum File AgeMinimum File AgeThe minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored0 sectruefalsefalseNONEfalsefalseMaximum File AgeMaximum File AgeThe maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignoredfalsefalsefalseNONEfalsefalseMinimum File SizeMinimum File SizeThe minimum size that a file must be in order to be pulled0 BtruefalsefalseNONEfalsefalseMaximum File SizeMaximum File SizeThe maximum size that a file can be in order to be pulledfalsefalsefalseNONEfalsefalseIgnore Hidden FilesIgnore Hidden FilesIndicates whether or not hidden files should be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalsetarget-system-timestamp-precisionTarget System Timestamp PrecisionSpecify timestamp precision at the target system. Since this processor uses timestamp of entities to decide which should be listed, it is crucial to use the right timestamp precision.auto-detectAuto Detectauto-detectAutomatically detect time unit deterministically based on candidate entries timestamp. Please note that this option may take longer to list entities unnecessarily, if none of entries has a precise precision timestamp. E.g. even if a target system supports millis, if all entries only have timestamps without millis, such as '2017-06-16 09:06:34.000', then its precision is determined as 'seconds'.MillisecondsmillisThis option provides the minimum latency for an entry from being available to being listed if target system supports millis, if not, use other options.SecondssecondsFor a target system that does not have millis precision, but has in seconds.MinutesminutesFor a target system that only supports precision in minutes.truefalsefalseNONEfalsefalseet-state-cacheEntity Tracking State CacheListed entities are stored in the specified cache storage so that this processor can resume listing across NiFi restart or in case of primary node change. 'Tracking Entities' strategy require tracking information of all listed entities within the last 'Tracking Time Window'. To support large number of entities, the strategy uses DistributedMapCache instead of managed state. Cache key format is 'ListedEntities::{processorId}(::{nodeId})'. If it tracks per node listed entities, then the optional '::{nodeId}' part is added to manage state separately. E.g. cluster wide cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b', per node cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b::nifi-node3' The stored cache content is Gzipped JSON string. The cache key will be deleted when target listing configuration is changed. Used by 'Tracking Entities' strategy.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseet-time-windowEntity Tracking Time WindowSpecify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile is emitted if one of following condition meets: 1. does not exist in the already-listed entities, 2. has newer timestamp than the cached entity, 3. has different size than the cached entity. If a cached entity's timestamp becomes older than specified time window, that entity will be removed from the cached already-listed entities. Used by 'Tracking Entities' strategy.3 hoursfalsefalsetrueENVIRONMENTfalsefalseet-initial-listing-targetEntity Tracking Initial Listing TargetSpecify how initial listing should be handled. Used by 'Tracking Entities' strategy.allTracking Time WindowwindowIgnore entities having timestamp older than the specified 'Tracking Time Window' at the initial listing activity.All AvailableallRegardless of entities timestamp, all existing entities will be listed at the initial listing activity.falsefalsefalseNONEfalsefalseet-node-identifierEntity Tracking Node IdentifierThe configured value will be appended to the cache key so that listing state can be tracked per NiFi node rather than cluster wide when tracking state is scoped to LOCAL. Used by 'Tracking Entities' strategy.${hostname()}falsefalsetrueENVIRONMENTfalsefalsetrack-performanceTrack PerformanceWhether or not the Processor should track the performance of disk access operations. If true, all accesses to disk will be recorded, including the file being accessed, the information being obtained, and how long it takes. This is then logged periodically at a DEBUG level. While the amount of data will be capped, this option may still consume a significant amount of heap (controlled by the 'Maximum Number of Files to Track' property), but it can be very useful for troubleshooting purposes if performance is poor is degraded.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsemax-performance-metricsMaximum Number of Files to TrackIf the 'Track Performance' property is set to 'true', this property indicates the maximum number of files whose performance metrics should be held onto. A smaller value for this property will result in less heap utilization, while a larger value may provide more accurate insights into how the disk access operations are performing100000truefalsetrueENVIRONMENTfalsefalsemax-operation-timeMax Disk Operation TimeThe maximum amount of time that any single disk operation is expected to take. If any disk operation takes longer than this amount of time, a warning bulletin will be generated for each operation that exceeds this amount of time.10 secsfalsefalsetrueENVIRONMENTfalsefalsemax-listing-timeMax Directory Listing TimeThe maximum amount of time that listing any single directory is expected to take. If the listing for the directory specified by the 'Input Directory' property, or the listing of any subdirectory (if 'Recurse' is set to true) takes longer than this amount of time, a warning bulletin will be generated for each directory listing that exceeds this amount of time.3 minsfalsefalsetrueENVIRONMENTfalsefalsesuccessAll FlowFiles that are received are routed to successfalsefilenameThe name of the file that was read from filesystem.pathThe path is set to the relative path of the file's directory on filesystem compared to the Input Directory property. For example, if Input Directory is set to /tmp, then files picked up from /tmp will have the path attribute set to "/". If the Recurse Subdirectories property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to "abc/1/2/3/".absolute.pathThe absolute.path is set to the absolute path of the file's directory on filesystem. For example, if the Input Directory property is set to /tmp, then files picked up from /tmp will have the path attribute set to "/tmp/". If the Recurse Subdirectories property is set to true and a file is picked up from /tmp/abc/1/2/3, then the path attribute will be set to "/tmp/abc/1/2/3/".file.ownerThe user that owns the file in filesystemfile.groupThe group that owns the file in filesystemfile.sizeThe number of bytes in the file in filesystemfile.permissionsThe permissions for the file in filesystem. This is formatted as 3 characters for the owner, 3 for the group, and 3 for other users. For example rw-rw-r--file.lastModifiedTimeThe timestamp of when the file in filesystem was last modified as 'yyyy-MM-dd'T'HH:mm:ssZ'file.lastAccessTimeThe timestamp of when the file in filesystem was last accessed as 'yyyy-MM-dd'T'HH:mm:ssZ'file.creationTimeThe timestamp of when the file in filesystem was created as 'yyyy-MM-dd'T'HH:mm:ssZ'trueAfter performing a listing of files, the timestamp of the newest file is stored. This allows the Processor to list only files that have been added or modified after this date the next time that the Processor is run. Whether the state is stored with a Local or Cluster scope depends on the value of the <Input Directory Location> property.LOCALCLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.standard.GetFileorg.apache.nifi.processors.standard.PutFileorg.apache.nifi.processors.standard.FetchFileTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.ListSFTPPROCESSORPerforms a listing of the files residing on an SFTP server. For each file that is found on the remote server, a new FlowFile will be created with the filename attribute set to the name of the file on the remote server. This can then be used in conjunction with FetchSFTP in order to fetch those files.listsftpremoteingestsourceinputfileslisting-strategyListing StrategySpecify how to determine new/updated entities. See each strategy descriptions for detail.timestampsTracking TimestampstimestampsThis strategy tracks the latest timestamp of listed entity to determine new/updated entities. Since it only tracks few timestamps, it can manage listing state efficiently. However, any newly added, or updated entity having timestamp older than the tracked latest timestamp can not be picked by this strategy. For example, such situation can happen in a file system if a file with old timestamp is copied or moved into the target directory without its last modified timestamp being updated. Also may miss files when multiple subdirectories are being written at the same time while listing is running.Tracking EntitiesentitiesThis strategy tracks information of all the listed entities within the latest 'Entity Tracking Time Window' to determine new/updated entities. This strategy can pick entities having old timestamp that can be missed with 'Tracking Timestamps'. Works even when multiple subdirectories are being written at the same time while listing is running. However additional DistributedMapCache controller service is required and more JVM heap memory is used. See the description of 'Entity Tracking Time Window' property for further details on how it works.No TrackingnoneThis strategy lists an entity without any tracking. The same entity will be listed each time on executing this processor. It is recommended to change the default run schedule value. Any property that related to the persisting state will be disregarded.Time Windowtime-windowThis strategy uses a sliding time window. The window starts where the previous window ended and ends with the 'current time'. One cycle will list files with modification time falling within the time window. Works even when multiple subdirectories are being written at the same time while listing is running. IMPORTANT: This strategy works properly only if the time on both the system hosting NiFi and the one hosting the files are accurate.truefalsefalseNONEfalsefalseHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers22truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalsePrivate Key PathPrivate Key PathThe fully qualified path to the Private Key filefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILEPrivate Key PassphrasePrivate Key PassphrasePassword for the private keyfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push files.falsefalsetrueENVIRONMENTfalsefalserecord-writerRecord WriterSpecifies the Record Writer to use for creating the listing. If not specified, one FlowFile will be created for each entity that is listed. If the Record Writer is specified, all entities will be written to a single FlowFile instead of adding attributes to individual FlowFiles.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseDistributed Cache ServiceDistributed Cache ServiceNOTE: This property is used merely for migration from old NiFi version before state management was introduced at version 0.5.0. The stored value in the cache service will be migrated into the state when this processor is started at the first time. The specified Controller Service was used to maintain state about what had been pulled from the remote server so that if a new node begins pulling data, it won't duplicate all of the work that has been done. If not specified, the information was not shared across the cluster. This property did not need to be set for standalone instances of NiFi but was supposed to be configured if NiFi had been running within a cluster.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseSearch RecursivelySearch RecursivelyIf true, will pull files from arbitrarily nested subdirectories; otherwise, will not traverse subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalsefollow-symlinkFollow symlinkIf true, will pull even symbolic files and also nested symbolic subdirectories; otherwise, will not read symbolic files and will not traverse symbolic link subdirectoriesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseFile Filter RegexFile Filter RegexProvides a Java Regular Expression for filtering Filenames; if a filter is supplied, only files whose names match that Regular Expression will be fetchedfalsefalsefalseNONEfalsefalsePath Filter RegexPath Filter RegexWhen Search Recursively is true, then only subdirectories whose path matches the given Regular Expression will be scannedfalsefalsefalseNONEfalsefalseIgnore Dotted FilesIgnore Dotted FilesIf true, files whose names begin with a dot (".") will be ignoredtruetruetruefalsefalsetruefalsefalseNONEfalsefalseRemote Poll Batch SizeRemote Poll Batch SizeThe value specifies how many file paths to find in a given directory on the remote system when doing a file listing. This value in general should not need to be modified but when polling against a remote system with a tremendous number of files this value can be critical. Setting this value too high can result very poor performance and setting it too low can cause the flow to be slower than normal.5000truefalsefalseNONEfalsefalseStrict Host Key CheckingStrict Host Key CheckingIndicates whether or not strict enforcement of hosts keys should be appliedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseHost Key FileHost Key FileIf supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be usedfalsefalsefalseNONEfalsefalseSINGLEFILEConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseSend Keep Alive On TimeoutSend Keep Alive On TimeoutSend a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.truetruetruefalsefalsetruefalsefalseNONEfalsefalsetarget-system-timestamp-precisionTarget System Timestamp PrecisionSpecify timestamp precision at the target system. Since this processor uses timestamp of entities to decide which should be listed, it is crucial to use the right timestamp precision.auto-detectAuto Detectauto-detectAutomatically detect time unit deterministically based on candidate entries timestamp. Please note that this option may take longer to list entities unnecessarily, if none of entries has a precise precision timestamp. E.g. even if a target system supports millis, if all entries only have timestamps without millis, such as '2017-06-16 09:06:34.000', then its precision is determined as 'seconds'.MillisecondsmillisThis option provides the minimum latency for an entry from being available to being listed if target system supports millis, if not, use other options.SecondssecondsFor a target system that does not have millis precision, but has in seconds.MinutesminutesFor a target system that only supports precision in minutes.truefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseet-state-cacheEntity Tracking State CacheListed entities are stored in the specified cache storage so that this processor can resume listing across NiFi restart or in case of primary node change. 'Tracking Entities' strategy require tracking information of all listed entities within the last 'Tracking Time Window'. To support large number of entities, the strategy uses DistributedMapCache instead of managed state. Cache key format is 'ListedEntities::{processorId}(::{nodeId})'. If it tracks per node listed entities, then the optional '::{nodeId}' part is added to manage state separately. E.g. cluster wide cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b', per node cache key = 'ListedEntities::8dda2321-0164-1000-50fa-3042fe7d6a7b::nifi-node3' The stored cache content is Gzipped JSON string. The cache key will be deleted when target listing configuration is changed. Used by 'Tracking Entities' strategy.org.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseet-time-windowEntity Tracking Time WindowSpecify how long this processor should track already-listed entities. 'Tracking Entities' strategy can pick any entity whose timestamp is inside the specified time window. For example, if set to '30 minutes', any entity having timestamp in recent 30 minutes will be the listing target when this processor runs. A listed entity is considered 'new/updated' and a FlowFile is emitted if one of following condition meets: 1. does not exist in the already-listed entities, 2. has newer timestamp than the cached entity, 3. has different size than the cached entity. If a cached entity's timestamp becomes older than specified time window, that entity will be removed from the cached already-listed entities. Used by 'Tracking Entities' strategy.3 hoursfalsefalsetrueENVIRONMENTfalsefalseet-initial-listing-targetEntity Tracking Initial Listing TargetSpecify how initial listing should be handled. Used by 'Tracking Entities' strategy.allTracking Time WindowwindowIgnore entities having timestamp older than the specified 'Tracking Time Window' at the initial listing activity.All AvailableallRegardless of entities timestamp, all existing entities will be listed at the initial listing activity.falsefalsefalseNONEfalsefalseMinimum File AgeMinimum File AgeThe minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored0 sectruefalsefalseNONEfalsefalseMaximum File AgeMaximum File AgeThe maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignoredfalsefalsefalseNONEfalsefalseMinimum File SizeMinimum File SizeThe minimum size that a file must be in order to be pulled0 BtruefalsefalseNONEfalsefalseMaximum File SizeMaximum File SizeThe maximum size that a file can be in order to be pulledfalsefalsefalseNONEfalsefalseCiphers AllowedCiphers AllowedA comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctrfalsefalsetrueENVIRONMENTfalsefalseKey Algorithms AllowedKey Algorithms AllowedA comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.comfalsefalsetrueENVIRONMENTfalsefalseKey Exchange Algorithms AllowedKey Exchange Algorithms AllowedA comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-cfalsefalsetrueENVIRONMENTfalsefalseMessage Authentication Codes AllowedMessage Authentication Codes AllowedA comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.comfalsefalsetrueENVIRONMENTfalsefalsesuccessAll FlowFiles that are received are routed to successfalsesftp.remote.hostThe hostname of the SFTP Serversftp.remote.portThe port that was connected to on the SFTP Serversftp.listing.userThe username of the user that performed the SFTP Listingfile.ownerThe numeric owner id of the source filefile.groupThe numeric group id of the source filefile.permissionsThe read/write/execute permissions of the source filefile.sizeThe number of bytes in the source filefile.lastModifiedTimeThe timestamp of when the file in the filesystem waslast modified as 'yyyy-MM-dd'T'HH:mm:ssZ'filenameThe name of the file on the SFTP ServerpathThe fully qualified name of the directory on the SFTP Server from which the file was pulledmime.typeThe MIME Type that is provided by the configured Record WritertruetrueAfter performing a listing of files, the timestamp of the newest file is stored. This allows the Processor to list only files that have been added or modified after this date the next time that the Processor is run. State is stored across the cluster so that this Processor can be run on Primary Node only and if a new Primary Node is selected, the new node will not duplicate the data that was listed by the previous Primary Node.CLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.standard.FetchSFTPorg.apache.nifi.processors.standard.GetSFTPorg.apache.nifi.processors.standard.PutSFTPTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.ListenFTPPROCESSORStarts an FTP server that listens on the specified port and transforms incoming files into FlowFiles. The URI of the service will be ftp://{hostname}:{port}. The default port is 2221.ingestFTPFTPSlistenbind-addressBind AddressThe address the FTP server should be bound to. If not set (or set to 0.0.0.0), the server binds to all available addresses (i.e. all network interfaces of the host machine).falsefalsetrueENVIRONMENTfalsefalselistening-portListening PortThe Port to listen on for incoming connections. On Linux, root privileges are required to use port numbers below 1024.2221truefalsetrueENVIRONMENTfalsefalseusernameUsernameThe name of the user that is allowed to log in to the FTP server. If a username is provided, a password must also be provided. If no username is specified, anonymous connections will be permitted.falsefalsetrueENVIRONMENTfalsefalsepasswordPasswordIf the Username is set, then a password must also be specified. The password provided by the client trying to log in to the FTP server will be checked against this password.falsetruetrueENVIRONMENTfalsefalsessl-context-serviceSSL Context ServiceSpecifies the SSL Context Service that can be used to create secure connections. If an SSL Context Service is selected, then a keystore file must also be specified in the SSL Context Service. Without a keystore file, the processor cannot be started successfully.Specifying a truststore file is optional. If a truststore file is specified, client authentication is required (the client needs to send a certificate to the server).Regardless of the selected TLS protocol, the highest available protocol is used for the connection. For example if NiFi is running on Java 11 and TLSv1.2 is selected in the controller service as the preferred TLS Protocol, TLSv1.3 will be used (regardless of TLSv1.2 being selected) because Java 11 supports TLSv1.3.org.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalsesuccessRelationship for successfully received files.falsefilenameThe name of the file received via the FTP/FTPS connection.pathThe path pointing to the file's target directory. E.g.: file.txt is uploaded to /Folder1/SubFolder, then the value of the path attribute will be "/Folder1/SubFolder/" (note that it ends with a separator character).INPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenHTTPPROCESSORStarts an HTTP Server and listens on a given base path to transform incoming requests into FlowFiles. The default URI of the Service will be http://{hostname}:{port}/contentListener. Only HEAD and POST requests are supported. GET, PUT, DELETE, OPTIONS and TRACE will result in an error and the HTTP response status code 405; CONNECT will also result in an error and the HTTP response status code 400. GET is supported on <service_URI>/healthcheck. If the service is available, it returns "200 OK" with the content "OK". The health check functionality can be configured to be accessible via a different port. For details see the documentation of the "Listening Port for health check requests" property.A Record Reader and Record Writer property can be enabled on the processor to process incoming requests as records. Record processing is not allowed for multipart requests and request in FlowFileV3 format (minifi).ingesthttphttpsrestlistenBase PathBase PathBase path for incoming connectionscontentListenertruefalsetrueENVIRONMENTfalsefalseListening PortListening PortThe Port to listen on for incoming connectionstruefalsetrueENVIRONMENTfalsefalsehealth-check-portListening Port for Health Check RequestsThe port to listen on for incoming health check requests. If set, it must be different from the Listening Port. Configure this port if the processor is set to use two-way SSL and a load balancer that does not support client authentication for health check requests is used. Only /<base_path>/healthcheck service is available via this port and only GET and HEAD requests are supported. If the processor is set not to use SSL, SSL will not be used on this port, either. If the processor is set to use one-way SSL, one-way SSL will be used on this port. If the processor is set to use two-way SSL, one-way SSL will be used on this port (client authentication not required).falsefalsetrueENVIRONMENTfalsefalseMax Data to Receive per SecondMax Data to Receive per SecondThe maximum amount of data to receive per second; this allows the bandwidth to be throttled to a specified data rate; if not specified, the data rate is not throttledfalsefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceSSL Context Service enables support for HTTPSorg.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseHTTP ProtocolsHTTP ProtocolsHTTP Protocols supported for Application Layer Protocol Negotiation with TLSHTTP_1_1http/1.1HTTP_1_1HTTP/1.1h2 http/1.1H2_HTTP_1_1HTTP/2 and HTTP/1.1 negotiated based on requested protocolsh2H2HTTP/2truefalsefalseNONEfalsefalseSSL Context ServiceSSL Context Serviceclient-authenticationClient AuthenticationClient Authentication policy for TLS connections. Required when SSL Context Service configured.AUTOAUTOAUTOInferred based on SSL Context Service properties. The presence of Trust Store properties implies REQUIRED, otherwise NONE is configured.WANTWANTRequests the client certificate on handshake and validates if present but does not require itREQUIREDREQUIREDRequests the client certificate on handshake and rejects the connection if it is not present and validNONENONEDoes not request the client certificate on handshakefalsefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceAuthorized DN PatternAuthorized Subject DN PatternA Regular Expression to apply against the Subject's Distinguished Name of incoming connections. If the Pattern does not match the Subject DN, the the processor will respond with a status of HTTP 403 Forbidden..*truefalsefalseNONEfalsefalseauthorized-issuer-dn-patternAuthorized Issuer DN PatternA Regular Expression to apply against the Issuer's Distinguished Name of incoming connections. If the Pattern does not match the Issuer DN, the processor will respond with a status of HTTP 403 Forbidden..*falsefalsefalseNONEfalsefalseMax Unconfirmed Flowfile TimeMax Unconfirmed Flowfile TimeThe maximum amount of time to wait for a FlowFile to be confirmed before it is removed from the cache60 secstruefalsefalseNONEfalsefalseHTTP Headers to receive as Attributes (Regex)HTTP Headers to receive as Attributes (Regex)Specifies the Regular Expression that determines the names of HTTP Headers that should be passed along as FlowFile attributesfalsefalsefalseNONEfalsefalseReturn CodeReturn CodeThe HTTP return code returned after every HTTP call200falsefalsefalseNONEfalsefalsemultipart-request-max-sizeMultipart Request Max SizeThe max size of the request. Only applies for requests with Content-Type: multipart/form-data, and is used to prevent denial of service type of attacks, to prevent filling up the heap or disk space1 MBtruefalsefalseNONEfalsefalsemultipart-read-buffer-sizeMultipart Read Buffer SizeThe threshold size, at which the contents of an incoming file would be written to disk. Only applies for requests with Content-Type: multipart/form-data. It is used to prevent denial of service type of attacks, to prevent filling up the heap or disk space.512 KBtruefalsefalseNONEfalsefalsemax-thread-pool-sizeMaximum Thread Pool SizeThe maximum number of threads to be used by the embedded Jetty server. The value can be set between 8 and 1000. The value of this property affects the performance of the flows and the operating system, therefore the default value should only be changed in justified cases. A value that is less than the default value may be suitable if only a small number of HTTP clients connect to the server. A greater value may be suitable if a large number of HTTP clients are expected to make requests to the server simultaneously.200truefalsefalseNONEfalsefalserecord-readerRecord ReaderThe Record Reader to use parsing the incoming FlowFile into Recordsorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalserecord-writerRecord WriterThe Record Writer to use for serializing Records after they have been transformedorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-readerRecord ReadersuccessRelationship for successfully received FlowFilesfalseINPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenRELPPROCESSORListens for RELP messages being sent to a given port over TCP. Each message will be acknowledged after successfully writing the message to a FlowFile. Each FlowFile will contain data portion of one or more RELP frames. In the case where the RELP frames contain syslog messages, the output of this processor can be sent to a ParseSyslog processor for further processing.listenrelptcplogsLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsePortPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalseReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor during these surges.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the received data.UTF-8truefalsefalseNONEfalsefalseMax Number of TCP ConnectionsMax Number of Worker ThreadsThe maximum number of worker threads available for servicing TCP connections.2truefalsefalseNONEfalsefalseMax Batch SizeMax Batch SizeThe maximum number of messages to add to a single FlowFile. If multiple messages are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages1truefalsefalseNONEfalsefalseMessage DelimiterBatching Message DelimiterSpecifies the delimiter to place between messages when multiple messages are bundled together (see <Max Batch Size> property).\ntruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection.org.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseClient AuthClient AuthThe client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.REQUIREDWANTWANTREQUIREDREQUIREDNONENONEfalsefalsefalseNONEfalsefalsesuccessMessages received successfully will be sent out this relationship.falserelp.commandThe command of the RELP frames.relp.senderThe sending host of the messages.relp.portThe sending port the messages were received over.relp.txnrThe transaction number of the message. Only included if <Batch Size> is 1.mime.typeThe mime.type of the content which is text/plainINPUT_FORBIDDENorg.apache.nifi.processors.standard.ParseSyslogorg.apache.nifi.processors.standard.ListenSyslogPROCESSORListens for Syslog messages being sent to a given port over TCP or UDP. Incoming messages are checked against regular expressions for RFC5424 and RFC3164 formatted messages. The format of each message is: (<PRIORITY>)(VERSION )(TIMESTAMP) (HOSTNAME) (BODY) where version is optional. The timestamp can be an RFC5424 timestamp with a format of "yyyy-MM-dd'T'HH:mm:ss.SZ" or "yyyy-MM-dd'T'HH:mm:ss.S+hh:mm", or it can be an RFC3164 timestamp with a format of "MMM d HH:mm:ss". If an incoming messages matches one of these patterns, the message will be parsed and the individual pieces will be placed in FlowFile attributes, with the original message in the content of the FlowFile. If an incoming message does not match one of these patterns it will not be parsed and the syslog.valid attribute will be set to false with the original message in the content of the FlowFile. Valid messages will be transferred on the success relationship, and invalid messages will be transferred on the invalid relationship.sysloglistenudptcplogsProtocolProtocolThe protocol for Syslog communication.UDPTCPTCPUDPUDPtruefalsefalseNONEfalsefalsePortPortThe port for Syslog communication. Note that Expression language is not evaluated per FlowFile.truefalsetrueENVIRONMENTfalsefalseLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsesocket-keep-aliveSocket Keep AliveWhether or not to have TCP socket keep alive turned on. Timing details depend on operating system properties.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseProtocolProtocolTCPSSL Context ServiceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, syslog messages will be received over a secure connection.org.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProtocolProtocolTCPClient AuthClient AuthThe client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.REQUIREDWANTWANTREQUIREDREQUIREDNONENONEfalsefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive Syslog messages. Adjust this value appropriately based on the expected size of the incoming Syslog messages. When UDP is selected each buffer will hold one Syslog message. When TCP is selected messages are read from an incoming connection until the buffer is full, or the connection is closed. 65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseProtocolProtocolTCPMax Number of TCP ConnectionsMax Number of TCP ConnectionsThe maximum number of concurrent connections to accept Syslog messages in TCP mode.2truefalsefalseNONEfalsefalseProtocolProtocolTCPMax Batch SizeMax Batch SizeThe maximum number of Syslog events to add to a single FlowFile. If multiple events are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages1truefalsefalseNONEfalsefalseMessage DelimiterMessage DelimiterSpecifies the delimiter to place between Syslog messages when multiple messages are bundled together (see <Max Batch Size> property).\ntruefalsefalseNONEfalsefalseParse MessagesParse MessagesIndicates if the processor should parse the Syslog messages. If set to false, each outgoing FlowFile will only contain the sender, protocol, and port, and no additional attributes.truetruetruefalsefalsetruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the Syslog messages. Note that Expression language is not evaluated per FlowFile.UTF-8truefalsetrueENVIRONMENTfalsefalsesuccessSyslog messages that match one of the expected formats will be sent out this relationship as a FlowFile per message.falseinvalidSyslog messages that do not match one of the expected formats will be sent out this relationship as a FlowFile per message.falsesyslog.priorityThe priority of the Syslog message.syslog.severityThe severity of the Syslog message derived from the priority.syslog.facilityThe facility of the Syslog message derived from the priority.syslog.versionThe optional version from the Syslog message.syslog.timestampThe timestamp of the Syslog message.syslog.hostnameThe hostname or IP address of the Syslog message.syslog.senderThe hostname of the Syslog server that sent the message.syslog.bodyThe body of the Syslog message, everything after the hostname.syslog.validAn indicator of whether this message matched the expected formats. If this value is false, the other attributes will be empty and only the original message will be available in the content.syslog.protocolThe protocol over which the Syslog message was received.syslog.portThe port over which the Syslog message was received.mime.typeThe mime.type of the FlowFile which will be text/plain for Syslog messages.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.PutSyslogorg.apache.nifi.processors.standard.ParseSyslogorg.apache.nifi.processors.standard.ListenTCPPROCESSORListens for incoming TCP connections and reads data from each connection using a line separator as the message demarcator. The default behavior is for each message to produce a single FlowFile, however this can be controlled by increasing the Batch Size to a larger value for higher throughput. The Receive Buffer Size must be set as large as the largest messages expected to be received, meaning if every 100kb there is a line separator, then the Receive Buffer Size must be greater than 100kb. The processor can be configured to use an SSL Context Service to only allow secure connections. When connected clients present certificates for mutual TLS authentication, the Distinguished Names of the client certificate's issuer and subject are added to the outgoing FlowFiles as attributes. The processor does not perform authorization based on Distinguished Name values, but since these values are attached to the outgoing FlowFiles, authorization can be implemented based on these attributes.listentcptlssslLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsePortPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalseReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor during these surges.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the received data.UTF-8truefalsefalseNONEfalsefalseMax Number of TCP ConnectionsMax Number of Worker ThreadsThe maximum number of worker threads available for servicing TCP connections.2truefalsefalseNONEfalsefalseMax Batch SizeMax Batch SizeThe maximum number of messages to add to a single FlowFile. If multiple messages are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages1truefalsefalseNONEfalsefalseMessage DelimiterBatching Message DelimiterSpecifies the delimiter to place between messages when multiple messages are bundled together (see <Max Batch Size> property).\ntruefalsefalseNONEfalsefalseidle-timeoutIdle Connection TimeoutThe amount of time a client's connection will remain open if no data is received. The default of 0 seconds will leave connections open until they are closed by the client.0 secondstruefalsefalseNONEfalsefalsemax-receiving-threadsMax Number of Receiving Message Handler ThreadsThis property is deprecated and no longer used.falsefalsefalseNONEfalsefalsepool-receive-buffersPool Receive BuffersEnable or disable pooling of buffers that the processor uses for handling bytes received on socket connections. The framework allocates buffers as needed during processing.TrueTrueTrueFalseFalsetruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection.org.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseClient AuthClient AuthThe client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.REQUIREDWANTWANTREQUIREDREQUIREDNONENONEfalsefalsefalseNONEfalsefalsesuccessMessages received successfully will be sent out this relationship.falsetcp.senderThe sending host of the messages.tcp.portThe sending port the messages were received.client.certificate.issuer.dnFor connections using mutual TLS, the Distinguished Name of the Certificate Authority that issued the client's certificate is attached to the FlowFile.client.certificate.subject.dnFor connections using mutual TLS, the Distinguished Name of the client certificate's owner (subject) is attached to the FlowFile.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenTCPRecordPROCESSORListens for incoming TCP connections and reads data from each connection using a configured record reader, and writes the records to a flow file using a configured record writer. The type of record reader selected will determine how clients are expected to send data. For example, when using a Grok reader to read logs, a client can keep an open connection and continuously stream data, but when using an JSON reader, the client cannot send an array of JSON documents and then send another array on the same connection, as the reader would be in a bad state at that point. Records will be read from the connection in blocking mode, and will timeout according to the Read Timeout specified in the processor. If the read times out, or if any other error is encountered when reading, the connection will be closed, and any records read up to that point will be handled according to the configured Read Error Strategy (Discard or Transfer). In cases where clients are keeping a connection open, the concurrent tasks for the processor should be adjusted to match the Max Number of TCP Connections allowed, so that there is a task processing each connection. The processor can be configured to use an SSL Context Service to only allow secure connections. When connected clients present certificates for mutual TLS authentication, the Distinguished Names of the client certificate's issuer and subject are added to the outgoing FlowFiles as attributes. The processor does not perform authorization based on Distinguished Name values, but since these values are attached to the outgoing FlowFiles, authorization can be implemented based on these attributes.listentcprecordtlssslLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalseportPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalsemax-size-socket-bufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalsemax-number-tcp-connectionsMax Number of TCP ConnectionsThe maximum number of concurrent TCP connections to accept. In cases where clients are keeping a connection open, the concurrent tasks for the processor should be adjusted to match the Max Number of TCP Connections allowed, so that there is a task processing each connection.2truefalsefalseNONEfalsefalseread-timeoutRead TimeoutThe amount of time to wait before timing out when reading from a connection.10 secondstruefalsefalseNONEfalsefalserecord-readerRecord ReaderThe Record Reader to use for incoming FlowFilesorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterThe Record Writer to use in order to serialize the data before writing to a FlowFileorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsereader-error-handling-strategyRead Error StrategyIndicates how to deal with an error while reading the next record from a connection, when previous records have already been read from the connection.TransferTransferTransferTransfers any records already received and closes the connection.DiscardDiscardDiscards any records already received and closes the connection.truefalsefalseNONEfalsefalserecord-batch-sizeRecord Batch SizeThe maximum number of records to write to a single FlowFile.1000truefalsefalseNONEfalsefalsessl-context-serviceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection.org.apache.nifi.ssl.RestrictedSSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseclient-authClient AuthThe client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.REQUIREDWANTWANTREQUIREDREQUIREDNONENONEfalsefalsefalseNONEfalsefalsesuccessMessages received successfully will be sent out this relationship.falsetcp.senderThe host that sent the data.tcp.portThe port that the processor accepted the connection on.record.countThe number of records written to the flow file.mime.typeThe mime-type of the writer used to write the records to the flow file.client.certificate.issuer.dnFor connections using mutual TLS, the Distinguished Name of the Certificate Authority that issued the client's certificate is attached to the FlowFile.client.certificate.subject.dnFor connections using mutual TLS, the Distinguished Name of the client certificate's owner (subject) is attached to the FlowFile.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenUDPPROCESSORListens for Datagram Packets on a given port. The default behavior produces a FlowFile per datagram, however for higher throughput the Max Batch Size property may be increased to specify the number of datagrams to batch together in a single FlowFile. This processor can be restricted to listening for datagrams from a specific remote host and port by specifying the Sending Host and Sending Host Port properties, otherwise it will listen for datagrams from all hosts and ports.ingestudplistensourceLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsePortPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalseReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the received data.UTF-8truefalsefalseNONEfalsefalseMax Batch SizeMax Batch SizeThe maximum number of messages to add to a single FlowFile. If multiple messages are available, they will be concatenated along with the <Message Delimiter> up to this configured maximum number of messages1truefalsefalseNONEfalsefalseMessage DelimiterBatching Message DelimiterSpecifies the delimiter to place between messages when multiple messages are bundled together (see <Max Batch Size> property).\ntruefalsefalseNONEfalsefalseSending HostSending HostIP, or name, of a remote host. Only Datagrams from the specified Sending Host Port and this host will be accepted. Improves Performance. May be a system property or an environment variable.falsefalsetrueENVIRONMENTfalsefalseSending Host PortSending Host PortPort being used by remote host to send Datagrams. Only Datagrams from the specified Sending Host and this port will be accepted. Improves Performance. May be a system property or an environment variable.falsefalsetrueENVIRONMENTfalsefalsesuccessMessages received successfully will be sent out this relationship.falseudp.senderThe sending host of the messages.udp.portThe sending port the messages were received.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.ListenUDPRecordPROCESSORListens for Datagram Packets on a given port and reads the content of each datagram using the configured Record Reader. Each record will then be written to a flow file using the configured Record Writer. This processor can be restricted to listening for datagrams from a specific remote host and port by specifying the Sending Host and Sending Host Port properties, otherwise it will listen for datagrams from all hosts and ports.ingestudplistensourcerecordLocal Network InterfaceLocal Network InterfaceThe name of a local network interface to be used to restrict listening to a specific LAN.falsefalsetrueENVIRONMENTfalsefalsePortPortThe port to listen on for communication.truefalsetrueENVIRONMENTfalsefalseReceive Buffer SizeReceive Buffer SizeThe size of each buffer used to receive messages. Adjust this value appropriately based on the expected size of the incoming messages.65507 BtruefalsefalseNONEfalsefalseMax Size of Message QueueMax Size of Message QueueThe maximum size of the internal queue used to buffer messages being transferred from the underlying channel to the processor. Setting this value higher allows more messages to be buffered in memory during surges of incoming messages, but increases the total memory used by the processor.10000truefalsefalseNONEfalsefalseMax Size of Socket BufferMax Size of Socket BufferThe maximum size of the socket buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseCharacter SetCharacter SetSpecifies the character set of the received data.UTF-8truefalsefalseNONEfalsefalsepoll-timeoutPoll TimeoutThe amount of time to wait when polling the internal queue for more datagrams. If no datagrams are found after waiting for the configured timeout, then the processor will emit whatever records have been obtained up to that point.50 mstruefalsefalseNONEfalsefalsebatch-sizeBatch SizeThe maximum number of datagrams to write as records to a single FlowFile. The Batch Size will only be reached when data is coming in more frequently than the Poll Timeout.1000truefalsefalseNONEfalsefalserecord-readerRecord ReaderThe Record Reader to use for reading the content of incoming datagrams.org.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterThe Record Writer to use in order to serialize the data before writing to a flow file.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsesending-hostSending HostIP, or name, of a remote host. Only Datagrams from the specified Sending Host Port and this host will be accepted. Improves Performance. May be a system property or an environment variable.falsefalsetrueENVIRONMENTfalsefalsesending-host-portSending Host PortPort being used by remote host to send Datagrams. Only Datagrams from the specified Sending Host and this port will be accepted. Improves Performance. May be a system property or an environment variable.falsefalsetrueENVIRONMENTfalsefalsesuccessMessages received successfully will be sent out this relationship.falseparse.failureIf a datagram cannot be parsed using the configured Record Reader, the contents of the message will be routed to this Relationship as its own individual FlowFile.falseudp.senderThe sending host of the messages.udp.portThe sending port the messages were received.record.countThe number of records written to the flow file.mime.typeThe mime-type of the writer used to write the records to the flow file.trueINPUT_FORBIDDENorg.apache.nifi.processors.standard.LogAttributePROCESSOREmits attributes of the FlowFile at the specified log levelattributesloggingLog LevelLog LevelThe Log Level to use when logging the AttributesinfotracetracedebugdebuginfoinfowarnwarnerrorerrortruefalsefalseNONEfalsefalseLog PayloadLog PayloadIf true, the FlowFile's payload will be logged, in addition to its attributes; otherwise, just the Attributes will be logged.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseAttributes to LogAttributes to LogA comma-separated list of Attributes to Log. If not specified, all attributes will be logged unless `Attributes to Log by Regular Expression` is modified. There's an AND relationship between the two properties.falsefalsefalseNONEfalsefalseattributes-to-log-regexAttributes to Log by Regular ExpressionA regular expression indicating the Attributes to Log. If not specified, all attributes will be logged unless `Attributes to Log` is modified. There's an AND relationship between the two properties..*falsefalsefalseNONEfalsefalseAttributes to IgnoreAttributes to IgnoreA comma-separated list of Attributes to ignore. If not specified, no attributes will be ignored unless `Attributes to Ignore by Regular Expression` is modified. There's an OR relationship between the two properties.falsefalsefalseNONEfalsefalseattributes-to-ignore-regexAttributes to Ignore by Regular ExpressionA regular expression indicating the Attributes to Ignore. If not specified, no attributes will be ignored unless `Attributes to Ignore` is modified. There's an OR relationship between the two properties.falsefalsefalseNONEfalsefalseLog FlowFile PropertiesLog FlowFile PropertiesSpecifies whether or not to log FlowFile "properties", such as Entry Date, Lineage Start Date, and content sizetruetruetruefalsefalsetruefalsefalseNONEfalsefalseOutput FormatOutput FormatSpecifies the format to use for logging FlowFile attributesLine per AttributeLine per AttributeLine per AttributeEach FlowFile attribute will be logged using a single line for the attribute name and another line for the attribute value. This format is often most advantageous when looking at the attributes of a single FlowFile.Single LineSingle LineAll FlowFile attribute names and values will be logged on a single line. This format is often most advantageous when comparing logs from multiple FlowFiles.truefalsefalseNONEfalsefalseLog prefixLog prefixLog prefix appended to the log lines. It helps to distinguish the output of multiple LogAttribute processors.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsecharacter-setCharacter SetThe name of the CharacterSet to useUTF-8truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessAll FlowFiles are routed to this relationshipfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.LogMessagePROCESSOREmits a log message at the specified log levelattributeslogginglog-levelLog LevelThe Log Level to use when logging the message: [trace, debug, info, warn, error]infotruefalsetrueFLOWFILE_ATTRIBUTESfalsefalselog-prefixLog prefixLog prefix appended to the log lines. It helps to distinguish the output of multiple LogMessage processors.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalselog-messageLog messageThe log message to emitfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessAll FlowFiles are routed to this relationshipfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.LookupAttributePROCESSORLookup attributes from a lookup servicelookupcacheenrichjoinattributesAttribute Expression Languagelookup-serviceLookup ServiceThe lookup service to use for attribute lookupsorg.apache.nifi.lookup.StringLookupServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseinclude-empty-valuesInclude Empty ValuesInclude null or blank values for keys that are null or blanktruetruetruefalsefalsetruefalsefalseNONEfalsefalseThe name of the attribute to add to the FlowFileThe name of the key or property to retrieve from the lookup serviceAdds a FlowFile attribute specified by the dynamic property's key with the value found in the lookup service using the the dynamic property's valueFLOWFILE_ATTRIBUTESfailureFlowFiles with failing lookups are routed to this relationshipfalseunmatchedFlowFiles with missing lookups are routed to this relationshipfalsematchedFlowFiles with matching lookups are routed to this relationshipfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.LookupRecordPROCESSORExtracts one or more fields from a Record and looks up a value for those fields in a LookupService. If a result is returned by the LookupService, that result is optionally added to the Record. In this case, the processor functions as an Enrichment processor. Regardless, the Record is then routed to either the 'matched' relationship or 'unmatched' relationship (if the 'Routing Strategy' property is configured to do so), indicating whether or not a result was returned by the LookupService, allowing the processor to also function as a Routing processor. The "coordinates" to use for looking up a value in the Lookup Service are defined by adding a user-defined property. Each property that is added will have an entry added to a Map, where the name of the property becomes the Map Key and the value returned by the RecordPath becomes the value for that key. If multiple values are returned by the RecordPath, then the Record will be routed to the 'unmatched' relationship (or 'success', depending on the 'Routing Strategy' property's configuration). If one or more fields match the Result RecordPath, all fields that match will be updated. If there is no match in the configured LookupService, then no fields will be updated. I.e., it will not overwrite an existing value in the Record with a null value. Please note, however, that if the results returned by the LookupService are not accounted for in your schema (specifically, the schema that is configured for your Record Writer) then the fields will not be written out to the FlowFile.lookupenrichmentrouterecordcsvjsonavrodatabasedblogsconvertfilterrecord-readerRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalselookup-serviceLookup ServiceThe Lookup Service to use in order to lookup a value in each Recordorg.apache.nifi.lookup.LookupServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseRoot Record PathRoot Record PathA RecordPath that points to a child Record within each of the top-level Records in the FlowFile. If specified, the additional RecordPath properties will be evaluated against this child Record instead of the top-level Record. This allows for performing enrichment against multiple child Records within a single top-level Record.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalserouting-strategyRouting StrategySpecifies how to route records after a Lookup has completedroute-to-successRoute to 'success'route-to-successRecords will be routed to a 'success' Relationship regardless of whether or not there is a match in the configured Lookup ServiceRoute to 'matched' or 'unmatched'route-to-matched-unmatchedRecords will be routed to either a 'matched' or an 'unmatched' Relationship depending on whether or not there was a match in the configured Lookup Service. A single input FlowFile may result in two different output FlowFiles. If the given Record Paths evaluate such that multiple sub-records are evaluated, the parent Record will be routed to 'unmatched' unless all sub-records match.truefalsefalseNONEfalsefalseresult-contentsRecord Result ContentsWhen a result is obtained that contains a Record, this property determines whether the Record itself is inserted at the configured path or if the contents of the Record (i.e., the sub-fields) will be inserted at the configured path.insert-entire-recordInsert Entire Recordinsert-entire-recordThe entire Record that is retrieved from the Lookup Service will be inserted into the destination path.Insert Record Fieldsrecord-fieldsAll of the fields in the Record that is retrieved from the Lookup Service will be inserted into the destination path.truefalsefalseNONEfalsefalserecord-update-strategyRecord Update StrategyThis property defines the strategy to use when updating the record with the value returned by the Lookup Service.use-propertyReplace Existing Valuesreplace-existing-valuesThe "Result RecordPath" property will be ignored and the lookup service must be a single simple key lookup service. Every dynamic property value should be a record path. For each dynamic property, the value contained in the field corresponding to the record path will be used as the key in the Lookup Service and the value returned by the Lookup Service will be used to replace the existing value. It is possible to configure multiple dynamic properties to replace multiple values in one execution. This strategy only supports simple types replacements (strings, integers, etc).Use "Result RecordPath" Propertyuse-propertyThe "Result RecordPath" property will be used to determine which part of the record should be updated with the value returned by the Lookup ServicetruefalsefalseNONEfalsefalseresult-record-pathResult RecordPathA RecordPath that points to the field whose value should be updated with whatever value is returned from the Lookup Service. If not specified, the value that is returned from the Lookup Service will be ignored, except for determining whether the FlowFile should be routed to the 'matched' or 'unmatched' Relationship.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalserecord-update-strategyRecord Update Strategyuse-propertyrecord-path-lookup-miss-result-cache-sizeCache SizeSpecifies how many lookup values/records should be cached.Setting this property to zero means no caching will be done and the table will be queried for each lookup value in each record. If the lookup table changes often or the most recent data must be retrieved, do not use the cache.0truefalsetrueENVIRONMENTfalsefalseValue To LookupValid Record PathA RecordPath that points to the field whose value will be looked up in the configured Lookup ServiceFLOWFILE_ATTRIBUTESsuccessAll records will be sent to this Relationship if configured to do so, unless a failure occursfalsefailureIf a FlowFile cannot be enriched, the unchanged FlowFile will be routed to this relationshipfalsemime.typeSets the mime.type attribute to the MIME Type specified by the Record Writerrecord.countThe number of records in the FlowFiletruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ConvertRecordorg.apache.nifi.processors.standard.SplitRecordorg.apache.nifi.lookup.SimpleKeyValueLookupServiceorg.apache.nifi.lookup.maxmind.IPLookupServiceorg.apache.nifi.lookup.db.DatabaseRecordLookupServiceorg.apache.nifi.processors.standard.MergeContentPROCESSORMerges a Group of FlowFiles together based on a user-defined strategy and packages them into a single FlowFile. It is recommended that the Processor be configured with only a single incoming connection, as Group of FlowFiles will not be created from FlowFiles in different connections. This processor updates the mime.type attribute as appropriate. NOTE: this processor should NOT be configured with Cron Driven for the Scheduling Strategy.mergecontentcorrelationtarzipstreamconcatenationarchiveflowfile-streamflowfile-stream-v3Merge StrategyMerge StrategySpecifies the algorithm used to merge content. The 'Defragment' algorithm combines fragments that are associated by attributes back into a single cohesive FlowFile. The 'Bin-Packing Algorithm' generates a FlowFile populated by arbitrarily chosen FlowFilesBin-Packing AlgorithmBin-Packing AlgorithmBin-Packing AlgorithmGenerates 'bins' of FlowFiles and fills each bin as full as possible. FlowFiles are placed into a bin based on their size and optionally their attributes (if the <Correlation Attribute> property is set)DefragmentDefragmentCombines fragments that are associated by attributes back into a single cohesive FlowFile. If using this strategy, all FlowFiles must have the attributes <fragment.identifier>, <fragment.count>, and <fragment.index>. All FlowFiles with the same value for "fragment.identifier" will be grouped together. All FlowFiles in this group must have the same value for the "fragment.count" attribute. All FlowFiles in this group must have a unique value for the "fragment.index" attribute between 0 and the value of the "fragment.count" attribute.truefalsefalseNONEfalsefalseMerge FormatMerge FormatDetermines the format that will be used to merge the content.Binary ConcatenationTARTARA bin of FlowFiles will be combined into a single TAR file. The FlowFiles' <path> attribute will be used to create a directory in the TAR file if the <Keep Paths> property is set to true; otherwise, all FlowFiles will be added at the root of the TAR file. If a FlowFile has an attribute named <tar.permissions> that is 3 characters, each between 0-7, that attribute will be used as the TAR entry's 'mode'.ZIPZIPA bin of FlowFiles will be combined into a single ZIP file. The FlowFiles' <path> attribute will be used to create a directory in the ZIP file if the <Keep Paths> property is set to true; otherwise, all FlowFiles will be added at the root of the ZIP file. The <Compression Level> property indicates the ZIP compression to use.FlowFile Stream, v3FlowFile Stream, v3A bin of FlowFiles will be combined into a single Version 3 FlowFile StreamFlowFile Stream, v2FlowFile Stream, v2A bin of FlowFiles will be combined into a single Version 2 FlowFile StreamFlowFile Tar, v1FlowFile Tar, v1A bin of FlowFiles will be combined into a single Version 1 FlowFile PackageBinary ConcatenationBinary ConcatenationThe contents of all FlowFiles will be concatenated together into a single FlowFileAvroAvroThe Avro contents of all FlowFiles will be concatenated together into a single FlowFiletruefalsefalseNONEfalsefalseAttribute StrategyAttribute StrategyDetermines which FlowFile attributes should be added to the bundle. If 'Keep All Unique Attributes' is selected, any attribute on any FlowFile that gets bundled will be kept unless its value conflicts with the value from another FlowFile. If 'Keep Only Common Attributes' is selected, only the attributes that exist on all FlowFiles in the bundle, with the same value, will be preserved.Keep Only Common AttributesKeep Only Common AttributesKeep Only Common AttributesAny attribute that is not the same on all FlowFiles in a bin will be dropped. Those that are the same across all FlowFiles will be retained.Keep All Unique AttributesKeep All Unique AttributesAny attribute that has the same value for all FlowFiles in a bin, or has no value for a FlowFile, will be kept. For example, if a bin consists of 3 FlowFiles and 2 of them have a value of 'hello' for the 'greeting' attribute and the third FlowFile has no 'greeting' attribute then the outbound FlowFile will get a 'greeting' attribute with the value 'hello'.truefalsefalseNONEfalsefalseCorrelation Attribute NameCorrelation Attribute NameIf specified, like FlowFiles will be binned together, where 'like FlowFiles' means FlowFiles that have the same value for this Attribute. If not specified, FlowFiles are bundled by the order in which they are pulled from the queue.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMerge StrategyMerge StrategyBin-Packing Algorithmmergecontent-metadata-strategyMetadata StrategyFor FlowFiles whose input format supports metadata (Avro, e.g.), this property determines which metadata should be added to the bundle. If 'Use First Metadata' is selected, the metadata keys/values from the first FlowFile to be bundled will be used. If 'Keep Only Common Metadata' is selected, only the metadata that exists on all FlowFiles in the bundle, with the same value, will be preserved. If 'Ignore Metadata' is selected, no metadata is transferred to the outgoing bundled FlowFile. If 'Do Not Merge Uncommon Metadata' is selected, any FlowFile whose metadata values do not match those of the first bundled FlowFile will not be merged.Do Not Merge Uncommon MetadataUse First MetadataUse First MetadataFor any input format that supports metadata (Avro, e.g.), the metadata for the first FlowFile in the bin will be set on the output FlowFile.Keep Only Common MetadataKeep Only Common MetadataFor any input format that supports metadata (Avro, e.g.), any FlowFile whose metadata values match those of the first FlowFile, any additional metadata will be dropped but the FlowFile will be merged. Any FlowFile whose metadata values do not match those of the first FlowFile in the bin will not be merged.Do Not Merge Uncommon MetadataDo Not Merge Uncommon MetadataFor any input format that supports metadata (Avro, e.g.), any FlowFile whose metadata values do not match those of the first FlowFile in the bin will not be merged.Ignore MetadataIgnore MetadataIgnores (does not transfer, compare, etc.) any metadata from a FlowFile whose content supports embedded metadata.truefalsefalseNONEfalsefalseMerge FormatMerge FormatAvroMinimum Number of EntriesMinimum Number of EntriesThe minimum number of files to include in a bundle1truefalsefalseNONEfalsefalseMerge StrategyMerge StrategyBin-Packing AlgorithmMaximum Number of EntriesMaximum Number of EntriesThe maximum number of files to include in a bundle1000truefalsefalseNONEfalsefalseMerge StrategyMerge StrategyBin-Packing AlgorithmMinimum Group SizeMinimum Group SizeThe minimum size for the bundle0 BtruefalsefalseNONEfalsefalseMerge StrategyMerge StrategyBin-Packing AlgorithmMaximum Group SizeMaximum Group SizeThe maximum size for the bundle. If not specified, there is no maximum.falsefalsefalseNONEfalsefalseMerge StrategyMerge StrategyBin-Packing AlgorithmMax Bin AgeMax Bin AgeThe maximum age of a Bin that will trigger a Bin to be complete. Expected format is <duration> <time unit> where <duration> is a positive integer and time unit is one of seconds, minutes, hoursfalsefalsefalseNONEfalsefalseMaximum number of BinsMaximum number of BinsSpecifies the maximum number of bins that can be held in memory at any one time5truefalsefalseNONEfalsefalseDelimiter StrategyDelimiter StrategyDetermines if Header, Footer, and Demarcator should point to files containing the respective content, or if the values of the properties should be used as the content.Do Not Use DelimitersDo Not Use DelimitersDo Not Use DelimitersNo Header, Footer, or Demarcator will be usedFilenameFilenameThe values of Header, Footer, and Demarcator will be retrieved from the contents of a fileTextTextThe values of Header, Footer, and Demarcator will be specified as property valuestruefalsefalseNONEfalsefalseMerge FormatMerge FormatBinary ConcatenationHeader FileHeaderFilename or text specifying the header to use. If not specified, no header is supplied.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILETEXTMerge FormatMerge FormatBinary ConcatenationDelimiter StrategyDelimiter StrategyFilenameTextFooter FileFooterFilename or text specifying the footer to use. If not specified, no footer is supplied.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILETEXTMerge FormatMerge FormatBinary ConcatenationDelimiter StrategyDelimiter StrategyFilenameTextDemarcator FileDemarcatorFilename or text specifying the demarcator to use. If not specified, no demarcator is supplied.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILETEXTMerge FormatMerge FormatBinary ConcatenationDelimiter StrategyDelimiter StrategyFilenameTextCompression LevelCompression LevelSpecifies the compression level to use when using the Zip Merge Format; if not using the Zip Merge Format, this value is ignored100112233445566778899truefalsefalseNONEfalsefalseMerge FormatMerge FormatZIPKeep PathKeep PathIf using the Zip or Tar Merge Format, specifies whether or not the FlowFiles' paths should be included in their entry names.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseMerge FormatMerge FormatZIPTARTar Modified TimeTar Modified TimeIf using the Tar Merge Format, specifies if the Tar entry should store the modified timestamp either by expression (e.g. ${file.lastModifiedTime} or static value, both of which must match the ISO8601 format 'yyyy-MM-dd'T'HH:mm:ssZ'.${file.lastModifiedTime}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMerge FormatMerge FormatTARfailureIf the bundle cannot be created, all FlowFiles that would have been used to created the bundle will be transferred to failurefalseoriginalThe FlowFiles that were used to create the bundlefalsemergedThe FlowFile containing the merged contentfalsefragment.identifierApplicable only if the <Merge Strategy> property is set to Defragment. All FlowFiles with the same value for this attribute will be bundled together.fragment.indexApplicable only if the <Merge Strategy> property is set to Defragment. This attribute indicates the order in which the fragments should be assembled. This attribute must be present on all FlowFiles when using the Defragment Merge Strategy and must be a unique (i.e., unique across all FlowFiles that have the same value for the "fragment.identifier" attribute) integer between 0 and the value of the fragment.count attribute. If two or more FlowFiles have the same value for the "fragment.identifier" attribute and the same value for the "fragment.index" attribute, the first FlowFile processed will be accepted and subsequent FlowFiles will not be accepted into the Bin.fragment.countApplicable only if the <Merge Strategy> property is set to Defragment. This attribute indicates how many FlowFiles should be expected in the given bundle. At least one FlowFile must have this attribute in the bundle. If multiple FlowFiles contain the "fragment.count" attribute in a given bundle, all must have the same value.segment.original.filenameApplicable only if the <Merge Strategy> property is set to Defragment. This attribute must be present on all FlowFiles with the same value for the fragment.identifier attribute. All FlowFiles in the same bundle must have the same value for this attribute. The value of this attribute will be used for the filename of the completed merged FlowFile.tar.permissionsApplicable only if the <Merge Format> property is set to TAR. The value of this attribute must be 3 characters; each character must be in the range 0 to 7 (inclusive) and indicates the file permissions that should be used for the FlowFile's TAR entry. If this attribute is missing or has an invalid value, the default value of 644 will be usedfilenameWhen more than 1 file is merged, the filename comes from the segment.original.filename attribute. If that attribute does not exist in the source FlowFiles, then the filename is set to the number of nanoseconds matching system time. Then a filename extension may be applied:if Merge Format is TAR, then the filename will be appended with .tar, if Merge Format is ZIP, then the filename will be appended with .zip, if Merge Format is FlowFileStream, then the filename will be appended with .pkgmerge.countThe number of FlowFiles that were merged into this bundlemerge.bin.ageThe age of the bin, in milliseconds, when it was merged and output. Effectively this is the greatest amount of time that any FlowFile in this bundle remained waiting in this processor before it was outputmerge.uuidUUID of the merged flow file that will be added to the original flow files attributes.merge.reasonThis processor allows for several thresholds to be configured for merging FlowFiles. This attribute indicates which of the Thresholds resulted in the FlowFiles being merged. For an explanation of each of the possible values and their meanings, see the Processor's Usage / documentation and see the 'Additional Details' page.truetrueINPUT_REQUIREDMEMORYWhile content is not stored in memory, the FlowFiles' attributes are. The configuration of MergeContent (maximum bin size, maximum group size, maximum bin age, max number of entries) will influence how much memory is used. If merging together many small FlowFiles, a two-stage approach may be necessary in order to avoid excessive use of memory.Concatenate FlowFiles with textual content together in order to create fewer, larger FlowFiles.concatenatebundleaggregatebinmergecombinesmashINPUT_ALLOWED"Merge Strategy" = "Bin Packing Algorithm" "Merge Format" = "Binary Concatenation" "Delimiter Strategy" = "Text" "Demarcator" = "\n" (a newline can be inserted by pressing Shift + Enter) "Minimum Number of Entries" = "1" "Maximum Number of Entries" = "500000000" "Minimum Group Size" = the minimum amount of data to write to an output FlowFile. A reasonable value might be "128 MB" "Maximum Group Size" = the maximum amount of data to write to an output FlowFile. A reasonable value might be "256 MB" "Max Bin Age" = the maximum amount of time to wait for incoming data before timing out and transferring the FlowFile along even though it is smaller than the Max Bin Age. A reasonable value might be "5 mins" Concatenate FlowFiles with binary content together in order to create fewer, larger FlowFiles.Not all binary data can be concatenated together. Whether or not this configuration is valid depends on the type of your data.concatenatebundleaggregatebinmergecombinesmashINPUT_ALLOWED"Merge Strategy" = "Bin Packing Algorithm" "Merge Format" = "Binary Concatenation" "Delimiter Strategy" = "Text" "Minimum Number of Entries" = "1" "Maximum Number of Entries" = "500000000" "Minimum Group Size" = the minimum amount of data to write to an output FlowFile. A reasonable value might be "128 MB" "Maximum Group Size" = the maximum amount of data to write to an output FlowFile. A reasonable value might be "256 MB" "Max Bin Age" = the maximum amount of time to wait for incoming data before timing out and transferring the FlowFile along even though it is smaller than the Max Bin Age. A reasonable value might be "5 mins" Reassemble a FlowFile that was previously split apart into smaller FlowFiles by a processor such as SplitText, UnpackContext, SplitRecord, etc.reassemblerepackmergerecombineINPUT_ALLOWED"Merge Strategy" = "Defragment" "Merge Format" = the value of Merge Format depends on the desired output format. If the file was previously zipped together and was split apart by UnpackContent, a Merge Format of "ZIP" makes sense. If it was previously a .tar file, a Merge Format of "TAR" makes sense. If the data is textual, "Binary Concatenation" can be used to combine the text into a single document. "Delimiter Strategy" = "Text" "Max Bin Age" = the maximum amount of time to wait for incoming data before timing out and transferring the fragments to 'failure'. A reasonable value might be "5 mins" For textual data, "Demarcator" should be set to a newline (\n), set by pressing Shift+Enter in the UI. For binary data, "Demarcator" should be left blank. org.apache.nifi.processors.standard.SegmentContentorg.apache.nifi.processors.standard.MergeRecordorg.apache.nifi.processors.standard.MergeRecordPROCESSORThis Processor merges together multiple record-oriented FlowFiles into a single FlowFile that contains all of the Records of the input FlowFiles. This Processor works by creating 'bins' and then adding FlowFiles to these bins until they are full. Once a bin is full, all of the FlowFiles will be combined into a single output FlowFile, and that FlowFile will be routed to the 'merged' Relationship. A bin will consist of potentially many 'like FlowFiles'. In order for two FlowFiles to be considered 'like FlowFiles', they must have the same Schema (as identified by the Record Reader) and, if the <Correlation Attribute Name> property is set, the same value for the specified attribute. See Processor Usage and Additional Details for more information. NOTE: this processor should NOT be configured with Cron Driven for the Scheduling Strategy.mergerecordcontentcorrelationstreameventrecord-readerRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsemerge-strategyMerge StrategySpecifies the algorithm used to merge records. The 'Defragment' algorithm combines fragments that are associated by attributes back into a single cohesive FlowFile. The 'Bin-Packing Algorithm' generates a FlowFile populated by arbitrarily chosen FlowFilesBin-Packing AlgorithmBin-Packing AlgorithmBin-Packing AlgorithmGenerates 'bins' of FlowFiles and fills each bin as full as possible. FlowFiles are placed into a bin based on their size and optionally their attributes (if the <Correlation Attribute> property is set)DefragmentDefragmentCombines fragments that are associated by attributes back into a single cohesive FlowFile. If using this strategy, all FlowFiles must have the attributes <fragment.identifier> and <fragment.count>. All FlowFiles with the same value for "fragment.identifier" will be grouped together. All FlowFiles in this group must have the same value for the "fragment.count" attribute. The ordering of the Records that are output is not guaranteed.truefalsefalseNONEfalsefalsecorrelation-attribute-nameCorrelation Attribute NameIf specified, two FlowFiles will be binned together only if they have the same value for this Attribute. If not specified, FlowFiles are bundled by the order in which they are pulled from the queue.falsefalsefalseNONEfalsefalseAttribute StrategyAttribute StrategyDetermines which FlowFile attributes should be added to the bundle. If 'Keep All Unique Attributes' is selected, any attribute on any FlowFile that gets bundled will be kept unless its value conflicts with the value from another FlowFile. If 'Keep Only Common Attributes' is selected, only the attributes that exist on all FlowFiles in the bundle, with the same value, will be preserved.Keep Only Common AttributesKeep Only Common AttributesKeep Only Common AttributesAny attribute that is not the same on all FlowFiles in a bin will be dropped. Those that are the same across all FlowFiles will be retained.Keep All Unique AttributesKeep All Unique AttributesAny attribute that has the same value for all FlowFiles in a bin, or has no value for a FlowFile, will be kept. For example, if a bin consists of 3 FlowFiles and 2 of them have a value of 'hello' for the 'greeting' attribute and the third FlowFile has no 'greeting' attribute then the outbound FlowFile will get a 'greeting' attribute with the value 'hello'.truefalsefalseNONEfalsefalsemin-recordsMinimum Number of RecordsThe minimum number of records to include in a bin1truefalsetrueENVIRONMENTfalsefalsemax-recordsMaximum Number of RecordsThe maximum number of Records to include in a bin. This is a 'soft limit' in that if a FlowFIle is added to a bin, all records in that FlowFile will be added, so this limit may be exceeded by up to the number of records in the last input FlowFile.1000falsefalsetrueENVIRONMENTfalsefalsemin-bin-sizeMinimum Bin SizeThe minimum size of for the bin0 BtruefalsefalseNONEfalsefalsemax-bin-sizeMaximum Bin SizeThe maximum size for the bundle. If not specified, there is no maximum. This is a 'soft limit' in that if a FlowFile is added to a bin, all records in that FlowFile will be added, so this limit may be exceeded by up to the number of bytes in last input FlowFile.falsefalsefalseNONEfalsefalsemax-bin-ageMax Bin AgeThe maximum age of a Bin that will trigger a Bin to be complete. Expected format is <duration> <time unit> where <duration> is a positive integer and time unit is one of seconds, minutes, hoursfalsefalsefalseNONEfalsefalsemax.bin.countMaximum Number of BinsSpecifies the maximum number of bins that can be held in memory at any one time. This number should not be smaller than the maximum number of concurrent threads for this Processor, or the bins that are created will often consist only of a single incoming FlowFile.10truefalsefalseNONEfalsefalsefailureIf the bundle cannot be created, all FlowFiles that would have been used to created the bundle will be transferred to failurefalseoriginalThe FlowFiles that were used to create the bundlefalsemergedThe FlowFile containing the merged recordsfalsefragment.identifierApplicable only if the <Merge Strategy> property is set to Defragment. All FlowFiles with the same value for this attribute will be bundled together.fragment.countApplicable only if the <Merge Strategy> property is set to Defragment. This attribute must be present on all FlowFiles with the same value for the fragment.identifier attribute. All FlowFiles in the same bundle must have the same value for this attribute. The value of this attribute indicates how many FlowFiles should be expected in the given bundle.record.countThe merged FlowFile will have a 'record.count' attribute indicating the number of records that were written to the FlowFile.mime.typeThe MIME Type indicated by the Record Writermerge.countThe number of FlowFiles that were merged into this bundlemerge.bin.ageThe age of the bin, in milliseconds, when it was merged and output. Effectively this is the greatest amount of time that any FlowFile in this bundle remained waiting in this processor before it was outputmerge.uuidUUID of the merged FlowFile that will be added to the original FlowFiles attributes<Attributes from Record Writer>Any Attribute that the configured Record Writer returns will be added to the FlowFile.truetrueINPUT_REQUIREDCombine together many arbitrary Records in order to create a single, larger fileINPUT_ALLOWEDConfigure the "Record Reader" to specify a Record Reader that is appropriate for the incoming data type. Configure the "Record Writer" to specify a Record Writer that is appropriate for the desired output data type. Set "Merge Strategy" to `Bin-Packing Algorithm`. Set the "Minimum Bin Size" to desired file size of the merged output file. For example, a value of `1 MB` will result in not merging data until at least 1 MB of data is available (unless the Max Bin Age is reached first). If there is no desired minimum file size, leave the default value of `0 B`. Set the "Minimum Number of Records" property to the minimum number of Records that should be included in the merged output file. For example, setting the value to `10000` ensures that the output file will have at least 10,000 Records in it (unless the Max Bin Age is reached first). Set the "Max Bin Age" to specify the maximum amount of time to hold data before merging. This can be thought of as a "timeout" at which time the Processor will merge whatever data it is, even if the "Minimum Bin Size" and "Minimum Number of Records" has not been reached. It is always recommended to set the value. A reasonable default might be `10 mins` if there is no other latency requirement. Connect the 'merged' Relationship to the next component in the flow. Auto-terminate the 'original' Relationship. Combine together many Records that have the same value for a particular field in the data, in order to create a single, larger filemergecombineaggregatelike recordssimilar dataorg.apache.nifi.processors.standard.PartitionRecordConfigure the "Record Reader" to specify a Record Reader that is appropriate for the incoming data type. Configure the "Record Writer" to specify a Record Writer that is appropriate for the desired output data type. Add a single additional property. The name of the property should describe the field on which the data is being merged together. The property's value should be a RecordPath that specifies which output FlowFile the Record belongs to. For example, to merge together data that has the same value for the "productSku" field, add a property named `productSku` with a value of `/productSku`. Connect the "success" Relationship to MergeRecord. Auto-terminate the "original" Relationship. org.apache.nifi.processors.standard.MergeRecordConfigure the "Record Reader" to specify a Record Reader that is appropriate for the incoming data type. Configure the "Record Writer" to specify a Record Writer that is appropriate for the desired output data type. Set "Merge Strategy" to `Bin-Packing Algorithm`. Set the "Minimum Bin Size" to desired file size of the merged output file. For example, a value of `1 MB` will result in not merging data until at least 1 MB of data is available (unless the Max Bin Age is reached first). If there is no desired minimum file size, leave the default value of `0 B`. Set the "Minimum Number of Records" property to the minimum number of Records that should be included in the merged output file. For example, setting the value to `10000` ensures that the output file will have at least 10,000 Records in it (unless the Max Bin Age is reached first). Set the "Maximum Number of Records" property to a value at least as large as the "Minimum Number of Records." If there is no need to limit the maximum number of records per file, this number can be set to a value that will never be reached such as `1000000000`. Set the "Max Bin Age" to specify the maximum amount of time to hold data before merging. This can be thought of as a "timeout" at which time the Processor will merge whatever data it is, even if the "Minimum Bin Size" and "Minimum Number of Records" has not been reached. It is always recommended to set the value. A reasonable default might be `10 mins` if there is no other latency requirement. Set the value of the "Correlation Attribute Name" property to the name of the property that you added in the PartitionRecord Processor. For example, if merging data based on the "productSku" field, the property in PartitionRecord was named `productSku` so the value of the "Correlation Attribute Name" property should be `productSku`. Set the "Maximum Number of Bins" property to a value that is at least as large as the different number of values that will be present for the Correlation Attribute. For example, if you expect 1,000 different SKUs, set this value to at least `1001`. It is not advisable, though, to set the value above 10,000. Connect the 'merged' Relationship to the next component in the flow. Auto-terminate the 'original' Relationship. org.apache.nifi.processors.standard.MergeContentorg.apache.nifi.processors.standard.SplitRecordorg.apache.nifi.processors.standard.PartitionRecordorg.apache.nifi.processors.standard.ModifyBytesPROCESSORDiscard byte range at the start and end or all content of a binary file.binarydiscardkeepStart OffsetStart OffsetNumber of bytes removed at the beginning of the file.0 BtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseEnd OffsetEnd OffsetNumber of bytes removed at the end of the file.0 BtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseRemove All ContentRemove All ContentRemove all content from the FlowFile superseding Start Offset and End Offset properties.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsesuccessProcessed flowfiles.falsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.MonitorActivityPROCESSORMonitors the flow for activity and sends out an indicator when the flow has not had any data for some specified amount of time and again when the flow's activity is restoredmonitorflowactiveinactiveactivitydetectionThreshold DurationThreshold DurationDetermines how much time must elapse before considering the flow to be inactive5 mintruefalsetrueENVIRONMENTfalsefalseContinually Send MessagesContinually Send MessagesIf true, will send inactivity indicator continually every Threshold Duration amount of time until activity is restored; if false, will send an indicator only when the flow first becomes inactivefalsetruetruefalsefalsetruefalsefalseNONEfalsefalseInactivity MessageInactivity MessageThe message that will be the content of FlowFiles that are sent to the 'inactive' relationshipLacking activity as of time: ${now():format('yyyy/MM/dd HH:mm:ss')}; flow has been inactive for ${inactivityDurationMillis:toNumber():divide(60000)} minutestruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseActivity Restored MessageActivity Restored MessageThe message that will be the content of FlowFiles that are sent to 'activity.restored' relationshipActivity restored at time: ${now():format('yyyy/MM/dd HH:mm:ss')} after being inactive for ${inactivityDurationMillis:toNumber():divide(60000)} minutestruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseWait for ActivityWait for ActivityWhen the processor gets started or restarted, if set to true, only send an inactive indicator if there had been activity beforehand. Otherwise send an inactive indicator even if there had not been activity beforehand.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseReset State on RestartReset State on RestartWhen the processor gets started or restarted, if set to true, the initial state will always be active. Otherwise, the last reported flow state will be preserved.truetruetruefalsefalsetruefalsefalseNONEfalsefalseCopy AttributesCopy AttributesIf true, will copy all flow file attributes from the flow file that resumed activity to the newly created indicator flow filefalsetruetruefalsefalsetruefalsefalseNONEfalsefalseMonitoring ScopeMonitoring ScopeSpecify how to determine activeness of the flow. 'node' means that activeness is examined at individual node separately. It can be useful if DFM expects each node should receive flow files in a distributed manner. With 'cluster', it defines the flow is active while at least one node receives flow files actively. If NiFi is running as standalone mode, this should be set as 'node', if it's 'cluster', NiFi logs a warning message and act as 'node' scope.nodenodenodeclusterclustertruefalsefalseNONEfalsefalseReporting NodeReporting NodeSpecify which node should send notification flow-files to inactive and activity.restored relationships. With 'all', every node in this cluster send notification flow-files. 'primary' means flow-files will be sent only from a primary node. If NiFi is running as standalone mode, this should be set as 'all', even if it's 'primary', NiFi act as 'all'.allallallprimaryprimarytruefalsefalseNONEfalsefalseMonitoring ScopeMonitoring ScopeclusterinactiveThis relationship is used to transfer an Inactivity indicator when no FlowFiles are routed to 'success' for Threshold Duration amount of timefalsesuccessAll incoming FlowFiles are routed to successfalseactivity.restoredThis relationship is used to transfer an Activity Restored indicator when FlowFiles are routing to 'success' following a period of inactivityfalseinactivityStartMillisThe time at which Inactivity began, in the form of milliseconds since EpochinactivityDurationMillisThe number of milliseconds that the inactivity has spannedtruetruetrueMonitorActivity stores the last timestamp at each node as state, so that it can examine activity at cluster wide. If 'Copy Attribute' is set to true, then flow file attributes are also persisted. In local scope, it stores last known activity timestamp if the flow is inactive.CLUSTERLOCALINPUT_REQUIREDorg.apache.nifi.processors.standard.NotifyPROCESSORCaches a release signal identifier in the distributed cache, optionally along with the FlowFile's attributes. Any flow files held at a corresponding Wait processor will be released once this signal in the cache is discovered.mapcachenotifydistributedsignalreleaserelease-signal-idRelease Signal IdentifierA value, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the release signal cache keytruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesignal-counter-nameSignal Counter NameA value, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the signal counter name. Signal counter name is useful when a corresponding Wait processor needs to know the number of occurrences of different types of events, such as success or failure, or destination data source names, etc.defaulttruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesignal-counter-deltaSignal Counter DeltaA value, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the signal counter delta. Specify how much the counter should increase. For example, if multiple signal events are processed at upstream flow in batch oriented way, the number of events processed can be notified with this property at once. Zero (0) has a special meaning, it clears target count back to 0, which is especially useful when used with Wait Releasable FlowFile Count = Zero (0) mode, to provide 'open-close-gate' type of flow control. One (1) can open a corresponding Wait processor, and Zero (0) can negate it as if closing a gate.1truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesignal-buffer-countSignal Buffer CountSpecify the maximum number of incoming flow files that can be buffered until signals are notified to cache service. The more buffer can provide the better performance, as it reduces the number of interactions with cache service by grouping signals by signal identifier when multiple incoming flow files share the same signal identifier.1truefalsefalseNONEfalsefalsedistributed-cache-serviceDistributed Cache ServiceThe Controller Service that is used to cache release signals in order to release files queued at a corresponding Wait processororg.apache.nifi.distributed.cache.client.AtomicDistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseattribute-cache-regexAttribute Cache RegexAny attributes whose names match this regex will be stored in the distributed cache to be copied to any FlowFiles released from a corresponding Wait processor. Note that the uuid attribute will not be cached regardless of this value. If blank, no attributes will be cached.falsefalsefalseNONEfalsefalsesuccessAll FlowFiles where the release signal has been successfully entered in the cache will be routed to this relationshipfalsefailureWhen the cache cannot be reached, or if the Release Signal Identifier evaluates to null or empty, FlowFiles will be routed to this relationshipfalsenotifiedAll FlowFiles will have an attribute 'notified'. The value of this attribute is true, is the FlowFile is notified, otherwise false.trueINPUT_REQUIREDorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientServiceorg.apache.nifi.distributed.cache.server.map.DistributedMapCacheServerorg.apache.nifi.processors.standard.Waitorg.apache.nifi.processors.standard.PackageFlowFilePROCESSORThis processor will package FlowFile attributes and content into an output FlowFile that can be exported from NiFi and imported back into NiFi, preserving the original attributes and content.flowfileflowfile-streamflowfile-stream-v3packageattributesmax-batch-sizeMaximum Batch SizeMaximum number of FlowFiles to package into one output FlowFile using a best effort, non guaranteed approach. Multiple input queues can produce unexpected batching behavior.1truefalsefalseNONEfalsefalseoriginalThe FlowFiles that were used to create the package are sent to this relationshipfalsesuccessThe packaged FlowFile is sent to this relationshipfalsemime.typeThe mime.type will be changed to application/flowfile-v3truetrueINPUT_REQUIREDSend FlowFile content and attributes from one NiFi instance to another NiFi instance.A Remote Process Group is preferred to send FlowFiles between two NiFi instances, but an alternative is to use PackageFlowFile then InvokeHTTP sending to ListenHTTP.flowfileattributescontentffv3flowfile-stream-v3transferorg.apache.nifi.processors.standard.PackageFlowFile "Maximum Batch Size" > 1 can help improve performance by batching many flowfiles together into 1 larger file that is transmitted by InvokeHTTP. Connect the success relationship of PackageFlowFile to the input of InvokeHTTP. org.apache.nifi.processors.standard.InvokeHTTP "HTTP Method" = "POST" to send data to ListenHTTP. "HTTP URL" should include the hostname, port, and path to the ListenHTTP. "Request Content-Type" = "${mime.type}" because PackageFlowFile output files have attribute mime.type=application/flowfile-v3. org.apache.nifi.processors.standard.ListenHTTP "Listening Port" = a unique port number. ListenHTTP automatically unpacks files that have attribute mime.type=application/flowfile-v3. If PackageFlowFile batches 99 FlowFiles into 1 file that InvokeHTTP sends, then the original 99 FlowFiles will be output by ListenHTTP. Export FlowFile content and attributes from NiFi to external storage and reimport.flowfileattributescontentffv3flowfile-stream-v3offlinestorageorg.apache.nifi.processors.standard.PackageFlowFile "Maximum Batch Size" > 1 can improve storage efficiency by batching many FlowFiles together into 1 larger file that is stored. Connect the success relationship to the input of any NiFi egress processor for offline storage. org.apache.nifi.processors.standard.UnpackContent "Packaging Format" = "application/flowfile-v3". Connect the output of a NiFi ingress processor that reads files stored offline to the input of UnpackContent. If PackageFlowFile batches 99 FlowFiles into 1 file that is read from storage, then the original 99 FlowFiles will be output by UnpackContent. org.apache.nifi.processors.standard.UnpackContentorg.apache.nifi.processors.standard.MergeContentorg.apache.nifi.processors.standard.ParseCEFPROCESSORParses the contents of a CEF formatted message and adds attributes to the FlowFile for headers and extensions of the parts of the CEF message. Note: This Processor expects CEF messages WITHOUT the syslog headers (i.e. starting at "CEF:0"logscefattributessystemeventmessageFIELDS_DESTINATIONParsed fields destinationIndicates whether the results of the CEF parser are written to the FlowFile content or a FlowFile attribute; if using flowfile-attributeattribute, fields will be populated as attributes. If set to flowfile-content, the CEF extension field will be converted into a flat JSON object.flowfile-contentflowfile-contentflowfile-contentflowfile-attributeflowfile-attributetruefalsefalseNONEfalsefalseAPPEND_RAW_MESSAGE_TO_JSONAppend raw message to JSONWhen using flowfile-content (i.e. JSON output), add the original CEF message to the resulting JSON object. The original message is added as a string to _raw.truetruefalsefalseNONEfalsefalseINCLUDE_CUSTOM_EXTENSIONSInclude custom extensionsIf set to true, custom extensions (not specified in the CEF specifications) will be included in the generated data/attributes.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseACCEPT_EMPTY_EXTENSIONSAccept empty extensionsIf set to true, empty extensions will be accepted and will be associated to a null value.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseTIME_REPRESENTATIONTimezoneTimezone to be used when representing date fields. UTC will convert all dates to UTC, while Local Timezone will convert them to the timezone used by NiFi.Local Timezone (system Default)UTCUTCLocal Timezone (system Default)Local Timezone (system Default)truefalsefalseNONEfalsefalseDATETIME_REPRESENTATIONDateTime LocaleThe IETF BCP 47 representation of the Locale to be used when parsing date fields with long or short month names (e.g. may <en-US> vs. mai. <fr-FR>. The defaultvalue is generally safe. Only change if having issues parsing CEF messagesen-UStruefalsefalseNONEfalsefalsesuccessAny FlowFile that is successfully parsed as a CEF message will be transferred to this Relationship.falsefailureAny FlowFile that could not be parsed as a CEF message will be transferred to this Relationship without any attributes being addedfalsecef.header.versionThe version of the CEF message.cef.header.deviceVendorThe Device Vendor of the CEF message.cef.header.deviceProductThe Device Product of the CEF message.cef.header.deviceVersionThe Device Version of the CEF message.cef.header.deviceEventClassIdThe Device Event Class ID of the CEF message.cef.header.nameThe name of the CEF message.cef.header.severityThe severity of the CEF message.cef.extension.*The key and value generated by the parsing of the message.truetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ParseSyslogorg.apache.nifi.processors.standard.ParseSyslogPROCESSORAttempts to parses the contents of a Syslog message in accordance to RFC5424 and RFC3164 formats and adds attributes to the FlowFile for each of the parts of the Syslog message.Note: Be mindfull that RFC3164 is informational and a wide range of different implementations are present in the wild. If messages fail parsing, considering using RFC5424 or using a generic parsing processors such as ExtractGrok.logssyslogattributessystemeventmessageCharacter SetCharacter SetSpecifies which character set of the Syslog messagesUTF-8truefalsefalseNONEfalsefalsesuccessAny FlowFile that is successfully parsed as a Syslog message will be to this Relationship.falsefailureAny FlowFile that could not be parsed as a Syslog message will be transferred to this Relationship without any attributes being addedfalsesyslog.priorityThe priority of the Syslog message.syslog.severityThe severity of the Syslog message derived from the priority.syslog.facilityThe facility of the Syslog message derived from the priority.syslog.versionThe optional version from the Syslog message.syslog.timestampThe timestamp of the Syslog message.syslog.hostnameThe hostname or IP address of the Syslog message.syslog.senderThe hostname of the Syslog server that sent the message.syslog.bodyThe body of the Syslog message, everything after the hostname.truetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ListenSyslogorg.apache.nifi.processors.standard.PutSyslogorg.apache.nifi.processors.standard.ParseSyslog5424PROCESSORAttempts to parse the contents of a well formed Syslog message in accordance to RFC5424 format and adds attributes to the FlowFile for each of the parts of the Syslog message, including Structured Data.Structured Data will be written to attributes as one attribute per item id + parameter see https://tools.ietf.org/html/rfc5424.Note: ParseSyslog5424 follows the specification more closely than ParseSyslog. If your Syslog producer does not follow the spec closely, with regards to using '-' for missing header entries for example, those logs will fail with this parser, where they would not fail with ParseSyslog.logssyslogsyslog5424attributessystemeventmessageCharacter SetCharacter SetSpecifies which character set of the Syslog messagesUTF-8truefalsefalseNONEfalsefalsenil_policyNIL PolicyDefines how NIL values are handled for header fields.NULLOMITOMITThe missing field will not have an attribute added.NULLNULLThe missing field will have an empty attribute added.DASHDASHThe missing field will have an attribute added with the value of '-'.truefalsefalseNONEfalsefalseinclude_policyInclude Message Body in AttributesIf true, then the Syslog Message body will be included in the attributes.truetruetruefalsefalsefalsefalsefalseNONEfalsefalsesuccessAny FlowFile that is successfully parsed as a Syslog message will be to this Relationship.falsefailureAny FlowFile that could not be parsed as a Syslog message will be transferred to this Relationship without any attributes being addedfalsesyslog.priorityThe priority of the Syslog message.syslog.severityThe severity of the Syslog message derived from the priority.syslog.facilityThe facility of the Syslog message derived from the priority.syslog.versionThe optional version from the Syslog message.syslog.timestampThe timestamp of the Syslog message.syslog.hostnameThe hostname or IP address of the Syslog message.syslog.appnameThe appname of the Syslog message.syslog.procidThe procid of the Syslog message.syslog.messageidThe messageid the Syslog message.syslog.structuredDataMultiple entries per structuredData of the Syslog message.syslog.senderThe hostname of the Syslog server that sent the message.syslog.bodyThe body of the Syslog message, everything after the hostname.truetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ListenSyslogorg.apache.nifi.processors.standard.ParseSyslogorg.apache.nifi.processors.standard.PutSyslogorg.apache.nifi.processors.standard.PartitionRecordPROCESSORSplits, or partitions, record-oriented data based on the configured fields in the data. One or more properties must be added. The name of the property is the name of an attribute to add. The value of the property is a RecordPath to evaluate against each Record. Two records will go to the same outbound FlowFile only if they have the same value for each of the given RecordPaths. Because we know that all records in a given output FlowFile have the same value for the fields that are specified by the RecordPath, an attribute is added for each field. See Additional Details on the Usage page for more information and examples.recordpartitionrecordpathrpathsegmentsplitgroupbinorganizerecord-readerRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseThe name given to the dynamic property is the name of the attribute that will be used to denote the value of the associated RecordPath.A RecordPath that points to a field in the Record.Each dynamic property represents a RecordPath that will be evaluated against each record in an incoming FlowFile. When the value of the RecordPath is determined for a Record, an attribute is added to the outgoing FlowFile. The name of the attribute is the same as the name of this property. The value of the attribute is the same as the value of the field in the Record that the RecordPath points to. Note that no attribute will be added if the value returned for the RecordPath is null or is not a scalar value (i.e., the value is an Array, Map, or Record).FLOWFILE_ATTRIBUTESsuccessFlowFiles that are successfully partitioned will be routed to this relationshipfalsefailureIf a FlowFile cannot be partitioned from the configured input format to the configured output format, the unchanged FlowFile will be routed to this relationshipfalseoriginalOnce all records in an incoming FlowFile have been partitioned, the original FlowFile is routed to this relationship.falserecord.countThe number of records in an outgoing FlowFilemime.typeThe MIME Type that the configured Record Writer indicates is appropriatefragment.identifierAll partitioned FlowFiles produced from the same parent FlowFile will have the same randomly generated UUID added for this attributefragment.indexA one-up number that indicates the ordering of the partitioned FlowFiles that were created from a single parent FlowFilefragment.countThe number of partitioned FlowFiles generated from the parent FlowFilesegment.original.filename The filename of the parent FlowFile<dynamic property name>For each dynamic property that is added, an attribute may be added to the FlowFile. See the description for Dynamic Properties for more information.trueINPUT_REQUIREDSeparate records into separate FlowFiles so that all of the records in a FlowFile have the same value for a given field or set of fields.separatesplitpartitionbreak apartcolocatesegregaterecordfieldrecordpathINPUT_ALLOWEDChoose a RecordReader that is appropriate based on the format of the incoming data. Choose a RecordWriter that writes the data in the desired output format. Add a single additional property. The name of the property should describe the type of data that is being used to partition the data. The property's value should be a RecordPath that specifies which output FlowFile the Record belongs to. For example, if we want to separate records based on their `transactionType` field, we could add a new property named `transactionType`. The value of the property might be `/transaction/type`. An input FlowFile will then be separated into as few FlowFiles as possible such that each output FlowFile has the same value for the `transactionType` field. Separate records based on whether or not they adhere to a specific criteriaseparatesplitpartitionbreak apartsegregaterecordfieldrecordpathcriteriaINPUT_ALLOWEDChoose a RecordReader that is appropriate based on the format of the incoming data. Choose a RecordWriter that writes the data in the desired output format. Add a single additional property. The name of the property should describe the criteria. The property's value should be a RecordPath that returns `true` if the Record meets the criteria or `false` otherwise. For example, if we want to separate records based on whether or not they have a transaction total of more than $1,000 we could add a new property named `largeTransaction` with a value of `/transaction/total > 1000`. This will create two FlowFiles. In the first, all records will have a total over `1000`. In the second, all records will have a transaction less than or equal to 1000. Each FlowFile will have an attribute named `largeTransaction` with a value of `true` or `false`. org.apache.nifi.processors.standard.ConvertRecordorg.apache.nifi.processors.standard.SplitRecordorg.apache.nifi.processors.standard.UpdateRecordorg.apache.nifi.processors.standard.QueryRecordorg.apache.nifi.processors.standard.PutDatabaseRecordPROCESSORThe PutDatabaseRecord processor uses a specified RecordReader to input (possibly multiple) records from an incoming flow file. These records are translated to SQL statements and executed as a single transaction. If any errors occur, the flow file is routed to failure or retry, and if the records are transmitted successfully, the incoming flow file is routed to success. The type of statement executed by the processor is specified via the Statement Type property, which accepts some hard-coded values such as INSERT, UPDATE, and DELETE, as well as 'Use statement.type Attribute', which causes the processor to get the statement type from a flow file attribute. IMPORTANT: If the Statement Type is UPDATE, then the incoming records must not alter the value(s) of the primary keys (or user-specified Update Keys). If such records are encountered, the UPDATE statement issued to the database may do nothing (if no existing records with the new primary key values are found), or could inadvertently corrupt the existing data (by changing records for which the new values of the primary keys exist).sqlrecordjdbcputdatabaseupdateinsertdeleteput-db-record-record-readerRecord ReaderSpecifies the Controller Service to use for parsing incoming data and determining the data's schema.org.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLfalsefalsefalseNONEfalsefalseput-db-record-statement-typeStatement TypeSpecifies the type of SQL Statement to generate. Please refer to the database documentation for a description of the behavior of each operation. Please note that some Database Types may not support certain Statement Types. If 'Use statement.type Attribute' is chosen, then the value is taken from the statement.type attribute in the FlowFile. The 'Use statement.type Attribute' option is the only one that allows the 'SQL' statement type. If 'SQL' is specified, the value of the field specified by the 'Field Containing SQL' property is expected to be a valid SQL statement on the target database, and will be executed as-is.UPDATEUPDATEINSERTINSERTUPSERTUPSERTINSERT_IGNOREINSERT_IGNOREDELETEDELETEUse statement.type AttributeUse statement.type AttributeUse Record PathUse Record PathtruefalsefalseNONEfalsefalseStatement Type Record PathStatement Type Record PathSpecifies a RecordPath to evaluate against each Record in order to determine the Statement Type. The RecordPath should equate to either INSERT, UPDATE, UPSERT, or DELETE. (Debezium style operation types are also supported: "r" and "c" for INSERT, "u" for UPDATE, and "d" for DELETE)truefalsefalseNONEfalsefalseput-db-record-statement-typeStatement TypeUse Record PathData Record PathData Record PathIf specified, this property denotes a RecordPath that will be evaluated against each incoming Record and the Record that results from evaluating the RecordPath will be sent to the database instead of sending the entire incoming Record. If not specified, the entire incoming Record will be published to the database.falsefalsefalseNONEfalsefalseput-db-record-dcbp-serviceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database for sending records.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseput-db-record-catalog-nameCatalog NameThe name of the catalog that the statement should update. This may not apply for the database that you are updating. In this case, leave the field empty. Note that if the property is set and the database is case-sensitive, the catalog name must match the database's catalog name exactly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-schema-nameSchema NameThe name of the schema that the table belongs to. This may not apply for the database that you are updating. In this case, leave the field empty. Note that if the property is set and the database is case-sensitive, the schema name must match the database's schema name exactly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-table-nameTable NameThe name of the table that the statement should affect. Note that if the database is case-sensitive, the table name must match the database's table name exactly.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-binary-formatBinary String FormatThe format to be applied when decoding string values to binary.UTF-8UTF-8UTF-8String values for binary columns contain the original value as text via UTF-8 character encodingHexadecimalHexadecimalString values for binary columns contain the original value in hexadecimal formatBase64Base64String values for binary columns contain the original value in Base64 encoded formattruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-translate-field-namesTranslate Field NamesIf true, the Processor will attempt to translate field names into the appropriate column names for the table specified. If false, the field names must match the column names exactly, or the column will not be updatedtruetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-db-record-unmatched-field-behaviorUnmatched Field BehaviorIf an incoming record has a field that does not map to any of the database table's columns, this property specifies how to handle the situationIgnore Unmatched FieldsIgnore Unmatched FieldsIgnore Unmatched FieldsAny field in the document that cannot be mapped to a column in the database is ignoredFail on Unmatched FieldsFail on Unmatched FieldsIf the document has any field that cannot be mapped to a column in the database, the FlowFile will be routed to the failure relationshipfalsefalsefalseNONEfalsefalseput-db-record-unmatched-column-behaviorUnmatched Column BehaviorIf an incoming record does not have a field mapping for all of the database table's columns, this property specifies how to handle the situationFail on Unmatched ColumnsIgnore Unmatched ColumnsIgnore Unmatched ColumnsAny column in the database that does not have a field in the document will be assumed to not be required. No notification will be loggedWarn on Unmatched ColumnsWarn on Unmatched ColumnsAny column in the database that does not have a field in the document will be assumed to not be required. A warning will be loggedFail on Unmatched ColumnsFail on Unmatched ColumnsA flow will fail if any column in the database that does not have a field in the document. An error will be loggedfalsefalsefalseNONEfalsefalseput-db-record-update-keysUpdate KeysA comma-separated list of column names that uniquely identifies a row in the database for UPDATE statements. If the Statement Type is UPDATE and this property is not set, the table's Primary Keys are used. In this case, if no Primary Key exists, the conversion to SQL will fail if Unmatched Column Behaviour is set to FAIL. This property is ignored if the Statement Type is INSERTfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-statement-typeStatement TypeUse statement.type AttributeUse Record PathUPSERTUPDATESQLput-db-record-field-containing-sqlField Containing SQLIf the Statement Type is 'SQL' (as set in the statement.type attribute), this field indicates which field in the record(s) contains the SQL statement to execute. The value of the field must be a single SQL statement. If the Statement Type is not 'SQL', this field is ignored.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseput-db-record-statement-typeStatement TypeUse statement.type AttributeUse Record Pathput-db-record-allow-multiple-statementsAllow Multiple SQL StatementsIf the Statement Type is 'SQL' (as set in the statement.type attribute), this field indicates whether to split the field value by a semicolon and execute each statement separately. If any statement causes an error, the entire set of statements will be rolled back. If the Statement Type is not 'SQL', this field is ignored.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseput-db-record-statement-typeStatement TypeUse statement.type AttributeUse Record Pathput-db-record-quoted-identifiersQuote Column IdentifiersEnabling this option will cause all column names to be quoted, allowing you to use reserved words as column names in your tables.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-db-record-quoted-table-identifiersQuote Table IdentifiersEnabling this option will cause the table name to be quoted to support the use of special characters in the table name.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseput-db-record-query-timeoutMax Wait TimeThe maximum amount of time allowed for a running SQL statement , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueENVIRONMENTfalsefalserollback-on-failureRollback On FailureSpecify how to handle error. By default (false), if an error occurs while processing a FlowFile, the FlowFile will be routed to 'failure' or 'retry' relationship based on error type, and processor can continue with next FlowFile. Instead, you may want to rollback currently processed FlowFiles and stop further processing immediately. In that case, you can do so by enabling this 'Rollback On Failure' property. If enabled, failed FlowFiles will stay in the input relationship without penalizing it and being processed repeatedly until it gets processed successfully or removed by other means. It is important to set adequate 'Yield Duration' to avoid retrying too frequently.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsetable-schema-cache-sizeTable Schema Cache SizeSpecifies how many Table Schemas should be cached100truefalsefalseNONEfalsefalseput-db-record-max-batch-sizeMaximum Batch SizeSpecifies maximum number of sql statements to be included in each batch sent to the database. Zero means the batch size is not limited, and all statements are put into a single batch which can cause high memory usage issues for a very large number of statements.1000falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedatabase-session-autocommitDatabase Session AutoCommitThe autocommit mode to set on the database connection being used. If set to false, the operation(s) will be explicitly committed or rolled back (based on success or failure respectively). If set to true, the driver/database automatically handles the commit/rollback.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseretryA FlowFile is routed to this relationship if the database cannot be updated but attempting the operation again may succeedfalsesuccessSuccessfully created FlowFile from SQL query result set.falsefailureA FlowFile is routed to this relationship if the database cannot be updated and retrying the operation will also fail, such as an invalid query or an integrity constraint violationfalsestatement.typeIf 'Use statement.type Attribute' is selected for the Statement Type property, the value of this attribute will be used to determine the type of statement (INSERT, UPDATE, DELETE, SQL, etc.) to generate and execute.putdatabaserecord.errorIf an error occurs during processing, the flow file will be routed to failure or retry, and this attribute will be populated with the cause of the error.INPUT_REQUIREDInsert records into a databaseINPUT_ALLOWEDorg.apache.nifi.processors.standard.PutDistributedMapCachePROCESSORGets the content of a FlowFile and puts it to a distributed map cache, using a cache key computed from FlowFile attributes. If the cache already contains the entry and the cache update strategy is 'keep original' the entry is not replaced.'mapcacheputdistributedCache Entry IdentifierCache Entry IdentifierA FlowFile attribute, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the cache keytruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseDistributed Cache ServiceDistributed Cache ServiceThe Controller Service that is used to cache flow filesorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseCache update strategyCache update strategyDetermines how the cache is updated if the cache already contains the entryreplaceReplace if presentreplaceAdds the specified entry to the cache, replacing any value that is currently set.Keep originalkeeporiginalAdds the specified entry to the cache, if the key does not exist.truefalsefalseNONEfalsefalseMax cache entry sizeMax cache entry sizeThe maximum amount of data to put into cache1 MBfalsefalsefalseNONEfalsefalsesuccessAny FlowFile that is successfully inserted into cache will be routed to this relationshipfalsefailureAny FlowFile that cannot be inserted into the cache will be routed to this relationshipfalsecachedAll FlowFiles will have an attribute 'cached'. The value of this attribute is true, is the FlowFile is cached, otherwise false.trueINPUT_REQUIREDorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientServiceorg.apache.nifi.distributed.cache.server.map.DistributedMapCacheServerorg.apache.nifi.processors.standard.FetchDistributedMapCacheorg.apache.nifi.processors.standard.PutEmailPROCESSORSends an e-mail to configured recipients for each incoming FlowFileemailputnotifysmtpSMTP HostnameSMTP HostnameThe hostname of the SMTP hosttruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP PortSMTP PortThe Port used for SMTP communications25truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseauthorization-modeAuthorization ModeHow to authorize sending email on the user's behalf.password-based-authorization-modeUse Passwordpassword-based-authorization-modeUse passwordUse OAuth2oauth-based-authorization-modeUse OAuth2 to acquire access tokentruefalsefalseNONEfalsefalseoauth2-access-token-providerOAuth2 Access Token ProviderOAuth2 service that can provide access tokens.org.apache.nifi.oauth2.OAuth2AccessTokenProviderorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseauthorization-modeAuthorization Modeoauth-based-authorization-modeSMTP UsernameSMTP UsernameUsername for the SMTP accountfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP PasswordSMTP PasswordPassword for the SMTP accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseauthorization-modeAuthorization Modepassword-based-authorization-modeSMTP AuthSMTP AuthFlag indicating whether authentication should be usedtruetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP TLSSMTP STARTTLSFlag indicating whether Opportunistic TLS should be enabled using STARTTLS commandfalsetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP Socket FactorySMTP Socket FactorySocket Factory to use for SMTP Connectionjavax.net.ssl.SSLSocketFactorytruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSMTP X-Mailer HeaderSMTP X-Mailer HeaderX-Mailer used in the header of the outgoing emailNiFitruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseattribute-name-regexAttributes to Send as Headers (Regex)A Regular Expression that is matched against all FlowFile attribute names. Any attribute whose name matches the regex will be added to the Email messages as a Header. If not specified, no FlowFile attributes will be added as headers.falsefalsefalseNONEfalsefalseContent TypeContent TypeMime Type used to interpret the contents of the email, such as text/plain or text/htmltext/plaintruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFromFromSpecifies the Email address to use as the sender. Comma separated sequence of addresses following RFC822 syntax.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseToToThe recipients to include in the To-Line of the email. Comma separated sequence of addresses following RFC822 syntax.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCCCCThe recipients to include in the CC-Line of the email. Comma separated sequence of addresses following RFC822 syntax.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseBCCBCCThe recipients to include in the BCC-Line of the email. Comma separated sequence of addresses following RFC822 syntax.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSubjectSubjectThe email subjectMessage from NiFitruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessageMessageThe body of the email messagefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseemail-ff-content-as-messageFlow file content as messageSpecifies whether or not the FlowFile content should be the message of the email. If true, the 'Message' property is ignored.falsetruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseinput-character-setInput Character SetSpecifies the character set of the FlowFile contents for reading input FlowFile contents to generate the message body or as an attachment to the message. If not set, UTF-8 will be the default value.UTF-8truefalsefalseNONEfalsefalseAttach FileAttach FileSpecifies whether or not the FlowFile content should be attached to the emailfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseInclude All Attributes In MessageInclude All Attributes In MessageSpecifies whether or not all FlowFile attributes should be recorded in the body of the email messagefalsetruetruefalsefalsetruefalsefalseNONEfalsefalsemail.propertyNameValue for a specific property to be set in the JavaMail Session objectDynamic property names that will be passed to the Mail session. Possible properties can be found in: https://javaee.github.io/javamail/docs/api/com/sun/mail/smtp/package-summary.html.FLOWFILE_ATTRIBUTEStruesuccessFlowFiles that are successfully sent will be routed to this relationshipfalsefailureFlowFiles that fail to send will be routed to this relationshipfalsetrueINPUT_REQUIREDMEMORYThe entirety of the FlowFile's content (as a String object) will be read into memory in case the property to use the flow file content as the email body is set to true.org.apache.nifi.processors.standard.PutFTPPROCESSORSends FlowFiles to an FTP ServerremotecopyegressputftparchivefilesHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers21truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push filesfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCreate DirectoryCreate DirectorySpecifies whether or not the remote directory should be created if it does not exist.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseBatch SizeBatch SizeThe maximum number of FlowFiles to send in a single connection500truefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseConflict ResolutionConflict ResolutionDetermines how to handle the problem of filename collisionsNONEREPLACEREPLACERemote file is replaced with new file, FlowFile goes to successIGNOREIGNOREFile is not transferred, FlowFile goes to successRENAMERENAMENew file is renamed with a one-up number at the beginning, FlowFile goes to successREJECTREJECTFile is not transferred, FlowFile goes to rejectFAILFAILFile is not transferred, FlowFile goes to failureNONENONEDo not check for conflict before transfer, FlowFile goes to success or failuretruefalsefalseNONEfalsefalseDot RenameDot RenameIf true, then the filename of the sent file is prepended with a "." and then renamed back to the original once the file is completely sent. Otherwise, there is no rename. This property is ignored if the Temporary Filename property is set.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseTemporary FilenameTemporary FilenameIf set, the filename of the sent file will be equal to the value specified during the transfer and after successful completion will be renamed to the original filename. If this value is set, the Dot Rename property is ignored.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseTransfer ModeTransfer ModeThe FTP Transfer ModeBinaryBinaryBinaryASCIIASCIIfalsefalsefalseNONEfalsefalseConnection ModeConnection ModeThe FTP Connection ModePassiveActiveActivePassivePassivefalsefalsefalseNONEfalsefalseReject Zero-Byte FilesReject Zero-Byte FilesDetermines whether or not Zero-byte files should be rejected without attempting to transfertruetruetruefalsefalsefalsefalsefalseNONEfalsefalseLast Modified TimeLast Modified TimeThe lastModifiedTime to assign to the file after transferring it. If not set, the lastModifiedTime will not be changed. Format must be yyyy-MM-dd'T'HH:mm:ssZ. You may also use expression language such as ${file.lastModifiedTime}. If the value is invalid, the processor will not be invalid but will fail to change lastModifiedTime of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePermissionsPermissionsThe permissions to assign to the file after transferring it. Format must be either UNIX rwxrwxrwx with a - in place of denied permissions (e.g. rw-r--r--) or an octal number (e.g. 644). If not set, the permissions will not be changed. You may also use expression language such as ${file.permissions}. If the value is invalid, the processor will not be invalid but will fail to change permissions of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseInternal Buffer SizeInternal Buffer SizeSet the internal buffer size for buffered data streams16KBfalsefalsefalseNONEfalsefalseftp-use-utf8Use UTF-8 EncodingTells the client to use UTF-8 encoding when processing files and filenames. If set to true, the server must also support UTF-8 encoding.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsepre.cmd._____Not usedThe command specified in the key will be executed before doing a put. You may add these optional properties to send any commands to the FTP server before the file is actually transferred (before the put command). This option is only available for the PutFTP processor, as only FTP has this functionality. This is essentially the same as sending quote commands to an FTP server from the command line. While this is the same as sending a quote command, it is very important that you leave off the .NONEpost.cmd._____Not usedThe command specified in the key will be executed after doing a put. You may add these optional properties to send any commands to the FTP server before the file is actually transferred (before the put command). This option is only available for the PutFTP processor, as only FTP has this functionality. This is essentially the same as sending quote commands to an FTP server from the command line. While this is the same as sending a quote command, it is very important that you leave off the .NONEsuccessFlowFiles that are successfully sent will be routed to successfalserejectFlowFiles that were rejected by the destination systemfalsefailureFlowFiles that failed to send to the remote system; failure is usually looped back to this processorfalsetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.GetFTPorg.apache.nifi.processors.standard.PutFilePROCESSORWrites the contents of a FlowFile to the local file systemputlocalcopyarchivefilesfilesystemDirectoryDirectoryThe directory to which files should be written. You may use expression language such as /aa/bb/${path}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseConflict Resolution StrategyConflict Resolution StrategyIndicates what should happen when a file with the same name already exists in the output directoryfailreplacereplaceignoreignorefailfailtruefalsefalseNONEfalsefalseCreate Missing DirectoriesCreate Missing DirectoriesIf true, then missing destination directories will be created. If false, flowfiles are penalized and sent to failure.truetruetruefalsefalsetruefalsefalseNONEfalsefalseMaximum File CountMaximum File CountSpecifies the maximum number of files that can exist in the output directoryfalsefalsefalseNONEfalsefalseLast Modified TimeLast Modified TimeSets the lastModifiedTime on the output file to the value of this attribute. Format must be yyyy-MM-dd'T'HH:mm:ssZ. You may also use expression language such as ${file.lastModifiedTime}.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePermissionsPermissionsSets the permissions on the output file to the value of this attribute. Format must be either UNIX rwxrwxrwx with a - in place of denied permissions (e.g. rw-r--r--) or an octal number (e.g. 644). You may also use expression language such as ${file.permissions}.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseOwnerOwnerSets the owner on the output file to the value of this attribute. You may also use expression language such as ${file.owner}. Note on many operating systems Nifi must be running as a super-user to have the permissions to set the file owner.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseGroupGroupSets the group on the output file to the value of this attribute. You may also use expression language such as ${file.group}.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessFiles that have been successfully written to the output directory are transferred to this relationshipfalsefailureFiles that could not be written to the output directory for some reason are transferred to this relationshipfalsefilenameThe filename to use when writing the FlowFile to disk.truewrite filesystemProvides operator the ability to write to any file that NiFi has access to.INPUT_REQUIREDorg.apache.nifi.processors.standard.FetchFileorg.apache.nifi.processors.standard.GetFileorg.apache.nifi.processors.standard.PutRecordPROCESSORThe PutRecord processor uses a specified RecordReader to input (possibly multiple) records from an incoming flow file, and sends them to a destination specified by a Record Destination Service (i.e. record sink).recordputsinkput-record-readerRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseput-record-sinkRecord Destination ServiceSpecifies the Controller Service to use for writing out the query result records to some destination.org.apache.nifi.record.sink.RecordSinkServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseput-record-include-zero-record-resultsInclude Zero Record ResultsIf no records are read from the incoming FlowFile, this property specifies whether or not an empty record set will be transmitted. The original FlowFile will still be routed to success, but if no transmission occurs, no provenance SEND event will be generated.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseretryThe original FlowFile is routed to this relationship if the records could not be transmitted but attempting the operation again may succeedfalsesuccessThe original FlowFile will be routed to this relationship if the records were transmitted successfullyfalsefailureA FlowFile is routed to this relationship if the records could not be transmitted and retrying the operation will also failfalseINPUT_REQUIREDorg.apache.nifi.processors.standard.PutSFTPPROCESSORSends FlowFiles to an SFTP ServerremotecopyegressputsftparchivefilesHostnameHostnameThe fully qualified hostname or IP address of the remote systemtruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePortPortThe port that the remote system is listening on for file transfers22truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseUsernameUsernameUsernametruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePasswordPasswordPassword for the user accountfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalsePrivate Key PathPrivate Key PathThe fully qualified path to the Private Key filefalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILEPrivate Key PassphrasePrivate Key PassphrasePassword for the private keyfalsetruetrueFLOWFILE_ATTRIBUTESfalsefalseRemote PathRemote PathThe path on the remote system from which to pull or push filesfalsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseCreate DirectoryCreate DirectorySpecifies whether or not the remote directory should be created if it does not exist.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseDisable Directory ListingDisable Directory ListingIf set to 'true', directory listing is not performed prior to create missing directories. By default, this processor executes a directory listing command to see target directory existence before creating missing directories. However, there are situations that you might need to disable the directory listing such as the following. Directory listing might fail with some permission setups (e.g. chmod 100) on a directory. Also, if any other SFTP client created the directory after this processor performed a listing and before a directory creation request by this processor is finished, then an error is returned because the directory already exists.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseBatch SizeBatch SizeThe maximum number of FlowFiles to send in a single connection500truefalsefalseNONEfalsefalseConnection TimeoutConnection TimeoutAmount of time to wait before timing out while creating a connection30 sectruefalsefalseNONEfalsefalseData TimeoutData TimeoutWhen transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems30 sectruefalsefalseNONEfalsefalseConflict ResolutionConflict ResolutionDetermines how to handle the problem of filename collisionsNONEREPLACEREPLACERemote file is replaced with new file, FlowFile goes to successIGNOREIGNOREFile is not transferred, FlowFile goes to successRENAMERENAMENew file is renamed with a one-up number at the beginning, FlowFile goes to successREJECTREJECTFile is not transferred, FlowFile goes to rejectFAILFAILFile is not transferred, FlowFile goes to failureNONENONEDo not check for conflict before transfer, FlowFile goes to success or failuretruefalsefalseNONEfalsefalseReject Zero-Byte FilesReject Zero-Byte FilesDetermines whether or not Zero-byte files should be rejected without attempting to transfertruetruetruefalsefalsefalsefalsefalseNONEfalsefalseDot RenameDot RenameIf true, then the filename of the sent file is prepended with a "." and then renamed back to the original once the file is completely sent. Otherwise, there is no rename. This property is ignored if the Temporary Filename property is set.truetruetruefalsefalsefalsefalsefalseNONEfalsefalseTemporary FilenameTemporary FilenameIf set, the filename of the sent file will be equal to the value specified during the transfer and after successful completion will be renamed to the original filename. If this value is set, the Dot Rename property is ignored.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseHost Key FileHost Key FileIf supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be usedfalsefalsefalseNONEfalsefalseSINGLEFILELast Modified TimeLast Modified TimeThe lastModifiedTime to assign to the file after transferring it. If not set, the lastModifiedTime will not be changed. Format must be yyyy-MM-dd'T'HH:mm:ssZ. You may also use expression language such as ${file.lastModifiedTime}. If the value is invalid, the processor will not be invalid but will fail to change lastModifiedTime of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsePermissionsPermissionsThe permissions to assign to the file after transferring it. Format must be either UNIX rwxrwxrwx with a - in place of denied permissions (e.g. rw-r--r--) or an octal number (e.g. 644). If not set, the permissions will not be changed. You may also use expression language such as ${file.permissions}. If the value is invalid, the processor will not be invalid but will fail to change permissions of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseRemote OwnerRemote OwnerInteger value representing the User ID to set on the file after transferring it. If not set, the owner will not be set. You may also use expression language such as ${file.owner}. If the value is invalid, the processor will not be invalid but will fail to change the owner of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseRemote GroupRemote GroupInteger value representing the Group ID to set on the file after transferring it. If not set, the group will not be set. You may also use expression language such as ${file.group}. If the value is invalid, the processor will not be invalid but will fail to change the group of the file.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseStrict Host Key CheckingStrict Host Key CheckingIndicates whether or not strict enforcement of hosts keys should be appliedfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseSend Keep Alive On TimeoutSend Keep Alive On TimeoutSend a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.truetruetruefalsefalsetruefalsefalseNONEfalsefalseUse CompressionUse CompressionIndicates whether or not ZLIB compression should be used when transferring filesfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseproxy-configuration-serviceProxy Configuration ServiceSpecifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN, SOCKS + AuthNorg.apache.nifi.proxy.ProxyConfigurationServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProxy TypeProxy TypeProxy type used for file transfersDIRECTDIRECTDIRECTHTTPHTTPSOCKSSOCKSfalsefalsefalseNONEfalsefalseProxy HostProxy HostThe fully qualified hostname or IP address of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseProxy PortProxy PortThe port of the proxy serverfalsefalsetrueENVIRONMENTfalsefalseHttp Proxy UsernameHttp Proxy UsernameHttp Proxy UsernamefalsefalsetrueENVIRONMENTfalsefalseHttp Proxy PasswordHttp Proxy PasswordHttp Proxy PasswordfalsetruetrueENVIRONMENTfalsefalseCiphers AllowedCiphers AllowedA comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctrfalsefalsetrueENVIRONMENTfalsefalseKey Algorithms AllowedKey Algorithms AllowedA comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.comfalsefalsetrueENVIRONMENTfalsefalseKey Exchange Algorithms AllowedKey Exchange Algorithms AllowedA comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-cfalsefalsetrueENVIRONMENTfalsefalseMessage Authentication Codes AllowedMessage Authentication Codes AllowedA comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.comfalsefalsetrueENVIRONMENTfalsefalsesuccessFlowFiles that are successfully sent will be routed to successfalserejectFlowFiles that were rejected by the destination systemfalsefailureFlowFiles that failed to send to the remote system; failure is usually looped back to this processorfalsetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.GetSFTPorg.apache.nifi.processors.standard.PutSQLPROCESSORExecutes a SQL UPDATE or INSERT command. The content of an incoming FlowFile is expected to be the SQL command to execute. The SQL command may use the ? to escape parameters. In this case, the parameters to use must exist as FlowFile attributes with the naming convention sql.args.N.type and sql.args.N.value, where N is a positive integer. The sql.args.N.type is expected to be a number indicating the JDBC Type. The content of the FlowFile is expected to be in UTF-8 format.sqlputrdbmsdatabaseupdateinsertrelationalJDBC Connection PoolJDBC Connection PoolSpecifies the JDBC Connection Pool to use in order to convert the JSON message to a SQL statement. The Connection Pool is necessary in order to determine the appropriate database column types.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseputsql-sql-statementSQL StatementThe SQL statement to execute. The statement can be empty, a constant value, or built from attributes using Expression Language. If this property is specified, it will be used regardless of the content of incoming FlowFiles. If this property is empty, the content of the incoming FlowFile is expected to contain a valid SQL statement, to be issued by the processor to the database.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSupport Fragmented TransactionsSupport Fragmented TransactionsIf true, when a FlowFile is consumed by this Processor, the Processor will first check the fragment.identifier and fragment.count attributes of that FlowFile. If the fragment.count value is greater than 1, the Processor will not process any FlowFile with that fragment.identifier until all are available; at that point, it will process all FlowFiles with that fragment.identifier as a single transaction, in the order specified by the FlowFiles' fragment.index attributes. This Provides atomicity of those SQL statements. Once any statement of this transaction throws exception when executing, this transaction will be rolled back. When transaction rollback happened, none of these FlowFiles would be routed to 'success'. If the <Rollback On Failure> is set true, these FlowFiles will stay in the input relationship. When the <Rollback On Failure> is set false,, if any of these FlowFiles will be routed to 'retry', all of these FlowFiles will be routed to 'retry'.Otherwise, they will be routed to 'failure'. If this value is false, these attributes will be ignored and the updates will occur independent of one another.truetruetruefalsefalsefalsefalsefalseNONEfalsefalsedatabase-session-autocommitDatabase Session AutoCommitThe autocommit mode to set on the database connection being used. If set to false, the operation(s) will be explicitly committed or rolled back (based on success or failure respectively), if set to true the driver/database handles the commit/rollback.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseTransaction TimeoutTransaction TimeoutIf the <Support Fragmented Transactions> property is set to true, specifies how long to wait for all FlowFiles for a particular fragment.identifier attribute to arrive before just transferring all of the FlowFiles with that identifier to the 'failure' relationshipfalsefalsefalseNONEfalsefalseBatch SizeBatch SizeThe preferred number of FlowFiles to put to the database in a single transaction100truefalsefalseNONEfalsefalseObtain Generated KeysObtain Generated KeysIf true, any key that is automatically generated by the database will be added to the FlowFile that generated it using the sql.generate.key attribute. This may result in slightly slower performance and is not supported by all databases.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalserollback-on-failureRollback On FailureSpecify how to handle error. By default (false), if an error occurs while processing a FlowFile, the FlowFile will be routed to 'failure' or 'retry' relationship based on error type, and processor can continue with next FlowFile. Instead, you may want to rollback currently processed FlowFiles and stop further processing immediately. In that case, you can do so by enabling this 'Rollback On Failure' property. If enabled, failed FlowFiles will stay in the input relationship without penalizing it and being processed repeatedly until it gets processed successfully or removed by other means. It is important to set adequate 'Yield Duration' to avoid retrying too frequently.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseretryA FlowFile is routed to this relationship if the database cannot be updated but attempting the operation again may succeedfalsesuccessA FlowFile is routed to this relationship after the database is successfully updatedfalsefailureA FlowFile is routed to this relationship if the database cannot be updated and retrying the operation will also fail, such as an invalid query or an integrity constraint violationfalsefragment.identifierIf the <Support Fragment Transactions> property is true, this attribute is used to determine whether or not two FlowFiles belong to the same transaction.fragment.countIf the <Support Fragment Transactions> property is true, this attribute is used to determine how many FlowFiles are needed to complete the transaction.fragment.indexIf the <Support Fragment Transactions> property is true, this attribute is used to determine the order that the FlowFiles in a transaction should be evaluated.sql.args.N.typeIncoming FlowFiles are expected to be parametrized SQL statements. The type of each Parameter is specified as an integer that represents the JDBC Type of the parameter.sql.args.N.valueIncoming FlowFiles are expected to be parametrized SQL statements. The value of the Parameters are specified as sql.args.1.value, sql.args.2.value, sql.args.3.value, and so on. The type of the sql.args.1.value Parameter is specified by the sql.args.1.type attribute.sql.args.N.formatThis attribute is always optional, but default options may not always work for your data. Incoming FlowFiles are expected to be parametrized SQL statements. In some cases a format option needs to be specified, currently this is only applicable for binary data types, dates, times and timestamps. Binary Data Types (defaults to 'ascii') - ascii: each string character in your attribute value represents a single byte. This is the format provided by Avro Processors. base64: the string is a Base64 encoded string that can be decoded to bytes. hex: the string is hex encoded with all letters in upper case and no '0x' at the beginning. Dates/Times/Timestamps - Date, Time and Timestamp formats all support both custom formats or named format ('yyyy-MM-dd','ISO_OFFSET_DATE_TIME') as specified according to java.time.format.DateTimeFormatter. If not specified, a long value input is expected to be an unix epoch (milli seconds from 1970/1/1), or a string value in 'yyyy-MM-dd' format for Date, 'HH:mm:ss.SSS' for Time (some database engines e.g. Derby or MySQL do not support milliseconds and will truncate milliseconds), 'yyyy-MM-dd HH:mm:ss.SSS' for Timestamp is used.sql.generated.keyIf the database generated a key for an INSERT statement and the Obtain Generated Keys property is set to true, this attribute will be added to indicate the generated key, if possible. This feature is not supported by all database vendors.trueINPUT_REQUIREDorg.apache.nifi.processors.standard.ConvertJSONToSQLorg.apache.nifi.processors.standard.PutSyslogPROCESSORSends Syslog messages to a given host and port over TCP or UDP. Messages are constructed from the "Message ___" properties of the processor which can use expression language to generate messages from incoming FlowFiles. The properties are used to construct messages of the form: (<PRIORITY>)(VERSION )(TIMESTAMP) (HOSTNAME) (BODY) where version is optional. The constructed messages are checked against regular expressions for RFC5424 and RFC3164 formatted messages. The timestamp can be an RFC5424 timestamp with a format of "yyyy-MM-dd'T'HH:mm:ss.SZ" or "yyyy-MM-dd'T'HH:mm:ss.S+hh:mm", or it can be an RFC3164 timestamp with a format of "MMM d HH:mm:ss". If a message is constructed that does not form a valid Syslog message according to the above description, then it is routed to the invalid relationship. Valid messages are sent to the Syslog server and successes are routed to the success relationship, failures routed to the failure relationship.syslogputudptcplogsHostnameHostnameThe IP address or hostname of the Syslog server.localhosttruefalsetrueENVIRONMENTfalsefalseProtocolProtocolThe protocol for Syslog communication.UDPTCPTCPUDPUDPtruefalsefalseNONEfalsefalsePortPortThe port for Syslog communication. Note that Expression language is not evaluated per FlowFile.truefalsetrueENVIRONMENTfalsefalseMax Size of Socket Send BufferMax Size of Socket Send BufferThe maximum size of the socket send buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsetrueENVIRONMENTfalsefalseSSL Context ServiceSSL Context ServiceThe Controller Service to use in order to obtain an SSL Context. If this property is set, syslog messages will be sent over a secure connection.org.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseProtocolProtocolTCPIdle Connection ExpirationIdle Connection ExpirationThe amount of time a connection should be held open without being used before closing the connection.5 secondstruefalsetrueENVIRONMENTfalsefalseTimeoutTimeoutThe timeout for connecting to and communicating with the syslog server. Does not apply to UDP. Note that Expression language is not evaluated per FlowFile.10 secondsfalsefalsetrueENVIRONMENTfalsefalseBatch SizeBatch SizeThe number of incoming FlowFiles to process in a single execution of this processor.25truefalsetrueENVIRONMENTfalsefalseCharacter SetCharacter SetSpecifies the character set of the Syslog messages. Note that Expression language is not evaluated per FlowFile.UTF-8truefalsetrueENVIRONMENTfalsefalseMessage PriorityMessage PriorityThe priority for the Syslog messages, excluding < >.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessage VersionMessage VersionThe version for the Syslog messages.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessage TimestampMessage TimestampThe timestamp for the Syslog messages. The timestamp can be an RFC5424 timestamp with a format of "yyyy-MM-dd'T'HH:mm:ss.SZ" or "yyyy-MM-dd'T'HH:mm:ss.S+hh:mm", " or it can be an RFC3164 timestamp with a format of "MMM d HH:mm:ss".${now():format('MMM d HH:mm:ss')}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessage HostnameMessage HostnameThe hostname for the Syslog messages.${hostname(true)}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMessage BodyMessage BodyThe body for the Syslog messages.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessFlowFiles that are sent successfully to Syslog are sent out this relationship.falsefailureFlowFiles that failed to send to Syslog are sent out this relationship.falseinvalidFlowFiles that do not form a valid Syslog message are sent out this relationship.falsetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ListenSyslogorg.apache.nifi.processors.standard.ParseSyslogorg.apache.nifi.processors.standard.PutTCPPROCESSORSends serialized FlowFiles or Records over TCP to a configurable destination with optional support for TLSremoteegressputtcpHostnameHostnameDestination hostname or IP addresslocalhosttruefalsetrueENVIRONMENTfalsefalsePortPortDestination port numbertruefalsetrueENVIRONMENTfalsefalseMax Size of Socket Send BufferMax Size of Socket Send BufferThe maximum size of the socket send buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseIdle Connection ExpirationIdle Connection ExpirationThe amount of time a connection should be held open without being used before closing the connection. A value of 0 seconds will disable this feature.15 secondstruefalsetrueENVIRONMENTfalsefalseTimeoutTimeoutThe timeout for connecting to and communicating with the destination. Does not apply to UDP10 secondsfalsefalsetrueENVIRONMENTfalsefalseConnection Per FlowFileConnection Per FlowFileSpecifies whether to send each FlowFile's content on an individual connection.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseSSL Context ServiceSSL Context ServiceSpecifies the SSL Context Service to enable TLS socket communicationorg.apache.nifi.ssl.SSLContextServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseTransmission StrategyTransmission StrategySpecifies the strategy used for reading input FlowFiles and transmitting messages to the destination socket addressFLOWFILE_ORIENTEDFlowFile-orientedFLOWFILE_ORIENTEDSend FlowFile content as a single streamRecord-orientedRECORD_ORIENTEDRead Records from input FlowFiles and send serialized Records as individual messagestruefalsefalseNONEfalsefalseOutgoing Message DelimiterOutgoing Message DelimiterSpecifies the delimiter to use when sending messages out over the same TCP stream. The delimiter is appended to each FlowFile message that is transmitted over the stream so that the receiver can determine when one message ends and the next message begins. Users should ensure that the FlowFile content does not contain the delimiter character to avoid errors. In order to use a new line character you can enter '\n'. For a tab character use '\t'. Finally for a carriage return use '\r'.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseTransmission StrategyTransmission StrategyFLOWFILE_ORIENTEDCharacter SetCharacter SetSpecifies the character set of the data being sent.UTF-8truefalsetrueENVIRONMENTfalsefalseTransmission StrategyTransmission StrategyFLOWFILE_ORIENTEDRecord ReaderRecord ReaderSpecifies the Controller Service to use for reading Records from input FlowFilesorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseTransmission StrategyTransmission StrategyRECORD_ORIENTEDRecord WriterRecord WriterSpecifies the Controller Service to use for writing Records to the configured socket addressorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseTransmission StrategyTransmission StrategyRECORD_ORIENTEDsuccessFlowFiles that are sent successfully to the destination are sent out this relationship.falsefailureFlowFiles that failed to send to the destination are sent out this relationship.falserecord.count.transmittedCount of records transmitted to configured destination addresstrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ListenTCPorg.apache.nifi.processors.standard.PutUDPorg.apache.nifi.processors.standard.PutUDPPROCESSORThe PutUDP processor receives a FlowFile and packages the FlowFile content into a single UDP datagram packet which is then transmitted to the configured UDP server. The user must ensure that the FlowFile content being fed to this processor is not larger than the maximum size for the underlying UDP transport. The maximum transport size will vary based on the platform setup but is generally just under 64KB. FlowFiles will be marked as failed if their content is larger than the maximum transport size.remoteegressputudpHostnameHostnameDestination hostname or IP addresslocalhosttruefalsetrueENVIRONMENTfalsefalsePortPortDestination port numbertruefalsetrueENVIRONMENTfalsefalseMax Size of Socket Send BufferMax Size of Socket Send BufferThe maximum size of the socket send buffer that should be used. This is a suggestion to the Operating System to indicate how big the socket buffer should be. If this value is set too low, the buffer may fill up before the data can be read, and incoming data will be dropped.1 MBtruefalsefalseNONEfalsefalseIdle Connection ExpirationIdle Connection ExpirationThe amount of time a connection should be held open without being used before closing the connection. A value of 0 seconds will disable this feature.15 secondstruefalsetrueENVIRONMENTfalsefalseTimeoutTimeoutThe timeout for connecting to and communicating with the destination. Does not apply to UDP10 secondsfalsefalsetrueENVIRONMENTfalsefalsesuccessFlowFiles that are sent successfully to the destination are sent out this relationship.falsefailureFlowFiles that failed to send to the destination are sent out this relationship.falsetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ListenUDPorg.apache.nifi.processors.standard.PutTCPorg.apache.nifi.processors.standard.QueryDatabaseTablePROCESSORGenerates a SQL select query, or uses a provided statement, and executes it to fetch all rows whose values in the specified Maximum Value column(s) are larger than the previously-seen maxima. Query result will be converted to Avro format. Expression Language is supported for several properties, but no incoming connections are permitted. The Environment/System properties may be used to provide values for any property containing Expression Language. If it is desired to leverage flow file attributes to perform these queries, the GenerateTableFetch and/or ExecuteSQL processors can be used for this purpose. Streaming is used so arbitrarily large result sets are supported. This processor can be scheduled to run on a timer or cron expression, using the standard scheduling methods. This processor is intended to be run on the Primary Node only. FlowFile attribute 'querydbtable.row.count' indicates how many rows were selected.sqlselectjdbcquerydatabaseDatabase Connection Pooling ServiceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-fetch-db-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLtruefalsefalseNONEfalsefalseTable NameTable NameThe name of the database table to be queried. When a custom query is used, this property is used to alias the query and appears as an attribute on the FlowFile.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseColumns to ReturnColumns to ReturnA comma-separated list of column names to be used in the query. If your database requires special treatment of the names (quoting, e.g.), each name should include such treatment. If no column names are supplied, all columns in the specified table will be returned. NOTE: It is important to use consistent column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-where-clauseAdditional WHERE clauseA custom clause to be added in the WHERE condition when building SQL queries.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-sql-queryCustom QueryA custom SQL query used to retrieve data. Instead of building a SQL query from other properties, this query will be wrapped as a sub-query. Query must have no ORDER BY statement.falsefalsetrueENVIRONMENTfalsefalseMaximum-value ColumnsMaximum-value ColumnsA comma-separated list of column names. The processor will keep track of the maximum value for each column that has been returned since the processor started running. Using multiple columns implies an order to the column list, and each column's values are expected to increase more slowly than the previous columns' values. Thus, using multiple columns implies a hierarchical structure of columns, which is usually used for partitioning tables. This processor can be used to retrieve only those rows that have been added/updated since the last retrieval. Note that some JDBC types such as bit/boolean are not conducive to maintaining maximum value, so columns of these types should not be listed in this property, and will result in error(s) during processing. If no columns are provided, all rows from the table will be considered, which could have a performance impact. NOTE: It is important to use consistent max-value column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseinitial-load-strategyInitial Load StrategyHow to handle existing rows in the database table when the processor is started for the first time (or its state has been cleared). The property will be ignored, if any 'initial.maxvalue.*' dynamic property has also been configured.Start at BeginningStart at BeginningStart at BeginningLoads all existing rows from the database table.Start at Current Maximum ValuesStart at Current Maximum ValuesLoads only the newly inserted or updated rows based on the maximum value(s) of the column(s) configured in the 'Maximum-value Columns' property.truefalsefalseNONEfalsefalseMax Wait TimeMax Wait TimeThe maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFetch SizeFetch SizeThe number of result rows to be fetched from the result set at a time. This is a hint to the database driver and may not be honored and/or exact. If the value specified is zero, then the hint is ignored. If using PostgreSQL, then 'Set Auto Commit' must be equal to 'false' to cause 'Fetch Size' to take effect.0truefalsetrueENVIRONMENTfalsefalseSet Auto CommitSet Auto CommitAllows enabling or disabling the auto commit functionality of the DB connection. Default value is 'No value set'. 'No value set' will leave the db connection's auto commit mode unchanged. For some JDBC drivers such as PostgreSQL driver, it is required to disable the auto commit functionality to get the 'Fetch Size' setting to take effect. When auto commit is enabled, PostgreSQL driver ignores 'Fetch Size' setting and loads all rows of the result set to memory at once. This could lead for a large amount of memory usage when executing queries which fetch large data sets. More Details of this behaviour in PostgreSQL driver can be found in https://jdbc.postgresql.org//documentation/head/query.html.truetruefalsefalsefalsefalsetrueENVIRONMENTfalsefalseqdbt-max-rowsMax Rows Per Flow FileThe maximum number of result rows that will be included in a single FlowFile. This will allow you to break up very large result sets into multiple FlowFiles. If the value specified is zero, then all rows are returned in a single FlowFile.0truefalsetrueENVIRONMENTfalsefalseqdbt-output-batch-sizeOutput Batch SizeThe number of output FlowFiles to queue before committing the process session. When set to zero, the session will be committed when all result set rows have been processed and the output FlowFiles are ready for transfer to the downstream relationship. For large result sets, this can cause a large burst of FlowFiles to be transferred at the end of processor execution. If this property is set, then when the specified number of FlowFiles are ready for transfer, then the session will be committed, thus releasing the FlowFiles to the downstream relationship. NOTE: The maxvalue.* and fragment.count attributes will not be set on FlowFiles when this property is set.0truefalsetrueENVIRONMENTfalsefalseqdbt-max-fragsMaximum Number of FragmentsThe maximum number of fragments. If the value specified is zero, then all fragments are returned. This prevents OutOfMemoryError when this processor ingests huge table. NOTE: Setting this property can result in data loss, as the incoming results are not ordered, and fragments may end at arbitrary boundaries where rows are not included in the result set.0truefalsetrueENVIRONMENTfalsefalsedbf-normalizeNormalize Table/Column NamesWhether to change non-Avro-compatible characters in column names to Avro-compatible characters. For example, colons and periods will be changed to underscores in order to build a valid Avro record.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsetransaction-isolation-levelTransaction Isolation LevelThis setting will set the transaction isolation level for the database connection for drivers that support this settingTRANSACTION_NONE0TRANSACTION_READ_COMMITTED2TRANSACTION_READ_UNCOMMITTED1TRANSACTION_REPEATABLE_READ4TRANSACTION_SERIALIZABLE8falsefalsefalseNONEfalsefalsedbf-user-logical-typesUse Avro Logical TypesWhether to use Avro Logical Types for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP columns. If disabled, written as string. If enabled, Logical types are used and written as its underlying type, specifically, DECIMAL/NUMBER as logical 'decimal': written as bytes with additional precision and scale meta data, DATE as logical 'date-millis': written as int denoting days since Unix epoch (1970-01-01), TIME as logical 'time-millis': written as int denoting milliseconds since Unix epoch, and TIMESTAMP as logical 'timestamp-millis': written as long denoting milliseconds since Unix epoch. If a reader of written Avro records also knows these logical types, then these values can be deserialized with more context depending on reader implementation.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-default-precisionDefault Decimal PrecisionWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'precision' denoting number of available digits is required. Generally, precision is defined by column data type definition or database engines default. However undefined precision (0) can be returned from some database engines. 'Default Decimal Precision' is used when writing those undefined precision numbers.10truefalsetrueENVIRONMENTfalsefalsedbf-default-scaleDefault Decimal ScaleWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'scale' denoting number of available decimal digits is required. Generally, scale is defined by column data type definition or database engines default. However when undefined precision (0) is returned, scale can also be uncertain with some database engines. 'Default Decimal Scale' is used when writing those undefined numbers. If a value has more decimals than specified scale, then the value will be rounded-up, e.g. 1.53 becomes 2 with scale 0, and 1.5 with scale 1.0truefalsetrueENVIRONMENTfalsefalseinitial.maxvalue.<max_value_column>Initial maximum value for the specified columnSpecifies an initial max value for max value column(s). Properties should be added in the format `initial.maxvalue.<max_value_column>`. This value is only used the first time the table is accessed (when a Maximum Value Column is specified).ENVIRONMENTsuccessSuccessfully created FlowFile from SQL query result set.falsetablenameName of the table being queriedquerydbtable.row.countThe number of rows selected by the queryfragment.identifierIf 'Max Rows Per Flow File' is set then all FlowFiles from the same query result set will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.fragment.countIf 'Max Rows Per Flow File' is set then this is the total number of FlowFiles produced by a single ResultSet. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming ResultSet. If Output Batch Size is set, then this attribute will not be populated.fragment.indexIf 'Max Rows Per Flow File' is set then the position of this FlowFile in the list of outgoing FlowFiles that were all derived from the same result set FlowFile. This can be used in conjunction with the fragment.identifier attribute to know which FlowFiles originated from the same query result set and in what order FlowFiles were producedmaxvalue.*Each attribute contains the observed maximum value of a specified 'Maximum-value Column'. The suffix of the attribute is the name of the column. If Output Batch Size is set, then this attribute will not be populated.truetrueAfter performing a query on the specified table, the maximum values for the specified column(s) will be retained for use in future executions of the query. This allows the Processor to fetch only those records that have max values greater than the retained values. This can be used for incremental fetching, fetching of newly added rows, etc. To clear the maximum values, clear the state of the processor per the State Management documentationCLUSTERINPUT_FORBIDDENorg.apache.nifi.processors.standard.GenerateTableFetchorg.apache.nifi.processors.standard.ExecuteSQLTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.QueryDatabaseTableRecordPROCESSORGenerates a SQL select query, or uses a provided statement, and executes it to fetch all rows whose values in the specified Maximum Value column(s) are larger than the previously-seen maxima. Query result will be converted to the format specified by the record writer. Expression Language is supported for several properties, but no incoming connections are permitted. The Environment/System properties may be used to provide values for any property containing Expression Language. If it is desired to leverage flow file attributes to perform these queries, the GenerateTableFetch and/or ExecuteSQL processors can be used for this purpose. Streaming is used so arbitrarily large result sets are supported. This processor can be scheduled to run on a timer or cron expression, using the standard scheduling methods. This processor is intended to be run on the Primary Node only. FlowFile attribute 'querydbtable.row.count' indicates how many rows were selected.sqlselectjdbcquerydatabaserecordDatabase Connection Pooling ServiceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-fetch-db-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLtruefalsefalseNONEfalsefalseTable NameTable NameThe name of the database table to be queried. When a custom query is used, this property is used to alias the query and appears as an attribute on the FlowFile.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseColumns to ReturnColumns to ReturnA comma-separated list of column names to be used in the query. If your database requires special treatment of the names (quoting, e.g.), each name should include such treatment. If no column names are supplied, all columns in the specified table will be returned. NOTE: It is important to use consistent column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-where-clauseAdditional WHERE clauseA custom clause to be added in the WHERE condition when building SQL queries.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedb-fetch-sql-queryCustom QueryA custom SQL query used to retrieve data. Instead of building a SQL query from other properties, this query will be wrapped as a sub-query. Query must have no ORDER BY statement.falsefalsetrueENVIRONMENTfalsefalseqdbtr-record-writerRecord WriterSpecifies the Controller Service to use for writing results to a FlowFile. The Record Writer may use Inherit Schema to emulate the inferred schema behavior, i.e. an explicit schema need not be defined in the writer, and will be supplied by the same logic used to infer the schema from the column types.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseMaximum-value ColumnsMaximum-value ColumnsA comma-separated list of column names. The processor will keep track of the maximum value for each column that has been returned since the processor started running. Using multiple columns implies an order to the column list, and each column's values are expected to increase more slowly than the previous columns' values. Thus, using multiple columns implies a hierarchical structure of columns, which is usually used for partitioning tables. This processor can be used to retrieve only those rows that have been added/updated since the last retrieval. Note that some JDBC types such as bit/boolean are not conducive to maintaining maximum value, so columns of these types should not be listed in this property, and will result in error(s) during processing. If no columns are provided, all rows from the table will be considered, which could have a performance impact. NOTE: It is important to use consistent max-value column names for a given table for incremental fetch to work properly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseinitial-load-strategyInitial Load StrategyHow to handle existing rows in the database table when the processor is started for the first time (or its state has been cleared). The property will be ignored, if any 'initial.maxvalue.*' dynamic property has also been configured.Start at BeginningStart at BeginningStart at BeginningLoads all existing rows from the database table.Start at Current Maximum ValuesStart at Current Maximum ValuesLoads only the newly inserted or updated rows based on the maximum value(s) of the column(s) configured in the 'Maximum-value Columns' property.truefalsefalseNONEfalsefalseMax Wait TimeMax Wait TimeThe maximum amount of time allowed for a running SQL select query , zero means there is no limit. Max time less than 1 second will be equal to zero.0 secondstruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseFetch SizeFetch SizeThe number of result rows to be fetched from the result set at a time. This is a hint to the database driver and may not be honored and/or exact. If the value specified is zero, then the hint is ignored. If using PostgreSQL, then 'Set Auto Commit' must be equal to 'false' to cause 'Fetch Size' to take effect.0truefalsetrueENVIRONMENTfalsefalseSet Auto CommitSet Auto CommitAllows enabling or disabling the auto commit functionality of the DB connection. Default value is 'No value set'. 'No value set' will leave the db connection's auto commit mode unchanged. For some JDBC drivers such as PostgreSQL driver, it is required to disable the auto commit functionality to get the 'Fetch Size' setting to take effect. When auto commit is enabled, PostgreSQL driver ignores 'Fetch Size' setting and loads all rows of the result set to memory at once. This could lead for a large amount of memory usage when executing queries which fetch large data sets. More Details of this behaviour in PostgreSQL driver can be found in https://jdbc.postgresql.org//documentation/head/query.html.truetruefalsefalsefalsefalsetrueENVIRONMENTfalsefalseqdbt-max-rowsMax Rows Per Flow FileThe maximum number of result rows that will be included in a single FlowFile. This will allow you to break up very large result sets into multiple FlowFiles. If the value specified is zero, then all rows are returned in a single FlowFile.0truefalsetrueENVIRONMENTfalsefalseqdbt-output-batch-sizeOutput Batch SizeThe number of output FlowFiles to queue before committing the process session. When set to zero, the session will be committed when all result set rows have been processed and the output FlowFiles are ready for transfer to the downstream relationship. For large result sets, this can cause a large burst of FlowFiles to be transferred at the end of processor execution. If this property is set, then when the specified number of FlowFiles are ready for transfer, then the session will be committed, thus releasing the FlowFiles to the downstream relationship. NOTE: The maxvalue.* and fragment.count attributes will not be set on FlowFiles when this property is set.0truefalsetrueENVIRONMENTfalsefalseqdbt-max-fragsMaximum Number of FragmentsThe maximum number of fragments. If the value specified is zero, then all fragments are returned. This prevents OutOfMemoryError when this processor ingests huge table. NOTE: Setting this property can result in data loss, as the incoming results are not ordered, and fragments may end at arbitrary boundaries where rows are not included in the result set.0truefalsetrueENVIRONMENTfalsefalseqdbtr-normalizeNormalize Table/Column NamesWhether to change characters in column names when creating the output schema. For example, colons and periods will be changed to underscores.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-user-logical-typesUse Avro Logical TypesWhether to use Avro Logical Types for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP columns. If disabled, written as string. If enabled, Logical types are used and written as its underlying type, specifically, DECIMAL/NUMBER as logical 'decimal': written as bytes with additional precision and scale meta data, DATE as logical 'date-millis': written as int denoting days since Unix epoch (1970-01-01), TIME as logical 'time-millis': written as int denoting milliseconds since Unix epoch, and TIMESTAMP as logical 'timestamp-millis': written as long denoting milliseconds since Unix epoch. If a reader of written Avro records also knows these logical types, then these values can be deserialized with more context depending on reader implementation.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-default-precisionDefault Decimal PrecisionWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'precision' denoting number of available digits is required. Generally, precision is defined by column data type definition or database engines default. However undefined precision (0) can be returned from some database engines. 'Default Decimal Precision' is used when writing those undefined precision numbers.10truefalsetrueENVIRONMENTfalsefalsedbf-default-scaleDefault Decimal ScaleWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'scale' denoting number of available decimal digits is required. Generally, scale is defined by column data type definition or database engines default. However when undefined precision (0) is returned, scale can also be uncertain with some database engines. 'Default Decimal Scale' is used when writing those undefined numbers. If a value has more decimals than specified scale, then the value will be rounded-up, e.g. 1.53 becomes 2 with scale 0, and 1.5 with scale 1.0truefalsetrueENVIRONMENTfalsefalseinitial.maxvalue.<max_value_column>Initial maximum value for the specified columnSpecifies an initial max value for max value column(s). Properties should be added in the format `initial.maxvalue.<max_value_column>`. This value is only used the first time the table is accessed (when a Maximum Value Column is specified).ENVIRONMENTsuccessSuccessfully created FlowFile from SQL query result set.falsetablenameName of the table being queriedquerydbtable.row.countThe number of rows selected by the queryfragment.identifierIf 'Max Rows Per Flow File' is set then all FlowFiles from the same query result set will have the same value for the fragment.identifier attribute. This can then be used to correlate the results.fragment.countIf 'Max Rows Per Flow File' is set then this is the total number of FlowFiles produced by a single ResultSet. This can be used in conjunction with the fragment.identifier attribute in order to know how many FlowFiles belonged to the same incoming ResultSet. If Output Batch Size is set, then this attribute will not be populated.fragment.indexIf 'Max Rows Per Flow File' is set then the position of this FlowFile in the list of outgoing FlowFiles that were all derived from the same result set FlowFile. This can be used in conjunction with the fragment.identifier attribute to know which FlowFiles originated from the same query result set and in what order FlowFiles were producedmaxvalue.*Each attribute contains the observed maximum value of a specified 'Maximum-value Column'. The suffix of the attribute is the name of the column. If Output Batch Size is set, then this attribute will not be populated.mime.typeSets the mime.type attribute to the MIME Type specified by the Record Writer.record.countThe number of records output by the Record Writer.truetrueAfter performing a query on the specified table, the maximum values for the specified column(s) will be retained for use in future executions of the query. This allows the Processor to fetch only those records that have max values greater than the retained values. This can be used for incremental fetching, fetching of newly added rows, etc. To clear the maximum values, clear the state of the processor per the State Management documentationCLUSTERINPUT_FORBIDDENRetrieve all rows from a database table.jdbcrdbmscdcdatabasetablestreamINPUT_ALLOWEDConfigure the "Database Connection Pooling Service" to specify a Connection Pooling Service so that the Processor knows how to connect to the database. Set the "Database Type" property to the type of database to query, or "Generic" if the database vendor is not listed. Set the "Table Name" property to the name of the table to retrieve records from. Configure the "Record Writer" to specify a Record Writer that is appropriate for the desired output format. Set the "Maximum-value Columns" property to a comma-separated list of columns whose values can be used to determine which values are new. For example, this might be set to an `id` column that is a one-up number, or a `last_modified` column that is a timestamp of when the row was last modified. Set the "Initial Load Strategy" property to "Start at Beginning". Set the "Fetch Size" to a number that avoids loading too much data into memory on the NiFi side. For example, a value of `1000` will load up to 1,000 rows of data. Set the "Max Rows Per Flow File" to a value that allows efficient processing, such as `1000` or `10000`. Set the "Output Batch Size" property to a value greater than `0`. A smaller value, such as `1` or even `20` will result in lower latency but also slightly lower throughput. A larger value such as `1000` will result in higher throughput but also higher latency. It is not recommended to set the value larger than `1000` as it can cause significant memory utilization. Perform an incremental load of a single database table, fetching only new rows as they are added to the table.incremental loadrdbmsjdbccdcdatabasetablestreamINPUT_ALLOWEDConfigure the "Database Connection Pooling Service" to specify a Connection Pooling Service so that the Processor knows how to connect to the database. Set the "Database Type" property to the type of database to query, or "Generic" if the database vendor is not listed. Set the "Table Name" property to the name of the table to retrieve records from. Configure the "Record Writer" to specify a Record Writer that is appropriate for the desired output format. Set the "Maximum-value Columns" property to a comma-separated list of columns whose values can be used to determine which values are new. For example, this might be set to an `id` column that is a one-up number, or a `last_modified` column that is a timestamp of when the row was last modified. Set the "Initial Load Strategy" property to "Start at Current Maximum Values". Set the "Fetch Size" to a number that avoids loading too much data into memory on the NiFi side. For example, a value of `1000` will load up to 1,000 rows of data. Set the "Max Rows Per Flow File" to a value that allows efficient processing, such as `1000` or `10000`. Set the "Output Batch Size" property to a value greater than `0`. A smaller value, such as `1` or even `20` will result in lower latency but also slightly lower throughput. A larger value such as `1000` will result in higher throughput but also higher latency. It is not recommended to set the value larger than `1000` as it can cause significant memory utilization. Perform an incremental load of multiple database tables, fetching only new rows as they are added to the tables.incremental loadrdbmsjdbccdcdatabasetablestreamorg.apache.nifi.processors.standard.ListDatabaseTablesConfigure the "Database Connection Pooling Service" property to specify a Connection Pool that is applicable for interacting with your database. Set the "Catalog" property to the name of the database Catalog; set the "Schema Pattern" property to a Java Regular Expression that matches all database Schemas that should be included; and set the "Table Name Pattern" property to a Java Regular Expression that matches the names of all tables that should be included. In order to perform an incremental load of all tables, leave the Catalog, Schema Pattern, and Table Name Pattern unset. Leave the RecordWriter property unset. Connect the 'success' relationship to QueryDatabaseTableRecord. org.apache.nifi.processors.standard.QueryDatabaseTableRecordConfigure the "Database Connection Pooling Service" to the same Connection Pool that was used in ListDatabaseTables. Set the "Database Type" property to the type of database to query, or "Generic" if the database vendor is not listed. Set the "Table Name" property to "${db.table.fullname}" Configure the "Record Writer" to specify a Record Writer that is appropriate for the desired output format. Set the "Maximum-value Columns" property to a comma-separated list of columns whose values can be used to determine which values are new. For example, this might be set to an `id` column that is a one-up number, or a `last_modified` column that is a timestamp of when the row was last modified. Set the "Initial Load Strategy" property to "Start at Current Maximum Values". Set the "Fetch Size" to a number that avoids loading too much data into memory on the NiFi side. For example, a value of `1000` will load up to 1,000 rows of data. Set the "Max Rows Per Flow File" to a value that allows efficient processing, such as `1000` or `10000`. Set the "Output Batch Size" property to a value greater than `0`. A smaller value, such as `1` or even `20` will result in lower latency but also slightly lower throughput. A larger value such as `1000` will result in higher throughput but also higher latency. It is not recommended to set the value larger than `1000` as it can cause significant memory utilization. org.apache.nifi.processors.standard.GenerateTableFetchorg.apache.nifi.processors.standard.ExecuteSQLTIMER_DRIVEN1 min1org.apache.nifi.processors.standard.QueryRecordPROCESSOREvaluates one or more SQL queries against the contents of a FlowFile. The result of the SQL query then becomes the content of the output FlowFile. This can be used, for example, for field-specific filtering, transformation, and row-level filtering. Columns can be renamed, simple calculations and aggregations performed, etc. The Processor is configured with a Record Reader Controller Service and a Record Writer service so as to allow flexibility in incoming and outgoing data formats. The Processor must be configured with at least one user-defined property. The name of the Property is the Relationship to route data to, and the value of the Property is a SQL SELECT statement that is used to specify how input data should be transformed/filtered. The SQL statement must be valid ANSI SQL and is powered by Apache Calcite. If the transformation fails, the original FlowFile is routed to the 'failure' relationship. Otherwise, the data selected will be routed to the associated relationship. If the Record Writer chooses to inherit the schema from the Record, it is important to note that the schema that is inherited will be from the ResultSet, rather than the input Record. This allows a single instance of the QueryRecord processor to have multiple queries, each of which returns a different set of columns and aggregations. As a result, though, the schema that is derived will have no schema name, so it is important that the configured Record Writer not attempt to write the Schema Name as an attribute if inheriting the Schema from the Record. See the Processor Usage documentation for more information.sqlquerycalciterouterecordtransformselectupdatemodifyetlfilterrecordcsvjsonlogstextavroaggregaterecord-readerRecord ReaderSpecifies the Controller Service to use for parsing incoming data and determining the data's schemaorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Controller Service to use for writing results to a FlowFileorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseinclude-zero-record-flowfilesInclude Zero Record FlowFilesWhen running the SQL statement against an incoming FlowFile, if the result has no data, this property specifies whether or not a FlowFile will be sent to the corresponding relationshiptruetruetruefalsefalsetruefalsefalseNONEfalsefalsecache-schemaCache SchemaThis property is no longer used. It remains solely for backward compatibility in order to avoid making existing Processors invalid upon upgrade. This property will be removed in future versions. Now, instead of forcing the user to understand the semantics of schema caching, the Processor caches up to 25 schemas and automatically rolls off the old schemas. This provides the same performance when caching was enabled previously and in some cases very significant performance improvements if caching was previously disabled.truetruetruefalsefalsetruefalsefalseNONEfalsefalsedbf-default-precisionDefault Decimal PrecisionWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'precision' denoting number of available digits is required. Generally, precision is defined by column data type definition or database engines default. However undefined precision (0) can be returned from some database engines. 'Default Decimal Precision' is used when writing those undefined precision numbers.10truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedbf-default-scaleDefault Decimal ScaleWhen a DECIMAL/NUMBER value is written as a 'decimal' Avro logical type, a specific 'scale' denoting number of available decimal digits is required. Generally, scale is defined by column data type definition or database engines default. However when undefined precision (0) is returned, scale can also be uncertain with some database engines. 'Default Decimal Scale' is used when writing those undefined numbers. If a value has more decimals than specified scale, then the value will be rounded-up, e.g. 1.53 becomes 2 with scale 0, and 1.5 with scale 1.0truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseThe name of the relationship to route data toA SQL SELECT statement that is used to determine what data should be routed to this relationship.Each user-defined property specifies a SQL SELECT statement to run over the data, with the data that is selected being routed to the relationship whose name is the property nameFLOWFILE_ATTRIBUTESfailureIf a FlowFile fails processing for any reason (for example, the SQL statement contains columns not present in input data), the original FlowFile it will be routed to this relationshipfalseoriginalThe original FlowFile is routed to this relationshipfalse<Property Name>Each user-defined property defines a new Relationship for this Processor.mime.typeSets the mime.type attribute to the MIME Type specified by the Record Writerrecord.countThe number of records selected by the queryQueryRecord.RouteThe relation to which the FlowFile was routedtruetrueINPUT_REQUIREDFilter out records based on the values of the records' fieldsfilter outremovedropstrip outrecord fieldsqlINPUT_ALLOWED"Record Reader" should be set to a Record Reader that is appropriate for your data. "Record Writer" should be set to a Record Writer that writes out data in the desired format. One additional property should be added. The name of the property should be a short description of the data to keep. Its value is a SQL statement that selects all columns from a table named `FLOW_FILE` for relevant rows. The WHERE clause selects the data to keep. I.e., it is the exact opposite of what we want to remove. It is recommended to always quote column names using double-quotes in order to avoid conflicts with SQL keywords. For example, to remove records where either the name is George OR the age is less than 18, we would add a property named "adults not george" with a value that selects records where the name is not George AND the age is greater than or equal to 18. So the value would be `SELECT * FROM FLOWFILE WHERE "name" <> 'George' AND "age" >= 18` Adding this property now gives us a new Relationship whose name is the same as the property name. So, the "adults not george" Relationship should be connected to the next Processor in our flow. Keep only specific recordskeepfilterretainselectincluderecordsqlINPUT_ALLOWED"Record Reader" should be set to a Record Reader that is appropriate for your data. "Record Writer" should be set to a Record Writer that writes out data in the desired format. One additional property should be added. The name of the property should be a short description of the data to keep. Its value is a SQL statement that selects all columns from a table named `FLOW_FILE` for relevant rows. The WHERE clause selects the data to keep. It is recommended to always quote column names using double-quotes in order to avoid conflicts with SQL keywords. For example, to keep only records where the person is an adult (aged 18 or older), add a property named "adults" with a value that is a SQL statement that selects records where the age is at least 18. So the value would be `SELECT * FROM FLOWFILE WHERE "age" >= 18` Adding this property now gives us a new Relationship whose name is the same as the property name. So, the "adults" Relationship should be connected to the next Processor in our flow. Keep only specific fields in a a Record, where the names of the fields to keep are knownkeepfilterretainselectincluderecordfieldssqlINPUT_ALLOWED"Record Reader" should be set to a Record Reader that is appropriate for your data. "Record Writer" should be set to a Record Writer that writes out data in the desired format. One additional property should be added. The name of the property should be a short description of the data to keep, such as `relevant fields`. Its value is a SQL statement that selects the desired columns from a table named `FLOW_FILE` for relevant rows. There is no WHERE clause. It is recommended to always quote column names using double-quotes in order to avoid conflicts with SQL keywords. For example, to keep only the `name`, `age`, and `address` fields, add a property named `relevant fields` with a value of `SELECT "name", "age", "address" FROM FLOWFILE` Adding this property now gives us a new Relationship whose name is the same as the property name. So, the `relevant fields` Relationship should be connected to the next Processor in our flow. Route record-oriented data for processing based on its contentsrecordrouteconditional processingfieldINPUT_ALLOWED"Record Reader" should be set to a Record Reader that is appropriate for your data. "Record Writer" should be set to a Record Writer that writes out data in the desired format. For each route that you want to create, add a new property. The name of the property should be a short description of the data that should be selected for the route. Its value is a SQL statement that selects all columns from a table named `FLOW_FILE`. The WHERE clause selects the data that should be included in the route. It is recommended to always quote column names using double-quotes in order to avoid conflicts with SQL keywords. A new outbound relationship is created for each property that is added. The name of the relationship is the same as the property name. For example, to route data based on whether or not it is a large transaction, we would add two properties: `small transaction` would have a value such as `SELECT * FROM FLOWFILE WHERE transactionTotal < 100` `large transaction` would have a value of `SELECT * FROM FLOWFILE WHERE transactionTotal >= 100` org.apache.nifi.processors.standard.RemoveRecordFieldPROCESSORModifies the contents of a FlowFile that contains Record-oriented data (i.e. data that can be read via a RecordReader and written by a RecordWriter) by removing selected fields. This Processor requires that at least one user-defined Property be added. The name of the property is ignored by the processor, but could be a meaningful identifier for the user. The value of the property should indicate a RecordPath that determines the field to be removed. The processor executes the removal in the order in which these properties are added to the processor. Set the "Record Writer" to "Inherit Record Schema" in order to use the updated Record Schema modified when removing Fields.updaterecordgenericschemajsoncsvavrofreeformtextremovedeleteRecord ReaderRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseRecord WriterRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseA description of the field to removeA RecordPath to the field to be removed.Any field that matches the RecordPath set as the value will be removed.FLOWFILE_ATTRIBUTESfailureIf a FlowFile cannot be transformed from the configured input format to the configured output format, the unchanged FlowFile will be routed to this relationshipfalsesuccessFlowFiles that are successfully transformed will be routed to this relationshipfalserecord.error.messageThis attribute provides on failure the error message encountered by the Reader or Writer.truetrueINPUT_REQUIREDRemove one or more fields from a Record, where the names of the fields to remove are known.recordfielddropremovedeleteexpungerecordpathINPUT_ALLOWEDConfigure the Record Reader according to the incoming data format. Configure the Record Writer according to the desired output format. For each field that you want to remove, add a single new property to the Processor. The name of the property can be anything but it's recommended to use a brief description of the field. The value of the property is a RecordPath that matches the field to remove. For example, to remove the `name` and `email` fields, add two Properties: `name` = `/name` `email` = `/email` org.apache.nifi.processors.standard.UpdateRecordorg.apache.nifi.processors.standard.RenameRecordFieldPROCESSORRenames one or more fields in each Record of a FlowFile. This Processor requires that at least one user-defined Property be added. The name of the Property should indicate a RecordPath that determines the field that should be updated. The value of the Property is the new name to assign to the Record Field that matches the RecordPath. The property value may use Expression Language to reference FlowFile attributes as well as the variables `field.name`, `field.value`, `field.type`, and `record.index`updaterecordrenamefieldgenericschemajsoncsvavrologlogsRecord ReaderRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseRecord WriterRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseA RecordPath that identifies which field(s) to updateThe new name to assign to the Record fieldAllows users to specify a new name for each field that matches the RecordPath.FLOWFILE_ATTRIBUTESfailureIf a FlowFile cannot be transformed from the configured input format to the configured output format, the unchanged FlowFile will be routed to this relationshipfalsesuccessFlowFiles that are successfully transformed will be routed to this relationshipfalserecord.indexThis attribute provides the current row index and is only available inside the literal value expression.truetrueINPUT_REQUIREDRename a field in each Record to a specific, known name.renamefieldstaticspecificnameINPUT_ALLOWEDConfigure the 'Record Reader' according to the input format. Configure the 'Record Writer' according to the desired output format. Add a property to the Processor such that the name of the property is a RecordPath to identifies the field to rename. The value of the property is the new name of the property. For example, to rename the `name` field to `full_name`, add a property with a name of `/name` and a value of `full_name`. Many properties can be added following this pattern in order to rename multiple fields. Rename a field in each Record to a name that is derived from a FlowFile attribute.renamefieldexpression languageELflowfileattributeINPUT_ALLOWEDConfigure the 'Record Reader' according to the input format. Configure the 'Record Writer' according to the desired output format. Add a property to the Processor such that the name of the property is a RecordPath to identifies the field to rename. The value of the property is an Expression Language expression that can be used to determine the new name of the field. For example, to rename the `addr` field to whatever value is stored in the `preferred_address_name` attribute, add a property with a name of `/name` and a value of `${preferred_address_name}`. Many properties can be added following this pattern in order to rename multiple fields. Rename a field in each Record to a new name that is derived from the current field name.This might be used, for example, to add a prefix or a suffix to some fields, or to transform the name of the field by making it uppercase.renamefieldexpression languageELfield.nameINPUT_ALLOWEDConfigure the 'Record Reader' according to the input format. Configure the 'Record Writer' according to the desired output format. Add a property to the Processor such that the name of the property is a RecordPath to identifies the field to rename. The value of the property is an Expression Language expression that references the `field.name` property. For example, to rename all fields with a prefix of `pre_`, we add a property named `/*` and a value of `pre_${field.name}`. If we would like this to happen recursively, to nested fields as well, we use a property name of `//*` with the value of `pre_${field.name}`. To make all field names uppercase, we can add a property named `//*` with a value of `${field.name:toUpper()}`. Many properties can be added following this pattern in order to rename multiple fields. org.apache.nifi.processors.standard.UpdateRecordorg.apache.nifi.processors.standard.RemoveRecordFieldorg.apache.nifi.processors.standard.ReplaceTextPROCESSORUpdates the content of a FlowFile by searching for some textual value in the FlowFile content (via Regular Expression/regex, or literal value) and replacing the section of the content that matches with some alternate value. It can also be used to append or prepend text to the contents of a FlowFile.TextRegular ExpressionUpdateChangeReplaceModifyRegexReplacement StrategyReplacement StrategyThe strategy for how and what to replace within the FlowFile's text content.Regex ReplacePrependPrependInsert the Replacement Value at the beginning of the FlowFile or the beginning of each line (depending on the Evaluation Mode). For "Line-by-Line" Evaluation Mode, the value will be prepended to each line. Similarly, for "First-Line", "Last-Line", "Except-Last-Line" and "Except-First-Line" Evaluation Modes,the value will be prepended to header alone, footer alone, all lines except header and all lines except footer respectively. For "Entire Text" evaluation mode,the value will be prepended to the entire text.AppendAppendInsert the Replacement Value at the end of the FlowFile or the end of each line (depending on the Evaluation Mode). For "Line-by-Line" Evaluation Mode, the value will be appended to each line. Similarly, for "First-Line", "Last-Line", "Except-Last-Line" and "Except-First-Line" Evaluation Modes,the value will be appended to header alone, footer alone, all lines except header and all lines except footer respectively. For "Entire Text" evaluation mode,the value will be appended to the entire text.SurroundSurroundPrepends text before the start of the FlowFile (or the start of each line, depending on the configuration of the Evaluation Mode property) as well as appending text to the end of the FlowFile (or the end of each line, depending on the configuration of the Evaluation Mode property)Regex ReplaceRegex ReplaceInterpret the Search Value as a Regular Expression and replace all matches with the Replacement Value. The Replacement Value may reference Capturing Groups used in the Search Value by using a dollar-sign followed by the Capturing Group number, such as $1 or $2. If the Search Value is set to .* then everything is replaced without even evaluating the Regular Expression.Literal ReplaceLiteral ReplaceSearch for all instances of the Search Value and replace the matches with the Replacement Value.Always ReplaceAlways ReplaceAlways replaces the entire line or the entire contents of the FlowFile (depending on the value of the <Evaluation Mode> property) and does not bother searching for any value. When this strategy is chosen, the <Search Value> property is ignored.Substitute VariablesSubstitute VariablesSubstitute variable references (specified in ${var} form) using FlowFile attributes for looking up the replacement value by variable name. When this strategy is chosen, both the <Search Value> and <Replacement Value> properties are ignored.truefalsefalseNONEfalsefalseRegular ExpressionSearch ValueThe Search Value to search for in the FlowFile content. Only used for 'Literal Replace' and 'Regex Replace' matching strategies(?s)(^.*$)truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseReplacement StrategyReplacement StrategyRegex ReplaceLiteral ReplaceReplacement ValueReplacement ValueThe value to insert using the 'Replacement Strategy'. Using "Regex Replace" back-references to Regular Expression capturing groups are supported, but back-references that reference capturing groups that do not exist in the regular expression will be treated as literal value. Back References may also be referenced using the Expression Language, as '$1', '$2', etc. The single-tick marks MUST be included, as these variables are not "Standard" attribute names (attribute names must be quoted unless they contain only numbers, letters, and _).$1truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseReplacement StrategyReplacement StrategyPrependRegex ReplaceAlways ReplaceAppendLiteral ReplaceText to PrependText to PrependThe text to prepend to the start of the FlowFile, or each line, depending on teh configured value of the Evaluation Mode propertytruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseReplacement StrategyReplacement StrategySurroundText to AppendText to AppendThe text to append to the end of the FlowFile, or each line, depending on teh configured value of the Evaluation Mode propertytruefalsetrueFLOWFILE_ATTRIBUTESfalsefalseReplacement StrategyReplacement StrategySurroundCharacter SetCharacter SetThe Character Set in which the file is encodedUTF-8truefalsefalseNONEfalsefalseMaximum Buffer SizeMaximum Buffer SizeSpecifies the maximum amount of data to buffer (per file or per line, depending on the Evaluation Mode) in order to apply the replacement. If 'Entire Text' (in Evaluation Mode) is selected and the FlowFile is larger than this value, the FlowFile will be routed to 'failure'. In 'Line-by-Line' Mode, if a single line is larger than this value, the FlowFile will be routed to 'failure'. A default value of 1 MB is provided, primarily for 'Entire Text' mode. In 'Line-by-Line' Mode, a value such as 8 KB or 16 KB is suggested. This value is ignored if the <Replacement Strategy> property is set to one of: Append, Prepend, Always Replace1 MBtruefalsefalseNONEfalsefalseEvaluation ModeEvaluation ModeRun the 'Replacement Strategy' against each line separately (Line-by-Line) or buffer the entire file into memory (Entire Text) and run against that.Line-by-LineLine-by-LineLine-by-LineEntire textEntire texttruefalsefalseNONEfalsefalseLine-by-Line Evaluation ModeLine-by-Line Evaluation ModeRun the 'Replacement Strategy' against each line separately (Line-by-Line) for all lines in the FlowFile, First Line (Header) alone, Last Line (Footer) alone, Except the First Line (Header) or Except the Last Line (Footer).AllAllAllFirst-LineFirst-LineLast-LineLast-LineExcept-First-LineExcept-First-LineExcept-Last-LineExcept-Last-LinefalsefalsefalseNONEfalsefalsesuccessFlowFiles that have been successfully processed are routed to this relationship. This includes both FlowFiles that had text replaced and those that did not.falsefailureFlowFiles that could not be updated are routed to this relationshipfalsetruetrueINPUT_REQUIREDMEMORYAn instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.Append text to the end of every line in a FlowFileraw textappendlineINPUT_ALLOWED"Evaluation Mode" = "Line-by-Line" "Replacement Strategy" = "Append" "Replacement Value" is set to whatever text should be appended to the line. For example, to insert the text `<fin>` at the end of every line, we would set "Replacement Value" to `<fin>`. We can also use Expression Language. So to insert the filename at the end of every line, we set "Replacement Value" to `${filename}` Prepend text to the beginning of every line in a FlowFileraw textprependlineINPUT_ALLOWED"Evaluation Mode" = "Line-by-Line" "Replacement Strategy" = "Prepend" "Replacement Value" is set to whatever text should be prepended to the line. For example, to insert the text `<start>` at the beginning of every line, we would set "Replacement Value" to `<start>`. We can also use Expression Language. So to insert the filename at the beginning of every line, we set "Replacement Value" to `${filename}` Replace every occurrence of a literal string in the FlowFile with a different valuereplacestringtextliteralINPUT_ALLOWED"Evaluation Mode" = "Line-by-Line" "Replacement Strategy" = "Literal Replace" "Search Value" is set to whatever text is in the FlowFile that needs to be replaced. "Replacement Value" is set to the text that should replace the current text. For example, to replace the word "spider" with "arachnid" we set "Search Value" to `spider` and set "Replacement Value" to `arachnid`. Transform every occurrence of a literal string in a FlowFilereplacetransformraw textINPUT_ALLOWED"Evaluation Mode" = "Line-by-Line" "Replacement Strategy" = "Regex Replace" "Search Value" is set to a regular expression that matches the text that should be transformed in a capturing group. "Replacement Value" is set to a NiFi Expression Language expression that references `$1` (in quotes to escape the reference name). For example, if we wanted to lowercase any occurrence of WOLF, TIGER, or LION, we would use a "Search Value" of `(WOLF|TIGER|LION)` and a "Replacement Value" of `${'$1':toLower()}`. If we want to replace any identifier with a hash of that identifier, we might use a "Search Value" of `identifier: (.*)` and a "Replacement Value" of `identifier: ${'$1':hash('sha256')}` Completely replace the contents of a FlowFile to a specific textreplaceraw textINPUT_ALLOWED"Evaluation Mode" = "Entire text" "Replacement Strategy" = "Always Replace" "Replacement Value" is set to the new text that should be written to the FlowFile. This text might include NiFi Expression Language to reference one or more attributes. org.apache.nifi.processors.standard.ReplaceTextWithMappingPROCESSORUpdates the content of a FlowFile by evaluating a Regular Expression against it and replacing the section of the content that matches the Regular Expression with some alternate value provided in a mapping file.TextRegular ExpressionUpdateChangeReplaceModifyRegexMappingRegular ExpressionRegular ExpressionThe Regular Expression to search for in the FlowFile content\S+truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseMatching GroupMatching GroupThe number of the matching group of the provided regex to replace with the corresponding value from the mapping file (if it exists).0truefalsetrueENVIRONMENTfalsefalseMapping FileMapping FileThe name of the file (including the full path) containing the Mappings.truefalsefalseNONEfalsefalseSINGLEFILEMapping File Refresh IntervalMapping File Refresh IntervalThe polling interval to check for updates to the mapping file. The default is 60s.60struefalsefalseNONEfalsefalseCharacter SetCharacter SetThe Character Set in which the file is encodedUTF-8truefalsefalseNONEfalsefalseMaximum Buffer SizeMaximum Buffer SizeSpecifies the maximum amount of data to buffer (per file) in order to apply the regular expressions. If a FlowFile is larger than this value, the FlowFile will be routed to 'failure'1 MBtruefalsefalseNONEfalsefalsesuccessFlowFiles that have been successfully updated are routed to this relationship, as well as FlowFiles whose content does not match the given Regular ExpressionfalsefailureFlowFiles that could not be updated are routed to this relationshipfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.RetryFlowFilePROCESSORFlowFiles passed to this Processor have a 'Retry Attribute' value checked against a configured 'Maximum Retries' value. If the current attribute value is below the configured maximum, the FlowFile is passed to a retry relationship. The FlowFile may or may not be penalized in that condition. If the FlowFile's attribute value exceeds the configured maximum, the FlowFile will be passed to a 'retries_exceeded' relationship. WARNING: If the incoming FlowFile has a non-numeric value in the configured 'Retry Attribute' attribute, it will be reset to '1'. You may choose to fail the FlowFile instead of performing the reset. Additional dynamic properties can be defined for any attributes you wish to add to the FlowFiles transferred to 'retries_exceeded'. These attributes support attribute expression language.RetryFlowFileretry-attributeRetry AttributeThe name of the attribute that contains the current retry count for the FlowFile. WARNING: If the name matches an attribute already on the FlowFile that does not contain a numerical value, the processor will either overwrite that attribute with '1' or fail based on configuration.flowfile.retriestruefalsetrueENVIRONMENTfalsefalsemaximum-retriesMaximum RetriesThe maximum number of times a FlowFile can be retried before being passed to the 'retries_exceeded' relationship3truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsepenalize-retriesPenalize RetriesIf set to 'true', this Processor will penalize input FlowFiles before passing them to the 'retry' relationship. This does not apply to the 'retries_exceeded' relationship.truetruetruefalsefalsetruefalsefalseNONEfalsefalseFail on Non-numerical OverwriteFail on Non-numerical OverwriteIf the FlowFile already has the attribute defined in 'Retry Attribute' that is *not* a number, fail the FlowFile instead of resetting that value to '1'falsetruetruefalsefalsetruefalsefalseNONEfalsefalsereuse-modeReuse ModeDefines how the Processor behaves if the retry FlowFile has a different retry UUID than the instance that received the FlowFile. This generally means that the attribute was not reset after being successfully retried by a previous instance of this processor.failFail on ReusefailIf the RetryFlowFile's UUID does not match the FlowFile's retry UUID, fail the FlowFile regardless of current retry countWarn on ReusewarnIf the RetryFlowFile's UUID does not match the FlowFile's retry UUID, log a warning message before resetting the retry attribute and UUID for this instanceReset ReuseresetIf the RetryFlowFile's UUID does not match the FlowFile's retry UUID, log a debug message before resetting the retry attribute and UUID for this instancetruefalsefalseNONEfalsefalseExceeded FlowFile Attribute KeyThe value of the attribute added to the FlowFileOne or more dynamic properties can be used to add attributes to FlowFiles passed to the 'retries_exceeded' relationshipFLOWFILE_ATTRIBUTESretries_exceededInput FlowFile has exceeded the configured maximum retry count, do not pass this relationship back to the input Processor to terminate the limited feedback loop.falseretryInput FlowFile has not exceeded the configured maximum retry count, pass this relationship back to the input Processor to create a limited feedback loop.falsefailureThe processor is configured such that a non-numerical value on 'Retry Attribute' results in a failure instead of resetting that value to '1'. This will immediately terminate the limited feedback loop. Might also include when 'Maximum Retries' contains attribute expression language that does not resolve to an Integer.trueRetry AttributeWill read the attribute or attribute expression language result as defined in 'Retry Attribute'Retry AttributeUser defined retry attribute is updated with the current retry countRetry Attribute .uuidUser defined retry attribute with .uuid that determines what processor retried the FlowFile lasttruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.RouteOnAttributePROCESSORRoutes FlowFiles based on their Attributes using the Attribute Expression LanguageattributesroutingAttribute Expression LanguageregexpregexRegular ExpressionExpression LanguagefindtextstringsearchfilterdetectRouting StrategyRouting StrategySpecifies how to determine which relationship to use when evaluating the Expression LanguageRoute to Property nameRoute to Property nameRoute to Property nameA copy of the FlowFile will be routed to each relationship whose corresponding expression evaluates to 'true'Route to 'matched' if all matchRoute to 'match' if all matchRequires that all user-defined expressions evaluate to 'true' for the FlowFile to be considered a matchRoute to 'matched' if any matchesRoute to 'match' if any matchesRequires that at least one user-defined expression evaluate to 'true' for the FlowFile to be considered a matchtruefalsefalseNONEfalsefalseRelationship NameExpression Language expression that returns a boolean value indicating whether or not the FlowFile should be routed to this RelationshipRoutes FlowFiles whose attributes match the Expression Language specified in the Dynamic Property Value to the Relationship specified in the Dynamic Property KeyFLOWFILE_ATTRIBUTESunmatchedFlowFiles that do not match any user-define expression will be routed herefalseName from Dynamic PropertyFlowFiles that match the Dynamic Property's Attribute Expression LanguageRouteOnAttribute.RouteThe relation to which the FlowFile was routedtruetrueINPUT_REQUIREDRoute data to one or more relationships based on its attributes using the NiFi Expression Language.attributesroutingexpression languageINPUT_ALLOWEDSet the "Routing Strategy" property to "Route to Property name". For each route that a FlowFile might be routed to, add a new property. The name of the property should describe the route. The value of the property is an Attribute Expression Language expression that returns a boolean value indicating whether or not a given FlowFile will be routed to the associated relationship. For example, we might route data based on its file extension using the following properties: - "Routing Strategy" = "Route to Property Name" - "jpg" = "${filename:endsWith('.jpg')}" - "png" = "${filename:endsWith('.png')}" - "pdf" = "${filename:endsWith('.pdf')}" The Processor will now have 3 relationships: `jpg`, `png`, and `pdf`. Each of these should be connected to the appropriate downstream processor. Keep data only if its attributes meet some criteria, such as its filename ends with .txt.keepfilterremovedeleteexpression languageINPUT_ALLOWEDAdd a new property for each condition that must be satisfied in order to keep the data. If the data should be kept in the case that any of the provided conditions is met, set the "Routing Strategy" property to "Route to 'matched' if any matches". If all conditions must be met in order to keep the data, set the "Routing Strategy" property to "Route to 'matched' if all match". For example, to keep files whose filename ends with .txt and have a file size of at least 1000 bytes, we will use the following properties: - "ends_with_txt" = "${filename:endsWith('.txt')}" - "large_enough" = "${fileSize:ge(1000)} - "Routing Strategy" = "Route to 'matched' if all match" Auto-terminate the 'unmatched' relationship. Connect the 'matched' relationship to the next processor in the flow. Discard or drop a file based on attributes, such as filename.discarddropfilterremovedeleteexpression languageINPUT_ALLOWEDAdd a new property for each condition that must be satisfied in order to drop the data. If the data should be dropped in the case that any of the provided conditions is met, set the "Routing Strategy" property to "Route to 'matched' if any matches". If all conditions must be met in order to drop the data, set the "Routing Strategy" property to "Route to 'matched' if all match". Here are a couple of examples for configuring the properties: Example 1 Use Case: Data should be dropped if its "uuid" attribute has an 'a' in it or ends with '0'. Here, we will use the following properties: - "has_a" = "${uuid:contains('a')}" - "ends_with_0" = "${uuid:endsWith('0')} - "Routing Strategy" = "Route to 'matched' if any matches" Example 2 Use Case: Data should be dropped if its 'uuid' attribute has an 'a' AND it ends with a '1'. Here, we will use the following properties: - "has_a" = "${uuid:contains('a')}" - "ends_with_1" = "${uuid:endsWith('1')} - "Routing Strategy" = "Route to 'matched' if all match" Auto-terminate the 'matched' relationship. Connect the 'unmatched' relationship to the next processor in the flow. Route record-oriented data based on whether or not the record's values meet some criteriarecordroutecontentdataorg.apache.nifi.processors.standard.PartitionRecordChoose a RecordReader that is appropriate based on the format of the incoming data. Choose a RecordWriter that writes the data in the desired output format. Add a single additional property. The name of the property should describe the criteria to route on. The property's value should be a RecordPath that returns `true` if the Record meets the criteria or `false` otherwise. This adds a new attribute to the FlowFile whose name is equal to the property name. Connect the 'success' Relationship to RouteOnAttribute. org.apache.nifi.processors.standard.RouteOnAttributeSet "Routing Strategy" to "Route to Property name" Add two additional properties. For the first one, the name of the property should describe data that matches the criteria. The value is an Expression Language expression that checks if the attribute added by the PartitionRecord processor has a value of `true`. For example, `${criteria:equals('true')}`. The second property should have a name that describes data that does not match the criteria. The value is an Expression Language that evaluates to the opposite of the first property value. For example, `${criteria:equals('true'):not()}`. Connect each of the newly created Relationships to the appropriate downstream processors. org.apache.nifi.processors.standard.RouteOnContentPROCESSORApplies Regular Expressions to the content of a FlowFile and routes a copy of the FlowFile to each destination whose Regular Expression matches. Regular Expressions are added as User-Defined Properties where the name of the property is the name of the relationship and the value is a Regular Expression to match against the FlowFile content. User-Defined properties do support the Attribute Expression Language, but the results are interpreted as literal values, not Regular Expressionsroutecontentregexregular expressionregexpfindtextstringsearchfilterdetectMatch RequirementMatch RequirementSpecifies whether the entire content of the file must match the regular expression exactly, or if any part of the file (up to Content Buffer Size) can contain the regular expression in order to be considered a matchcontent must match exactlycontent must match exactlycontent must match exactlycontent must contain matchcontent must contain matchtruefalsefalseNONEfalsefalseCharacter SetCharacter SetThe Character Set in which the file is encodedUTF-8truefalsefalseNONEfalsefalseContent Buffer SizeContent Buffer SizeSpecifies the maximum amount of data to buffer in order to apply the regular expressions. If the size of the FlowFile exceeds this value, any amount of this value will be ignored1 MBtruefalsefalseNONEfalsefalseRelationship NameA Regular ExpressionRoutes FlowFiles whose content matches the regular expression defined by Dynamic Property's value to the Relationship defined by the Dynamic Property's keyFLOWFILE_ATTRIBUTESunmatchedFlowFiles that do not match any of the user-supplied regular expressions will be routed to this relationshipfalseName from Dynamic PropertyFlowFiles that match the Dynamic Property's Regular ExpressiontruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.RouteTextPROCESSORRoutes textual data based on a set of user-defined rules. Each line in an incoming FlowFile is compared against the values specified by user-defined Properties. The mechanism by which the text is compared to these user-defined properties is defined by the 'Matching Strategy'. The data is then routed according to these rules, routing each line of the text individually.attributesroutingtextregexpregexRegular ExpressionExpression LanguagecsvfilterlogsdelimitedfindstringsearchfilterdetectRouting StrategyRouting StrategySpecifies how to determine which Relationship(s) to use when evaluating the lines of incoming text against the 'Matching Strategy' and user-defined properties.Route to each matching Property NameRoute to each matching Property NameRoute to each matching Property NameLines will be routed to each relationship whose corresponding expression evaluates to 'true'Route to 'matched' if line matches all conditionsRoute to 'matched' if line matches all conditionsRequires that all user-defined expressions evaluate to 'true' for the line to be considered a matchRoute to 'matched' if lines matches any conditionRoute to 'matched' if lines matches any conditionRequires that at least one user-defined expression evaluate to 'true' for the line to be considered a matchtruefalsefalseNONEfalsefalseMatching StrategyMatching StrategySpecifies how to evaluate each line of incoming text against the user-defined properties.Satisfies ExpressionSatisfies ExpressionMatch lines based on whether or not the the text satisfies the given Expression Language expression. I.e., the line will match if the property value, evaluated as an Expression, returns true. The expression is able to reference FlowFile Attributes, as well as the variables 'line' (which is the text of the line to evaluate) and 'lineNo' (which is the line number being evaluated. This will be 1 for the first line, 2 for the second and so on).Starts WithStarts WithMatch lines based on whether the line starts with the property valueEnds WithEnds WithMatch lines based on whether the line ends with the property valueContainsContainsMatch lines based on whether the line contains the property valueEqualsEqualsMatch lines based on whether the line equals the property valueMatches Regular ExpressionMatches Regular ExpressionMatch lines based on whether the line exactly matches the Regular Expression that is provided as the Property valueContains Regular ExpressionContains Regular ExpressionMatch lines based on whether the line contains some text that matches the Regular Expression that is provided as the Property valuetruefalsefalseNONEfalsefalseCharacter SetCharacter SetThe Character Set in which the incoming text is encodedUTF-8truefalsefalseNONEfalsefalseIgnore Leading/Trailing WhitespaceIgnore Leading/Trailing WhitespaceIndicates whether or not the whitespace at the beginning and end of the lines should be ignored when evaluating the line.truetruefalsefalseNONEfalsefalseIgnore CaseIgnore CaseIf true, capitalization will not be taken into account when comparing values. E.g., matching against 'HELLO' or 'hello' will have the same result. This property is ignored if the 'Matching Strategy' is set to 'Satisfies Expression'.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseGrouping Regular ExpressionGrouping Regular ExpressionSpecifies a Regular Expression to evaluate against each line to determine which Group the line should be placed in. The Regular Expression must have at least one Capturing Group that defines the line's Group. If multiple Capturing Groups exist in the Regular Expression, the values from all Capturing Groups will be concatenated together. Two lines will not be placed into the same FlowFile unless they both have the same value for the Group (or neither line matches the Regular Expression). For example, to group together all lines in a CSV File by the first column, we can set this value to "(.*?),.*". Two lines that have the same Group but different Relationships will never be placed into the same FlowFile.falsefalsefalseNONEfalsefalseRelationship Namevalue to match againstRoutes data that matches the value specified in the Dynamic Property Value to the Relationship specified in the Dynamic Property Key.FLOWFILE_ATTRIBUTESoriginalThe original input file will be routed to this destination when the lines have been successfully routed to 1 or more relationshipsfalseunmatchedData that does not satisfy the required user-defined rules will be routed to this RelationshipfalseName from Dynamic PropertyFlowFiles that match the Dynamic Property's valueRouteText.RouteThe name of the relationship to which the FlowFile was routed.RouteText.GroupThe value captured by all capturing groups in the 'Grouping Regular Expression' property. If this property is not set or contains no capturing groups, this attribute will not be added.truetrueINPUT_REQUIREDDrop blank or empty lines from the FlowFile's content.filterdropemptyblankremovedeletestrip outlinestextINPUT_ALLOWED"Routing Strategy" = "Route to each matching Property Name" "Matching Strategy" = "Matches Regular Expression" "Empty Line" = "^$" Auto-terminate the "Empty Line" relationship. Connect the "unmatched" relationship to the next processor in your flow. Remove specific lines of text from a file, such as those containing a specific word or having a line length over some threshold.filterdropemptyblankremovedeletestrip outlinestextexpression languageINPUT_ALLOWED"Routing Strategy" = "Route to each matching Property Name" "Matching Strategy" = "Satisfies Expression" An additional property should be added named "Filter Out." The value should be a NiFi Expression Language Expression that can refer to two variables (in addition to FlowFile attributes): `line`, which is the line of text being evaluated; and `lineNo`, which is the line number in the file (starting with 1). The Expression should return `true` for any line that should be dropped. For example, to remove any line that starts with a # symbol, we can set "Filter Out" to `${line:startsWith("#")}`. We could also remove the first 2 lines of text by setting "Filter Out" to `${lineNo:le(2)}`. Note that we use the `le` function because we want lines numbers less than or equal to `2`, since the line index is 1-based. Auto-terminate the "Filter Out" relationship. Connect the "unmatched" relationship to the next processor in your flow. org.apache.nifi.processors.standard.SampleRecordPROCESSORSamples the records of a FlowFile based on a specified sampling strategy (such as Reservoir Sampling). The resulting FlowFile may be of a fixed number of records (in the case of reservoir-based algorithms) or some subset of the total number of records (in the case of probabilistic sampling), or a deterministic number of records (in the case of interval sampling).recordsamplereservoirrangeintervalrecord-readerRecord ReaderSpecifies the Controller Service to use for parsing incoming data and determining the data's schemaorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Controller Service to use for writing results to a FlowFileorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsesample-record-sampling-strategySampling StrategySpecifies which method to use for sampling records from the incoming FlowFilereservoirInterval SamplingintervalSelects every Nth record where N is the value of the 'Interval Value' propertyRange SamplingrangeCreates a sample of records based on the index (i.e. record number) of the records using the specified range. An example is '3,6-8,20-' which includes the third record, the sixth, seventh and eighth record, and all records from the twentieth record on. Commas separate intervals that don't overlap, and an interval can be between two numbers (i.e. 6-8) or up to a given number (i.e. -5), or from a number to the number of the last record (i.e. 20-).Probabilistic SamplingprobabilisticSelects each record with probability P where P is the value of the 'Selection Probability' propertyReservoir SamplingreservoirCreates a sample of K records where each record has equal probability of being included, where K is the value of the 'Reservoir Size' property. Note that if the value is very large it may cause memory issues as the reservoir is kept in-memory.truefalsefalseNONEfalsefalsesample-record-intervalSampling IntervalSpecifies the number of records to skip before writing a record to the outgoing FlowFile. This property is only used if Sampling Strategy is set to Interval Sampling. A value of zero (0) will cause no records to be included in theoutgoing FlowFile, a value of one (1) will cause all records to be included, and a value of two (2) will cause half the records to be included, and so on.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesample-record-sampling-strategySampling Strategyintervalsample-record-rangeSampling RangeSpecifies the range of records to include in the sample, from 1 to the total number of records. An example is '3,6-8,20-' which includes the third record, the sixth, seventh and eighth records, and all records from the twentieth record on. Commas separate intervals that don't overlap, and an interval can be between two numbers (i.e. 6-8) or up to a given number (i.e. -5), or from a number to the number of the last record (i.e. 20-). If this property is unset, all records will be included.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesample-record-sampling-strategySampling Strategyrangesample-record-probabilitySampling ProbabilitySpecifies the probability (as a percent from 0-100) of a record being included in the outgoing FlowFile. This property is only used if Sampling Strategy is set to Probabilistic Sampling. A value of zero (0) will cause no records to be included in theoutgoing FlowFile, and a value of 100 will cause all records to be included in the outgoing FlowFile..truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesample-record-sampling-strategySampling Strategyprobabilisticsample-record-reservoirReservoir SizeSpecifies the number of records to write to the outgoing FlowFile. This property is only used if Sampling Strategy is set to reservoir-based strategies such as Reservoir Sampling.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesample-record-sampling-strategySampling Strategyreservoirsample-record-random-seedRandom SeedSpecifies a particular number to use as the seed for the random number generator (used by probabilistic strategies). Setting this property will ensure the same records are selected even when using probabilistic strategies.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesample-record-sampling-strategySampling StrategyprobabilisticreservoirsuccessThe FlowFile is routed to this relationship if the sampling completed successfullytruefailureIf a FlowFile fails processing for any reason (for example, any record is not valid), the original FlowFile will be routed to this relationshipfalseoriginalThe original FlowFile is routed to this relationship if sampling is successfultruemime.typeThe MIME type indicated by the record writerrecord.countThe number of records in the resulting flow filetruetrueINPUT_REQUIREDMEMORYAn instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.org.apache.nifi.processors.standard.ScanAttributePROCESSORScans the specified attributes of FlowFiles, checking to see if any of their values are present within the specified dictionary of termsscanattributessearchlookupfindtextDictionary FileDictionary FileA new-line-delimited text file that includes the terms that should trigger a match. Empty lines are ignored. The contents of the text file are loaded into memory when the processor is scheduled and reloaded when the contents are modified.truefalsetrueENVIRONMENTfalsefalseSINGLEFILEAttribute PatternAttribute PatternRegular Expression that specifies the names of attributes whose values will be matched against the terms in the dictionary.*truefalsefalseNONEfalsefalseMatch CriteriaMatch CriteriaIf set to All Must Match, then FlowFiles will be routed to 'matched' only if all specified attributes' values are found in the dictionary. If set to At Least 1 Must Match, FlowFiles will be routed to 'matched' if any attribute specified is found in the dictionaryAt Least 1 Must MatchAt Least 1 Must MatchAt Least 1 Must MatchAll Must MatchAll Must MatchtruefalsefalseNONEfalsefalseDictionary Filter PatternDictionary Filter PatternA Regular Expression that will be applied to each line in the dictionary file. If the regular expression does not match the line, the line will not be included in the list of terms to search for. If a Matching Group is specified, only the portion of the term that matches that Matching Group will be used instead of the entire term. If not specified, all terms in the dictionary will be used and each term will consist of the text of the entire line in the filefalsefalsefalseNONEfalsefalseunmatchedFlowFiles whose attributes are not found in the dictionary will be routed to this relationshipfalsematchedFlowFiles whose attributes are found in the dictionary will be routed to this relationshipfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ScanContentPROCESSORScans the content of FlowFiles for terms that are found in a user-supplied dictionary. If a term is matched, the UTF-8 encoded version of the term will be added to the FlowFile using the 'matching.term' attributeaho-corasickscancontentbyte sequencesearchfinddictionaryDictionary FileDictionary FileThe filename of the terms dictionarytruefalsefalseNONEfalsefalseSINGLEFILEDictionary EncodingDictionary EncodingIndicates how the dictionary is encoded. If 'text', dictionary terms are new-line delimited and UTF-8 encoded; if 'binary', dictionary terms are denoted by a 4-byte integer indicating the term length followed by the term itselftexttexttextbinarybinarytruefalsefalseNONEfalsefalseunmatchedFlowFiles that do not match any term in the dictionary are routed to this relationshipfalsematchedFlowFiles that match at least one term in the dictionary are routed to this relationshipfalsematching.termThe term that caused the Processor to route the FlowFile to the 'matched' relationship; if FlowFile is routed to the 'unmatched' relationship, this attribute is not addedtruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.SegmentContentPROCESSORSegments a FlowFile into multiple smaller segments on byte boundaries. Each segment is given the following attributes: fragment.identifier, fragment.index, fragment.count, segment.original.filename; these attributes can then be used by the MergeContent processor in order to reconstitute the original FlowFilesegmentsplitSegment SizeSegment SizeThe maximum data size in bytes for each segmenttruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesegmentsAll segments will be sent to this relationship. If the file was small enough that it was not segmented, a copy of the original is sent to this relationship as well as originalfalseoriginalThe original FlowFile will be sent to this relationshipfalsefragment.identifierAll segments produced from the same parent FlowFile will have the same randomly generated UUID added for this attributefragment.indexA one-up number that indicates the ordering of the segments that were created from a single parent FlowFilefragment.countThe number of segments generated from the parent FlowFilesegment.original.filename The filename of the parent FlowFilesegment.original.filename The filename will be updated to include the parent's filename, the segment index, and the segment counttruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.MergeContentorg.apache.nifi.processors.standard.SplitContentPROCESSORSplits incoming FlowFiles by a specified byte sequencecontentsplitbinaryByte Sequence FormatByte Sequence FormatSpecifies how the <Byte Sequence> property should be interpretedHexadecimalHexadecimalHexadecimalThe Byte Sequence will be interpreted as a hexadecimal representation of bytesTextTextThe Byte Sequence will be interpreted as UTF-8 Encoded texttruefalsefalseNONEfalsefalseByte SequenceByte SequenceA representation of bytes to look for and upon which to split the source file into separate filestruefalsefalseNONEfalsefalseKeep Byte SequenceKeep Byte SequenceDetermines whether or not the Byte Sequence should be included with each SplitfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseByte Sequence LocationByte Sequence LocationIf <Keep Byte Sequence> is set to true, specifies whether the byte sequence should be added to the end of the first split or the beginning of the second; if <Keep Byte Sequence> is false, this property is ignored.TrailingTrailingTrailingKeep the Byte Sequence at the end of the first split if <Keep Byte Sequence> is trueLeadingLeadingKeep the Byte Sequence at the beginning of the second split if <Keep Byte Sequence> is truetruefalsefalseNONEfalsefalsesplitsAll Splits will be routed to the splits relationshipfalseoriginalThe original filefalsefragment.identifierAll split FlowFiles produced from the same parent FlowFile will have the same randomly generated UUID added for this attributefragment.indexA one-up number that indicates the ordering of the split FlowFiles that were created from a single parent FlowFilefragment.countThe number of split FlowFiles generated from the parent FlowFilesegment.original.filename The filename of the parent FlowFiletruetrueINPUT_REQUIREDMEMORYThe FlowFile with its attributes is stored in memory, not the content of the FlowFile. If many splits are generated due to the size of the content, or how the content is configured to be split, a two-phase approach may be necessary to avoid excessive use of memory.org.apache.nifi.processors.standard.MergeContentorg.apache.nifi.processors.standard.SplitJsonPROCESSORSplits a JSON File into multiple, separate FlowFiles for an array element specified by a JsonPath expression. Each generated FlowFile is comprised of an element of the specified array and transferred to relationship 'split,' with the original file transferred to the 'original' relationship. If the specified JsonPath is not found or does not evaluate to an array element, the original file is routed to 'failure' and no files are generated.jsonsplitjsonpathJsonPath ExpressionJsonPath ExpressionA JsonPath expression that indicates the array element to split into JSON/scalar fragments.truefalsefalseNONEfalsefalseNull Value RepresentationNull Value RepresentationIndicates the desired representation of JSON Path expressions resulting in a null value.empty stringempty stringempty stringthe string 'null'the string 'null'truefalsefalseNONEfalsefalseMax String LengthMax String LengthThe maximum allowed length of a string value when parsing the JSON document20 MBtruefalsefalseNONEfalsefalsefailureIf a FlowFile fails processing for any reason (for example, the FlowFile is not valid JSON or the specified path does not exist), it will be routed to this relationshipfalseoriginalThe original FlowFile that was split into segments. If the FlowFile fails processing, nothing will be sent to this relationshipfalsesplitAll segments of the original FlowFile will be routed to this relationshipfalsefragment.identifierAll split FlowFiles produced from the same parent FlowFile will have the same randomly generated UUID added for this attributefragment.indexA one-up number that indicates the ordering of the split FlowFiles that were created from a single parent FlowFilefragment.countThe number of split FlowFiles generated from the parent FlowFilesegment.original.filename The filename of the parent FlowFiletruetrueINPUT_REQUIREDMEMORYThe entirety of the FlowFile's content (as a JsonNode object) is read into memory, in addition to all of the generated FlowFiles representing the split JSON. If many splits are generated due to the size of the JSON, or how the JSON is configured to be split, a two-phase approach may be necessary to avoid excessive use of memory.org.apache.nifi.processors.standard.SplitRecordPROCESSORSplits up an input FlowFile that is in a record-oriented data format into multiple smaller FlowFilessplitgenericschemajsoncsvavrologlogsfreeformtextRecord ReaderRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseRecord WriterRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseRecords Per SplitRecords Per SplitSpecifies how many records should be written to each 'split' or 'segment' FlowFiletruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsefailureIf a FlowFile cannot be transformed from the configured input format to the configured output format, the unchanged FlowFile will be routed to this relationship.falsesplitsThe individual 'segments' of the original FlowFile will be routed to this relationship.falseoriginalUpon successfully splitting an input FlowFile, the original FlowFile will be sent to this relationship.falsemime.typeSets the mime.type attribute to the MIME Type specified by the Record Writer for the FlowFiles routed to the 'splits' Relationship.record.countThe number of records in the FlowFile. This is added to FlowFiles that are routed to the 'splits' Relationship.fragment.identifierAll split FlowFiles produced from the same parent FlowFile will have the same randomly generated UUID added for this attributefragment.indexA one-up number that indicates the ordering of the split FlowFiles that were created from a single parent FlowFilefragment.countThe number of split FlowFiles generated from the parent FlowFilesegment.original.filename The filename of the parent FlowFiletruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.SplitTextPROCESSORSplits a text file into multiple smaller text files on line boundaries limited by maximum number of lines or total size of fragment. Each output split file will contain no more than the configured number of lines or bytes. If both Line Split Count and Maximum Fragment Size are specified, the split occurs at whichever limit is reached first. If the first line of a fragment exceeds the Maximum Fragment Size, that line will be output in a single split file which exceeds the configured maximum size limit. This component also allows one to specify that each split should include a header lines. Header lines can be computed by either specifying the amount of lines that should constitute a header or by using header marker to match against the read lines. If such match happens then the corresponding line will be treated as header. Keep in mind that upon the first failure of header marker match, no more matches will be performed and the rest of the data will be parsed as regular lines for a given split. If after computation of the header there are no more data, the resulting split will consists of only header lines.splittextLine Split CountLine Split CountThe number of lines that will be added to each split file, excluding header lines. A value of zero requires Maximum Fragment Size to be set, and line count will not be considered in determining splits.truefalsefalseNONEfalsefalseMaximum Fragment SizeMaximum Fragment SizeThe maximum size of each split file, including header lines. NOTE: in the case where a single line exceeds this property (including headers, if applicable), that line will be output in a split of its own which exceeds this Maximum Fragment Size setting.falsefalsefalseNONEfalsefalseHeader Line CountHeader Line CountThe number of lines that should be considered part of the header; the header lines will be duplicated to all split files0truefalsefalseNONEfalsefalseHeader Line Marker CharactersHeader Line Marker CharactersThe first character(s) on the line of the datafile which signifies a header line. This value is ignored when Header Line Count is non-zero. The first line not containing the Header Line Marker Characters and all subsequent lines are considered non-headerfalsefalsefalseNONEfalsefalseRemove Trailing NewlinesRemove Trailing NewlinesWhether to remove newlines at the end of each split file. This should be false if you intend to merge the split files later. If this is set to 'true' and a FlowFile is generated that contains only 'empty lines' (i.e., consists only of \r and \n characters), the FlowFile will not be emitted. Note, however, that if header lines are specified, the resultant FlowFile will never be empty as it will consist of the header lines, so a FlowFile may be emitted that contains only the header lines.truetruetruefalsefalsetruefalsefalseNONEfalsefalsefailureIf a file cannot be split for some reason, the original file will be routed to this destination and nothing will be routed elsewherefalseoriginalThe original input file will be routed to this destination when it has been successfully split into 1 or more filesfalsesplitsThe split files will be routed to this destination when an input file is successfully split into 1 or more split filesfalsetext.line.countThe number of lines of text from the original FlowFile that were copied to this FlowFilefragment.sizeThe number of bytes from the original FlowFile that were copied to this FlowFile, including header, if applicable, which is duplicated in each split FlowFilefragment.identifierAll split FlowFiles produced from the same parent FlowFile will have the same randomly generated UUID added for this attributefragment.indexA one-up number that indicates the ordering of the split FlowFiles that were created from a single parent FlowFilefragment.countThe number of split FlowFiles generated from the parent FlowFilesegment.original.filename The filename of the parent FlowFiletruetrueINPUT_REQUIREDMEMORYThe FlowFile with its attributes is stored in memory, not the content of the FlowFile. If many splits are generated due to the size of the content, or how the content is configured to be split, a two-phase approach may be necessary to avoid excessive use of memory.org.apache.nifi.processors.standard.MergeContentorg.apache.nifi.processors.standard.SplitXmlPROCESSORSplits an XML File into multiple separate FlowFiles, each comprising a child or descendant of the original root elementxmlsplitSplit DepthSplit DepthIndicates the XML-nesting depth to start splitting XML fragments. A depth of 1 means split the root's children, whereas a depth of 2 means split the root's children's children and so forth.1truefalsefalseNONEfalsefalsefailureIf a FlowFile fails processing for any reason (for example, the FlowFile is not valid XML), it will be routed to this relationshipfalseoriginalThe original FlowFile that was split into segments. If the FlowFile fails processing, nothing will be sent to this relationshipfalsesplitAll segments of the original FlowFile will be routed to this relationshipfalsefragment.identifierAll split FlowFiles produced from the same parent FlowFile will have the same randomly generated UUID added for this attributefragment.indexA one-up number that indicates the ordering of the split FlowFiles that were created from a single parent FlowFilefragment.countThe number of split FlowFiles generated from the parent FlowFilesegment.original.filename The filename of the parent FlowFiletruetrueINPUT_REQUIREDMEMORYThe entirety of the FlowFile's content (as a Document object) is read into memory, in addition to all of the generated FlowFiles representing the split XML. A Document object can take approximately 10 times as much memory as the size of the XML. For example, a 1 MB XML document may use 10 MB of memory. If many splits are generated due to the size of the XML, a two-phase approach may be necessary to avoid excessive use of memory.org.apache.nifi.processors.standard.TailFilePROCESSOR"Tails" a file, or a list of files, ingesting data from the file as it is written to the file. The file is expected to be textual. Data is ingested only when a new line is encountered (carriage return or new-line character or combination). If the file to tail is periodically "rolled over", as is generally the case with log files, an optional Rolling Filename Pattern can be used to retrieve data from files that have rolled over, even if the rollover occurred while NiFi was not running (provided that the data still exists upon restart of NiFi). It is generally advisable to set the Run Schedule to a few seconds, rather than running with the default value of 0 secs, as this Processor will consume a lot of resources if scheduled very aggressively. At this time, this Processor does not support ingesting files that have been compressed when 'rolled over'.tailfilelogtextsourcetail-modeTailing modeMode to use: single file will tail only one file, multiple file will look for a list of file. In Multiple mode the Base directory is required.Single fileSingle fileSingle fileIn this mode, only the one file indicated in the 'Files to tail' property will be watched by the processor. In this mode, the file may not exist when starting the processor.Multiple filesMultiple filesIn this mode, the 'Files to tail' property accepts a regular expression and the processor will look for files in 'Base directory' to list the files to tail by the processor.truefalsefalseNONEfalsefalseFile to TailFile(s) to TailPath of the file to tail in case of single file mode. If using multifile mode, regular expression to find files to tail in the base directory. In case recursivity is set to true, the regular expression will be used to match the path starting from the base directory (see additional details for examples).truefalsetrueENVIRONMENTfalsefalseRolling Filename PatternRolling Filename PatternIf the file to tail "rolls over" as would be the case with log files, this filename pattern will be used to identify files that have rolled over so that if NiFi is restarted, and the file has rolled over, it will be able to pick up where it left off. This pattern supports wildcard characters * and ?, it also supports the notation ${filename} to specify a pattern based on the name of the file (without extension), and will assume that the files that have rolled over live in the same directory as the file being tailed. The same glob pattern will be used for all files.falsefalsefalseNONEfalsefalsePost-Rollover Tail PeriodPost-Rollover Tail PeriodWhen a file is rolled over, the processor will continue tailing the rolled over file until it has not been modified for this amount of time. This allows for another process to rollover a file, and then flush out any buffered data. Note that when this value is set, and the tailed file rolls over, the new file will not be tailed until the old file has not been modified for the configured amount of time. Additionally, when using this capability, in order to avoid data duplication, this period must be set longer than the Processor's Run Schedule, and the Processor must not be stopped after the file being tailed has been rolled over and before the data has been fully consumed. Otherwise, the data may be duplicated, as the entire file may be written out as the contents of a single FlowFile.0 secfalsefalsefalseNONEfalsefalsetail-base-directoryBase directoryBase directory used to look for files to tail. This property is required when using Multifile mode.falsefalsetrueENVIRONMENTfalsefalseInitial Start PositionInitial Start PositionWhen the Processor first begins to tail data, this property specifies where the Processor should begin reading data. Once data has been ingested from a file, the Processor will continue from the last point from which it has received data.Beginning of FileBeginning of TimeBeginning of TimeStart with the oldest data that matches the Rolling Filename Pattern and then begin reading from the File to TailBeginning of FileBeginning of FileStart with the beginning of the File to Tail. Do not ingest any data that has already been rolled overCurrent TimeCurrent TimeStart with the data at the end of the File to Tail. Do not ingest any data thas has already been rolled over or any data in the File to Tail that has already been written.truefalsefalseNONEfalsefalseFile LocationState LocationSpecifies where the state is located either local or cluster so that state can be stored appropriately in order to ensure that all data is consumed without duplicating data upon restart of NiFiLocalLocalLocalState is stored locally. Each node in a cluster will tail a different file.RemoteRemoteState is located on a remote resource. This Processor will store state across the cluster so that it can be run on Primary Node Only and a new Primary Node can pick up where the last one left off.truefalsefalseNONEfalsefalsetailfile-recursive-lookupRecursive lookupWhen using Multiple files mode, this property defines if files must be listed recursively or not in the base directory.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsetailfile-lookup-frequencyLookup frequencyOnly used in Multiple files mode. It specifies the minimum duration the processor will wait before listing again the files to tail.10 minutesfalsefalsefalseNONEfalsefalsetailfile-maximum-ageMaximum ageOnly used in Multiple files mode. It specifies the necessary minimum duration to consider that no new messages will be appended in a file regarding its last modification date. This should not be set too low to avoid duplication of data in case new messages are appended at a lower frequency.24 hoursfalsefalsefalseNONEfalsefalsereread-on-nulReread when NUL encounteredIf this option is set to 'true', when a NUL character is read, the processor will yield and try to read the same part again later. (Note: Yielding may delay the processing of other files tailed by this processor, not just the one with the NUL character.) The purpose of this flag is to allow users to handle cases where reading a file may return temporary NUL values. NFS for example may send file contents out of order. In this case the missing parts are temporarily replaced by NUL values. CAUTION! If the file contains legitimate NUL values, setting this flag causes this processor to get stuck indefinitely. For this reason users should refrain from using this feature if they can help it and try to avoid having the target file on a file system where reads are unreliable.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseLine Start PatternLine Start PatternA Regular Expression to match against the start of a log line. If specified, any line that matches the expression, and any following lines, will be buffered until another line matches the Expression. In doing this, we can avoid splitting apart multi-line messages in the file. This assumes that the data is in UTF-8 format.falsefalsefalseNONEfalsefalsetail-modeTailing modeSingle filepre-allocated-buffer-sizePre-Allocated Buffer SizeSets the amount of memory that is pre-allocated for each tailed file.65536 BtruefalsefalseNONEfalsefalseMax Buffer SizeMax Buffer SizeWhen using the Line Start Pattern, there may be situations in which the data in the file being tailed never matches the Regular Expression. This would result in the processor buffering all data from the tailed file, which can quickly exhaust the heap. To avoid this, the Processor will buffer only up to this amount of data before flushing the buffer, even if it means ingesting partial data from the file.64 KBtruefalsefalseNONEfalsefalseLine Start PatternLine Start PatternsuccessAll FlowFiles are routed to this Relationship.falsetailfile.original.pathPath of the original file the flow file comes from.trueStores state about where in the Tailed File it left off so that on restart it does not have to duplicate data. State is stored either local or clustered depend on the <File Location> property.LOCALCLUSTERread filesystemProvides operator the ability to read from any file that NiFi has access to.INPUT_FORBIDDENTIMER_DRIVEN30 sec1org.apache.nifi.processors.standard.TransformXmlPROCESSORApplies the provided XSLT file to the FlowFile XML payload. A new FlowFile is created with transformed content and is routed to the 'success' relationship. If the XSL transform fails, the original FlowFile is routed to the 'failure' relationshipxmlxslttransformXSLT file nameXSLT file nameProvides the name (including full path) of the XSLT file to apply to the FlowFile XML content.One of the 'XSLT file name' and 'XSLT Lookup' properties must be defined.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSINGLEFILExslt-controllerXSLT LookupController lookup used to store XSLT definitions. One of the 'XSLT file name' and 'XSLT Lookup' properties must be defined. WARNING: note that the lookup controller service should not be used to store large XSLT files.org.apache.nifi.lookup.StringLookupServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalsexslt-controller-keyXSLT Lookup keyKey used to retrieve the XSLT definition from the XSLT lookup controller. This property must be set when using the XSLT controller property.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseindent-outputIndentWhether or not to indent the output.truetruetruefalsefalsetruefalsefalseNONEfalsefalsesecure-processingSecure processingWhether or not to mitigate various XML-related attacks like XXE (XML External Entity) attacks.truetruetruefalsefalsetruefalsefalseNONEfalsefalsecache-sizeCache sizeMaximum number of stylesheets to cache. Zero disables the cache.10truefalsefalseNONEfalsefalsecache-ttl-after-last-accessCache TTL after last accessThe cache TTL (time-to-live) or how long to keep stylesheets in the cache after last access.60 secstruefalsefalseNONEfalsefalseAn XSLT transform parameter nameAn XSLT transform parameter valueThese XSLT parameters are passed to the transformerFLOWFILE_ATTRIBUTESsuccessThe FlowFile with transformed content will be routed to this relationshipfalsefailureIf a FlowFile fails processing for any reason (for example, the FlowFile is not valid XML), it will be routed to this relationshipfalsetruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.UnpackContentPROCESSORUnpacks the content of FlowFiles that have been packaged with one of several different Packaging Formats, emitting one to many FlowFiles for each input FlowFile. Supported formats are TAR, ZIP, and FlowFile Stream packages.Unpackun-mergetarziparchiveflowfile-streamflowfile-stream-v3Packaging FormatPackaging FormatThe Packaging Format used to create the fileuse mime.type attributeuse mime.type attributeuse mime.type attributetartarzipzipflowfile-stream-v3flowfile-stream-v3flowfile-stream-v2flowfile-stream-v2flowfile-tar-v1flowfile-tar-v1truefalsefalseNONEfalsefalseFilename Character SetFilename Character SetIf supplied this character set will be supplied to the Zip utility to attempt to decode filenames using the specific character set. If not specified the default platform character set will be used. This is useful if a Zip was created with a different character set than the platform default and the zip uses non standard values to specify.UTF-8falsefalsefalseNONEfalsefalsePackaging FormatPackaging Formatzipuse mime.type attributeFile FilterFile FilterOnly files contained in the archive whose names match the given regular expression will be extracted (tar/zip only).*truefalsefalseNONEfalsefalsePasswordPasswordPassword used for decrypting Zip archives encrypted with ZipCrypto or AES. Configuring a password disables support for alternative Zip compression algorithms.falsetruefalseNONEfalsefalseallow-stored-entries-wddAllow Stored Entries With Data DescriptorSome zip archives contain stored entries with data descriptors which by spec should not happen. If this property is true they will be read anyway. If false and such an entry is discovered the zip will fail to process.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsePackaging FormatPackaging FormatzipsuccessUnpacked FlowFiles are sent to this relationshipfalsefailureThe original FlowFile is sent to this relationship when it cannot be unpacked for some reasonfalseoriginalThe original FlowFile is sent to this relationship after it has been successfully unpackedfalsemime.typeIf the <Packaging Format> property is set to use mime.type attribute, this attribute is used to determine the FlowFile's MIME Type. In this case, if the attribute is set to application/tar, the TAR Packaging Format will be used. If the attribute is set to application/zip, the ZIP Packaging Format will be used. If the attribute is set to application/flowfile-v3 or application/flowfile-v2 or application/flowfile-v1, the appropriate FlowFile Packaging Format will be used. If this attribute is missing, the FlowFile will be routed to 'failure'. Otherwise, if the attribute's value is not one of those mentioned above, the FlowFile will be routed to 'success' without being unpacked. Use the File Filter property only extract files matching a specific regular expression.mime.typeIf the FlowFile is successfully unpacked, its MIME Type is no longer known, so the mime.type attribute is set to application/octet-stream.fragment.identifierAll unpacked FlowFiles produced from the same parent FlowFile will have the same randomly generated UUID added for this attributefragment.indexA one-up number that indicates the ordering of the unpacked FlowFiles that were created from a single parent FlowFilefragment.countThe number of unpacked FlowFiles generated from the parent FlowFilesegment.original.filename The filename of the parent FlowFile. Extensions of .tar, .zip or .pkg are removed because the MergeContent processor automatically adds those extensions if it is used to rebuild the original FlowFilefile.lastModifiedTimeThe date and time that the unpacked file was last modified (tar only).file.creationTimeThe date and time that the file was created. This attribute holds always the same value as file.lastModifiedTime (tar only).file.ownerThe owner of the unpacked file (tar only)file.groupThe group owner of the unpacked file (tar only)file.permissionsThe read/write/execute permissions of the unpacked file (tar only)file.encryptionMethodThe encryption method for entries in Zip archivestruetrueINPUT_REQUIREDUnpack Zip containing filenames with special characters, created on Windows with filename charset 'Cp437' or 'IBM437'.INPUT_ALLOWEDSet "Packaging Format" value to "zip" or "use mime.type attribute". Set "Filename Character Set" value to "Cp437" or "IBM437". org.apache.nifi.processors.standard.MergeContentorg.apache.nifi.processors.standard.UpdateCounterPROCESSORThis processor allows users to set specific counters and key points in their flow. It is useful for debugging and basic counting functions.counterdebuginstrumentationcounter-nameCounter NameThe name of the counter you want to set the value of - supports expression language like ${counterName}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsedeltaDeltaAdjusts the counter by the specified delta for each flow file received. May be a positive or negative integer.1truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessCounter was updated/retrievedfalsecounterNameThe name of the counter to update/get.INPUT_REQUIREDorg.apache.nifi.processors.standard.UpdateDatabaseTablePROCESSORThis processor uses a JDBC connection and incoming records to generate any database table changes needed to support the incoming records. It expects a 'flat' record layout, meaning none of the top-level record fields has nested fields that are intended to become columns themselves.metadatajdbcdatabasetableupdatealterrecord-readerRecord ReaderThe service for reading incoming flow files. The reader is only used to determine the schema of the records, the actual records will not be processed.org.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseupdatedatabasetable-dbcp-serviceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain connection(s) to the databaseorg.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalsedb-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses.GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLfalsefalsefalseNONEfalsefalseupdatedatabasetable-catalog-nameCatalog NameThe name of the catalog that the statement should update. This may not apply for the database that you are updating. In this case, leave the field empty. Note that if the property is set and the database is case-sensitive, the catalog name must match the database's catalog name exactly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseupdatedatabasetable-schema-nameSchema NameThe name of the database schema that the table belongs to. This may not apply for the database that you are updating. In this case, leave the field empty. Note that if the property is set and the database is case-sensitive, the schema name must match the database's schema name exactly.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseupdatedatabasetable-table-nameTable NameThe name of the database table to update. If the table does not exist, then it will either be created or an error thrown, depending on the value of the Create Table property.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseupdatedatabasetable-create-tableCreate Table StrategySpecifies how to process the target table when it does not exist (create it, fail, e.g.).Fail If Not ExistsCreate If Not ExistsCreate If Not ExistsCreate a table with the given schema if it does not already existFail If Not ExistsFail If Not ExistsIf the target does not already exist, log an error and route the flowfile to failuretruefalsefalseNONEfalsefalseupdatedatabasetable-primary-keysPrimary Key FieldsA comma-separated list of record field names that uniquely identifies a row in the database. This property is only used if the specified table needs to be created, in which case the Primary Key Fields will be used to specify the primary keys of the newly-created table. IMPORTANT: Primary Key Fields must match the record field names exactly unless 'Quote Column Identifiers' is false and the database allows for case-insensitive column names. In practice it is best to specify Primary Key Fields that exactly match the record field names, and those will become the column names in the created table.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseupdatedatabasetable-create-tableCreate Table StrategyCreate If Not Existsupdatedatabasetable-translate-field-namesTranslate Field NamesIf true, the Processor will attempt to translate field names into the corresponding column names for the table specified, for the purposes of determining whether the field name exists as a column in the target table. NOTE: If the target table does not exist and is to be created, this property is ignored and the field names will be used as-is. If false, the field names must match the column names exactly, or the column may not be found and instead an error my be reported that the column already exists.truetruetruefalsefalsetruefalsefalseNONEfalsefalseupdatedatabasetable-update-field-namesUpdate Field NamesThis property indicates whether to update the output schema such that the field names are set to the exact column names from the specified table. This should be used if the incoming record field names may not match the table's column names in terms of upper- and lower-case. For example, this property should be set to true if the output FlowFile is destined for Oracle e.g., which expects the field names to match the column names exactly. NOTE: The value of the 'Translate Field Names' property is ignored when updating field names; instead they are updated to match the column name as returned by the database.falsetruetruefalsefalsetruefalsefalseNONEfalsefalseupdatedatabasetable-record-writerRecord WriterSpecifies the Controller Service to use for writing results to a FlowFile. The Record Writer should use Inherit Schema to emulate the inferred schema behavior, i.e. an explicit schema need not be defined in the writer, and will be supplied by the same logic used to infer the schema from the column types. If Create Table Strategy is set 'Create If Not Exists', the Record Writer's output format must match the Record Reader's format in order for the data to be placed in the created table location. Note that this property is only used if 'Update Field Names' is set to true and the field names do not all match the column names exactly. If no update is needed for any field names (or 'Update Field Names' is false), the Record Writer is not used and instead the input FlowFile is routed to success or failure without modification.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseupdatedatabasetable-update-field-namesUpdate Field Namestrueupdatedatabasetable-quoted-table-identifiersQuote Table IdentifiersEnabling this option will cause the table name to be quoted to support the use of special characters in the table name and/or forcing the value of the Table Name property to match the target table name exactly.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseupdatedatabasetable-quoted-column-identifiersQuote Column IdentifiersEnabling this option will cause all column names to be quoted, allowing you to use reserved words as column names in your tables and/or forcing the record field names to match the column names exactly.falsetruetruefalsefalsefalsefalsefalseNONEfalsefalseupdatedatabasetable-query-timeoutQuery TimeoutSets the number of seconds the driver will wait for a query to execute. A value of 0 means no timeout. NOTE: Non-zero values may not be supported by the driver.0truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesuccessA FlowFile containing records routed to this relationship after the record has been successfully transmitted to the database.falsefailureA FlowFile containing records routed to this relationship if the record could not be transmitted to the database.falseoutput.tableThis attribute is written on the flow files routed to the 'success' and 'failure' relationships, and contains the target table name.output.pathThis attribute is written on the flow files routed to the 'success' and 'failure' relationships, and contains the path on the file system to the table (or partition location if the table is partitioned).mime.typeSets the mime.type attribute to the MIME Type specified by the Record Writer, only if a Record Writer is specified and Update Field Names is 'true'.record.countSets the number of records in the FlowFile, only if a Record Writer is specified and Update Field Names is 'true'.INPUT_REQUIREDorg.apache.nifi.processors.standard.UpdateRecordPROCESSORUpdates the contents of a FlowFile that contains Record-oriented data (i.e., data that can be read via a RecordReader and written by a RecordWriter). This Processor requires that at least one user-defined Property be added. The name of the Property should indicate a RecordPath that determines the field that should be updated. The value of the Property is either a replacement value (optionally making use of the Expression Language) or is itself a RecordPath that extracts a value from the Record. Whether the Property value is determined to be a RecordPath or a literal value depends on the configuration of the <Replacement Value Strategy> Property.updaterecordgenericschemajsoncsvavrologlogsfreeformtextRecord ReaderRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseRecord WriterRecord WriterSpecifies the Controller Service to use for writing out the recordsorg.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseReplacement Value StrategyReplacement Value StrategySpecifies how to interpret the configured replacement valuesliteral-valueLiteral Valueliteral-valueThe value entered for a Property (after Expression Language has been evaluated) is the desired value to update the Record Fields with. Expression Language may reference variables 'field.name', 'field.type', and 'field.value' to access information about the field and the value of the field being evaluated.Record Path Valuerecord-path-valueThe value entered for a Property (after Expression Language has been evaluated) is not the literal value to use but rather is a Record Path that should be evaluated against the Record, and the result of the RecordPath will be used to update the Record. Note that if this option is selected, and the Record Path results in multiple values for a given Record, the input FlowFile will be routed to the 'failure' Relationship.truefalsefalseNONEfalsefalseA RecordPath.The value to use to replace fields in the record that match the RecordPathAllows users to specify values to use to replace fields in the record that match the RecordPath.FLOWFILE_ATTRIBUTESfailureIf a FlowFile cannot be transformed from the configured input format to the configured output format, the unchanged FlowFile will be routed to this relationshipfalsesuccessFlowFiles that are successfully transformed will be routed to this relationshipfalserecord.indexThis attribute provides the current row index and is only available inside the literal value expression.record.error.messageThis attribute provides on failure the error message encountered by the Reader or Writer.truetrueINPUT_REQUIREDCombine multiple fields into a single field.combineconcatenaterecordpathINPUT_ALLOWED"Replacement Value Strategy" = "Record Path Value" A single additional property is added to the Processor. The name of the property is a RecordPath identifying the field to place the result in. The value of the property uses the CONCAT Record Path function to concatenate multiple values together, potentially using other string literal values. For example, to combine the `title`, `firstName` and `lastName` fields into a single field named `fullName`, we add a property with the name `/fullName` and a value of `CONCAT(/title, ' ', /firstName, ' ', /lastName)` Change the value of a record field to an explicit value.changeupdatereplacetransformINPUT_ALLOWED "Replacement Value Strategy" = "Literal Value" A single additional property is added to the Processor. The name of the property is a RecordPath identifying the field to place the result in. The value of the property is the explicit value to set the field to. For example, we can set any field with a name of `txId`, regardless of its level in the data's hierarchy, to `1111-1111` by adding a property with a name of `//txId` and a value of `1111-1111` Copy the value of one record field to another record field.changeupdatecopyrecordpathhierarchytransformINPUT_ALLOWED "Replacement Value Strategy" = "Record Path Value" A single additional property is added to the Processor. The name of the property is a RecordPath identifying the field to update. The value of the property is a RecordPath identifying the field to copy the value from. For example, we can copy the value of `/identifiers/all/imei` to the `identifier` field at the root level, by adding a property named `/identifier` with a value of `/identifiers/all/imei`. Enrich data by injecting the value of an attribute into each Record.enrichattributechangeupdatereplaceinserttransformINPUT_ALLOWED"Replacement Value Strategy" = "Literal Value" A single additional property is added to the Processor. The name of the property is a RecordPath identifying the field to place the result in. The value of the property is an Expression Language expression that references the attribute of interest. We can, for example, insert a new field name `filename` into each record by adding a property named `/filename` with a value of `${filename}` Change the format of a record field's value.Use the RenameRecordField Processor in order to change a field's name.changeupdatereplaceinserttransformformatdate/timetimezoneexpression languageINPUT_ALLOWED"Replacement Value Strategy" = "Literal Value" A single additional property is added to the Processor. The name of the property is a RecordPath identifying the field to update. The value is an Expression Language expression that references the `field.value` variable. For example, to change the date/time format of a field named `txDate` from `year-month-day` format to `month/day/year` format, we add a property named `/txDate` with a value of `${field.value:toDate('yyyy-MM-dd'):format('MM/dd/yyyy')}`. We could also change the timezone of a timestamp field (and insert the timezone for clarity) by using a value of `${field.value:toDate('yyyy-MM-dd HH:mm:ss', 'UTC-0400'):format('yyyy-MM-dd HH:mm:ss Z', 'UTC')}`. org.apache.nifi.processors.standard.ConvertRecordorg.apache.nifi.processors.standard.ValidateCsvPROCESSORValidates the contents of FlowFiles against a user-specified CSV schema. Take a look at the additional documentation of this processor for some schema examples.csvschemavalidationvalidate-csv-schemaSchemaThe schema to be used for validation. Is expected a comma-delimited string representing the cell processors to apply. The following cell processors are allowed in the schema definition: [ParseBigDecimal, ParseBool, ParseChar, ParseDate, ParseDouble, ParseInt, ParseLong, Optional, DMinMax, Equals, ForbidSubStr, LMinMax, NotNull, Null, RequireHashCode, RequireSubStr, Strlen, StrMinMax, StrNotNullOrEmpty, StrRegEx, Unique, UniqueHashCode, IsIncludedIn]. Note: cell processors cannot be nested except with Optional.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsevalidate-csv-headerHeaderTrue if the incoming flow file contains a header to ignore, false otherwise.truetruetruefalsefalsetruefalsefalseNONEfalsefalsevalidate-csv-delimiterDelimiter characterCharacter used as 'delimiter' in the incoming data. Example: ,,truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsevalidate-csv-quoteQuote characterCharacter used as 'quote' in the incoming data. Example: ""truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsevalidate-csv-eolEnd of line symbolsSymbols used as 'end of line' in the incoming data. Example: \n\ntruefalsetrueFLOWFILE_ATTRIBUTESfalsefalsevalidate-csv-strategyValidation strategyStrategy to apply when routing input files to output relationships.FlowFile validationLine by line validationLine by line validationIn case an error is found, the input CSV file will be split into two FlowFiles: one routed to the 'valid' relationship containing all the correct lines and one routed to the 'invalid' relationship containing all the incorrect lines. Take care if choosing this option while using Unique cell processors in schema definition:the first occurrence will be considered valid and the next ones as invalid.FlowFile validationFlowFile validationAs soon as an error is found in the CSV file, the validation will stop and the whole flow file will be routed to the 'invalid' relationship. This option offers best performances.truefalsefalseNONEfalsefalsevalidate-csv-violationsInclude all violationsIf true, the validation.error.message attribute would include the list of all the violations for the first invalid line. Note that setting this property to true would slightly decrease the performances as all columns would be validated. If false, a line is invalid as soon as a column is found violating the specified constraint and only this violation for the first invalid line will be included in the validation.error.message attribute.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsevalidFlowFiles that are successfully validated against the schema are routed to this relationshipfalseinvalidFlowFiles that are not valid according to the specified schema are routed to this relationshipfalsecount.valid.linesIf line by line validation, number of valid lines extracted from the source datacount.invalid.linesIf line by line validation, number of invalid lines extracted from the source datacount.total.linesIf line by line validation, total number of lines in the source datavalidation.error.messageFor flow files routed to invalid, message of the first validation errortruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ValidateJsonPROCESSORValidates the contents of FlowFiles against a configurable JSON Schema. See json-schema.org for specification standards. This Processor does not support input containing multiple JSON objects, such as newline-delimited JSON. If the input FlowFile contains newline-delimited JSON, only the first line will be validated.JSONschemavalidationSchema Access StrategySchema Access StrategySpecifies how to obtain the schema that is to be used for interpreting the data.SCHEMA_CONTENT_PROPERTYSchema Name PropertySCHEMA_NAME_PROPERTYThe name of the Schema to use is specified by the 'Schema Name' Property. The value of this property is used to lookup the Schema in the configured JSON Schema Registry Service.JSON Schema PropertySCHEMA_CONTENT_PROPERTYA URL or file path to the JSON schema or the actual JSON schema is specified by the 'JSON Schema' Property. No matter how the JSON schema is specified, it must be a valid JSON schematruefalsefalseNONEfalsefalseSchema NameSchema NameSpecifies the name of the schema to lookup in the Schema Registry property${schema.name}truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseSchema Access StrategySchema Access StrategySCHEMA_NAME_PROPERTYJSON Schema RegistryJSON Schema RegistrySpecifies the Controller Service to use for the JSON Schema Registryorg.apache.nifi.schemaregistry.services.JsonSchemaRegistryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseSchema Access StrategySchema Access StrategySCHEMA_NAME_PROPERTYJSON SchemaJSON SchemaA URL or file path to the JSON schema or the actual JSON schema contenttruefalsefalseNONEfalsefalseSINGLEURLFILETEXTSchema Access StrategySchema Access StrategySCHEMA_CONTENT_PROPERTYJSON Schema VersionJSON Schema VersionThe JSON schema specificationDRAFT_2020_12Draft 4DRAFT_4Draft Version 4Draft 6DRAFT_6Draft Version 6Draft 7DRAFT_7Draft Version 7Draft 2019-09DRAFT_2019_09Draft Version 2019-09Draft 2020-12DRAFT_2020_12Draft Version 2020-12truefalsefalseNONEfalsefalseSchema Access StrategySchema Access StrategySCHEMA_CONTENT_PROPERTYvalidFlowFiles that are successfully validated against the schema are routed to this relationshipfalseinvalidFlowFiles that are not valid according to the specified schema are routed to this relationshipfalsefailureFlowFiles that cannot be read as JSON are routed to this relationshipfalsejson.validation.errorsIf the flow file is routed to the invalid relationship , this attribute will contain the error message resulting from the validation failure.truetruereference remote resourcesSchema configuration can reference resources over HTTPINPUT_REQUIREDMEMORYValidating JSON requires reading FlowFile content into memoryorg.apache.nifi.processors.standard.ValidateRecordPROCESSORValidates the Records of an incoming FlowFile against a given schema. All records that adhere to the schema are routed to the "valid" relationship while records that do not adhere to the schema are routed to the "invalid" relationship. It is therefore possible for a single incoming FlowFile to be split into two individual FlowFiles if some records are valid according to the schema and others are not. Any FlowFile that is routed to the "invalid" relationship will emit a ROUTE Provenance Event with the Details field populated to explain why records were invalid. In addition, to gain further explanation of why records were invalid, DEBUG-level logging can be enabled for the "org.apache.nifi.processors.standard.ValidateRecord" logger.recordschemavalidaterecord-readerRecord ReaderSpecifies the Controller Service to use for reading incoming dataorg.apache.nifi.serialization.RecordReaderFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalserecord-writerRecord WriterSpecifies the Controller Service to use for writing out the records. Regardless of the Controller Service schema access configuration, the schema that is used to validate record is used to write the valid results.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseinvalid-record-writerRecord Writer for Invalid RecordsIf specified, this Controller Service will be used to write out any records that are invalid. If not specified, the writer specified by the "Record Writer" property will be used with the schema used to read the input records. This is useful, for example, when the configured Record Writer cannot write data that does not adhere to its schema (as is the case with Avro) or when it is desirable to keep invalid records in their original format while converting valid records to another format.org.apache.nifi.serialization.RecordSetWriterFactoryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseschema-access-strategySchema Access StrategySpecifies how to obtain the schema that should be used to validate recordsreader-schemaUse Reader's Schemareader-schemaThe schema to validate the data against is determined by asking the configured Record Reader for its schemaUse Schema Name Propertyschema-name-propertyThe schema to validate the data against is determined by looking at the 'Schema Name' Property and looking up the schema in the configured Schema RegistryUse Schema Text Propertyschema-text-propertyThe schema to validate the data against is determined by looking at the 'Schema Text' Property and parsing the schema as an Avro schematruefalsefalseNONEfalsefalseschema-registrySchema RegistrySpecifies the Controller Service to use for the Schema Registry. This is necessary only if the Schema Access Strategy is set to "Use 'Schema Name' Property".org.apache.nifi.schemaregistry.services.SchemaRegistryorg.apache.nifinifi-standard-services-api-nar2.0.0-M4falsefalsefalseNONEfalsefalseschema-nameSchema NameSpecifies the name of the schema to lookup in the Schema Registry property${schema.name}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseschema-textSchema TextThe text of an Avro-formatted Schema${avro.schema}falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalseallow-extra-fieldsAllow Extra FieldsIf the incoming data has fields that are not present in the schema, this property determines whether or not the Record is valid. If true, the Record is still valid. If false, the Record will be invalid due to the extra fields.truetruetruefalsefalsetruefalsefalseNONEfalsefalsestrict-type-checkingStrict Type CheckingIf the incoming data has a Record where a field is not of the correct type, this property determines how to handle the Record. If true, the Record will be considered invalid. If false, the Record will be considered valid and the field will be coerced into the correct type (if possible, according to the type coercion supported by the Record Writer). This property controls how the data is validated against the validation schema.truetruetruefalsefalsetruefalsefalseNONEfalsefalsecoerce-typesForce Types From Reader's SchemaIf enabled, the processor will coerce every field to the type specified in the Reader's schema. If the value of a field cannot be coerced to the type, the field will be skipped (will not be read from the input data), thus will not appear in the output. If not enabled, then every field will appear in the output but their types may differ from what is specified in the schema. For details please see the Additional Details page of the processor's Help. This property controls how the data is read by the specified Record Reader.falsetruetruefalsefalsetruefalsefalseNONEfalsefalsevalidation-details-attribute-nameValidation Details Attribute NameIf specified, when a validation error occurs, this attribute name will be used to leave the details. The number of characters will be limited by the property 'Maximum Validation Details Length'.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsemaximum-validation-details-lengthMaximum Validation Details LengthSpecifies the maximum number of characters that validation details value can have. Any characters beyond the max will be truncated. This property is only used if 'Validation Details Attribute Name' is set1024falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsevalidRecords that are valid according to the schema will be routed to this relationshipfalseinvalidRecords that are not valid according to the schema will be routed to this relationshipfalsefailureIf the records cannot be read, validated, or written, for any reason, the original FlowFile will be routed to this relationshipfalsemime.typeSets the mime.type attribute to the MIME Type specified by the Record Writerrecord.countThe number of records in the FlowFile routed to a relationshiptruetrueINPUT_REQUIREDorg.apache.nifi.processors.standard.ValidateXmlPROCESSORValidates XML contained in a FlowFile. By default, the XML is contained in the FlowFile content. If the 'XML Source Attribute' property is set, the XML to be validated is contained in the specified attribute. It is not recommended to use attributes to hold large XML documents; doing so could adversely affect system performance. Full schema validation is performed if the processor is configured with the XSD schema details. Otherwise, the only validation performed is to ensure the XML syntax is correct and well-formed, e.g. all opening tags are properly closed.xmlschemavalidationxsdSchema FileSchema FileThe file path or URL to the XSD Schema file that is to be used for validation. If this property is blank, only XML syntax/structure will be validated.falsefalsetrueENVIRONMENTfalsefalseSINGLEURLFILEXML Source AttributeXML Source AttributeThe name of the attribute containing XML to be validated. If this property is blank, the FlowFile content will be validated.falsefalsetrueENVIRONMENTfalsefalsevalidFlowFiles that are successfully validated against the schema, if provided, or verified to be well-formed XML are routed to this relationshipfalseinvalidFlowFiles that are not valid according to the specified schema or contain invalid XML are routed to this relationshipfalsevalidatexml.invalid.errorIf the flow file is routed to the invalid relationship the attribute will contain the error message resulting from the validation failure.truetruereference remote resourcesSchema configuration can reference resources over HTTPINPUT_REQUIREDMEMORYWhile this processor supports processing XML within attributes, it is strongly discouraged to hold large amounts of data in attributes. In general, attribute values should be as small as possible and hold no more than a couple hundred characters.org.apache.nifi.processors.standard.WaitPROCESSORRoutes incoming FlowFiles to the 'wait' relationship until a matching release signal is stored in the distributed cache from a corresponding Notify processor. When a matching release signal is identified, a waiting FlowFile is routed to the 'success' relationship. The release signal entry is then removed from the cache. The attributes of the FlowFile that produced the release signal are copied to the waiting FlowFile if the Attribute Cache Regex property of the corresponding Notify processor is set properly. If there are multiple release signals in the cache identified by the Release Signal Identifier, and the Notify processor is configured to copy the FlowFile attributes to the cache, then the FlowFile passing the Wait processor receives the union of the attributes of the FlowFiles that produced the release signals in the cache (identified by Release Signal Identifier). Waiting FlowFiles will be routed to 'expired' if they exceed the Expiration Duration. If you need to wait for more than one signal, specify the desired number of signals via the 'Target Signal Count' property. This is particularly useful with processors that split a source FlowFile into multiple fragments, such as SplitText. In order to wait for all fragments to be processed, connect the 'original' relationship to a Wait processor, and the 'splits' relationship to a corresponding Notify processor. Configure the Notify and Wait processors to use the '${fragment.identifier}' as the value of 'Release Signal Identifier', and specify '${fragment.count}' as the value of 'Target Signal Count' in the Wait processor.It is recommended to use a prioritizer (for instance First In First Out) when using the 'wait' relationship as a loop.mapcachewaitholddistributedsignalreleaserelease-signal-idRelease Signal IdentifierA value that specifies the key to a specific release signal cache. To decide whether the FlowFile that is being processed by the Wait processor should be sent to the 'success' or the 'wait' relationship, the processor checks the signals in the cache specified by this key.truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsetarget-signal-countTarget Signal CountThe number of signals that need to be in the cache (specified by the Release Signal Identifier) in order for the FlowFile processed by the Wait processor to be sent to the ‘success’ relationship. If the number of signals in the cache has reached this number, the FlowFile is routed to the 'success' relationship and the number of signals in the cache is decreased by this value. If Signal Counter Name is specified, this processor checks a particular counter, otherwise checks against the total number of signals in the cache.1truefalsetrueFLOWFILE_ATTRIBUTESfalsefalsesignal-counter-nameSignal Counter NameWithin the cache (specified by the Release Signal Identifier) the signals may belong to different counters. If this property is specified, the processor checks the number of signals in the cache that belong to this particular counter. If not specified, the processor checks the total number of signals in the cache.falsefalsetrueFLOWFILE_ATTRIBUTESfalsefalsewait-buffer-countWait Buffer CountSpecify the maximum number of incoming FlowFiles that can be buffered to check whether it can move forward. The more buffer can provide the better performance, as it reduces the number of interactions with cache service by grouping FlowFiles by signal identifier. Only a signal identifier can be processed at a processor execution.1truefalsefalseNONEfalsefalsereleasable-flowfile-countReleasable FlowFile CountA value, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the releasable FlowFile count. This specifies how many FlowFiles can be released when a target count reaches target signal count. Zero (0) has a special meaning, any number of FlowFiles can be released as long as signal count matches target.1truefalsetrueFLOWFILE_ATTRIBUTESfalsefalseexpiration-durationExpiration DurationIndicates the duration after which waiting FlowFiles will be routed to the 'expired' relationship10 mintruefalsefalseNONEfalsefalsedistributed-cache-serviceDistributed Cache ServiceThe Controller Service that is used to check for release signals from a corresponding Notify processororg.apache.nifi.distributed.cache.client.AtomicDistributedMapCacheClientorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseattribute-copy-modeAttribute Copy ModeSpecifies how to handle attributes copied from FlowFiles entering the Notify processorkeeporiginalReplace if presentreplaceWhen cached attributes are copied onto released FlowFiles, they replace any matching attributes.Keep originalkeeporiginalAttributes on released FlowFiles are not overwritten by copied cached attributes.truefalsefalseNONEfalsefalsewait-modeWait ModeSpecifies how to handle a FlowFile waiting for a notify signalwaitTransfer to wait relationshipwaitTransfer a FlowFile to the 'wait' relationship when whose release signal has not been notified yet. This mode allows other incoming FlowFiles to be enqueued by moving FlowFiles into the wait relationship. It is recommended to set a prioritizer (for instance First In First Out) on the 'wait' relationship.Keep in the upstream connectionkeepTransfer a FlowFile to the upstream connection where it comes from when whose release signal has not been notified yet. This mode helps keeping upstream connection being full so that the upstream source processor will not be scheduled while back-pressure is active and limit incoming FlowFiles. truefalsefalseNONEfalsefalsewait-penalty-durationWait Penalty DurationIf configured, after a signal identifier got processed but did not meet the release criteria, the signal identifier is penalized and FlowFiles having the signal identifier will not be processed again for the specified period of time, so that the signal identifier will not block others to be processed. This can be useful for use cases where a Wait processor is expected to process multiple signal identifiers, and each signal identifier has multiple FlowFiles, and also the order of releasing FlowFiles is important within a signal identifier. The FlowFile order can be configured with Prioritizers. IMPORTANT: There is a limitation of number of queued signals can be processed, and Wait processor may not be able to check all queued signal ids. See additional details for the best practice.falsefalsefalseNONEfalsefalseexpiredA FlowFile that has exceeded the configured Expiration Duration will be routed to this relationshipfalsesuccessA FlowFile with a matching release signal in the cache will be routed to this relationshipfalsewaitA FlowFile with no matching release signal in the cache will be routed to this relationshipfalsefailureWhen the cache cannot be reached, or if the Release Signal Identifier evaluates to null or empty, FlowFiles will be routed to this relationshipfalsewait.start.timestampAll FlowFiles will have an attribute 'wait.start.timestamp', which sets the initial epoch timestamp when the file first entered this processor. This is used to determine the expiration time of the FlowFile. This attribute is not written when the FlowFile is transferred to failure, expired or successwait.counter.<counterName>The name of each counter for which at least one signal has been present in the cache since the last time the cache was empty gets copied to the current FlowFile as an attribute.trueINPUT_REQUIREDorg.apache.nifi.distributed.cache.client.DistributedMapCacheClientServiceorg.apache.nifi.distributed.cache.server.map.DistributedMapCacheServerorg.apache.nifi.processors.standard.Notifyorg.apache.nifi.controller.ControllerStatusReportingTaskREPORTING_TASKLogs the 5-minute stats that are shown in the NiFi Summary Page for Processors and Connections, as well optionally logging the deltas between the previous iteration and the current iteration. Processors' stats are logged using the org.apache.nifi.controller.ControllerStatusReportingTask.Processors logger, while Connections' stats are logged using the org.apache.nifi.controller.ControllerStatusReportingTask.Connections logger. These can be configured in the NiFi logging configuration to log to different files, if desired.statslogShow DeltasShow DeltasSpecifies whether or not to show the difference in values between the current status and the previous statustruetruetruefalsefalsetruefalsefalseNONEfalsefalsereporting-granularityReporting GranularityWhen reporting information, specifies the granularity of the metrics to reportfive-minutesFive Minutesfive-minutesThe stats that are reported will reflect up to the last 5 minutes' worth of processing, which will coincide with the stats that are shown in the UI.One Secondone-secondThe stats that are reported will be an average of the value per second, gathered over the last 5 minutes. This is essentially obtained by dividing the stats that are shown in the UI by 300 (300 seconds in 5 minutes), with the exception of when NiFi has been running for less than 5 minutes. In that case, the stats will be divided by the amount of time NiFi has been running.falsefalsefalseNONEfalsefalseorg.apache.nifi.controller.MonitorDiskUsageREPORTING_TASKChecks the amount of storage space available for the specified directory and warns (via a log message and a System-Level Bulletin) if the partition on which it lives exceeds some configurable threshold of storage spacediskstoragewarningmonitoringrepoThresholdThresholdThe threshold at which a bulletin will be generated to indicate that the disk usage of the partition on which the directory found is of concern80%truefalsefalseNONEfalsefalseDirectory LocationDirectory LocationThe directory path of the partition to be monitored.truefalsefalseNONEfalsefalseDirectory Display NameDirectory Display NameThe name to display for the directory in alerts.Un-NamedfalsefalsefalseNONEfalsefalseorg.apache.nifi.controller.MonitorMemoryREPORTING_TASKChecks the amount of Java Heap available in the JVM for a particular JVM Memory Pool. If the amount of space used exceeds some configurable threshold, will warn (via a log message and System-Level Bulletin) that the memory pool is exceeding this threshold.monitormemoryheapjvmgcgarbage collectionwarningMemory PoolMemory PoolThe name of the JVM Memory Pool to monitor. The allowed values for Memory Pools are platform and JVM dependent and may vary for different versions of Java and from published documentation. This reporting task will become invalidated if configured to use a Memory Pool that is not available on the currently running host platform and JVMG1 Old GenG1 Eden SpaceG1 Eden SpaceG1 Old GenG1 Old GenG1 Survivor SpaceG1 Survivor SpacetruefalsefalseNONEfalsefalseUsage ThresholdUsage ThresholdIndicates the threshold at which warnings should be generated. This can be a percentage or a Data Size65%truefalsefalseNONEfalsefalseReporting IntervalReporting IntervalIndicates how often this reporting task should report bulletins while the memory utilization exceeds the configured thresholdfalsefalsefalseNONEfalsefalseorg.apache.nifi.reporting.ganglia.StandardGangliaReporterREPORTING_TASKReports metrics to Ganglia so that Ganglia can be used for external monitoring of the application. Metrics reported include JVM Metrics (optional); the following 5-minute NiFi statistics: FlowFiles Received, Bytes Received, FlowFiles Sent, Bytes Sent, Bytes Read, Bytes Written, Total Task Duration; and the current values for FlowFiles Queued, Bytes Queued, and number of Active Threads.gangliastatsHostnameHostnameThe fully-qualified name of the host on which Ganglia is runninglocalhosttruefalsefalseNONEfalsefalsePortPortThe Port on which Ganglia is listening for incoming connections8649truefalsefalseNONEfalsefalseSend JVM MetricsSend JVM MetricsSpecifies whether or not JVM Metrics should be gathered and sent, in addition to NiFi-specific metricsfalsetruetruefalsefalsetruefalsefalseNONEfalsefalseorg.apache.nifi.parameter.DatabaseParameterProviderPARAMETER_PROVIDERFetches parameters from database tablesdatabasedbcpsqldb-typeDatabase TypeThe type/flavor of database, used for generating database-specific code. In many cases the Generic type should suffice, but some databases (such as Oracle) require custom SQL clauses. GenericGenericGenericGenerates ANSI SQLOracleOracleGenerates Oracle compliant SQLOracle 12+Oracle 12+Generates Oracle compliant SQL for version 12 or greaterMS SQL 2012+MS SQL 2012+Generates MS SQL Compatible SQL, for version 2012 or greaterMS SQL 2008MS SQL 2008Generates MS SQL Compatible SQL for version 2008MySQLMySQLGenerates MySQL compatible SQLPostgreSQLPostgreSQLGenerates PostgreSQL compatible SQLPhoenixPhoenixGenerates Phoenix compliant SQLtruefalsefalseNONEfalsefalsedbcp-serviceDatabase Connection Pooling ServiceThe Controller Service that is used to obtain a connection to the database.org.apache.nifi.dbcp.DBCPServiceorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseparameter-grouping-strategyParameter Grouping StrategyThe strategy used to group parameters.grouping-by-columnColumngrouping-by-columnA single table is partitioned by the 'Parameter Group Name Column'. All rows with the same value in this column will map to a group of the same name.Table Namegrouping-by-table-nameAn entire table maps to a Parameter Group. The group name will be the table name.truefalsefalseNONEfalsefalsetable-nameTable NameThe name of the database table containing the parameters.truefalsefalseNONEfalsefalseparameter-grouping-strategyParameter Grouping Strategygrouping-by-columntable-namesTable NamesA comma-separated list of names of the database tables containing the parameters.truefalsefalseNONEfalsefalseparameter-grouping-strategyParameter Grouping Strategygrouping-by-table-nameparameter-name-columnParameter Name ColumnThe name of a column containing the parameter name.truefalsefalseNONEfalsefalseparameter-value-columnParameter Value ColumnThe name of a column containing the parameter value.truefalsefalseNONEfalsefalseparameter-group-name-columnParameter Group Name ColumnThe name of a column containing the name of the parameter group into which the parameter should be mapped.truefalsefalseNONEfalsefalseparameter-grouping-strategyParameter Grouping Strategygrouping-by-columnsql-where-clauseSQL WHERE clauseA optional SQL query 'WHERE' clause by which to filter all results. The 'WHERE' keyword should not be included.falsefalsefalseNONEfalsefalseorg.apache.nifi.parameter.EnvironmentVariableParameterProviderPARAMETER_PROVIDERFetches parameters from environment variablesenvironmentvariableparameter-group-nameParameter Group NameThe name of the parameter group that will be fetched. This indicates the name of the Parameter Context that may receive the fetched parameters.Environment VariablestruefalsefalseNONEfalsefalseenvironment-variable-inclusion-strategyEnvironment Variable Inclusion StrategyIndicates how Environment Variables should be includedinclude-allInclude Allinclude-allAll Environment Variables will be includedComma-Separatedcomma-separatedList comma-separated Environment Variable names to includeRegular ExpressionregexInclude Environment Variable names that match a Regular ExpressiontruefalsefalseNONEfalsefalseinclude-environment-variablesInclude Environment VariablesSpecifies environment variable names that should be included from the fetched environment variables.falsefalsefalseNONEfalsefalseenvironment-variable-inclusion-strategyEnvironment Variable Inclusion Strategyregexcomma-separatedaccess environment credentialsProvides operator the ability to read environment variables, which may contain environment credentials.org.apache.nifi.parameter.KubernetesSecretParameterProviderPARAMETER_PROVIDERFetches parameters from files, in the format provided by Kubernetes mounted secrets. Parameter groups are indicated by a set of directories, and files within the directories map to parameter names. The content of the file becomes the parameter value. Since Kubernetes mounted Secrets are base64-encoded, the parameter provider defaults to Base64-decoding the value of the parameter from the file.fileparameter-group-directoriesParameter Group DirectoriesA comma-separated list of directory absolute paths that will map to named parameter groups. Each directory that contains files will map to a parameter group, named after the innermost directory in the path. Files inside the directory will map to parameter names, whose values are the content of each respective file.truefalsefalseNONEfalsefalseparameter-value-byte-limitParameter Value Byte LimitThe maximum byte size of a parameter value. Since parameter values are pulled from the contents of files, this is a safeguard that can prevent memory issues if large files are included.256 BtruefalsefalseNONEfalsefalseparameter-value-encodingParameter Value EncodingIndicates how parameter values are encoded inside Parameter files.base64Base64base64File content is Base64-encoded, and will be decoded before providing the value as a Parameter.Plain textplaintextFile content is not encoded, and will be provided directly as a Parameter value.truefalsefalseNONEfalsefalseread filesystemProvides operator the ability to read from any file that NiFi has access to.org.apache.nifi.parameter.OnePasswordParameterProviderPARAMETER_PROVIDERFetches parameters from 1Password Connect Server1PasswordWeb Client Service ProviderWeb Client Service ProviderController service for HTTP client operations.org.apache.nifi.web.client.provider.api.WebClientServiceProviderorg.apache.nifinifi-standard-services-api-nar2.0.0-M4truefalsefalseNONEfalsefalseConnect ServerConnect ServerHTTP endpoint of the 1Password Connect Server to connect to. Example: http://localhost:8080truefalsefalseNONEfalsefalseAccess TokenAccess TokenAccess Token used for authentication against the 1Password APIs.truetruefalseNONEfalsefalseorg.apache.nifi.flowanalysis.rules.DisallowComponentTypeFLOW_ANALYSIS_RULEProduces rule violations for each component (i.e. processors or controller services) of a given type.componentprocessorcontroller servicetypecomponent-typeComponent TypeComponents of the given type will produce a rule violation (i.e. they shouldn't exist). Either the simple or the fully qualified name of the type should be provided.truefalsefalseNONEfalsefalse