How to configure pylintrc for Python Processors in Apache NiFi?

With Pylint you can ensure the quality of your Python projects. You can do manual checks locally or automated in a pipeline. The following article gives a quick explanation of how to achieve excellent code quality with your Apache NiFi Python Processors with just a few steps.

Installing and customizing pylint

To get started, you will need to install pylint using the following command:

pip install pylint
Once installed, you can create a custom configuration file to tailor pylint for your specific needs. Generate the pylintrc file by running:

pylint --generate-rcfile > pylintrc
After generating the file, you can customize it by disabling certain checks in categories like convention, refactor, warning and errors. This does not mean your code is riddled with issues; rather, it helps you adapt the linting process to accommodate the unique structure of Python Processors in Apache NiFi, ensuring you get the best possible code quality within those constraints.

The specific structure of the processors in Apache NiFi makes it necessary to disable the following checks:

• invalid-name / C0103 ⇾ Appears in every processor, because the argument flowFile in method "transform" does not conform to the snake case convention
• duplicate-code / R0801 ⇾ Similar processors could lead to this error, even if they do not have the same purpose
• too-few-public-methods / R0903 ⇾ The classes Java or ProcessorDetails do not require any methods
• unused-argument / W01613 ⇾ The methods 'transform' or 'create' of the processors must have the argument context. If neither method is used in the processor, this error appears.
• import-error / E0401 ⇾ This error appears on specific NiFi imports, because Pylint cannot find the module

TL; DR

The final part of disabling in pylintrc file looks like this (defaults included):

disable=raw-checker-failed,
        bad-inline-option,
        locally-disabled,
        file-ignored,
        suppressed-message,
        useless-suppression,
        deprecated-pragma,
        use-symbolic-message-instead,
        use-implicit-booleaness-not-comparison-to-string,
        use-implicit-booleaness-not-comparison-to-zero,
        invalid-name,
        duplicate-code,
        too-few-public-methods,
        unused-argument,
        import-error 

You can use your configuration with the command 'pylint folder_with_your_processors --rcfile=path/to/your/pylintrc'. 

Do you like to avoid linting errors in your IDE while coding?

Use the same configuration in your preferred IDE, in Visual Studio Code you could set this in the settings.json.

Do you have questions about Apache NiFi Processors? Talk to us. 😊

Seminarempfehlungen