How Checks are Run¶
In Flake8 2.x, Flake8 delegated check running to pep8. In 3.0 Flake8
takes on that responsibility. This has allowed for simpler
handling of the --jobs
parameter (using multiprocessing
) and
simplified our fallback if something goes awry with concurrency.
At the lowest level we have a FileChecker
. Instances of FileChecker
are
created for each file to be analyzed by Flake8. Each instance, has a copy
of all of the plugins registered with setuptools in the flake8.extension
entry-point group.
The FileChecker
instances are managed by an instance of Manager
. The
Manager
instance handles creating sub-processes with
multiprocessing
module and falling back to running checks in serial if
an operating system level error arises. When creating FileChecker
instances,
the Manager
is responsible for determining if a particular file has been
excluded.
Processing Files¶
Unfortunately, since Flake8 took over check running from pep8/pycodestyle,
it also had to take over parsing and processing files for the checkers
to use. Since it couldn’t reuse pycodestyle’s functionality (since it did not
separate cleanly the processing from check running) that function was isolated
into the FileProcessor
class. We moved
several helper functions into the flake8.processor
module (see also
Processor Utility Functions).
API Reference¶
- class flake8.checker.FileChecker(*, filename, plugins, options)[source]¶
Manage running checks for a file and aggregate the results.
- check_physical_eol(token, prev_physical)[source]¶
Run physical checks if and only if it is at the end of the line.
- Parameters
token (TokenInfo) –
prev_physical (str) –
- Return type
None
- handle_newline(token_type)[source]¶
Handle the logic when encountering a newline token.
- Parameters
token_type (int) –
- Return type
None
- process_tokens()[source]¶
Process tokens and trigger checks.
Instead of using this directly, you should use
flake8.checker.FileChecker.run_checks()
.- Return type
None
- report(error_code, line_number, column, text)[source]¶
Report an error by storing it in the results list.
- class flake8.checker.Manager(style_guide, plugins)[source]¶
Manage the parallelism and checker instances for each plugin and file.
This class will be responsible for the following:
Determining the parallelism of Flake8, e.g.:
Do we use
multiprocessing
or is it unavailable?Do we automatically decide on the number of jobs to use or did the user provide that?
Falling back to a serial way of processing files if we run into an OSError related to
multiprocessing
Organizing the results of each checker so we can group the output together and make our output deterministic.
- Parameters
style_guide (StyleGuideManager) –
plugins (Checkers) –
- report()[source]¶
Report all of the errors found in the managed file checkers.
This iterates over each of the checkers and reports the errors sorted by line number.
- run()[source]¶
Run all the checkers.
This will intelligently decide whether to run the checks in parallel or whether to run them in serial.
If running the checks in parallel causes a problem (e.g., #117) this also implements fallback to serial processing.
- Return type
None
- class flake8.processor.FileProcessor(filename, options, lines=None)[source]¶
Processes a file and holdes state.
This processes a file by generating tokens, logical and physical lines, and AST trees. This also provides a way of passing state about the file to checks expecting that state. Any public attribute on this object can be requested by a plugin. The known public attributes are:
- blank_before¶
Number of preceding blank lines
- blank_lines¶
Number of blank lines
- generate_tokens()[source]¶
Tokenize the file and yield the tokens.
- Return type
Generator[TokenInfo, None, None]
- hang_closing¶
User provided option for hang closing
- indent_level¶
Current level of indentation
- indent_size¶
Number of spaces used for indentation
- keyword_arguments_for(parameters, arguments)[source]¶
Generate the keyword arguments for a list of parameters.
- line_number¶
Line number in the file
- logical_line¶
Current logical line
- max_doc_length¶
Maximum docstring / comment line length as configured by the user
- max_line_length¶
Maximum line length as configured by the user
- multiline¶
Whether the current physical line is multiline
- next_logical_line()[source]¶
Record the previous logical line.
This also resets the tokens list and the blank_lines count.
- Return type
None
- noqa = False¶
always
False
, included for compatibility
- previous_indent_level¶
Previous level of indentation
- previous_logical¶
Previous logical line
- previous_unindented_logical_line¶
Previous unindented (i.e. top-level) logical line
- should_ignore_file()[source]¶
Check if
flake8: noqa
is in the file to be ignored.- Returns
True if a line matches
defaults.NOQA_FILE
, otherwise False- Return type
- split_line(token)[source]¶
Split a physical line’s line based on new-lines.
This also auto-increments the line number for the caller.
- statistics¶
Statistics dictionary
- total_lines¶
Total number of lines in the file
- update_checker_state_for(plugin)[source]¶
Update the checker_state attribute for the plugin.
- Parameters
plugin (LoadedPlugin) –
- Return type
None
- verbose¶
Verbosity level of Flake8
Utility Functions¶
- flake8.processor.count_parentheses(current_parentheses_count, token_text)[source]¶
Count the number of parentheses.
- flake8.processor.expand_indent(line)[source]¶
Return the amount of indentation.
Tabs are expanded to the next multiple of 8.
>>> expand_indent(' ') 4 >>> expand_indent('\t') 8 >>> expand_indent(' \t') 8 >>> expand_indent(' \t') 16
- flake8.processor.is_eol_token(token)[source]¶
Check if the token is an end-of-line token.
- Parameters
token (TokenInfo) –
- Return type
- flake8.processor.is_multiline_string(token)[source]¶
Check if this is a multiline string.
- Parameters
token (TokenInfo) –
- Return type