coalib.core package

Submodules

coalib.core.Bear module

class coalib.core.Bear.Bear(section: coalib.settings.Section.Section, file_dict: dict)[source]

Bases: object

A bear contains the actual subroutine that is responsible for checking source code for certain specifications. However, it can actually do whatever it wants with the files it gets.

This is the base class for every bear. If you want to write a bear, you will probably want to look at the ProjectBear and FileBear classes that inherit from this class.

To indicate which languages your bear supports, just give it the LANGUAGES value which should be a set of string(s):

>>> class SomeBear(Bear):
...     LANGUAGES = {'C', 'CPP', 'C#', 'D'}

To indicate the requirements of the bear, assign REQUIREMENTS a set with instances of PackageRequirements.

>>> from dependency_management.requirements.PackageRequirement import (
...     PackageRequirement)
>>> class SomeBear(Bear):
...     REQUIREMENTS = {
...         PackageRequirement('pip', 'coala_decorators', '0.2.1')}

If your bear uses requirements from a manager we have a subclass from, you can use the subclass, such as PipRequirement, without specifying manager:

>>> from dependency_management.requirements.PipRequirement import (
...     PipRequirement)
>>> class SomeBear(Bear):
...     REQUIREMENTS = {PipRequirement('coala_decorators', '0.2.1')}

To specify additional attributes to your bear, use the following:

>>> class SomeBear(Bear):
...     AUTHORS = {'Jon Snow'}
...     AUTHORS_EMAILS = {'jon_snow@gmail.com'}
...     MAINTAINERS = {'Catelyn Stark'}
...     MAINTAINERS_EMAILS = {'catelyn_stark@gmail.com'}
...     LICENSE = 'AGPL-3.0'
...     ASCIINEMA_URL = 'https://asciinema.org/a/80761'

If the maintainers are the same as the authors, they can be omitted:

>>> class SomeBear(Bear):
...     AUTHORS = {'Jon Snow'}
...     AUTHORS_EMAILS = {'jon_snow@gmail.com'}
>>> SomeBear.maintainers
{'Jon Snow'}
>>> SomeBear.maintainers_emails
{'jon_snow@gmail.com'}

If your bear needs to include local files, then specify it giving strings containing relative file paths to the INCLUDE_LOCAL_FILES set:

>>> class SomeBear(Bear):
...     INCLUDE_LOCAL_FILES = {'checkstyle.jar', 'google_checks.xml'}

To keep track easier of what a bear can do, simply tell it to the CAN_FIX and the CAN_DETECT sets. Possible values are:

>>> CAN_DETECT = {'Syntax', 'Formatting', 'Security', 'Complexity',
... 'Smell', 'Unused Code', 'Redundancy', 'Variable Misuse', 'Spelling',
... 'Memory Leak', 'Documentation', 'Duplication', 'Commented Code',
... 'Grammar', 'Missing Import', 'Unreachable Code', 'Undefined Element',
... 'Code Simplification'}
>>> CAN_FIX = {'Syntax', ...}

Specifying something to CAN_FIX makes it obvious that it can be detected too, so it may be omitted:

>>> class SomeBear(Bear):
...     CAN_DETECT = {'Syntax', 'Security'}
...     CAN_FIX = {'Redundancy'}
>>> sorted(SomeBear.can_detect)
['Redundancy', 'Security', 'Syntax']

Every bear has a data directory which is unique to that particular bear:

>>> class SomeBear(Bear): pass
>>> class SomeOtherBear(Bear): pass
>>> SomeBear.data_dir == SomeOtherBear.data_dir
False

A bear can be dependent from other bears. BEAR_DEPS contains bear classes that are to be executed before this bear gets executed. The results of these bears will then be passed inside self.dependency_results as a dict. The dict will have the name of the bear as key and a list of its results as values:

>>> class SomeBear(Bear): pass
>>> class SomeOtherBear(Bear):
...     BEAR_DEPS = {SomeBear}
>>> SomeOtherBear.BEAR_DEPS
{<class 'coalib.core.Bear.SomeBear'>}
ASCIINEMA_URL = ''
AUTHORS = set()
AUTHORS_EMAILS = set()
BEAR_DEPS = set()
CAN_DETECT = set()
CAN_FIX = set()
INCLUDE_LOCAL_FILES = set()
LANGUAGES = set()
LICENSE = ''
MAINTAINERS = set()
MAINTAINERS_EMAILS = set()
PLATFORMS = {'any'}
REQUIREMENTS = set()
analyze(*args, **kwargs)[source]

Performs the code analysis.

Returns:An iterable of results.
can_detect = set()
classmethod check_prerequisites()[source]

Checks whether needed runtime prerequisites of the bear are satisfied.

This function gets executed at construction.

Section value requirements shall be checked inside the run method.

>>> from dependency_management.requirements.PipRequirement import (
...     PipRequirement)
>>> class SomeBear(Bear):
...     REQUIREMENTS = {PipRequirement('pip')}
>>> SomeBear.check_prerequisites()
True
>>> class SomeOtherBear(Bear):
...     REQUIREMENTS = {PipRequirement('really_bad_package')}
>>> SomeOtherBear.check_prerequisites()
'Following requirements are not installed: really_bad_package (...)'
Returns:True if prerequisites are satisfied, else False or a string that serves a more detailed description of what’s missing.
data_dir = '/home/docs/.local/share/coala-bears/Bear'
dependency_results

Contains all dependency results.

This variable gets set during bear execution from the core and can be used from analyze.

Modifications to the returned dictionary while the core is running leads to undefined behaviour.

>>> section = Section('my-section')
>>> file_dict = {'file1.txt': ['']}
>>> bear = Bear(section, file_dict)
>>> bear.dependency_results
defaultdict(<class 'list'>, {})
>>> dependency_bear = Bear(section, file_dict)
>>> bear.dependency_results[type(dependency_bear)] += [1, 2]
>>> bear.dependency_results
defaultdict(<class 'list'>, {<class 'coalib.core.Bear.Bear'>: [1, 2]})
Returns:A dictionary with bear-types as keys and their results received.
classmethod download_cached_file(url, filename)[source]

Downloads the file if needed and caches it for the next time. If a download happens, the user will be informed.

Take a sane simple bear:

>>> section = Section('my-section')
>>> file_dict = {'file1.txt': ['']}
>>> bear = Bear(section, file_dict)

We can now carelessly query for a neat file that doesn’t exist yet:

>>> from os import remove
>>> if exists(join(bear.data_dir, 'a_file')):
...     remove(join(bear.data_dir, 'a_file'))
>>> file = bear.download_cached_file('https://github.com/', 'a_file')

If we download it again, it’ll be much faster as no download occurs:

>>> newfile = bear.download_cached_file(
...     'https://github.com/', 'a_file')
>>> newfile == file
True
Parameters:
  • url – The URL to download the file from.
  • filename – The filename it should get, e.g. “test.txt”.
Returns:

A full path to the file ready for you to use!

execute_task(args, kwargs)[source]

Executes a task.

By default returns a list of results collected from this bear.

This function has to return something that is picklable to make bears work in multi-process environments.

Parameters:
  • args – The arguments of a task.
  • kwargs – The keyword-arguments of a task.
Returns:

A list of results from the bear.

generate_tasks()[source]

This method is responsible for providing the job arguments analyze gets called with.

Returns:An iterable containing the positional and keyword arguments organized in pairs: (args-tuple, kwargs-dict)
get_config_dir()[source]

Gives the directory where the configuration file resides.

Returns:Directory of the config file.
classmethod get_metadata()[source]
Returns:Metadata for the analyze function extracted from its signature. Excludes parameter self.
classmethod get_non_optional_settings()[source]

This method has to determine which settings are needed by this bear. The user will be prompted for needed settings that are not available in the settings file so don’t include settings where a default value would do.

Note: This function also queries settings from bear dependencies in recursive manner. Though circular dependency chains are a challenge to achieve, this function would never return on them!

Returns:A dictionary of needed settings as keys and a tuple of help text and annotation as values
maintainers = set()
maintainers_emails = set()
name = 'Bear'
new_result

Returns a partial for creating a result with this bear already bound.

static setup_dependencies()[source]

This is a user defined function that can download and set up dependencies (via download_cached_file or arbitrary other means) in an OS independent way.

source_location = '/home/docs/checkouts/readthedocs.org/user_builds/coala-api/envs/latest/lib/python3.5/site-packages/coala-0.12.0.dev20180614080551-py3.5.egg/coalib/core/Bear.py'

coalib.core.CircularDependencyError module

exception coalib.core.CircularDependencyError.CircularDependencyError(names=None)[source]

Bases: RuntimeError

An error identifying a circular dependency.

coalib.core.Core module

class coalib.core.Core.Session(bears, result_callback, cache=None, executor=None)[source]

Bases: object

Maintains a session for a coala execution. For each session, there are set of bears to run along with a callback function, which is called when results are available.

Dependencies of bears (provided via bear.BEAR_DEPS) are automatically handled. If BearA requires BearB as dependency, then on running BearA, first BearB will be executed, followed by BearA.

run()[source]

Runs the coala session.

coalib.core.Core.group(iterable, key=<function <lambda>>)[source]

Groups elements (out-of-order) together in the given iterable.

Supports non-hashable keys by comparing keys with ==.

Accessing the groups is supported using the iterator as follows:

>>> for key, elements in group([1, 3, 7, 1, 2, 1, 2]):
...     print(key, list(elements))
1 [1, 1, 1]
3 [3]
7 [7]
2 [2, 2]

You can control how elements are grouped by using the key parameter. It takes a function with a single parameter and maps to the group.

>>> data = [(1, 2), (3, 4), (1, 9), (2, 10), (1, 11), (7, 2), (10, 2),
...         (2, 1), (3, 7), (4, 5)]
>>> for key, elements in group(data, key=sum):
...     print(key, list(elements))
3 [(1, 2), (2, 1)]
7 [(3, 4)]
10 [(1, 9), (3, 7)]
12 [(2, 10), (1, 11), (10, 2)]
9 [(7, 2), (4, 5)]
Parameters:
  • iterable – The iterable to group elements in.
  • key – The key-function mapping an element to its group.
Returns:

An iterable yielding tuples with key, elements, where elements is also an iterable yielding the elements grouped under key.

coalib.core.Core.initialize_dependencies(bears)[source]

Initializes and returns a DependencyTracker instance together with a set of bears ready for scheduling.

This function acquires, processes and registers bear dependencies accordingly using a consumer-based system, where each dependency bear has only a single instance per section and file-dictionary.

The bears set returned accounts for bears that have dependencies and excludes them accordingly. Dependency bears that have themselves no further dependencies are included so the dependency chain can be processed correctly.

Parameters:bears – The set of instantiated bears to run that serve as an entry-point.
Returns:A tuple with (dependency_tracker, bears_to_schedule).
coalib.core.Core.run(bears, result_callback, cache=None, executor=None)[source]

Initiates a session with the given parameters and runs it.

Parameters:
  • bears – The bear instances to run.
  • result_callback

    A callback function which is called when results are available. Must have following signature:

    def result_callback(result):
        pass
    

    Only those results are passed for bears that were explicitly requested via the bears parameter, implicit dependency results do not call the callback.

  • cache

    A cache bears can use to speed up runs. If None, no cache will be used.

    The cache stores the results that were returned last time from the parameters passed to execute_task in bears. If the parameters to execute_task are the same from a previous run, the cache will be queried instead of executing execute_task.

    The cache has to be a dictionary-like object, that maps bear types to respective cache-tables. The cache-tables itself are dictionary-like objects that map hash-values (generated by PersistentHash.persistent_hash from the task objects) to actual bear results. When bears are about to be scheduled, the core performs a cache-lookup. If there’s a hit, the results stored in the cache are returned and the task won’t be scheduled. In case of a miss, execute_task is called normally in the executor.

  • executor – Custom executor used to run the bears. If None, a ProcessPoolExecutor is used using as many processes as cores available on the system.

coalib.core.DependencyBear module

class coalib.core.DependencyBear.DependencyBear(section, file_dict)[source]

Bases: coalib.core.Bear.Bear

This bear base class parallelizes tasks for each dependency result.

You can specify dependency bears with the BEAR_DEPS field.

generate_tasks()[source]
classmethod get_metadata()[source]
Returns:Metadata for the analyze function extracted from its signature. Excludes parameters self, filename and file.

coalib.core.DependencyTracker module

class coalib.core.DependencyTracker.DependencyTracker[source]

Bases: object

A DependencyTracker allows to register and manage dependencies between objects.

This class uses a directed graph to track relations.

Add a dependency relation between two objects:

>>> object1 = object()
>>> object2 = object()
>>> tracker = DependencyTracker()
>>> tracker.add(object2, object1)

This would define that object1 is dependent on object2.

If you define that object2 has its dependency duty fulfilled, you can resolve it:

>>> resolved = tracker.resolve(object2)
>>> resolved
{<object object at ...>}
>>> resolved_object = resolved.pop()
>>> resolved_object is object1
True

This returns all objects that are now freed, meaning they have no dependencies any more.

>>> object3 = object()
>>> tracker.add(object2, object1)
>>> tracker.add(object3, object1)
>>> tracker.resolve(object2)
set()
>>> tracker.resolve(object3)
{<object object at ...>}

The ones who instantiate a DependencyTracker are responsible for resolving dependencies in the right order. Dependencies which are itself dependent will be forcefully resolved and removed from their according dependencies too.

add(dependency, dependant)[source]

Add a dependency relation.

This function does not check for circular dependencies.

>>> tracker = DependencyTracker()
>>> tracker.add(0, 1)
>>> tracker.add(0, 2)
>>> tracker.resolve(0)
{1, 2}
Parameters:
  • dependency – The object that is the dependency.
  • dependant – The object that is the dependant.
are_dependencies_resolved

Checks whether all dependencies in this DependencyTracker instance are resolved.

>>> tracker = DependencyTracker()
>>> tracker.are_dependencies_resolved
True
>>> tracker.add(0, 1)
>>> tracker.are_dependencies_resolved
False
>>> tracker.resolve(0)
{1}
>>> tracker.are_dependencies_resolved
True
Returns:True when all dependencies resolved, False if not.
check_circular_dependencies()[source]

Checks whether there are circular dependency conflicts.

>>> tracker = DependencyTracker()
>>> tracker.add(0, 1)
>>> tracker.add(1, 0)
>>> tracker.check_circular_dependencies()
Traceback (most recent call last):
 ...
coalib.core.CircularDependencyError.CircularDependencyError: ...
Raises:CircularDependencyError – Raised on circular dependency conflicts.
dependants

Returns a set of all registered dependants.

>>> tracker = DependencyTracker()
>>> tracker.add(0, 1)
>>> tracker.add(0, 2)
>>> tracker.add(1, 3)
>>> tracker.dependants
{1, 2, 3}
dependencies

Returns a set of all registered dependencies.

>>> tracker = DependencyTracker()
>>> tracker.add(0, 1)
>>> tracker.add(0, 2)
>>> tracker.add(1, 3)
>>> tracker.dependencies
{0, 1}
get_all_dependants(dependency)[source]

Returns a set of all dependants of the given dependency, even indirectly related ones.

>>> tracker = DependencyTracker()
>>> tracker.add(0, 1)
>>> tracker.add(1, 2)
>>> tracker.get_all_dependants(0)
{1, 2}
Parameters:dependency – The dependency to get all dependants for.
Returns:A set of dependants.
get_all_dependencies(dependant)[source]

Returns a set of all dependencies of the given dependants, even indirectly related ones.

>>> tracker = DependencyTracker()
>>> tracker.add(0, 1)
>>> tracker.add(1, 2)
>>> tracker.get_all_dependencies(2)
{0, 1}
Parameters:dependant – The dependant to get all dependencies for.
Returns:A set of dependencies.
get_dependants(dependency)[source]

Returns all immediate dependants for the given dependency.

>>> tracker = DependencyTracker()
>>> tracker.add(0, 1)
>>> tracker.add(0, 2)
>>> tracker.add(1, 3)
>>> tracker.get_dependants(0)
{1, 2}
>>> tracker.get_dependants(1)
{3}
>>> tracker.get_dependants(2)
set()
Parameters:dependency – The dependency to retrieve all dependants from.
Returns:A set of dependants.
get_dependencies(dependant)[source]

Returns all immediate dependencies of a given dependant.

>>> tracker = DependencyTracker()
>>> tracker.add(0, 1)
>>> tracker.add(0, 2)
>>> tracker.add(1, 2)
>>> tracker.get_dependencies(0)
set()
>>> tracker.get_dependencies(1)
{0}
>>> tracker.get_dependencies(2)
{0, 1}
Parameters:dependant – The dependant to retrieve all dependencies from.
Returns:A set of dependencies.
resolve(dependency)[source]

Resolves all dependency-relations from the given dependency, and frees and returns dependants with no more dependencies. If the given dependency is itself a dependant, all those relations are also removed.

>>> tracker = DependencyTracker()
>>> tracker.add(0, 1)
>>> tracker.add(0, 2)
>>> tracker.add(2, 3)
>>> tracker.resolve(0)
{1, 2}
>>> tracker.resolve(2)
{3}
>>> tracker.resolve(2)
set()
Parameters:dependency – The dependency.
Returns:Returns a set of dependants whose dependencies were all resolved.

coalib.core.FileBear module

class coalib.core.FileBear.FileBear(section, file_dict)[source]

Bases: coalib.core.Bear.Bear

This bear base class parallelizes tasks for each file given.

generate_tasks()[source]
classmethod get_metadata()[source]
Returns:Metadata for the analyze function extracted from its signature. Excludes parameters self, filename and file.

coalib.core.Graphs module

coalib.core.Graphs.traverse_graph(start_nodes, get_successive_nodes, run_on_edge=<function <lambda>>)[source]

Traverses all edges of a directed, possibly disconnected graph once. Detects cyclic graphs by raising a CircularDependencyError.

>>> graph = {1: [2], 2: [3, 4], 5: [3], 3: [6]}
>>> def get_successive_nodes(node):
...     return graph.get(node, [])
>>> edges = set()
>>> def append_to_edges(prev, nxt):
...     edges.add((prev, nxt))
>>> traverse_graph([1, 5], get_successive_nodes, append_to_edges)
>>> sorted(edges)
[(1, 2), (2, 3), (2, 4), (3, 6), (5, 3)]

You can also use this function to detect cyclic graphs:

>>> graph = {1: [2], 2: [3], 3: [1]}
>>> traverse_graph([1], get_successive_nodes)
Traceback (most recent call last):
 ...
coalib.core.CircularDependencyError.CircularDependencyError: ...
Parameters:
  • start_nodes – The nodes where to start traversing the graph.
  • get_successive_nodes – A callable that takes in a node and returns an iterable of nodes to traverse next.
  • run_on_edge – A callable that is run on each edge during traversing. Takes in two parameters, the previous- and next-node which form an edge. The default is an empty function.
Raises:

CircularDependencyError – Raised when the graph is cyclic.

coalib.core.PersistentHash module

coalib.core.PersistentHash.persistent_hash(obj)[source]

Calculates a persistent hash of the given object.

This hash method uses pickle (protocol 4) to serialize the given object and hash the byte-stream. The hashing algorithm used is SHA-1.

Parameters:obj – The object to calculate the persistent hash for.
Returns:The persistent hash.

coalib.core.ProjectBear module

class coalib.core.ProjectBear.ProjectBear(section, file_dict)[source]

Bases: coalib.core.Bear.Bear

This bear base class does not parallelize tasks at all, it runs on the whole file base provided.

generate_tasks()[source]
classmethod get_metadata()[source]
Returns:Metadata for the analyze function extracted from its signature. Excludes parameters self and files.

Module contents