coalib.misc package¶
Submodules¶
coalib.misc.BuildManPage module¶
-
class
coalib.misc.BuildManPage.
BuildManPage
(dist)[source]¶ Bases:
distutils.cmd.Command
Add a
build_manpage
command to your setup.py. To use this Command class add a command to call this class:# For setuptools setup( entry_points={ "distutils.commands": [ "build_manpage = coalib.misc.BuildManPage:BuildManPage" ] } ) # For distutils from coalib.misc.BuildManPage import BuildManPage setup( cmdclass={'build_manpage': BuildManPage} )
You can then use the following setup command to produce a man page:
$ python setup.py build_manpage --output=coala.1 --parser=coalib.parsing.DefaultArgParser:default_arg_parser
If automatically want to build the man page every time you invoke your build, add to your
setup.cfg
the following:[build_manpage] output = <appname>.1 parser = <path_to_your_parser>
-
user_options
= [('output=', 'O', 'output file'), ('parser=', None, 'module path to an ArgumentParser instance(e.g. mymod:func, where func is a method or function which returnan arparse.ArgumentParser instance.')]¶
-
coalib.misc.Caching module¶
-
class
coalib.misc.Caching.
FileCache
(log_printer, project_dir: str, flush_cache: bool = False)[source]¶ Bases:
object
This object is a file cache that helps in collecting only the changed and new files since the last run. Example/Tutorial:
>>> import logging >>> import copy, time >>> logging.getLogger().setLevel(logging.CRITICAL)
To initialize the cache create an instance for the project:
>>> cache = FileCache(None, "test", flush_cache=True)
Now we can track new files by running:
>>> cache.track_files(["a.c", "b.c"])
Since all cache operations are lazy (for performance), we need to explicitly write the cache to disk for persistence in future uses: (Note: The cache will automatically figure out the write location)
>>> cache.write()
Let’s go into the future:
>>> time.sleep(1)
Let’s create a new instance to simulate a separate run:
>>> cache = FileCache(None, "test", flush_cache=False)
>>> old_data = copy.deepcopy(cache.data)
We can mark a file as changed by doing:
>>> cache.untrack_files({"a.c"})
Again write to disk after calculating the new cache times for each file:
>>> cache.write() >>> new_data = cache.data
Since we marked ‘a.c’ as a changed file:
>>> "a.c" not in cache.data True >>> "a.c" in old_data True
Since ‘b.c’ was untouched after the second run, its time was updated to the latest value:
>>> old_data["b.c"] < new_data["b.c"] True
-
get_uncached_files
(files)[source]¶ Returns the set of files that are not in the cache yet or have been untracked.
Parameters: files – The list of collected files. Returns: A set of files that are uncached.
-
track_files
(files)[source]¶ Start tracking files given in
files
by adding them to the database.Parameters: files – A set of files that need to be tracked. These files are initialized with their last modified tag as -1.
-
coalib.misc.CachingUtilities module¶
-
coalib.misc.CachingUtilities.
delete_files
(log_printer, identifiers)[source]¶ Delete the given identifiers from the user’s coala data directory.
Parameters: - log_printer – A LogPrinter object to use for logging.
- identifiers – The list of files to be deleted.
Returns: True if all the given files were successfully deleted. False otherwise.
-
coalib.misc.CachingUtilities.
get_data_path
(log_printer, identifier)[source]¶ Get the full path of
identifier
present in the user’s data directory.Parameters: - log_printer – A LogPrinter object to use for logging.
- identifier – The file whose path needs to be expanded.
Returns: Full path of the file, assuming it’s present in the user’s config directory. Returns
None
if there is aPermissionError
in creating the directory.
-
coalib.misc.CachingUtilities.
get_settings_hash
(sections, targets=[], ignore_settings: list = ['disable_caching'])[source]¶ Compute and return a unique hash for the settings.
Parameters: - sections – A dict containing the settings for each section.
- targets – The list of sections that are enabled.
- ignore_settings – Setting keys to remove from sections before hashing.
Returns: A MD5 hash that is unique to the settings used.
-
coalib.misc.CachingUtilities.
hash_id
(text)[source]¶ Hashes the given text.
Parameters: text – String to to be hashed Returns: A MD5 hash of the given string
-
coalib.misc.CachingUtilities.
pickle_dump
(log_printer, identifier, data)[source]¶ Write
data
into the filefilename
present in the user config directory.Parameters: - log_printer – A LogPrinter object to use for logging.
- identifier – The name of the file present in the user config directory.
- data – Data to be serialized and written to the file using pickle.
Returns: True if the write was successful. False if there was a permission error in writing.
-
coalib.misc.CachingUtilities.
pickle_load
(log_printer, identifier, fallback=None)[source]¶ Get the data stored in
filename
present in the user config directory. Example usage:>>> test_data = {'answer': 42} >>> pickle_dump(None, 'test_project', test_data) True >>> pickle_load(None, 'test_project') {'answer': 42} >>> pickle_load(None, 'nonexistent_project') >>> pickle_load(None, 'nonexistent_project', fallback=42) 42
Parameters: - log_printer – A LogPrinter object to use for logging.
- identifier – The name of the file present in the user config directory.
- fallback – Return value to fallback to in case the file doesn’t exist.
Returns: Data that is present in the file, if the file exists. Otherwise the
default
value is returned.
-
coalib.misc.CachingUtilities.
settings_changed
(log_printer, settings_hash)[source]¶ Determine if the settings have changed since the last run with caching.
Parameters: - log_printer – A LogPrinter object to use for logging.
- settings_hash – A MD5 hash that is unique to the settings used.
Returns: Return True if the settings hash has changed Return False otherwise.
coalib.misc.Compatibility module¶
coalib.misc.DeprecationUtilities module¶
-
coalib.misc.DeprecationUtilities.
check_deprecation
(param_list)[source]¶ Shows a deprecation warning message if the parameters passed are not
None
.Parameters: param_list – A dictionary of parameters with their names mapped to their values being checked for deprecation. >>> from testfixtures import LogCapture >>> from collections import OrderedDict >>> param_list = OrderedDict([('foo', None), ... ('bar', 'Random'), ... ('baz', 1773)]) >>> with LogCapture() as capture: ... check_deprecation(param_list) ... print(capture) root WARNING bar parameter is deprecated root WARNING baz parameter is deprecated
coalib.misc.DictUtilities module¶
-
coalib.misc.DictUtilities.
inverse_dicts
(*dicts)[source]¶ Inverts the dicts, e.g. {1: 2, 3: 4} and {2: 3, 4: 4} will be inverted {2: [1], 3: [2], 4: [3, 4]}. This also handles dictionaries with Iterable items as values e.g. {1: [1, 2, 3], 2: [3, 4, 5]} and {2: [1], 3: [2], 4: [3, 4]} will be inverted to {1: [1, 2], 2: [1, 3], 3: [1, 2, 4], 4: [2, 4], 5: [2]}. No order is preserved.
Parameters: dicts (dict) – The dictionaries to invert. Returns: The inversed dictionary which merges all dictionaries into one. Return type: defaultdict
coalib.misc.Exceptions module¶
coalib.misc.IterUtilities module¶
coalib.misc.Shell module¶
-
class
coalib.misc.Shell.
ShellCommandResult
(code, stdout, stderr)[source]¶ Bases:
tuple
The result of a
coalib.misc.run_shell_command()
call.It is based on a
(stdout, stderr)
string tuple like it is returned formsubprocess.Popen.communicate
and was originally returned fromcoalib.misc.run_shell_command()
. So it is backwards-compatible.It additionally stores the return
.code
:>>> import sys >>> process = Popen([sys.executable, '-c', ... 'import sys; print(sys.stdin.readline().strip() +' ... ' " processed")'], ... stdin=PIPE, stdout=PIPE, stderr=PIPE, ... universal_newlines=True)
>>> stdout, stderr = process.communicate(input='data') >>> stderr '' >>> result = ShellCommandResult(process.returncode, stdout, stderr) >>> result[0] 'data processed\n' >>> result[1] '' >>> result.code 0
-
coalib.misc.Shell.
run_interactive_shell_command
(command, **kwargs)[source]¶ Runs a single command in shell and provides stdout, stderr and stdin streams.
This function creates a context manager that sets up the process (using
subprocess.Popen()
), returns to caller and waits for process to exit on leaving.By default the process is opened in
universal_newlines
mode and creates pipes for all streams (stdout, stderr and stdin) usingsubprocess.PIPE
special value. These pipes are closed automatically, so if you want to get the contents of the streams you should retrieve them before the context manager exits.>>> with run_interactive_shell_command(["echo", "TEXT"]) as p: ... stdout = p.stdout ... stdout_text = stdout.read() >>> stdout_text 'TEXT\n' >>> stdout.closed True
Custom streams provided are not closed except of
subprocess.PIPE
.>>> from tempfile import TemporaryFile >>> stream = TemporaryFile() >>> with run_interactive_shell_command(["echo", "TEXT"], ... stdout=stream) as p: ... stderr = p.stderr >>> stderr.closed True >>> stream.closed False
Parameters: - command – The command to run on shell. This parameter can either
be a sequence of arguments that are directly passed to
the process or a string. A string gets splitted beforehand
using
shlex.split()
. If providingshell=True
as a keyword-argument, noshlex.split()
is performed and the command string goes directly tosubprocess.Popen()
. - kwargs – Additional keyword arguments to pass to
subprocess.Popen
that are used to spawn the process.
Returns: A context manager yielding the process started from the command.
- command – The command to run on shell. This parameter can either
be a sequence of arguments that are directly passed to
the process or a string. A string gets splitted beforehand
using
-
coalib.misc.Shell.
run_shell_command
(command, stdin=None, **kwargs)[source]¶ Runs a single command in shell and returns the read stdout and stderr data.
This function waits for the process (created using
subprocess.Popen()
) to exit. Effectively it wrapsrun_interactive_shell_command()
and usescommunicate()
on the process.See also
run_interactive_shell_command()
.Parameters: - command – The command to run on shell. This parameter can either
be a sequence of arguments that are directly passed to
the process or a string. A string gets splitted beforehand
using
shlex.split()
. - stdin – Initial input to send to the process.
- kwargs – Additional keyword arguments to pass to
subprocess.Popen
that is used to spawn the process.
Returns: A tuple with
(stdoutstring, stderrstring)
.- command – The command to run on shell. This parameter can either
be a sequence of arguments that are directly passed to
the process or a string. A string gets splitted beforehand
using