Coverage for /home/anselor/src/cmd2/cmd2/argparse_completer.py : 90%

Hot-keys on this page
r m x p toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
# coding=utf-8 AutoCompleter interprets the argparse.ArgumentParser internals to automatically generate the completion options for each argument.
How to supply completion options for each argument: argparse Choices - pass a list of values to the choices parameter of an argparse argument. ex: parser.add_argument('-o', '--options', dest='options', choices=['An Option', 'SomeOtherOption'])
arg_choices dictionary lookup arg_choices is a dict() mapping from argument name to one of 3 possible values: ex: parser = argparse.ArgumentParser() parser.add_argument('-o', '--options', dest='options') choices = {} mycompleter = AutoCompleter(parser, completer, 1, choices)
- static list - provide a static list for each argument name ex: choices['options'] = ['An Option', 'SomeOtherOption']
- choices function - provide a function that returns a list for each argument name ex: def generate_choices(): return ['An Option', 'SomeOtherOption'] choices['options'] = generate_choices
- custom completer function - provide a completer function that will return the list of completion arguments ex 1: def my_completer(text: str, line: str, begidx: int, endidx:int): my_choices = [...] return my_choices choices['options'] = (my_completer) ex 2: def my_completer(text: str, line: str, begidx: int, endidx:int, extra_param: str, another: int): my_choices = [...] return my_choices completer_params = {'extra_param': 'my extra', 'another': 5} choices['options'] = (my_completer, completer_params)
How to supply completion choice lists or functions for sub-commands: subcmd_args_lookup is used to supply a unique pair of arg_choices and subcmd_args_lookup for each sub-command in an argparser subparser group. This requires your subparser group to be named with the dest parameter ex: parser = ArgumentParser() subparsers = parser.add_subparsers(title='Actions', dest='action')
subcmd_args_lookup maps a named subparser group to a subcommand group dictionary The subcommand group dictionary maps subcommand names to tuple(arg_choices, subcmd_args_lookup)
For more details of this more complex approach see tab_autocompletion.py in the examples
Copyright 2018 Eric Lin <anselor@gmail.com> Released under MIT license, see LICENSE file """
# imports copied from argparse to support our customized argparse functions
# attribute that can optionally added to an argparse argument (called an Action) to # define the completion choices for the argument. You may provide a Collection or a Function.
# pre-process special ranged nargs else: # this shouldn't use a range tuple, but yet here we are else: else:
# noinspection PyShadowingBuiltins,PyShadowingBuiltins option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None):
option_strings=option_strings, dest=dest, nargs=self.nargs_adjusted, const=const, default=default, type=type, choices=choices, required=required, help=help, metavar=metavar)
# noinspection PyShadowingBuiltins,PyShadowingBuiltins option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None):
option_strings=option_strings, dest=dest, nargs=self.nargs_adjusted, const=const, default=default, type=type, choices=choices, required=required, help=help, metavar=metavar)
"""Register custom argument action types"""
"""Automatically command line tab completion based on argparse parameters"""
"""reset tracking values"""
parser: argparse.ArgumentParser, token_start_index: int = 1, arg_choices: Dict[str, Union[List, Tuple, Callable]] = None, subcmd_args_lookup: dict = None, tab_for_arg_help: bool = True, cmd2_app=None): """ Create an AutoCompleter
:param parser: ArgumentParser instance :param token_start_index: index of the token to start parsing at :param arg_choices: dictionary mapping from argparse argument 'dest' name to list of choices :param subcmd_args_lookup: mapping a sub-command group name to a tuple to fill the child\ AutoCompleter's arg_choices and subcmd_args_lookup parameters :param tab_for_arg_help: Enable of disable argument help when there's no completion result :param cmd2_app: reference to the Cmd2 application. Enables argparse argument completion with class methods """ else:
# maps action name to sub-command autocompleter: # action_name -> dict(sub_command -> completer)
# Start digging through the argparse structures. # _actions is the top level container of parameter definitions # if there are choices defined, record them in the arguments dictionary # if completion choices are tagged on the action, record them
# if the parameter is flag based, it will have option_strings # record each option flag else:
if action.dest in subcmd_args_lookup else {} subcmd in args_for_action else \ (arg_choices, subcmd_args_lookup) if forward_arg_choices else ({}, {}) arg_choices=subcmd_args, subcmd_args_lookup=subcmd_lookup, cmd2_app=cmd2_app)
"""Complete the command using the argparse metadata and provided argument dictionary""" # Count which positional argument index we're at now. Loop through all tokens on the command line so far # Skip any flags or flag parameter tokens
# the following are nested functions that have full access to all variables in the parent # function including variables declared and updated after this function. Variable values # are current at the point the nested functions are invoked (as in, they do not receive a # snapshot of these values, they directly access the current state of variables in the # parent function)
"""Consuming token as a flag argument""" # we're consuming flag arguments # if this is not empty and is not another potential flag, count towards flag arguments
# does this complete a option item for the flag # if the current token matches the current position's autocomplete argument list, # track that we've used it already. Unless this is the current token, then keep it.
"""Consuming token as positional argument"""
# does this complete a option item for the flag # if the current token matches the current position's autocomplete argument list, # track that we've used it already. Unless this is the current token, then keep it.
# Only start at the start token index # Are we consuming flag arguments? # we're not consuming flag arguments, is the current argument a potential flag? (is_last_token or (not is_last_token and token != '-')): # reset some tracking values # don't reset positional tracking because flags can be interspersed anywhere between positionals
# does the token fully match a known flag?
# resolve argument counts
# current token isn't a potential flag # - does the last flag accept variable arguments? # - have we reached the max arg count for the flag? # previous flag doesn't accept variable arguments, count this as a positional argument
# reset flag tracking variables
# we have positional action match and we haven't reached the max arg count, consume # the positional argument and move on. # if we don't have a current positional action or we've reached the max count for the action # close out the current positional argument state and set up for the next one
# are we at a sub-command? If so, forward to the matching completer begidx, endidx)
elif not is_last_token and pos_arg.max is not None: pos_action = None pos_arg.reset()
else:
else:
# don't reset this if we're on the last token - this allows completion to occur on the current token
# if we don't have a flag to populate with arguments and the last token starts with # a flag prefix then we'll complete the list of flag options [flag for flag in self._flags if flag not in matched_flags]) # we're not at a positional argument, see if we're in a flag argument # current_items = [] if flag_action.dest in consumed_arg_values else [] # current_items.extend(self._resolve_choices_for_arg(flag_action, consumed))
# ok, we're not a flag, see if there's a positional argument to complete else:
"""Supports the completion of sub-commands for commands through the cmd2 help command.""" # For now argparse only allows 1 sub-command group per level # so this will only loop once. return completers[token].complete_command_help(tokens, text, line, begidx, endidx) else: return []
arg_state.min = 0 arg_state.max = 1 arg_state.variable = True else:
text: str, line: str, begidx: int, endidx: int, used_values=()) -> List[str]:
# if arg_choices is a tuple # Let's see if it's a custom completion function. If it is, return what it provides # To do this, we make sure the first element is either a callable # or it's the name of a callable in the application (callable(arg_choices[0]) or (isinstance(arg_choices[0], str) and hasattr(self._cmd2_app, arg_choices[0]) and callable(getattr(self._cmd2_app, arg_choices[0])) ) ):
elif isinstance(arg_choices[0], str) and callable(getattr(self._cmd2_app, arg_choices[0])): completer = getattr(self._cmd2_app, arg_choices[0])
# extract the positional and keyword arguments from the tuple # call the provided function differently depending on the provided positional and keyword arguments elif kw_args is not None: return completer(text, line, begidx, endidx, **kw_args) else: return completer(text, line, begidx, endidx) except TypeError: # assume this is due to an incorrect function signature, return nothing. return [] else: self._resolve_choices_for_arg(action, used_values))
# is the argument a string? If so, see if we can find an attribute in the # application matching the string. except AttributeError: # Couldn't find anything matching the name return []
# is the provided argument a callable. If so, call it else: except TypeError: return []
except TypeError: pass else: # filter out arguments we already used
return
else: else:
else: help_lines = [''] else:
# Redraw prompt and input line
# noinspection PyUnusedLocal """ Performs tab completion against a list
:param text: str - the string prefix we are attempting to match (all returned matches must begin with it) :param line: str - the current input line with leading whitespace removed :param begidx: int - the beginning index of the prefix text :param endidx: int - the ending index of the prefix text :param match_against: Collection - the list being matched against :return: List[str] - a list of possible tab completions """
############################################################################### # Unless otherwise noted, everything below this point are copied from Python's # argparse implementation with minor tweaks to adjust output. # Changes are noted if it's buried in a block of copied code. Otherwise the # function will check for a special case and fall back to the parent function ###############################################################################
# noinspection PyCompatibility,PyShadowingBuiltins,PyShadowingBuiltins """Custom help formatter to configure ordering of help text"""
# if usage is specified, use that usage %= dict(prog=self._prog)
# if no optionals or positionals are available, usage is just prog
# if optionals and positionals are available, calculate usage
# split optionals from positionals # Begin cmd2 customization (separates required and optional, applies to all changes in this function) else: else: # End cmd2 customization
# build full usage string
# wrap the usage parts if it's too long
# Begin cmd2 customization
# break usage into wrappable parts
# End cmd2 customization
# helper for wrapping lines # noinspection PyMissingOrEmptyDocstring,PyShadowingNames else: lines.append(indent + ' '.join(line)) line = [] line_len = len(indent) - 1
# if prog is short, follow it with optionals or positionals # Begin cmd2 customization elif pos_parts: lines = get_lines([prog] + pos_parts, indent, prefix) lines.extend(get_lines(req_parts, indent)) else: lines = [prog] # End cmd2 customization
# if prog is long, put it on its own line else: indent = ' ' * len(prefix) # Begin cmd2 customization parts = pos_parts + req_parts + opt_parts lines = get_lines(parts, indent) if len(lines) > 1: lines = [] lines.extend(get_lines(pos_parts, indent)) lines.extend(get_lines(req_parts, indent)) lines.extend(get_lines(opt_parts, indent)) # End cmd2 customization lines = [prog] + lines
# join lines into usage
# prefix with 'usage:'
else:
# if the Optional doesn't take a value, format is: # -s, --long
# Begin cmd2 customization (less verbose) # if the Optional takes a value, format is: # -s, --long ARGS else:
# End cmd2 customization
result = action.metavar # Begin cmd2 customization (added space after comma) # End cmd2 customization else:
# noinspection PyMissingOrEmptyDocstring return result else:
# Begin cmd2 customization (less verbose) action.nargs_min is not None and action.nargs_max is not None: # End cmd2 customization else:
# noinspection PyCompatibility """Custom argparse class to override error method to change default help text."""
# Begin cmd2 customization """ Allows an error message override to the error() function, useful when forcing a re-parse of arguments with newly required parameters """ self._custom_error_message = custom_message # End cmd2 customization
"""Custom error override. Allows application to control the error being displayed by argparse""" message = self._custom_error_message self._custom_error_message = ''
else: formatted_message += '\n ' + line
# sys.stderr.write('{}\n\n'.format(formatted_message))
"""Copy of format_help() from argparse.ArgumentParser with tweaks to separately display required parameters"""
# usage self._mutually_exclusive_groups)
# description
# Begin cmd2 customization (separate required and optional arguments)
# positionals, optionals and user-defined groups # check if the arguments are required, group accordingly else:
# separately display required arguments
# now display truly optional arguments else:
# End cmd2 customization
# epilog
# determine help from format above
# Override _get_nargs_pattern behavior to use the nargs ranges provided by AutoCompleter action.nargs_min is not None and action.nargs_max is not None:
# if this is an optional action, -- is not allowed
# match the pattern for this action to the arg strings
# raise an exception if we weren't able to find a match if isinstance(action, _RangeAction) and \ action.nargs_min is not None and action.nargs_max is not None: raise ArgumentError(action, 'Expected between {} and {} arguments'.format(action.nargs_min, action.nargs_max))
# This is the official python implementation with a 5 year old patch applied # See the comment below describing the patch def _parse_known_args(self, arg_strings, namespace): # pragma: no cover # replace arg strings that are file references if self.fromfile_prefix_chars is not None: arg_strings = self._read_args_from_files(arg_strings)
# map all mutually exclusive arguments to the other arguments # they can't occur with action_conflicts = {} for mutex_group in self._mutually_exclusive_groups: group_actions = mutex_group._group_actions for i, mutex_action in enumerate(mutex_group._group_actions): conflicts = action_conflicts.setdefault(mutex_action, []) conflicts.extend(group_actions[:i]) conflicts.extend(group_actions[i + 1:])
# find all option indices, and determine the arg_string_pattern # which has an 'O' if there is an option at an index, # an 'A' if there is an argument, or a '-' if there is a '--' option_string_indices = {} arg_string_pattern_parts = [] arg_strings_iter = iter(arg_strings) for i, arg_string in enumerate(arg_strings_iter):
# all args after -- are non-options if arg_string == '--': arg_string_pattern_parts.append('-') for arg_string in arg_strings_iter: arg_string_pattern_parts.append('A')
# otherwise, add the arg to the arg strings # and note the index if it was an option else: option_tuple = self._parse_optional(arg_string) if option_tuple is None: pattern = 'A' else: option_string_indices[i] = option_tuple pattern = 'O' arg_string_pattern_parts.append(pattern)
# join the pieces together to form the pattern arg_strings_pattern = ''.join(arg_string_pattern_parts)
# converts arg strings to the appropriate and then takes the action seen_actions = set() seen_non_default_actions = set()
def take_action(action, argument_strings, option_string=None): seen_actions.add(action) argument_values = self._get_values(action, argument_strings)
# error if this argument is not allowed with other previously # seen arguments, assuming that actions that use the default # value don't really count as "present" if argument_values is not action.default: seen_non_default_actions.add(action) for conflict_action in action_conflicts.get(action, []): if conflict_action in seen_non_default_actions: msg = _('not allowed with argument %s') action_name = _get_action_name(conflict_action) raise ArgumentError(action, msg % action_name)
# take the action if we didn't receive a SUPPRESS value # (e.g. from a default) if argument_values is not SUPPRESS: action(self, namespace, argument_values, option_string)
# function to convert arg_strings into an optional action def consume_optional(start_index):
# get the optional identified at this index option_tuple = option_string_indices[start_index] action, option_string, explicit_arg = option_tuple
# identify additional optionals in the same arg string # (e.g. -xyz is the same as -x -y -z if no args are required) match_argument = self._match_argument action_tuples = [] while True:
# if we found no optional action, skip it if action is None: extras.append(arg_strings[start_index]) return start_index + 1
# if there is an explicit argument, try to match the # optional's string arguments to only this if explicit_arg is not None: arg_count = match_argument(action, 'A')
# if the action is a single-dash option and takes no # arguments, try to parse more single-dash options out # of the tail of the option string chars = self.prefix_chars if arg_count == 0 and option_string[1] not in chars: action_tuples.append((action, [], option_string)) char = option_string[0] option_string = char + explicit_arg[0] new_explicit_arg = explicit_arg[1:] or None optionals_map = self._option_string_actions if option_string in optionals_map: action = optionals_map[option_string] explicit_arg = new_explicit_arg else: msg = _('ignored explicit argument %r') raise ArgumentError(action, msg % explicit_arg)
# if the action expect exactly one argument, we've # successfully matched the option; exit the loop elif arg_count == 1: stop = start_index + 1 args = [explicit_arg] action_tuples.append((action, args, option_string)) break
# error if a double-dash option did not use the # explicit argument else: msg = _('ignored explicit argument %r') raise ArgumentError(action, msg % explicit_arg)
# if there is no explicit argument, try to match the # optional's string arguments with the following strings # if successful, exit the loop else: start = start_index + 1 selected_patterns = arg_strings_pattern[start:] arg_count = match_argument(action, selected_patterns) stop = start + arg_count args = arg_strings[start:stop] action_tuples.append((action, args, option_string)) break
# add the Optional to the list and return the index at which # the Optional's string args stopped assert action_tuples for action, args, option_string in action_tuples: take_action(action, args, option_string) return stop
# the list of Positionals left to be parsed; this is modified # by consume_positionals() positionals = self._get_positional_actions()
# function to convert arg_strings into positional actions def consume_positionals(start_index): # match as many Positionals as possible match_partial = self._match_arguments_partial selected_pattern = arg_strings_pattern[start_index:] arg_counts = match_partial(positionals, selected_pattern)
#################################################################### # Applied mixed.patch from https://bugs.python.org/issue15112 if 'O' in arg_strings_pattern[start_index:]: # if there is an optional after this, remove # 'empty' positionals from the current match
while len(arg_counts) > 1 and arg_counts[-1] == 0: arg_counts = arg_counts[:-1] ####################################################################
# slice off the appropriate arg strings for each Positional # and add the Positional and its args to the list for action, arg_count in zip(positionals, arg_counts): args = arg_strings[start_index: start_index + arg_count] start_index += arg_count take_action(action, args)
# slice off the Positionals that we just parsed and return the # index at which the Positionals' string args stopped positionals[:] = positionals[len(arg_counts):] return start_index
# consume Positionals and Optionals alternately, until we have # passed the last option string extras = [] start_index = 0 if option_string_indices: max_option_string_index = max(option_string_indices) else: max_option_string_index = -1 while start_index <= max_option_string_index:
# consume any Positionals preceding the next option next_option_string_index = min([ index for index in option_string_indices if index >= start_index]) if start_index != next_option_string_index: positionals_end_index = consume_positionals(start_index)
# only try to parse the next optional if we didn't consume # the option string during the positionals parsing if positionals_end_index > start_index: start_index = positionals_end_index continue else: start_index = positionals_end_index
# if we consumed all the positionals we could and we're not # at the index of an option string, there were extra arguments if start_index not in option_string_indices: strings = arg_strings[start_index:next_option_string_index] extras.extend(strings) start_index = next_option_string_index
# consume the next optional and any arguments for it start_index = consume_optional(start_index)
# consume any positionals following the last Optional stop_index = consume_positionals(start_index)
# if we didn't consume all the argument strings, there were extras extras.extend(arg_strings[stop_index:])
# make sure all required actions were present and also convert # action defaults which were not given as arguments required_actions = [] for action in self._actions: if action not in seen_actions: if action.required: required_actions.append(_get_action_name(action)) else: # Convert action default now instead of doing it before # parsing arguments to avoid calling convert functions # twice (which may fail) if the argument was given, but # only if it was defined already in the namespace if (action.default is not None and isinstance(action.default, str) and hasattr(namespace, action.dest) and action.default is getattr(namespace, action.dest)): setattr(namespace, action.dest, self._get_value(action, action.default))
if required_actions: self.error(_('the following arguments are required: %s') % ', '.join(required_actions))
# make sure all required groups had one option present for group in self._mutually_exclusive_groups: if group.required: for action in group._group_actions: if action in seen_non_default_actions: break
# if no actions were used, report the error else: names = [_get_action_name(action) for action in group._group_actions if action.help is not SUPPRESS] msg = _('one of the arguments %s is required') self.error(msg % ' '.join(names))
# return the updated namespace and the extra arguments return namespace, extras |