coma.config.cli

Utilities for overriding config attributes with command line arguments.

class OverrideProtocol(*args, **kwargs)

Protocol for the function signature of coma.config.cli.Override.__call__(). To make use of other default coma components, user-defined alternative implementation should adhere to this same protocol.

Protocol:

Callable[[OverrideData, InstanceKey], None]

with OverrideData and InstanceKey

class OverridePolicy(value)

Policy for handling cases where one parameter will override another in a Callable’s signature. For example, suppose a function defined as:

def fn(x, **kwargs):
    ...

is invoked as:

kwargs = dict(x=1, y=2)
fn(x=3, **kwargs)

How should x be treated?

SILENT_OVERRIDE

Silently override the parameter value. In the above example, the result is x=1 since kwargs["x"] applies last.

VERBOSE_OVERRIDE

Like SILENT_OVERRIDE, but emit a warning listing which parameter is overridden and what the old and new values are.

SILENT_SKIP

Silently skip over any parameter whose value has already been assigned. In the above example, the result is x=3 since kwargs["x"] is silently skipped.

VERBOSE_SKIP

Like SILENT_SKIP, but emit a warning listing which parameter is being skipped and what the current and skipped values are.

RAISE

Raise an error if an override is being attempted.

class OverrideData(config_id: str, configs: dict[str, Config], instance_key: str | None, unknown_args: list[str])

All relevant data for overriding a config instance.

config_id

The identifier of the specific config in configs to possibly override.

Type:

ConfigID

configs

All configs that may be override-relevant. Configs that are not corresponding to config_id inform concerns of override exclusivity and uniqueness. See Override for details.

Type:

Configs

instance_key

The specific instance of the config_id config to override. If None, the latest is used instead. If not None, the same key is used to probe the other configs for override exclusivity and uniqueness. See Override for details.

Type:

InstanceKey, optional

unknown_args

The list of unknown command line arguments, some of which may specify overrides for this config_id config. Typically, this is the second return value of parse_known_args().

Type:

list[str]

class Override(sep: str = '::', exclusive_prefixed: bool = True, exclusive_shared: bool = False, unique_overrides: bool = True)

Attempts to override a config instance’s attributes with command line arguments.

sep

The prefix separation string to use. Can be any string, though some options (such as "=", ":", "{", "}", "[", "]", ",", "'", '"', ".", "$", etc.) will likely cause parsing errors. Use these with caution.

Type:

str

exclusive_prefixed

Whether prefixed overrides should match at most one config.

Type:

bool

exclusive_shared

Whether shared overrides should match at most one config.

Type:

bool

unique_overrides

Whether each override should be defined at most one.

Type:

bool

Similar to from_dotlist() followed by merge(), but with additional features and support for list-like configs. In particular, omegaconf always overrides list configs entirely when merging (discarding the original), whereas here we ensure top-level lists are extended instead. Non-top-level lists (for example, a list as one of the fields of a dict-like config) are treated as conventional omegaconf list configs (override instead of merge).

Specifically, since coma commands accept an arbitrary number of configs, config attributes’ names may end up clashing when using pure omegaconf dot-list notation. To resolve these clashes, a prefix notation is introduced.

Prefix Notation

For a config with identifier config_id, any omegaconf dot-list notation can be prefixed with config_id followed by sep to uniquely link the override to the corresponding config.

In addition, an attempt is made to match all non-prefixed arguments in dot-list notation to the config corresponding to config_id. These shared config overrides are not consumed, and so can be used to override multiple configs without duplication. However, this powerful feature can also be error-prone. To disable it, set exclusive_shared to True. This raises a ValueError if shared overrides match more than one config.

Note

If the config is not structured, omegaconf will happily add any attributes to it. To prevent this, ensure that the config is structured (by instantiating it from a dataclass backend type or by using structured() or set_struct() on a dict-based config).

Finally, prefixes can be shortened to any leading substring. For example, 'long' or even just 'l' matches against the config identifier 'long_config_id'. By default, prefixes have to be unambiguous (i.e., have to match against at most one config identifier). To disable this, set exclusive_prefixed to False. Then, all matching configs to a given prefix will be overridden. Be cautious.

To toggle whether each command line argument should itself be unique, set unique_overrides accordingly.

Note

The uniqueness of command line arguments is based on their (non-prefixed) string value of the field key, not on their effects on any config object. For example, "x=1" and "x=2" are not correctly determined as being not unique because "x" == "x", whereas "a.b=1" and "a[b]=2" will slip by the uniqueness detection because "a.b" != "a[b]" (as a string value).

Note

Regardless of their ordering as command line arguments, all prefixed overrides are processed before all shared overrides. This is not a problem when unique_overrides is False, but can lead to an unexpected outcome when it is True. For example, "x=1" followed by "prefix::x=2" will lead to a final value of x == 1. To avoid this unexpected outcome, makes sure to place all prefixed command line arguments before all shared arguments, or disable shared arguments entirely by setting exclusive_shared to False.

Examples

Resolving clashing dot-list notations with (abbreviated) prefixes:

@dataclass
class Person:
    name: str

@dataclass
class School:
    name: str

@coma.command
def enroll_student(person: Person, school: School):
        ...

Invoking on the command line (assuming sep is "::"):

$ python main.py enroll_student p::name="..." s::name="..."

class ParamData(configs: dict[str, Config] = <factory>, supplemental_configs: dict[str, Config] = <factory>, inline_identifier: str = 'inline', inline_config: ~coma.config.base.Config | None = None, other_parameters: dict[str, ~typing.Any] = <factory>, args_id: str | None = None, kwargs_id: str | None = None)

Utilities for creating configs and other parameters from a Callable’s signature, manipulating these, and calling the Callable using the result.

configs

The configs pulled from the Callable’s signature.

Type:

Configs

supplemental_configs

Any additional configs to manipulate that don’t appear in the Callable’s signature and won’t be called on it. Helpful for providing additional information in the manipulation process.

Type:

Configs

inline_identifier

The identifier of the inline config (whether it exists or not).

Type:

ConfigID

inline_config

The special config collecting all inline parameter from the Callable’s signature.

Type:

Config, optional

other_parameters

Every non-config parameter in the Callable’s signature.

Type:

Parameters

args_id

The identifier of the variadic positional parameter (if any) of the Callable’s signature. Depending on the signature’s specifics, the associated data (if any) is either in configs or in other_parameters.

Type:

Identifier, optional

kwargs_id

The identifier of the variadic keyword parameter (if any) of the Callable’s signature. Depending on the signature’s specifics, the associated data (if any) is either in configs or in other_parameters.

Type:

Identifier

See also

• from_signature():

  For specifics on how the above attributes are inferred from the Callable’s signature.

get_inline_id() → str

Returns the identifier of the inline config (whether it exists or not).

is_inline_id(config_id: str) → bool

Returns whether config_id is the identifier of the inline config.

get_all_configs(include_inline: bool = True) → dict[str, Config]

Returns all configs and supplemental_configs. If include_inline is True and an inline config exists, it is also returned.

Parameters:

include_inline (bool) – Whether the inline config is also returned. Ignored if an inline config does not exist.

Returns:

All configs, possibly including inline.

Return type:

Configs

is_serializable(config_id: str) → bool

Returns whether config_id corresponds to a serializable config. All configs are serializable except for variadic positional (*args), variadic keyword (**kwargs), and the special inline_config.

Parameters:

config_id (ConfigID) – A config identifier.

Returns:

Whether config_id corresponds to a serializable config.

Return type:

bool

select(*ids: str, default: ~typing.Any = <object object>) → dict[str, Any]

Returns the objects associated with the selected ids, keyed by identifier. These can be in any of configs, supplemental_configs, other_parameters, or inline_config. For inline_config, only the aggregate inline config can be retrieved, not the individual parameters that collectively make it up. To retrieve it, use get_inline_id().

Parameters:

• *ids (Identifier) – Collection of identifiers for which to select data.

• default (Any) – If given, the value for identifiers in ids with no existing data is set to this value. If not given, raise a KeyError if data does not exist for at least one identifier.

Returns:

A mapping between the selected identifiers and their associated value.

Return type:

dict[Identifier, typing.Any]

Raises:

KeyError – If default is not specified, and data does not exist for at least one identifier in ids.

select_config(*config_ids: str, default: ~typing.Any = <object object>) → dict[str, Config]

Returns the configs associated with the selected config_ids, keyed by identifier. These can be in any of configs, supplemental_configs, or inline_config. For inline_config, only the aggregate inline config can be retrieved, not the individual parameters that collectively make it up. To retrieve it, use get_inline_id().

Parameters:

• *config_ids (ConfigID) – Collection of config identifiers for which to select configs.

• default (Any) – If given, missing configs for identifiers in config_ids are set to this value. If not given, raise a KeyError if a config does not exist for at least one identifier.

Returns:

A mapping between the selected config identifiers and their associated configs.

Return type:

Configs

Raises:

KeyError – If default is not specified, and a config does not exist for at least one config identifier in config_ids; or if any identifier in config_ids refers to a parameter instead of a config.

get(identifier: str, default: ~typing.Any = <object object>) → Any

Returns the object associated with identifier. This object can be in any of configs, supplemental_configs, other_parameters, or inline_config. For inline_config, only the aggregate inline config can be retrieved, not the individual parameters that collectively make it up. To retrieve it, use get_inline_id().

Parameters:

• identifier (Identifier) – The identifier for which to retrieve data.

• default (Any) – If given, returns this value if no data exists for identifier. If not given, raise a KeyError on missing data.

Returns:

The data associated with identifier, or default if no such data exists and default is given.

Return type:

Any

Raises:

KeyError – If default is not given, and data for identifier does not exist.

get_config(config_id: str, default: ~typing.Any = <object object>) → Config

Returns the config associated with config_id. The config can be in any of configs, supplemental_configs, or inline_config. For inline_config, only the aggregate inline config can be retrieved, not the individual parameters that collectively make it up. To retrieve it, use get_inline_id().

Parameters:

• config_id (ConfigID) – The config identifier for which to retrieve a config.

• default (Any) – If given, returns this value if no config exists for config_id. If not given, raise a KeyError on missing.

Returns:

The config associated with config_id, or default if no such config exists and default is given.

Return type:

Config

Raises:

KeyError – If default is not given, and a config for config_id does not exist; or if config_id refers to a parameter instead of a config.

replace(identifier: str, new_value: Any) → None

Replaces the data associated with identifier. This object can be in any of configs, supplemental_configs, or other_parameters, or inline_config. For inline_config, only the aggregate inline config can be replaced (as a whole), not the individual parameters that collectively make it up. Use get_inline_id().

Parameters:

• identifier (Identifier) – The identifier for which to replace the data.

• new_value (Any) – The new value.

Raises:

KeyError – If data for identifier does not already exist. To add new data for a new identifier, add an entry directly to the desired attribute dictionary.

delete(*ids: str, raise_on_missing: bool = True) → None

Deletes the data associated with each identifier in ids. These can be in any of configs, supplemental_configs, or other_parameters, or inline_config. For inline_config, only the aggregate inline config can be deleted, not the individual parameters that collectively make it up. To delete it, use get_inline_id().

Parameters:

• *ids (Identifier) – Collection of identifiers for which to delete data.

• raise_on_missing (bool) – If True, raise a KeyError if data does not already exist for at least one identifier in ids. If False, silently ignore missing identifiers.

Raises:

KeyError – If raise_on_missing is True, and data does not already exist for at least one identifier in ids.

classmethod from_signature(signature: Signature, *, args_as_config: bool, kwargs_as_config: bool, inline_identifier: str, inline: Sequence[str | tuple[str, Callable[[], Any]]], supplemental_configs: dict[str, Any]) → ParamData

Returns a ParamData filled according to the specifics of signature and the various additional criteria. All supplemental_configs are invariably treated as configs and converted into Configs without additional checks besides ensuring that the identifiers of supplemental configs do not clash with any identifiers in signature or with inline_identifier.

The distinction between configs and other_parameters in signature is determined by inspecting its type annotation (if any), its default value (if any), its kind, and whether the parameter identifier is marked as inline.

An inline parameter is a one-off config field. Specifically, all inline parameters are aggregated into a special inline_config, which is backed by a programmatic dataclass. This provides all the rigorous runtime type validation of a standard dataclass-backed omegaconf config without requiring a dataclass to be created just for those one-off fields. Moreover, inline configs are considered non-serializable by default.

configs take priority over other_parameters: If a parameter can be considered a config (as per the criteria below), it is treated as one. A non-config parameter is assumed to be regular parameters unless it meets the inline criteria (below) in which case it is treated as inline.

Criteria for interpreting a parameter as a config:

1. The parameter has a type annotation that exactly matches one of list, dict, or any dataclass type. We refer to these as “config annotations”.

2. The parameter does not have a default value. Since configs employ a dedicated initialization protocol, default parameter values are not needed.

Note

This means that a convenient way to ensure that a config-annotated parameter is interpreted as a regular parameter is to give it a default. For example, list_cfg: list is interpreted as a config whereas non_cfg_list: list = None is interpreted as a regular parameter.

3. The parameter’s identifier in not found in inline. Even if the parameter has a config annotation, being in inline disqualifies.

4. Special case: Because variadic positional (*args) and variadic keyword (**kwargs) parameters cannot be assigned defaults in Python, and because they can never be marked as inline, criteria (2) and (3) cannot be used. Instead, use the flags args_as_config and kwargs_as_config.

Checklist for interpreting a parameter as inline:

1. The parameter has a type annotation. A missing annotation is disqualifying.

2. The parameter has a default value. A missing default value is disqualifying.

Note

On mutable inline default values. Because it is un-Pythonic to declare a mutable default value in a function definition, it can be tricky to set a good default value for inline parameters. So, items in inline can consist of either just a ParamIDs, or be 2-tuple where the first value is a ParamID and the second value is a default_factory conforming to the requirements of the same argument in dataclasses.field(). It is an error to give both a signature-level default and an inline-level default factory.

3. The default value is a valid instance of the annotation type. If not, the underlying omegaconf call will raise a ValidationError.

4. The parameter’s identifier is found in inline. If this is true, but one of the above criteria are violated, an error is raised.

5. The parameter’s kind is not variadic positional or variadic keyword. These two special cases can be configs or regular parameters, but never inline.

Example

Even though Data below is a dataclass, it is not considered a config because of its non-config annotation and its None default value (either one of which is disqualifying on its own). On the other hand, out_file can be overridden on the command line because of its inline declaration. Any list-like command line arguments are not fed to *args because args_as_config is False whereas the opposite is true for **kwargs and dict-like command line arguments because kwargs_as_config is True (by default).

@dataclass
class Data:
    x: int = 42

@dataclass
class Config:
    y: float = 3.14

@coma.command(
    signature_inspector=SignatureInspector(
        args_as_config=False, inline=["out_file"],
    ),
)
def cmd(
        cfg: Config,
        data: Optional[Data] = None,
        out_file: str = "out.txt",
        *args,
        **kwargs,
    ):
    print("cfg is:", cfg)
    print("data is:", data or Data())
    print("out_file is:", out_file)
    print("*args is:", args)
    print("*kwargs is:", kwargs)

Invoking on the command line with some overrides results in the following:

$ python main.py cmd x=1 y=2 z inline::out_file=foo.txt
cfg is: Config(y=2.0)
data is: Data(x=42)
out_file is: foo.txt
*args is: ()
*kwargs is: {'x': 1, 'y': 2}

In the example above, notice that:

1. The list-like argument 'z' is not in *args because *args is not a config.

2. **kwargs includes both dict-like arguments.

3. out_file is overridden.

4. out_file is prefixed with the reserved config identifier "inline" to prevent **kwargs from also containing an "out_file" entry. This prevents a runtime error resulting from "out_file" appearing multiple times in the Callable’s parameter list.

5. Because cfg is a config, it’s y attribute was also overridden (this is the default override model where overrides are applied as widely as possible; to disable, see Override).

6. Because data is not a config, it’s x attribute is not overridden. In fact, because the default value of data is not replaced in any command() hook, its value when invoking this command will invariably be None. Use replace() in a hook to change this.

Parameters:

• signature (inspect.Signature) – The signature of the Callable from which to create and fill a ParamData.

• args_as_config (bool) – Whether to treat the variadic positional parameter in signature (if any) as a list config or as a regular parameter.

• kwargs_as_config (bool) – Whether to treat the variadic keyword parameter in signature (if any) as a dict config or as a regular parameter.

• inline_identifier (ConfigID) – The config identifier to use for the inline config.

• inline (Sequence) – The parameters in signature to mark as inline config parameters (if any). Items in this sequence must either be ParamID s or be 2-tuple where the first value is a ParamID and the second value is a default_factory conforming to the requirements of dataclasses.field().

• supplemental_configs (Parameters) – Any additional parameters not in signature to convert into configs.

Returns:

Filled according to the specifics of signature and the various allowance criteria.

Return type:

ParamData

Raises:

ValueError – If any parameter identifier in supplemental_configs matches an existing parameter in signature; or if any parameter identifier or supplemental config identifier is the (case-insensitive) inline_identifier; or if any parameter is misspecified for its type (e.g., missing a default value on an inline parameter).

call_on(fn: Callable[[...], T], policy: OverridePolicy, instance_key: str | None = None) → T

Calls fn using the current state of configs and other_parameters, returning the return value of fn.

Parameters:

• fn (Callable) – The Callable to call using internal signature data.

• policy (OverridePolicy) – Policy for when some keyword-based parameter would override another parameter.

• instance_key (InstanceKey, optional) – The specific instance of the various self.configs to pass to fn. If None, the latest is used instead. If not None, the same key is used for all self.configs and must exist for all of them.

Returns:

The return value from calling fn.

Raises:

• ValueError – If one of the parameters in the signature of fn cannot be filled by internal data; or if at least one of the configs was never instantiated; or if policy causes a raise on a parameter.

• KeyError – If instance_key is not a valid key for at least one config.

• Others – As may be raised by the underlying omegaconf implementation of the configs.

class SignatureInspectorProtocol(*args, **kwargs)

Protocol for a command signature inspector that takes in a command signature object and supplemental configs and returns a (possible subclassed) instance of ParamData.

To make use of other default coma components, user-defined alternative implementation should adhere to this same protocol and to the protocols (all public methods except from_signature()) of ParamData.

Protocol:

Callable[[inspect.Signature, dict[ConfigID, Any]], ParamData]

with ConfigID and ParamData.

class SignatureInspector(args_as_config: bool = True, kwargs_as_config: bool = True, inline_identifier: str = 'inline', inline: Sequence[str | tuple[str, Callable[[], Any]]] = ())

Lightweight wrapper around from_signature() acting as the default SignatureInspectorProtocol when no user-defined alternative is provided.

args_as_config

Whether to treat the variadic positional parameter in the command signature (if any) as a list config or a regular parameter.

Type:

bool

kwargs_as_config

Whether to treat the variadic keyword parameter in the command signature (if any) as a dict config or a regular parameter.

Type:

bool

inline_identifier

The config identifier to use for the inline config (regardless of whether there is one).

Type:

ConfigID

inline

The parameters in the command signature to mark as inline config parameters (if any).

Type:

Sequence, of str or tuple[str, Callable]

See also

• from_signature()