API references#

analysis#

config#

class utilsd.config.BaseConfig(*args, **kwds)#

Interface for config

asdict() → Dict[str, Any]#

Convert config to JSON object.

classmethod fromcli(*, shortcuts: Optional[Dict[str, str]] = None, allow_rest: bool = False, receive_nni: bool = False) → Union[T, Tuple[T, List[str]]]#

Parse command line from a mandatory config file and optional arguments, like

python main.py exp.yaml –learning_rate 1e-4

Parameters:

  • shortcuts (Optional[Dict[str, str]], optional) – To create short command line arguments. In the form of {'-lr': 'trainer.learning_rate'}. Defaults to None.

  • allow_rest (bool, optional) – If false, check if there is any unrecognized command line arguments. If false, ignore them. Defaults to False.

  • receive_nni (bool, optional) – Receive next parameters from NNI. Defaults to False.

Raises:

ValidationError – Config file is invalid.

Returns:

If allow_rest, a tuple of parsed config and the rest of command line arguments will be returned. Otherwise, the parse config only.

Return type:

Union[PythonConfig, Tuple[PythonConfig, List[str]]]

classmethod fromdict(data: Any) → T#

Create a config by parsing data.

Parameters:

data (any) – Data to parse. Should be a dict in most cases.

classmethod fromfile(filename: Path, **kwargs) → T#

Create a config by reading from filename.

Parameters:

filename (path) – Path to read from

meta() → dict#

Export metadata.

class utilsd.config.ClassConfig(*args, **kwds)#

Dataclass based on __init__ of one single class.

class utilsd.config.PythonConfig#

The original base class for config classes. Deprecated now.

class utilsd.config.Registry(clsname, bases, attrs, name=None)#

Registry holds a collection of modules, that can be looked up with their names. Useful when configuring the system with config files (e.g., JSON, YAML).

Examples

To create a registry:

class Converters(metaclass=Registry, name='converter'):
    pass

To register a module into registry:

@Converters.register_module()
class MyConverter:
    ...
class utilsd.config.RegistryConfig(*args, **kwds)#

Dataclass based on __init__ of classes in a specific registry. Special field type is used to specify the targeted class name.

class utilsd.config.RuntimeConfig(seed: int = 42, output_dir: Optional[Path] = None, checkpoint_dir: Optional[Path] = None, tb_log_dir: Optional[Path] = None, debug: bool = False, use_cuda: bool = True)#

Built-in runtime config that supports most commonly used fields for a deep learning experiment, including:

  • seed (int)

  • output_dir (path)

  • checkpoint_dir (path)

  • tb_log_dir (path)

  • debug (bool, default false)

  • use_cuda (bool, default true)

This can be used as an argument to setup an experiment: setup_experiment().

class utilsd.config.SubclassConfig(*args, **kwds)#

Dataclass based on __init__ of classes inheriting a specific base class. Special field type is used to specify the targeted class import path.

exception utilsd.config.ValidationError#
utilsd.config.configclass(cls: T) → Union[T, BaseConfig]#

Make a class a config class. The class should be written like a dataclass, and annotated with @configclass. Then it can load content from YAML files or python dict.

Returns:

The annotated config class.

Return type:

BaseConfig

Examples

To create a config class:

@configclass
class UserConf:
    username: str
    password: str
    attempt: Optional[int] = None

To load from a file or content:

config = UserConf.fromfile('config.yml')
config = UserConf.fromdict({'username': 'john', 'password': 123})

To parse from commnad line:

config = UserConf.fromcli()

Then in command line, it accepts a file which is considered as “base config”. The base config should be load-able with UserConf.fromfile. Most of the fields in the config can be overridden with command line arguments.

$ python test.py config.yml --username "grace"

Use -h for help.

$ python test.py config.yml -h

It is essentially a dataclass, and thus all the methods that apply to dataclass similarly apply.

UserConf('john', '123', attempt=0)

fileio#

class utilsd.fileio.Config(cfg_dict=None, cfg_text=None, filename=None)#

A facility for config and config files. It supports common file formats as configs: python/json/yaml. The interface is the same as a dict object and also allows access config values as attributes. .. rubric:: Example

>>> cfg = Config(dict(a=1, b=dict(b1=[0, 1])))
>>> cfg.a
1
>>> cfg.b
{'b1': [0, 1]}
>>> cfg.b.b1
[0, 1]
>>> cfg = Config.fromfile('tests/data/config/a.py')
>>> cfg.filename
"/home/kchen/projects/mmcv/tests/data/config/a.py"
>>> cfg.item4
'test'
>>> cfg
"Config [path: /home/kchen/projects/mmcv/tests/data/config/a.py]: "
"{'item1': [1, 2], 'item2': {'a': 0}, 'item3': True, 'item4': 'test'}"
static auto_argparser(description=None)#

Generate argparser from config file automatically (experimental)

static fromstring(cfg_str, file_format)#

Generate config from config str. :param cfg_str: Config str. :type cfg_str: str :param file_format: Config file format corresponding to the :type file_format: str :param config str. Only py/yml/yaml/json type are supported now!:

Returns:

Config: Config obj.

Return type:

obj

merge_from_dict(options, allow_list_keys=True)#

Merge list into cfg_dict. Merge the dict parsed by MultipleKVAction into this cfg. .. rubric:: Examples

>>> options = {'model.backbone.depth': 50,
...            'model.backbone.with_cp':True}
>>> cfg = Config(dict(model=dict(backbone=dict(type='ResNet'))))
>>> cfg.merge_from_dict(options)
>>> cfg_dict = super(Config, self).__getattribute__('_cfg_dict')
>>> assert cfg_dict == dict(
...     model=dict(backbone=dict(depth=50, with_cp=True)))
# Merge list element
>>> cfg = Config(dict(pipeline=[
...     dict(type='LoadImage'), dict(type='LoadAnnotations')]))
>>> options = dict(pipeline={'0': dict(type='SelfLoadImage')})
>>> cfg.merge_from_dict(options, allow_list_keys=True)
>>> cfg_dict = super(Config, self).__getattribute__('_cfg_dict')
>>> assert cfg_dict == dict(pipeline=[
...     dict(type='SelfLoadImage'), dict(type='LoadAnnotations')])
Parameters:

  • options (dict) – dict of configs to merge from.

  • allow_list_keys (bool) – If True, int string keys (e.g. ‘0’, ‘1’) are allowed in options and will replace the element of the corresponding index in the config if the config is a list. Default: True.

class utilsd.fileio.ConfigDict(*args, **kwargs)#
class utilsd.fileio.DictAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)#

argparse action to split an argument into KEY=VALUE form on the first = and append to a dictionary. List options can be passed as comma separated values, i.e ‘KEY=V1,V2,V3’, or with explicit brackets, i.e. ‘KEY=[V1,V2,V3]’. It also support nested brackets to build list/tuple values. e.g. ‘KEY=[(V1,V2),(V3,V4)]’

utilsd.fileio.dict_from_file(filename, key_type=<class 'str'>)#

Load a text file and parse the content as a dict.

Each line of the text file will be two or more columns splited by whitespaces or tabs. The first column will be parsed as dict keys, and the following columns will be parsed as dict values.

Parameters:

  • filename (str) – Filename.

  • key_type (type) – Type of the dict’s keys. str is user by default and type conversion will be performed if specified.

Returns:

The parsed contents.

Return type:

dict

utilsd.fileio.dump(obj, file=None, file_format=None, **kwargs)#

Dump data to json/yaml/pickle strings or files.

This method provides a unified api for dumping data as strings or to files, and also supports custom arguments for each file format.

Parameters:

  • obj (any) – The python object to be dumped.

  • file (str or Path or file-like object, optional) – If not specified, then the object is dump to a str, otherwise to a file specified by the filename or file-like object.

  • file_format (str, optional) – Same as load().

Returns:

True for success, False otherwise.

Return type:

bool

utilsd.fileio.list_from_file(filename, prefix='', offset=0, max_num=0)#

Load a text file and parse the content as a list of strings.

Parameters:

  • filename (str) – Filename.

  • prefix (str) – The prefix to be inserted to the begining of each item.

  • offset (int) – The offset of lines.

  • max_num (int) – The maximum number of lines to be read, zeros and negatives mean no limitation.

Returns:

A list of strings.

Return type:

list[str]

utilsd.fileio.load(file, file_format=None, **kwargs)#

Load data from json/yaml/pickle files.

This method provides a unified api for loading data from serialized files.

Parameters:

  • file (str or Path or file-like object) – Filename or a file-like object.

  • file_format (str, optional) – If not specified, the file format will be inferred from the file extension, otherwise use the specified one. Currently supported formats include “json”, “yaml/yml” and “pickle/pkl”.

Returns:

The content from the file.

search#

misc#

Currently it’s same as https://kaiyangzhou.github.io/deep-person-reid/_modules/torchreid/utils/avgmeter.html#MetricMeter

class utilsd.avgmeter.AverageMeter#

Computes and stores the average and current value.

Examples::
>>> # Initialize a meter to record loss
>>> losses = AverageMeter()
>>> # Update meter after every minibatch update
>>> losses.update(loss_value, batch_size)
class utilsd.avgmeter.MetricMeter(delimiter='  ')#

A collection of metrics.

Source: KaiyangZhou/Dassl.pytorch

Examples::
>>> # 1. Create an instance of MetricMeter
>>> metric = MetricMeter()
>>> # 2. Update using a dictionary as input
>>> input_dict = {'loss_1': value_1, 'loss_2': value_2}
>>> metric.update(input_dict)
>>> # 3. Convert to string and print
>>> print(str(metric))
class utilsd.earlystop.EarlyStopStatus(value)#

An enumeration.

utilsd.logging.print_log(msg, logger=None, level=20)#

Print a log message.

Parameters:

  • msg (str) – The message to be logged.

  • logger (logging.Logger | str | None) – The logger to be used. Some special loggers are: - “silent”: no message will be printed. - other str: the logger obtained with get_root_logger(logger). - None: The print() method will be used to print log messages.

  • level (int) – Logging level. Only available when logger is a Logger object or “root”.

utilsd.logging.setup_logger(name, log_file=None, log_level=20, file_mode='w')#

Initialize and get a logger by name.

If the logger has not been initialized, this method will initialize the logger by adding one or two handlers, otherwise the initialized logger will be directly returned. During initialization, a StreamHandler will always be added. If log_file is specified and the process rank is 0, a FileHandler will also be added.

Parameters:

  • name (str) – Logger name.

  • log_file (str | None) – The log filename. If specified, a FileHandler will be added to the logger.

  • log_level (int) – The logger level. Note that only the process of rank 0 is affected, and other processes will set the level to “Error” thus be silent most of the time.

  • file_mode (str) – The file mode used in opening log file. Defaults to ‘w’.

Returns:

The expected logger.

Return type:

logging.Logger