Logging

Functions to log data for experiments.

All functions can be accessed under the module: exputils.data.logging

Usage

Import the logging module and directly use its functions to log values or objects. It is not necessary to create a logging object.

Scalars and arrays will be logged as numpy arrays in the memory.

To write the logged values to disk it is necessary to call the save function.

Example:

# import the logging module as log
import exputils.data.logging as log

# use the log to log some scalars under the name 'val'
for i in range(10):
    log.add_value('val', i)

# save the log, only then will the log be written from memory to a file!
log.save()

Writting

add_value

Adds a value to a log entry with optional parallel TensorBoard logging.

Parameters:

Name	Type	Description	Default
name	str	The name of the entry where the value is added.	required
value	Any	The value to be added. Can be a scalar or an array.	required
log_to_tb	bool	Defines of the value should be logged to TensorBoard in parallel to the standard log. If True, log the value to TensorBoard. If False, do not log the value to TensorBoard. If not specified, then it gets logged if TensorBoard is globally activated. See activate_tensorboard for more details.	None
tb_global_step	int	If logging to TensorBoard is active, then this is the global step value to record with the value in TensorBoard.	None
tb_walltime	float	If logging to TensorBoard is active, then this is an optional override for the walltime in TensorBoard.	None

Source code in exputils/data/logging.py

def add_value(name: str,
              value,
              log_to_tb: Optional[bool] = None,
              tb_global_step: Optional[int] = None,
              tb_walltime: Optional[float] = None):
    """
    Adds a value to a log entry with optional parallel TensorBoard logging.

    Parameters:
        name (str):
            The name of the entry where the value is added.
        value (Any):
            The value to be added. Can be a scalar or an array.
        log_to_tb (bool):
            Defines of the value should be logged to TensorBoard in parallel to the standard log.
            If True, log the value to TensorBoard.
            If False, do not log the value to TensorBoard.
            If not specified, then it gets logged if TensorBoard is globally activated.
            See [activate_tensorboard][exputils.data.logging.activate_tensorboard] for more details.
        tb_global_step (int):
            If logging to TensorBoard is active, then this is the global step value to record with
            the value in TensorBoard.
        tb_walltime (float):
            If logging to TensorBoard is active, then this is an optional override for the walltime
            in TensorBoard.
    """
    log.add_value(name, value, log_to_tb, tb_global_step, tb_walltime)

add_scalar

Adds a scalar value to a log entry with optional parallel TensorBoard logging.

Note: Has the same functionality as add_value and exists to have a similar named log function as TensorBoard.

Parameters:

Name	Type	Description	Default
name	str	The name of the entry where the value is added.	required
scalar	Any	The scalar value to be added.	required
log_to_tb	bool	Defines of the value should be logged to TensorBoard in parallel to the standard log. If True, log the value to TensorBoard. If False, do not log the value to TensorBoard. If not specified, then it gets logged if TensorBoard is globally activated. See activate_tensorboard for more details.	None
tb_global_step	int	If logging to TensorBoard is active, then this is the global step value to record with the value in TensorBoard.	None
tb_walltime	float	If logging to TensorBoard is active, then this is an optional override for the walltime in TensorBoard.	None

Source code in exputils/data/logging.py

def add_scalar(name: str,
              scalar,
              log_to_tb: Optional[bool] = None,
              tb_global_step: Optional[int] = None,
              tb_walltime: Optional[float] = None):
    """
    Adds a scalar value to a log entry with optional parallel TensorBoard logging.

    Note: Has the same functionality as [add_value][exputils.data.logging.add_value] and exists to
    have a similar named log function as TensorBoard.

    Parameters:
        name (str):
            The name of the entry where the value is added.
        scalar (Any):
            The scalar value to be added.
        log_to_tb (bool):
            Defines of the value should be logged to TensorBoard in parallel to the standard log.
            If True, log the value to TensorBoard.
            If False, do not log the value to TensorBoard.
            If not specified, then it gets logged if TensorBoard is globally activated.
            See [activate_tensorboard][exputils.data.logging.activate_tensorboard] for more details.
        tb_global_step (int):
            If logging to TensorBoard is active, then this is the global step value to record with
            the value in TensorBoard.
        tb_walltime (float):
            If logging to TensorBoard is active, then this is an optional override for the walltime
            in TensorBoard.
    """
    log.add_scalar(name, scalar, log_to_tb, tb_global_step, tb_walltime)

add_histogram

Adds a histogram which is a one-dimensional array a log entry with optional parallel TensorBoard logging.

This allows to add the values as a histogram plot to TensorBoard.

Parameters:

Name	Type	Description	Default
name	str	The name of the entry where the value is added.	required
values	Any	The array of values to be added.	required
log_to_tb	bool	Defines of the value should be logged to TensorBoard in parallel to the standard log. If True, log the value to TensorBoard. If False, do not log the value to TensorBoard. If not specified, then it gets logged if TensorBoard is globally activated. See activate_tensorboard for more details.	None
tb_global_step	int	If logging to TensorBoard is active, then this is the global step value to record with the value in TensorBoard.	None
tb_walltime	float	If logging to TensorBoard is active, then this is an optional override for the walltime in TensorBoard.	None

Source code in exputils/data/logging.py

def add_histogram(name: str,
                  values,
                  log_to_tb: Optional[bool] = None,
                  tb_global_step: Optional[int] = None,
                  tb_walltime: Optional[float] = None):
    """
    Adds a histogram which is a one-dimensional array a log entry with optional parallel TensorBoard
    logging.

    This allows to add the values as a histogram plot to TensorBoard.

    Parameters:
        name (str):
            The name of the entry where the value is added.
        values (Any):
            The array of values to be added.
        log_to_tb (bool):
            Defines of the value should be logged to TensorBoard in parallel to the standard log.
            If True, log the value to TensorBoard.
            If False, do not log the value to TensorBoard.
            If not specified, then it gets logged if TensorBoard is globally activated.
            See [activate_tensorboard][exputils.data.logging.activate_tensorboard] for more details.
        tb_global_step (int):
            If logging to TensorBoard is active, then this is the global step value to record with
            the value in TensorBoard.
        tb_walltime (float):
            If logging to TensorBoard is active, then this is an optional override for the walltime
            in TensorBoard.
    """
    log.add_histogram(name, values, log_to_tb, tb_global_step, tb_walltime)

add_object

Adds an object to a log entry. Objects are stored in a list and saved as dill files.

Parameters:

Name	Type	Description	Default
name	str	The name of the log entry where the object is added.	required
obj	object	The object to be added to the log.	required

Source code in exputils/data/logging.py

def add_object(name: str,
               obj: object):
    """
    Adds an object to a log entry. Objects are stored in a list and saved as dill files.

    Parameters:
        name (str):
            The name of the log entry where the object is added.
        obj (object):
            The object to be added to the log.
    """
    log.add_object(name, obj)

add_single_object

Logs a single object which is directly written to a dill file and not stored in memory.

Parameters:

Name	Type	Description	Default
name	str	The name of the object which is used for the filename.	required
obj	object	The object to be logged.	required
directory	str	Optional directory path where the dill file for the object is saved. Default is the log directory.	None

Source code in exputils/data/logging.py

def add_single_object(name: str,
                      obj: object,
                      directory: Optional[str] = None):
    """
    Logs a single object which is directly written to a dill file and not stored in memory.

    Parameters:
        name (str):
            The name of the object which is used for the filename.
        obj (object):
            The object to be logged.
        directory (str):
            Optional directory path where the dill file for the object is saved.
            Default is the log directory.
    """
    log.add_single_object(name, obj, directory=directory)

save

Saves the log. All logged values are stored in memory and only written to disk when this function is called.

Parameters:

Name	Type	Description	Default
directory	str	Optional directory path where the dill file for the object is saved. Default is the log directory.	None

Source code in exputils/data/logging.py

def save(directory: Optional[str] = None):
    """
    Saves the log.
    All logged values are stored in memory and only written to disk when this function is called.

    Parameters:
        directory (str):
            Optional directory path where the dill file for the object is saved.
            Default is the log directory.
    """
    log.save(directory=directory)

clear

Clears the data of all or a specific log entry.

Data that has been logged after the save function was called will be lost.

Parameters:

Name	Type	Description	Default
name	str	Name of the log entry. If no name is given, then all log entries will be cleared.	None

Source code in exputils/data/logging.py

def clear(name: Optional[str] = None):
    """
    Clears the data of all or a specific log entry.

    :warning: Data that has been logged after the [save][exputils.data.logging.save] function
    was called will be lost.

    Parameters:
        name (str):
            Name of the log entry.
            If no name is given, then all log entries will be cleared.
    """
    log.clear(name=name)

reset

Resets the log which deletes all data in the memory and resets all changed configuration such as the directory path of the log.

Data that has been logged after the save function was called will be lost.

Source code in exputils/data/logging.py

def reset():
    """
    Resets the log which deletes all data in the memory and resets all changed configuration such
    as the directory path of the log.

    :warning: Data that has been logged after the [save][exputils.data.logging.save] function
    was called will be lost.
    """
    global log
    log = Logger()

Reading

Values that were logged can also be accessed. It is also possible to load a complete log from disk. This can be used to continue experiment and add new values to an existing log.

contains

Check if a log entry for the given name exists.

Parameters:

Name	Type	Description	Default
name	str	The name to be checked in the log.	required

Returns:

Name	Type	Description
is_contained	bool	True if a log for the name exists, otherwise False.

Source code in exputils/data/logging.py

def contains(name: str) -> bool:
    """
    Check if a log entry for the given name exists.

    Parameters:
        name (str): The name to be checked in the log.

    Returns:
        is_contained (bool): True if a log for the name exists, otherwise False.
    """
    return (name in log)

items

Returns all log entries as a list of tuples with the name and values of the entries.

Returns:

Name	Type	Description
entries	list	All logged entries.

Source code in exputils/data/logging.py

def items() -> list:
    """
    Returns all log entries as a list of tuples with the name and values of the entries.

    Returns:
        entries (list): All logged entries.
    """
    return log.items()

get_item

Returns the logged data for a certain entry.

Parameters:

Name	Type	Description	Default
name	str	Name of the log entry.	required

Returns:

Type	Description
object	Logged data. Usually in form of a numpy array.

Source code in exputils/data/logging.py

def get_item(name: str) -> object:
    """
    Returns the logged data for a certain entry.

    Parameters:
        name (str):
            Name of the log entry.

    Returns:
        Logged data. Usually in form of a numpy array.
    """
    return log[name]

get_values

Returns the logged data for a certain entry.

Note: Has the same functionality as get_item and exists to have a similar named log function as TensorBoard.

Parameters:

Name	Type	Description	Default
name	str	Name of the log entry.	required

Returns:

Type	Description
	Logged data. Usually in form of a numpy array.

Source code in exputils/data/logging.py

def get_values(name: str):
    """
    Returns the logged data for a certain entry.

    Note: Has the same functionality as [get_item][exputils.data.logging.get_item] and exists to
    have a similar named log function as TensorBoard.

    Parameters:
        name (str):
            Name of the log entry.

    Returns:
        Logged data. Usually in form of a numpy array.
    """
    return log[name]

get_objects

Returns the logged objects for a certain entry.

Parameters:

Name	Type	Description	Default
name	str	Name of the log entry.	required

Returns:

Name	Type	Description
objects	list	Logged objects.

Source code in exputils/data/logging.py

def get_objects(name: str) -> list:
    """
    Returns the logged objects for a certain entry.

    Args:
        name (str): Name of the log entry.

    Returns:
        objects (list): Logged objects.
    """
    return log[name]

load

Loads entries from a log directory into the log. Afterwards the loaded entries can be accessed via the items and get_item functions.

Parameters:

Name	Type	Description	Default
directory	str	Optional directory path where the dill file for the object is saved. Default is the log directory.	None
load_objects	bool	True if objects (dill files) that are in the directory should also be loaded. Default is False to avoid unintended large loads of objects.	False

Source code in exputils/data/logging.py

def load(directory: Optional[str] = None,
         load_objects: bool = False):
    """
    Loads entries from a log directory into the log.
    Afterwards the loaded entries can be accessed via the [items][exputils.data.logging.items] and
    [get_item][exputils.data.logging.get_item] functions.

    Parameters:
        directory (str):
            Optional directory path where the dill file for the object is saved.
            Default is the log directory.
        load_objects (bool):
            True if objects (dill files) that are in the directory should also be loaded.
            Default is False to avoid unintended large loads of objects.
    """
    log.load(directory=directory, load_objects=load_objects)

load_single_object

Loads a single object from the log folder and returns it. The object is not stored in the log memory.

Parameters:

Name	Type	Description	Default
name	str	Name of the object which is used for the filename. Either with or without the '.dill' extension.	required

Returns:

Name	Type	Description
obj	object	Loaded object.

Source code in exputils/data/logging.py

def load_single_object(name: str) -> object:
    """
    Loads a single object from the log folder and returns it.
    The object is not stored in the log memory.

    Parameters:
        name (str):
            Name of the object which is used for the filename.
            Either with or without the `'.dill'` extension.

    Returns:
        obj (object): Loaded object.
    """
    return log.load_single_object(name)

Tensorboard

The log has the ability to log values in parallel to Tensorboard which can be used to visualize them while an experiment is running.

tensorboard

Returns the tensorboard SummaryWriter object used to log values to TensorBoard.

Returns:

Name	Type	Description
writer	SummaryWriter	TensorBoard SummaryWriter object.

Source code in exputils/data/logging.py

def tensorboard():
    """
    Returns the tensorboard SummaryWriter object used to log values to TensorBoard.

    Returns:
        writer (SummaryWriter): TensorBoard SummaryWriter object.
    """
    return log.tensorboard

create_tensorboard

Creates the SummaryWriter object used to log values to TensorBoard. This allows to set its configuration parameters.

For more details see: https://pytorch.org/docs/stable/tensorboard.html

Parameters:

Name	Type	Description	Default
config	dict	Optional dictionary with the configuration of the SummaryWriter. Has the same entries as the set of "Other Parameters" below.	None

Other Parameters:

Name	Type	Description
log_dir	str	Directly location of the TensorBoard log files. Default is experiments/tensorboard_logs/exp_\<experiment_id>/rep_\<repetition_id>/\<date>_\<time>.
purge_step	int	When logging crashes at step T+XT+XT+X and restarts at step TTT, any events whose global_step larger or equal to TTT will be purged and hidden from TensorBoard. Note that crashed and resumed experiments should have the same log_dir.
max_queue	int	Size of the queue for pending events and summaries before one of the ‘add’ calls forces a flush to disk. (default = 10)
flush_secs	int	How often, in seconds, to flush the pending events and summaries to disk. (default = 120)
filename_suffix	string	Suffix added to all event filenames in the log_dir directory. (default = '.tblog')

Returns:

Name	Type	Description
writer	SummaryWriter	TensorBoard SummaryWriter object.

Source code in exputils/data/logging.py

def create_tensorboard(config: Optional[dict] = None,
                       **kwargs):
    """
    Creates the SummaryWriter object used to log values to TensorBoard.
    This allows to set its configuration parameters.

    For more details see: https://pytorch.org/docs/stable/tensorboard.html

    Parameters:
        config (dict):
            Optional dictionary with the configuration of the SummaryWriter.
            Has the same entries as the set of "Other Parameters" below.

    Other Parameters:
        log_dir (str):
            Directly location of the TensorBoard log files.
            Default is `experiments/tensorboard_logs/exp_\<experiment_id>/rep_\<repetition_id>/\<date>_\<time>.`
        purge_step (int):
            When logging crashes at step T+XT+XT+X and restarts at step TTT, any events whose
            global_step larger or equal to TTT will be purged and hidden from TensorBoard.
            Note that crashed and resumed experiments should have the same log_dir.
        max_queue (int):
            Size of the queue for pending events and summaries before one of the ‘add’ calls forces
            a flush to disk. (default = 10)
        flush_secs (int):
            How often, in seconds, to flush the pending events and summaries to disk. (default = 120)
        filename_suffix (string):
            Suffix added to all event filenames in the log_dir directory. (default = '.tblog')

    Returns:
        writer (SummaryWriter): TensorBoard SummaryWriter object.
    """

    return log.create_tensorboard(config=config, **kwargs)

activate_tensorboard

Activates parallel TensorBoard logging. When it is activated, then the logging functions ( add_value, add_scalar, add_histogram) will automatically log each value also in TensorBoard if not otherwise specified for them.

Creates a TensorBoard SummaryWriter if non is created yet with the create_tensorboard function. If non is created then the configureation of the writer can be defined by the parameters of this function. For more details see: https://pytorch.org/docs/stable/tensorboard.html

Other Parameters:

Name	Type	Description
log_dir	str	Directly location of the TensorBoard log files. Default is experiments/tensorboard_logs/exp_\<experiment_id>/rep_\<repetition_id>/\<date>_\<time>.
purge_step	int	When logging crashes at step T+XT+XT+X and restarts at step TTT, any events whose global_step larger or equal to TTT will be purged and hidden from TensorBoard. Note that crashed and resumed experiments should have the same log_dir.
max_queue	int	Size of the queue for pending events and summaries before one of the ‘add’ calls forces a flush to disk. (default = 10)
flush_secs	int	How often, in seconds, to flush the pending events and summaries to disk. (default = 120)
filename_suffix	string	Suffix added to all event filenames in the log_dir directory. (default = '.tblog')

Returns:

Name	Type	Description
writer	SummaryWriter	TensorBoard SummaryWriter object.

Source code in exputils/data/logging.py

def activate_tensorboard(config: Optional[dict] = None,
                         **kwargs):
    """
    Activates parallel TensorBoard logging.
    When it is activated, then the logging functions (
    [add_value][exputils.data.logging.add_value],
    [add_scalar][exputils.data.logging.add_scalar],
    [add_histogram][exputils.data.logging.add_histogram])
    will automatically log each value also in TensorBoard if not otherwise specified for them.

    Creates a TensorBoard SummaryWriter if non is created yet with the [create_tensorboard][exputils.data.logging.create_tensorboard] function.
    If non is created then the configureation of the writer can be defined by the parameters of this function.
    For more details see: https://pytorch.org/docs/stable/tensorboard.html

    Other Parameters:
        log_dir (str):
            Directly location of the TensorBoard log files.
            Default is `experiments/tensorboard_logs/exp_\<experiment_id>/rep_\<repetition_id>/\<date>_\<time>.`
        purge_step (int):
            When logging crashes at step T+XT+XT+X and restarts at step TTT, any events whose
            global_step larger or equal to TTT will be purged and hidden from TensorBoard.
            Note that crashed and resumed experiments should have the same log_dir.
        max_queue (int):
            Size of the queue for pending events and summaries before one of the ‘add’ calls forces
            a flush to disk. (default = 10)
        flush_secs (int):
            How often, in seconds, to flush the pending events and summaries to disk. (default = 120)
        filename_suffix (string):
            Suffix added to all event filenames in the log_dir directory. (default = '.tblog')

    Returns:
        writer (SummaryWriter): TensorBoard SummaryWriter object.
    """

    return log.activate_tensorboard(config=config, **kwargs)

deactivate_tensorboard

Deactivates tensorboard logging. Afterwards, values will not be automatically logged via the add_value / add_scalar function to the tensorboard.

Source code in exputils/data/logging.py

def deactivate_tensorboard():
    """
    Deactivates tensorboard logging.
    Afterwards, values will not be automatically logged via the add_value / add_scalar function
    to the tensorboard.
    """
    return log.deactivate_tensorboard()

is_tensorboard_active

Returns true, if the tensorboard is active.

Returns:

Name	Type	Description
is_active	bool	True if the tensorboard is active, otherwise False.

Source code in exputils/data/logging.py

def is_tensorboard_active() -> bool:
    """Returns true, if the tensorboard is active.

    Returns:
        is_active (bool): True if the tensorboard is active, otherwise False.
    """
    return log.is_tensorboard_active

Configuration

To configure the default directory the logging is using.

set_directory

Sets the directory path under which the logs will be saved. The default is ./data.

If the directory does not exist it will be created.

Parameters:

Name	Type	Description	Default
directory	str	Path to the directory.	required

Source code in exputils/data/logging.py

def set_directory(directory: str):
    """
    Sets the directory path under which the logs will be saved.
    The default is `./data`.

    If the directory does not exist it will be created.

    Parameters:
        directory (str):
            Path to the directory.

    """
    log.directory = directory

get_directory

Returns the path to the directory the log.

Returns:

Name	Type	Description
directory	str	Path to the directory.

Source code in exputils/data/logging.py

def get_directory() -> str:
    """
    Returns the path to the directory the log.

    Returns:
        directory (str):
            Path to the directory.
    """
    return log.directory