glide.utils module

Common utilities

class glide.utils.DateTimeWindowAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: argparse.Action

An argparse Action for handling datetime window CLI args

class glide.utils.DateWindowAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: argparse.Action

An argparse Action for handling date window CLI args

glide.utils.cancel_asyncio_tasks(tasks, loop, cancel_timeout=None)

Cancel a set of asyncio tasks

Parameters

• tasks (iterable) – An iterable of asyncio tasks to cancel

• loop – asyncio Event Loop

• cancel_timeout (int or float, optional) – A timeout to use when waiting for tasks to finish cancelling

glide.utils.closer(x)

Helper to call close on x

glide.utils.date_from_str(s)

glide.utils.date_window_cli()

An argparse parent CLI that adds date window support

glide.utils.datetime_cmp(d1, d2)

glide.utils.datetime_window_cli()

An argparse parent CLI that adds datetime window support

glide.utils.dbg(msg, **kwargs)

Call tlbx dbg with glide logger

glide.utils.dbgsql(msg, **kwargs)

Call tlbx dbgsql with glide logger

glide.utils.divide_data(data, n)

Divide data into n chunks, with special handling for pandas objects

glide.utils.error(msg, **kwargs)

Call tlbx error with glide logger

glide.utils.excel_file_type(f)

Best guess at Excel file type from name

glide.utils.find_class_in_dict(cls, d, include=None, exclude=None)

Get a list of keys that are an instance of a class in a dict

glide.utils.flatten(l)

Flatten a list of iterables

glide.utils.get_class_list_docstring(heading, classes)

Helper to generate a part of a module docstring from a list of classes

glide.utils.get_date_windows(start_date, end_date, reverse=False)

glide.utils.get_datetime_windows(start_date, end_date, window_size_hours=None, num_windows=None, reverse=False, add_second=True)

Produce a list of start/end date tuples

Parameters

• start_date (date, datetime, or str) – The absolute start date of the range

• end_date (date, datetime, or str) – The absolute end date of the range

• window_size_hours (float, optional) – The size of the windows in hours. May be a float to represent partial hours.

• num_windows (int, optional) – The number of windows to split the date range into. One of num_windows or window_size_hours must be specified.

• reverse (bool, optional) – If true return the windows in reverse order

• add_second (bool, optional) – If true, offset the start of each window to be one second past the end date of the previous window.

Returns

dt_windows – A list of tuples of start / end datetime pairs

Return type

list

glide.utils.get_file_handle(*args, **kwargs)

Context manager pass through to open_filepath_or_buffer. This will automatically close the file if and only if it was opened here. If file handles are passed in it is assumed the caller will manage them.

glide.utils.get_or_create_event_loop()

Get an existing asyncio Event Loop or create one if necessary

glide.utils.info(msg, **kwargs)

Call tlbx info with glide logger

glide.utils.is_file_obj(o)

Test if an object is a file object

glide.utils.is_function(f)

Test if f is a function

glide.utils.is_pandas(o)

Test if an object is a Pandas object

glide.utils.iterize(o)

Automatically wrap certain objects that you would not normally process item by item.

TODO: this function should probaly be improved/generalized.

glide.utils.join(tables, on=None, how='left', rsuffixes=None)

Join a list of iterables or DataFrames

glide.utils.listify(o)

Ensure an object is a list by wrapping if necessary

glide.utils.load_ini_config(filename, key=None)

Load a config from an ini file, optionally extracting a key

glide.utils.load_json_config(filename, key=None)

Load a config from a json file, optionally extracting a key

glide.utils.load_yaml_config(filename, key=None)

Load a config from a yaml file, optionally extracting a key

glide.utils.nchunks(a, n)

Divide iterable a into n chunks

glide.utils.not_none(*args)

glide.utils.open_filepath_or_buffer(f, open_flags='r', compression=None, is_text=True)

Use pandas IO functions to return a handle from a filepath or buffer.

Parameters

• f (str or buffer) – filepath or buffer to open

• open_flags (str, optional) – mode to open file

• compression (str, optional) – compression arg passed to pandas functions

• is_text (bool) – Whether file/buffer is in text format, Passed through to pandas helpers.

Returns

• f (file-like) – A file-like object

• handles (list of file-like) – A list of file-like objects opened. Seems mostly relevant for zipped archives.

• close (bool) – A flag indicating whether the caller should close the file object when done

glide.utils.read_excel(f, **kwargs)

Read data from an Excel file using pyexcel

Parameters

• f (str or buffer) – Excel file to read from

• **kwargs – Keyword arguments passed to pyexcel

glide.utils.save_excel(f, data, **kwargs)

Write data to an Excel file using pyexcel

Note

If f is a file that ends in .xls, pyexcel_xls will be used, otherwise it defaults to pyexcel_xlsx.

Parameters

• f (str or buffer) – Excel file to write to

• data (dict) – Data to write to the file. This is expected to be a dict of {sheet_name: sheet_data} format.

• **kwargs – Keyword arguments passed to pyexcel’s save_data

glide.utils.size(o, default=None)

Helper to return the len() of an object if it is available

glide.utils.split_count_helper(data, split_count)

Helper to override the split count if data len is shorter

glide.utils.to_date(d)

glide.utils.to_datetime(d)

glide.utils.warn(msg, **kwargs)

Call tlbx warn with glide logger

glide.utils.window(seq, size=2)

Returns a sliding window over data from the iterable