glide.pipelines module

Basic ETL pipeline templates for common nodes

Pipelines:

  • SQL2SQL

  • SQL2CSV

  • SQLParam2SQL

  • SQLParam2CSV

  • CSV2SQL

  • CSV2CSV

  • File2File

  • File2Email

  • File2URL

  • Email2Email

  • Email2File

  • URL2Email

  • URL2File

  • URL2URL

class glide.pipelines.GliderTemplate(nodes)[source]

Bases: object

A template for a Glider. It will create a new pipeline with a copy of its templated nodes when __call__’d.

Parameters

nodes – A top node potentially tied to other downstream nodes

nodes

A top node potentially tied to other downstream nodes

class glide.pipelines.NodeTemplate(nodes)[source]

Bases: object

A set of nodes that can be used as a template

glide.pipelines.basic_glider(extract=<class 'glide.core.PlaceholderNode'>, transform=<class 'glide.core.PlaceholderNode'>, load=<class 'glide.core.PlaceholderNode'>)[source]

Convenience function to produce a basic ETL template

Parameters

  • extract (type, optional) – A Node class to use as the extractor

  • transform (type, optional) – A Node class to use as the transformer

  • load (type, optional) – A Node class to use as the loader

Returns

Return type

A GliderTemplate that can be called to produce Gliders from the template.