airtunnel.operators.sql package

Submodules

airtunnel.operators.sql.sql_helpers module

A module with several SQL helpers used for Airtunnel.

airtunnel.operators.sql.sql_helpers.execute_script(connection, statements: Iterable[str]) → None

Executes a SQL script.

Parameters

• connection – a live DB connection, implementing execute() as in Python DB API (PEP:249)

• statements – list of individual SQL statements to execute

Returns

None

airtunnel.operators.sql.sql_helpers.execute_statement(db_conn, sql_statement) → None

Executes a single SQL statement on a DB connection.

airtunnel.operators.sql.sql_helpers.format_sql_script(sql_script: str, sql_params_dict: Dict[str, str]) → str

Formats a SQL script using the standard Python string format method.

Parameters

• sql_script – raw SQL script string with formatting parameters expressed as {parameter_key}

• sql_params_dict – the dictionary (of type string to string) that will be passed to the format function

Returns

the formatted SQL script as a string

airtunnel.operators.sql.sql_helpers.load_sql_script(script_file_relative_path: Union[str, Iterable[str]]) → str

Load a SQL script from Airtunnel’s script store.

Parameters

script_file_relative_path – single or multiple relative script path(s) from the Airtunnel script store SQL folder

Returns

the full loaded SQL script as a raw string

airtunnel.operators.sql.sql_helpers.prepare_sql_params(compute_sql_params_function, airflow_context) → Dict[str, str]

Prepares further dynamic SQL parameters using an anonymous passed function that computes them.

The dynamic function will be passed the live Airflow context, so that it can leverage it’s information.

Returns

a dict (of type string to string) with additional formatting key-value pairs that are added to the default static formatting parameters.

airtunnel.operators.sql.sql_helpers.split_sql_script(full_script: str) → List[str]

Splits a full SQL script into its individual statements (using ;) and removes commented lines.

Parameters

full_script – the full raw SQL script

Returns

list of individual SQL statements

airtunnel.operators.sql.sqloperator module

Module for Airtunnel’s SQLOperator.

class airtunnel.operators.sql.sqloperator.SQLOperator(sql_hook: airflow.hooks.dbapi_hook.DbApiHook, script_file_relative_path: Union[str, List[str]] = None, parameters: dict = None, dynamic_parameters: False = None, *args, **kwargs)

Bases: airflow.models.baseoperator.BaseOperator

Airtunnel’s SQLOperator.

execute(context)

Execute the operator instance using Airflow.

Module contents

A package with several SQL related helpers.