SmallNet ETL Framework

SmallNet has produced its own ETL (Extract Transform Load) framework for the development and deployment of data warehousing and data integration projects.

The SmallNet framework includes:

  • Development and design standards and best practice.
  • Project plan templates.
  • A deployment architecture comprising:
    • Scheduling and control layer.
      • Runs and logs all jobs in the application, tracking their progress.
    • Error and event handling layer.
      • Manages the logging of all errors raised.
    • Audit and reconciliation elements.
      • Manage the capture of the data processed through the framework.
    • Job structure:
      1. Extract – the extraction of data from the source to the DataStage environment. Once extracted the data is validated for completeness and cleansed (and, if necessary, standardised) to ensure quality.
      2. Transform – cleansed and validated data is mapped and enhanced.
      3. Load – loads data into a specific target file. This process
    • Pre-built components that save you time and money.