ETL - extract, transform and load - is the set of
processes by which data is extracted from numerous
databases, applications and systems, transformed as
appropriate, and loaded into target systems - including, but
not limited to, data warehouses, data marts, analytical
The first part of the extract, transform and load (ETL) process is understanding the data sources. The transformations are organization-specific and Integration is sometimes included in the ETL process; because it requires an in-depth knowledge of the organization and its business.
More than half of all development work for data warehousing projects is typically dedicated to the design and implementation of ETL processes. Poorly designed ETL processes are costly to maintain, change and update, so it is critical it is to make the right choices in terms of the right technology and tools that will be used for developing and maintaining the ETL processes.
Click to enlarge
KnowledgeBase consultants has vast experience
developing ETL complying with Star
Schema/Dimensional/Snowflake approach or Normalized
approach depending on the Data Warehouse design.
KnowledgeBase consultants have expertise across Extract, Transform & Load (ETL) tools that are effectively used to extract and unite data from disparate sources and deliver meaningful and actionable business intelligence across the organization.
KnowledgeBase handles the entire data movement lifecycle
– from source systems to the staging area to the
datawarehouse to the final data mart.
Defining data sources and the relevant mappings
- Maintenance and monitoring of transformation scripts
- Creation of new transformation scripts for the new business requirements
- Preparation of batch scripts
Update mapping specifications with meta data
Update monitoring and responsibility processes on source data changes
Ensure accurate and efficient ETL process in production
- Rectification of a broken ETL process
- Updation of the ETL process
- Streamline/ eliminate disjointed extraction and transformation programs
- Validation routines to check for data consistency between source and destination
Performance tuning, error handling and interdependent scheduling
Some of the numerous technological approaches and solutions available on the market include:
- Traditional engine-based ETL products
- RDBMS proprietary solutions
- Third-generation ELT solutions, based on a code-generation approach that uses the power of the RDBMS engines to perform the data transformations
Some of the key technologies used by KnowledgeBase for ETL are:
- Microsoft SQL Server 2005 Integration Services
- Pentaho ETL
- SAP ETL
- Data Stage
- Oracle OWB
|back to top|