Since 1992, Edgewater Technology has been providing IT clients with the necessary knowledge and experience to successfully integrate data from the most complex systems.  With proven methodologies, Edgewater has delivered data integration solutions that have enabled organizations to access, integrate, transform, and deliver enterprise data from any source.

If you require the ability for improved reporting, query and analysis, analytic applications, and overall performance management, then leverage Edgewater’s expertise to provide your organization with the ability of greater data integrity and Information Technology (IT) efficiencies.

What is Extraction, Transformation, and Load (ETL)?

ETL is an essential part of uniting disparate operational systems for use in a Data Warehouse.  The ETL process is responsible for extracting data from the operational systems – either in near real-time or batched at regular intervals.  The transformation part of the process converts the data from each system into a common representation, while the load process stores it in a normalized staging database.  The staging database is then used to construct the star schemas, snowflake schemas, data cubes, and other structures in a data warehouse.  The data warehouse can then be broken down into multiple data marts.  This logical progression is shown in the following diagram:

ETL technology addresses the challenges that arise when creating a central reporting repository from several source systems.  Developing a single, consistent view of the data that exists in multiple systems requires data to be altered or transformed into a common format, also known as cleansing.  Data cleansing ensures differences in data among systems storing the same data are reconciled before loading a data warehouse.

These exceptions need to be identified and reconciled through business rules before loading the data warehouse.  Some examples are:

1. A sales database and an inventory database may use different product codes.  The data transformation process ensures a single set of product codes is used within the data warehouse. 
2. A data attribute such as a nine-digit zip code in a customer database may need to be split into two attributes, a five-digit zip and a four-digit extension, to match the data structure of another customer database. 
3. Two databases that both store customer data and may have a different address for the same customer. 

Several ETL tools are available that support rapid development of data marts and data warehouses.  Such a tool allows definition of business rules for the ETL process using a graphical “drag and drop” interface.  In recent years, vendors of ETL tools have moved into the BI space, while established Business Intelligence (BI) vendors have either developed their own ETL capability or purchased a company that has such a product.  The result is that once a BI platform is selected, it is likely to have sophisticated ETL tools already integrated.

The Complex Data Integration required to transform data to a common representation is frequently the most costly portion of a BI implementation.  Edgewater Technology has the necessary experience with definition process and ETL tools to minimize the cost and time needed to build your data warehouse.