Data transformation and movement is the process by which source data is selected, converted, and mapped to the format required by targeted systems. The process manipulates data to bring it into compliance with business, domain, and integrity rules and with other data in the target environment. Transformation can take some of the following forms:

  • Aggregation Consolidating or summarizing data values into a single value. Collecting daily sales data to be aggregated to the weekly level is a common example of aggregation.
  • Basic conversion Ensuring that data types are correctly converted and mapped from source to target columns.
  • Cleansing Resolving inconsistencies and fixing the anomalies in source data.
  • Derivation Transforming data from multiple sources by using a complex business rule or algorithm.
  • Enrichment Combining data from internal or external sources to provide additional meaning to the data. Normalizing Reducing the amount of redundant and potentially duplicated data.
  • Combining The process of combining data from multiple sources via parallel Lookup, Join, or Merge operations.
  • Pivoting Converting records in an input stream to many records in the appropriate table in the data warehouse or data mart.
  • Sorting Grouping related records and sequencing data based on data or string values

Leave a Reply