|
|
|
Forum Newbie
Group: Forum Members
Last Login: 7/1/2009 12:49:08 PM
Posts: 1,
Visits: 7
|
|
I want to perform complex transformation on staging data and load them in fact tables. both staging and fact tables resides in same teradata database.
What should be the better choice?
Informatica or BTEQ scrips?
Volume of data is appx 200,000 rows every day.
I am thinking to use informatica for complex transformation and Mload as external loader(Informatica External Loader) to load data.
Pls Suggest.
Many Thanks
Raj Shah
|
|
|
|
|
Supreme Being
Group: PAC and SFT Members
Last Login: 9/9/2009 4:34:06 PM
Posts: 407,
Visits: 889
|
|
The real question to ask is whether you can do the transformations entirely within the database (Informatica calls it "push down") or if the processing is such that you MUST export it to another server, transform, then load back to Teradata. Transforming within the database is usually much faster.
If you absolutely can't do the transformations within the Teradata database, BTEQ alone won't solve the problem. On the other hand, if you already have Informatica as your standard ETL tool and can use "push down", there is no overwhelming benefit to using BTEQ instead.
|
|
|
|
|
Forum Newbie
Group: Forum Members
Last Login: 2/6/2009 3:21:21 PM
Posts: 1,
Visits: 3
|
|
This scenario looks like a perfect fit for Informatica PowerCenter’s Pushdown Optimization capability where you can use the same design environment in PowerCenter to develop the transformations and have PowerCenter manage the “push” of processing into the Teradata database.
Informatica introduced this specifically for the Teradata database and for scenarios where staging and production are in the same database instance to reduce/eliminate the need to move data in and out of the database.
You can find more information here: http://www.informatica.com/INFA_Resources/ds_pushdown_optimization_6675.pdf
|
|
|
|
|
Supreme Being
Group: Forum Members
Last Login: 8/4/2009 12:24:24 PM
Posts: 144,
Visits: 93
|
|
One potential showstopper for any bteq dependent job is the connection sustainability which will not be there in your case...because both source and target are in the same database. I guess this is a test to your SQL capability to code the complex tranformations which might prove to be a better choice than Informatica.
Regards
Ramakrishna Vedantam
Telecom Technology Excellence Group
Tata Consultancy Services
Website: http://www.tcs.com
|
|
|
|