-
Lookup Stage Partitioning
With a Lookup stage the input stage has 1million records and reference link has 1million records. What is the best Partition method and why?
Kumar Shourav
- Oct 24th, 2009
Better to use JOIN or MERGE stage in place of Lookup. If input data is low in compare to reference data then Lookup is recommendable. Assumption is 1:100 or more.
gopalanrr
- Mar 27th, 2009
When the reference link is too high in terms of records fetched, you should be performing a join instead of performing a lookup
-
Problems with Datastage - Date format & Null Handling of a Date field in a schema file
Hi,I have a .csv file as my source.I am reading the data using asequential file stage.Then I am validating the format of the data using a schema file.The date format of the incoming file is '%dd/%mm/%yyyy'.So the datecolumn has a fixed length 10.Now according to the provided incoming data set the date column shouldbe nullable.Can anybody please provide an idea how to handle a nullable datecolumn using...
Abhijain
- Oct 19th, 2009
Read the date field column as varchar and use transformer to check nullabiltiy and if null found assign some default value say '12-Dec-9999' and use the appropriate conversion function an...
Abhijain
- Oct 19th, 2009
Read date field from sequential file as varchar datatype with nullable yes. Use modify stage or transformer to convert the datatype and also check the nullability of the incoming data and if null foun...
-
SCD Type2 in Enterprise Edition
How to implement Slowly changing dimesions Type2 in Parallel jobs . What are the stages we use and How to assign the key?
Abhijain
- Oct 19th, 2009
Datastage Px 8.0.1 is having SCD stage to perform the required operation. Or it can be achived by using lookup/change capture stage in between source and target database.
-
Move data using DataStage
How to move data from operational and external environments to BI environment using datastage software?
Abhijain
- Oct 19th, 2009
Question is not clear.Are you talking about moving OLTP data to OLAP system?
-
Datastage Incremental Loading
How to do incremental loading in datastage?
veer.dwh
- Oct 15th, 2009
Abhijain is right. If you compare source with target using either lookup or change capture you can perform Incremental loading. But my requirement is I have a job like,Source-------->Transformer...
Abhijain
- Oct 14th, 2009
There are ways to perform the incremental load. While performing incremental load, Pls take care of below points:1. Retrieve changed records from last load run from source by using appropriate ex...
-
What is the difference betwen Merge Stage and Lookup Stage?
veer.dwh
- Oct 7th, 2009
Both stages use for joining the data. using merger you can capture rejected data from N no of update datasets but you can't capture master source data. but using lookup you can capture 1 reject data set from source only not from the refrence datasets.Regards,Veeru.
srinivasguptha
- Jan 29th, 2007
Lookup stage:1. Used to perform lookups.2. Multiple reference links, single input link, single output link, single rejects link, single primary link. 3. Large amount of memory usage. Because paging re...
-
Real-Time Data Integration
What is meant by real-time integration in terms of datastage and how is different from the same in Informatica.Explain batch and real time processing capabilities of both ETL tools with their characteristic differences?
-
How the job status changed when the datasets it process is locked by another job
When the data processed by job1 is locked by job2 with higher priority, what will happen about the status of job1, changing from running to aborted , or still running. does it have any relationship with the database
vujjini1974
- Sep 30th, 2009
If its reference dataset and you are doing Lookup then job will run but no records will come from that Dataset, so you should rerun the dataset job and then run the other job to get the records.
-
How I can convert Server Jobs into Parallel Jobs?
veer.dwh
- Oct 1st, 2009
Hi,I have worked on server to px migration project. you can't directly convert a server job to PX job. the only way is, you have to redevelop the px job using px environment.Regards,Veeru.
Abhijain
- Feb 20th, 2008
On a job level- to convert server jobs into a parellel job is not possible.The mechnism of server jobs is different then that of parellel jobs.You have to re-build the job.
-
Purpose of using the key and difference between Surrogate keys and natural key
-
How many types of look up stages in datastage parallel ext?
vujjini1974
- Sep 30th, 2009
There is only one look up stage in parallel extender, and it is of two types one is normal look up and another one is Sparse look up,mostly we normal look up for the reference datasets.
-
XML File
How to read an XML File?
vujjini1974
- Sep 29th, 2009
Use a XML Folder stage to read the XML file and pass the content to XML Input stage to parse out tabular records.
-
What is the sequencer stage??
veer.dwh
- Sep 28th, 2009
Sequencer stage comes in job sequencer.
Job sequencer uses to run multiple jobs based on dependency.
Sequencer stage activity is one type of activity stage in job sequencer, using this you can run n number of jobs parallely.amardeep.reddy
- Dec 6th, 2006
Lets say there are two jobs (J1 & J2) as the input links and one job (J3) as output link for a sequencer stage in a DS job sequencer. The sequencer can be set to "ALL" or "ANY". If it is set to "A...
-
How can we improve the job performance?
veer.dwh
- Sep 28th, 2009
There are several performance techniques are avilable to help run the job faster1. Eliminate unwanted data and columns as early as possible in your job design, each unwanred column and data requ...
Bhaskar
- Dec 29th, 2006
We can put hash file in lookup. This will index the input data based on key column (which we define). Thus improve performance. Also Array size can be increased in final table stage.
-
What is a sequential file that has single input link??
veer.dwh
- Sep 28th, 2009
Sequntial file stage supports either one input link or one output link.it runs sequntially(like row by row process), if it is reading single input file and won't support any parallism.it runs parallely, if it is reading multiple files and will support parallism techniques.
Raghu
- Aug 1st, 2007
Sequential file always has?single link because its it cannot accept the multiple links or threads.Data in sequential file always runs sequentially..HTHRaghu Mule
-
What are different types of star schema??
veer.dwh
- Sep 28th, 2009
There are 2 types of star schemas in datastage1.Starschema: A single fact table sits in middele and surounded dimension tables radically connect with fact table like a star. in star schema dimension t...
PRASHANT NARAYANI
- Dec 27th, 2006
Multi star schema or galaxy schema is one of the type of star schema
-
Feature of DataStage 8.01 Version
What are the main features of DataStage 8.01 Version?
veer.dwh
- Sep 23rd, 2009
There are several new futures includes in DS 8.0.1 compare to 7.5 version.In 8.0.1 IBM included Quality stage, IBM information server (profile stage and audit stage) . these are not available in 7.5...
veer.dwh
- Jul 20th, 2009
Compare to DataStage 7.5 Accential DataStage, with DataStage 8.0 IBM websphere datastage Architecture got changedIn 8.0 IBM included client components of IBM information server which supports profiling stage along with Quality stage features are included in designer client component.
-
SCD2 Parallel Jobs
How to implement SCD2 in parallel jobs?
veer.dwh
- Sep 23rd, 2009
If you are using DS 8.0.1 version, it is very east to implement SCD2 using SCD stage. it supports 2 input links and 2 output links. from the 2 input links. primary Dataset it takes as a current dates...
-
How can we improve the performance of a link collector in the case of handling more than 25000 records?
veer.dwh
- Sep 23rd, 2009
Link partitiner and collecter are used to develop the parallism in server jobs.if source has huge amount of data that situation you can use these stages to perfoem parallism in server job environment....
-
Hashing Algorithm
Explain What is mean by Hashing Algorithm?
veer.dwh
- Sep 23rd, 2009
Hashing alogarithm is used when you create hash files in server jobs. it takes key field and distribute records avilable cpus in DS sercver while running the DS job. the cpu utiliztion in this process may vary based on server configuration.Regards,Veeru.
DataStage Interview Questions
Ans