-
Performance in sort stage
i am running huge amount of data through a sort stage and it is very slow. Is there any option which can increase the performance with in the sort stage?
manoharkolukula
- Jan 25th, 2008
See if it is orcle db then u can write user def query for sort and remove duplicates in the source itself. and maintaining some key partition teqniques u can improve the performence. If it is not the ...
-
Memory allocation while using lookup stage
Hi friends, i know while using lookup stage if the lookup table has huge amount of data we will go for sparse lookup or else will use normal lookup. would anybody explain how the memory will be allocated for both type of lookups?
harishsj
- Jan 22nd, 2008
"Hi friends, i know while using lookup stage if the lookup table has huge amount of data we will go for sparse lookup or else will use normal lookup. would anybody explain how the memory will be ...
-
Even and Odd Number Nodes
To run Job why do we use only even number nodes like 2,4,6,8.., Why don't we use 1,3,5..,?
anshulsri2
- Jan 23rd, 2008
There are no odd number nodes used in datastage, for the simple reason we are not allowed to configure odd number nodes on the datastage server.
-
Datastage jobs
if job compiled with "job completed successfully". but The DS director window of Log page shows warning messages.what will we do?giv answer.
veer.dwh
- Sep 19th, 2009
Data Stage job can finish successfully with n no of wrnings in DS. if suppose some records are rejected which are coming from source also DS director give warning message but those records are not mee...
-
What is datastage DSTX
sinhasaurabh014
- Apr 9th, 2008
DataStage Transformation Extender was developed,owned and marketed my Meractor(an aviation company). e.g. Version 6.5.2 is Mercator. Later on Ascential Softwares purchased it and named it as DataStage...
-
How do you implement SCD in Datastage
How we will implement scd in DS?Explain with example ?Any scd ie scd1,scd2,scd3 .
wahi80
- Feb 19th, 2008
If you are using Server version, you will have to use a combination of transformer, hashfiles for lookup and stage variables. If you are using the parallel version you will have touse the Change Capture and Change Apply stages
-
How can we load the flat file
How can i load a flat file into target as fast as i can?Assuming that the source bottleneck is not there,that is there is no performance issues in the source side.
nikhilanshuman
- Dec 18th, 2010
A Flat file can be read using sequential file stage.To make data load faster,try implementing least stages in you job and use minimal transformations.if you are trying to load the data of a flat file ...
-
When we have to contact with admin people...
Have you ever contact with the DS Administration people?What are the situations and what problems you phased?
-
How you will define Global
How you will define global variables in DS?
ketanshah123
- Feb 19th, 2008
The user variable stage allows you to define global variables within a sequence. These variables can then be used elsewhere in the sequence,The values of the user variables are set by expressions in the stage’s properties
-
Abort a job after loading 10 records
How to implement in datastgae like the job should abort after it loads 10 records into the target using lookup?
nagoosk
- Apr 29th, 2008
You can abort a job after n records which is available while you r running the job you can see tha option as "abort job after _____" . there you can specify the number
-
Conductor Node
What is a conductor node?
nagoosk
- Feb 20th, 2008
Actually every process contains a conductor process where the execution was started and a section leader process for each processing node and a player process for each set of combined operators and a ...
-
SCD Type2 in Enterprise Edition
How to implement Slowly changing dimesions Type2 in Parallel jobs . What are the stages we use and How to assign the key?
Abhijain
- Oct 19th, 2009
Datastage Px 8.0.1 is having SCD stage to perform the required operation. Or it can be achived by using lookup/change capture stage in between source and target database.
-
DataStage Transformer stage Server Jobs and Parallel Jobs
What is the difference between transformer stage in serverjobs and transformer stage in parallel job?
Abhijain
- Sep 19th, 2008
Transformer stage in server jobs vs Transformer stage in Parallel jobs In server jobs, support single input stream link and multiple input reference link where in parallel support only single input st...
-
Dataset stage/lookup file stage/file stage
When do one use dataset stage/lookup file stage/file stage in a parallel job?or in other way what is the significance of these files...wat are their differences?
nagoosk
- Feb 28th, 2008
DataSetFile: DataStage PX jobs uses the dataset file to organize the data.Dataset files preserves partitioning means u need not to partition the data when ever u read the data.It maintaines the persis...
-
Job Control and Sequencer
What is the difference between jobcontrol and sequencer?
nagoosk
- Mar 18th, 2008
Jobcontrol: It just inserts the compilation code. Whenever you add the job it just places executable code there. There is no diagrammatical representation here.Sequencer: Sequencer is a diagrammat...
-
Hashing Algorithm
Explain What is mean by Hashing Algorithm?
veer.dwh
- Sep 23rd, 2009
Hashing alogarithm is used when you create hash files in server jobs. it takes key field and distribute records avilable cpus in DS sercver while running the DS job. the cpu utiliztion in this process may vary based on server configuration.Regards,Veeru.
-
Remove Duplicate Records
what is process to remove duplicate records using Copy Stage.
Abhijain
- Sep 26th, 2008
In the copy stage, there is an option to sort the input data where you can select the key on the basis of which you can perform the sort operation. There is an option named 'Unique'. Select th...
-
How to assign the one path of file to PROJDEF using the job parameters?
At the time to incremental loading I need to take the reusable files' path in this situation How to assign the one path of file to PROJDEF using the job parameters?
nikhilanshuman
- Dec 11th, 2009
From your question it seems that you have defined the file path parameter as $PROJDEF in you parallel job.Setting a parameter to projdef :Step 1: Go to datastage adminstrator and set a user defined va...
-
Write the shell script using Unix to run the 2 jobs?
write the shell script using Unix to run the 2 jobs?How to schedule the 2 jobs using Unix crontab to run for particular time?
nikhilanshuman
- Nov 29th, 2009
Inside the shell script, first you have to set the path using "dsenv".Then using following command the datastage job can be run.dsjob -run mode -Normal -jobstatus Projectname JobNameafter typing this in the editor,save file as testrun.sh.Now run the command using bash testrun.sh.
-
Datastage flow of parallel Project
Any body explain the flow of project(parallel) in real time thanx
srikanth.ds
- Oct 30th, 2009
Project flow is nothing but the data flow in your project. For example in many of the projects data is moved from source to staging, and then perfoem ETL and staging to ware house. So here you encountered 2 levels of data flow i.e source to staging and staging to warehouse.
DataStage Interview Questions
Ans