-
Explain the best approch to do a SCD type2 mapping in parallel job?
manoharkolukula
- Feb 12th, 2008
use CDC, lookup, trasformer stages to impliment SCD2. In CDC just give the changed values and key column, then capture the changed records, in transformer just give the constraint for 0,1,2,3(del,insert,copy,updated data), u will get inserted data and updated data, copied data, deleted data.
-
Problems with Datastage - Date format & Null Handling of a Date field in a schema file
Hi,I have a .csv file as my source.I am reading the data using asequential file stage.Then I am validating the format of the data using a schema file.The date format of the incoming file is '%dd/%mm/%yyyy'.So the datecolumn has a fixed length 10.Now according to the provided incoming data set the date column shouldbe nullable.Can anybody please provide an idea how to handle a nullable datecolumn using...
Abhijain
- Oct 19th, 2009
Read the date field column as varchar and use transformer to check nullabiltiy and if null found assign some default value say '12-Dec-9999' and use the appropriate conversion function an...
Abhijain
- Oct 19th, 2009
Read date field from sequential file as varchar datatype with nullable yes. Use modify stage or transformer to convert the datatype and also check the nullability of the incoming data and if null foun...
-
What is Pipeline Parallelism?
What is Pipeline Parallelism in Datastage?
leelasankar.pr
- Jul 22nd, 2008
It is the ability of a down streem stage to process arow ,which the up streem stage processed that row
-
Check for Job Errors in datastage
in datastage job successfully finished. but showing errors in director window. How do we fix and find the cause for the errors
manoharkolukula
- Feb 12th, 2008
I think in designer u find the green colour but in Director -> log view --> u find some fatal errors and job is aborted, is it, Means job is successfully completed but the data is not loaded in ...
-
What is the use of clear status file
What is the use of clear status file in DS?How we will use it?explain with examples?
nikhilanshuman
- Dec 30th, 2009
If status is cleared for a datastage job,all of its instaces will be deleted.I mean if there is a instance say jobname.inst1 or jobname.inst2 these will be deleted.The entries for that particular jobs will be cleared from RT_Status(n) table.Here n is the job number.
ketanshah123
- Feb 19th, 2008
When you clear a job’s status file you reset the status records associated with all stages in that job. You should therefore use this option with great care, and only when you believe a job has ...
-
Add 10 days to a particular date
How to implement the scenario: Add 10 days to a particular date in Datastage PX
Venkatesh Mekewar
- May 31st, 2016
You can use Date function named "DateFromDaySince()" present inbuilt in Datastage. Example: Consider Link.DaystoAdd contains 10 and Link.GivenDate contains 2016-10-31. Following function will retur...
nagoosk
- Mar 5th, 2008
extract the day part of the date by using the string function and then add how many days u want and then again add all those parts into single string and then convert the string into date object.
-
Find table count in Datastage
How to find out individual counts for a particular table
karthikgeekkarthik
- Jul 9th, 2008
While implementing the scd type-2, due to maintianing of history, the primary key constraint will be violated. to have a primary key for the table for the sake of unique key we use surrogate key.
-
Surrogate key
What is the Disadvantage of surrogate key stage?
rajul.ds
- Jul 22nd, 2008
To my concern if the data is not properly processed on nodes..in such suitations this surrogate key generates holes(simply skips the ordering )ex:aa-1bb-2ba..bg...cc-3dd-4ddd-5here for ba and bg do not have seq key ,here ba and bg have holes
debojyoti_datastage
- Jul 21st, 2008
The main disadvantage of this stage comes when you use it second time around..There you have to give the initial value of the surrogate key....I faced the similar situation and the only way i found wa...
-
Change capture
What is the difference between change capture and change apply stages
ritu singhai
- Jul 7th, 2008
Change capture stage is used to get the difference between two sources i.e. after dataset and before dataset. The source which is used as a reference to capture the changes is called after dataset. Th...
HemaDS
- May 29th, 2008
Change Capture is used to capture the changes between the two sources.Change Apply will apply those changes in the output file.
-
BASIC Transformer and NORMAL Transformer
What is the Exact difference between BASIC Transformer and NORMAL Transformer?When we will go for BASIC Or NORMAL Transformer?
nikhilanshuman
- Mar 19th, 2010
Difference: - A Basic transformer compiles in "Basic Language" whereas a Normal Transformer compiles in "C++".- Basic transformer does not run on multiple nodes wheras a Normal Tra...
Abhijain
- Sep 19th, 2008
The basic transformer stage is similar in appearance and function to the Normal Transformer Stage. Basic transformer stage can only be used for SMP(Symmetric Multiprocessors) systems and not for MPP(Massively Parellel Processing) system or clusters systems.
-
Filter and External Filter stages
What is the difference between Filter and External Filter stages?
shrushti
- Nov 12th, 2015
FILTER EXTERNAL FILTER Filters based on some condition datastage level filter------ Uses UNIX commands to filter Can have optional re...
nagoosk
- May 29th, 2008
Filter is used to pass the records basing on some condition.We will have multiple where conditions to specify.We can send to different output linksExternal filter is used to execute commands of unix.You can give the commands in the specified box with the corresponding arguements.
-
DataStage Engine Commands
I am eager to know about DataStage Engine commands. Can someone tell me something about that?
prasanth37
- Jun 12th, 2008
The following commands can be taken as DS Engine commands, used to start and stop the DS EngineDSHOME/bin/uv -admin -startDSHOME/bin/uv -admin -stop
HemaDS
- May 29th, 2008
According to me, Datastage Engine command is otherwise called as Universe Commands. It is used to retrieve the details that are stored in the datstage repository in the form of files and tables. We ca...
-
Transform XML file to sequential file
Using server job, how to transform data in XML file into sequential file?? i have used XML input, XML transformer and a sequential file.
shm_Ariel
- Jul 4th, 2008
Import the XSD for the xml file via manager-metadata importer and store it. If transformation functions are not required to be done on the data in XML file, there is no need for the tra...
dwh_com
- Jun 24th, 2008
i will explain u the stages used inorder..FOLDER STAGE--------->XMLINPUTSTAGE--------->TRANSFORMER------>SEQUENTIAL FILEfolder stage is to check for the folder which has xmlfile and u have to...
-
High Level Design
How high level design will be prepared in datastage?
vjrockings
- Sep 14th, 2009
HLD (High Level Design) is the overview of a project in very high level prospectus. Here we have to mention the project definition goal and objective of the project, flow diagram of the process, which functions are in scope and are not etc.
k2_swarupa
- Apr 16th, 2009
HLD (High Level Design) is the overview of a project in very high level prospectus. Here we have to mention the project definition, goal and objective of the project, which functions are in scope and which are not, flow diagram of the process etc.
-
Design Phase
How technical design will be prepared in datastage?
k2_swarupa
- Apr 16th, 2009
If you want to prepare a design doc, then you should know the complete flow of your project along with business rule. Technical design doc is nothing but the flow of the process,?to describe in jobs.
-
Use of Terminator Activity
What is the Use of Terminator Activity in Datastage server Jobs?
nikhilanshuman
- Mar 19th, 2010
The terminator activity is used to terminate/abort a job in particulat conditions.user can define his own flow to the terminator activity.
Abhijain
- Sep 19th, 2008
A terminator stage can be used in a job sequence to ensure that the sequence is stopped cleanly if certain situations arise. You can have multiple terminator activities and can place them anywhere in ...
-
Dataset and Transformers Performance
How can you say that performance will be improve if we make use of a dataset?Why does the performance degrade when we use 4-5 transformers in a single job?
ashokmadduri
- Jan 17th, 2009
Dataset are referring to control file due to this dataset will give fast performance while processing the data, if you use transformers in job, it should compile the C++ backend due to this datastage ...
anurangevk
- Dec 6th, 2008
1. Using Dataset?- The storage method is internal to datastage, hence performance increases as less time is required to read from or write to a dataset2. Transformers are very bulky with lot of functions inbuilt them, so usage of more transformers slows down the speed at which?job executes
-
Job Sequence
How to create Job Sequence?
Abhijain
- Feb 8th, 2010
Have you ever tried opening Datastage and browse all the available options.To create a job sequence. Below are the steps;1. Open Datastage2. Open New Job Sequence3. Save it with some name.4. A your job sequence is created.5. Now drag the job sequence stages-->as per your business requirement.
veer.dwh
- Jul 20th, 2009
You can create job sequencer using designer client, if you click create new job like server or px job in Datastage, it will get open window there you can find to create the job sequencer.Job sequencer uses to run multiple jobs either server or px based on the dependencies.
-
Job Schedule
How to schedule jobs in UNIX Server?
Abhijain
- Feb 8th, 2010
Couple of ways to schedule the Datastage jobs/sequences-1. Can be done using Datastage Director Tool 2. Normally done using crontab script. You can easily get it through google.3. Can use Third party scheduling tools like Autosys
veer.dwh
- Jul 20th, 2009
DataStage job scheduling we can do using datastage director client tool, but in real time they use UNIX for DataStage job scheduling which supports 3rd party scheduling tools like Autosys, Triviam, UNIX corn tab etc.
-
Feature of DataStage 8.01 Version
What are the main features of DataStage 8.01 Version?
veer.dwh
- Sep 23rd, 2009
There are several new futures includes in DS 8.0.1 compare to 7.5 version.In 8.0.1 IBM included Quality stage, IBM information server (profile stage and audit stage) . these are not available in 7.5...
veer.dwh
- Jul 20th, 2009
Compare to DataStage 7.5 Accential DataStage, with DataStage 8.0 IBM websphere datastage Architecture got changedIn 8.0 IBM included client components of IBM information server which supports profiling stage along with Quality stage features are included in designer client component.
DataStage Interview Questions
Ans