-
How can we remove duplicates using sort stage?
Alex
- Feb 28th, 2007
TreeSet<String> set = new TreeSet<String>(Arrays.asList(names)); for (String name : set) &nb...
-
Why copy stage is better than transformer?
Incase of only dropping columns(without any transformations or business rules), we can go for copy stage instead of transformer.But can anyone tell me exactly why copy stage is better in performance than transformer?
Lalit
- Oct 9th, 2017
Look Copy stage is made up for only copying data and transformer stage is made up for multiple functionalities and also it is made up of c++ code. So if you want to only copy data and rename datatypes...
Atangel
- Oct 12th, 2012
Transformer is a heavier stage as apart from the orchestrate we do have the C++ operators for various derivations. Right from compilation to execution a Transformer will always take more time than a copy stage which is a simple straight forward passive(if we can call) stage
-
What is the necessity to run the datastage job in Unix background?
SudhirKanojia
- Feb 9th, 2010
We need to execute dsenv file, before executing DataSatge job in UNIX background using DsRunJob commandRegard Sudhir Kanojia
veer.dwh
- Sep 23rd, 2009
You can run DS jobs using DS desiner and director client components. but in real time we will have huge no of jobs and those has sequnce of order dependency? to prform client requirment. in that situ...
-
What is the difference between Squential Stage & Dataset Stage. When do u use them.
manoharkolukula
- Jan 25th, 2008
Sequential file can used as a target file but Dataset can't be used as final target, it can be used as temperary target.Dataset can be used as a hashfile(just like in server hashfile), it improves...
vijaykumar_ads
- Jan 18th, 2008
Parallel jobs use data sets to manage data within a job. You can think of each link in a job as carrying a data set. The Data Set stage allows you to store data being operated on in a persistent form,...
-
What is the max size of Data set stage?
Venkat Duvvuri
- Sep 9th, 2011
S I agree with Hari's answer..Thae max size of the dataset stage is entirely depends upon the size of the resource disk space, which we have specified under config file.
Regards,
Venkat Duvvuri
harishsj
- Mar 14th, 2008
The Max size of a Dataset is equal to the summation of the space available in the Resource disk specified in the configuration file.
-
How to seperate two diff datatypes and load it into two files?
i have a source file1 consist of two datatypes
file1:
no(integer)
1
2
3
&
dept(char)
cs
it
ie
and i want to seperate these two datatypes and load it into target files
file2 & file3.
how can i do this in datastage and by using which stage?
lalit
- Oct 10th, 2017
So you can do it with transformer constraints like
allnum(), alpha() like function and separate both datatypes in two different outputsdileep Janga
- Jan 31st, 2014
I think this Question is to confuse the Job Aspirant by using Datatypes and all... Its very simple... File1-->2 Columns. 1.NO(Integer) 2.DEPT(Char). Target1: NO(Integer), Target2: DEPT(Char). Take ...
-
Why the sequential file not used in Hash Lookup?
manoharkolukula
- Feb 12th, 2008
Hi Rashmi,Can u please tell me ur telling about server jobs or parallel jobs.If it is server jobs means seq file can't be used as lookup(ref link), bcoz it doesn't accept that link, only hash ...
-
How to develop the SCD using LOOKUP stage?
-
How can we improve the performance of the job while handling huge amount of data
manoharkolukula
- Feb 12th, 2008
1. decrease usage of transformer stage.2. use sorted input for aggregator.3. Filter the data in source itself(if it is oracle database, write user def sql)4. use tunables tab in Administrator for cach...
sunayan
- Dec 13th, 2007
this require a job level tuning or server level tuning.in job level we can do the follwing.job level tuninguse Join for huge amount of data rather than lookup.use modify stage rather than transformer ...
-
Server transformer and Parallel transformer
Explain the difference between server transformer and parallel transformer
kairunnisha.k
- Aug 4th, 2008
Server transfomer is same as basic transfomer used in SMP systems. Uses a server interface to call a server transformer stage. But parallel transfomer are same as normal transfomer used in parallel jobs. They use a c-based environment that is called osh (orchestrate shell).
santhosh.gsk
- Dec 27th, 2007
The main difference is server Transformer supports basic transforms only,but in pararllel both basic and pararllel transforms. server transformer is basic language compatability,pararllel transformer is c++ language compatabillity.
-
Why you need Modify Stage?
When you are able to handle Null handling and Data type changes in ODBC stages why you need Modify Stage?
leelasankar.pr
- Jul 22nd, 2008
By using this modify stage we can change data types,keep and drop columns and also we can handle null values
-
Sparse Lookup and Normal Lookup
What are the differences between Sparse Lookup and Normal Lookup?
karthikgeekkarthik
- Jul 9th, 2008
normal look-up:all the reference table data is stored in the buffer for cross- check with the primary table data.sparse lookup:each record of the primary table is cross checked with the reference tabl...
nagakalyan
- Jun 23rd, 2008
Sparse means we will generate query source bases and one by one record ,and normal is we can take look up on target table at a time multiple records
-
Can we pass the parameters in job sequence
nikhilanshuman
- Mar 10th, 2010
Parameters can be passed to the job sequence using a shell script.Following is the command to run the sequence with passing the parameters :dsjob -run mode Normal -param param=value projectname S...
nikhilanshuman
- Dec 25th, 2009
Please provide more details.=> Are you asking to pass the parameters to the job sequence from any external script?=> Are you asking passing parameters to a parallel job from sequence?
-
Remove duplicates using transformer
How do you remove duplicates using transformer stage in datastage
Jaimy Chacko
- Oct 26th, 2009
To capture rejected duplicates use a Transformer. Partition and sort on your primary key. In a transformer keep the primary key stored in a Stage Variable. Compare incoming primary key to the stored p...
umamahesh.ds
- Jul 25th, 2008
In that Time double click on transformer stage---> Go to Stage properties(its having in hedder line first icon) ---->double click on stage properties --->Go to inputs ---->go to part...
-
Setting up of node
How do we set up nodes in Datastage?
amitarudra
- Jan 29th, 2010
We can set the nodes in the follwing path C:AscentialDataStageConfigurations, which is a text file containing { node "node1" { &nb...
john_345
- Nov 19th, 2008
Here not the question of admin or developer.For the above question we have provide proper answer, like Go to DS manager and create nodes into Configuaration file according to the requirement. Please if any of DS admin registered expecting step by step process. Mine is also not clear answer.
-
How do you use Surrogatekey key in reporting?
Mallikarjuna_G
- Aug 10th, 2013
In a typical data warehouse environment, we normally have a requirement to keep history. So, we would end up having multiple rows for a given Primary key. So, we define a new column that doesn't have ...
djkool4u
- Jul 11th, 2013
Is Slowly Changing Dimension (SCD) Implementation we use Surrogate key as the primary key is being duplicated for the sake of keeping history data for the records with the same pk.
-
User Created Stages
Explain Wrapped, Build and Custom stages?
Phuzzball
- Feb 15th, 2012
Datastage allows you to create your own stages with custom properties. There are three types of custom stages: Custom: This is Orchestrate based custom stage. Orchestrate operators are used for def...
MArcus_Datastage
- Mar 17th, 2009
Custom: This allows knowledgeable Orchestrate users to specifyan Orchestrate operator as a DataStage stage. This is then availableto use in DataStage Parallel jobs.Build: This allows you to design and...
-
Sequential File Stage
If are given a list of .txt files and asked to read only the first 3 files using seq file stage. How will you do it?
matan
- Nov 16th, 2012
In sequential file we can take a single file by using the file as Specified file.
But we can take the more than one file use File Pattern with different file names.
Metadata must be same.narra satish
- Mar 25th, 2011
HI, In sequential file we can take a single file byusing the file as Specifyed file.But we can take the more than one file use File Pattern with different file names.Metadata must be same.
-
Link sort/Stage sort
What is the use of sort stage when you have explicit sort stage?
nikhilanshuman
- Dec 9th, 2010
Link Sort and stage sort,both do the same thing.Only the Sort Stage provides you with more options like the amount of memory to be used,remove duplicates,sort in Ascending or descending order,Create change key columns and etc.These options will not be available to you while using Link Sort.
praveen.sarva
- Dec 1st, 2010
Simple... For huge amount of data we prefer link sorting... Under partitions for small and compatable data we use stage sort(nothing but sort with in data set).
-
Stage Variables Initialization
What are the initial values of stage variables? How can we set this values?
glaciya
- Dec 12th, 2011
In my thought, the inital variables are the variables which can be defined when you just have installed the software, in another word they are usually called default variable or you have the first chance to create and config some variables that are used in a stage.
G. Venu
- Oct 14th, 2011
In Transformer stage properties we can set the stage variable initial value.
DataStage Interview Questions
Ans