lavanya

• Dec 29th, 2016

 

We have many ways to achieve this one of them is
Create sequence job as

• Write a routine to count the no of files and read the individual filename

• Use looping activity --to perform no of iterations
No of iterations=count/5

• Next use 5 job activities by calling the same parallel job in all job activities with different parameter values

For example:
If source file folder has 10 files
No of iterations =10/5=2 iterations
the loop perform 2 iterations each time 5 instances total 10 files will be processed.
If source folder has 100 files
No of iterations=100/5=20 iterations
The loop performs 20 iterations to execute 100 instances by processing 100 files.

Routine passes the count to loop activity ex 20

Ram

• Jan 10th, 2017

 

Hi Aloka,
we can achieve this if you know the shell scripting. First design a job Source -->Target. then write a shell script for dsjob command to run the job and pass the parameters using parameter file for Source,schema & target tables. and run the job in loop.

Sirisha d

 Profile Answers by Sirisha d 

• Feb 13th, 2017

 

In the below logic, I am making sure the input file names have similar name format like Input1,Input2,Input3.... to parameterize input file names .
Step 1 : Place all the input files in a single txt file with on file name in each line.
Make sure there is a new line at the end of all the input file names.
Step 2 : Open a new sequence job.
In the Execute command stage , give the following command.
wc -l <#file_path#
Step 3: In the User Varaiable Stage, create a new variable to hold the output of execute Command stage.
Make sure you are removing Field Marks.
Use following expression.
Convert(@FM,,Execute_Command_0.$CommandOutput)
Step 4: Take a start loop and define starting values as 1, Step =1 and To =#UserVariables_Activity.New#
Step 5 : Pass this Start loop counter value to a parallel job (in which logic is developed to load target files) .
The loop will be repeated as many times as the number of number of input files defined in the single input text file.
Each time a target file will be created for each input file.
Parellel Job : In the parallel job to load different text files, make sure the input file names and output file names are parameterized to accept value from the start loop counter.
If the input files have different formats or if we want to load tables instead of files, we can include more logic to the above design to achieve the result.
Hope this helps.

shasank

• Apr 24th, 2017

 

We can achieve that by allowing multiple instances in a single job.
Just design a job that loads sources to target file and enable the multiple instances for that job.

I don't know exactly how to do it. As I have no Datastage software with. Try using that.

shasank

• Apr 24th, 2017

 

We can do that allowing multiple instances for a single job in the job properties tab.

Reddy

• Jun 27th, 2017

 

This can be achieved by creating multiple instance job and parameterizing the source file name, table name and also need to enable run time column propagation.