Solve with steps

employee_id, salary
-----------------------------
10, 1000
20, 2000
30, 3000
40, 5000

output for (previous row salary in the current row) as below

employee_id, salary, pre_row_salary
-----------------------------------
10, 1000, Null
20,...

There is a sequencer 1 scheduled at 12:30 that contains 2 jobs. After sequencer 1 finishes sequencer 2 will execute that again contains 2 jobs. Now if one of the jobs on sequencer 1 fails, then sequencer 2 will not execute. Sequencer 1 should exactly start at 12:30 AM every day. Write the jil script for these sequencers.

I am getting a file in and I want to be sure that it has specific traits. The first record must say HEADER, the second record must say PROVIDER, the last record must say TRAILER, Record Count must equal TRAILER count (HEADER - TRAILER = Record Count; TRAILER COLUMN has number of Records)

To further this, it is probably simple, but :

TABLENAME(Column) CLOB(Column)
HEADER(first row) PROVIDER12082015
MISCROWS IN BETWEEN
TRAILER(last row) 00000000669

If I have three columns created in the transformer (HeadName, HeadDate, RecCount), how would I put this in the derivations?
HeadName is supposed to be equal to PROVIDER
HeadDate...

How to integrate an Informatica Data Quality mapplet with DataStage?

Explain telecom related project in data stage? For example You are handling data usage and roaming related data. How you will get the data? What mediation team will do? What are the terms such as CDR (Call Detail Record), CDF will you use?

Input:
almost
banana
very
begins
order
in


Output:
almost IN ORDER
banana NOT IN ORDER
very NOT IN ORDER
begins IN ORDER
order NOT IN ORDER
in IN ORDER

In source we have 10 files with different metadata and different Source files we need to load the files to target with 10 different target but we should do in a single datastage job how can we do this

In detailed-
I have 5 input files that are completely different formats (the number of columns and metadata are different). How can I read all files using a sequential file stage? When I read the data from each file, how can I confirm which data belongs to which file?

Sincere request, Please suggest for the correct answer. I appreciate any help you can provide.

Container is a collection of stages used for the purpose of Reusability. There are 2 types of Containers. a) Local Container: Job Specific b) Shared Container: Used in any job within a project. 

jagannimmagadda

 Profile Answers by jagannimmagadda Questions by jagannimmagadda

• Jan 10th, 2006

Container is a collection of stages used for the purpose of Reusability. There are 2 types of Containers. a) Local Container: Job Specific b) Shared Container: Used in any job within a project. ? Ther...

Answer Question  Select Best Answer

Link Partitioner : It actually splits data into various partitions or data flows using various partition methods . Link Collector : It collects the data coming from partitions, merges it into a single data flow and loads to target. 

jagannimmagadda

 Profile Answers by jagannimmagadda Questions by jagannimmagadda

• Jan 10th, 2006

server jobs mainly execute the jobs in sequential fashion,the ipc stage as well as link partioner and link collector will simulate the parllel mode of execution over the sever jobs having single cpu L...

Answer Question  Select Best Answer

narsimhulu.D

• Sep 11th, 2005

Using Orabulk

Answer Question  Select Best Answer

Give the OS of the Server and the OS of the Client of your recent most project

rcp

• Oct 13th, 2005

Server is unix and client machine i.e is ur machine where u design a job is windows xp professional

Answer Question  Select Best Answer

Used for Look-ups. It is like a reference table. It is also used in-place of ODBC, OCI tables for better performance.   

RamaPrasad Batchu

• Mar 8th, 2006

We can also use the Hash File stage to avoid / remove dupilcate rowsby specifying the hash key on a particular fileld

Answer Question  Select Best Answer

CNU

• Aug 2nd, 2005

You use the Designer to build jobs by creating a visual design that models the flow and transformation of data from the data source through to the target warehouse. The Designer graphical interface lets you select stage icons, drop them onto the Designer work area, and add links. 

Answer Question  Select Best Answer

Yes. One of the most important requirements.     

sekr

 Profile Answers by sekr Questions by sekr

• Dec 10th, 2005

some times u need to write unix progrms in back round !like batch progms ! bcz data stage can invoke a batch processing in every 24 hrs .soo.......unix must... so that we can run the unix prog in back round even min/ hrs

Answer Question  Select Best Answer

IConv() - Converts a string to an internal storage formatOConv() - Converts an expression to an output format.

Read Best Answer

Editorial / Best Answer

sekr  

 Profile Answers by sekr Questions by sekr

• Member Since Dec-2005 | Jan 30th, 2006

---

iconv is used to convert the date into into internal format i.e only datastage can understand

example :- date comming in mm/dd/yyyy format

datasatge will conver this ur date into some number like :- 740

u can use this 740 in derive in ur own format by using oconv.

suppose u want to change mm/dd/yyyy  to dd/mm/yyyy

now u will use iconv and oconv.

ocnv(iconv(datecommingfromi/pstring,SOMEXYZ(seein help which is iconvformat),defineoconvformat))

sekr

 Profile Answers by sekr Questions by sekr

• Jan 30th, 2006

iconv is used to convert the date into into internal format i.e only datastage can understand example :- date comming in mm/dd/yyyy format datasatge will conver this ur date into some number like :- 7...

Improve Answer

Typically a Reject-link is defined and the rejected data is loaded back into data warehouse. So Reject link has to be defined every Output link you wish to collect rejected data. Rejected data is typically bad data like duplicates of Primary keys or null-rows where data is expected. 

sunilsavalasang

• Oct 18th, 2006

we can handle rejected data by collecting them seperately in sequential file......

Answer Question  Select Best Answer

Staged the data coming from ODBC/OCI/DB2UDB stages or any database on the server using Hash/Sequential files for optimum performance also for data recovery in case job aborts.Tuned the OCI stage for 'Array Size' and 'Rows per Transaction' numerical values for faster inserts, updates and selects.Tuned the 'Project Tunables' in Administrator for better performance.Used sorted data for Aggregator.Sorted...

Read Best Answer

Editorial / Best Answer

sistlasatish  

 Profile Answers by sistlasatish Questions by sistlasatish

• Member Since Nov-2005 | Nov 14th, 2005

---

1. Minimise the usage of Transformer (Instead of this use Copy, modify, Filter, Row Generator)
2. Use SQL Code while extracting the data
3. Handle the nulls
4. Minimise the warnings
5. Reduce the number of lookups in a job design
6. Use not more than 20stages in a job
7. Use IPC stage between two passive stages Reduces processing time
8. Drop indexes before data loading and recreate after loading data into tables
9. Gen\'ll we cannot avoid no of lookups if our requirements to do lookups compulsory.
10. There is no limit for no of stages like 20 or 30 but we can break the job into small jobs then we use dataset Stages to store the data.
11. IPC Stage that is provided in Server Jobs not in Parallel Jobs
12. Check the write cache of Hash file. If the same hash file is used for Look up and as well as target, disable this Option.
13. If the hash file is used only for lookup then \"enable Preload to memory\". This will improve the performance. Also, check the order of execution of the routines.
14. Don\'t use more than 7 lookups in the same transformer; introduce new transformers if it exceeds 7 lookups.
15. Use Preload to memory option in the hash file output.
16. Use Write to cache in the hash file input.
17. Write into the error tables only after all the transformer stages.
18. Reduce the width of the input record - remove the columns that you would not use.
19. Cache the hash files you are reading from and writting into. Make sure your cache is big enough to hold the hash files.
20. Use ANALYZE.FILE or HASH.HELP to determine the optimal settings for your hash files.

       This would also minimize overflow on the hash file.

21. If possible, break the input into multiple threads and run multiple instances of the job.
22. Staged the data coming from ODBC/OCI/DB2UDB stages or any database on the server using Hash/Sequential files for optimum performance also for data recovery in case job aborts.
23. Tuned the OCI stage for 'Array Size' and 'Rows per Transaction' numerical values for faster inserts, updates and selects.
24. Tuned the 'Project Tunables' in Administrator for better performance.
25. Used sorted data for Aggregator.
26. Sorted the data as much as possible in DB and reduced the use of DS-Sort for better performance of jobs
27. Removed the data not used from the source as early as possible in the job.
28. Worked with DB-admin to create appropriate Indexes on tables for better performance of DS queries
29. Converted some of the complex joins/business in DS to Stored Procedures on DS for faster execution of the jobs.
30. If an input file has an excessive number of rows and can be split-up then use standard logic to run jobs in parallel.
31. Before writing a routine or a transform, make sure that there is not the functionality required in one of the standard routines supplied in the sdk or ds utilities categories.
    Constraints are generally CPU intensive and take a significant amount of time to process. This may be the case if the constraint calls routines or external macros but if it is inline code then the overhead will be minimal.
32. Try to have the constraints in the 'Selection' criteria of the jobs itself. This will eliminate the unnecessary records even getting in before joins are made.
33. Tuning should occur on a job-by-job basis.
34. Use the power of DBMS.
35. Try not to use a sort stage when you can use an ORDER BY clause in the database.
36. Using a constraint to filter a record set is much slower than performing a SELECT ? WHERE?.
37. Make every attempt to use the bulk loader for your particular database. Bulk loaders are generally faster than using ODBC or OLE.

sistlasatish

 Profile Answers by sistlasatish Questions by sistlasatish

• Nov 14th, 2005

Minimise the usage of Transformer (Instead of this use Copy, modify, Filter, Row Generator)Use SQL Code while extracting the dataHandle the nullsMinimise the warningsReduce the number of lookups in a ...

Improve Answer

A. Under Windows: Use the 'WaitForFileActivity' under the Sequencers and then run the job. May be you can schedule the sequencer around the time the file is expected to arrive.B. Under UNIX: Poll for the file. Once the file has start the job or sequencer depending on the file.

jyothi

• Mar 21st, 2006

Under Unix :what does Poll for file mean?.can u plz explain in detail

Answer Question  Select Best Answer