IPC Stage:

An inter-process (IPC) stage is a passive stage which provides a communication channel between DataStage processes running simultaneously in the same job. It allows you to design jobs that run on SMP systems with great performance benefits. To understand the benefits of using IPC stages, you need to know a bit about how DataStage jobs actually run as processes, see “DataStage Jobs and Processes”.

The output link connecting IPC stage to the stage reading data can be opened as soon as the input link connected to the stage writing data has been opened. You can use Inter-process stages to join passive stages together. For example you could use them to speed up data transfer between two data sources:

In this example the job will run as two processes, one handling the communication from sequential file stage to IPC stage, and one handling communication from IPC stage to ODBC stage. As soon as the Sequential File stage has opened its output link, the IPC stage can start passing data to the ODBC stage. If the job is running on a multi processor system, the two processor can run simultaneously so the transfer will be much faster.

Defining IPC Stage Properties

The Properties tab allows you to specify two properties for the IPC stage:

  • Buffer Size.  Defaults to 128 Kb. The IPC stage uses two blocks of memory; one block can be written to while the other is read from. This property defines the size of each block, so that by default 256 Kb is allocated in total.
  • Timeout.  Defaults to 10 seconds. This gives time limit for how long the stage will wait for a process to connect to it before timing out. This normally will not need changing, but may be important where you are prototyping multi-processor jobs on single processor platforms and there are likely to be delays.

Leave a Reply