Job Compilation
1. Operators. These
underlie the stages in a DataStage job. A single stage may correspond
to a single operator, or a number of operators, depending on the properties you
have set, and whether you have chosen to partition or collect or sort data on
the input link to a stage. At compilation, WebSphere DataStage evaluates your
job design and will sometimes optimize operators out if they are judged to be
superfluous, or insert other operators if they are needed for the logic of the
job.
2. OSH. This
is the scripting language used internally by the WebSphere DataStage parallel
engine.
3. Players.
Players are the workhorse processes in a parallel job. There is generally a
player for each operator on each node. Players are the children of section
leaders; there is one section leader per processing node. Section leaders are
started by the conductor process running on the conductor node (the conductor
node is defined in the configuration file).
DataStage Designer client generates all code - Validates
link requirements, mandatory stage options, transformer logic, etc.
1. Generates
OSH representation of job data flow and stages
GUI
“stages” are representations of Framework “operators”
Stages in parallel
shared containers are statically inserted in the job flow
Each server shared container becomes a
dsjobsh operator
2. Generates
transform code for each parallel Transformer
Compiled on the
DataStage server into C++ and then to corresponding native operators
To
improve compilation times, previously compiled Transformers that have not been
modified are not recompiled
Force
Compile recompiles all Transformers (use after client upgrades)
3. Buildop
stages must be compiled manually within the GUI or using buildop UNIX command
line
Viewing of generated OSH is enabled in DS Administrator
OSH is visible in
1. Job
Properties
2. Job run
log
3. View
Data
4. Table Definitions