Mainframes 360
The one stop destination for System Z professionals

Thursday, July 30, 2009

Formatting Output Records – DFSORT Tutorial


Q. What does one understand by the phrase – Formatting of Output Records? Why do we need to format them?

When we generate an Output using DFSORT sometimes, the records are not in a presentable format. For example, in the output records, there might not be a space between two fields. Or you might want to insert a $ sign for the currency field. You might want to re-order the fields, or display only selected fields/columns in the Output. This type of formatting of records can be done using DFSORT.

You can format the records to be sorted in a particular way, prior to the sorting. This is also possible. So, for better presentation/easier understanding of the report, we re-format the records produced in Output.

For re-formatting of records, there are mainly 3 tools at your disposal -
BUILD - Reformat each record by specifying all of its items one by one. Here, you can add,delete or re-arrange fields.

OVERLAY - It allows you to change (edit) the values in specific columns.

IFTHEN - Reformats different records in different ways.

The above re-formatting tools can be used with INREC, OUTREC and OUTFIL.

INREC - Used to reformat the records before applying sort.
OUTREC - Used to reformat the records after applying sort.

Q. How to generate a Report, which displays only specific(desired) columns in the Output?


Suppose you are given the following Input Dataset.

----+----1----+----2----+----3----+----4----+----5----+----6----+----7--
AG10001  RAMESH CHANDRA GUHA INDIA     MAHARASHT MUMBAI    01000       
AG10002  QUASAR SHABBIR CHUN INDIA     MAHARASHT PUNE      02000       
AG10003  VEERAMANI DARYAGANJ INDIA     MAHARASHT NAGPUR    03000       
AG10004  DELPHAN FERANDAZ    INDIA     TAMIL NAD CHENNAI   04000       
AG10005  PADMAVATHI SUNDARAM INDIA     TAMIL NAD SALEM     05000       
AG10006  NEELESHWARI IYER    INDIA     TAMIL NAD ERNAKULAM 06000       
AG10007  AJAY NANUBHAI PARSA INDIA     WEST BENG KOLKATA   07000       
AG10008  MEHUL POPATLAL SUTA INDIA     WEST BENG KHARAGPUR 08000       
AG10009  KRITHIKA RAMANUJAM  INDIA     WEST BENG SUNDARBAN 09000       
AG10010  ANKUR CHOUDHARY     INDIA     PUNJAB    AMRITSAR  10000       
AG10011  HIREN NITIN SHETH   INDIA     PUNJAB    JALANDHAR 11000       
AG10012  DARSHAN SHAH        INDIA     HARYANA   CHANDIGAR 12000       


In the output report, you are asked to display only the name of the Insurance Agent and the Sales Figure. Since, we want reformat records after applying SORT, we will OUTREC FIELDS Control statement.

We want to have two columns/fields in the ouput -
NAME   Start-Position 10    Length 20
SALES  Start-Position 60    Length 05

Thus, we can write the Batch JOB/JCL for the above requirement as follows :

----+----1----+----2----+----3----+----4----+----5----+----6----+----7--
//CAA0176J JOB A123,QUASAR,NOTIFY=&SYSUID                              
//STEP01 EXEC  PGM=SORT,REGION=1024K                                   
//SORTIN DD    DSN=CAA0176.DEMO.SORTIN,DISP=SHR                        
//SYSPRINT DD  SYSOUT=*                                                
//SYSOUT DD    SYSOUT=*                                                
//SORTOUT DD   DSN=CAA0176.DEMO.SORTOUT,DISP=(OLD,KEEP,KEEP)           
//SYSIN  DD    *                                                       
  SORT FIELDS=COPY -                                                   
  OUTREC FIELDS=(10,20,60,5)                                           
//                                                                     


Note that, we have written SORT FIELDS=COPY, which simply copies the Input Records to the Output, and displays NAME and SALES Columns. Upon submitting the above Batch JOB/JCL, we get the following Output :

----+----1----+----2----+
RAMESH CHANDRA GUHA 01000
QUASAR SHABBIR CHUN 02000
VEERAMANI DARYAGANJ 03000
DELPHAN FERANDAZ    04000
PADMAVATHI SUNDARAM 05000
NEELESHWARI IYER    06000
AJAY NANUBHAI PARSA 07000
MEHUL POPATLAL SUTA 08000
KRITHIKA RAMANUJAM  09000
ANKUR CHOUDHARY     10000
HIREN NITIN SHETH   11000
DARSHAN SHAH        12000


A lot more formatting is possible on the above report. Read on to know more...

Q. What if I wanted the SALES field/column to be displayed before(first) and then the NAME Field/Column? How do you re-order columns in the Output Report?


If we would like to display the SALES Column first, and then the agent-names, we can simply change the order in OUTREC FIELDS statement.

  SORT FIELDS=COPY -                                                   
  OUTREC FIELDS=(60,5,10,20)  


Upon submitting a Batch JOB/JCL with the above control statements, we get the following output :

----+----1----+----2----+
01000RAMESH CHANDRA GUHA
02000QUASAR SHABBIR CHUN
03000VEERAMANI DARYAGANJ
04000DELPHAN FERANDAZ   
05000PADMAVATHI SUNDARAM
06000NEELESHWARI IYER   
07000AJAY NANUBHAI PARSA
08000MEHUL POPATLAL SUTA
09000KRITHIKA RAMANUJAM 
10000ANKUR CHOUDHARY    
11000HIREN NITIN SHETH  
12000DARSHAN SHAH       

Q. I’ve understood, how to re-order the columns, but the above output report looks like greek(Cyrilic) – there should be some space between the fields SALES and NAME. How do you insert a SPACE/SPACES?


Since the above output looks very clumsy, to make this report more readable we would like to insert 2 blank spaces between SALES and AGENT-NAME. To insert a blank space, we use X. To insert 2 blank spaces, we write 2X.

  SORT FIELDS=COPY -                                                    
  OUTREC FIELDS=(60,5,2X,10,20)
                                         

Thus, upon submitting a Batch JOB/JCL with the above Control Statements, we get the following Output Report.


----+----1----+----2----+-
01000  RAMESH CHANDRA GUHA
02000  QUASAR SHABBIR CHUN
03000  VEERAMANI DARYAGANJ
04000  DELPHAN FERANDAZ  
05000  PADMAVATHI SUNDARAM
06000  NEELESHWARI IYER  
07000  AJAY NANUBHAI PARSA
08000  MEHUL POPATLAL SUTA
09000  KRITHIKA RAMANUJAM
10000  ANKUR CHOUDHARY   
11000  HIREN NITIN SHETH 
12000  DARSHAN SHAH      


If you want to fill in blanks in the last field, you use the syntax c:X, where c is the column till where the padding takes place.

For example,

OUTREC FIELDS=(60,5,2X,10,20,140:X)

Q. What if I wanted to insert a Character string like ‘IS SALES FOR’ between the two fields? What if I wanted to insert a Date and Timestamp?

You can insert characters at any position by tying C'<Characters>'.
Suppose all records have length 65 bytes. Suppose, you would like to insert at the end of each record the current date and timestamp.

Then, we would insert DATE1 and TIME2 fields respectively. This is the Batch JOB/JCL for doing it.

----+----1----+----2----+----3----+----4----+----5----+----6----+----7--
//CAA0176L JOB A123,QUASAR,NOTIFY=&SYSUID                              
//STEP01 EXEC  PGM=SORT,REGION=1024K                                   
//SORTIN DD    DSN=CAA0176.DEMO.SORTIN,DISP=SHR                        
//SYSPRINT DD  SYSOUT=*                                                
//SYSOUT DD    SYSOUT=*                                                
//SORTOUT DD   DSN=CAA0176.DEMO.SORTOUT,DISP=(OLD,KEEP,KEEP)           
//SYSIN  DD    *                                                       
  SORT FIELDS=COPY -                                                   
  OUTREC FIELDS=(60,5,2X,10,20,X,DATE1(/),X,TIME2(:))                  
//                                                                     


Upon submitting the above Batch JOB/JCL, we get the following Output Report, with the Date and Time printed :

----+----1----+----2----+----3----+----4----+
01000  RAMESH CHANDRA GUHA  2009/07/29 03:32
02000  QUASAR SHABBIR CHUN  2009/07/29 03:32
03000  VEERAMANI DARYAGANJ  2009/07/29 03:32
04000  DELPHAN FERANDAZ     2009/07/29 03:32
05000  PADMAVATHI SUNDARAM  2009/07/29 03:32
06000  NEELESHWARI IYER     2009/07/29 03:32
07000  AJAY NANUBHAI PARSA  2009/07/29 03:32
08000  MEHUL POPATLAL SUTA  2009/07/29 03:32
09000  KRITHIKA RAMANUJAM   2009/07/29 03:32
10000  ANKUR CHOUDHARY      2009/07/29 03:32
11000  HIREN NITIN SHETH    2009/07/29 03:32
12000  DARSHAN SHAH         2009/07/29 03:32


In the next tutorial, find out how to replace certain characters/fields, eliminate junk characters like LF, changing lower case to upper case and vice versa and much more.. Hope you’ve enjoyed!

Tuesday, July 28, 2009

MVS System Catalog – Finding information about VSAM Cluster from the Catalog


Q. What’s a System Catalog? Why have one?
A Catalog is like an address book. A real-life address book stores the names of people, and their addresses(where they live/reside, where they work etc..). In a Mainframe System, there are many files(Datasets), that live in different areas of Memory(Storage). Each dataset is found in a different area/location. If you wanted to find a particular dataset, do anything with it, read it, update it etc., you would need to lookup where the dataset is. How do you know, in this vast expanse of storage space consisting several DASD Devices, each having many volumes, where your dataset lies. Its like finding a needle in a haystack.

Thanks to the MVS System Catalog. I call it the Address Book of the MVS O/S. You want to access a dataset, but don’t know where it is? Ask the System Catalog. The System Catalog tells you, where in the whole world of storage space, you dataset is..

Storage Devices like DASD are just like Hard-disk Drives. Inside, there are a no. of plates(platters)/circular disks. Each disk is called a volume. Every disk volume records information in tracks. These are just like the tracks on vinyl record/compact disk.

The Address Book of the MVS O/S – System Catalog stores the name of the dataset, along with the disk volume on which it resides.

Have you ever wondered, on Windows OS, why can't you have 2 files with the same name in a folder? In MVS O/S, on the same lines, you can't create two datasets with the same identical names. Its pretty easy to understand why. The MVS O/S keeps track of all the datasets. It stores information about the name of a dataset, and where this dataset is present on the disk(which volume). In other words, MVS O/S records the name and disk volume of the dataset.The list of all the datasets and their locations are stored in an area called MVS System Catalog. We say, that the dataset is cataloged. If MVS, does not have any information about the whereabouts of a dataset, that is it does not know, where to look for a dataset, we say that the dataset is un-cataloged.
Q. I would like to see what the MVS System Catalog looks like? How do see what’s in there?

When you would like to see(peek in) the entries of the MVS System Catalog, you must use the LISTCAT Command of the IDCAMS utility. In case of VSAM Files, the  MVS System Catalog also stores information about the Alternate Indexed, no. of records, control interval size, free space etc. So, you can find a lot meta-information(Information about data) for VSAM Files in the MVS System Catalog.

You can run LISTCAT from TSO. You must type TSO LISTCAT, and you can see the output.

The LISTCAT Command can be used to find the entries of the VSAM Cluster in the Catalog. So, if you want to see the entries of your VSAM file in the Catalog, you can use the LISTCAT AMS Control Statement. The LISTCAT Command has a few options :
1) ENTRIES - This is used to filter out the entry that you want to be listed.

For example, if you want to see all the catalog entries with the name CAA0176.DEMO.KSDS, you can code the ENTRIES option as

ENTRIES(CAA0176.DEMO.KSDS)

To code multiple files, you can simply put the continuation character :

ENTRIES(CAA0176.DEMO.KSDS1,          -
        CAA0176.DEMO.KSDS2,          -
        CAA0176.DEMO.KSDS3)

You can also make use of a generic name. Suppose, you wanted to list all entries in the catalog having the dataset names starting with CAA0176.DEMO.

ENTRIES(CAA0176.DEMO.*)

2) CLUSTER - This is used to specify that only the cluster entry is to be listed. If this is not mentioned, all the index entries would also be printed.
3) ALL - This means all field of the entry are to printed.

A Simple Batch JOB/JCL making use of the LISTCAT Command is shown here :

----+----1----+----2----+----3----+----4----+----5----+----6----+----7--
//LISTCAT3 JOB A123,QUASAR,NOTIFY=&SYSUID                              
//STEP1  EXEC  PGM=IDCAMS                                              
//SYSPRINT DD  DSN=CAA0176.DEMO.JOBLOG,DISP=(OLD,KEEP,KEEP)            
//SYSIN    DD    *                                                     
  LISTCAT -                                                            
        ENTRIES(CAA0176.DEMO.KSDS) -                                   
        CLUSTER -                                                      
        ALL                                                            
/*                                                                     


Upon submitting the above Batch JOB/JCL. the JOB Log shows the following information -

DCAMS  SYSTEM SERVICES                                           TIME:
  LISTCAT -                                                            
        ENTRIES(CAA0176.DEMO.KSDS) -                                   
        CLUSTER -                                                      
        ALL                                                            
CLUSTER ------- CAA0176.DEMO.KSDS                                      
     IN-CAT --- CATALOG.T.ICFTST03                                     
     HISTORY                                                           
       DATASET-OWNER-----(NULL)     CREATION--------2009.208           
       RELEASE----------------2     EXPIRATION------0000.000           
     SMSDATA                                                           
       STORAGECLASS -------WORK     MANAGEMENTCLASS---SORTMC           
       DATACLASS --------(NULL)     LBACKUP ---0000.000.0000           
       BWO STATUS------00000000     BWO TIMESTAMP---00000 00:00:00.0   
       BWO---------------(NULL)                                        
     RLSDATA                                                           
       LOG ----------------(NULL)   RECOVERY REQUIRED --(NO)     FRLOG -
       VSAM QUIESCED -------(NO)    RLS IN USE ---------(NO)           
       LOGSTREAMID-----------------------------(NULL)                  
       RECOVERY TIMESTAMP LOCAL-----X'0000000000000000'                
       RECOVERY TIMESTAMP GMT-------X'0000000000000000'                
     PROTECTION-PSWD-----(NULL)     RACF----------------(NO)           
     ASSOCIATIONS                                                      
       DATA-----CAA0176.DEMO.KSDS.DATA                                 
       INDEX----CAA0176.DEMO.KSDS.INDEX                                
IDCAMS  SYSTEM SERVICES                                           TIME:
         THE NUMBER OF ENTRIES PROCESSED WAS:                          
                   AIX -------------------0                            
                   ALIAS -----------------0                            
                   CLUSTER ---------------1                            
                   DATA ------------------0                            
                   GDG -------------------0                            
                   INDEX -----------------0                            
                   NONVSAM ---------------0                            
                   PAGESPACE -------------0                            
                   PATH ------------------0                            
                   SPACE -----------------0                            
                   USERCATALOG -----------0                            
                   TAPELIBRARY -----------0                            
                   TAPEVOLUME ------------0                            
                   TOTAL -----------------1                            
         THE NUMBER OF PROTECTED ENTRIES SUPPRESSED WAS 0              
IDC0001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 0              
IDC0002I IDCAMS PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 0      


As you might notice, the above Catalog Entry gives all information like AIX, ALIAS, CLUSTER, DATA Cluster’s name, INDEX Cluster’s Name. Moreover, it also gives all the SMS-related(where on the disk this dataset resides) information.

Monday, July 27, 2009

VSAM Tutorial – How to create a ESDS and RRDS Cluster


Q. How do you create an ESDS Object using JCL/Batch JOB?
An ESDS Cluster is a VSAM Dataset, which is similar to a sequential file. An ESDS Dataset has no Index Structure. Now, you might wonder, how to access the records of an ESDS Cluster. Well, to access a record in an ESDS Cluster, you need to know the exact address/physical location, where record lies on the Disk Volume. It’s like saying, if you had a 1000 page book, and you wanted to access the line having keyword laugh, you must know beforehand on which page no(address) laugh is.

Thus, Records in ESDS Cluster are accessed by their Physical Address. This address is called Relative Byte Address(RBA). ESDS works just like a stack/pile of books. If you add records, they always get added to the top of the pile/stack, i.e. the records get appended. Since the pile is too heavy, it is not possible to insert records in between.

Here, I’ve shown how to create an empty ESDS Cluster using IDCAMS, DEFINE CLUSTER Control statement.

----+----1----+----2----+----3----+----4----+----5----+----6----+----7--
//MYJOB004 JOB A123,QUASAR,NOTIFY=&SYSUID                              
//STEP01 EXEC PGM=IDCAMS                                               
//SYSPRINT DD DSN=CAA0176.DEMO.JOBLOG,DISP=(OLD,KEEP,KEEP)             
//SYSIN  DD   *                                                        
  DEFINE CLUSTER(                                                      -
                 NAME(CAA0176.DEMO.ESDS)                               -
                 TRACKS(1 1)                                           -
                 RECORDSIZE(80 80)                                     -
                 NONINDEXED                                            -
                 )                                                     -
  DATA(                                                                -
                 NAME(CAA0176.DEMO.ESDS.DATA)                          -
       )                                                               
/*                                                                     
//                                                                     


In the above example, I’ve created a new ESDS Cluster with the name CAA0176.DEMO.ESDS. This ESDS Cluster will be allocated space(extents) in terms of 1 Primary Track and 1 Secondary Track. All records in this ESDS Cluster will be of length 80 bytes. Moreover, since an ESDS Cluster does not have any key, we don’t write KEYS keyword. Also, ESDS Cluster is specified using the option NONINDEXED.

Since, it has on DATA Component, only the DATA Sub-parameter has been coded. The ESDS Data component will have the name – CAA0176.DEMO.ESDS.DATA.
Q. How do you create an RRDS Object using a Batch JOB/JCL?
In a Relative Record Dataset, you must recall that each record is identified by a Relative Record Number(RRN), which is nothing but the sequence number relative to the first record.

Note : As opposed to KSDS, ESDS and RRDS support only fixed length records.

In RRDS, space is divided into fixed length slots. A slot can be either completely vacant(does not contain a record) or completely full(contains a record). Thus, you can add new records to empty slots, and delete existing records from slots which are filled.

Let’s see a simple JCL to create an RRDS Cluster :

----+----1----+----2----+----3----+----4----+----5----+----6----+----7--
//MYJOB004 JOB A123,QUASAR,NOTIFY=&SYSUID                              
//STEP01 EXEC PGM=IDCAMS                                               
//SYSPRINT DD DSN=CAA0176.DEMO.JOBLOG,DISP=(OLD,KEEP,KEEP)             
//SYSIN  DD   *                                                        
  DEFINE CLUSTER(                                                      -
                 NAME(CAA0176.DEMO.RRDS)                               -
                 TRACKS(1 1)                                           -
                 RECORDSIZE(80 80)                                     -
                 NUMBERED                                              -
                 REUSE                                                 -
                 )                                                     -
  DATA(                                                                -
                 NAME(CAA0176.DEMO.RRDS.DATA)                          -
       )                                                               
/*                                                                     
//                                                                     


RRDS is created on the same lines, like how we created an ESDS. The only difference being that since RRDS records are accessed by RRN(Sequence Number), we must code NUMBERED inside the DEFINE CLUSTER AMS Command.

Sunday, July 26, 2009

Creating a New KSDS File


Q. What is IDCAMS Software? How do you run it?
The free IBM-Software Program IDCAMS is used to create, alter, load and delete VSAM Files. I am going to elaborate, on how you write a job to run the IDCAMS Software to create a new KSDS-File.

The IDCAMS Software expects you two supply two important files – //SYSPRINT File and //SYSIN File.

 Image357
The skeleton of the job, that you must write to run IDCAMS Software is shown in this picture.

Image358

To tell IDCAMS, what to do, you type Control Instructions or Commands, which IDCAMS Software understands in the //SYSIN DD File. Here are some of the commands, that you can issue to IDCAMS -
1. DEFINE Command to create a new Object.
2. ALTER Command to alter an existing Object.
3. DELETE Command to delete objects.
4. PRINT Command to Print the contents of an Object.
5. REPRO Command to take Backups.

You must understand, how to write the Control-Card which contains the Special-Instructions for IDCAMS Software.
Q. How do you write the Control-Card to create a new KSDS File?
You type the DEFINE CLUSTER Command to create a new KSDS File. The model or format of this Command is shown in the picture below.

Image359

You type IDCAMS Commands like DEFINE, ALTER, DELETE etc. beginning from
Column 03. Also, if the Command spans across several lines, you must put a continuation character '-' in Column 72.

NAME – The NAME Parameter specifies the name of the VSAM File to be created newly.
TRACKS – The space allocation of the VSAM File is in terms of Tracks. This says, my new VSAM File will occupy 10 Tracks of Primary Space, and shall grow by 10 Tracks of secondary, when full. You may specify CYLINDERS or RECORDS as well.
VOLUMES – The VOLUMES Parameter specifies one or more DASD Volumes on which the VSAM File may be created.
CONTROLINTERVALSIZE – The CISZ Parameter tells the size of the Control-Interval in the VSAM File in Bytes. Generally, you would skip off this, and let AMS decide the best CI Size. It defaults to 4096 Bytes.
ERASE or NOERASE – The ERASE Parameter specifies, that when the Cluster is Deleted, the space must be over-written with Binary Zeroes, prior to marking the space free for Re-use.
FREESPACE(ci ca) – FREESPACE Parameter specifies the percentage of space to be left unallocated for future expansion. If no FREESPACE is specified, the Control-Intervals will be filled upto the brim.
IMBED and REPLICATE – These options are now obsolete. IMBED specifies that the Sequence-Set Records are to be embedded with the Data in Data-Cluster. Without Embedding, the Sequence-Set Records are kept with the Index-Set Records in the Index Cluster.
KEYS(length offset) – This is used to tell the Key Column in the VSAM File. 3 is the length of the Key, and it is at a Displacement of 0 Bytes from the Column 1.
RECORDSIZE(avg max) – This tells average and Maximum length of the Logical-Reocrds in the VSAM File. For Fixed-Length records, avg=max.
INDEXED, NONINDEXED and NUMBERED – This tells the Type of Cluster – KSDS(Data is accessed using Key Value), ESDS(Data is accessed using RBA), and RRDS(Data is accessed using Relative Record Number) respectively.

Wednesday, July 15, 2009

Introduction to VSAM Files


  This post is partially complete and several sections may require editing and proof-reading. The contents on the history of the organization of a VSAM File, CI and CA splits is under updation.

Access Methods
Each peripheral device such as a disk-drive or a tape cartridge has a different geometry, organization of data. Programs often process the data on a disk/tape. An application program must then contain logic to communicate with various I-O devices.

An access method is a software (bunch of tiny programs) to write device independent code. What device drivers are to Windows operating system, access methods are to mainframe zOS. Access methods assume the burden of interacting directly with the hardware. Developers can include high-level business logic, and avoid the gory details of the specific hardware in their program source-code. Access-methods are an interface between the program and the hardware device.

VSAM is one of the several access methods in the zOS. VSAM makes it easier for application programs to execute an I/O operation. There are several other access methods - QSAM, BSAM, ISAM etc.

Why VSAM?
QSAM is an access-method used to read/write data from Sequential Files. QSAM stands for Queued Sequential Access Method. QSAM queues records from the file, into an area of the main storage called System buffers. This speeds up the execution of the program.

The "V" in VSAM stands for Virtual. VSAM is an access method used to read/write data from direct-access devices, such as disk-drives.



Monday, July 13, 2009

Storage Management System - SMS

 
Read more tutorials on JCL >>

Q. What is the Storage Management System(SMS)? What is it used for?

The Storage Management System(SMS) is important part of the MVS O/S. Its function is to manage Storage Space. In other words, the memory manager of the MVS is called Storage Management System(SMS).

What meaning does this have when you write JCL? Its quite simple. When you write JCL/Batch JOBs, you often need to store your output/results, in OUTPUT Dataset. Moreover, you also need to store intermediate results of processing in Temporary/Work Datasets. When you create a new Dataset, for storing output, you must tell the MVS, how much storage space should be RESERVED for the dataset. That is, you need to declare/announce to the MVS, how much Storage Space(which is either in TRACKS,CYLS,BLKS,KB,MB) to keep aside for your dataset. Why do you need to do this?
Q. Why do you need to announce or make a declaration to MVS, “I need 1 TRACK Primary space, and 1 TRACK of secondary space?”

The reason is because, MVS O/S is a memory guard. It protects the memory. You cannot directly go ahead and store data in a Mainframe Computer’s memory. You need to first request the MVS O/S, “I need a few tracks of memory space to store my output data”. You need to properly/explicitly say, How much Storage Space you need – 1 track, 2 tracks, 5 cylinders, exactly how much? Not just this, you need to tell the MVS, the device and the volume on which you desire this storage space. Only when the MVS grants your request, and exclusively reserves Storage Space for you, you can then go ahead and store output dataset in it.

Thus, the MVS O/S acts like a miser. It does not give any TRACKS for free. Even if you want a single(1) byte of Storage Space, you need to first ask the MVS O/S.
Q. And how do you make this request to MVS O/S?

You already know it..! You do this by coding the SPACE parameter on the DD Statement, which creates a new dataset in the Batch JOB/JCL. If you need to brush up, on how the SPACE Parameter works, you can jump to the relevant tutorial by click here.
Q. Alright, I’ve had enough of writing the SPACE Parameter every time, I wanted to create a new dataset. Isn’t there some automatic way of doing this?

That’s precisely, what Storage Management System(SMS) is all about. Storage Management System(SMS) handles all the storage-space related issues for you automatically. For example, it can calculate how much Storage Space, your dataset needs, on which volumes to store your dataset, when is your dataset no longer needed, and how often to backup your dataset etc. However, one must realize that you will need to specify in your JCL, that you want SMS to manage your dataset.

Thus, in a nutshell, SMS is a tool that helps you by automatically managing Storage-space related issues for you.

You simply need to rub the Magic Lamp, and SMS-genie will manage storage-related issues for you!
 
Q. How does SMS know, how much storage space you’ll need, what volume to store the dataset on, bla bla bla...?

Storage Management Subsystem(SMS) manages your datasets, by asking you a few questions :

1)What’s the Storage Class ?

- Storage Class STORCLAS : The Storage Class represents different categories of devices and volumes. Every category has a meaning. Category name can be 1-8 characters long. For example, a STORCLAS=CLASSOA could be representing all volumes on 3380 DASD Device.

Note : You need to specify STORCLAS parameter in JCL, to signal to the SMS to manage your dataset. Otherwise, you will have to specify SPACE and VOL parameters for the dataset.

- Looking at a JCL/Batch JOB, you can make out whether a dataset is SMS-Managed or not, by seeing the STORCLAS parameter.

2)What’s the Management Class ?

- Management Class MGMTCLAS : Management class represents different categories of management activities. For example, a MGMTCLAS=MGMT01 could mean all datasets which are backed up everyday, and retained upto 1 month.
It is not mandatory to specify the MGMTCLAS parameter.

3) What’s the Data Class ?

- Data Class DATACLAS : Data Class represents different characteristics of the dataset’s records, what will be their record length, record organisation, record format etc. Since SMS can also be used to manage VSAM Datasets, the DATACLAS can be used to specify the Control Interval Size, Control Areas, Free Space, Imbed option etc. relevant to VSAM Clusters.

Once again, it is not mandatory to specify the DATACLAS.

Apart from these, if you wish, you could also specify other parameters to the MVS such as (i) RECORG, which is used for VSAM Datasets and tells what type of cluster to create – KSDS, ESDS, RRDS (ii) KEYOFF and KEYLEN, which in the context of VSAM Dataset tells, where the key starts, and what is the length of the key respectively, as well as (iii) AVGREC.
Q. Cut the crap out! Just show me a Batch JOB/JCL which uses SMS to manage my dataset...

Very well... It couldn’t be easier.

//HERC04A JOB A123,QUASAR
//STEP01 EXEC PGM=IEFBR14
//OUTDD DD DSN=HERC04.SMS.MNGDATA,
//   STORCLAS=ST01,
//   DISP=(NEW,CATLG,DELETE)
//    

Read more tutorials on JCL >>
References :
- Storage Management System – A marvel of IBM

Saturday, July 11, 2009

IBM Utilities Tutorial – 04 : Generation Data Group - GDG


References :
- Drona’s Tutorial – How to create and Delete GDGs
- Simotime – Everything about Generation Data Groups

Q. What is a GDG(Generation Data Group)?
Suppose we have stored a large amount of data of all credit-card holders in a Sequential Dataset. The Credit-card holders dataset, stores their name, billing address, total credit limit, available balance, minimum amount due etc. The data in the Credit-card holder file has to be updated every to reflect the purchases and payments made by the card-holder. So, periodically, every month, a Batch JOB is run to update the Credit Card Accounts dataset.

Suppose, the Credit Card Accounts have been updated till the month of July. But, what if we wanted to see the status of Credit Card Accounts, for a previous month say January, this wouldn’t be possible, since, we have lost the old data. In other words, we need to preserve the past historical data. What if the customer wanted a report which shows him, his Bank Account Statement of the last 6 months?

Thus, we need to store older/previous versions of the Credit Card Accounts dataset. When we want to maintain not only the current version of the file, but also previous versions, this calls for Generation Data Group(GDG).

A Generation Data Group(GDG) is a group of datasets, each having a father-son relationship. Every dataset/generation in the group is given a unique Generation Number. Suppose the name of the GDG is BANK.CREDITCARDS.ACCOUNT. Then, the first dataset which is created in this GDG, will have a generation number G0001V00. If we update this dataset, to create the next generation, the generation number becomes G0002V00. Again, for the next generation, the generation number will be G0003V00, and so on...

How to access a particular generation? There are two ways to do this. First, the name of a dataset inside a GDG is always => Name-of-GDG.generation-number. So, the generation G0002V00 can be accessed directly as BANK.CREDITCARDS.ACCOUNT.G0002V00. The other way, we can access generations is by using relative referencing. Suppose the current/latest generation in the GDG is G0003V00. The current generation is called Generation 0. So, the generation G0003V00 can be accessed as BANK.CREDITCARDS.ACCOUNT(0). To access the previous generations/ancestors of G0003V00, that is G0002V00, we must subtract (-1) from the generation number. So, the generation G0002V00 can be accessed as BANK.CREDITCARDS.ACCOUNT(-1). The successors of G0003V00, such as G0004V00 can be accessed as BANK.CREDITCARDS.ACCOUNT(+1). Thus, we can access datasets within a GDG in 2 ways : 1) Absolute dataset name 2) Relative reference
Q. What is a GDG Base or GDG Index?
To a build a GDG, you first need to build a GDG Base entry/GDG Index. This is nothing but an entry in the VTOC(Volume Table of Contents), a catalog which keeps track of all datasets on a volume.

To build a GDG Base, we use the IBM Supplied IDCAMS Utility program. The following JCL shows how :



The DEFINE GDG Control card is required to define a GDG Base.

NAME
– Specifies the name of the GDG. Here, the name of the GDG is HERC04.BANK.ACCOUNTS.

LIMIT
– Specifies how many generation will be stored in the GDG. Here, we have kept limit as 3. This means, a maximum of 3 generations will be stored in the GDG. You can specify any no. from 1 – 255.

NOEMPTY – This specifies, what action is to be taken when the GDG becomes full. NOEMPTY means, that when GDG becomes full(contains 3 generations), and another generation(4th generation) is added, the oldest generation will be deleted. EMPTY means that, when GDG is full, and 4th generation is added, all the old generations should be deleted, that the GDG should be made empty/flushed.

SCRATCH – This specifies, what happens to the Data Control Block of a generation when it it deleted. Every generation’s DCB is stored in VTOC. SCRATCH implies that when a generation is uncataloged, its DCB from the VTOC, should also be deleted/scratched(erased).
Q. What is GDG Model? Show me the JCL to create a GDG Model.
Now, that you know, how to create a GDG Base entry, the second thing before we can get started with storing generations in the GDG, is we have to create a GDG Model.

A GDG Model is a description of the DCB of all the datasets(generations) within the GDG. This means, all the generations within the GDG follow, the same record format, the same record length, the same block size etc. In other words, GDG Model dataset acts as a model/blueprint, a generalised template for all the generations that you will create and add to the GDG.

To create a GDG Model, we simply need to create an empty dataset, no space allocation, and bearing the DCB of the generations, you want to create. We can do this with the help of IEFBR14.

 
Q. Alright, so we’ve created GDG Base, GDG Model. Now, How about creating a Generation Dataset (GDS) inside the GDG?
Yup.. That’s what we’ll do now, without beating around the bush much, let’s come straight to the point. How do you create a GDS(Generation Dataset) inside a GDG?

To create a new Generation Dataset, we need to add (+1) to the GDG name, so as to indicate this is a new generation. That’s it. Rest all remains the same, just like we create an ordinary dataset.

Note : It is must to code UNIT and DDN Parameters on all new Generation Datasets.

So, here’s the JCL that adds a new generation to the above GDG HERC04.BANK.ACCOUNTS.

//HERC04A JOB A123,QUASAR
//STEP01 EXEC PGM=IEBGENER
//SYSUT1 DD *
RECORD-1 100001  
 RECORD-2 100002  
/*    
//SYSUT2 DD DSN=HERC04.BANK.ACCOUNTS(+1),
//   DISP=(NEW,CATLG,DELETE),
//   SPACE=(TRK,5),
//   DCB=HERC04.DEMO.GDGMODEL
//   UNIT=SYSDA
//SYSIN DD DUMMY
//    

Note : Only after this Batch JOB/JCL is complete, this GDS will become the current generation 0. If you are writing the code to access the GDS within the same Batch Job, it will still be referred to as Generation +1.

To many people who are thrown to work at a mainframe computer on their first job, they feel lost. Mainframe people seem to speak a completely different language and that doesn't make life easy. What's more, the books and manuals are incredibly hard to comprehend.

"What on earth is a Mainframe?" is an absolute beginner's guide to mainframe computers. We'll introduce you to the hardware and peripherals. We'll talk about the operating system, the software installed on a mainframe. We'll also talk about the different people who work on a mainframe. In a nutshell, we'll de-mystify the mainframe.

Readers based in India, can buy the e-book for Rs. 50 only or the print book. International readers based in the US and other countries can click here to purchase the e-book.