Getting started

This section covers basic tutorials of smf2db including:

Installation

If you already know how to install python packages, then you can simply create a Python virtual environment and install the package like this:

pip install smf2db

See also

Refer to the Installing smf2db for more detail instructions.

Printing the report

Assuming that you have had some SMF JSON files, (if not, please following the instructions on CBTTape to get ready of some SMF JSON files,) let’s print the reports to have an overview of your data. It is started with the report program:

smf2db report <smf type> <jsonfiles> -o <outfile>

where smf type is the SMF type (e.g. 30, 70, 123, etc) and jsonfiles is the JSON files in which you want to print the report. The -o <outfile> option outputs the report to a file outfile.


The following table is a reference of the corresponding reports supported:

SMF Type

Subtype

Report

30

1,2,3,4,5,6

Address Space Activity

70

1

CPU Activity

‘’

2

Cryptographic Hardware Activity

71

1

Paging Activity

72

3

Workload Activty

‘’

5

Serialization Delay

73

1

Channel Path Activity

74

1

Device Activity

‘’

2

XCF Activity

‘’

3

OMVS Activity

‘’

4

CF Activity

‘’

5

Cache Subsystem Activity

‘’

6

HFS Statistics

‘’

7

FCD Activity

‘’

8

ESS Activity

‘’

9

PCIE Activity

‘’

10

EADM Activity

75

1

Page Data Set Activity

77

1

Enqueue Activity

78

2

Virtual Storage Activity

‘’

3

I/O Queuing Activity

110

1

CICS Performance Summary

‘’

2

CICS Statistics Summary

123

1

z/OS Connect EE Requeust Overview

See also

Refer to the smf2db man page for all options that report supports.

Usage Examples


Configuring Database

This section is for those who would like to use the upload function. (It will not cover how to install and use PostgreSQL.)

smf2db comes with a script called initcfg that create a yaml file, say, config.yaml in your preferred config directory, which you can configure all aspects of databases which will be used later for uploading to the database . The configuration values are set from a few queustions it asks you. Currently, only three db drivers are supported: sqlite, pg8000 and psycopg2 and it depends on whether it is supported on your platform. In addition, you can choose which partition scheme for your database according to your data volume: no partition, weekday, day of month and week number. Here is how to run this:

smf2db db initcfg --config_file <config file path>

For z/OS platform, you can run a batch job to create the config yaml file using a JCL like this:

//PYJOB    EXEC PGM=BPXBATCH,REGION=0M
//SYSPRINT DD SYSOUT=*
//STDERR   DD SYSOUT=*
//STDOUT   DD SYSOUT=*
//STDPARM  DD *
sh
<smf2db path>
db
initcfg
--config_fle <config file path>
--db_driver <db driver>
--sqlite_path <sqlite db path>
--partitions "<partition scheme>"
--db_prefix "<database name prefix>"

where config file path is the target config file in full path or relative path with extension yaml and it will be overrided if exist.

See also

Refer to the smf2db man page for all options that initcfg supports.

Initializing database

After creating the config yaml file using Configuring Database, you will need to run initdb. It will create the corresponding schemas, databases and tables in your chosen DBMS. You can initialize multiple smf types in one go. To do this, run:

smf2db db initdb <smf type> --config_file <config file>

Warning

This script will drop and recreate the tables if they are exist in your database with the same names.

For z/OS platform, you can run a batch job to initialize the databases using a JCL like this:

//PYJOB    EXEC PGM=BPXBATCH,REGION=0M
//SYSPRINT DD SYSOUT=*
//STDERR   DD SYSOUT=*
//STDOUT   DD SYSOUT=*
//STDPARM  DD *
sh
echo y |
<smf2db path>
db
initdb
<smf type list>
--config_fle <config file path>

where smf type is the SMF type (e.g. 30, 70, 123, etc) and config file is the target config file in full path or relative path with extension yaml which must be exist before running.

See also

Refer to the smf2db man page for all options that initdb supports.

Uploading data to DB

Assuming that you have had some SMF JSON files, (if not, please following the instructions on CBTTape to get ready of some SMF JSON files,) and you have run the Initializing database, you can now upload the data to your chosen DBMS based on the config.yaml. It is started with the upload program which is an interactive script which will prompt you for user ID and password to connect to the database if you are using PostgreSQL and you does not provide it on the command like this:

smf2db db upload <smf type> <jsonfiles> --config_file <config file>

For z/OS platform, you can run a batch job to upload data to database using a JCL like this:

//PYJOB    EXEC PGM=BPXBATCH,REGION=0M
//SYSPRINT DD SYSOUT=*
//STDERR   DD SYSOUT=*
//STDOUT   DD SYSOUT=*
//STDPARM  DD *
sh
<smf2db path>
db
upload
<smf type>
<json files>
--config_fle <config file path>

where smf type is the SMF type (e.g. 30, 70, 123, etc), jsonfiles is the JSON files in which you want to upload and config file is the target config file in full path or relative path with extension yaml which must be exist before running.

See also

Refer to the smf2db man page for all options that upload supports.

Usage Examples

Upload smf 70

By running the following command or submiting the JCL:

smf2db db upload 70 smf70_1.json smf70_2.json --config_file configs/config.yaml

//PYJOB    EXEC PGM=BPXBATCH,REGION=0M
//SYSPRINT DD SYSOUT=*
//STDERR   DD SYSOUT=*
//STDOUT   DD SYSOUT=*
//STDPARM  DD *
sh
/u/myuser/.venv/bin/smf2db
db
upload
70
/u/myuser/data/smf70_1.json
/u/myuser/data/smf70_2.json
--config_fle /u/myuser/configs/config.yaml

You will see something like the following content on your screen or job output, which is, the Execution time for each of the JSON file and the total number of records added to each of the tables after the process will be shown.

Execution time (/u/myuser/data/smf70_1.json): 0.043489535649617515 minutes
Execution time (/u/myuser/data/smf70_2.json): 0.012916600704193116 minutes
Upload Result (table name: row count): {'smf70_pro': 204, 'smf70_ctl': 102, 'smf70_cpu': 372, 'smf70_aid': 102, 'smf70_bct': 1212, 'smf70_bct_cpu': 1266, 'smf70_bpd': 4008, 'smf70_trg': 0, 'smf70_ccf': 102, 'smf70_typ3': 96, 'smf70_typ4': 96, 'smf70_typ5': 0}

Note

smf70 data is the pre-requsite of smf30 and other smf7x data. If you prepare to upload these data to database, you are required to upload smf70 data first.

Initializing sumup database

After creating the config yaml file by running Configuring Database and loading interval data to database from Uploading data to DB, you can sum up the interval data in database by running initsum to initialize the sumup database. It will create the corresponding schemas, databases and tables in your chosen DBMS. There are three summary levels you can choose: 15-minute, hourly and daily although only some smf-types support 15-minute. To use this, run:

smf2db db initsum <smf type> <summary level> --config_file <config file>

For z/OS platform, you can also run a batch job to initialize summary database using a JCL like this:

//PYJOB    EXEC PGM=BPXBATCH,REGION=0M
//SYSPRINT DD SYSOUT=*
//STDERR   DD SYSOUT=*
//STDOUT   DD SYSOUT=*
//STDPARM  DD *
sh
echo y |
<smf2db path>
db
initsum
<smf type>
<summary level>
--config_fle <config file path>

where smf type is the SMF type (e.g. 30, 70, 123, etc), summary level is either 15-min, hourly or daily depends on what you want to sumup and config file is the target config file in full path or relative path with extension yaml which must be exist before running.

Warning

This script will drop and recreate the summary tables if they are exist in your database with the same names.

See also

Refer to the smf2db man page for all options that initsum supports.

Summing up database

Assuming that you have uploaded some SMF JSON files to database, (if not, please following the instructions on Uploading data to DB to get ready of the interval databases,) and you have run the Initializing sumup database to initialize the database, you can now sum up the database to a specified summary level. It is started with the sumup program which is an interactive script and will prompt you for user ID and password to connect to the database if you are using PostgreSQL and does not provide it on the command like this:

smf2db db sumup <smf type> --summary_levl <summary level> --config_file <config file>

For z/OS platform, you can also run a batch job to summarize the database in different supported summary level using a JCL like this:

//PYJOB    EXEC PGM=BPXBATCH,REGION=0M
//SYSPRINT DD SYSOUT=*
//STDERR   DD SYSOUT=*
//STDOUT   DD SYSOUT=*
//STDPARM  DD *
sh
<smf2db path>
db
sumup
<smf type>
--summary_lvel <summary level>
--config_fle <config file path>

where smf type is the SMF type (e.g. 30, 70, 123, etc), summary level is either 15-min, hourly or daily depends on what summary level you want to sumup and config file is the target config file in full path or relative path with extension yaml which must be exist before running.

See also

Refer to the smf2db man page for all options that sumup supports.