Read z/OS files in Python on Linux from COZLAUNCH

Discussion of the Co:Z Co-Processing Toolkit for z/OS
Post Reply
tsdjim
Posts: 64
Joined: Fri May 07, 2010 2:21 am

Read z/OS files in Python on Linux from COZLAUNCH

Post by tsdjim » Wed Mar 07, 2018 12:14 am

What is the correct way to read a z/OS Sequential file using Python on Linux from COZLAUNCH.

Something like this...
//INPUT DD DSN=...
python
f = open("//DD INPUT","r")
print f.read()

dovetail
Site Admin
Posts: 1856
Joined: Thu Jul 29, 2004 12:12 pm

Re: Read z/OS files in Python on Linux from COZLAUNCH

Post by dovetail » Wed Mar 07, 2018 9:32 am

You can do it, but it you have to run the "fromdsn" command on the remote Linux server and then pipe the output of that into your Python program.

BTW: doing what you suggest has some nice benefits: no data at rest, and you can process data while it is being transferred.

So, something like this, where your python code reads the data from stdin:
//RUNCOZ EXEC PROC=COZPROC,ARGS='user@linux1.myco.com'
//INPUT DD DSN=...
//STDIN DD *
fromdsn //DD:INPUT | python myprog.py
//
You can also get more sophisticated, where your python program internally runs the fromdsn (or todsn) commands and then reads (or writes) pipes to those commands.
See: https://stackoverflow.com/questions/476 ... the-output
(contrary to Python, there seems to be many ways to do it :-)

FYI, here is something similar done with another language, System R:
// EXEC PROC=COZPROC,ARGS='u@linux.myco.com'
//LIFEEXP DD DISP=SHR,DSN=KIRK.LIFEEXP.DATA
//STDIN DD *
#Run the R statistical system with inline commands
#This example is a 2-dimensional multiple regression

R --no-save <<EOB
mypipe = pipe("fromdsn DD:LIFEEXP", open="r")
life = read.table(mypipe, header=TRUE)
close(mypipe)
multilinearFit = lm(LifeExp~PeoplePerTV+PeoplePerDoctor,data=life)
summary(multilinearFit)
EOB
//

tsdjim
Posts: 64
Joined: Fri May 07, 2010 2:21 am

Re: Read z/OS files in Python on Linux from COZLAUNCH

Post by tsdjim » Thu Mar 08, 2018 12:12 am

I managed to get the following to work, however I had to move the data to the target platform, and was not successful in piping the data dynamically
from z/OS. Could you give me an example of how I can do this.

//STDIN DD *
fromdsn -s IBM-420 '//DD:INDATA' > first.data
fromdsn -s IBM-420 '//DD:INPGM > first.py
python < first.py

//INPGM DD *
f = open("first.data","r")
print f.read()
/*
//INDATA DD DSN=MY.FILE,DISP=SHR

dovetail
Site Admin
Posts: 1856
Joined: Thu Jul 29, 2004 12:12 pm

Re: Read z/OS files in Python on Linux from COZLAUNCH

Post by dovetail » Thu Mar 08, 2018 10:42 am

The reason that you couldn't get it to work was that you were trying put your python code inline (stdin) and if you use stdin for that, then you can't also use stdin for piping fromdsn data.

What I would do in this case is to use Bash "process substitution" for the fromdsn output.
This automatically sets up a temporary named pipe file pointing to the fromdsn output, and passes that filename as an argument to your python code. The temp file is only a named pipe (the data isn't stored on disk), and bash cleans up the temp pipe file automatically.

Then, use a '<<here' document for your inline python code.
//RUNCOZ EXEC PROC=COZPROC,ARGS='user@linux1.myco.com'
//INPUT DD DSN=...
//STDIN DD *
python - <(fromdsn //DD:INPUT) <<'__HERE'
import sys
f = open(sys.argv[1],"r")
print f.read()
__HERE
//
Cool or what?

tsdjim
Posts: 64
Joined: Fri May 07, 2010 2:21 am

Re: Read z/OS files in Python on Linux from COZLAUNCH

Post by tsdjim » Thu Mar 08, 2018 12:12 pm

Thanks, that is incredibly sleek, I will try that next week and update here.
Last edited by tsdjim on Thu Mar 08, 2018 2:51 pm, edited 1 time in total.

dovetail
Site Admin
Posts: 1856
Joined: Thu Jul 29, 2004 12:12 pm

Re: Read z/OS files in Python on Linux from COZLAUNCH

Post by dovetail » Thu Mar 08, 2018 12:29 pm

BTW: when using process substitution in bash: <() or >(), you can have more than one and also have output pipes.

Code: Select all

//RUNCOZ EXEC PROC=COZPROC,ARGS='user@linux1.myco.com'
//INPUT DD DSN=...
//OUT1  DD DSN=...
//OUT2  DD DSN=...
//STDIN  DD *
python - <(fromdsn //DD:INPUT) \
         >(todsn //DD:OUT1) \
         >(todsn //DD:OUT2) \
         <<'__EOP'
# your python code ...
import sys
# the (temp pipe) file names:
infile = sys.argv[1]
outfile1  = sys.argv[2]
outfile2  = sys.argv[3]
....
__EOP

# other shell commands....
//

dovetail
Site Admin
Posts: 1856
Joined: Thu Jul 29, 2004 12:12 pm

Re: Read z/OS files in Python on Linux from COZLAUNCH

Post by dovetail » Fri Mar 09, 2018 9:32 am

And maybe you want to keep your Python code in a PDS:
//PYCODE DD DISP=SHR,DSN=MY.PYTHON.SRCLIB
//STDIN DD *
python <(fromdsn //DD:PYCODE\(PROG1\)) <other args>
...

tsdjim
Posts: 64
Joined: Fri May 07, 2010 2:21 am

Re: Read z/OS files in Python on Linux from COZLAUNCH

Post by tsdjim » Mon Mar 12, 2018 12:45 am

I tried the code but I get the following error, could you guide me:

python - <(fromdsn //DD:INPUT) ×
>(todsn //DD:OUT1) ×
>(todsn //DD:OUT2) ×
<<'__EOP'
# your python code ...
import sys
infile = sys.argv[1]
outfile1 =sys.argv[2]
outfile2 = sys.argv[3]
f = open(infile,"r")
g = open(outfile1,"w")
h = open(outfile2,"w")
print f.read();
print g.write();
print h.write();
# the (temp pipe) file names:
__EOP
Traceback (most recent call last):
File "<stdin>", line 10, in <module>
TypeError: function takes exactly 1 argument (0 given)
todsn-client(56686)قEن: todsn(DD:OUT2): error: no input data available, rc=103,
todsn-client(56686)قEن: server exit_code=103
todsn-client(56687)قEن: todsn(DD:OUT1): error: no input data available, rc=103,
todsn-client(56687)قEن: server exit_code=103

dovetail
Site Admin
Posts: 1856
Joined: Thu Jul 29, 2004 12:12 pm

Re: Read z/OS files in Python on Linux from COZLAUNCH

Post by dovetail » Mon Mar 12, 2018 8:37 am

This isn't correct in Python:

print g.write();

If you try the same code using regular text files you would have the same problem

Post Reply