REXX Code to read HTML data from a URL
Select messages from
# through # FAQ
[/[Print]\]

MVSFORUMS.com -> TSO and ISPF

#1: REXX Code to read HTML data from a URL Author: superk PostPosted: Tue Aug 02, 2005 7:30 am
    —
Good morning folks. I'm working on a potential automation project where I need to access and read HTML data from a URL. So far, I've been working on this using SAS to read and process the data from the URL. I'll include the SAS code here just in case anyone is interested:
Code:

//STEP0001 EXEC SAS                                                     
//SAS.SASLOG   DD SYSOUT=*                                             
//SAS.SYSDUMP  DD SYSOUT=*                                             
//SAS.SASLIST  DD SYSOUT=*                                             
//T1       DD   DSN=&SYSUID..T1,DISP=(,CATLG,DELETE),                   
//         UNIT=SYSDA,SPACE=(CYL,(1,1))                                                               
//SASUSER  DD   UNIT=VIO,SPACE=(TRK,(10,10),RLSE)                       
//SYSTCPD  DD   DSN=HLQ.TCPIP.SEZAINST(TCPDATB1),DISP=SHR           
//SYSIN    DD   DATA                                                   
 FILENAME READHTM1                                                     
   URL 'http://server:port/directory/file.html'; 
 DATA _NULL_;                                                           
    INFILE READHTM1 LENGTH=LEN;                                         
    INPUT RECORD $VARYING256. LEN;                                   
    FILE T1 NOPRINT NOTITLES LRECL=256;                               
    PUT RECORD $VARYING256. LEN;                                     
 RUN;         
/*
//*

I would like to, for the purposes of automation, write something similar in REXX using TCP/IP sockets. Would anyone have a code sample that they may have already developed?

#2:  Author: kolusuLocation: San Jose PostPosted: Tue Aug 02, 2005 8:01 am
    —
superk,

Can you use ISPF Work station agent?

btw here is a snippet of REXX TCP/IP Sockets Source Code for IP look up. May be you can use this as basis and work on

http://www.tsotimes.com/articles/archive/fall03/rexxtcpipsocketssc.html

Also check this which discusses about parsing special characters

http://www.mainframeweek.com/journals/articles/0004/A+REXX+parsing+program

Hope this helps...

Cheers

Kolusu

#3:  Author: Cogito-Ergo-SumLocation: Bengaluru, INDIA PostPosted: Tue Aug 02, 2005 8:49 am
    —
Hi Kevin,
Can the HTML be in any way be converted to XML format at the client side? If yes, then I suggest, you try the SOAP route. SOAP as you would know, is a XML based protocol. Both COBOL and PL/I (Enterprise) support the XML parser. Of course, this involves a messaging service (HTTP or WebSphere MQ) too.

If no, then SOAP has now been enhanced to handle attachments. Therefore, you can have the client send a SOAP message with the HTML as the attachment. The rest of the design of course must consider the XML parser, messaging service, etc.

The only reason I am suggesting this, if you go the SOAP way, then your application will automatically be industry-standard compliant. SOAP seems to be the 'in thing' nowadays.

#4:  Author: superk PostPosted: Tue Aug 02, 2005 9:14 am
    —
kolusu wrote:
Can you use ISPF Work station agent?

No, this exec will be executed by the Automation started task within an MVS address space. That's why I'm looking into using sockets.

Cogito-Ergo-Sum, the content of this URL does happen to be XML data. I'm planning on parsing through the content until I find a set of start and end tags (<DSV>,</DSV>) and then checking the tags within these two for a set of values (SUCCESS/FAILED):
Code:

<DSV>
    <MAINFRAME>SUCCESS</MAINFRAME>
    <SUNGARD>SUCCESS</SUNGARD>
    ....
</DSV>

at which time, if the value for any test is FAILED, I will post an alert to the Command Center and open a problem ticket in our problem tracking system.

#5:  Author: Cogito-Ergo-SumLocation: Bengaluru, INDIA PostPosted: Tue Aug 02, 2005 11:13 pm
    —
I see.
How about the link for 'REXX TCP/IP Sockets Source Code for IP look up' that Kolusu posted? Did that help?

#6:  Author: superk PostPosted: Wed Aug 03, 2005 9:14 pm
    —
Yeah, I already have that document, as well as the IP Socket Programming Manual, and some others. The main problem occurs when I attempt to issue the "Socket 'Connect' " command to port 80 and the url. I'm always getting back a message that the specified hostname does not exist.

There must be a special process to connect to a url instead of an actual server (like we already do with the SMTP servers), so I'll keep digging for an answer. Meanwhile, the SAS solution is working quite well.

Thanks to everyone who has responded so far.

#7: I Could not understand the error message ?? Author: vvallaturuLocation: Hyderabad PostPosted: Thu Aug 04, 2005 3:20 am
    —
Hi all,

When I executed the REXX Edit macro, which is in the link posted before ( the edit macro for REXX TCP/IP Socket Connection), I got the below error message :

1004 EIBMIUCVERR IUCV error

I searched but could not find any details.
Can anyone please explain to me the error message and also guide me on how to rectify the same.

Thank you.

Looking forward to your response.

#8:  Author: Cogito-Ergo-SumLocation: Bengaluru, INDIA PostPosted: Thu Aug 04, 2005 3:52 am
    —
Kevin,
Quote:
There must be a special process to connect to a url instead of an actual server (like we already do with the SMTP servers), so I'll keep digging for an answer.

Are you looking for the Gethostbyname command? This is from the TCP/IP Implementation Guide. Maybe this helps...
Code:
/* REXX */
/*------------------------------------------------------------------*/
/* */
/* Name: MISCLIEN - Miscellaneous client program */
/* */
/* Function: Tests the three following protocols available in */
/* the Miscellaneous Server using TCP protocol. */
/* ECHO protocol (PORT 7) */
/* DISCARD protocol (PORT 9) */
/* CHARACTER GENERATOR (PORT 19) */
/* */
/* Interface: - none - */
/* */
/* Logic: The REXX program sends the message ' Are you There ?'   */
/* to the MISCSERV Server over a socket connection. */
/* 1- When using ECHO protocol (tpiport=7), MISCSERV */
/* sends back the ' Are You There ?'   message. */
/* 2- When using DISCARD protocol (tpiport=9), MISCSERV */
/* throws away the ' Are You There ?'   message and */
/* send no response. */
/* 3- When using CHARACTER GENERATOR protocol */
/* (tpiport=19), MISCSERV sends back the */
/* Character Generator Data. */
/* */
/* Returncode: RC = 0, processing OK */
/* Everything else is non-successful return code */
/* from socket interface */
/* */
/* Written: June the 10' th 1994 at ITSO Raleigh */
/* */
/* Changes: */
/* */
/*------------------------------------------------------------------*/
parse upper arg tpiport
actrace = 1 /*Controls tracing */
record = ' Are You There ?' /*Message test */
tpiserver = ' mvs18' /*Server host name */
sockval = DoSocket(' Terminate' ) /*Ensure clean interface*/
if actrace then say ' Terminate returned; ' sockval
/*------------------------------------------------------------------*/
/* */
/* Initialize REXX socket interface */
/* and get our own TCP/IP Client id. */
/* */
/*------------------------------------------------------------------*/
sockval = DoSocket(' Initialize' ,   ' echorexx' )
if actrace then say ' Initialize returned: ' sockval
if sockrc <> 0 then do
say ' Socket initialize failed, rc=' sockrc
say sockval
exit(sockrc)
end
sockval = DoSocket(' Getclientid' )
if actrace then say ' Getclientid returned: ' sockval
if sockrc <> 0 then do
say ' Getclientid failed, rc=' sockrc
say sockval
exit(sockrc)
end
servipaddr = DoSocket(' Gethostbyname' ,   tpiserver)
if actrace then say ' Gethostbyname returned: ' servipaddr
if sockrc <> 0 then do
say ' Gethostbyname failed, rc=' sockrc
say sockval
x=Doclean
exit(sockrc)
end
/*------------------------------------------------------------------*/
/* */
/* Get a socket to use for connection to the MISCSERV Server */
/* address space. */
/* */
/*------------------------------------------------------------------*/
sockdescr = DoSocket(' Socket' , ' AF_INET' , ' SOCK_STREAM' , ' TCP' )
if actrace then say ' Socket returned: ' sockdescr
if sockrc <> 0 then do
say ' Socket call for socket descriptor failed, rc=' sockrc
say sockval
x=Doclean
exit(sockrc)
end
/*------------------------------------------------------------------*/
/* */
/* Connect the socket to the MISCSERV Server */
/* */
/*------------------------------------------------------------------*/
name = ' AF_INET ' | | tpiport||'   ' | | servipaddr
sockval = DoSocket(' Connect' ,   sockdescr, name)
if actrace then say ' Connect returned: ' sockdescr
if sockrc <> 0 then do
say ' Connect failed, rc=' sockrc
say sockval
x=Doclean
exit(sockrc)
end
/*------------------------------------------------------------------*/
/* */
/* Send the message to the MISCSERV server */
/* */
/*------------------------------------------------------------------*/
sockval = DoSocket(' Write' ,   sockdescr, record)
if actrace then say ' Write returned: ' sockval
if sockrc <> 0 then do
say ' Write failed, rc=' sockrc
say sockval
x=Doclean
exit(sockrc)
end
/*------------------------------------------------------------------*/
/* */
/* Read the response from the MISCSERV Server */
/* */
/*------------------------------------------------------------------*/
if tpiport <> ' 9 '   then do
respdata = DoSocket(' Read' ,   sockdescr)
if actrace then say ' Read returned: ' respdata
if sockrc <> 0 then do
say ' Read failed, rc=' sockrc
say sockval
x=Doclean
exit(sockrc)
end
end
/*------------------------------------------------------------------*/
/* */
/* Close the socket */
/* */
/*------------------------------------------------------------------*/
parse value respdata with resplen response
sockval = DoSocket(' Close' ,   sockdescr)
if actrace then say ' Close returned: ' sockval
if sockrc <> 0 then do
say ' Socket Close failed, rc=' sockrc
say sockval
x=Doclean
exit(sockrc)
end
/*------------------------------------------------------------------*/
/* */
/* Terminate socket interface */
/* */
/*------------------------------------------------------------------*/
sockval = DoSocket(' Terminate' )
if actrace then say ' Terminate returned; ' sockval
if sockrc <> 0 then do
say ' Socket Close failed, rc=' sockrc
say sockval
exit(sockrc)
end
Exit(0)
/*------------------------------------------------------------------*/
/* */
/* Doclean Procedure. */
/* */
/* If a socket call failed, and we are about to exit this */
/* Rexx application, close the socket and terminate the */
/* socket interface. */
/* */
/*------------------------------------------------------------------*/
Doclean:
sockval = DoSocket(' Close' ,   sockdescr)
sockval = DoSocket(' Terminate' )
return sockres
/*------------------------------------------------------------------*/
/* */
/* DoSocket procedure. */
/* */
/* Do the actual socket call, and parse the return code. */
/* Return rest of string returned from socket call. */
/* */
/*------------------------------------------------------------------*/
DoSocket:
a0 = arg(1)
a1 = arg(2)
a2 = arg(3)
a3 = arg(4)
a4 = arg(5)
a5 = arg(6)
a6 = arg(7)
a7 = arg(8)
a8 = arg(9)
a9 = arg(10)
if actrace then do
say ' Socket Parms=' a0 a1 a2 a3 a4 a5 a6 a7 a8 a9
end
else do
msgstat = msg()
z = msg("OFF")
end
Parse value ' SOCKET' ( a0,a1,a2,a3,a4,a5,a6,a7,a8,a9) with sockrc sockres
if actrace then do
say ' Socket return code=' sockrc
end
else do
z = msg(msgstat)
end
return sockres


(I hope the formatting is OK. You might see some funny characters in the above code. That is because, I copy-pasted from the PDF to notepad and pasted it here. I have changed some of them to meaningful characters.)



MVSFORUMS.com -> TSO and ISPF


output generated using printer-friendly topic mod. All times are GMT - 5 Hours

Page 1 of 1

Powered by phpBB © 2001, 2005 phpBB Group