MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

REXX Code to read HTML data from a URL

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> TSO and ISPF
View previous topic :: View next topic  
Author Message
superk
Advanced


Joined: 19 Dec 2002
Posts: 684
Topics: 5

PostPosted: Tue Aug 02, 2005 7:30 am    Post subject: REXX Code to read HTML data from a URL Reply with quote

Good morning folks. I'm working on a potential automation project where I need to access and read HTML data from a URL. So far, I've been working on this using SAS to read and process the data from the URL. I'll include the SAS code here just in case anyone is interested:
Code:

//STEP0001 EXEC SAS                                                     
//SAS.SASLOG   DD SYSOUT=*                                             
//SAS.SYSDUMP  DD SYSOUT=*                                             
//SAS.SASLIST  DD SYSOUT=*                                             
//T1       DD   DSN=&SYSUID..T1,DISP=(,CATLG,DELETE),                   
//         UNIT=SYSDA,SPACE=(CYL,(1,1))                                                               
//SASUSER  DD   UNIT=VIO,SPACE=(TRK,(10,10),RLSE)                       
//SYSTCPD  DD   DSN=HLQ.TCPIP.SEZAINST(TCPDATB1),DISP=SHR           
//SYSIN    DD   DATA                                                   
 FILENAME READHTM1                                                     
   URL 'http://server:port/directory/file.html'; 
 DATA _NULL_;                                                           
    INFILE READHTM1 LENGTH=LEN;                                         
    INPUT RECORD $VARYING256. LEN;                                   
    FILE T1 NOPRINT NOTITLES LRECL=256;                               
    PUT RECORD $VARYING256. LEN;                                     
 RUN;         
/*
//*

I would like to, for the purposes of automation, write something similar in REXX using TCP/IP sockets. Would anyone have a code sample that they may have already developed?
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12358
Topics: 75
Location: San Jose

PostPosted: Tue Aug 02, 2005 8:01 am    Post subject: Reply with quote

superk,

Can you use ISPF Work station agent?

btw here is a snippet of REXX TCP/IP Sockets Source Code for IP look up. May be you can use this as basis and work on

http://www.tsotimes.com/articles/archive/fall03/rexxtcpipsocketssc.html

Also check this which discusses about parsing special characters

http://www.mainframeweek.com/journals/articles/0004/A+REXX+parsing+program

Hope this helps...

Cheers

Kolusu
_________________
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Cogito-Ergo-Sum
Advanced


Joined: 15 Dec 2002
Posts: 637
Topics: 43
Location: Bengaluru, INDIA

PostPosted: Tue Aug 02, 2005 8:49 am    Post subject: Reply with quote

Hi Kevin,
Can the HTML be in any way be converted to XML format at the client side? If yes, then I suggest, you try the SOAP route. SOAP as you would know, is a XML based protocol. Both COBOL and PL/I (Enterprise) support the XML parser. Of course, this involves a messaging service (HTTP or WebSphere MQ) too.

If no, then SOAP has now been enhanced to handle attachments. Therefore, you can have the client send a SOAP message with the HTML as the attachment. The rest of the design of course must consider the XML parser, messaging service, etc.

The only reason I am suggesting this, if you go the SOAP way, then your application will automatically be industry-standard compliant. SOAP seems to be the 'in thing' nowadays.
_________________
ALL opinions are welcome.

Debugging tip:
When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.
-- Sherlock Holmes.
Back to top
View user's profile Send private message
superk
Advanced


Joined: 19 Dec 2002
Posts: 684
Topics: 5

PostPosted: Tue Aug 02, 2005 9:14 am    Post subject: Reply with quote

kolusu wrote:
Can you use ISPF Work station agent?

No, this exec will be executed by the Automation started task within an MVS address space. That's why I'm looking into using sockets.

Cogito-Ergo-Sum, the content of this URL does happen to be XML data. I'm planning on parsing through the content until I find a set of start and end tags (<DSV>,</DSV>) and then checking the tags within these two for a set of values (SUCCESS/FAILED):
Code:

<DSV>
    <MAINFRAME>SUCCESS</MAINFRAME>
    <SUNGARD>SUCCESS</SUNGARD>
    ....
</DSV>

at which time, if the value for any test is FAILED, I will post an alert to the Command Center and open a problem ticket in our problem tracking system.
Back to top
View user's profile Send private message
Cogito-Ergo-Sum
Advanced


Joined: 15 Dec 2002
Posts: 637
Topics: 43
Location: Bengaluru, INDIA

PostPosted: Tue Aug 02, 2005 11:13 pm    Post subject: Reply with quote

I see.
How about the link for 'REXX TCP/IP Sockets Source Code for IP look up' that Kolusu posted? Did that help?
_________________
ALL opinions are welcome.

Debugging tip:
When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.
-- Sherlock Holmes.
Back to top
View user's profile Send private message
superk
Advanced


Joined: 19 Dec 2002
Posts: 684
Topics: 5

PostPosted: Wed Aug 03, 2005 9:14 pm    Post subject: Reply with quote

Yeah, I already have that document, as well as the IP Socket Programming Manual, and some others. The main problem occurs when I attempt to issue the "Socket 'Connect' " command to port 80 and the url. I'm always getting back a message that the specified hostname does not exist.

There must be a special process to connect to a url instead of an actual server (like we already do with the SMTP servers), so I'll keep digging for an answer. Meanwhile, the SAS solution is working quite well.

Thanks to everyone who has responded so far.
Back to top
View user's profile Send private message
vvallaturu
Beginner


Joined: 28 Feb 2005
Posts: 8
Topics: 2
Location: Hyderabad

PostPosted: Thu Aug 04, 2005 3:20 am    Post subject: I Could not understand the error message ?? Reply with quote

Hi all,

When I executed the REXX Edit macro, which is in the link posted before ( the edit macro for REXX TCP/IP Socket Connection), I got the below error message :

1004 EIBMIUCVERR IUCV error

I searched but could not find any details.
Can anyone please explain to me the error message and also guide me on how to rectify the same.

Thank you.

Looking forward to your response.
_________________
Thank you.

Venkata Krishna
MVS system programmer
Back to top
View user's profile Send private message Send e-mail AIM Address Yahoo Messenger
Cogito-Ergo-Sum
Advanced


Joined: 15 Dec 2002
Posts: 637
Topics: 43
Location: Bengaluru, INDIA

PostPosted: Thu Aug 04, 2005 3:52 am    Post subject: Reply with quote

Kevin,
Quote:
There must be a special process to connect to a url instead of an actual server (like we already do with the SMTP servers), so I'll keep digging for an answer.

Are you looking for the Gethostbyname command? This is from the TCP/IP Implementation Guide. Maybe this helps...
Code:
/* REXX */
/*------------------------------------------------------------------*/
/* */
/* Name: MISCLIEN - Miscellaneous client program */
/* */
/* Function: Tests the three following protocols available in */
/* the Miscellaneous Server using TCP protocol. */
/* ECHO protocol (PORT 7) */
/* DISCARD protocol (PORT 9) */
/* CHARACTER GENERATOR (PORT 19) */
/* */
/* Interface: - none - */
/* */
/* Logic: The REXX program sends the message ' Are you There ?'   */
/* to the MISCSERV Server over a socket connection. */
/* 1- When using ECHO protocol (tpiport=7), MISCSERV */
/* sends back the ' Are You There ?'   message. */
/* 2- When using DISCARD protocol (tpiport=9), MISCSERV */
/* throws away the ' Are You There ?'   message and */
/* send no response. */
/* 3- When using CHARACTER GENERATOR protocol */
/* (tpiport=19), MISCSERV sends back the */
/* Character Generator Data. */
/* */
/* Returncode: RC = 0, processing OK */
/* Everything else is non-successful return code */
/* from socket interface */
/* */
/* Written: June the 10' th 1994 at ITSO Raleigh */
/* */
/* Changes: */
/* */
/*------------------------------------------------------------------*/
parse upper arg tpiport
actrace = 1 /*Controls tracing */
record = ' Are You There ?' /*Message test */
tpiserver = ' mvs18' /*Server host name */
sockval = DoSocket(' Terminate' ) /*Ensure clean interface*/
if actrace then say ' Terminate returned; ' sockval
/*------------------------------------------------------------------*/
/* */
/* Initialize REXX socket interface */
/* and get our own TCP/IP Client id. */
/* */
/*------------------------------------------------------------------*/
sockval = DoSocket(' Initialize' ,   ' echorexx' )
if actrace then say ' Initialize returned: ' sockval
if sockrc <> 0 then do
say ' Socket initialize failed, rc=' sockrc
say sockval
exit(sockrc)
end
sockval = DoSocket(' Getclientid' )
if actrace then say ' Getclientid returned: ' sockval
if sockrc <> 0 then do
say ' Getclientid failed, rc=' sockrc
say sockval
exit(sockrc)
end
servipaddr = DoSocket(' Gethostbyname' ,   tpiserver)
if actrace then say ' Gethostbyname returned: ' servipaddr
if sockrc <> 0 then do
say ' Gethostbyname failed, rc=' sockrc
say sockval
x=Doclean
exit(sockrc)
end
/*------------------------------------------------------------------*/
/* */
/* Get a socket to use for connection to the MISCSERV Server */
/* address space. */
/* */
/*------------------------------------------------------------------*/
sockdescr = DoSocket(' Socket' , ' AF_INET' , ' SOCK_STREAM' , ' TCP' )
if actrace then say ' Socket returned: ' sockdescr
if sockrc <> 0 then do
say ' Socket call for socket descriptor failed, rc=' sockrc
say sockval
x=Doclean
exit(sockrc)
end
/*------------------------------------------------------------------*/
/* */
/* Connect the socket to the MISCSERV Server */
/* */
/*------------------------------------------------------------------*/
name = ' AF_INET ' | | tpiport||'   ' | | servipaddr
sockval = DoSocket(' Connect' ,   sockdescr, name)
if actrace then say ' Connect returned: ' sockdescr
if sockrc <> 0 then do
say ' Connect failed, rc=' sockrc
say sockval
x=Doclean
exit(sockrc)
end
/*------------------------------------------------------------------*/
/* */
/* Send the message to the MISCSERV server */
/* */
/*------------------------------------------------------------------*/
sockval = DoSocket(' Write' ,   sockdescr, record)
if actrace then say ' Write returned: ' sockval
if sockrc <> 0 then do
say ' Write failed, rc=' sockrc
say sockval
x=Doclean
exit(sockrc)
end
/*------------------------------------------------------------------*/
/* */
/* Read the response from the MISCSERV Server */
/* */
/*------------------------------------------------------------------*/
if tpiport <> ' 9 '   then do
respdata = DoSocket(' Read' ,   sockdescr)
if actrace then say ' Read returned: ' respdata
if sockrc <> 0 then do
say ' Read failed, rc=' sockrc
say sockval
x=Doclean
exit(sockrc)
end
end
/*------------------------------------------------------------------*/
/* */
/* Close the socket */
/* */
/*------------------------------------------------------------------*/
parse value respdata with resplen response
sockval = DoSocket(' Close' ,   sockdescr)
if actrace then say ' Close returned: ' sockval
if sockrc <> 0 then do
say ' Socket Close failed, rc=' sockrc
say sockval
x=Doclean
exit(sockrc)
end
/*------------------------------------------------------------------*/
/* */
/* Terminate socket interface */
/* */
/*------------------------------------------------------------------*/
sockval = DoSocket(' Terminate' )
if actrace then say ' Terminate returned; ' sockval
if sockrc <> 0 then do
say ' Socket Close failed, rc=' sockrc
say sockval
exit(sockrc)
end
Exit(0)
/*------------------------------------------------------------------*/
/* */
/* Doclean Procedure. */
/* */
/* If a socket call failed, and we are about to exit this */
/* Rexx application, close the socket and terminate the */
/* socket interface. */
/* */
/*------------------------------------------------------------------*/
Doclean:
sockval = DoSocket(' Close' ,   sockdescr)
sockval = DoSocket(' Terminate' )
return sockres
/*------------------------------------------------------------------*/
/* */
/* DoSocket procedure. */
/* */
/* Do the actual socket call, and parse the return code. */
/* Return rest of string returned from socket call. */
/* */
/*------------------------------------------------------------------*/
DoSocket:
a0 = arg(1)
a1 = arg(2)
a2 = arg(3)
a3 = arg(4)
a4 = arg(5)
a5 = arg(6)
a6 = arg(7)
a7 = arg(8)
a8 = arg(9)
a9 = arg(10)
if actrace then do
say ' Socket Parms=' a0 a1 a2 a3 a4 a5 a6 a7 a8 a9
end
else do
msgstat = msg()
z = msg("OFF")
end
Parse value ' SOCKET' ( a0,a1,a2,a3,a4,a5,a6,a7,a8,a9) with sockrc sockres
if actrace then do
say ' Socket return code=' sockrc
end
else do
z = msg(msgstat)
end
return sockres


(I hope the formatting is OK. You might see some funny characters in the above code. That is because, I copy-pasted from the PDF to notepad and pasted it here. I have changed some of them to meaningful characters.)
_________________
ALL opinions are welcome.

Debugging tip:
When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.
-- Sherlock Holmes.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> TSO and ISPF All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group