The SPL provides the httpfile function to access webpages, WebService and Restful services on HTTP server, passing parameters required by the service to get the return data or download files.

The full syntax of the httpfile function is as follows:

httpfile(url:cs, post:cs; header:value,....)

The function parameter url is the webpage address to be accessed, cs is the charset of return data, and default is the specified charSet in raqsoftConfig.xml, which is the configuration file of SPL.

post is a parameter string that is passed as a POST to access the url whose format is param1=value1&param2=value2…… or JSON. The following cs is the charset for this parameter string, which defaults to UTF-8.

header is the property name of RequestHeader, and value is its value. Multiple request header properties can be passed.

 

1. access without parameter

For example,to visit the home page of RaqForum for contents requires the SPL script that reads as follows:

	A	B
1	=httpfile("http://c.raqsoft.com.cn")	//create   a file object of http
2	=A1.read()	//read   the file contents of A1

If the value of charSet in the SPL configuration file is GBK and the charset of the RaqForum home page is UTF-8, the value of cell A2 after executing the script will be messy code, so the SPL script should be modified as follows:

	A	B
1	=httpfile("http://c.raqsoft.com.cn":"UTF-8")	//create   a file object of http, specify that the return content is encoded as UTF-8
2	=A1.read()	//read   the file contents of A1

 

2. Download files

For example, to download the SPL installation pack writes the SPL script as follows:

	A	B
1	=httpfile("http://download.raqsoft.com.cn/esproc/esProc-install-20210811.zip")	//create   a file object of http
2	=file("d:/esProc-install-20210811.zip").write@b(A1.read@b())

read@b()in cell A2 reads the binary data of the file to be downloaded, and the binary data is written to file d:/esProc-install-20210811.zip with write@b(). Note: be sure to add the option @b for reading and writing when downloading files. When the file is large, the downloading time will be longer.

 

3. Pass accessing parameters

Some services also need to pass some parameters when accessing. There are two methods to pass parameters: GET and POST. Some services require only GET method, some only POST method, and some both. Choose which method to use according to the requirements of the server.

 

3.1. GET method

Directly concatenate the name and value of the parameter in the url, link the parameter to the url with a question mark, and link various parameters with &, which looks like this:

urladdress?p1=v1&p2=v2&p3=v3……

Passing parameters with GET method will be subject to the entire length of the URL string. Different types of servers have different restrictions to the lengths, and generally no more than 512 is appropriate.

	A	B
1	=httpfile("http://www.test.com/query?user=sjr&year=2020")	//pass   the parameters in GET

 

3.2. Encoding

URL transcoding is required for parameter values that have special characters, and SPL provides the urlencode function to encode the parameter, for example:

	A	B
1	=httpfile("http://www.test.com/query?user="+urlencode("Miker     Jackson","utf-8")+"&year=2020")	//The   value of user parameter has a space in it and needs to be converted

When the value contains letters, numbers and characters other than “.”, “-”, “*” and “_”, URL encoding is required.

 

3.3. POST method

In POST method, the part after the question mark of url string in GET method is submitted as the second parameter to the httpfile function. When sending with POST, there is no length limitation. For example:

	A	B
1	=httpfile("http://www.test.com/query","user=sjr&year=2020")	//parameter   values have no special characters and do not require encoding conversion
2	=httpfile("http://www.test.com/query","user="+urlencode("Miker   Jackson","utf-8")+"&year=2020")	//the   user parameter has a space in it and needs to be converted
3	=httpfile("http://www.test2.com/query","user="+urlencode("张三","GBK")+"&year=2020":"GBK")	//the   server requires the parameters in GBK charset, and specify the charset after   the post parameter string

 

3.4. JSON method

It is essentially the POST method, except that the submitted parameter string is in JSON format. And that the property value of Content-Type is application/json needs to be specified in the requestHeader property. For example:

	A
1	=httpfile("http://www.test.com/rtdb/access","{\"nodeIds\":[\"ns=102;s=AI_002_0600.PV\"],"startTime\":\"2018/6/28   13:10:00\",\"endTime\":\"2018/6/28 13:11:0\",   \"returnBounds\":\"false\",   \"maxSizePerNode\":\"10\"}";   "Content-Type":"application/json")

 

4. Pass the properties of RequestHeader

When accessing some services, parameters are not enough; the properties need be added in the RequestHeader. These properties can be set after the semicolon in the httpfile function parameter, such as the Content-Type specified when passing the parameter in JSON method in the previous example, and the Cookie properties specified in a later example.

 

5. Request the properties of ResponseHeader

In addition to returning page data, the server sometimes returns properties in the ResponseHeader. In the SPL script, after reading the data returned by the server through httpfile.read(), the value of propName property in ResponseHeader can be accessed through the httpfile.property(propName) function. For more information, see the Set-Cookie properties in the example of next section.

 

6. Access control

For data security, some servers authenticate the identity of the accessor, and the page data can only be read through the authenticated access. There are two types of authentication, one is that after the user accesses the authentication page, the server records the authenticated information in the Session and sends the Session number back to the Cookie at the client, or send the authenticated information back to the Cookie at the client as well. When accessing page data that has access control, the content stored in the Cookie needs to be placed in the request header, then the server can identify the accessor and decides whether to allow access to the page data. The other is that after the user visits the authentication page, the server returns an access Token, which is passed back as a parameter during its period of validity when the user visits the page data.

How do we complete the authentication steps in SPL and access data that have access control?

 

6.1. The server saving authentication as a Session or Cookie

RaqForum has a column of internal posts. Only those authenticated as the internal staff can read the contents.

	A
1	=httpfile("http://c.raqsoft.com.cn/article/1628656263716")
2	=A1.read()

If we read the post content like the above, what cell A2 returns is “no access”, not the actual content of the page. The following SPL script reads after authentication: 

	A
1	=httpfile("http://c.raqsoft.com.cn/login4get?nameOrEmail=tom&userPassword="+md5("mypass")+"&rememberLogin=true")
2	=A1.read()
3	=A1.property("Set-Cookie")
4	=httpfile("http://c.raqsoft.com.cn/article/1628656263716";"Cookie":A3)
5	=A4.read()

A1 Define the httpfile object to access the RaqForum login page. This server requires the password being a MD5-encrypted string of the user’s original password

A2 Read the returned content from the authentication page to complete the authentication. The returned content usually indicates whether the authentication was successful

A3 Read the Set-Cookie properties from the response header of this authentication request, which will be written into the Cookie of the client

A4 Define the httpfile object that accesses the internal page, and put the content of A3 in the Cookie of the request header

A5 Read the content of internal page and what returns is the actual content of the page

 

Accessors need to know the login interface of the server, the data service vendor will explain it in related documents. In this case, it is submitted in GET method, and some servers require POST method or only JSON format submission.

 

6.2. The server returning Token

Here instead of specific examples, the general process is as follows:

	A
1	=httpfile("https://xxxxxx","{\"userId\":\"abc\",\"password\":\"sdfikje87kd908\"}";"Content-Type":"application/json")
2	=A1.read()
3	=json(A2).accessToken
4	=httpfile("https://xxxxxx","{\"accessToken\":\""+A3+"\",\"other\":\"xxx\"}";"Content-Type":"application/json")
5	=A4.read()

A1 Define the httpfile object to access the login interface page. In the example, the user name, password, or other required parameters are submitted in JSON format. How it is actually submitted depends on the server’s requirements

A2 Read the returned content from the authentication page to complete the authentication. The returned content generally indicates whether the authentication was successful, and the returned content will contain information about accessToken and the duration of validity after succeeding

A3 If the returned content is in JSON format, convert the content into a JSON object and get the value of accessToken

A4 Define the httpfile object that accesses the page with access control. Assuming that the server requires parameters in JSON format, the content of A3 is passed as the accessToken parameter value, along with other required parameters

A5 Read the data that needs to be accessed