Deploy and Use RSRV in esProc Enterprise Edition

 

esProc Enterprise Edition provides the Remote Server (abbreviated as RSRV). The feature enables users to upload/download files to/from a remote server or the remote storage, or modify and debug a SPL script on the remote server.

You need to deploy the Remote Server (RSRV) application before using the remote server. esProc Enterprise Edition has the QVS deployed in its built-in Tomcat, so you can directly use it after installing esProc in your remote machines. You can also deploy the RSRV in an independent application.

Quick start

esProc Enterprise edition offers a complete set of RSRV project in its installer. The path isesproc installation directory\esProc\q-server\webapps\qvs. Just install esProc Enterprise Edition in the local and remote machines to log on a remote server quickly through the RSRV application.

The following example will walk you through the how to use the RSRV.

There is an employee table that records employee information in database mydemoon remote server 192.168.124.13. We want to retrieve information of employees whose salaries are above 8,000 per month from the employee table on a local machine using a script. The script is stored on the remote server for the convenience of later debugging.

Step 1: Edit a script locally

Edit a script on the local esProc as required:

..

Save the script on the local machine as D:\tmp\test1.splx.

Step 2: Configure IP address and port number

Configure the IP address and the port number in esproc installation directory\esProc\q-server\conf\server.xml on the remote server:

<?xml version="1.0" encoding="UTF-8"?>
<Server port="8041" shutdown="SHUTDOWN">
 …
  <Service name="Catalina">
    <Connector port="8090" protocol="HTTP/1.1"
               connectionTimeout="0"
               />
    <Engine name="Catalina" defaultHost="localhost">
     …
<Host name="192.168.124.13"  appBase="webapps"
            unpackWARs="true"  autoDeploy="false" 
			deployXML ="false" deployOnStartup ="false" >
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log" suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />
               
              <Context path="/qvs" docBase="qvs" reloadable="true"  privileged="true"> </Context>
            
      </Host>
    </Engine>
  </Service>
</Server>

Step 3: Configure data source

Modify configuration file raqsoftConfig.xml:

Configure the data source inesProc installation directory\esProc\q-server\webapps\qvs\WEB-INF\home\raqsoftConfig.xmlon the remote server:


<DBList>
 <DB name="mydemo">
   <property name="url" value="jdbc:hsqldb:hsql://127.0.0.1/demo" ></property> 
   <property name="driver" value="org.hsqldb.jdbcDriver" ></property> 
   <property name="type" value="13" ></property> 
   <property name="user" value="sa" ></property> 
   <property name="password" ></property> 
   <property name="batchSize" value="1000" ></property>
   <property name="autoConnect" value="true" ></property>
   <property name="useSchema" value="false" ></property>
   <property name="addTilde" value="false" ></property>
   <property name="dbCharset" value="UTF-8" ></property>
   <property name="clientCharset" value="UTF-8" ></property>
   <property name="needTransContent" value="false" ></property>
   <property name="needTransSentence" value="false" ></property>
   <property name="caseSentence" value="false" ></property>
 </DB>
</DBList>

Note: The path of the driver to be automatically loaded for the RSRV application isinstallation root directory\common\jdbc. In the directory there are drivers for various commonly used databases, such as HSQL,Oracle,MySQLand SQL Server. To integrate and deploy the RSRV to a certain database application, put the corresponding database driver in a class path that can be loaded.

If you need to configure the other databases, just do as the first two steps show.


Step 4: Start the database

Execute startDataBase.bat(startDataBase.shfor Linux) in esProc installation directory\esProc\binon the remote server.

Step 5: Start the RSRV

Start RSRV through the starup file or through the Start menu.

The starup file is StartRSRV.bat (StartRSRV.shfor Linux)under esProc installation directory\esProc \bin.

Step 6: Log in RSRV

Open esProc IDE and clickRSRV -> Log inon the menu bar:

..


..

* Server Name: the user-defined server name; namesake servers are not allowed;

* URL: “http:// 192.168.124.13:8090/qvs”;

* Verification string: The string for verifying RSRV connection; default is "demoqvs". Learn more in Quick Integration – Step 4: Configuring RSRV.

Enter all the information correctly and you can log in and access the RSRV.

Then the logged in RSRV resource myRSRV is displayed under File Resource on the left part of the IDE. Under myRSRV, there are RSRV main directory and remote storage directory. As no remote storage address is configured for the RSRV, only the main directory is displayed. To use the remote storage service, see Quick Integration – Step 4: Configuring RSRV.

The main directory is by defaultesProc installation directory\esProc\q-server\webapps\qvs\WEB-INF\home\splx. To change the main directory, do as Quick Integration – Step 3: Set QVS RSRV main directoryexplains.

..

Step 7: Debug script remotely

1.Upload the local script test1.splxonto main directory of the RSRV 192.168.124.13.

Right-click the main directory to select “Upload File”:

..

Or Select RSRV -> Upload Fileon the menu bar:

..

Click “Select Node” and then select the “Main path”:

..

Select D:\tmp\test1.splx and click “Upload File”. Now you get the following information after the upload is finished:

..

2.Execute the remote script.

Double-click script test1.splx in the RSRV’s main directory to open and execute it.

..

Actually, the script is now executed directly on the remote server, rather than being downloaded to the local for execution.

3.Debug the remote script

Add a condition to the script: find the number of employees whose salaries are above 8,000 in each department. Now change the script test1.splxas follows:

..

Click “Save” to synchronously update the modified script in the remote server.

Quick Integration

The above is how to use the RSRV deployed in the esProc built-in Tomcat. Now look at how to deploy the RSRV in your independent applications and the remote storage directory.

Step 1: Integrate application directory

The RSRV application’s path is esProc installation directory\esProc\q-server\webapps\qvs. Integrate all contents below into the WEB application.

....

There are commons-fileupload-1.4.jar and commons-io-2.11.0.jar under the \WEB-INF\libdirectory in the integrated RSRV application. You also need to put jars in the directories below to the lib directory:

esProc installation directory\esProc\lib;
esProc installation directory\esProc\q-extlib, which contains the third-party jars of various remote storage platforms. You just get the corresponding jars for the remote storage service used. In this step you should put the jar in s3directory to the RSRV application’s libdirectory.

Note:

1. To connect to the database, see Quick Start - Configure data source.

2. All files can be directly copied except for web.xml. You need to copy the content of QVS’s web.xmlto the web.xmlto be integrated. The order of the content should be consistent with the rule of related application server.

Step 2: Set IP address and port number

Step 3: Set RSRV main directory

Set the RSRV main directory inproject directory\webapps\qvs\WEB-INF\home\raqsoftConfig.xml; the default main directory isproject directory\webapps\qvs\WEB-INF\home\splx.

<?xml version="1.0" encoding="UTF-8"?>
<Config Version="3">
    <Runtime>
		…
			<!-- Set RSRV main directory -->
            <mainPath>splx</mainPath>
        …
    </Runtime>
</Config>

Step 4: Configure RSRV

	<?xml version="1.0" encoding="UTF-8"?>
	<qvs>
		…
			<verifies>
		<!-- code:Verification string-->
                 <verify code="demoqvs">
<!-- Remote storage configurations: “store” is debugger’s remote storage information; “type” specifies storage service provider type (S3, GCS, NFS, Azure, and HDFS). The content between tags is RSRV’s remote storage user information. The example uses S3’s user information format; need to configure as user’s own remote storage address. -->
                 <store type="S3">
{"region":"cn-north-2","accessKey":"AKEETAZQDCXFENKIIIXO","secretKey":"aYI3JBZOuRGkU3bck2H6u4W5O3FvNhjDhoVQU2yN","endPoint":"https://s3.cn-north-2.amazonaws.com.cn"}
		 		</store>
</verify>
            </verifies>
		… 
	

You can change the verification string and add remote storage address inproject directory\webapps\qvs\WEB-INF\home\qvsConfig.xml. The default verification string the RSRV application provides is "demoqvs".

Step 5: Start RSRV

Step 6: Log in RSRV

Open esProc IDE and click RSRV -> Log inon the menu bar.

..


* Server Name: the user-defined server name; namesake servers are not allowed;

* URL: http://[ip address]:[port number]/[projectname];

* Verification string: The string for verifying RSRV connection; there can be multiple verification strings.

Enter all the information correctly and click “Log in” to access the RSRV.

Then the logged in remote resource myRSRV is displayed under File Resource on the left part of the IDE. Under myRSRV, there are the RSRV main directory and the remote storage directory.

..

WEB scriptdebug

After finishing deploying the RSRV application, we can not only log in the remote server from the designer and debug a script file stored in its main directory, but use the “WEB script debug” functionality to debug a script file under the directory without logging int the remote server.

URL: http://[ip address]:[port number]/[project name]/index.jsp

..

Execute:Execute the program in the whole cellset;

Compute:Compute the selected cell only;

Save:Save the modified script file;

On the left part of the interface, directories and script files in the SPL main directory are automatically displayed. Click a file in the list, content of the script will be displayed in the cellset area, and you can modify the script. Cannot add or delete rows or columns; you can only save and execute the script, and check value of a cell after the computation is finished.

Application Directory

Look at content of the RSRV application directory and that of configuration files.

Content

The RSRV’s application path is esProc installation directory\esProc\q-server\webapps\qvs. Below is the content of the qvs application directory:

WEB-INF

Necessary directory, where configuration file and jars are held

home

log

Log directory

splx

Default RSRV main directory

qvsConfig.xml

RSRV configuration file

raqsoftConfig.xml

SPL configuration file

lib

Holds the jars referenced by the WEB application

web.xml

For configuring servlet

images

The directory needed for script debugging on the WEB page

js

The directory needed for script debugging on the WEB page

websplx

The directory needed for script debugging on the WEB page

Configuration files

qvsConfig.xml

Below is the content of RSRV configuration file qvsConfig.xml:

<?xml version="1.0" encoding="UTF-8"?>
<qvs>
	<runMode>1</runMode>
	<locale language="en" country=""/>
    <!—cacheSystem: Configurations of downloading user files from the remote storage service provider to RSRV.
                 cachePath: The directory recording file downloading information;
                 blockSize: Specify the size of each downloaded block;
                 minFreeSpace: The minimum free space for storing the downloaded file directory; when the available server disk space is less than the specified minimum free space, buffered files under the download directory will be automatically deleted. -->
	<cacheSystem cachePath="cache" blockSize="1024" minFreeSpace="2" />
	<verifies>
		<verify code="demoqvs">
        <!-- Remote storage configurations: “store” is debugger’s remote storage information; “type” specifies storage service provider type (S3,GCS, NFS, Azure and HDFS). The content between tags is RSRV’s remote storage user information. The example uses S3’s user information format; need to configure as user’s own remote storage address. -->
         <store type="S3">        {"region":"cn-north-2","accessKey":"AKEETAZQDCXFENKIIIXO","secretKey":"aYI3JBZOuRGkU3bck2H6u4W5O3FvNhjDhoVQU2yN","endPoint":"https://s3.cn-north-2.amazonaws.com.cn"}
		 </store>
		</verify>
	</verifies>
</qvs>

raqsoftConfig.xml

Below is the content of SPL configuration file raqsoftConfig.xml:

<?xml version="1.0" encoding="UTF-8"?>
<Config Version="3">
    <Runtime>
		<!-- Configure the data source the application requires; the related driver jar files should be put in a directory that can be loaded by the RSRV application-->
        <DBList>
            <DB name="mydemo">
            <property name="url" value="jdbc:hsqldb:hsql://127.0.0.1/demo" ></property> 
            <property name="driver" value="org.hsqldb.jdbcDriver" ></property> 
            <property name="type" value="13" ></property> 
            <property name="user" value="sa" ></property> 
            <property name="password" ></property> 
            <property name="batchSize" value="1000" ></property>
            <property name="autoConnect" value="true" ></property>
            <property name="useSchema" value="false" ></property>
            <property name="addTilde" value="false" ></property>
            <property name="dbCharset" value="UTF-8" ></property>
            <property name="clientCharset" value="UTF-8" ></property>
            <property name="needTransContent" value="false" ></property>
            <property name="needTransSentence" value="false" ></property>
            <property name="caseSentence" value="false" ></property>
         </DB>
        </DBList>
        <Esproc>
            <charSet>GBK</charSet>
            <splPathList>
                <splPath></splPath>
            </splPathList>
            <dateFormat>yyyy-MM-dd</dateFormat>
            <timeFormat>HH🇲🇲ss</timeFormat>
            <dateTimeFormat>yyyy-MM-dd HH🇲🇲ss</dateTimeFormat>
			<!—Set RSRV main directory, which will be displayed on the tree structured list on the left part of the IDE after the remote service is logged in; when it is a relative path, it is relative to raqsoftConfig.xml -->
            <mainPath>splx</mainPath>
            <tempPath>temp</tempPath>
            <bufSize>65536</bufSize>
            <parallelNum>16</parallelNum>
            <cursorParallelNum>16</cursorParallelNum>
            <blockSize>1048576</blockSize>
            <nullStrings>nan,null,n/a</nullStrings>
            <fetchCount>9999</fetchCount>
            <customFunctionFile></customFunctionFile>
			<!-- Set external library -->
			<extLibsPath>F:\raqsoft\esProc\extlib</extLibsPath>
            <importLibs>
                <lib>WebserviceCli</lib>
            </importLibs>
		</Esproc>
        <Logger>
            <Level>DEBUG</Level>
        </Logger>
    </Runtime>
    <JDBC>
    </JDBC>
</Config>

web.xml

Below is the content of web.xml for servlet configuration:

<?xml version="1.0" encoding="UTF-8"?>
<web-app version="3.0">
<servlet>
		<servlet-name>QVSServlet</servlet-name>
		<servlet-class>com.scudata.ecloud.server.QVSServlet</servlet-class>
	    <load-on-startup>1</load-on-startup>
	</servlet>
	<servlet>
		<servlet-name>DebuggerServlet</servlet-name>
		<servlet-class>com.scudata.ecloud.server.debug.DebuggerServlet</servlet-class>
	</servlet>
	<servlet>
    	<servlet-name>ideFileServlet</servlet-name>
    	<servlet-class>com.scudata.ecloud.server.debug.file.FileServlet</servlet-class>
    </servlet>
	<servlet>
    	<servlet-name>ideResumableUpDownServlet</servlet-name>
    	<servlet-class>com.scudata.ecloud.server.debug.file.stream.StreamServlet</servlet-class>
    </servlet>
     <servlet>
    	<servlet-name>qvmServlet</servlet-name>
    	<servlet-class>com.scudata.ecloud.server.vm.QVMServlet</servlet-class>
    	<load-on-startup>3</load-on-startup>
    </servlet>
    <servlet>
    	<servlet-name>qvmResumableServlet</servlet-name>
    	<servlet-class>com.scudata.ecloud.vm.common.ResumableStreamController</servlet-class>
    </servlet>
    <servlet>
    	<servlet-name>webSplxServlet</servlet-name>
    	<servlet-class>com.scudata.websplx.WebSplxServlet</servlet-class>
    </servlet>
    
    <servlet-mapping>
		<servlet-name>qvmResumableServlet</servlet-name>
		<url-pattern>/resumable/servlet</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>qvmServlet</servlet-name>
		<url-pattern>/asqvm</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>QVSServlet</servlet-name>
		<url-pattern>/qvs</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>DebuggerServlet</servlet-name>
		<url-pattern>/spl/debug</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
    	<servlet-name>ideFileServlet</servlet-name>
    	<url-pattern>/file/ide</url-pattern>
    </servlet-mapping>
    <servlet-mapping>
    	<servlet-name>ideResumableUpDownServlet</servlet-name>
    	<url-pattern>/file/resumable</url-pattern>
    </servlet-mapping>    
    <servlet-mapping>
    	<servlet-name>webSplxServlet</servlet-name>
    	<url-pattern>/websplx</url-pattern>
    </servlet-mapping>    
    
	<welcome-file-list>
		<welcome-file>index.jsp</welcome-file>
	</welcome-file-list>
</web-app>