How Java Handles Data Returned from WebService/RESTful
The XML/JSON returned from WebService/RESTful is multilevel data, which is easy to parse yet difficult to compute. The popular approach is to parse XML/JSON using an open-source class library, such as Dom4J and JsonPath, and then compute data with XPath or JsonPath. The approach is lightweight and integration-friendly, but it can manage conditional queries only and hardcodes all the other computations. If high computing performance is required, we can store XML/JSON in a small database, such as SQLite and MySQL, and parse data using the corresponding special function and perform computations in regular SQL. This involves a heavy framework and a complicated and time-consuming loading process. Moreover, SQL performs far better in computing structured records than it does in handling multilevel data.
An ideal alternative is esProc SPL, an open-source class library. It has a lightweight integration framework for parsing data returned from WebService/RESTful directly and rich functions that enable capabilities for computing multilevel data.
SPL’s JDBC driver has a simple framework and is easy to learn. A Java main program receives JSON strings sent from RESTful and integrates SPL, for instance, and we can parse the JSON into ResultSet using the code below for further processing:
…
String jsonStr=… //JSON string
Class.forName("com.esproc.jdbc.InternalDriver");
Connection connection =DriverManager.getConnection("jdbc:esproc:local://");
Statement statement = connection.createStatement();
String query="=json($["+jsonStr+"])";
ResultSet result = statement.executeQuery(query);
…
SPL has the built-in multilevel data object for computing and generating JSON/XML conveniently, making it particularly suitable for processing data returned from WebService/RESTful. After all, computing instead of parsing is our ultimate target. Let’s look at an example. The JSON returned from RESTful has two levels. The first level records employee records. In each record, the Orders field contains orders records. We are trying to parse the JSON, perform a conditional query on all orders, and output the result as JSON strings. SPL does this with the following code:
=json(json($\["+jsonStr+"\]).conj(Orders)
.select((Amount>1000 && Amount<=2000) && like@c(Client,"\*business\*")))
SPL handles XML in the same way.
SPL offers simple and easy-to-use receive functions to obtain data from WebService/RESTful directly. This gets rid of complex receiving processes from the Java main program and reduces use of the third-party class libraries, producing a lighter integration framework. To receive data returned from RESTful, perform a conditional query, and convert the result into JSON, for instance, we have the following SPL code:
=json(json(httpfile("http://127.0.0.1:6868/api/getData").read()).conj(Orders)
.select((Amount>1000 && Amount<=2000) && like@c(Client,"*business*")))
SPL can access servers with access controls, too. You can refer to relative documentation to learn more details.
The SPL code can be stored separately and independently, substantially decoupling it from the Java code and making it suited to handling complicated computations or computations that probably undergo frequent changes. The above code for achieving conditional query, for instance, can be saved as an SPL script file:
A |
B |
|
1 |
=json(parameter_jsonStr) |
/Multilevel JSON |
2 |
=A1.conj(Orders) |
/Concatenate orders records |
3 |
=A2.select((Amount>1000 && Amount<=2000) && like@c(Client,"*business*")) |
/Perform conditional query |
4 |
return json(A3) |
Then we can call the script file in JDBC as we call a stored procedure:
…
String jsonStr=… //JSONstrings
Class.forName("com.esproc.jdbc.InternalDriver");
Connection conn =DriverManager.getConnection("jdbc:esproc:local://");
CallableStatement statement = conn.prepareCall("{call getQuery(?)}");
statement.setObject(1, jsonStr);
ResultSet result statement.execute();
result.next();
String jsonResult=result.getString(1);
…
SPL provides universal syntax to compute data coming from various sources in a consistent way. To perform a conditional query on employees-orders data originated from WebService and return the result as XML, for instance, XPath and JsonPath cannot be used at the same time, and SQLite does not support handling XML (but it can handle JSON) while SPL can achieve the computation using the same syntax, as shown below:
A |
B |
|
1 |
=ws_call(ws_client("http://.../entityWS.asmx?wsdl"),"entityWS ":" entityWSSoap":"getData") |
/Receive data from WebService |
2 |
=A1.conj(Orders) |
/Concatenate orders records |
3 |
=A2.select((Amount>1000 && Amount<=2000) && like@c(Client,"*business*")) |
/Perform conditional query |
4 |
return xml(A3) |
/Convert result to XML |
SPL supports various data sources, including relational databases, all types of NoSQL databases, CSV, Excel, JSON files, and XML files, and handles them using the same syntax. Here we just skip detailed explanation.
SPL has a wealth of functions that provide computational capabilities as good as SQL. Below are more examples:
A |
B |
|
1 |
…. |
/Multilevel data parsing |
2 |
=A1.conj(Orders).groups(Client;sum(Amount)) |
/Grouping & aggregation |
3 |
=A1.groups(State,Gender;avg(Salary),count(1)) |
/ Grouping & aggregation by multiple fields |
4 |
=A1.sort(Salary) |
/Sorting |
5 |
=A1.id(State) |
/Distinct |
6 |
=A1.new(Name,Gender,Dept,Orders.OrderID,Orders.Client,Orders.Client, Orders.SellerId,Orders.Amount,Orders.OrderDate |
/Join |
SPL has agile syntax to effortlessly express algorithms with complex logic that are hard to achieve using SQL or stored procedure with rather simple code. Suppose are trying to find the first n big clients whose amounts takes up at least half of the total and sort them by amount in descending order:
A |
B |
|
1 |
… |
/Parse data |
2 |
=A1.sort(amount:-1) |
/Sort data by amount in descending order |
3 |
=A2.cumulate(amount) |
/Get the sequence of cumulative amounts |
4 |
=A3.m(-1)/2 |
/The final cumulative amount is the total |
5 |
=A3.pselect(~>=A4) |
/Get the position of record where the cumulative amount reaches at least half of the total |
6 |
=A2(to(A5)) |
/Get records according to A4’s position |
SPL has a professional IDE equipped with all-around debugging functionalities and letting users to observe the result of each step, increasing efficiency of coding algorithms with complex logics.
So, with simple and light integration framework, built-in multilevel data object, rich functions and agile syntax, SPL is naturally suitable for processing data returned from WebService and RESTful.
SPL Official Website 👉 https://www.scudata.com
SPL Feedback and Help 👉 https://www.reddit.com/r/esProcSPL
SPL Learning Material 👉 https://c.scudata.com
SPL Source Code and Package 👉 https://github.com/SPLWare/esProc
Discord 👉 https://discord.gg/cFTcUNs7
Youtube 👉 https://www.youtube.com/@esProc_SPL
Chinese version