SPL: order-related grouping

 

Sometimes the order of the data makes sense when grouping. We at times group the adjacent records that have the same field values or that meet certain conditions. For example, find out the nation that ranks in the first of consecutive Olympic gold medals, find out how many days at most that the closing price of a stock has been increased, and so on. This is where order-related grouping comes in.

 

1.  Grouping by consecutive the same values

When grouping an ordered set, a new group will be created when the values of the fields for grouping change.

 

[e.g. 1] According to the table of the Olympic medal tally, find out the nation with the most consecutive first places and its medal information. Some of the data are as follows:

Game

Nation

Gold

Silver

Copper

30

USA

46

29

29

30

China

38

27

23

30

UK

29

17

19

30

Russia

24

26

32

30

Korea

13

8

7

 

The option @o of A.group() function in SPL enables to create a new group when field values change.

The SPL script looks like this:


A

1

=T("Olympic.txt")

2

=A1.sort@z(GAME,GOLD,SILVER,COPPER)

3

=A2.group@o1(GAME)

4

=A3.group@o(NATION)

5

=A4.maxp(~.len())

A1: import the Olympic medal table.

A2: sort the Olympic Games and the number of medals (gold, silver, bronze) in descending order.

A3: select one of every Olympic Game, because order is the first one of each game.

A4: create new groups when nations change.

A5: select the group with the largest number of members, which is the group with the most consecutive gold medals.

 

2. Grouping by adjacent conditions

When an ordered set is grouped, a new group will be created when evaluation result of the grouping condition is true.

 

[e.g. 2] How many days at most are the closing prices of the Shanghai Composite Index in 2020 consecutively rise? (the rising of the first trading day index). Some of the data are as follows:

DATE

CLOSE

OPEN

VOLUME

AMOUNT

2020/01/02

3085.1976

3066.3357

292470208

3.27197122606E11

2020/01/03

3083.7858

3089.022

261496667

2.89991708382E11

2020/01/06

3083.4083

3070.9088

312575842

3.31182549906E11

2020/01/07

3104.8015

3085.4882

276583111

2.88159227657E11

2020/01/08

3066.8925

3094.2389

297872553

3.06517394459E11

 

The option @i of the A.group() function in SPL enables to create a new group when conditions change.

The SPL script looks like this:


A

1

=T("SSEC.csv")

2

=A1.select(year(DATE)==2020).sort(DATE)

3

=A2.group@i(CLOSE<CLOSE[-1])

4

=A3.max(~.len())

A1: import the Shanghai Composite Index table.

A2: select the records of 2020 and sort them in ascending order of date.

A3: create a new group when the closing price is less than the closing price of the previous day.

A4: calculate the maximum number of days with consecutive rising.

 

3. Grouping by sequence numbers

Sometimes, we can directly or indirectly get the group number (members should be assigned to which group). In this case, we can directly group by the group number.

 

[e.g. 3] Divide the employee into three groups based on their working years (numbers with a remainder are assigned to certain group), and calculate the average salary of each group. Some of the data are as follows:

ID

NAME

BIRTHDAY

ENTRYDATE

DEPT

SALARY

1

Rebecca

1974/11/20

2005/03/11

R&D

7000

2

Ashley

1980/07/19

2008/03/16

Finance

11000

3

Rachel

1970/12/17

2010/12/01

Sales

9000

4

Emily

1985/03/07

2006/08/15

HR

7000

5

Ashley

1975/05/13

2004/07/30

R&D

16000

 

The option @n of the A.group()function in SPL is used to group by sequence number, and records with the same number are assigned to the same group (number N is assigned to Group N, N starts at 1) .

The SPL script looks like this:


A

1

=T("Employee.csv").sort(ENTRYDATE)

2

=A1.group@n((#-1)*3\A1.len()+ 1)

3

=A2.new(#:GROUP_NO,   ~.avg(SALARY):AVG_SALARY)

A1: import the employee table, and sort them by date of entry.

A2: calculate the number of the group to which they belong by the sorted row number, and group them by the number.

A3: calculate the average salary of each group.

Employee.csv

Olympic.txt

SSEC.csv