11.8 Traverse directory and summarize files

 

Traverse files in a directory and summarize them through calling SPL script recursively.
Traverse all text files in a specified directory and under its subdirectories, and put line17 in each file to a separate file.

imagepng

Below is a text file sample:

16 …
17 Middleware for report source data computing
18 …

The target result.txt after summary:

Middleware for report source data computing
SPL parsing and exporting Excel
SQL Headaches Therapies – For Loop Operations
The skill of updating database with esProc
…

The parameter:

ID Name Value Note
1 path File directory

Read the specified row of each file and export it to a text file and save the result file as readfile.dfx.

SPL script:

A B
1 =directory@p(path)
2 =A1.(file(~).cursor@s())
3 =A2.((~.skip(16),~.fetch@x(1)))
4 =A3.union()
5 >file(“result.txt”).export@a(A4)
6 =directory@dp(path)
7 if A6.len()==0 return
8 else =A6.(call(“readfile.dfx”,~))

A1 List files under the current directory.
A2 Open file cursor in loop.
A3 Skip 16 lines to get line 17 from each file cursor.
A4 Union all retrieved records.
A5 Export union result to result.txt.
A6 List subdirectories in the current directory.
B7 Return if there isn’t any subdirectories.
B8 Execute the script recursively if there is a subdirectory.

In SPL, we use call() function to execute an edited readfile.dfx.

SPL script:

A
1 =call(“readfile.dfx”,“D:/Documents”)

A1 Execute the cellset program, where the parameter specifies a directory.

Traversing a directory recursively and reading files with the cursor consume little memory. The method is suitable for handling scenarios where a large number of files or a huge volume of data is involved.