- [Instructor] Welcome to the fourth video
of section two, Playing with xargs.
In the previous video we learned
about finding files and file listing.
In this video we'll learn how to formulate one-liners
for solving text processing problems.
We use pipes to redirect stdout, standard output
of a command to stdin, standard input for another command.
For example,
some of the arguments accept data as command line arguments
rather than a data stream through stdin, standard input.
In this case we cannot use pipe to supply data
through command line arguments.
We should try alternate methods.
xargs is a command that is very helpful
in handling standard input data
to the command line argument conversions.
It can manipulate stdin and convert
to command line arguments for the specified command.
Also xargs can convert any one line
or multiple line text inputs into other formats
such as multiple lines, specified number of columns,
or a single line and vice versa.
All bash users love one-liner commands
which are command sequences that are joined
by the pipe operator but do not use
the semi-colon terminator between the commands used.
Crafting one-line commands makes tasks
more efficient and simpler to solve.
It requires proper understanding and practice
to formulate one-liners
for solving text processing problems.
xargs is one of the important components
for building one-liner commands.
When using the pipe operator the xargs command
should always be the first thing
to appear after the operator.
xargs uses standard input as the primary data stream source.
It uses stdin and executes another command
by providing command line arguments
for that executing command using the stdin datasource.
For example,
let's see how to do it.
The xargs command can supply arguments to a command
by reformatting the data received through stdin.
xargs can act as a substitute that can
perform similar actions as the -exec argument
in the case of the find command.
Let's see a variety of hacks that can be performed
using the xargs command.
Converting multiple lines of input to a single line output.
Multiple line inputs can be converted
simply by removing the new line character
and replacing with the space character.
/n is interpreted as a new line character
which is a delimiter for the lines.
By using xargs we can ignore all the new lines
with a space so that multiple lines
can be converted into single line text as follows.
Converting single line into multiple line format.
Given a maximum number of arguments in a line equals n
we can split any stdin, standard input text,
into texts of n arguments each.
An argument is a piece of string delimited by space.
Space is the default delimiter.
A single line can be split into multiple lines as follows.
Now we will see how the function works.
The xargs command is appropriate to be applied
to many problem scenarios with its many options.
Let's see how these options
can be used wisely to solve problems.
We can also use our own delimiter
towards separating arguments.
To specify a custom limiter for input
use the -d option as follows.
We've learned how to format stdin to different output
as arguments from the previous examples.
Now let's learn how to supply this formatted output
as arguments to commands.
Passing formatted arguments to a command by reading stdin,
write a small custom echo script for better understanding
of usage examples with xargs to provide command arguments.
When arguments are passed to the C echo to S-H shell
it will print the arguments terminated
by the hash character.
For example,
let's have a look at our problem.
I have a list of arguments in a file,
one argument in each line to be provided
to a command, say cecho.sh.
I need to provide arguments in two methods.
In the first method I need to provide
one argument for each of the commands as follows.
Or, alternatively, I need to provide two or three arguments
for each execution of the command.
For two arguments each it would be similar to the following.
In the second method I need to provide
all arguments at once to the command as follows.
Run the preceding commands and not the output
before going through the following section.
These problems can be solved using xargs.
We have the list of arguments in a file called args.txt.
The content are as follows.
For the first problem we can execute the command
multiple times with one argument per execution.
Therefore, use
For executing the command
with x arguments per each execution use
for example.
For the second problem, in order to execute the command
at once for all the arguments use
In this code standard in contains a string
consisting of multiple x characters.
We can use x as the input delimiter
while using it with the -d.
Here we've explicitly specified x as the input delimiter
whereas in the default case xargs
takes the internal field separator space
as the input delimiter.
In the preceding command execution arg1
is the only variable text.
All others should remain constant.
We should read arguments from a file args.txt
and supply it as
to provide a command execution sequence as shown.
xargs has an option -I.
By using -I we can specify a replacement string
that will be replaced while xargs expands.
When -I is used with xargs it will execute
as one command execution per argument.
Let's do it as follows.
- I {} specifies the replacement string.
For each of the arguments supplied for the command
the {} string will be replaced
with arguments read through stdin.
When used with the -I command the command
is executed in a loop.
When there are three arguments the command is executed
three times along with the command {}.
Each time {} is replaced with the arguments one by one.
Using xargs with find xargs and find are best friends.
They can be combined to perform tasks easy.
Usually people combine them in the wrong way.
For example,
this is dangerous.
It may sometimes cause removal of unnecessary files.
Here we cannot predict the delimiting character.
Whether it is /n or space
for the output of the find command.
Many of the filenames may contain a space character
and hence xargs may misinterpret as a delimiter.
For example, hell text.txt
is misinterpreted by xargs as hell and text.txt.
Hence we must use the -print0 along with finds
to produce an output with a delimited character null
whenever we use the find output as the xargs input.
Let's use find to match and list all of the .txt files
and remove them using xargs.
This removes all .txt files.
xargs -0 interprets the delimiting character as /0.
Counting the number of lines of c code
in a source code directory.
This is a task most programmers do.
That is counting all c program files for lines of code loc.
The code for the task is as follows.
If you want more statistics about your source code
there is utility called sloccount which is very useful.
Modern G and ULinux distributions
usually have packages or you can get it
from http code on //www.dwheeler.com\sloccount/.
While and subshell trick with stdin args is restricted
to providing arguments in limited ways to supply arguments.
Also xargs cannot supply arguments
to multiple sets of commands.
For executing commands with collected arguments
from the standard input we have a very flexible method.
A subshell with a while loop can be used to read arguments
and execute commands in a trickier way as follows.
Here, by replacing cat $arg with any number of commands
while using a while loop we can perform
many command actions with the same arguments.
We can also pass the output
to other commands without using pipes.
Subshell tricks can be used
in a variety of problematic environments.
When enclosed within subshell operators
it acts as a single unit
where there're multiple commands inside like so.
If cmd1 is cd/ within the subshell
the path of the working directory changes.
However, this change resides inside the subshell only.
cmd4 will not see the directory change.
Awesome.
In this video we've seen how to play with xargs.
In the next video we'll learn about translating with tr.