Bash

Heredoc

cat << EOF > file.txt
The current working directory is $PWD.
You are logged in as $(whoami).
EOF

Plain-print the difference between two files

Suppose we have two files: packages.fedora and packages.

packages.fedora:

autossh
bash-completion
bat
bc
borgmatic
bzip2
cmake
curl
diff-so-fancy
diffutils
dnf-plugins-core

packages:

bash-completion
bc
bzip2
curl
diffutils
dnf-plugins-core

To plain-print the lines that exist in packages.fedora but do not exist packages:

comm -23 <(sort packages.fedora) <(sort packages)

Output:

autossh
bat
borgmatic
cmake
diff-so-fancy
  • The comm command compares two sorted files line by line.
  • The -23 flag is shorthand for -2 and -3.
  • -2 : suppress column 2 (lines unique to packages)
  • -3 : suppress column 3 (lines that appear in both files)

Split large text file into smaller files with equal number of lines

split -l 60 bigfile.txt prefix-

Loop through lines of file

while read line; do
    echo "$line";
done </path/to/file.txt

Use grep to find URLs from HTML file

cat urls.html | grep -Eo "(http|https)://[a-zA-Z0-9./?=_%:-]*"
  • grep -E: egrep
  • grep -o: only output what has been grepped
  • (http|https): either http OR https
  • a-zA-Z0-9: match all lowercase, uppercase, and digits
  • .: match period
  • /: match slash
  • ?: match ?
  • =: match =
  • _: match underscore
  • %: match percent
  • :: match colon
  • -: match dash
  • *: repeat the [...] group any number of times

Use Awk to print the first line of ps aux output followed by each grepped line

To find all cron processes with ps aux.

ps aux | awk 'NR<2{print $0;next}{print $0 | grep "cron"}' | grep -v "awk"
  • ps aux : equivalent to ps -aux. -a displays info about other users processes besides to current user. -u displays info associated with keywords user, pid, %cpu, %mem, vsz, rss, tt, state, start, time, and command. -x includes processes which do not have a controlling terminal. See man 1 ps.
  • awk 'NR<2{print $0;next}{print $0 | "grep cron"}' | grep -v "awk" : For number of input records (NR) less than 2, print the input record ($0), go to the next input record and repeat the {print $0} pattern until the end is reached, then execute the END rule. The End rule in this case is {print $0 | "grep cron"}, it prints the remaining input records after piping them through the "grep cron" command. This allows printing the first line of the ps aux output, which consists of the column labels, and filters out everything besides what you want to grep for (e.g. "cron" processes).
  • grep -v "awk" : avoids printing the line containing this command.