Bash Techne

A practical knowledge base

Split large text file into smaller files with equal number of lines

split -l 60 bigfile.txt prefix-

Loop through lines of file

while read line; do
    echo "$line";
done </path/to/file.txt

Use grep to find URLs from HTML file

cat urls.html | grep -Eo "(http|https)://[a-zA-Z0-9./?=_%:-]*"

grep -E: egrep
grep -o: only output what has been grepped
(http|https): either http OR https
a-zA-Z0-9: match all lowercase, uppercase, and digits
.: match period
/: match slash
?: match ?
=: match =
_: match underscore
%: match percent
:: match colon
-: match dash
*: repeat the […] group any number of times