Bash Techne
A practical knowledge base
Split large text file into smaller files with equal number of lines
split -l 60 bigfile.txt prefix-
Loop through lines of file
while read line; do echo "$line"; done </path/to/file.txt
Use grep to find URLs from HTML file
cat urls.html | grep -Eo "(http|https)://[a-zA-Z0-9./?=_%:-]*"
grep -E
: egrepgrep -o
: only output what has been grepped(http|https)
: either http OR httpsa-zA-Z0-9
: match all lowercase, uppercase, and digits.
: match period/
: match slash?
: match ?=
: match =_
: match underscore%
: match percent:
: match colon-
: match dash*
: repeat the […] group any number of times