Linux Filesystem

/bin  binaries
/boot boot files
/dev device files
/etc config files
/home home directories
/tmp temporary files
/var logs

Package System

.deb: Debian based: Ubuntu (apt)

.rmp: Fedora, RHEL, Suse, Photon OS, Centos (yum)

Users & groups

sudo useradd -m chang
sudo useradd -g k8s chang
chmod ugo+rwx
chmod u+rwx
chmod u+x

Access Control Lists

getfacl # Get access control list
setfacl # set access control over file

CPU and Processes

top
lscpu
ps
kill

memory check

free
vmstat

CPU and IO check

iostat

Disk space check

df -T
du -h -d 1  /home/chang
fdisk -l # Check file sizes
lsblk -f # This shows type mount points

Symbolic Link

ln -s /usr/share/doc .

Cat

sql=$(cat <<EOF
SELECT foo, bar FROM db
WHERE foo='baz'
EOF
)

cat <<EOF > print.sh
#!/bin/bash
echo \$PWD
echo $PWD
EOF

cat <<EOF | grep 'b' | tee b.txt
foo
bar
baz
EOF

HOST=$(kubectl -n $NAMESPACE get secret audit --template '{{ .data.host | base64decode }}')

cat <<EOF | kubectl -n default apply -f - 
apiVersion: v1
kind: Pod
metadata:
  name: audit-socat
spec:
  containers:
  - name: socat
  image: alpine/socat
  args:
  - TCP4-LISTEN:27017,fork,reuseaddr
  - TCP4:$HOST:27017
EOF

kubectl -n $NAMESPACE port-forward pod/audit-socat 27017

– Usage (reference)

# Using “-” to Specify a Standard Input

<htop-2.2.0.tar.gz | tar -xzf -

wget -c http://geolite.maxmind.com/eoLite2-Country.tar.gz -O - | sudo tar -xz -C /etc/nginx/

curl http://geolite.maxmind.com/eoLite2-Country.tar.gz | tar -xz

awk ( drived from the deveper names)

• Advanced filtering
• Data validations
• Custom transformations
• Formatted report FS: fild separator. Default :’t’ $0= whole line, $1=1st field,$2=2nd field NF: Number of fields in current record NR: Current line number RS: record separator. Default: “n” print(): display message on screen length(): length of a string substr(): extract portion of a string gsub(): replace part of a string with another string index(): find position of a string tolower() toupper()

  awk '{print $1, $6}' /etc/passwd # print from col1 , col 6, This has t delimeter in this file. So onlycol1 will be displayed
  awk -F: '{print $1, $6}' /etc/passwd # show  username and home directory
  echo "mary has a little lamb, little lab, little lamb" | awk '{print substr($0, 1,4)}'   # mary
  echo "mary has" | awk '{if (length($0) > 10) print "String too long"; else print length($0)}' # 8
  cat *.log| awk '{print ($9)} ' | sort | uniq -c

sed (stream editor)

• Search and replace
• Delete
• Insert
• Append
• in-place edit

sed -e 's/BOOK: //'  books.cvs # remove BOOK:
 sed -e 's/book: //I'  books.cvs # remove BOOK: or BoOk: book: 
 sed -e '1, 3 s/book: //I'  books.cvs # remove only from line 1 to line 3
 sed -e 's/book: //I' -e 's/author: //I'  books.cvs # remove book: and author: with case insensitive
 sed -i 's/book: //I' -e 's/author: //I'  books.cvs  # edit files in place

paste

Create new dataset from input files

 paste -d : statesfile capitalfiles

cut

Used to extract selected parts of lines from one or more files

cut -f 1 -d , continent_country_capital.csv
cut -f 1 -d , continent_country_capital.csv | sort | uniq -f
cut -f 2,3 -d , continent_country_capital.csv

sort

  sort file1
  sort -r file1 # reverse order
  sort -k 1,2 file1 | sort -o file2 # sort from col1 and col2  and outputs to file
  sort -n -k 2 -t , -r file1 # numeric values with 2nd col from comma delimeter
  sort -u -f fruits.txt # sort with only unique names and ignore cases (-f)

uniq

uniq cannot remove non-adjacent repeated values

 sort fruits.txt| uniq -i  # show unique fruits
 sort fruits.txt| uniq -i -c # show how many times unique fruits are repeated
 sort fruits.txt| uniq -i -c | sort -r # get from most repeated fruit to least

tr (translate)

  cat file1 | tr [:lower:] [:upper:] # change all words to upper
  cat file1 | tr \t , # place tab to comma
  cat file1 | tr \t ,| tr -s \n  # -s replaces repeated new lines to one newline

wc (word count)

  wc file1 # shows lines | words | characters | filename
  wc -m file1 # characters
  wc -w file1 # words
  wc -l file1 # lines
  wc -c file1 # bites```

diff

“<” means: this need to be removed or changed in the first file
“>” means: this line needs to be added to the first file

diff file1 file2
diff -y file1 file2  # easier to identify side by side
diff -u file1 file2  # unified comparison
diff -i file1 file2  # case insensitive
diff -B file1 file2  # Ignore blank lines for comparison
diff -w file1 file2  # Ignore white spaces
diff  -r /bin/ /sbin/  # comparing directories```
diff -r /bin/ /sbin/ | grep -i "only in /bin"

head and tail

head -n 20 /var/log/messages tail -n 20 /var/log/messages tail -vn 20 /var/log/messages # you can see the tails as system added new lines

cat

cat > outfile.txt << EOF some text
> EOF

find

find / -type f -name *.log find -type d -name www find . -type f -name ".*" # find all the hidden files in the current directory

grep

grep 1978 oscars.tsv| sort > 1978_films.txt 
cut -f 3 oscar.tsv | grep 4 | wc -l grep -i "system" /var/log/messages # lower & upper cases 
grep -i -w "system" /var/log/messages # Only show system insteadof systemd, esystem 
grep -i "shut *down" /var/log/messages # when we do not know shutdown or shut down 
grep -i -n "shut* down" /var/log/messages # Added line number 
grep -i -n -B 2 -A 3"shut *down" /var/log/messages # show 2 previous lines and 3 following lines if matching 
grep -v "system" shut* down" /var/log/messages # all lines excluding shut down or shutdown 
grep -v -c "system" shut* down" /var/log/messages # show number of lines that excluded 
grep -i -w -r 'apache " var" # Directory search 
grep -i -w -r -l 'apache " var" # Directory search and display file pathes only 
grep tiny *  # all files from current folder

File information

file sh.sh
sh.sh: Bourne-Again shell script, ASCII text executable

Last argument

vi Workspace/pyPractice/foo1.py
python3 !$

If tty is messed up in terminal, use

reset