©2015 - 2022 Chad’s Technoworks. Disclaimer and Terms of Use

Chad’s TechnoWorks My Journal On Technology

LINUX COMMAND LINE CHEAT SHEET

Information Technology

TABLE OF CONTENTS


System and Environment

How To Get The Core Count And Threads In Linux

How To Clear The Bash History

How To Disable and Enable A User Shell Environment

How To Convert DOS To Unix File Format

Infinite Loop Script Template


Disable Swappiness

Disable Transparent Huge Pages (THP)

Firewall Service Management (stop,start,disable,enable)



Commands Related To Process

How To Check Top Process Memory Consumption

How To List The Process tree


How To Check A User File Descriptor Limit

How To Set A File Descriptor Limit

How To Check File Descriptor Usage

How To Open Open Socket Range


Check Process Swap Usage

List All Process Using Swap

To Clear Swap Usage



Commands Related To Network

How To Check Number Of Connections To Target Server From Host Server

How to Check Number Of Client Connections On Host Server

How To Check Number Of Connections To A Port

How To Get A Count Of All Connection State

How To Find The Port Of A Process



String Manipulation

Remove Duplicate Lines

Print Lines Not Matching A Word





SYSTEM AND ENVIRONMENT

How To Get The Core Count And Threads In Linux


[chad@rhpacifica ~]$ lscpu | egrep 'Thread|Core|Socket|^CPU\('

CPU(s):                32

Thread(s) per core:    2

Core(s) per socket:    8

Socket(s):             2

[chad@rhpacifica ~]$


CPU - is the equivalent of total threads.

Socket - is the number of processor on board.

In the example above, total cores = 16 (socket x Core per socket)



How To Clear The Bash History


cat /dev/null > ~/.bash_history && history -c && exit




How To Disable and Enable A User Shell Environment


TO ENABLE NO LOGIN OF A USER ACCOUNT

In this mode, the user doesn't have a shell and a password set.

Unlike in Solaris Unix, one can have a shell but had no login property. Thus as root (or sudo), one can switch to the user and run shell scripts. A flexibility that Linux doesn't have.


# usermod -s /sbin/nologin userchad


TO ENABLE SHELL OF A USER


# usermod -s /bin/bash userchad




DOS To Unix File Convertion


perl -pi -e 's/\r\n/\n/g' INPUT.FILE



Infinite Loop Script (template)


while :; do echo 'Hit CTRL+C'; sleep 1; done



DISABLE SWAPPINESS


Edit: /etc/sysctl.conf


# Engage swapping only when memory is exhausted

vm.swappiness = 0


 

DISABLE THP (Transparent Huge Pages)

 

Edit: /etc/rc.d/rc.local

 

# For NoSQL, disable THP

for i in /sys/kernel/mm/*transparent_hugepage/enabled; do

echo never > $i;

done

 

for i in /sys/kernel/mm/*transparent_hugepage/defrag; do

echo never > $i;

Done


Change the permission of the run script:

 

chmod u+x,g+x /etc/rc.d/rc.local



FIREWALL SERVICE MANAGEMENT (start, stop, disable)


Staring on Linux 7, the firewalld service is enabled at bootup.

For lab test purposes, you may want to either stop the service or disable the firewall service to allow all ports to be opened without the annoyance of mapping each application port. Disabling of firewall is NOT recommended for production implementation.


Stop Firewall Service


# systemctl stop firewalld



Start Firewall Service


# systemctl start firewalld



Disable Firewall Service


# systemctl disable firewalld



Enable Firewall Service


# systemctl enable firewalld



COMMANDS RELATED TO NETWORK


How To Check Open Socket Range


[chad@lxnode11 ~]$ cat /proc/sys/net/ipv4/ip_local_port_range

32768 61000

[chad@lxnode11 ~]$


Open sockets allowed in a system is the difference of the above port ranges (28,232).

Note when adjusting the starting port range, make sure that it will not conflict with the existing required ports of the software itself (i.e. 8091,4984,11210, etc.).


Open Socket Connection Queue

KERNAL PARAM:

net.core.somaxconn

 

from 128 to 1024

The net.core.somaxconn value has an important role. It sets the maximum number of requests queued (backlog) per listener socket port.




How To Check Number Of Connections To Target Server From Host Server


netstat -anp | grep -i <destination IP> | wc -l




How To Check Number Of Client Connections On Host Server


netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n



How To Check Number Of Connections To A Port


netstat -an | grep :8093


$ netstat -an | grep :8093

tcp        0      0 127.0.0.1:44510             127.0.0.1:8093              ESTABLISHED

tcp        0      0 :::8093                     :::*                        LISTEN      

tcp        0      0 ::ffff:10.164.32.120:8093   ::ffff:10.164.32.124:54584  ESTABLISHED

tcp        0      0 ::ffff:127.0.0.1:8093       ::ffff:127.0.0.1:44510      ESTABLISHED

$



How To Get A Count Of All Connection State


netstat -ant | awk '{print $6}' | sort | uniq -c | sort -n


$ netstat -ant | awk '{print $6}' | sort | uniq -c | sort -n

      1 established)

      1 Foreign

     55 LISTEN

    106 TIME_WAIT

    564 ESTABLISHED

$


Note: If there were more TIME_WAIT than established, consider adjusting the socket port range (/proc/sys/net/ipv4/ip_local_port_range) to allow high volume of traffic. Or, adjust the time out (/proc/sys/net/ipv4/tcp_keepalive_time) to make it short. If TIME_WAIT still persist, check out-of-memory error in target host.




How To Find The Port Of A Process


Example: find the port of mongod 


[root@mdb01 chad]# netstat -ltnp | grep mongo

tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      12151/mongod       

[root@mdb01 chad]#

 







COMMANDS RELATED TO PROCESS


How To Check Top Process Memory Consumption


ps aux | sort -rn -k 6 | head -7


USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

27810    38893  8.5 20.4 2332800 1221992 ?     Sl   Jul25 2826:25 /opt/couchbase-sync-gateway/bin/sync_gateway /home/sync_gateway/sync_gateway.json

cbase    63201  0.0  1.2 2588748 74308 ?       Sl    2016  29:31 ./cbq-gui -datastore=http://bopcdcrbcrfcb04.ual.com:8091 -localPort=:8095 -user= -pass=

root      6671  0.0  0.4 654084 25664 ?        Sl    2016  17:37 /opt/OV/lbin/perf/coda

root     17812  0.0  0.2 839164 15480 ?        Sl   Jun14  78:45 /opt/OV/lbin/agtrep/agtrep -start

root      6611  0.0  0.2 2423660 14748 ?       Ssl   2016 212:16 /opt/OV/bin/ovcd

root      6544  0.0  0.2 111972 14336 ?        S<l   2016   0:00 /opt/perf/bin/perfd

root      6452  2.0  0.2  42844 12432 ?        S<s   2016 10483:04 /opt/perf/bin/scopeux


NOTE:

VSZ is Virtual Memory Size (KB)

RSS is Resident Set Size KB (Real Memory)



How To List The Process Tree


$ ps -ejH


$ ps axjf



How To Check A User File Descriptor Limit


  ulimit -Hn    #-- check hard limit


  ulimit -Sn    #-- check soft limit



How To Set A File Descriptor Limit


Modify or add the following lines in /etc/security/limits.conf:


couchbase              soft    nofile                  <value>

couchbase              hard    nofile                  <value>



Where <value> is greater than 40960.


example:


couchbase     soft  nofile  51200

couchbase     hard  nofile  51200

sync_gateway  soft  nofile  131072

sync_gateway  hard  nofile  131072


How To Check File Descriptor Usage


Get total File Descriptor usage of a host (run as root):


lsof | wc -l


The total file descriptor usage of a user (sync_gateway):


/usr/sbin/lsof -u sync_gateway | wc -l



Get File Descriptor usage of a process:


1. Get the PID of a process


ps aux | grep processName

or,

pidof processName


2. List the file descriptor in use


lsof -a -p <PID> | wc -l


Optionally, you can find where the File Descriptors are for a process:


ls -l /proc/<your PID>/fd



How To Check Specific Process Swap Usage

 

[chad@mdb01 ~]$ pidof mongod

4929

[chad@mdb01 ~]$

[chad@mdb01 ~]$ grep --color VmSwap /proc/4929/status

VmSwap:   260804 kB

[chad@mdb01 ~]$



How To List All Processes Using Swap Space


for procID in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $procID; done

 

Script with sorted output:

 

## Get swap space in Linux using bash for loop ##

for procID in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $procID; done | sort -k 2 -n -r | less

 

[chad@mdb01 ~]$ for procID in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $procID; done | sort -k 2 -n -r | less

mongod 416208 kB

tet-enforcer 1620 kB

amsHelper 1308 kB

tet-sensor 456 kB

tet-enforcer 20 kB

tet-main 16 kB

hpasmlited 12 kB



How To Clear The Swap Usage

1. Check if you have enough memory to accommodate the swap size.

      $ free -m


2. Turn off swap to clear.

      # swapoff -a


3. Turn on swap to re-enable swap.

      # swapon -a





STRING MANIPULATION


REMOVE DUPLICATE LINES

cat index.txt | sort -u >index_uq.txt


PRINT LINE NOT MATCHING A WORD (MISSING KEYWORD)


cat index_uq.txt | awk '!/missingWord/ {print}'