Information Technology

SYSTEM AND ENVIRONMENT

How To Get The Core Count And Threads In Linux

[chad@rhpacifica ~]$ lscpu | egrep 'Thread|Core|Socket|^CPU\('

CPU(s):                32

Thread(s) per core:    2

Core(s) per socket:    8

Socket(s):             2

[chad@rhpacifica ~]$

CPU - is the equivalent of total threads.

Socket - is the number of processor on board.

In the example above, total cores = 16 (socket x Core per socket)

How To Clear The Bash History

cat /dev/null > ~/.bash_history && history -c && exit

How To Disable and Enable A User Shell Environment

TO ENABLE NO LOGIN OF A USER ACCOUNT

In this mode, the user doesn't have a shell and a password set.

Unlike in Solaris Unix, one can have a shell but had no login property. Thus as root (or sudo), one can switch to the user and run shell scripts. A flexibility that Linux doesn't have.

# usermod -s /sbin/nologin userchad

TO ENABLE SHELL OF A USER

# usermod -s /bin/bash userchad

DOS To Unix File Convertion

perl -pi -e 's/\r\n/\n/g' INPUT.FILE

Infinite Loop Script (template)

while :; do echo 'Hit CTRL+C'; sleep 1; done

DISABLE SWAPPINESS

Edit: /etc/sysctl.conf

# Engage swapping only when memory is exhausted

vm.swappiness = 0

 

DISABLE THP (Transparent Huge Pages)

 

Edit: /etc/rc.d/rc.local

 

# For NoSQL, disable THP

for i in /sys/kernel/mm/*transparent_hugepage/enabled; do

echo never > $i;

done

 

for i in /sys/kernel/mm/*transparent_hugepage/defrag; do

echo never > $i;

Done

Change the permission of the run script:

 

chmod u+x,g+x /etc/rc.d/rc.local

FIREWALL SERVICE MANAGEMENT (start, stop, disable)

Staring on Linux 7, the firewalld service is enabled at bootup.

For lab test purposes, you may want to either stop the service or disable the firewall service to allow all ports to be opened without the annoyance of mapping each application port. Disabling of firewall is NOT recommended for production implementation.

Stop Firewall Service

# systemctl stop firewalld

Start Firewall Service

# systemctl start firewalld

Disable Firewall Service

# systemctl disable firewalld

Enable Firewall Service

# systemctl enable firewalld

COMMANDS RELATED TO NETWORK

How To Check Open Socket Range

[chad@lxnode11 ~]$ cat /proc/sys/net/ipv4/ip_local_port_range

32768 61000

[chad@lxnode11 ~]$

Open sockets allowed in a system is the difference of the above port ranges (28,232).

Note when adjusting the starting port range, make sure that it will not conflict with the existing required ports of the software itself (i.e. 8091,4984,11210, etc.).

Open Socket Connection Queue

KERNAL PARAM:

net.core.somaxconn

 

from 128 to 1024

The net.core.somaxconn value has an important role. It sets the maximum number of requests queued (backlog) per listener socket port.

How To Check Number Of Connections To Target Server From Host Server

netstat -anp | grep -i <destination IP> | wc -l

How To Check Number Of Client Connections On Host Server

netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

How To Check Number Of Connections To A Port

netstat -an | grep :8093

$ netstat -an | grep :8093

tcp        0      0 127.0.0.1:44510             127.0.0.1:8093              ESTABLISHED

tcp        0      0 :::8093                     :::*                        LISTEN      

tcp        0      0 ::ffff:10.164.32.120:8093   ::ffff:10.164.32.124:54584  ESTABLISHED

tcp        0      0 ::ffff:127.0.0.1:8093       ::ffff:127.0.0.1:44510      ESTABLISHED

$

How To Get A Count Of All Connection State

netstat -ant | awk '{print $6}' | sort | uniq -c | sort -n

$ netstat -ant | awk '{print $6}' | sort | uniq -c | sort -n

      1 established)

      1 Foreign

     55 LISTEN

    106 TIME_WAIT

    564 ESTABLISHED

$

Note: If there were more TIME_WAIT than established, consider adjusting the socket port range (/proc/sys/net/ipv4/ip_local_port_range) to allow high volume of traffic. Or, adjust the time out (/proc/sys/net/ipv4/tcp_keepalive_time) to make it short. If TIME_WAIT still persist, check out-of-memory error in target host.

How To Find The Port Of A Process

Example: find the port of mongod 

[root@mdb01 chad]# netstat -ltnp | grep mongo

tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      12151/mongod       

[root@mdb01 chad]#

 

COMMANDS RELATED TO PROCESS

How To Check Top Process Memory Consumption

ps aux | sort -rn -k 6 | head -7

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

27810    38893  8.5 20.4 2332800 1221992 ?     Sl   Jul25 2826:25 /opt/couchbase-sync-gateway/bin/sync_gateway /home/sync_gateway/sync_gateway.json

cbase    63201  0.0  1.2 2588748 74308 ?       Sl    2016  29:31 ./cbq-gui -datastore=http://bopcdcrbcrfcb04.ual.com:8091 -localPort=:8095 -user= -pass=

root      6671  0.0  0.4 654084 25664 ?        Sl    2016  17:37 /opt/OV/lbin/perf/coda

root     17812  0.0  0.2 839164 15480 ?        Sl   Jun14  78:45 /opt/OV/lbin/agtrep/agtrep -start

root      6611  0.0  0.2 2423660 14748 ?       Ssl   2016 212:16 /opt/OV/bin/ovcd

root      6544  0.0  0.2 111972 14336 ?        S<l   2016   0:00 /opt/perf/bin/perfd

root      6452  2.0  0.2  42844 12432 ?        S<s   2016 10483:04 /opt/perf/bin/scopeux

NOTE:

VSZ is Virtual Memory Size (KB)

RSS is Resident Set Size KB (Real Memory)

How To List The Process Tree

$ ps -ejH

$ ps axjf

How To Check A User File Descriptor Limit

  ulimit -Hn    #-- check hard limit

  ulimit -Sn    #-- check soft limit

How To Set A File Descriptor Limit

Modify or add the following lines in /etc/security/limits.conf:

couchbase              soft    nofile                  <value>

couchbase              hard    nofile                  <value>

Where <value> is greater than 40960.

example:

couchbase     soft  nofile  51200

couchbase     hard  nofile  51200

sync_gateway  soft  nofile  131072

sync_gateway  hard  nofile  131072

How To Check File Descriptor Usage

Get total File Descriptor usage of a host (run as root):

lsof | wc -l

The total file descriptor usage of a user (sync_gateway):

/usr/sbin/lsof -u sync_gateway | wc -l

Get File Descriptor usage of a process:

1. Get the PID of a process

ps aux | grep processName

or,

pidof processName

2. List the file descriptor in use

lsof -a -p <PID> | wc -l

Optionally, you can find where the File Descriptors are for a process:

ls -l /proc/<your PID>/fd

How To Check Specific Process Swap Usage

 

[chad@mdb01 ~]$ pidof mongod

4929

[chad@mdb01 ~]$

[chad@mdb01 ~]$ grep --color VmSwap /proc/4929/status

VmSwap:   260804 kB

[chad@mdb01 ~]$

How To List All Processes Using Swap Space

for procID in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $procID; done

 

Script with sorted output:

 

## Get swap space in Linux using bash for loop ##

for procID in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $procID; done | sort -k 2 -n -r | less

 

[chad@mdb01 ~]$ for procID in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $procID; done | sort -k 2 -n -r | less

mongod 416208 kB

tet-enforcer 1620 kB

amsHelper 1308 kB

tet-sensor 456 kB

tet-enforcer 20 kB

tet-main 16 kB

hpasmlited 12 kB

How To Clear The Swap Usage

1. Check if you have enough memory to accommodate the swap size.

      $ free -m

2. Turn off swap to clear.

      # swapoff -a

3. Turn on swap to re-enable swap.

      # swapon -a

STRING MANIPULATION

REMOVE DUPLICATE LINES

cat index.txt | sort -u >index_uq.txt

PRINT LINE NOT MATCHING A WORD (MISSING KEYWORD)

cat index_uq.txt | awk '!/missingWord/ {print}'