Linux Shell TIPS
This list will be regularly updated.
Find a string inside a file in a given directory
For example search of ‘ErrorDocument’ in /etc directory and display filename and line where the string was found:
find /etc -type f -exec grep -H 'ErrorDocument' {} \;
Something equivalent with grep (ignoring case (-i) and recursively searching (-r) all files in the current and child folders plus symbolic links, and displaying line number (-n)):
$> grep -inr 'pattern' $> grep -inr 'error'
The following finds finds including the string error and then filters the list to files that do not contain help using the -v option.
$> grep -inr 'error' | grep -v 'help'
Regular expression and pattern
Here note the importance of double square brackets.
Test filename extension:
[[ $filename == *.sql.bz2 ]] # or [[ "$filename" == *.sql.bz2 ]]
Test a value is a number:
regex_number='^[0-9]+$' if [[ ! "$port" =~ $regex_number ]]; then echo "error $port is not a number" fi
Ternary operator: booleanExpression ? expression1 : expression2
booleanExpression ? expression1 : expression2 # java version
booleanExpression && expression1 || expression2 # Shell versionExample:
# example 1: [ $myVariable -eq 5 ] && echo "equal 5" || echo "not equal 5" #example 2: a=false [ $a = true ] && b=1 || b=2 echo $b # will display 2
Last linux boot time
$> who -b
system boot 2022-06-07 12:51Use -x option to run the shell interpreter to facilitate your debugging session
It can be useful for debuging:
#!/bin/bash -x # # myscript.sh echo "hello world" i=0 ((i++)) echo "i = $i"
$> ./test.sh
+ echo 'hello world'
hello world
+ i=0
+ (( i++ ))
+ echo 'i = 1'
i = 1
IFS: Input Field Separator
IFS is a Unix shell variable which defines the character separators for the shell command line interpreter
You can see it’s value:
$> set | grep ^IFS
IFS=$' \t\n'As you can see, it’s default value is:
- space
- tabbing (\t)
- line feed (\n)
You can change this value, for example when you don’t want space to be consider as a separator (for instance if you have filename with spaces, …).
Impact of the default IFS value
[root@instance-nicotest2 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 4.7G 0 4.7G 0% /dev
tmpfs 4.8G 0 4.8G 0% /dev/shm
tmpfs 4.8G 41M 4.7G 1% /run
tmpfs 4.8G 0 4.8G 0% /sys/fs/cgroup
/dev/mapper/ocivolume-root 36G 8.4G 28G 24% /
/dev/mapper/ocivolume-oled 10G 117M 9.9G 2% /var/oled
/dev/sda2 1014M 314M 701M 31% /boot
/dev/sda1 100M 5.1M 95M 6% /boot/efi
tmpfs 967M 0 967M 0% /run/user/0
tmpfs 967M 0 967M 0% /run/user/987
/dev/sdb1 51G 53M 49G 1% /volumetest
tmpfs 967M 0 967M 0% /run/user/1000
[root@instance-nicotest2 ~]# df=$(df -h)
[root@instance-nicotest2 ~]# echo $df
Filesystem Size Used Avail Use% Mounted on devtmpfs 4.7G 0 4.7G 0% /dev tmpfs 4.8G 0 4.8G 0% /dev/shm tmpfs 4.8G 41M 4.7G 1% /run tmpfs 4.8G 0 4.8G 0% /sys/fs/cgroup /dev/mapper/ocivolume-root 36G 8.4G 28G 24% / /dev/mapper/ocivolume-oled 10G 117M 9.9G 2% /var/oled /dev/sda2 1014M 314M 701M 31% /boot /dev/sda1 100M 5.1M 95M 6% /boot/efi tmpfs 967M 0 967M 0% /run/user/0 tmpfs 967M 0 967M 0% /run/user/987 /dev/sdb1 51G 53M 49G 1% /volumetest tmpfs 967M 0 967M 0% /run/user/1000
[root@instance-nicotest2 ~]# IFS=""
[root@instance-nicotest2 ~]# echo $df
Filesystem Size Used Avail Use% Mounted on
devtmpfs 4.7G 0 4.7G 0% /dev
tmpfs 4.8G 0 4.8G 0% /dev/shm
tmpfs 4.8G 41M 4.7G 1% /run
tmpfs 4.8G 0 4.8G 0% /sys/fs/cgroup
/dev/mapper/ocivolume-root 36G 8.4G 28G 24% /
/dev/mapper/ocivolume-oled 10G 117M 9.9G 2% /var/oled
/dev/sda2 1014M 314M 701M 31% /boot
/dev/sda1 100M 5.1M 95M 6% /boot/efi
tmpfs 967M 0 967M 0% /run/user/0
tmpfs 967M 0 967M 0% /run/user/987
/dev/sdb1 51G 53M 49G 1% /volumetest
tmpfs 967M 0 967M 0% /run/user/1000
[root@instance-nicotest2 ~]# IFS=$' \t\n'
[root@instance-nicotest2 ~]# echo $df
Filesystem Size Used Avail Use% Mounted on devtmpfs 4.7G 0 4.7G 0% /dev tmpfs 4.8G 0 4.8G 0% /dev/shm tmpfs 4.8G 41M 4.7G 1% /run tmpfs 4.8G 0 4.8G 0% /sys/fs/cgroup /dev/mapper/ocivolume-root 36G 8.4G 28G 24% / /dev/mapper/ocivolume-oled 10G 117M 9.9G 2% /var/oled /dev/sda2 1014M 314M 701M 31% /boot /dev/sda1 100M 5.1M 95M 6% /boot/efi tmpfs 967M 0 967M 0% /run/user/0 tmpfs 967M 0 967M 0% /run/user/987 /dev/sdb1 51G 53M 49G 1% /volumetest tmpfs 967M 0 967M 0% /run/user/1000
[root@instance-nicotest2 ~]# echo "$df"
Filesystem Size Used Avail Use% Mounted on
devtmpfs 4.7G 0 4.7G 0% /dev
tmpfs 4.8G 0 4.8G 0% /dev/shm
tmpfs 4.8G 41M 4.7G 1% /run
tmpfs 4.8G 0 4.8G 0% /sys/fs/cgroup
/dev/mapper/ocivolume-root 36G 8.4G 28G 24% /
/dev/mapper/ocivolume-oled 10G 117M 9.9G 2% /var/oled
/dev/sda2 1014M 314M 701M 31% /boot
/dev/sda1 100M 5.1M 95M 6% /boot/efi
tmpfs 967M 0 967M 0% /run/user/0
tmpfs 967M 0 967M 0% /run/user/987
/dev/sdb1 51G 53M 49G 1% /volumetest
tmpfs 967M 0 967M 0% /run/user/1000Above we can see 3 cases that are detailed below:
1/ echo $df without double quote and with IFS set to its default value
df=$(df -h)
# here we don't change IFS value, it is set to its default value
echo $dfWhen we first echo $df we have an output on only one line. Why? It’s because the command line interpreter broke the text containing by $df into several input parameters to echo, each word separated by a space, or a tabbing, or LF is considered as a parameter. Of course in this example it gives a lot of parameters… 🙂
2/ echo $df without double quotes and IFS set to an empty string
df=$(df -h)
IFS=''
echo $dfWhen we set IFS to blank, it means there is no input separator character. Thus with the second echo $df we can see the content of $df displayed exactly as the same it was when executing the command df -h.
If you change IFS value for your needs, don’t forget to set it back to its initial value when your treatments are done.
3/ echo $df surrounded with double quotes, and IFS set to its default value
df=$(df -h)
IFS=$' \t\n' # here we set IFS to its default value because it has been changed in step number 2
echo $dfIn this case you englobe the whole text between double quote to tell to the interpreter to consider it as only one input field.
Many functions have an option to specify field separator without changing IFS
Here two examples grab from internet:
# Search for the user named vivek and print home directory
# The -F':' set ':' for the input field separator and value of $IFS always ignored
awk -F':' '/vivek/{ print $6}' /etc/passwd
# Search for the user named vivek and print home directory
# The -d':' set ':' for the input field separator and value of $IFS always ignored
grep -w "^vivek" /etc/passwd | cut -d':' -f6
For and while loops
IFS is used by for and while loops.
text="text1 text2 text3 text4"
for txt in $text
do
echo $txt
done
# it will display this:
text1
text2
text3
text4Test url or resources is available
httpCode=$(curl -o /dev/null --silent -Iw '%{http_code}' https://example.com/my.remote.tarball.gz)
[ "$httpCode" -eq "200" ] && echo "Resource is available"scp copy file from a host to another through a jump host
If you want to copy a file from a distant hidden host to your local host:
scp -o 'ProxyJump <user_public_host>@<public_ip>' <user_hidden_host>@<private_ip>:/source/path/yourfiletocopy /target/localpath/
Or if you wan to copy a file from your host to an hidden host (hidden behind a JumpHost, it means a public host):
scp -o 'ProxyJump <user_public_host>@<public_ip>' /source/localpath/yourfiletocopy <user_hidden_host>@<private_ip>:/target/path/
#some examples:
scp -o 'ProxyJump nicodevlog@193.193.193.193' nicodevlog@192.168.201.10:/mnt/source/nicodevlog.txt /root/nicodevlog/
scp -o 'ProxyJump nicodevlog@193.193.193.193' /root/nicodevlog/nicodevlog.txt nicodevlog@192.168.201.10:/mnt/target/What is my public IP on my linux server, command line
curl ifconfig.me
This won’t give you the correct IP if your server is behind a proxy.