Difference between revisions of "HowTo One Liners"

From NST Wiki
Jump to navigationJump to search
(References)
(Use SSH To Login On Remote System Using A Different New Shell)
(47 intermediate revisions by 2 users not shown)
Line 2: Line 2:
 
= Overview =
 
= Overview =
 
This page provides a quick reference to common '''One Liner''' administrative command line operations.
 
This page provides a quick reference to common '''One Liner''' administrative command line operations.
 +
 +
== One Liner Resources ==
 +
 +
* '''[http://www.grymoire.com/Unix/Regular.html A Regular Expression Reference]'''
 +
 +
* '''[https://regexr.com/ Regular Expression Online Tool]'''
 +
 +
* '''[http://main.rtfiber.com.tw/~changyj/sed/ Sed and Regular Expressions]'''
 +
 +
* '''[https://www.tutorialspoint.com/unix/unix-regular-expressions.htm Linux - Regular Expressions Tutorial with Sed]'''
 +
 +
* '''[https://eloquentjavascript.net/09_regexp.html A Javascript Regular Expressions Tutorial]
 +
 +
* Handy One Liners for Sed: '''[http://www.linuxhowtos.org/System/sedoneliner.htm Sed - LinuxHowtos]''', '''[http://sed.sourceforge.net/sed1line.txt Sed - SourceForge]''', '''[https://gist.github.com/jasonm23/396693/b9135d0dbe821d2ff8bd1b8a0a452cb27b4c2f68 Sed - Gist]'''
 +
 +
* '''[https://www.gnu.org/software/sed/ GNU Sed Homepage]'''
 +
 +
* '''[http://www.grymoire.com/Unix/Sed.html A Sed Tutorial]'''
 +
 +
* '''[http://www.ibm.com/developerworks/linux/library/l-sed1/index.html Sed Part1]''', '''[http://www.ibm.com/developerworks/linux/library/l-sed2/index.html Sed Part2]''' and '''[http://www.ibm.com/developerworks/linux/library/l-sed3/index.html Sed Part3]'''
 +
 +
* '''[https://posts.specterops.io/fawk-yeah-advanced-sed-and-awk-usage-parsing-for-pentesters-3-e5727e11a8ad Advanced Parsing for Pentesters]'''
 +
 +
* '''[https://www.gnu.org/software/gawk/ GNU Gawk Homepage]'''
 +
 +
* '''[https://www.gnu.org/software/gawk/manual/gawk.html The GNU Awk User’s Guide]'''
 +
 +
* '''[http://www.grymoire.com/Unix/Awk.html An Awk Tutorial]'''
 +
 +
* '''[http://www.thegeekstuff.com/2010/03/awk-arrays-explained-with-5-practical-examples Awk Array Examples]'''
 +
 +
* '''[https://www.ibm.com/developerworks/library/l-lpic1-103-2/l-lpic1-103-2-pdf.pdf Text Streams and Filters]'''
 +
 +
* '''[http://www.grymoire.com/Unix/Grep.html A Grep Tutorial]'''
 +
 +
* '''[http://www.grymoire.com/Unix/Find.html A Find Example]'''
 +
 +
* '''[http://www.grymoire.com/Unix/Tar.html A Tar Tutorial]'''
 +
 +
= Get Syntax Color In Less =
 +
 +
The NST includes the source-highlight package which can "smartly" apply color to a wide variety of file formats. You can set some '''less''' environment variables to make use of the source-hightlight package to color code files in your terminal with the following settings:
 +
 +
export LESSOPEN="| source-highlight --out-format=esc -o STDOUT -i %s 2>/dev/null"; export LESS=" -R "
 +
 +
Then try something like:
 +
 +
less /usr/share/nstwui/apps/arp-scan/arp-scan.js
 +
less /usr/bin/lsusb.py
 +
 +
Unfortunately, source-highlight only works by filename extensions (it won't try to guess the input format based on the contents of the file).
  
 
= Find The Largest Files Within A File System =
 
= Find The Largest Files Within A File System =
Line 16: Line 67:
 
  75497472 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-000000000004d2bc-0004fbc9efdbc627.journal
 
  75497472 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-000000000004d2bc-0004fbc9efdbc627.journal
 
  64720632 /var/lib/clamav/main.cvd
 
  64720632 /var/lib/clamav/main.cvd
 +
 +
= Use SSH To Login On Remote System Using A Different New Shell =
 +
The command below demonstrates how to login on to a remote system using a different shell (i.e., ''/bin/ash''):
 +
 +
imac2012:~ rwhalb$ ssh -t root@10.222.222.8 /bin/ash
 +
root@10.222.222.8's password:
 +
Warning: untrusted X11 forwarding setup failed: xauth key data not generated
 +
 +
BusyBox v1.30.1 () built-in shell (ash)
 +
 +
~ # exit
 +
Connection to 10.222.222.8 closed.
 +
imac2012:~ rwhalb$
  
 
= Remove Incorrect Host Key from ~/.ssh/known_hosts (Delete 1 Line from File) =
 
= Remove Incorrect Host Key from ~/.ssh/known_hosts (Delete 1 Line from File) =
Line 22: Line 86:
  
 
   sed -i -e 12d ~/.ssh/known_hosts
 
   sed -i -e 12d ~/.ssh/known_hosts
 +
 +
Alternatively, you can add a ''rmsshhost'' function to your ''~/.bash_profile'':
 +
 +
  rmsshhost() {
 +
    sed -i -e ${1:-999999999}d ${2:-~/.ssh/known_hosts};
 +
  }
  
 
This is particularly useful in situations where ''ssh'' host keys are expected to change. For example, depending on which micro SD card is loaded on a Beagle Bone Black, it's host key might change. The following demonstrates the output from ''ssh'' when it detects this change in the host key (note how it reports the problem line as 54). The ''sed'' command is then used to quickly remove the old key.
 
This is particularly useful in situations where ''ssh'' host keys are expected to change. For example, depending on which micro SD card is loaded on a Beagle Bone Black, it's host key might change. The following demonstrates the output from ''ssh'' when it detects this change in the host key (note how it reports the problem line as 54). The ''sed'' command is then used to quickly remove the old key.
Line 53: Line 123:
 
  taco:~ pkb$ sed -i -e 54d ~/.ssh/known_hosts
 
  taco:~ pkb$ sed -i -e 54d ~/.ssh/known_hosts
 
  taco:~ pkb$
 
  taco:~ pkb$
 +
 +
Or, if using the ''rmsshhost'' function, you can remove line 54 using the following command:
 +
 +
rmsshhost 54
  
 
= Find File Differences in Two Directories =
 
= Find File Differences in Two Directories =
Line 64: Line 138:
 
  [root@rice 1.1.4]#
 
  [root@rice 1.1.4]#
  
= References =
+
= Modifying An ISO Image for Booting =
 +
The example below mounts an iso image and copies both the "'''EFI'''" and "'''isolinux'''" directories to a Read / Write directory: "'''/DATA/nstboot/'''" for the purpose of modifying isolinux and EFI booting:
 +
 
 +
[root@shopper2 iso]# mount -o loop ./nst-30-11210.x86_64.iso /mnt/iso/;
 +
[root@shopper2 iso]# cd /DATA/nstboot/;
 +
[root@shopper2 iso]# cp -aR /mnt/iso/EFI .
 +
[root@shopper2 iso]# cp -aR /mnt/iso/isolinux .
 +
[root@shopper2 iso]# ls -al /DATA/nstboot/
 +
total 16
 +
drwxr-xr-x 4 root root 4096 Jan  3 09:14 .
 +
drwxr-xr-x 9 root root 4096 Jan  3 09:08 ..
 +
dr-xr-xr-x 3 root root 4096 Jul 16 09:10 EFI
 +
dr-xr-xr-x 2 root root 4096 Jul 16 09:10 isolinux
 +
[root@shopper2 iso]#
 +
[root@shopper2 iso]# umount /mnt/iso;
 +
[root@shopper2 iso]#
  
* '''[https://en.wikipedia.org/wiki/Sed sed]''' one lines: "'''[http://www.linuxhowtos.org/System/sedoneliner.htm HANDY ONE-LINERS FOR SED]'''"
+
After making modifications, the following ''mkisofs'' command can be used to rebuild the ISO boot image for testing.
  
* Advanced '''[https://en.wikipedia.org/wiki/Sed sed]''' and '''[https://en.wikipedia.org/wiki/AWK awk]''' usage: "'''[https://posts.specterops.io/fawk-yeah-advanced-sed-and-awk-usage-parsing-for-pentesters-3-e5727e11a8ad Advanced Parsing for Pentesters]'''"
+
[root@shopper2 iso]# mkisofs -o /DATA/iso/nstboot.iso -b isolinux/isolinux.bin -J -R -l -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e isolinux/efiboot.img -no-emul-boot -graft-points -V "NST30-BOOT" /DATA/nstboot/;
 +
[root@shopper2 iso]#
 +
[root@shopper2 iso]# ls -al /DATA/iso/nstboot.iso;
 +
-rw-r--r-- 1 root root 110659584 Jan  3 09:17 /DATA/iso/nstboot.iso
 +
[root@shopper2 iso]#

Revision as of 13:13, 12 January 2020

Overview

This page provides a quick reference to common One Liner administrative command line operations.

One Liner Resources

Get Syntax Color In Less

The NST includes the source-highlight package which can "smartly" apply color to a wide variety of file formats. You can set some less environment variables to make use of the source-hightlight package to color code files in your terminal with the following settings:

export LESSOPEN="| source-highlight --out-format=esc -o STDOUT -i %s 2>/dev/null"; export LESS=" -R "

Then try something like:

less /usr/share/nstwui/apps/arp-scan/arp-scan.js
less /usr/bin/lsusb.py

Unfortunately, source-highlight only works by filename extensions (it won't try to guess the input format based on the contents of the file).

Find The Largest Files Within A File System

This example finds the 10 largest files, descending sorted, using the "/var" top level directory:

[root@vortex wui]# find /var -printf '%s %p\n' | sort -nr | head -10;
29956694633 /var/named/chroot/var/named/data/default_debug.log
182947840 /var/lib/rpm/Packages
134217728 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-00000000000b1d98-0005092323239c17.journal
125829120 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-000000000008eadb-000506c496be90cb.journal
125829120 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-00000000000251f3-0004f57678d900a6.journal
125829120 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-0000000000000001-0004f10922bc1e86.journal
95967232 /var/cache/yum/x86_64/20/fedora/gen/primary_db.sqlite
83886080 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-0000000000077d06-00050460486ab015.journal
75497472 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-000000000004d2bc-0004fbc9efdbc627.journal
64720632 /var/lib/clamav/main.cvd

Use SSH To Login On Remote System Using A Different New Shell

The command below demonstrates how to login on to a remote system using a different shell (i.e., /bin/ash):

imac2012:~ rwhalb$ ssh -t root@10.222.222.8 /bin/ash
root@10.222.222.8's password: 
Warning: untrusted X11 forwarding setup failed: xauth key data not generated

BusyBox v1.30.1 () built-in shell (ash)

~ # exit
Connection to 10.222.222.8 closed.
imac2012:~ rwhalb$

Remove Incorrect Host Key from ~/.ssh/known_hosts (Delete 1 Line from File)

The sed command can be very useful when you want to remove a specific line from a file. For example, the following command can be used to remove line 12 out of the file: ~/.ssh/known_hosts.

 sed -i -e 12d ~/.ssh/known_hosts

Alternatively, you can add a rmsshhost function to your ~/.bash_profile:

 rmsshhost() {
   sed -i -e ${1:-999999999}d ${2:-~/.ssh/known_hosts};
 }

This is particularly useful in situations where ssh host keys are expected to change. For example, depending on which micro SD card is loaded on a Beagle Bone Black, it's host key might change. The following demonstrates the output from ssh when it detects this change in the host key (note how it reports the problem line as 54). The sed command is then used to quickly remove the old key.

taco:~ pkb$ ssh salsa-e
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
fb:a7:a9:09:1a:f3:d2:4a:aa:89:9d:34:47:1c:d5:3c.
Please contact your system administrator.
Add correct host key in /Users/pkb/.ssh/known_hosts to get rid of this message.
Offending RSA key in /Users/pkb/.ssh/known_hosts:54
Password authentication is disabled to avoid man-in-the-middle attacks.
Keyboard-interactive authentication is disabled to avoid man-in-the-middle attacks.
Agent forwarding is disabled to avoid man-in-the-middle attacks.
X11 forwarding is disabled to avoid man-in-the-middle attacks.
Debian GNU/Linux 7

BeagleBoard.org Debian Image 2015-03-01

Support/FAQ: http://elinux.org/Beagleboard:BeagleBoneBlack_Debian

default username:password is [debian:temppwd]

Permission denied (publickey,password).
(reverse-i-search)`se': cd release/
taco:~ pkb$ sed -i -e 54d ~/.ssh/known_hosts
taco:~ pkb$

Or, if using the rmsshhost function, you can remove line 54 using the following command:

rmsshhost 54

Find File Differences in Two Directories

This one is handy when you have two directories (DIRA and DIRB) with a similar set of files and you want to determine if any of the files in DIRB are different than the files in DIRA. As an example, if you are looking for differences in your CSS files under the css directory (DIRA) with the css files in the 1.1.7 release found at ../1.1.7/css (DIRB).

[root@rice 1.1.4]# find css -type f | wc -l 
4 
[root@rice 1.1.4]# find css -type f | while read src; do cmp ${src} ../1.1.7/${src}; done
css/site.css ../1.1.7/css/site.css differ: byte 31, line 3
[root@rice 1.1.4]#

Modifying An ISO Image for Booting

The example below mounts an iso image and copies both the "EFI" and "isolinux" directories to a Read / Write directory: "/DATA/nstboot/" for the purpose of modifying isolinux and EFI booting:

[root@shopper2 iso]# mount -o loop ./nst-30-11210.x86_64.iso /mnt/iso/;
[root@shopper2 iso]# cd /DATA/nstboot/;
[root@shopper2 iso]# cp -aR /mnt/iso/EFI .
[root@shopper2 iso]# cp -aR /mnt/iso/isolinux .
[root@shopper2 iso]# ls -al /DATA/nstboot/
total 16
drwxr-xr-x 4 root root 4096 Jan  3 09:14 .
drwxr-xr-x 9 root root 4096 Jan  3 09:08 ..
dr-xr-xr-x 3 root root 4096 Jul 16 09:10 EFI
dr-xr-xr-x 2 root root 4096 Jul 16 09:10 isolinux
[root@shopper2 iso]#
[root@shopper2 iso]# umount /mnt/iso;
[root@shopper2 iso]#

After making modifications, the following mkisofs command can be used to rebuild the ISO boot image for testing.

[root@shopper2 iso]# mkisofs -o /DATA/iso/nstboot.iso -b isolinux/isolinux.bin -J -R -l -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e isolinux/efiboot.img -no-emul-boot -graft-points -V "NST30-BOOT" /DATA/nstboot/;
[root@shopper2 iso]#
[root@shopper2 iso]# ls -al /DATA/iso/nstboot.iso;
-rw-r--r-- 1 root root 110659584 Jan  3 09:17 /DATA/iso/nstboot.iso
[root@shopper2 iso]#