HowTo One Liners: Difference between revisions
(155 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
__TOC__ | __TOC__ | ||
= Overview = | = Overview = | ||
This page provides a quick reference to common '''One Liner''' administrative command line operations. | This page provides a quick reference to common '''One Liner''' administrative command line operations. | ||
== One Liner Resources == | |||
* '''[http://www.grymoire.com/Unix/Regular.html A Regular Expression Reference]''' | |||
* '''[https://regexr.com/ Regular Expression Online Tool]''' | |||
* '''[http://main.rtfiber.com.tw/~changyj/sed/ Sed and Regular Expressions]''' | |||
* '''[https://www.tutorialspoint.com/unix/unix-regular-expressions.htm Linux - Regular Expressions Tutorial with Sed]''' | |||
* '''[https://eloquentjavascript.net/09_regexp.html A Javascript Regular Expressions Tutorial] | |||
* Handy One Liners for Sed: '''[http://www.linuxhowtos.org/System/sedoneliner.htm Sed - LinuxHowtos]''', '''[http://sed.sourceforge.net/sed1line.txt Sed - SourceForge]''', '''[https://gist.github.com/jasonm23/396693/b9135d0dbe821d2ff8bd1b8a0a452cb27b4c2f68 Sed - Gist]''' | |||
* '''[https://www.gnu.org/software/sed/ GNU Sed Homepage]''' | |||
* '''[http://www.grymoire.com/Unix/Sed.html A Sed Tutorial]''' | |||
* '''[http://www.ibm.com/developerworks/linux/library/l-sed1/index.html Sed Part1]''', '''[http://www.ibm.com/developerworks/linux/library/l-sed2/index.html Sed Part2]''' and '''[http://www.ibm.com/developerworks/linux/library/l-sed3/index.html Sed Part3]''' | |||
* '''[https://posts.specterops.io/fawk-yeah-advanced-sed-and-awk-usage-parsing-for-pentesters-3-e5727e11a8ad Advanced Parsing for Pentesters]''' | |||
* '''[https://www.gnu.org/software/gawk/ GNU Gawk Homepage]''' | |||
* '''[https://www.gnu.org/software/gawk/manual/gawk.html The GNU Awk User’s Guide]''' | |||
* '''[http://www.grymoire.com/Unix/Awk.html An Awk Tutorial]''' | |||
* '''[http://www.thegeekstuff.com/2010/03/awk-arrays-explained-with-5-practical-examples Awk Array Examples]''' | |||
* '''[https://www.ibm.com/developerworks/library/l-lpic1-103-2/l-lpic1-103-2-pdf.pdf Text Streams and Filters]''' | |||
* '''[http://www.grymoire.com/Unix/Grep.html A Grep Tutorial]''' | |||
* '''[http://www.grymoire.com/Unix/Find.html A Find Example]''' | |||
* '''[http://www.grymoire.com/Unix/Tar.html A Tar Tutorial]''' | |||
= Get Syntax Color In Less = | |||
The NST includes the source-highlight package which can "smartly" apply color to a wide variety of file formats. You can set some '''less''' environment variables to make use of the source-hightlight package to color code files in your terminal with the following settings: | |||
export LESSOPEN="| source-highlight --out-format=esc -o STDOUT -i %s 2>/dev/null"; export LESS=" -R " | |||
Then try something like: | |||
less /usr/share/nstwui/apps/arp-scan/arp-scan.js | |||
less /usr/bin/lsusb.py | |||
Unfortunately, source-highlight only works by filename extensions (it won't try to guess the input format based on the contents of the file). | |||
= Find The Largest Files Within A File System = | = Find The Largest Files Within A File System = | ||
Line 16: | Line 67: | ||
75497472 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-000000000004d2bc-0004fbc9efdbc627.journal | 75497472 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-000000000004d2bc-0004fbc9efdbc627.journal | ||
64720632 /var/lib/clamav/main.cvd | 64720632 /var/lib/clamav/main.cvd | ||
= Use SSH To Login On Remote System Using A Different New Shell = | |||
The command below demonstrates how to login on to a remote system using a different shell (i.e., ''/bin/ash''): | |||
imac2012:~ rwhalb$ ssh -t root@10.222.222.8 /bin/ash | |||
root@10.222.222.8's password: | |||
Warning: untrusted X11 forwarding setup failed: xauth key data not generated | |||
BusyBox v1.30.1 () built-in shell (ash) | |||
~ # exit | |||
Connection to 10.222.222.8 closed. | |||
imac2012:~ rwhalb$ | |||
= Used sed to Remove All Carriage Lines (\r, ^M) from a File = | |||
sed -i -e 's/\r//g' <file> | |||
= Use sed to Remove the Next Line from a Previous Line Match = | |||
Lets say you need to remove a specific line from a file that is replicated throughout the file: | |||
Example: The second '''#endif''' line from a file: "'''types.h'''" will be removed after matching line: '''typedef uint64_t __u64;''' | |||
. | |||
. | |||
. | |||
#ifndef LIBZWAVE_EXCLUDE_TYPES | |||
typedef ZWAVE_NULL 0; | |||
#endif | |||
#ifndef LIBWIFIUSERSPACE_EXCLUDE_TYPES | |||
typedef uint8_t __u8; | |||
typedef uint16_t __u16; | |||
typedef uint32_t __u32; | |||
typedef uint64_t __u64; | |||
#endif | |||
. | |||
. | |||
. | |||
Use this sed one liner to remove the second '''#endif''' after line: '''typedef uint64_t __u64;''': | |||
sed -i -e '/typedef uint64_t __u64;/{n;d;}' "./types.h"; | |||
Results: | |||
. | |||
. | |||
. | |||
#ifndef LIBZWAVE_EXCLUDE_TYPES | |||
typedef ZWAVE_NULL 0; | |||
#endif | |||
#ifndef LIBWIFIUSERSPACE_EXCLUDE_TYPES | |||
typedef uint8_t __u8; | |||
typedef uint16_t __u16; | |||
typedef uint32_t __u32; | |||
typedef uint64_t __u64; | |||
. | |||
. | |||
. | |||
= Use sed to Remove Multiple Next Lines from a Previous Line Match = | |||
Lets say you need to remove multiple lines from a file after a previous line match: | |||
Example: Remove lines containing: '''4''', '''6''' and '''8''' from a file: "'''letters.txt'''" after matching line: '''abc1''' | |||
. | |||
. | |||
. | |||
abc1 | |||
4 | |||
6 | |||
8 | |||
9 | |||
. | |||
. | |||
. | |||
Use this sed one liner: | |||
sed -i -e '/abc1/{n;N;N;d;}' "./letters.txt"; | |||
Results: | |||
. | |||
. | |||
. | |||
abc1 | |||
9 | |||
. | |||
. | |||
. | |||
= Remove Incorrect Host Key from ~/.ssh/known_hosts (Delete 1 Line from File) = | = Remove Incorrect Host Key from ~/.ssh/known_hosts (Delete 1 Line from File) = | ||
Line 22: | Line 158: | ||
sed -i -e 12d ~/.ssh/known_hosts | sed -i -e 12d ~/.ssh/known_hosts | ||
Alternatively, you can add a ''rmsshhost'' function to your ''~/.bash_profile'': | |||
rmsshhost() { | |||
sed -i -e ${1:-999999999}d ${2:-~/.ssh/known_hosts}; | |||
} | |||
This is particularly useful in situations where ''ssh'' host keys are expected to change. For example, depending on which micro SD card is loaded on a Beagle Bone Black, it's host key might change. The following demonstrates the output from ''ssh'' when it detects this change in the host key (note how it reports the problem line as 54). The ''sed'' command is then used to quickly remove the old key. | This is particularly useful in situations where ''ssh'' host keys are expected to change. For example, depending on which micro SD card is loaded on a Beagle Bone Black, it's host key might change. The following demonstrates the output from ''ssh'' when it detects this change in the host key (note how it reports the problem line as 54). The ''sed'' command is then used to quickly remove the old key. | ||
Line 53: | Line 195: | ||
taco:~ pkb$ sed -i -e 54d ~/.ssh/known_hosts | taco:~ pkb$ sed -i -e 54d ~/.ssh/known_hosts | ||
taco:~ pkb$ | taco:~ pkb$ | ||
Or, if using the ''rmsshhost'' function, you can remove line 54 using the following command: | |||
rmsshhost 54 | |||
= Find File Differences in Two Directories = | = Find File Differences in Two Directories = | ||
Line 64: | Line 210: | ||
[root@rice 1.1.4]# | [root@rice 1.1.4]# | ||
= | = Modifying An ISO Disk Image for Booting = | ||
The example below mounts an iso disk image and copies both the "'''EFI'''" and "'''isolinux'''" directories to a Read / Write directory: "'''/DATA/nstboot/'''" for the purpose of modifying isolinux and EFI booting: | |||
[root@shopper2 iso]# mount -o loop ./nst-30-11210.x86_64.iso /mnt/iso/; | |||
[root@shopper2 iso]# cd /DATA/nstboot/; | |||
[root@shopper2 iso]# cp -aR /mnt/iso/EFI . | |||
[root@shopper2 iso]# cp -aR /mnt/iso/isolinux . | |||
[root@shopper2 iso]# ls -al /DATA/nstboot/ | |||
total 16 | |||
drwxr-xr-x 4 root root 4096 Jan 3 09:14 . | |||
drwxr-xr-x 9 root root 4096 Jan 3 09:08 .. | |||
dr-xr-xr-x 3 root root 4096 Jul 16 09:10 EFI | |||
dr-xr-xr-x 2 root root 4096 Jul 16 09:10 isolinux | |||
[root@shopper2 iso]# | |||
[root@shopper2 iso]# umount /mnt/iso; | |||
[root@shopper2 iso]# | |||
After making modifications, the following ''mkisofs'' command can be used to rebuild the ISO boot image for testing. | |||
[root@shopper2 iso]# mkisofs -o /DATA/iso/nstboot.iso -b isolinux/isolinux.bin -J -R -l -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e isolinux/efiboot.img -no-emul-boot -graft-points -V "NST30-BOOT" /DATA/nstboot/; | |||
[root@shopper2 iso]# | |||
[root@shopper2 iso]# ls -al /DATA/iso/nstboot.iso; | |||
-rw-r--r-- 1 root root 110659584 Jan 3 09:17 /DATA/iso/nstboot.iso | |||
[root@shopper2 iso]# | |||
= Locate Command = | |||
The '''[https://en.wikipedia.org/wiki/Locate_(Unix)#:~:text=locate%20is%20a%20Unix%20utility,and%20compressed%20using%20incremental%20encoding. locate]''' command is a linux utility which serves to find files on filesystems. It searches through a prebuilt database: "'''/var/lib/mlocate/mlocate.db'''" generated by the ''updatedb'' command or by a systemd service: "'''/usr/lib/systemd/system/mlocate-updatedb.service'''" and timer unit: "'''/usr/lib/systemd/system/mlocate-updatedb.timer'''" which is compressed using incremental encoding. | |||
The example below uses the ''locate'' command to search for '''htop''' files using the regular expression: "'''/htop$'''" | |||
[root@shopper2 ~]# locate --regexp '/htop$' | |||
/etc/pcp/pmlogconf/tools/htop | |||
/root/.config/htop | |||
/usr/bin/htop | |||
/usr/share/bash-completion/completions/htop | |||
/usr/share/doc/htop | |||
/usr/share/licenses/htop | |||
/var/lib/pcp/config/pmlogconf/tools/htop | |||
The output below demonstrates how to ''manually'' update the default searched database: "'''/var/lib/mlocate/mlocate.db'''" using the utility: "'''/usr/libexec/mlocate-run-updatedb'''" | |||
[root@shopper2 ~]# ls -al /var/lib/mlocate/mlocate.db; | |||
-rw-r----- 1 root slocate 13715680 Aug 16 20:17 /var/lib/mlocate/mlocate.db | |||
[root@shopper2 ~]# /usr/libexec/mlocate-run-updatedb; | |||
[root@shopper2 ~]# ls -al /var/lib/mlocate/mlocate.db; | |||
-rw-r----- 1 root slocate 13715680 Aug 16 20:18 /var/lib/mlocate/mlocate.db | |||
= Ncat One Liners = | |||
== Overview == | |||
'''[https://nmap.org/ncat/ Ncat]''' is a similar tool to '''[https://en.wikipedia.org/wiki/Netcat netcat]''' provided by '''[https://en.wikipedia.org/wiki/Nmap Nmap]''' suite. Ncat features includes: ability to chain Ncats together, redirect both TCP and UDP ports to other sites, SSL support, and proxy connections via SOCKS4 or HTTP (CONNECT method) proxies (with optional proxy authentication as well). | |||
Ncat always operates in one of two basic modes: ''connect mode'' and ''listen mode''. In connect mode, Ncat initiates a connection (or sends UDP data) to a service that is listening somewhere. For those familiar with socket programming, connect mode is like using the connect function. In listen mode, Ncat waits for an incoming connection (or data receipt), like using the bind and listen functions. You can think of connect mode as “client” mode and listen mode as “server” mode. | |||
To use Ncat in connect mode, run | |||
'''ncat <host> [<port>]''' | |||
''<host>'' may be a hostname or IP address, and ''<port>'' is a port number. Listen mode is the same, with the addition of the --listen option (or its -l alias): | |||
'''ncat --listen [<host>] [<port>]''' | |||
'''ncat -l [<host>] [<port>]''' | |||
In listen mode, ''<host>'' controls the address on which Ncat listens; if you omit it, Ncat will bind to all local interfaces (INADDR_ANY). If the port number is omitted, Ncat uses its default port 31337. Typically only privileged (root) users may bind to a port number lower than 1024. A listening TCP server normally accepts only one connection and will exit after the client disconnects. Combined with the ''--keep-open'' option, Ncat accepts multiple concurrent connections up to the connection limit. With ''--keep-open'' (or -k for short), the server receives everything sent by any of its clients, and anything the server sends is sent to all of them. A UDP server will communicate with only one client (the first one to send it data), because in UDP there is no list of “connected” clients. | |||
See the '''[https://nmap.org/ncat/guide/ncat-usage.html Ncat Basic Users' Guide]''' for a detailed infomation. | |||
By default, Ncat uses TCP. The option --udp or -u enables UDP instead, and --sctp enables SCTP. Ncat listens on both IPv4 and IPv6, and connects to either address family as well. The -6 option forces IPv6-only, and -4 forces IPv4-only. For a quick summary of options at any time, run ''ncat --help'' or '''[http://man7.org/linux/man-pages/man1/ncat.1.html man ncat]'''. | |||
== Ncat As a Web Browser == | |||
After the connection type in "'''GET / HTTP/1.0'''" followed by two(2) '''enter''' key strokes. | |||
[root@10.222.222.10 ~]# ncat -C www.openwrt.org 80; | |||
GET / HTTP/1.0 | |||
HTTP/1.1 301 Moved Permanently | |||
Server: nginx | |||
Date: Mon, 10 Feb 2020 13:44:37 GMT | |||
Content-Type: text/html | |||
Content-Length: 178 | |||
Connection: close | |||
Location: https://openwrt.org/ | |||
<nowiki> | |||
<html> | |||
<head><title>301 Moved Permanently</title></head> | |||
<body bgcolor="white"> | |||
<center><h1>301 Moved Permanently</h1></center> | |||
<hr><center>nginx</center> | |||
</body> | |||
</html> | |||
</nowiki> | |||
== SMTP Connection == | |||
After the connection type in SMTP command (helo <domain>): "'''helo verizon.net'''" followed by one(1) '''enter''' key stroke. | |||
[root@10.222.222.10 tmp]# ncat -C smtp.aol.com 587; | |||
220 smtp.mail.yahoo.com ESMTP ready | |||
helo verizon.net | |||
250 smtp406.mail.bf1.yahoo.com Hello verizon.net [71.164.79.94]) | |||
^C | |||
[root@10.222.222.10 tmp]# | |||
== Command Execution TCP/IP == | |||
Run a command with '''--exec''' using TCP/IP port: '''30000''' on host: 10.222.222.10 (i.e., Server Side). Execute a TCP/IP connection on host: 10.222.222.22 (i.e., Client Side) to '''10.222.222.10:30000''': | |||
Server Side: '''10.222.222.10''' | |||
[root@10.222.222.10 ~]# ncat -l -k -p 30000 --exec "/bin/echo Hello."; | |||
Check if TCP/IP port: '''30000''' is active on host: '''10.222.222.10''': | |||
[root@10.222.222.10 ~]# netstat -tnap | grep 30000; | |||
tcp 0 0 0.0.0.0:30000 0.0.0.0:* LISTEN 6076/ncat | |||
tcp6 0 0 :::30000 :::* LISTEN 6076/ncat | |||
Client Side: '''10.222.222.22''' | |||
[root@10.222.222.22 ~]# ncat 10.222.222.10 30000; | |||
Hello. | |||
^C | |||
[root@10.222.222.22 ~]# | |||
== Command Execution UDP == | |||
Run a command with '''--exec''' using UDP port: '''33333''' on host: 10.222.222.10 (i.e., Server Side). Execute a UDP connection on host: 10.222.222.22 (i.e., Client Side) to '''10.222.222.10:33333''': | |||
Server Side: '''10.222.222.10''' | |||
[root@10.222.222.10 ~]# ncat -l -u -k -p 33333 --exec "/bin/echo Hello."; | |||
Check if UDP port: '''33333''' is active on host: '''10.222.222.10''': | |||
[root@10.222.222.10 ~]# netstat -unap | grep 33333; | |||
tcp 0 0 0.0.0.0:33333 0.0.0.0:* LISTEN 968/ncat | |||
tcp6 0 0 :::33333 :::* LISTEN 968/ncat | |||
Client Side: '''10.222.222.22''' | |||
[root@10.222.222.22 ~]# ncat -C -u 10.222.222.10 33333; | |||
<enter> | |||
Hello. | |||
^C | |||
[root@10.222.222.22 ~]# | |||
== Serial Over TCP/IP - A Remote Serial Stream (stty, ncat) == | |||
In this example we will used a GPS serial stream (4800 Baud) connected to USB-to-Serial adapter: "'''/dev/ttyUSB1'''" on host: "'''10.222.222.23'''" and use ncat to establish a TCP/IP listening socket on port: "'''22222'''". On host: "'''10.222.222.10'''" we will connect via TCP/IP to this service on host: "'''10.222.222.23'''". | |||
On host: "'''10.222.222.23'''" | |||
* First set the serial baud rate and any special settings for the USB-to-Serial device: "'''/dev/ttyUSB1'''" using the "'''[http://man7.org/linux/man-pages/man1/stty.1.html stty]'''" utility: | |||
[root@10.222.222.23 ~]# stty -F /dev/ttyUSB1 4800 raw; | |||
* Next establish the TCP/IP GPS service on port: "'''22222'''" via ncat: | |||
[root@10.222.222.23 ~]# ncat -l 22222 > /dev/ttyUSB1 < /dev/ttyUSB1; | |||
On host: "'''10.222.222.10'''" | |||
* Connect to the TCP/IP GPS service on host: "'''10.222.222.23'''" using ncat: | |||
[root@10.222.222.10 ~]# ncat 10.222.222.23 22222; | |||
$PGRME,14.4,M,21.0,M,25.4,M*1F | |||
$GPRMC,132904,A,4241.6671,N,07351.6808,W,000.0,239.2,110220,014.2,W,A*18 | |||
$GPGGA,132904,4241.6671,N,07351.6808,W,1,04,2.1,109.6,M,-33.5,M,,*7F | |||
$GPGSA,M,2,10,,12,,,24,,,32,,,,2.4,2.1,1.0*3E | |||
$GPGSV,3,2,11,20,48,150,00,24,33,051,32,25,25,137,00,31,14,218,00*78 | |||
$PGRME,14.4,M,21.0,M,25.4,M*1F | |||
$GPRMC,132905,A,4241.6673,N,07351.6809,W,000.0,239.2,110220,014.2,W,A*1A | |||
$GPGGA,132905,4241.6673,N,07351.6809,W,1,04,2.1,110.1,M,-33.5,M,,*72 | |||
$GPGSA,M,2,10,,12,,,24,,,32,,,,2.4,2.1,1.0*3E | |||
$GPGSV,3,3,11,32,59,297,37,34,00,000,00,35,00,000,00*4D | |||
^C | |||
[root@10.222.222.10 ~]# | |||
== Serial Over TCP/IP - A Remote Serial Login Console (stty, ncat, socat, minicom) == | |||
In this example we will used a Raspberry PI 4 serial console connection (115200 Baud) attached to USB-to-Serial adapter: "'''/dev/ttyUSB0'''" on host: "'''172.16.1.245'''" and use ncat to establish a TCP/IP listening socket on port: "'''22222'''". On host: "'''172.16.1.71'''" we will connect via TCP/IP to this console service on remote host: "'''172.16.1.245'''". The '''[https://linux.die.net/man/1/socat socat]''' utility and the terminal emulator "'''[https://en.wikipedia.org/wiki/Minicom minicom]'''" application will be used to establish an interactive session to the remote Raspberry Pi 4 serial login console. | |||
On host: "'''172.16.1.245'''" | |||
* First set the serial baud rate and any special settings for the USB-to-Serial device: "'''/dev/ttyUSB0'''" using the "'''[http://man7.org/linux/man-pages/man1/stty.1.html stty]'''" utility: | |||
[root@172.16.1.245 ~]# stty -F /dev/ttyUSB0 115200 raw; | |||
* Next establish the TCP/IP Raspberry Pi 4 serial console service on port: "'''22222'''" via ncat and put the process in the background: | |||
[root@172.16.1.245 ~]# ncat -l 22222 > /dev/ttyUSB0 < /dev/ttyUSB0 & | |||
On host: "'''172.16.1.71'''" | |||
* Connect to the TCP/IP Raspberry Pi 4 serial console service on host: "'''172.16.1.245'''", port: "'''22222'''" using socat and put the process in the background. The pseudo terminal: "'''/dev/ttyV0'''" will be created by socat for attaching minicom. | |||
[root@172.16.1.245 ~]# socat pty,link=/dev/ttyV0 tcp:172.16.1.245:22222 & | |||
* Create a minicom configuration in file: "'''/etc/minirc.ttyV0'''": | |||
[root@172.16.1.71]# cat /etc/minirc.ttyV0 | |||
# | |||
# Raspbery Pi 4 Serial Login Console | |||
pu port /dev/ttyV0 | |||
pu baudrate 115200 | |||
pu bits 8 | |||
pu parity N | |||
pu stopbits 1 | |||
pu rtscts No | |||
pu xonxoff No | |||
# | |||
# Disable modem initialization values... | |||
pu minit | |||
pu mreset | |||
pu mdialpre | |||
pu mdialsuf | |||
pu mdialpre2 | |||
pu mdialsuf2 | |||
pu mdialpre3 | |||
pu mdialsuf3 | |||
pu mhangup | |||
pu mdialcan | |||
* Next attach minicom to the newly created pseudo terminal: "'''/dev/V0'''" - An interactive serial login console session to a Raspberry Pi 4 attached to remote system: "'''172.16.1.245'''". The user: "'''pi'''" will be used for logging in. | |||
[root@172.16.1.71]# minicom -w -c on -t xterm ttyV0; | |||
Welcome to minicom 2.7.1 | |||
OPTIONS: I18n | |||
Compiled on Feb 10 2020, 00:00:00. | |||
Port /dev/ttyV0, 12:27:47 | |||
Press CTRL-A Z for help on special keys | |||
rpi4 login: pi | |||
Password: ************ | |||
Last login: Fri Oct 2 11:34:47 EDT 2020 on ttyS0 | |||
Linux rpi4 5.4.51-v7l+ #1327 SMP Thu Jul 23 11:04:39 BST 2020 armv7l | |||
The programs included with the Debian GNU/Linux system are free software; | |||
the exact distribution terms for each program are described in the | |||
individual files in /usr/share/doc/*/copyright. | |||
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent | |||
permitted by applicable law. | |||
pi@rpi4:~$ lscpu | |||
Architecture: armv7l | |||
Byte Order: Little Endian | |||
CPU(s): 4 | |||
On-line CPU(s) list: 0-3 | |||
Thread(s) per core: 1 | |||
Core(s) per socket: 4 | |||
Socket(s): 1 | |||
Vendor ID: ARM | |||
Model: 3 | |||
Model name: Cortex-A72 | |||
Stepping: r0p3 | |||
CPU max MHz: 1500.0000 | |||
CPU min MHz: 600.0000 | |||
BogoMIPS: 108.00 | |||
Flags: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 | |||
pi@rpi4:~$ exit | |||
logout | |||
Raspbian GNU/Linux 10 rpi4 ttyS0 | |||
rpi4 login: | |||
== Tunneling UDP Traffic Through An SSH Connection == | |||
See article: [[Tunnelling_UDP_Traffic_Through_An_SSH_Connection | Tunneling UDP Traffic Through An SSH Connection]] |
Latest revision as of 16:53, 25 August 2024
Overview
This page provides a quick reference to common One Liner administrative command line operations.
One Liner Resources
- Handy One Liners for Sed: Sed - LinuxHowtos, Sed - SourceForge, Sed - Gist
Get Syntax Color In Less
The NST includes the source-highlight package which can "smartly" apply color to a wide variety of file formats. You can set some less environment variables to make use of the source-hightlight package to color code files in your terminal with the following settings:
export LESSOPEN="| source-highlight --out-format=esc -o STDOUT -i %s 2>/dev/null"; export LESS=" -R "
Then try something like:
less /usr/share/nstwui/apps/arp-scan/arp-scan.js less /usr/bin/lsusb.py
Unfortunately, source-highlight only works by filename extensions (it won't try to guess the input format based on the contents of the file).
Find The Largest Files Within A File System
This example finds the 10 largest files, descending sorted, using the "/var" top level directory:
[root@vortex wui]# find /var -printf '%s %p\n' | sort -nr | head -10; 29956694633 /var/named/chroot/var/named/data/default_debug.log 182947840 /var/lib/rpm/Packages 134217728 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-00000000000b1d98-0005092323239c17.journal 125829120 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-000000000008eadb-000506c496be90cb.journal 125829120 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-00000000000251f3-0004f57678d900a6.journal 125829120 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-0000000000000001-0004f10922bc1e86.journal 95967232 /var/cache/yum/x86_64/20/fedora/gen/primary_db.sqlite 83886080 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-0000000000077d06-00050460486ab015.journal 75497472 /var/log/journal/597d443ff603490286135ca186ed9c7d/system@f9cb0e593f6c413d8fdfaa88bd1c9f42-000000000004d2bc-0004fbc9efdbc627.journal 64720632 /var/lib/clamav/main.cvd
Use SSH To Login On Remote System Using A Different New Shell
The command below demonstrates how to login on to a remote system using a different shell (i.e., /bin/ash):
imac2012:~ rwhalb$ ssh -t root@10.222.222.8 /bin/ash root@10.222.222.8's password: Warning: untrusted X11 forwarding setup failed: xauth key data not generated BusyBox v1.30.1 () built-in shell (ash) ~ # exit Connection to 10.222.222.8 closed. imac2012:~ rwhalb$
Used sed to Remove All Carriage Lines (\r, ^M) from a File
sed -i -e 's/\r//g' <file>
Use sed to Remove the Next Line from a Previous Line Match
Lets say you need to remove a specific line from a file that is replicated throughout the file:
Example: The second #endif line from a file: "types.h" will be removed after matching line: typedef uint64_t __u64;
. . . #ifndef LIBZWAVE_EXCLUDE_TYPES typedef ZWAVE_NULL 0; #endif #ifndef LIBWIFIUSERSPACE_EXCLUDE_TYPES typedef uint8_t __u8; typedef uint16_t __u16; typedef uint32_t __u32; typedef uint64_t __u64; #endif . . .
Use this sed one liner to remove the second #endif after line: typedef uint64_t __u64;:
sed -i -e '/typedef uint64_t __u64;/{n;d;}' "./types.h";
Results:
. . . #ifndef LIBZWAVE_EXCLUDE_TYPES typedef ZWAVE_NULL 0; #endif #ifndef LIBWIFIUSERSPACE_EXCLUDE_TYPES typedef uint8_t __u8; typedef uint16_t __u16; typedef uint32_t __u32; typedef uint64_t __u64; . . .
Use sed to Remove Multiple Next Lines from a Previous Line Match
Lets say you need to remove multiple lines from a file after a previous line match:
Example: Remove lines containing: 4, 6 and 8 from a file: "letters.txt" after matching line: abc1
. . . abc1 4 6 8 9 . . .
Use this sed one liner:
sed -i -e '/abc1/{n;N;N;d;}' "./letters.txt";
Results:
. . . abc1 9 . . .
Remove Incorrect Host Key from ~/.ssh/known_hosts (Delete 1 Line from File)
The sed command can be very useful when you want to remove a specific line from a file. For example, the following command can be used to remove line 12 out of the file: ~/.ssh/known_hosts.
sed -i -e 12d ~/.ssh/known_hosts
Alternatively, you can add a rmsshhost function to your ~/.bash_profile:
rmsshhost() { sed -i -e ${1:-999999999}d ${2:-~/.ssh/known_hosts}; }
This is particularly useful in situations where ssh host keys are expected to change. For example, depending on which micro SD card is loaded on a Beagle Bone Black, it's host key might change. The following demonstrates the output from ssh when it detects this change in the host key (note how it reports the problem line as 54). The sed command is then used to quickly remove the old key.
taco:~ pkb$ ssh salsa-e @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the RSA key sent by the remote host is fb:a7:a9:09:1a:f3:d2:4a:aa:89:9d:34:47:1c:d5:3c. Please contact your system administrator. Add correct host key in /Users/pkb/.ssh/known_hosts to get rid of this message. Offending RSA key in /Users/pkb/.ssh/known_hosts:54 Password authentication is disabled to avoid man-in-the-middle attacks. Keyboard-interactive authentication is disabled to avoid man-in-the-middle attacks. Agent forwarding is disabled to avoid man-in-the-middle attacks. X11 forwarding is disabled to avoid man-in-the-middle attacks. Debian GNU/Linux 7 BeagleBoard.org Debian Image 2015-03-01 Support/FAQ: http://elinux.org/Beagleboard:BeagleBoneBlack_Debian default username:password is [debian:temppwd] Permission denied (publickey,password). (reverse-i-search)`se': cd release/ taco:~ pkb$ sed -i -e 54d ~/.ssh/known_hosts taco:~ pkb$
Or, if using the rmsshhost function, you can remove line 54 using the following command:
rmsshhost 54
Find File Differences in Two Directories
This one is handy when you have two directories (DIRA and DIRB) with a similar set of files and you want to determine if any of the files in DIRB are different than the files in DIRA. As an example, if you are looking for differences in your CSS files under the css directory (DIRA) with the css files in the 1.1.7 release found at ../1.1.7/css (DIRB).
[root@rice 1.1.4]# find css -type f | wc -l 4 [root@rice 1.1.4]# find css -type f | while read src; do cmp ${src} ../1.1.7/${src}; done css/site.css ../1.1.7/css/site.css differ: byte 31, line 3 [root@rice 1.1.4]#
Modifying An ISO Disk Image for Booting
The example below mounts an iso disk image and copies both the "EFI" and "isolinux" directories to a Read / Write directory: "/DATA/nstboot/" for the purpose of modifying isolinux and EFI booting:
[root@shopper2 iso]# mount -o loop ./nst-30-11210.x86_64.iso /mnt/iso/; [root@shopper2 iso]# cd /DATA/nstboot/; [root@shopper2 iso]# cp -aR /mnt/iso/EFI . [root@shopper2 iso]# cp -aR /mnt/iso/isolinux . [root@shopper2 iso]# ls -al /DATA/nstboot/ total 16 drwxr-xr-x 4 root root 4096 Jan 3 09:14 . drwxr-xr-x 9 root root 4096 Jan 3 09:08 .. dr-xr-xr-x 3 root root 4096 Jul 16 09:10 EFI dr-xr-xr-x 2 root root 4096 Jul 16 09:10 isolinux [root@shopper2 iso]# [root@shopper2 iso]# umount /mnt/iso; [root@shopper2 iso]#
After making modifications, the following mkisofs command can be used to rebuild the ISO boot image for testing.
[root@shopper2 iso]# mkisofs -o /DATA/iso/nstboot.iso -b isolinux/isolinux.bin -J -R -l -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e isolinux/efiboot.img -no-emul-boot -graft-points -V "NST30-BOOT" /DATA/nstboot/; [root@shopper2 iso]# [root@shopper2 iso]# ls -al /DATA/iso/nstboot.iso; -rw-r--r-- 1 root root 110659584 Jan 3 09:17 /DATA/iso/nstboot.iso [root@shopper2 iso]#
Locate Command
The locate command is a linux utility which serves to find files on filesystems. It searches through a prebuilt database: "/var/lib/mlocate/mlocate.db" generated by the updatedb command or by a systemd service: "/usr/lib/systemd/system/mlocate-updatedb.service" and timer unit: "/usr/lib/systemd/system/mlocate-updatedb.timer" which is compressed using incremental encoding.
The example below uses the locate command to search for htop files using the regular expression: "/htop$"
[root@shopper2 ~]# locate --regexp '/htop$' /etc/pcp/pmlogconf/tools/htop /root/.config/htop /usr/bin/htop /usr/share/bash-completion/completions/htop /usr/share/doc/htop /usr/share/licenses/htop /var/lib/pcp/config/pmlogconf/tools/htop
The output below demonstrates how to manually update the default searched database: "/var/lib/mlocate/mlocate.db" using the utility: "/usr/libexec/mlocate-run-updatedb"
[root@shopper2 ~]# ls -al /var/lib/mlocate/mlocate.db; -rw-r----- 1 root slocate 13715680 Aug 16 20:17 /var/lib/mlocate/mlocate.db
[root@shopper2 ~]# /usr/libexec/mlocate-run-updatedb;
[root@shopper2 ~]# ls -al /var/lib/mlocate/mlocate.db; -rw-r----- 1 root slocate 13715680 Aug 16 20:18 /var/lib/mlocate/mlocate.db
Ncat One Liners
Overview
Ncat is a similar tool to netcat provided by Nmap suite. Ncat features includes: ability to chain Ncats together, redirect both TCP and UDP ports to other sites, SSL support, and proxy connections via SOCKS4 or HTTP (CONNECT method) proxies (with optional proxy authentication as well).
Ncat always operates in one of two basic modes: connect mode and listen mode. In connect mode, Ncat initiates a connection (or sends UDP data) to a service that is listening somewhere. For those familiar with socket programming, connect mode is like using the connect function. In listen mode, Ncat waits for an incoming connection (or data receipt), like using the bind and listen functions. You can think of connect mode as “client” mode and listen mode as “server” mode.
To use Ncat in connect mode, run
ncat <host> [<port>]
<host> may be a hostname or IP address, and <port> is a port number. Listen mode is the same, with the addition of the --listen option (or its -l alias):
ncat --listen [<host>] [<port>] ncat -l [<host>] [<port>]
In listen mode, <host> controls the address on which Ncat listens; if you omit it, Ncat will bind to all local interfaces (INADDR_ANY). If the port number is omitted, Ncat uses its default port 31337. Typically only privileged (root) users may bind to a port number lower than 1024. A listening TCP server normally accepts only one connection and will exit after the client disconnects. Combined with the --keep-open option, Ncat accepts multiple concurrent connections up to the connection limit. With --keep-open (or -k for short), the server receives everything sent by any of its clients, and anything the server sends is sent to all of them. A UDP server will communicate with only one client (the first one to send it data), because in UDP there is no list of “connected” clients.
See the Ncat Basic Users' Guide for a detailed infomation.
By default, Ncat uses TCP. The option --udp or -u enables UDP instead, and --sctp enables SCTP. Ncat listens on both IPv4 and IPv6, and connects to either address family as well. The -6 option forces IPv6-only, and -4 forces IPv4-only. For a quick summary of options at any time, run ncat --help or man ncat.
Ncat As a Web Browser
After the connection type in "GET / HTTP/1.0" followed by two(2) enter key strokes.
[root@10.222.222.10 ~]# ncat -C www.openwrt.org 80; GET / HTTP/1.0 HTTP/1.1 301 Moved Permanently Server: nginx Date: Mon, 10 Feb 2020 13:44:37 GMT Content-Type: text/html Content-Length: 178 Connection: close Location: https://openwrt.org/ <html> <head><title>301 Moved Permanently</title></head> <body bgcolor="white"> <center><h1>301 Moved Permanently</h1></center> <hr><center>nginx</center> </body> </html>
SMTP Connection
After the connection type in SMTP command (helo <domain>): "helo verizon.net" followed by one(1) enter key stroke.
[root@10.222.222.10 tmp]# ncat -C smtp.aol.com 587; 220 smtp.mail.yahoo.com ESMTP ready helo verizon.net 250 smtp406.mail.bf1.yahoo.com Hello verizon.net [71.164.79.94]) ^C [root@10.222.222.10 tmp]#
Command Execution TCP/IP
Run a command with --exec using TCP/IP port: 30000 on host: 10.222.222.10 (i.e., Server Side). Execute a TCP/IP connection on host: 10.222.222.22 (i.e., Client Side) to 10.222.222.10:30000:
Server Side: 10.222.222.10
[root@10.222.222.10 ~]# ncat -l -k -p 30000 --exec "/bin/echo Hello.";
Check if TCP/IP port: 30000 is active on host: 10.222.222.10:
[root@10.222.222.10 ~]# netstat -tnap | grep 30000; tcp 0 0 0.0.0.0:30000 0.0.0.0:* LISTEN 6076/ncat tcp6 0 0 :::30000 :::* LISTEN 6076/ncat
Client Side: 10.222.222.22
[root@10.222.222.22 ~]# ncat 10.222.222.10 30000; Hello. ^C [root@10.222.222.22 ~]#
Command Execution UDP
Run a command with --exec using UDP port: 33333 on host: 10.222.222.10 (i.e., Server Side). Execute a UDP connection on host: 10.222.222.22 (i.e., Client Side) to 10.222.222.10:33333:
Server Side: 10.222.222.10
[root@10.222.222.10 ~]# ncat -l -u -k -p 33333 --exec "/bin/echo Hello.";
Check if UDP port: 33333 is active on host: 10.222.222.10:
[root@10.222.222.10 ~]# netstat -unap | grep 33333; tcp 0 0 0.0.0.0:33333 0.0.0.0:* LISTEN 968/ncat tcp6 0 0 :::33333 :::* LISTEN 968/ncat
Client Side: 10.222.222.22
[root@10.222.222.22 ~]# ncat -C -u 10.222.222.10 33333; <enter> Hello. ^C [root@10.222.222.22 ~]#
Serial Over TCP/IP - A Remote Serial Stream (stty, ncat)
In this example we will used a GPS serial stream (4800 Baud) connected to USB-to-Serial adapter: "/dev/ttyUSB1" on host: "10.222.222.23" and use ncat to establish a TCP/IP listening socket on port: "22222". On host: "10.222.222.10" we will connect via TCP/IP to this service on host: "10.222.222.23".
On host: "10.222.222.23"
- First set the serial baud rate and any special settings for the USB-to-Serial device: "/dev/ttyUSB1" using the "stty" utility:
[root@10.222.222.23 ~]# stty -F /dev/ttyUSB1 4800 raw;
- Next establish the TCP/IP GPS service on port: "22222" via ncat:
[root@10.222.222.23 ~]# ncat -l 22222 > /dev/ttyUSB1 < /dev/ttyUSB1;
On host: "10.222.222.10"
- Connect to the TCP/IP GPS service on host: "10.222.222.23" using ncat:
[root@10.222.222.10 ~]# ncat 10.222.222.23 22222; $PGRME,14.4,M,21.0,M,25.4,M*1F $GPRMC,132904,A,4241.6671,N,07351.6808,W,000.0,239.2,110220,014.2,W,A*18 $GPGGA,132904,4241.6671,N,07351.6808,W,1,04,2.1,109.6,M,-33.5,M,,*7F $GPGSA,M,2,10,,12,,,24,,,32,,,,2.4,2.1,1.0*3E $GPGSV,3,2,11,20,48,150,00,24,33,051,32,25,25,137,00,31,14,218,00*78 $PGRME,14.4,M,21.0,M,25.4,M*1F $GPRMC,132905,A,4241.6673,N,07351.6809,W,000.0,239.2,110220,014.2,W,A*1A $GPGGA,132905,4241.6673,N,07351.6809,W,1,04,2.1,110.1,M,-33.5,M,,*72 $GPGSA,M,2,10,,12,,,24,,,32,,,,2.4,2.1,1.0*3E $GPGSV,3,3,11,32,59,297,37,34,00,000,00,35,00,000,00*4D ^C [root@10.222.222.10 ~]#
Serial Over TCP/IP - A Remote Serial Login Console (stty, ncat, socat, minicom)
In this example we will used a Raspberry PI 4 serial console connection (115200 Baud) attached to USB-to-Serial adapter: "/dev/ttyUSB0" on host: "172.16.1.245" and use ncat to establish a TCP/IP listening socket on port: "22222". On host: "172.16.1.71" we will connect via TCP/IP to this console service on remote host: "172.16.1.245". The socat utility and the terminal emulator "minicom" application will be used to establish an interactive session to the remote Raspberry Pi 4 serial login console.
On host: "172.16.1.245"
- First set the serial baud rate and any special settings for the USB-to-Serial device: "/dev/ttyUSB0" using the "stty" utility:
[root@172.16.1.245 ~]# stty -F /dev/ttyUSB0 115200 raw;
- Next establish the TCP/IP Raspberry Pi 4 serial console service on port: "22222" via ncat and put the process in the background:
[root@172.16.1.245 ~]# ncat -l 22222 > /dev/ttyUSB0 < /dev/ttyUSB0 &
On host: "172.16.1.71"
- Connect to the TCP/IP Raspberry Pi 4 serial console service on host: "172.16.1.245", port: "22222" using socat and put the process in the background. The pseudo terminal: "/dev/ttyV0" will be created by socat for attaching minicom.
[root@172.16.1.245 ~]# socat pty,link=/dev/ttyV0 tcp:172.16.1.245:22222 &
- Create a minicom configuration in file: "/etc/minirc.ttyV0":
[root@172.16.1.71]# cat /etc/minirc.ttyV0 # # Raspbery Pi 4 Serial Login Console pu port /dev/ttyV0 pu baudrate 115200 pu bits 8 pu parity N pu stopbits 1 pu rtscts No pu xonxoff No # # Disable modem initialization values... pu minit pu mreset pu mdialpre pu mdialsuf pu mdialpre2 pu mdialsuf2 pu mdialpre3 pu mdialsuf3 pu mhangup pu mdialcan
- Next attach minicom to the newly created pseudo terminal: "/dev/V0" - An interactive serial login console session to a Raspberry Pi 4 attached to remote system: "172.16.1.245". The user: "pi" will be used for logging in.
[root@172.16.1.71]# minicom -w -c on -t xterm ttyV0; Welcome to minicom 2.7.1 OPTIONS: I18n Compiled on Feb 10 2020, 00:00:00. Port /dev/ttyV0, 12:27:47 Press CTRL-A Z for help on special keys rpi4 login: pi Password: ************ Last login: Fri Oct 2 11:34:47 EDT 2020 on ttyS0 Linux rpi4 5.4.51-v7l+ #1327 SMP Thu Jul 23 11:04:39 BST 2020 armv7l The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. pi@rpi4:~$ lscpu Architecture: armv7l Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 1 Vendor ID: ARM Model: 3 Model name: Cortex-A72 Stepping: r0p3 CPU max MHz: 1500.0000 CPU min MHz: 600.0000 BogoMIPS: 108.00 Flags: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 pi@rpi4:~$ exit logout Raspbian GNU/Linux 10 rpi4 ttyS0 rpi4 login:
Tunneling UDP Traffic Through An SSH Connection
See article: Tunneling UDP Traffic Through An SSH Connection