Latest revision as of 14:33, 11 July 2021

USB General

Can all systems boot from a USB device?

No. In order to boot directly from a USB device, the BIOS in your system must be capable of booting from a USB drive. In addition, depending on your BIOS, it may only be able to boot from a USB drive.

However, even if your system is not able to boot directly from a USB device, you might be able to indirectly boot from a USB device using the following strategy:

Insert the NST Live DVD or NST Live Minimal CD into your system.

Insert your USB Flash Drive into a USB port.

When the NST Live DVD boots, press the down arrow keep to prevent it from automatically booting from the DVD.

Use the down arrow key until you reach the "PLoP Boot Manager (Boot USB)" menu option.

Press the Enter key to bring up the PLoP boot loader.

Select the USB boot option from the PLoP boot loader menu.

The boot process should then transfer to your USB Flash drive.

This procedure might not always work, but it doesn't hurt to try.

This procedure might work for other USB distributions as well as NST.

This procedure is not recommended for systems having only a USB 1.1 interface.

What key do I press to bring up the BIOS boot menu?

This varies from system to system. Try F10 or F12. Sometimes a BIOS won't offer a boot chooser keyboard shortcut. In this case you'll need to enter your BIOS setup utility (F2 or Delete keys are common) and change the boot order such that USB drives are checked first.

Can all USB Drives be booted from?

No. While most USB drives can be booted from, it is our understanding that the ability to boot from a USB drive is not necessarily a requirement (that being said, we are not sure we have run across a drive which can not be booted from).

What are the different ways to boot NST from a USB device?

There are three different ways in which you can boot the NST from a USB device:

A USB Live boot without a persistence overlay requires the least amount of space and behaves almost identically to booting from the ISO image burned to a DVD.
A USB Live Persistence boot is similar to a USB Live boot with the addition of a persistence overlay. This is a bit like a hybrid between a Live boot and a full hard disk installation (updates to the system will persist between boots, but you will periodically need to start over).
A USB Full Install boot is very similar to a full hard disk installation.

Can a Mac Mini 2009 boot the NST from a USB device?

While the Mac Mini can boot the NST from a ISO image burned onto a DVD, it can not boot a NST distribution on a USB attached drive.

What is the performance hit when booting from a USB 2.0 Flash drive?

It depends upon the USB 2.0 Flash drive (speeds will differ). Typically USB Flash drives will offer good read performance (15 to 25 MB/sec), but with a lower write performance (5 to 10 MB/sec).

Here are a couple of links related to speed tests that will be influencing my next purchase:

USB Flash Drive Comparison – 21 Tested and Compared

16gb USB Drive Comparison – 17 Drives Compared

This article has some interesting points in the evaluation of USB flash drive perfomance:

Two Fast and Functional USB Flash Drives

Can I edit the boot menu?

Yes.

If you have a USB Live or USB Live Persistence installation, use a text editor to modify the file: isolinux/isolinux.cfg on the partition of your installation.

If you have a USB Full Install, boot your NST system and then edit the file: /boot/grub/grub.conf. It should be noted that the /boot/grub/grub.conf file is modified whenever a kernel update is installed.

While in the "NST isolunix Boot Menu", you can use the "Tab" key to edit the boot options for a particular menu entry on the fly.

Can I select Local from the boot menu?

You can not use the Local boot method when booting from a live USB device. Feel free to delete this choice from your syslinux.cfg file.

How do you rescan a USB device so that you don't have to plug in the card reader every time you want to add/remove a flash drive?

The "hdparm -z <device>" command forces a kernel re-read of the partition table of the specified device(s). This can come in handy with a system that has a built-in memory card reader. The following example will re-read the partition table for the attached USB flash drive associated with device: "/dev/sdc":

 hdparm -z /dev/sdc

How do I safely remove an attached USB flash drive with power down?

This example assumes an attached USB flash drive: "/dev/sde1" mounted on mount point: "/mnt/DATA_8GB".

First flush file system buffers using the "sync" utility. It never hurts to do it twice.

 sync; sync;

Next remove the mount with the "umount" command.

umount /mnt/DATA_8GB;

Lastly power down the device using the "udisks" command.

udisks --detach /dev/sde;

How do I spin-down the disk for an attached USB SATA hard drive?

Use this command to perform an immediate spin-down of an attached USB SATA drive thus saving on power usage but could sacrifice on I/O performance (i.e., The drive will need to be spun-up prior to accessing it again). Install the RPM package: "sg3_utils" and then use the "sg_start" utility to spin-down a disk. The example below will spin-down the attached USB SATA hard drive associated with device: "/dev/sdc1".

yum install sg3_utils;
sg_start --stop /dev/sdc;

USB Live

What is the minimum size memory stick for a USB Live boot?

2 GB (if you could find a 1.5GB USB memory stick it would probably work as well).

Does a USB Live boot remember changes between boots?

No. You need a USB Live Persistence or USB Full Install boot if you want to remember changes to the systems between reboots.

Can I update packages on a USB Live boot?

Yes, but there are some caveats:

A lot of RAM is consumed both when downloading the files and during the installation process.

You will lose your updates when your reboot the system.

It is not possible to update the kernel. You can download and install it, but you can't reboot your system to actually load it.

How do I create NST Live on a USB stick from the command line after booting the NST DVD?

Please review the man page for livecd-iso-to-disk from the livecd-tools package before attempting this process.

To create a USB Live memory stick:

Install the livecd-tools package if it isn't already installed on your system:

rpm -q livecd-tools || yum install livecd-tools

Determine the device entry of your DVD drive which you booted from (typically: /dev/sr0).

Determine the device entry of your USB memory stick (it will typically be something like: /dev/sdf1). Running fdisk -l will often help give you a clue.

If your USB memory stick already contains a VFAT file system, you can do a non-destructive installation from the contents of the NST DVD (/dev/sr0) to your USB memory stick (replace /dev/sdXX with your device entry):

livecd-iso-to-disk /dev/sr0 /dev/sdXX

If you want to destroy the current contents of your USB memory stick (start from a clean slate), use the following command (DESTROYS ALL PRE-EXISTING DATA ON: /dev/sdXX):

livecd-iso-to-disk --format --reset-mbr /dev/sr0 /dev/sdXX

Read the man page (man livecd-iso-to-disk) for additional features of this handy utility (such as adding a persistence layer, adding a /home partition, or using the --noverify option to skip the ISO verification step).

How do I create NST Live on a USB stick from the command line after downloading an NST ISO?

First you need to be on a system that has access to the livecd-tools package (e.g., a Fedora based system or an NST system). The USB drive must have enough storage space on the device for the ISO image plus any persistence overlay file plus any configured home directory. The following is an example of creating NST Live on a USB stick using these configuration options:

Sets the Master Boot Record (MBR): --reset-mbr
Disables image verification: --noverify
Creates a 1GB persistence overlay file: --overlay-size-mb 1024
Creates a 256MB home directory: --home-size-mb 256
The NST ISO image location: /tmp/nst-2.13.0.x86_64.iso (In this case a 64 bit NST13 distro)
The USB storage device partition for installation: /dev/sdc1 (Assumes a VFAT file system already exists on "/dev/sdc1")

livecd-iso-to-disk --reset-mbr --noverify --overlay-size-mb 1024 --home-size-mb 256 /tmp/nst-2.13.0.x86_64.iso /dev/sdc1

USB Live Persistence

What is the minimum size memory stick for a USB Live Persistence boot?

2 GB. However, this will limit you to roughly 0.5 GB for your persistence layer.

Does a USB Live Persistence boot remember changes between boots?

Yes - until you fill the persistence layer. Using this boot method, your NST system will gradually consume memory in the persistence layer. Once all of the memory in the persistence layer is consumed, you will need to clear the entire persistence layer and start over from the initial ISO boot.

Can I update packages on a USB Live Persistence boot?

Yes, but there are some caveats:

A lot of your persistence overlay will be consumed each time you update your system.

You will lose your updates when your persistence overlay fills and you need to reset it (you'll have to start over).

How do I tell if a persistence overlay is being used?

During a live boot, a overlay image is used. This overlay may be created in RAM, or it may be persistent in which case changes will be preserved across boots.

The current method for testing for the presence of a persistent overlay is to look for the presence of the string: "overlay=" on the command line (/proc/cmdline).

How do I reset (clear) the persistence layer?

When your persistence layer fills (it will eventually), you will need to reset it in order to use the system again. This reset your system as if it booted from the initial ISO image.

To reset the persistence overlay, you'll need to append the following string to the end of the kernel boot options:

reset_overlay

USB Full Install

What is the minimum size memory stick for a USB Full Install?

10 GB (10240 MB).

Can I update packages on a USB Full Install?

Yes. A USB Full Install behaves like a normal hard disk installation and updates are fully supported (though they seem to take a long time to install).

Does a USB Full Install remember changes between boots?

Yes - a USB Full Install behaves pretty much the same as a normal USB hard disk installation.

Why does a yum update take so long on a USB Full Install and how can I speed it up?

According to Fedora bug reports 600716 and 529948, it appears that yum and the associated rpm commands perform a lot of fdatasync() operations and that USB devices are notorious for being slow at performing these operations.

As an example, the following shows that it takes over 2 minutes to remove the existing wireshark package and then re-install it from a USB based NST distribution (NOTE: This test was performed twice to verify that the time was repeatable).

[root@probe ~]# time (rpm --erase --nodeps wireshark && rpm -ivh wireshark-1.4.0-10.nst13.i686.rpm && echo "Remove/Install OK")
Preparing...                ########################################### [100%]
   1:wireshark              ########################################### [100%]
Remove/Install OK

real	2m18.124s
user	0m3.390s
sys	0m0.289s
[root@probe ~]# time (rpm --erase --nodeps wireshark && rpm -ivh wireshark-1.4.0-10.nst13.i686.rpm && echo "Remove/Install OK")
Preparing...                ########################################### [100%]
   1:wireshark              ########################################### [100%]
Remove/Install OK

real	2m22.688s
user	0m3.388s
sys	0m0.286s

If you look at the output above, you'll notice that while the total time was around 138 seconds, the CPU time was less than 4 seconds. In other words most of the time spent was due to waiting on something other than the CPU (in this case it was waiting on I/O).

Reading through the comments in the 600716 bug report, it is mentioned that its possible to reduce the number fsync operations performed, by running the following command to create a /etc/rpm/macros.dbi file.

[root@probe ~]# echo "%__dbi_other %{?__dbi_cdb} nofsync" > /etc/rpm/macros.dbi

After running the above command to reduce the number of fsync operations, we repeated the test:

[root@probe ~]# time (rpm --erase --nodeps wireshark && rpm -ivh wireshark-1.4.0-10.nst13.i686.rpm && echo "Remove/Install OK")
Preparing...                ########################################### [100%]
   1:wireshark              ########################################### [100%]
Remove/Install OK

real	0m3.718s
user	0m3.372s
sys	0m0.276s
[root@probe ~]# time (rpm --erase --nodeps wireshark && rpm -ivh wireshark-1.4.0-10.nst13.i686.rpm && echo "Remove/Install OK")
Preparing...                ########################################### [100%]
   1:wireshark              ########################################### [100%]
Remove/Install OK

real	0m3.721s
user	0m3.384s
sys	0m0.256s

The time taken to perform the operation dropped from 138 seconds down to less than 4 seconds! This is a dramatic performance increase. However, we thought we had read (unfortunately, we've lost the link), that disabling the fsync operation can cause irreparable damage to the system should a failure (like a power outage) occur during the upgrade. So, enabling this performance enhancement may involve taking on some risk. Out of curiosity, we decided to repeat the experiment once again, but with a single sync invocation after the RPM operations finished:

[root@probe ~]# time (rpm --erase --nodeps wireshark && rpm -ivh wireshark-1.4.0-10.nst13.i686.rpm && sync && echo "Remove/Install OK")
Preparing...                ########################################### [100%]
   1:wireshark              ########################################### [100%]
Remove/Install OK

real	1m18.388s
user	0m3.371s
sys	0m0.287s
[root@probe ~]# time (rpm --erase --nodeps wireshark && rpm -ivh wireshark-1.4.0-10.nst13.i686.rpm && sync && echo "Remove/Install OK")
Preparing...                ########################################### [100%]
   1:wireshark              ########################################### [100%]
Remove/Install OK

real	1m17.144s
user	0m3.345s
sys	0m0.302s

Added the sync invocation raised the time from 4 seconds to 77 seconds as we forced all of the buffered I/O operations to be written to the USB flash drive. However, 77 seconds is still significantly less than the original 138 seconds. Reducing the multiple sync operations into a single sync at the end definitely helped.

As a final word of advice, if you decide to enable this feature, it is recommended that you invoke the sync command once after performing a yum update.

Can I move a USB Full Install between systems?

Yes. However, this can confuse the udev system. The udev system tends to keep track of all hardware which the system sees. If you plan on moving your USB Full Install to different systems (which is common), it is recommended that you add the following line to your: /sbin/halt.local script:

/usr/bin/nstboot --move --verbose;

The above line will clear the hardware information collected by the udev system whenever you shutdown the system.

If you don't have a /sbin/halt.local script, you can use the following as a template (make sure to mark the file as executable):

#!/bin/bash
 
/usr/bin/nstboot --move --verbose;

Can I dual boot from a USB Flash Drive?

Yes. It is possible to dual boot off of a USB memory stick. This allows one to put both the NST Live version and the NST hard disk installation on the same memory stick. This may require a 32GB memory stick.

The following steps are required:

Setup up a 4GB (minimum) partition as primary partition 1 on the system according to the requirements of the liveusb-creator utility.

Install the NST Live version onto the 4GB partition.

Verify that the NST Live version installed boots.

After booting NST Live, perform a full hard disk installation to the remaining portion of the memory stick. NOTE: The hard disk installer, may not permit you to install to the same memory stick that you booted from (in this case, you will need to boot from a second NST Live memory stick).

Reboot the system, it should boot up using the hard disk installation on your USB memory stick (you should see the grub boot loader).

After logging in, determine the UUID of your LIVE partition. For example if your LIVE partition is /dev/sdb1, you can use either of the following commands to find which UUID is associated with a disk partition:

[root@chimi ~]# blkid /dev/sdb1
/dev/sdb1: LABEL="usbvfat" UUID="A794-9976" TYPE="vfat" PARTUUID="8ea02519-05"
[root@chimi ~]# ls -l /dev/disk/by-uuid/ | grep sdb1
lrwxrwxrwx 1 root root 10 Nov 15 08:50 A794-9976 -> ../../sdb1
[root@chimi ~]#

Edit (or create) the /etc/grub.d/43_nst_live grub configuration file. Change the UUID value to match the value you determined previously:

#!/bin/bash
#
# This file provides an easy way to add custom menu entry to boot a
# LIVE USB system that you know the UUID of the partion on the USB
# thumbdrive to boot from. To determine the UUID when you know the partion,
# you can use the following command (substitute your partition for sdb1):
#
# [root@chimi ext4]# ls -l /dev/disk/by-uuid | grep sdb1
# lrwxrwxrwx 1 root root 10 Nov 15 07:00 BE64-F4F4 -> ../../sdb1
# [root@chimi ext4]#

# Change this to match the UUID of the partition you want to boot from
declare liveUsbUuid="A794-9976"

# Don't think this matters (I think the search commands will override)
declare liveUsbRoot="hd1,msdos2";

# If we can verify that disk partition exists with specified UUID, then
# add a boot entry
if [ -e "/dev/disk/by-uuid/${liveUsbUuid}" ]; then

  echo -e "
menuentry 'NST Live USB' --class gnu-linux --class gnu --class os \$menuentry_id_option 'osprober-chain-${liveUsbUuid}' {
	set root='${liveUsbRoot}'
	if [ x\$feature_platform_search_hint = xy ]; then
	  search --no-floppy --fs-uuid --set=root --hint-bios=${liveUsbRoot} --hint-efi=${liveUsbRoot} --hint-baremetal=${liveUsbRoot//hd/ahci} --hint=${liveUsbRoot} ${liveUsbUuid}
	else
	  search --no-floppy --fs-uuid --set=root ${liveUsbUuid}
	fi
	parttool \${root} hidden-
	chainloader +1
}
";
fi

Make sure the /etc/grub.d/43_nst_live grub configuration file is executable then run the nstboot command and tell it to rebuild your grub configuration (it should include the new NST Live USB entry).

[root@chimi ~]# nstboot --grub2 -v
+ BEGIN   + Rebuilding the 'Grub2' configuration file: "/boot/grub2/grub.cfg"

Check/Adding NST Boot Grub2 Defaults file: "/etc/nst/grub2/nst_grub2_defaults"
===============================================================================
Already included...

Generating the NST Grub2 Configuration file: "/boot/grub2/grub.cfg"
===============================================================================
/sbin/grub2-mkconfig --output="/boot/grub2/grub.cfg";
Generating grub configuration file ...
Found theme: /boot/grub2/themes/system/nst_theme.txt
Found linux image: /boot/vmlinuz-4.2.5-201.fc22.x86_64
Found initrd image: /boot/initramfs-4.2.5-201.fc22.x86_64.img
Found linux image: /boot/vmlinuz-4.2.3-200.fc22.x86_64
Found initrd image: /boot/initramfs-4.2.3-200.fc22.x86_64.img
Found linux image: /boot/vmlinuz-4.1.8-200.fc22.x86_64
Found initrd image: /boot/initramfs-4.1.8-200.fc22.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-2e275e043f324b9b9fa1beeb09d9155a
Found initrd image: /boot/initramfs-0-rescue-2e275e043f324b9b9fa1beeb09d9155a.img
Found Windows 7 (loader) on /dev/sda1
Found NST release 22 (NST22) on /dev/sdb2
Found linux image: /boot/vmlinuz-4.2.5-201.fc22.x86_64
Found initrd image: /boot/initramfs-4.2.5-201.fc22.x86_64.img
done

+ END     + Successfully ran the 'Grub2' configuration rebuild
[root@chimi ~]#

From this point on, when you boot off of your USB memory stick, you should see the grub boot loader come up and you should have the option to select "NST Live USB" from the grub boot screen (you may need to scroll down to find it).

USB Flash Drive Speed Tests

USB Flash Drive Read Test

Use the following command to test the read speed of an attached USB flash device. The hdparm command will be use. It is assumed that USB device location is at: "/dev/sdc"

[root@nst-hd64 ~]# hdparm -t -T "/dev/sdc";

/dev/sdc:
  Timing cached reads:   5356 MB in  2.00 seconds = 2678.25 MB/sec
  Timing buffered disk reads: 100 MB in  3.95 seconds =  25.32 MB/sec
[root@nst-hd64 ~]#

USB Flash Drive Write Test

Use the following command to test the write speed of an attached USB flash device. The "/usr/share/nst-disk-speed/disk-write-test" command will be use. It is assumed that USB drive is mounted at: "/mnt/vfat".

[root@nst-hd64 ~]# mkdir -p "/mnt/vfat/tmp";
[root@nst-hd64 ~]#
[root@nst-hd64 ~]#  /usr/share/nst-disk-speed/disk-write-test -h;

disk-write-test <-h | path-file-name> [count [block-size]]]

[root@nst-hd64 ~]#  /usr/share/nst-disk-speed/disk-write-test "/mnt/vfat/tmp/wtest" 102400 1024;
Starting Storage Media Write Test
======== ======= ===== ===== ====
      Input File: /dev/zero
     Output File: /mnt/vfat/tmp/wtest
      Block Size: 1024 (MB = 1000*1000)
           Count: 102400
Output File Size: 104857600 bytes, 104857600 MB
        Duration: 7.767230000 (secs)
=================Results=================
13.500 MB/sec

[root@nst-hd64 ~]# rmdir "/mnt/vfat/tmp";
[root@nst-hd64 ~]#

USB Bus Reset

This section describes how to reset the USB 2.0 / 3.0 Bus without rebooting the system. One may need to do this for a variety of reasons. An example could be that a USB 3.0 Gigabit Ethernet Adapter is not recognized after power on. By resetting the USB 3.0 driver (i.e., unbind / bind) the adapter will now be recognized. One could to this automatically after a system reboot or power on via a shell script (See Step: 3 below.).

Step: 1

Get the Bus number, Device number and Function number for the USB devices on your system using the lspci command.

[root@nst26-rwh ~]# lspci | grep "USB";
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)
26:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 04)

Step: 2

Use the following commands to manually reset the USB bus. These commands must be run as the 'root' user. These commands simply remove the USB drivers "ehci_hcd" and "xhci_hcd" from USB 2.0 and the USB 3.0 hubs respectively, then reattaches them, thus simulating a logical reset of the entire hub.

USB 2.0

[root@nst26-rwh ~]# echo -n "0000:00:1a.0" | tee "/sys/bus/pci/drivers/ehci-pci/unbind";
[root@nst26-rwh ~]# echo -n "0000:00:1d.0" | tee "/sys/bus/pci/drivers/ehci-pci/unbind";
[root@nst26-rwh ~]# echo -n "0000:00:1a.0" | tee "/sys/bus/pci/drivers/ehci-pci/bind";
[root@nst26-rwh ~]# echo -n "0000:00:1d.0" | tee "/sys/bus/pci/drivers/ehci-pci/bind";

USB 3.0

[root@nst26-rwh ~]# echo -n "0000:26:00.0" | tee "/sys/bus/pci/drivers/xhci_hcd/unbind";
[root@nst26-rwh ~]# echo -n "0000:26:00.0" | tee "/sys/bus/pci/drivers/xhci_hcd/bind";

Step: 3

Below is a script one can use to reset all USB Buses. Make sure your script is executable prior to running.

#!/bin/bash
 
#
# Attempt to reset the USB 2.0 Bus...
echo -n "0000:00:1a.0" | tee "/sys/bus/pci/drivers/ehci-pci/unbind";
echo;
echo -n "0000:00:1d.0" | tee "/sys/bus/pci/drivers/ehci-pci/unbind";
echo;
echo -n "0000:00:1a.0" | tee "/sys/bus/pci/drivers/ehci-pci/bind";
echo;
echo -n "0000:00:1d.0" | tee "/sys/bus/pci/drivers/ehci-pci/bind";
echo;

#
# Attempt to reset the USB 3.0 Bus...
echo -n "0000:26:00.0" | tee "/sys/bus/pci/drivers/xhci_hcd/unbind";
echo;
echo -n "0000:26:00.0" | tee "/sys/bus/pci/drivers/xhci_hcd/bind";
echo;

Fedora Grub2 References

GRUB 2 is the latest version of GNU GRUB, the GRand Unified Bootloader. Use the following references for NST implementation:

NST USB FAQ: Difference between revisions