How does the Network Security Toolkit use the serial port at boot?
The Network Security Toolkit makes use of serial port /dev/ttyS0 (COM1) at boot up. If you have another system connected to the NST's serial port when you power on the system, you should be able to select Server mode and access the NST system even if it doesn't have a monitor or keyboard attached.
What settings should I use to connect to the serial port?
The default settings for the serial port are: 57600 baud, no parity, 8 data bits, and 1 stop bit. VT220 terminal emulation is also desirable.