Building From Source

From NST Wiki
Revision as of 17:46, 3 August 2007 by Paul Blankenbaker (talk | contribs) (Installing Packages)
Jump to navigationJump to search

Preparing The Build Machine

You will need to install Fedora Core 6 (the i386 variant) onto your development system.

If you want to build a 64 bit variant (x86_64) of the NST, you will need to download the x86_64 variant of Fedora Core 6. While the i386 variant will run on both 32 bit and 64 bit machines, the x86_64 variant will only run on 64 bit machines. Most, if not all, 64 bit AMD processors and the Intel Core 2 series should be compatible with the x86_64 variant.

If you don't want to use physical hardware, you can use a Virtual Machine (even running under Windows) to do your NST development under. We have found that either VMWare Server (our preference) or VMWare Player from VMware can be used (even if you want to build a x86_64 version of the NST). We have also found that Virtual PC can be used.

Getting the NST Source Code

There are several ways to acquire the source code for NST developement (one can find an entire section related to this in the NST FAQ).

For our purposes we will choose the anonymous CVS approach. This permits one to build a current snapshot of the NST distribution as it is being developed.

The following demonstrates how one can create a: nst sub-directory on a Fedora Core 6 based system, and then populate it with the current NST source code (NOTE: Just press the Enter key when prompted for the password as there is no password required for anonymous access):

Checking Out The Current Source Code
[root@localhost ~]# mkdir $HOME/nst
[root@localhost ~]# cd $HOME/nst
[root@localhost nst]# CVSROOT=":pserver:anonymous@nst.cvs.sourceforge.net:/cvsroot/nst"
[root@localhost nst]# export CVSROOT
[root@localhost nst]# cvs login
Logging in to :pserver:anonymous@nst.cvs.sourceforge.net:2401/cvsroot/nst
CVS password:
[root@localhost nst]# cvs co .

   ... You should see file names streaming by in your
       console window - it may take several moments to
       download all of the source files ...

[root@localhost nst]#

As time goes on, you may want to update your source files (the NST developers check in new code quite frequently). Use the following commands:

Updating The Source Code
[root@localhost ~]# cd $HOME/nst
[root@localhost nst]# CVSROOT=":pserver:anonymous@nst.cvs.sourceforge.net:/cvsroot/nst"
[root@localhost nst]# export CVSROOT
[root@localhost nst]# cvs update -Pd

   ... You should see directory names streaming by in your
       console window as CVS searches for updated files ...

[root@localhost nst]#

As setting the CVSROOT variable can be quite tedious, it is recommended that you include its definition in your: "~/.bashrc" file. Add the following lines to the end of: "~/.bashrc".

CVSROOT=":pserver:anonymous@nst.cvs.sourceforge.net:/cvsroot/nst"; export CVSROOT;

Configuring The System

After downloading the source code, you will need to run the: configure command from the top level directory. Before doing so, here are a couple of things to note:

  • Before configuring the system, it is recommend that one run: "yum update" to make sure the latest kernel and package updates are installed.
  • Sun's Java Run Time Environment (JRE) is used when building the NST. The configure script will attempt to download and install it automatically (if it isn't found on your development system). If the script is unable to automatically install it, you will need to download and install the JRE from: [1] (get the appropriate RPM version).
  • You will probably need to run the configure multiple times the first time you setup your development system (as you will most likely be missing some packages).
  • If configure determines that there are missing packages, it will indicate the yum command which you need to run in order to add the necessary packages to your system (or at least indicate which packages need to be added).

Here's an example of what one might go through when trying to initially configure a development system (NOTE: In this example, yum found that the current packages installed on the system were up to date):

Initial System Configuration
[root@localhost ~]# cd $HOME/nst
[root@localhost nst]# yum update
Loading "installonlyn" plugin
Setting up Update Process
Setting up repositories
core                      100% |=========================| 1.1 kB    00:00
updates                   100% |=========================| 1.2 kB    00:00
extras                    100% |=========================| 1.1 kB    00:00
Reading repository metadata in from local files
primary.xml.gz            100% |=========================| 1.6 MB    00:11
extras    : ################################################## 5097/5097
No Packages marked for Update/Obsoletion
[root@localhost nst]# ./configure
***ERROR*** unable to find executable program 'docbook2html' on system
***ERROR*** unable to find executable program 'docbook2pdf' on system
***ERROR*** unable to find executable program 'make' on system
***ERROR*** unable to find executable program 'mkzftree' on system
***ERROR*** unable to find executable program 'ncftpput' on system
***ERROR*** unable to find executable program 'pear' on system
***ERROR*** unable to find executable program 'rpmbuild' on system
***ERROR*** unable to find executable program 'svn' on system
  Try the following to add the missing packages:

    yum install  docbook-utils docbook-utils-pdf make zisofs-tools ncftp php-pear rpm-build subversion
[root@localhost nst]# yum install docbook-utils docbook-utils-pdf make zisofs-tools ncftp php-pear rpm-build subversion

   ... Lot's of yum output as it figures out package dependencies,
       then you will be prompted. Enter 'y' when you see:

Transaction Summary
= Installing Packages =

At this point you might be tempted to run: "'''make'''" and produce a bootable '''ISO''' image. However, you would be missing MANY of the extra packages which are typically included in a '''NST''' distribution.

To get "most" of the extra packages installed onto your development system, you can use the: "'''make package-update'''" command as shown below. NOTE: This may take ''hours'' to complete. Also, if you don't run it from a graphical login (you can use '''VNC'''), the ''Firefox Add-Ons'' (like: "'''image_zoom'''") will fail to install:

<div class="screen">
  <div class="screenTitle">''Installing the Extra Packages''</div>
  <div class="userInput"><span class="prompt">[root@localhost ~]# </span>cd $HOME/nst</div>
  <div class="userInput"><span class="prompt">[root@localhost nst]# </span>make package-update</div>
<pre class="computerOutput">

   ... First a check will be made for the RPM packages which
       can be installed or updated via yum (you will need to
       answer yes if any packages are found ...

   ... After the yum installation of RPM packages completes,
       most of the packages defined in the include/data/packages.tsv
       (or include/data/packages.x86_64.tsv) file will be
       installed using custom scripts (some will be binary installs
       and some will build from source) ...

-------------------------------------------------------------------------------
***ERROR*** Following package updates failed:  snort unicornscan wireshark autohide colorful_tabs dictionarysearch image_zoom LinkChecker long_titles quickrestart reloadevery tidy webdeveloper

         Check the log files under: /root/nst/tmp/updates for details

-------------------------------------------------------------------------------
make: *** [tsv-update] Error 1
[root@localhost nst]#

The above output shows that several of the "automatic" package installations failed. This typically occurs for one of the following reasons:

  • The package required that a X desktop be running. All of the firefox add-ons fall into this category (autohide, colorful_tabs, ...).
  • The package is no longer available. This often occurs when a new release of a package comes out and indicates that the file: "include/data/packages.tsv" needs to be updated. This is what happened with snort (they were at release 2.7.0 where as the "include/data/packages.tsv" file was configured for 2.6.1.5).
  • A library was not found because the ldconfig command needs to be run (this is what caused the issue with wireshark shown in the above output).
  • The package has dissappeared, moved, or no longer builds cleanly (this is what occurred with unicornscan in the output shown above).

When you encounter packages which fail to install cleanly, you should be able to review the entire log related to the installation by looking in the: "tmp/updates" directory (for example, the file: "tmp/updates/unicornscan.log" contained many error messages). Also, if you have failures and have taken corrective actions, you can either run the: "make package-update" command again, or install packages individually using the scripts found in the: "src/bin" directory.

While the "make package-update" will automate the installation of 98% of the packages needed to build a full NST system, there are a handful of packages which require user interaction to complete the installation. For each of these packages, you should find an appropriately named script under the: "src/bin" directory.

To see what packages still need to be installed, the "make package-check" command can be used. The following demonstrates how to identify which packages still need to be installed, and then installs the ieee80211 package:

Identifying Missing Packages
[root@localhost ~]# cd $HOME/nst
[root@localhost nst]# make package-check | grep -i fail

autohide (version 1.1.5)                                           [failed]
colorful_tabs (version 2.0.1)                                      [failed]
dictionarysearch (version 2.0.1)                                   [failed]
ieee80211 (version 1.2.16)                                         [failed]
image_zoom (version 0.2.7)                                         [failed]
ipw2200 (version 1.2.1)                                            [failed]
LinkChecker (version 0.6.1)                                        [failed]
long_titles (version 1.2.4)                                        [failed]
madwifi (version 0.9.3)                                            [failed]
metasploit (version 2.7)                                           [failed]
nessus (version 2.2.9)                                             [failed]
netw (version 5.34.0)                                              [failed]
ntop (version 3.3)                                                 [failed]
quickrestart (version 1.1.0)                                       [failed]
reloadevery (version 2.0)                                          [failed]
snort (version 2.6.1.5)                                            [failed]
snort_inline (version 2.6.1.5)                                     [failed]
snort-mysql (version 2.6.1.5)                                      [failed]
snorter (version 2.1)                                              [failed]
tidy (version 0.8.3.9)                                             [failed]
vmware-tools (version 1.0.3)                                       [failed]
webdeveloper (version 1.1.4)                                       [failed]
[root@localhost nst]# src/bin/ieee80211_fetch_build

   ... Lots of output followed by a series of questions you must
       answer, followed by the build/install of the package ...

-------------------------------------------------------------------------------
SUCCESS: Successfully installed ieee80211
-------------------------------------------------------------------------------

NOTE: You may need to re-build modules which depended upon
this one. Look for failures when you run "make package-check".

[root@localhost nst]# make package-check | grep ieee80211
ieee80211 (version 1.2.16)                                         [ok]
[root@localhost nst]#

Installing Packages

At this point you might be tempted to run: "make" and produce a bootable ISO image. However, you would be missing MANY of the extra packages which are typically included in a NST distribution.

To get "most" of the extra packages installed onto your development system, you can use the: "make package-update" command as shown below (NOTE: This may take hours to complete):

Installing the Extra Packages
[root@localhost ~]# cd $HOME/nst
[root@localhost nst]# make package-update

   ... First a check will be made for the RPM packages which
       can be installed or updated via yum (you will need to
       answer yes if any packages are found ...

   ... After the yum installation of RPM packages completes,
       most of the packages defined in the include/data/packages.tsv
       (or include/data/packages.x86_64.tsv) file will be
       installed using custom scripts (some will be binary installs
       and some will build from source) ...

-------------------------------------------------------------------------------
***ERROR*** Following package updates failed:  snort unicornscan wireshark autohide colorful_tabs dictionarysearch image_zoom LinkChecker long_titles quickrestart reloadevery tidy webdeveloper

         Check the log files under: /root/nst/tmp/updates for details

-------------------------------------------------------------------------------
make: *** [tsv-update] Error 1
[root@localhost nst]#

The above output shows that several of the "automatic" package installations failed. This typically occurs for one of the following reasons:

  • The package required that a X desktop be running. All of the firefox add-ons fall into this category (autohide, colorful_tabs, ...).
  • The package is no longer available. This often occurs when a new release of a package comes out and indicates that the file: "include/data/packages.tsv" needs to be updated. This is what happened with snort (they were at release 2.7.0 where as the "include/data/packages.tsv" file was configured for 2.6.1.5).
  • A library was not found because the ldconfig command needs to be run (this is what caused the issue with wireshark shown in the above output).
  • The package has dissappeared, moved, or no longer builds cleanly (this is what occurred with unicornscan in the output shown above).

When you encounter packages which fail to install cleanly, you should be able to review the entire log related to the installation by looking in the: "tmp/updates" directory (for example, the file: "tmp/updates/unicornscan.log" contained many error messages).

While the "make package-update" will automate the installation of 98% of the packages needed to build a full NST system, there are a handful of packages which require user interaction to complete the installation. For each of these packages, you should find an appropriately named script under the: "src/bin" directory.

To see what packages still need to be installed, the "make package-check" command can be used. The following demonstrates how to identify which packages still need to be installed, and then installs the ieee80211 package:

Identifying Missing Packages
[root@localhost ~]# cd $HOME/nst
[root@localhost nst]# make package-check | grep -i fail

autohide (version 1.1.5)                                           [failed]
colorful_tabs (version 2.0.1)                                      [failed]
dictionarysearch (version 2.0.1)                                   [failed]
ieee80211 (version 1.2.16)                                         [failed]
image_zoom (version 0.2.7)                                         [failed]
ipw2200 (version 1.2.1)                                            [failed]
LinkChecker (version 0.6.1)                                        [failed]
long_titles (version 1.2.4)                                        [failed]
madwifi (version 0.9.3)                                            [failed]
metasploit (version 2.7)                                           [failed]
nessus (version 2.2.9)                                             [failed]
netw (version 5.34.0)                                              [failed]
ntop (version 3.3)                                                 [failed]
quickrestart (version 1.1.0)                                       [failed]
reloadevery (version 2.0)                                          [failed]
snort (version 2.6.1.5)                                            [failed]
snort_inline (version 2.6.1.5)                                     [failed]
snort-mysql (version 2.6.1.5)                                      [failed]
snorter (version 2.1)                                              [failed]
tidy (version 0.8.3.9)                                             [failed]
vmware-tools (version 1.0.3)                                       [failed]
webdeveloper (version 1.1.4)                                       [failed]
[root@localhost nst]# src/bin/ieee80211_fetch_build

   ... Lots of output followed by a series of questions you must
       answer, followed by the build/install of the package ...

-------------------------------------------------------------------------------
SUCCESS: Successfully installed ieee80211
-------------------------------------------------------------------------------

NOTE: You may need to re-build modules which depended upon
this one. Look for failures when you run "make package-check".

[root@localhost nst]# make package-check | grep ieee80211
ieee80211 (version 1.2.16)                                         [ok]
[root@localhost nst]#

Building a ISO Image

Tweaking The Configure/Build Process

The Magic "${HOME}/.nst" Directory

"${HOME}/.nst/configure.sh"

"${HOME}/.nst/disable.txt"

"${HOME}/.nst/post_install.sh"