Difference between revisions of "HowTo Build NST 18"

From NST Wiki
Jump to navigationJump to search
(Check Out The Source Code)
(Building NST 18)
 
(4 intermediate revisions by the same user not shown)
Line 7: Line 7:
 
* Installing the necessary packages to build.
 
* Installing the necessary packages to build.
 
* Building the yum repository and ISO image.
 
* Building the yum repository and ISO image.
 +
 +
<div class="noteMessage">[[File:Warning.png|left]]  There appears to be issue with building the NST ISO image if you are using a GNOME 3 desktop and logged in as the ''root'' user. You are strongly discouraged from attempting to build the NST ISO image when logged into a GNOME 3 desktop.</div>
 +
 +
<div class="noteMessage">[[File:Warning.png|left]]  Do not use '''sudo''' when invoking commands to build the NST. If you are logged in as a non-root user, the build process will automatically insert the necessary '''sudo''' invocations as needed.</div>
  
 
= Setting Up A Development Machine =
 
= Setting Up A Development Machine =
Line 42: Line 46:
  
 
= Initial Configuration =
 
= Initial Configuration =
 +
 +
When you run the top level ''./configure'' script, the configuration script may automatically be able to determine what packages are missing and add them to your development system. In order for the script to do this, you must either be logged in as the ''root'' user or using a account which has ''sudo'' privileges. To check whether or not your account has ''sudo'' privileges, try running the following:
 +
 +
sudo tail /var/log/messages
 +
 +
If the above command fails, it means you skipped a step during the development machine setup section. You will need to fix this before continuing.
 +
 +
If the above command passes (or you are logged in as ''root''), you should be able to run the following commands:
 +
 +
cd $HOME/repo
 +
./configure
 +
 +
The first invocation will take a long time as many packages will need to be installed onto your development machine in order to make it capable of building the NST.
 +
 +
If the configuration script does not complete successfully, it indicates one of the following issues:
 +
 +
* A core package needs to be installed by hand. In this case, a error message should be displayed showing you what ''yum'' command to run in order to install the core package.
 +
* There was a networking issue in downloading and installing one of the required packages.
 +
* One of the packages that needed to be built and installed locally failed. This typically indicates that the remote server where the source is downloaded from was temporarily down (or that we need to update the source URL for the package in the NST source code).
 +
 +
= Initial Build =
 +
 +
Once you have successfully completed the configuration process, you should be able to build the NST yum repository and NST ISO image using the following commands:
 +
 +
cd $HOME/repo
 +
make
 +
 +
If things work correctly, this should take a long time (many hours) and produce:
 +
 +
;nst-18.x86_64.iso or nst-18.i686.iso
 +
: The 64 bit or 32 bit ISO image which can be burned to DVD or installed on a USB memory stick (using the '''live-iso-to-disk''' command).
 +
;yum/repo
 +
: The directory containing the NST yum repository. You can push this out to a web server if you want to maintain your own yum repository for your deployed NST systems to get updates from.
 +
 +
= Subsequent Builds =
 +
 +
Subsequent builds will run much faster (less than two hours) as less things should need to be downloaded, installed and built. Use the following commands to update your subversion area to the latest code and rebuild the NST yum repository and ISO image:
 +
 +
cd $HOME/repo
 +
svn update && ./configure && make
 +
 +
= Troubleshooting Notes =
 +
 +
A build may fail for one of the following reasons:
 +
 +
* One of the web servers (there are many) that the build downloads source code is unreachable at the time of your build. For example, if the http://www.thc.org/ web site is down, you won't be able to download the source code for the ''hydra'' package. In this case the build will fail if you don't already have a copy of the ''hydra'' source code downloaded to your ''yum/cache/downloads'' directory.
 +
* The download URL for a source package is no longer valid. For example if there is a new release of the ''wireshark'' source code and current release the NST is trying to download a older release which is no longer available from the ''wireshark'' web site. In this case the build will fail if you don't already have a copy of the ''wireshark'' source code downloaded to your ''yum/cache/downloads'' directory.
 +
* You have not set up your development machine correctly.
 +
* You did not check out the ''repo'' area of the source code (this is the area we actively maintain).
 +
* Your ''repo'' area is not up to date (''svn update && ./configure'').
 +
* We made a mistake when merging updates into the ''repo'' area. If you could notify us via the NST forum (http://sourceforge.net/p/nst/discussion/361524/) we would appreciate it.
 +
 +
If the build fails because it was unable to download the source code for a particular package it will probably tell you the name of the file it was trying to download. If you can find this file elsewhere on the Internet (or in the ''yum/cache/downloads'' directory of another of your NST build machines), you can copy it into the ''yum/cache/downloads'' directory on your build machine to work through the process.
 +
 +
Unfortunately, the easiest method to get a successful build often requires some patience. If a build fails one day, it may just be easier to wait another day, update, configure and try again. Once you get your first successful build  things typically go much more smoothly (and quickly) as files are cached.

Latest revision as of 10:44, 29 May 2013

Building NST 18

The following page provides details on building the NST 18 yum repository and ISO images directly from source code. The procedure boils down to the following steps:

  • Setting up a development machine.
  • Checking out copy of the source code.
  • Installing the necessary packages to build.
  • Building the yum repository and ISO image.
Warning.png
There appears to be issue with building the NST ISO image if you are using a GNOME 3 desktop and logged in as the root user. You are strongly discouraged from attempting to build the NST ISO image when logged into a GNOME 3 desktop.
Warning.png
Do not use sudo when invoking commands to build the NST. If you are logged in as a non-root user, the build process will automatically insert the necessary sudo invocations as needed.

Setting Up A Development Machine

In order to set up a development machine, you will need to download either the 32 bit or 64 bit Live Installable version of Fedora 18 (http://fedoraproject.org/get-fedora). Choose the 32 bit version if you want to build the 32 bit version of NST and choose the 64 bit version if you want to build the 64 bit version of NST. If you want to build both a 32 bit version and 64 bit version of the NST, you will need to set up two development machines.

You can use a virtual environment as your development machines. If you do this, I'd recommend allocated at least 2GB of RAM and 40 GB of disk space and two CPUs. Even though it takes longer to build within a virtual environment, I find that the convenience is typically worth it. I typically use VirtualBox (https://www.virtualbox.org/) for my personal builds. The NST project uses VMware (http://www.vmware.com/) environments for nightly test builds.

Be aware that there are parts of the build process which require root access. Because of this, it is recommended that you set up a dedicated machine for NST development.

Regardless of whether you are using real hardware or virtual hardware, after you install Fedora 18, make sure you do the following:

  • Update the /etc/sudo file such that your account (or wheel group) can run the sudo command without being prompted for a password (NOTE: you can skip this step if you will be using the root account).
sudo visudo
  • Update all of the packages on the system:
sudo yum update
  • Reboot the system to load the newest kernel:
sudo reboot
  • Install/update any device drivers for you hardware or virtual environment.
  • Make sure that the subversion package is installed.
sudo yum install subversion 
  • Reboot the system to make sure the device drivers load properly.
sudo reboot

Check Out The Source Code

Once you have your Fedora 18 development machine set up, you can check out a copy of the NST source code. It is recommended that you check out the repo area from subversion as this is the active area that is kept up to date.

The following commands will check out a copy of the repo area to the $HOME/repo directory on your system:

cd $HOME
svn checkout http://svn.code.sf.net/p/nst/code/repo repo

It may take a bit, but the above should get a current copy of what we are currently using to build the current version of the NST distribution.

Initial Configuration

When you run the top level ./configure script, the configuration script may automatically be able to determine what packages are missing and add them to your development system. In order for the script to do this, you must either be logged in as the root user or using a account which has sudo privileges. To check whether or not your account has sudo privileges, try running the following:

sudo tail /var/log/messages

If the above command fails, it means you skipped a step during the development machine setup section. You will need to fix this before continuing.

If the above command passes (or you are logged in as root), you should be able to run the following commands:

cd $HOME/repo
./configure

The first invocation will take a long time as many packages will need to be installed onto your development machine in order to make it capable of building the NST.

If the configuration script does not complete successfully, it indicates one of the following issues:

  • A core package needs to be installed by hand. In this case, a error message should be displayed showing you what yum command to run in order to install the core package.
  • There was a networking issue in downloading and installing one of the required packages.
  • One of the packages that needed to be built and installed locally failed. This typically indicates that the remote server where the source is downloaded from was temporarily down (or that we need to update the source URL for the package in the NST source code).

Initial Build

Once you have successfully completed the configuration process, you should be able to build the NST yum repository and NST ISO image using the following commands:

cd $HOME/repo
make

If things work correctly, this should take a long time (many hours) and produce:

nst-18.x86_64.iso or nst-18.i686.iso
The 64 bit or 32 bit ISO image which can be burned to DVD or installed on a USB memory stick (using the live-iso-to-disk command).
yum/repo
The directory containing the NST yum repository. You can push this out to a web server if you want to maintain your own yum repository for your deployed NST systems to get updates from.

Subsequent Builds

Subsequent builds will run much faster (less than two hours) as less things should need to be downloaded, installed and built. Use the following commands to update your subversion area to the latest code and rebuild the NST yum repository and ISO image:

cd $HOME/repo
svn update && ./configure && make

Troubleshooting Notes

A build may fail for one of the following reasons:

  • One of the web servers (there are many) that the build downloads source code is unreachable at the time of your build. For example, if the http://www.thc.org/ web site is down, you won't be able to download the source code for the hydra package. In this case the build will fail if you don't already have a copy of the hydra source code downloaded to your yum/cache/downloads directory.
  • The download URL for a source package is no longer valid. For example if there is a new release of the wireshark source code and current release the NST is trying to download a older release which is no longer available from the wireshark web site. In this case the build will fail if you don't already have a copy of the wireshark source code downloaded to your yum/cache/downloads directory.
  • You have not set up your development machine correctly.
  • You did not check out the repo area of the source code (this is the area we actively maintain).
  • Your repo area is not up to date (svn update && ./configure).
  • We made a mistake when merging updates into the repo area. If you could notify us via the NST forum (http://sourceforge.net/p/nst/discussion/361524/) we would appreciate it.

If the build fails because it was unable to download the source code for a particular package it will probably tell you the name of the file it was trying to download. If you can find this file elsewhere on the Internet (or in the yum/cache/downloads directory of another of your NST build machines), you can copy it into the yum/cache/downloads directory on your build machine to work through the process.

Unfortunately, the easiest method to get a successful build often requires some patience. If a build fails one day, it may just be easier to wait another day, update, configure and try again. Once you get your first successful build things typically go much more smoothly (and quickly) as files are cached.