Automated Tests

From MediaWiki
Revision as of 07:56, 30 December 2007 by Paul Blankenbaker (talk | contribs) (Initial overview and how to run tests)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

Automated Tests

The "test" sub-directory contains a collection of test scripts which can be run by a NST developer to help assure that a new build is behaving properly.

The design of the test framework is as follows:

  • Tests are run from the development machine against a remote probe (ssh is used to transfer and run test scripts).
  • Tests may be DESTRUCTIVE to the target machine! Run tests against a NST system BEFORE deployment or final installation.
  • The automated tests are good base line checks, but they are not exhaustive. They are useful in determining if the current development environment is ready for human testing, but do not replace human testing.
  • Running one, many or all tests is a simple process.
  • Creating tests is a simple process.
  • Automated tests do not need to worry about missing libraries, unknown owners or broken symbolic links as existing make targets already exist for this purpose.

Running Tests

To run tests, you must have the following two systems at your disposal:

  • A Fedora based NST Development system with the NST source code checked out and configured (the same system used to build the NST ISO image).
  • A running NST system booted from either the NST ISO image OR a hard disk installation of the NST. ***WARNING*** The tests are run against this system and will change the state/configuration of the system - you will likely need to reboot, re-configure, and/or re-install after running tests against this NST system.

Hanging Tests

If a test hangs, you may need to ssh to the NST system being tested, and kill the hanging processes by hand.

The nessus test has sometimes triggered this situation.

Running The Full "probe-check"

To run the full test suite against a running NST system, use the probe-check target on the development system as shown below (change the IP address shown to the IP address of the NST system to be checked):

make HOST=192.168.22.13 probe-check
[root@localhost nst]# make HOST=192.168.22.13 probe-check

   ... All testing is done

       - Runs "ldd" on files looking for missing modules.
       - Checks if there are any obvious ownership issues.
       - Checks for symbolic links which point nowhere.
       - Runs each of the test scripts under the "test" directory.

       Takes a long time to complete ...
[root@localhost nst]# less -R probe-check.log

   ... By passing the "-R" option to less, you should
       see color while reviewing the log file ...
[root@localhost nst]#

Running All Tests

To run all of the tests found under the: "test" sub-directory without running the ldd, ownership, and symbolic link tests, use the test target on the development system as shown below (change the IP address shown to the IP address of the NST system to be checked):

make HOST=192.168.22.13 test
[root@localhost nst]# make HOST=192.168.22.13 test

   ... Runs each of the test scripts under the "test" directory.

       Takes a long time to complete ...
[root@localhost nst]# less -R test.log

   ... By passing the "-R" option to less, you should
       see color while reviewing the log file ...
[root@localhost nst]#

Running A Single Test

To quickly run a specific test found under the: "test" sub-directory, include the TEST=NAME option when invoking the test target (change the IP address shown to the IP address of the NST system to be checked):

make HOST=192.168.22.13 TEST=mbrowse test
[root@localhost nst]# make HOST=192.168.22.13 TEST=mbrowse test

Running test: "mbrowse" on: "192.168.22.13"

      Script: "/root/nst/tmp/test/mbrowse/runtest"
         Log: "/root/nst/tmp/test/mbrowse/runtest.log"

  Locating: "mbrowse" ........................................ [  OK  ]
  Verify "mbrowse -help" returns expected version ............ [  OK  ]
  Success! All 2 tests passed ................................ [  OK  ]

SUCCESS!    All of the tests passed - congratulations.
[root@localhost nst]#

Running Multiple Tests

To quickly run a specific set of tests found under the: "test" sub-directory, include the TEST="NAME0 NAME1 ..." option when invoking the test target (change the IP address shown to the IP address of the NST system to be checked):

make HOST=192.168.22.13 TEST="getipaddr mbrowse nfs" test
[root@localhost nst]# make HOST=192.168.22.13 TEST="getipaddr mbrowse nfs" test

Running test: "getipaddr" on: "192.168.22.13"

      Script: "/root/nst/tmp/test/getipaddr/runtest"
         Log: "/root/nst/tmp/test/getipaddr/runtest.log"

  Locating: "getipaddr" ...................................... [  OK  ]
  Locating: "wget" ........................................... [  OK  ]
  Getting public IP address .................................. [  OK  ]
  Checking: "http://nst.sourceforge.net/nst/cgi-bin/ip.cgi" .. [  OK  ]
  Checking: "http://www.networksecuritytoolkit.org/nst/cgi-bin [  OK  ]
  Checking: "http://whatismyip.org/" ......................... [  OK  ]
  Success! All 6 tests passed ................................ [  OK  ]

Running test: "mbrowse" on: "192.168.22.13"

      Script: "/root/nst/tmp/test/mbrowse/runtest"
         Log: "/root/nst/tmp/test/mbrowse/runtest.log"

  Locating: "mbrowse" ........................................ [  OK  ]
  Verify "mbrowse -help" returns expected version ............ [  OK  ]
  Success! All 2 tests passed ................................ [  OK  ]

Running test: "nfs" on: "192.168.22.13"

      Script: "/root/nst/tmp/test/nfs/runtest"
         Log: "/root/nst/tmp/test/nfs/runtest.log"

  Locating: "mount" .......................................... [  OK  ]
  Locating: "service" ........................................ [  OK  ]
  Locating: "showmount" ...................................... [  OK  ]
  Locating: "sleep" .......................................... [  OK  ]
  Locating: "sync" ........................................... [  OK  ]
  Locating: "umount" ......................................... [  OK  ]
  Saving: "/etc/exports" ..................................... [  OK  ]
  Installing custom: "/etc/exports" .......................... [  OK  ]
  Starting service: rpcbind .................................. [  OK  ]
  Starting service: nfslock .................................. [  OK  ]
  Starting service: rpcidmapd ................................ [  OK  ]
  Starting service: rstatd ................................... [  OK  ]
  Starting service: nfs ...................................... [  OK  ]
  Testing /usr/local/sbin/showmount output ................... [  OK  ]
  Mounting exported filesystem to: "/root/runtest/nfs/nfs" ... [  OK  ]
  Verifying that we can see file on NFS mount ................ [  OK  ]
  Verify that we see removal of file on NFS mount ............ [  OK  ]
  Umount of: "/root/runtest/nfs/nfs" ......................... [  OK  ]
  Restoring: "/etc/exports" .................................. [  OK  ]
  Stopping serivce: nfs ...................................... [  OK  ]
  Stopping serivce: rstatd ................................... [  OK  ]
  Stopping serivce: rpcidmapd ................................ [  OK  ]
  Stopping serivce: nfslock .................................. [  OK  ]
  Stopping serivce: rpcbind .................................. [  OK  ]
  Success! All 24 tests passed ............................... [  OK  ]

SUCCESS!    All of the tests passed - congratulations.
[root@localhost nst]#

Creating Tests

Files Making Up The Test

Functions Which Can Be Used

Simple Example

Complex Example

Retrieved from "https://wiki.networksecuritytoolkit.org/nstwiki/index.php?title=Automated_Tests&oldid=734"