Automated Tests - Revision history

Paul Blankenbaker: /* When Tests Fail */

2010-09-22T13:17:04Z

When Tests Fail

← Older revision		Revision as of 09:17, 22 September 2010
Line 180:		Line 180:
	Should a test fail, it is often desirable to dig a bit deeper into the log files produced by running the test to see what happened.		Should a test fail, it is often desirable to dig a bit deeper into the log files produced by running the test to see what happened.

	== Log Files Left On The ~~Development~~ System ==		== Log Files Left On The NST System ==

	When each test is run, a temporary directory is created ~~having~~ the ~~form: "~~'''~~tmp/test/TEST_NAME~~'''~~" on~~ the ~~development system. '''NOTE'''~~: ~~The~~ "'''~~tmp~~'''" ~~may be a different location if you didn't use the default configuration options when configuring your development system~~.		When each test is run, a temporary directory is created on the '''NST''' system having the form: "'''/root/runtest/TEST_NAME'''".

	For example, after running the ''getipaddr'', one should find the: "'''~~tmp~~/~~test~~/getipaddr'''" directory on the ~~development~~ system. At a minimum, one should find the following files under this directory:		For example, after running the ''getipaddr'', one should find the: "'''/root/runtest/getipaddr'''" directory on the '''NST''' system. At a minimum, one should find the following files under this directory:

	; '''runtest.log''' : Verbose output from the running of the test.		; '''runtest.log''' : Verbose output from the running of the test.
Line 190:		Line 190:
	; '''test.bash''' : The source test script.		; '''test.bash''' : The source test script.

	~~These~~ files ~~should ALWAYS be present after~~ running a test ~~- regardless of whether~~ the test ~~passes or fails~~.		In addition to the above files, one may also find temporary files generated by running the test under this directory. For example, the ''clamav'' test makes requests to the '''NST WUI''' and saves the '''HTML''' responses to temporary files.

	== Log Files Left On The ~~NST~~ System ==		'''NOTE''': These files will only be present if the test fails.

			== Log Files Left On The Development System ==

			Regardless of whether a test passes or fails, all of the files created under the "'''/root/runtest/TEST_NAME'''" directory will be transferred back to the: "'''tmp/test/TEST_NAME'''" directory on the development system. '''NOTE''': The "'''tmp'''" may be a different location if you didn't use the default configuration options when configuring your development system.

			For example, after running the ''getipaddr'', one should find the: "'''tmp/test/getipaddr'''" directory on the development system. At a minimum, one should find the following files under this directory:

			<div class="screen">
			<div class="screenTitle">''make HOST=192.168.22.13 TEST="getipaddr"''</div>
			<div class="userInput"><span class="prompt">[root@localhost nst]# </span>make HOST=192.168.22.13 TEST="getipaddr" test</div>
			<pre class="computerOutput">

	~~When each~~ test ~~is run, a temporary directory is created~~ on ~~the '''NST''' system having the form~~: "~~'''/root/runtest/TEST_NAME'''~~".		Running test: "getipaddr" on: "192.168.22.13"

	~~For example, after running the ''~~getipaddr~~'', one should find the~~: "~~'''~~/root/~~runtest~~/getipaddr~~'''~~" ~~directory on the '''NST''' system. At a minimum, one should find the following files under this directory:~~		Script: "/root/nst/tmp/test/getipaddr/runtest"
			Log: "/root/nst/tmp/test/getipaddr/runtest.log"

	~~; '''runtest~~.~~log'''~~ : ~~Verbose output from the running of the test~~.		Locating: "getipaddr" ...................................... [ OK ]
	~~; '''runtest'''~~ : ~~The fully formed test script (supporting '''bash''' functions joined with '''test~~.~~bash''')~~.		Locating: "wget" ........................................... [ OK ]
	~~; '''test~~.~~bash'''~~ : ~~The source test script~~.		Getting public IP address .................................. [ OK ]
			Checking: "http://nst.sourceforge.net/nst/cgi-bin/ip.cgi" .. [ OK ]
			Checking: "http://www.networksecuritytoolkit.org/nst/cgi-bin [ OK ]
			Checking: "http://whatismyip.org/" ......................... [ OK ]
			Success! All 6 tests passed ................................ [ OK ]

	~~In addition to~~ the above files, one may also find temporary files generated by running the test under this directory. For example, the ''clamav'' test makes requests to the '''NST WUI''' and saves the '''HTML''' responses to temporary files.		SUCCESS! All of the tests passed - congratulations.

	~~'''NOTE''': These files will only be present if the~~ test ~~fails~~.		</pre>
			<div class="userInput"><span class="prompt">[root@localhost nst]# </span>ls tmp/test/getipaddr</div>
			<pre class="computerOutput">
			runtest runtest.log state-after.txt state-before.txt test.bash
			</pre>
			<div class="userInput"><span class="prompt">[root@localhost nst]# </span></div>
			</div>

	= Creating Tests =		= Creating Tests =

Paul Blankenbaker: /* Automated Tests */

2010-09-22T13:03:13Z

Automated Tests

← Older revision		Revision as of 09:03, 22 September 2010
Line 15:		Line 15:
	* Creating tests is a simple process.		* Creating tests is a simple process.

	* Automated tests do not need to worry about missing libraries, unknown owners or broken symbolic links ~~as existing make targets~~ already ~~exist for this purpose~~.		* Automated tests do not need to worry about missing libraries, unknown owners or broken symbolic links. There are system wide test which already perform these checks.

	= Running Tests =		= Running Tests =

Paul Blankenbaker: /* Complex Example */

2007-12-31T13:03:13Z

Complex Example

Show changes

Rwh: /* Running Multiple Tests */

2007-12-31T01:54:11Z

Running Multiple Tests

← Older revision		Revision as of 21:54, 30 December 2007
Line 110:		Line 110:
	== Running Multiple Tests ==		== Running Multiple Tests ==

	To quickly run a specific set of tests found under the: "'''test'''" sub-directory, include the ''TEST="NAME0 NAME1 ..."'' option when invoking the ''test'' target (change the '''IP''' address shown to the '''IP''' address of the '''NST''' system to be checked):		To quickly run a specific set of tests found under the: "'''test'''" sub-directory, include the ''TEST="NAME0 NAME1 ..."'' option when invoking the "''test''" '''make''' target (change the '''IP''' address shown to the '''IP''' address of the '''NST''' system to be checked):

	<div class="screen">		<div class="screen">

Rwh: /* Running A Single Test */

2007-12-31T01:53:26Z

Running A Single Test

← Older revision		Revision as of 21:53, 30 December 2007
Line 86:		Line 86:
	== Running A Single Test ==		== Running A Single Test ==

	To quickly run a specific test found under the: "'''test'''" sub-directory, include the ''TEST=NAME'' option when invoking the ''test'' target (change the '''IP''' address shown to the '''IP''' address of the '''NST''' system to be checked):		To quickly run a specific test found under the: "'''test'''" sub-directory, include the ''TEST=NAME'' option when invoking the "''test''" '''make''' target (change the '''IP''' address shown to the '''IP''' address of the '''NST''' system to be checked):

	<div class="screen">		<div class="screen">

Rwh: /* Running All Tests */

2007-12-31T01:52:30Z

Running All Tests

← Older revision		Revision as of 21:52, 30 December 2007
Line 62:		Line 62:
	== Running All Tests ==		== Running All Tests ==

	To run all of the tests found under the: "'''test'''" sub-directory without running the ldd, ownership, and symbolic link tests, use the ''test'' target on the development system as shown below (change the '''IP''' address shown to the '''IP''' address of the '''NST''' system to be checked):		To run all of the tests found under the: "'''test'''" sub-directory without running the ldd, ownership, and symbolic link tests, use the "''test''" '''make''' target on the development system as shown below (change the '''IP''' address shown to the '''IP''' address of the '''NST''' system to be checked):

	<div class="screen">		<div class="screen">

Rwh: /* Running The Full "probe-check" */

2007-12-31T01:51:26Z

Running The Full "probe-check"

← Older revision		Revision as of 21:51, 30 December 2007
Line 33:		Line 33:
	== Running The Full "probe-check" ==		== Running The Full "probe-check" ==

	To run the full test suite against a running '''NST''' system, use the ''probe-check'' '''make''' target on the development system in the top level directory as shown below (change the '''IP''' address shown to the '''IP''' address of the '''NST''' system to be checked):		To run the full test suite against a running '''NST''' system, use the "''probe-check''" '''make''' target on the development system in the top level directory as shown below (change the '''IP''' address shown to the '''IP''' address of the '''NST''' system to be checked):

	<div class="screen">		<div class="screen">

Rwh: /* Running The Full "probe-check" */

2007-12-31T00:02:36Z

Running The Full "probe-check"

← Older revision		Revision as of 20:02, 30 December 2007
Line 33:		Line 33:
	== Running The Full "probe-check" ==		== Running The Full "probe-check" ==

	To run the full test suite against a running '''NST''' system, use the ''probe-check'' target on the development system in the top level directory as shown below (change the '''IP''' address shown to the '''IP''' address of the '''NST''' system to be checked):		To run the full test suite against a running '''NST''' system, use the ''probe-check'' '''make''' target on the development system in the top level directory as shown below (change the '''IP''' address shown to the '''IP''' address of the '''NST''' system to be checked):

	<div class="screen">		<div class="screen">

Rwh: /* Running The Full "probe-check" */

2007-12-31T00:01:56Z

Running The Full "probe-check"

← Older revision		Revision as of 20:01, 30 December 2007
Line 33:		Line 33:
	== Running The Full "probe-check" ==		== Running The Full "probe-check" ==

	To run the full test suite against a running '''NST''' system, use the ''probe-check'' target on the development system as shown below (change the '''IP''' address shown to the '''IP''' address of the '''NST''' system to be checked):		To run the full test suite against a running '''NST''' system, use the ''probe-check'' target on the development system in the top level directory as shown below (change the '''IP''' address shown to the '''IP''' address of the '''NST''' system to be checked):

	<div class="screen">		<div class="screen">

Paul Blankenbaker: /* Creating Tests */

2007-12-30T21:33:26Z

Creating Tests

@@ Line 215: / Line 215: @@
 To create a new test:
-* Create a directory under the: "[http://nst.cvs.sourceforge.net/nst/test test]" with a name related to the test to be performed. For example, "[http://nst.cvs.sourceforge.net/nst/test/snort test/snort]" is the directory containing the test for '''snort'''.
+* Create a directory under the: "[http://nst.cvs.sourceforge.net/nst/test test]" directory with a name related to the test to be performed. For example, "[http://nst.cvs.sourceforge.net/nst/test/snort test/snort]" is the directory containing the test for '''snort'''.
 * Create a '''bash''' script under the new directory with the name: '''test.bash'''.  For example, "[http://nst.cvs.sourceforge.net/nst/test/snort/test.bash?revision=HEAD&view=markup test/snort/test.bash]" is the '''bash''' script used to test '''snort'''.
@@ Line 224: / Line 224: @@
 == Functions Which Can Be Used ==
 == Simple Example ==