Test Script

A test script is a testing work product modeling a software program (often written in a procedural scripting language) that executes a test suite of test cases.

The goals of a test script :
1) Automate the execution of test cases.
2) Support regression testing

Objectives of a single test script :
1) Execute each test case in the test suite.
2) Report the results of the test suite.

A test script provides the following benefits:
1) Automates a single test suite, thereby supporting regression testing.
2) Failure to produce test scripts makes regression testing more expensive and less likely to occur.

Contents
1) Test script objectives
2) Test preparation (e.g., to place objects under test into the appropriate pre-test states)
3) Test stimuli (e.g., to send test messages or raise test exceptions)
4) Expected behavior (i.e., test oracle)
5) Test reporting script
6) Test finalization script

To know more visit : Test Script

Test Execution Engine

A test execution engine is a type of software used to test software, hardware or complete systems.

A test execution engine may appear in two forms:
1) Module of a test software suite (test bench) or an integrated development environment
2) Stand-alone application software

The test specification is software. Test specification is sometimes referred to as test sequence, which consists of test steps.

The test specification should be stored in the test repository in a text format (such as source code). Test data is sometimes generated by some test data generator tool. Test data can be stored in binary or text files. Test data should also be stored in the test repository together with the test specification.

Test specification is selected, loaded and executed by the test execution engine similarly, as application software is selected, loaded and executed by operation systems. The test execution engine should not operate on the tested object directly, but though plug-in modules similarly as an application software accesses devices through drivers which are installed on the operation system.

The difference between the concept of test execution engine and operation system is that the test execution engine monitors, presents and stores the status, results, time stamp, length and other information for every Test Step of a Test Sequence, but typically an operation system does not perform such profiling of a software execution.

Advantages of using a test execution engine:
1) Test results are stored and can be viewed in a uniform way, independent of the type of the test
2) Easier to keep track of the changes
3) Easier to reuse components developed for testing

Difference between Monkey testing an Ad-hoc testing

Monkey Testing : Monkey is random testing, you don't know about the application.
Ad-hoc Testing : Ad-hoc is informal testing where you know about the application well in hand.

Monkey Testing : Testing a system or an Application on the fly, i.e just few tests here and there to ensure the system or an application does not crash out.
Ad-hoc testing : A testing phase where the tester tries to 'break' the system by randomly trying the system's functionality. Can include negative testing as well.

Monkey Testing

Testing by means of a random selection from a large range of inputs and by randomly pushing buttons, ignorant on how the product is being used.

Or

Testing a system or an Application on the fly, i.e. just few tests here and there to ensure the system or an application does not crash out.

Or

In computer science, a monkey test is a unit test that runs with no specific test in mind. The monkey in this case is the producer of any input. For example, a monkey test can enter random strings into text boxes to ensure handling of all possible user input or provide garbage files to check for loading routines that have blind faith in their data.

Or

Testers use the term monkey when referring to a fully automated testing tool. This tool doesn’t know how to use any application, so it performs mouse clicks on the screen or keystrokes on the keyboard randomly. The test monkey is technically known to conduct stochastic testing, which is in the category of black-box testing.

There are two types :
1) Smart Monkeys
2) Dumb Monkeys

1) Smart Monkeys : Are valuable for load and stress testing.
                             They will find a significant number of bugs.
                             Very expensive to develop.

2) Dumb Monkeys : Inexpensive to develop.
                             Able to do basic testing.
                             Can find only few bugs.

Some Major Test cases for web application cookie testing:

The first obvious test case is to test if your application is writing cookies properly on disk.
You can use the Cookie Tester application also if you don’t have any web application to test but you want to understand the cookie concept for testing.

Test cases:

1) As a Cookie privacy policy make sure from your design documents that no personal or sensitive data is stored in the cookie.

2) If you have no option than saving sensitive data in cookie make sure data stored in cookie is stored in encrypted format.

3) Make sure that there is no overuse of cookies on your site under test. Overuse of cookies will annoy users if browser is prompting for cookies more often and this could result in loss of site traffic and eventually loss of business.

4) Disable the cookies from your browser settings: If you are using cookies on your site, your sites major functionality will not work by disabling the cookies. Then try to access the web site under test. Navigate through the site. See if appropriate messages are displayed to user like “For smooth functioning of this site make sure that cookies are enabled on your browser”. There should not be any page crash due to disabling the cookies. (Please make sure that you close all browsers, delete all previously written cookies before performing this test)

5) Accepts/Reject some cookies: The best way to check web site functionality is, not to accept all cookies. If you are writing 10 cookies in your web application then randomly accept some cookies say accept 5 and reject 5 cookies. For executing this test case you can set browser options to prompt whenever cookie is being written to disk. On this prompt window you can either accept or reject cookie. Try to access major functionality of web site. See if pages are getting crashed or data is getting corrupted.

6) Delete cookie: Allow site to write the cookies and then close all browsers and manually delete all cookies for web site under test. Access the web pages and check the behavior of the pages.

7) Corrupt the cookies: Corrupting cookie is easy. You know where cookies are stored. Manually edit the cookie in notepad and change the parameters to some vague values. Like alter the cookie content, Name of the cookie or expiry date of the cookie and see the site functionality. In some cases corrupted cookies allow to read the data inside it for any other domain. This should not happen in case of your web site cookies. Note that the cookies written by one domain say rediff.com can’t be accessed by other domain say yahoo.com unless and until the cookies are corrupted and someone trying to hack the cookie data.

8) Checking the deletion of cookies from your web application page: Some times cookie written by domain say rediff.com may be deleted by same domain but by different page under that domain. This is the general case if you are testing some ‘action tracking’ web portal. Action tracking or purchase tracking pixel is placed on the action web page and when any action or purchase occurs by user the cookie written on disk get deleted to avoid multiple action logging from same cookie. Check if reaching to your action or purchase page deletes the cookie properly and no more invalid actions or purchase get logged from same user.

9) Cookie Testing on Multiple browsers: This is the important case to check if your web application page is writing the cookies properly on different browsers as intended and site works properly using these cookies. You can test your web application on Major used browsers like Internet explorer (Various versions), Mozilla Firefox, Netscape, Opera etc.

10) If your web application is using cookies to maintain the logging state of any user then log in to your web application using some username and password. In many cases you can see the logged in user ID parameter directly in browser address bar. Change this parameter to different value say if previous user ID is 100 then make it 101 and press enter. The proper access message should be displayed to user and user should not be able to see other users account.

To more information about What is Cookie? visit : Cookie

STAF

The Software Testing Automation Framework (STAF) is an open source, multi-platform, multi-language framework designed around the idea of reusable components, called services (such as process invocation, resource management, logging, and monitoring). STAF removes the tedium of building an automation infrastructure, thus enabling you to focus on building your automation solution. The STAF framework provides the foundation upon which to build higher level solutions, and provides a pluggable approach supported across a large variety of platforms and languages.

To know more about STAF please visit : http://staf.sourceforge.net/

Test Automation Framework

A Test Automation Framework is a set of assumptions, concepts and tools that provide support for automated software testing. The main advantage of such a framework is the low cost for maintenance. If there is change to any test case then only the test case file needs to be updated and the Driver Script and Startup script will remain the same. There's no need to update the scripts in case of changes to the application.

What should a test harness include?

Test harnesses should include the following capabilities:

1) A standard way to specify setup (i.e., creating an artificial runtime environment) and cleanup.

2) A method for selecting individual tests to run, or all tests.

3) A means of analyzing output for expected (or unexpected) results.

4) A standardized form of failure reporting.

Test Harness

A test harness or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs.

It has two main parts:

1) Test execution engine

2) Test script repository.

Test harnesses allow for the automation of tests. They can call functions with supplied parameters and print out and compare the results to the desired value. The test harness is a hook to the developed code, which can be tested using an automation framework.

The typical objectives of a test harness are to:

1) Automate the testing process.

2) Execute test suites of test cases.

3) Generate associated test reports.

Benefits of test harness :

1) Increased productivity due to automation of the testing process.

2) Increased probability that regression testing will occur.

3) Increased quality of software components and application.

Globalization Testing

The goal of globalization testing is to detect potential problems in application design that could inhibit globalization. It makes sure that the code can handle all international support without breaking functionality that would cause either data loss or display problems.

Globalization testing checks proper functionality of the product with any of the culture/locale settings using every type of international input possible.

Proper functionality of the product assumes both a stable component that works according to design specification, regardless of international environment settings or cultures/locales, and the correct representation of data.

For more details please visit :
http://www.onestoptesting.com/globalization-testing/

Localization Testing

Localization (L10N) is the process of customizing a software application that was originally designed for a domestic market so that it can be released in foreign markets.

This process involves translating all native language strings to the target language and customizing the GUI so that it is appropriate for the target market. Depending on the size and complexity of the software, localization can range from a simple process involving a small team of translators, linguists, desktop publishers and engineers to a complex process requiring a Localization Project Manager directing a team of a hundred specialists.

Localization is usually done using some combination of in-house resources, independent contractors and full-scope services of a localization company.

For more details please visit : http://www.onestoptesting.com/localization-testing/

Customer Test

This is another term for an acceptance test.

GUI Test

GUI tests test the graphical user interface. GUI tests are considered functional tests. Applications are used to simulate users interacting with the system such as entering text into a field or clicking a button. Verifications are then made based on the response from the UI or system.

Developer Test

This is another term for a unit test.

Unit Test

A unit test is a method used to verify that a small unit of source code is working properly. Unit tests should be independent of external resources such as databases and files. A unit is generally considered a method.

Dummy Objects

Dummy objects are used when methods require an object as part of their method or constructor. However, in this case the object is never used by the code under test. As such, a common dummy object is null.

Fake

Fake objects are yet another type of test doubles. Fake objects are similar to test stubs, but replace parts of the functionality with their own implementation to enable testing to be easier for the method.

Mock

Mock objects are also a form of test double and work in a similar fashion to stub objects. Mocks are used to simulate the behavior of a complex object. Any interactions made with the mock object are verified for correctness, unlike stub objects.

Stub

A test stub is a specific type of test double. A stub is used when you need to replicate an object and control the output, but without verifying any interactions with the stub object for correctness. Many types of stubs exist, such as the responder, saboteur, temporary, procedural, and entity chain.

Test Double

When we cannot, or choose not, to use a real component in unit tests, the object that is substituted for the real component is called a test double.

Behavior Driven Development (BDD)

Building on top of the fundamentals of TDD, BDD aims to take more advantage of the design and documentation aspects of TDD to provide more value to the customer and business.

Test Driven Development (TDD)

Test Driven Development is an Agile Software Development process where a test for a procedure is created before the code is created.

Test Fixture

Test fixtures refer to the state a test must be in before the test can be run. Test fixtures prepare any objects that need to be in place before the test is run. Fixtures ensure a known, repeatable state for the tests to be run in.

Fail

In the case of a fail, the functionality being tested has changed and as a result no longer works as expected. When represented on a report, this is represented as red.

Pass

A pass indicates that everything is working correctly. When represented on a report or user interface (UI), it is represented as green.

Test

A test is a systematic procedure to ensure that a particular unit of an application is working correctly.

xUnit

xUnit refers to the various testing frameworks which were originally ported from sUnit.
Tools such as jUnit, qUnit, and nUnit fall into the xUnit family.

JUnit

JUnit is a unit testing framework for the Java programming language. JUnit has been important in the development of test-driven development, and is one of a family of unit testing frameworks collectively known as xUnit that originated with SUnit.

Penetration test

A penetration test is a method of evaluating the security of a computer system or network by simulating an attack from a malicious source, known as a Black Hat Hacker, or Cracker. The process involves an active analysis of the system for any potential vulnerabilities that may result from poor or improper system configuration, known and/or unknown hardware or software flaws, or operational weaknesses in process or technical countermeasures. This analysis is carried out from the position of a potential attacker, and can involve active exploitation of security vulnerabilities. Any security issues that are found will be presented to the system owner together with an assessment of their impact and often with a proposal for mitigation or a technical solution. The intent of a penetration test is to determine feasibility of an attack and the amount of business impact of a successful exploit, if discovered. It is a component of a full security audit.

Dictionary attack

A dictionary attack is a method of breaking into a password-protected computer or server by systematically entering every word in a dictionary as a password. A dictionary attack can also be used in an attempt to find the key necessary to decrypt an encrypted message or document.

Dictionary attacks work because many computer users and businesses insist on using ordinary words as passwords. Dictionary attacks are rarely successful against systems that employ multiple-word phrases, and unsuccessful against systems that employ random combinations of uppercase and lowercase letters mixed up with numerals. In those systems, the brute-force method of attack (in which every possible combination of characters and spaces is tried up to a certain maximum length) can sometimes be effective, although this approach can take a long time to produce results.

Vulnerability to password or decryption-key assaults can be reduced to near zero by limiting the number of attempts allowed within a given period of time, and by wisely choosing the password or key. For example, if only three attempts are allowed and then a period of 15 minutes must elapse before the next three attempts are allowed, and if the password or key is a long, meaningless jumble of letters and numerals, a system can be rendered immune to dictionary attacks and practically immune to brute-force attacks.

A form of dictionary attack is often used by spammers. A message is sent to every e-mail address consisting of a word in the dictionary, followed by the at symbol (@), followed by the name of a particular domain. Lists of given names (such as frank, george, judith, or donna) can produce amazing results. So can individual letters of the alphabet followed by surnames (such as csmith, jwilson, or pthomas). E-mail users can minimize their vulnerability to this type of spam by choosing usernames according to the same rules that apply to passwords and decryption keys -- long, meaningless sequences of letters interspersed with numerals.

Brute force attack

In cryptography, a brute force attack is a strategy used to break the encryption of data. It involves traversing the search space of possible keys until the correct key is found.

The selection of an appropriate key length depends on the practical feasibility of performing a brute force attack. By obfuscating the data to be encoded, brute force attacks are made less effective as it is more difficult to determine when one has succeeded in breaking the code.

Warchalking

Warchalking is the drawing of symbols in public places to advertise an open Wi-Fi wireless network.

The word is formed by analogy to wardriving, the practice of driving around an area in a car to detect open Wi-Fi nodes. That term in turn is based on wardialing, the practice of dialing many phone numbers hoping to find a modem.

Having found a Wi-Fi node, the warchalker draws a special symbol on a nearby object, such as a wall, the pavement, or a lamp post. Those offering Wi-Fi service might also draw such a symbol to advertise the availability of their Wi-Fi location, whether commercial or personal.

War dialing And wardriving

War dialing or wardialing is a technique of using a modem to automatically scan a list of telephone numbers, usually dialing every number in a local area code to search for computers, Bulletin board systems and fax machines. Hackers use the resulting lists for various purposes, hobbyists for exploration, and crackers - hackers that specialize in computer security - for password guessing.

A single wardialing call would involve calling an unknown number, and waiting for one or two rings, since answering computers usually pick up on the first ring. If the phone rings twice, the modem hangs up and tries the next number. If a modem or fax machine answers, the wardialer program makes a note of the number. If a human or answering machine answers, the wardialer program hangs up. Depending on the time of day, wardialing 10,000 numbers in a given area code might annoy dozens or hundreds of people, some who attempt and fail to answer a phone in two rings, and some who succeed, only to hear the wardialing modem's carrier tone and hang up. The repeated incoming calls are especially annoying to businesses that have many consecutively numbered lines in the exchange, such as used with a Centrex telephone system.

A more recent phenomenon is wardriving, the searching for wireless networks (Wi-Fi) from a moving vehicle. Wardriving was named after wardialing, since both techniques involve brute-force searches to find computer networks. The aim of wardriving is to collect information about wireless access points.

Common techniques for Security Testing

1) Network scanning

2) Vulnerability scanning

3) Password cracking

4) Log review

5) Integrity checkers

6) Virus detection

7) War dialing

8) War driving (wireless LAN testing)

9) Penetration testing

In actual practice combination of many such techniques may be used to have a more comprehensive assessment of the overall security aspect.

Who should do the Security Testing?

Majority of the security testing techniques are manual, requiring an individual to initiate and conduct the test. Automation tools can be helpful in executing simple tasks, whereas complicated tasks continue to depend largely on the intelligentsia of the test engineer.

Irrespective of the type of testing, the testing engineers that plan and conduct security testing should have significant security and networking related knowledge, including expertise of following areas:

1) Network security

2) Firewalls

3) Intrusion detection system

4) Operating systems

5) Programming and networking protocols like TCP/IP

Objectives of Security Testing

1. To ensure that adequate attention is provided to identify the security risks,
2. To ensure that a realistic mechanism to define & enforce access to the system is in place,
3. To ensure that sufficient expertise exists to perform adequate security testing,
4. To conduct reasonable tests to confirm the proper functioning of the implemented security measures.

When do we use Security Testing?

Security testing is carried out when some important information and assets managed by the software application are of significant importance to the organization. Failures in the software security system can be serious especially when not detected, thereby resulting in a loss or compromise of information without the knowledge of that loss.

The security testing should be performed both prior to the system going into the operation and after the system is put into operation.

Rigorous security testing activities are performed to demonstrate that the system meets the specified security requirements & identify the left out security vulnerabilities, if any.

The extent of testing largely depends upon the security risks, and the test engineers assigned to conduct the security testing are selected according to the estimated sophistication that might be used to penetrate the security.