Software Testing Techniques: A Comprehensive Guide

Posted on May 16, 2024 in Computers

Software Testing Techniques

9: Input Domain Partitioning & Boundary Test Cases

Domain Testing

Generate test cases by assigning specific values to input variables based on analysis of the input domain.

  • Input Space: x1,…,xn represents an input space where each input corresponds to a single data item. Together, they form the input vector. When the vector takes on a specific value, it is called a test point or test case.
  • Input Domain: All points that are allowable input combinations.
  • Sub Domain: Subset of input in the form f(x1,…,xn) < K, where < can be (>, =, =/=, ≤, ≥).
  • Domain Partition: Partition of the input domain into mutually exclusive sub-domains.
  • Boundary: Where two sub-domains meet in the form f(x1,…,xn) = K. It is linear if defined by a1x1+…+anxn=K.
  • Boundary Point: A point on a boundary. A boundary is open to a sub-domain if none of the boundary points belong to the sub-domain. If all points are boundary points, then it is closed.
  • Vertex Point: Where two boundaries intersect.

Boundary Problems

  • Closure Problems: Problems with whether the boundary points belong to the sub-domain under consideration.
  • Boundary Shift: Disagreement with where a boundary is between sub-domains.
  • Missing Boundary: Self-explanatory.
  • Extra Boundary: Too many partitions.

EPC (Extreme Point Combination)

EPC is a testing tool for domains. It tests extreme input values to challenge system limits.

  • EPC for 1-Dimensional Sub-Domains: Use four values: “under, min, max, and over” + an interior point. For sample input 0 ≤ x < 21, u, m, m, o would be (-1, 0, 10, 21, 22).
  • EPC for Higher Dimension Sub-Domains: See Fig 1.

Weak Nx1

Uses n ON points and 1 OFF point for each boundary. Boundary points are “ON” points. Pick an OFF point close to the boundary so a small change is detected. The OFF point is ε distance away from the boundary. Testing for Weak N x 1 of [0,21), the ON points would be 0 and 21, while the OFF points would be x = -1 and 20.

Weak 1×1

Uses fewer points than Weak Nx1. Uses one ON point for each boundary. The OFF point is ε distance from the ON point and perpendicular to the boundary.

Other Testing Strategies

  • Strong boundary testing
  • Linear approximations

Queueing Testing as Boundary Testing

Queueing testing is associated with systems where a queue is involved, such as the internet.

  • Priority: Selections based on priority.
  • Buffer Capacity: Boundary testing for queues.
  • Pre-emptive: Can items be emptied to make way for higher priority items?
  • Batching: Wait until a batch is full before processing.

Comparing Testing Systems

  • Weak 1×1: Uses few testing points and can detect most boundary problems; should be used most often.
  • Weak Nx1: Should be used to patch holes of Weak 1×1 if needed.
  • Strong Testing: Should be used if inconsistencies exist in boundaries.

10: Coverage and Usage Testing Based on Finite-State Machines and Markov Chains

FSM (Finite-State Machine)

A basic model of program execution or behavior.

  • Static Elements: States and state transitions. The number of states is finite. State A to B should be labeled A-B.
  • Dynamic Elements: Input provided to the FSM and output generated.

The system is in one state at a time and moves between states with transitions. If the output and input are uniquely determined by the current state and input, it is called “deterministic”.

Graphical Representations of FSMs

  • Each state as a node.
  • Transitions as a link from one node to another.
  • Input and output are associated with transitions and represented as link weights or annotations (Fig 2).

The program starts in the “initial” state and ends in the “final” state.

Representations of FSMs

  • States are lettered.
  • Can be represented as a table (Fig 3).
  • Can be represented as a list, i.e., {C, B “unable to receive package”}.

Problems Associated with FSMs

  • State Problems: Missing, extra, or incorrect states.
  • Transition Problems: Missing, extra, or incorrect transitions.
  • Input Problems and Output Problems: Use a checklist to prevent these problems with information source and data collection.

Testing for Correct States and Transitions

  • State or Node Coverage: Ensure that every state can be reached in a test.
  • Transition or Link Coverage: Ensure that each link is covered by a test case.
  • Test Case Sanitization: Ensure that test cases are valid and cover all necessary scenarios.

Operational Profile

A statistical representation of how a software system will be used in production. It describes the various types of operations that will be performed, the frequency of these operations, and the conditions under which they will be executed. Essentially, an operational profile models the expected usage patterns and environmental conditions of the software.

Markov Chains & Operational Profiles

For larger systems, use Markov chains and operational profiles. A Markov chain that constitutes the specific operational profile for the system is called a Markov OP.

UMM (Unified Markov Model)

Hierarchical Markov chains in a framework called Unified Markov Models to support testing, performance evaluation, and reliability. Makes it easier to combine similar operations into one big one.

Constructing a UMM

  1. Construct basic FSMs with an emphasis on external functions and operations.
  2. Complete the usage model by assigning transition probabilities based on measurements or surveys of target customers.

Thresholds for UMM

  • Overall probability threshold
  • Stationary probability threshold
  • Transition probability threshold

11: Control Flow, Data Dependency, and Interaction Testing

Control Flow Testing (CFT)

An extension to coverage-based FSM testing with a specialized type of FSM called Control Flow Graphs (CFGs).

CFGs have:

  • Nodes: Represent a unit of information processing.
  • Links: Represent the relation “is followed by”.
  • Initial/Entry and Final/Exit Nodes: Represent the start and end points of the control flow.
  • Outlinks: A link that originates from a node.
  • Inlinks: A link that ends up in a node.
  • Decision Junction and Processing Nodes: A node associated with multiple outlinks is a decision junction, a node with multiple inlinks is a junction, and a node that is neither is called a processing node.
  • Path: A path that starts from an entry node and goes to an exit node.
  • Segment: A subpart of a path.
  • Loop: A path that contains a loop.

The basic idea for CFT is to select paths and test cases and sensitize them by assigning corresponding input values. Fig(5) is a CFG.

Strategy for Path Selection for CFG

  1. CFG decomposition
  2. Bottom-up path definition

(Fig 6) = while loop CFG

CFT vs. FSM

  • Test cases are larger than FSM-based testing.
  • CFT is necessary for large traditional software systems.

P-Use and C-Use

The use of variables or data items in CFG decisions is called P-use or C-use.

Data Dependency Relations

  • D-U Relation: When a data item is used, we need to obtain its value defined previously.
  • D-D Relation: A data item is redefined without a previous definition.
  • U-U Relation: Ignored in DDA and DFT.
  • U-D Relation: Called anti-usage.

Data Flow Testing (DFT)

DFT is used to test the correct handling of data dependencies during program execution. Test cases are derived from data dependency analysis (DDA) with a focus on D-U relations and modeled as a data dependency graph (DDG).

DDGs

Each node represents the definition of a data item, such as a variable or constant. For the statement “z <- x + y”, it can be represented by (Fig 7 188). In DFT, we directly focus on the data dependencies captured in DDGs instead of the computational sequences or control flow in CFT.

Constructing DDGs

  1. Each node represents the definition of a data item x, denoted as D(x) and represented as an x inside an oval in a DDG.
  2. There are three types of nodes: output or result nodes, input or constant nodes, and intermediate or storage nodes (Fig 8 190).

Characteristics of DDGs

  1. Usually one output data item or variable or at most a few.
  2. More input variables and constants.
  3. Multiple inlinks are common.
  4. Tree-shaped is the most common shape.

DDG Construction

  1. Identify output variables.
  2. Backward chaining to resolve these variables using other variables and constants.
  3. If there are unresolved variables, repeat the previous step for each one until none are left. An unresolved variable is one that is neither an input variable nor a pre-defined constant.

DDG for code (Fig 9 194)

Differences Between DFT, CFT, and FSM

  • CFT relies on CFGs, a type of FSM, while DFT relies on DDGs, which are not FSMs.
  • CFGs closely resemble program code.
  • DDGs capture more details about interactions.
  • DDGs are more complex than CFGs.
  • The ability for loop handling is much more limited in DFT than in CFT.

Testing Techniques: Adaptation, Specialization, and Integration

Large-scale testing is split into various sub-phases, which allow testing to be carried out on a more manageable scale.

Testing Sub-Phases

(Fig 10) V-model / waterfall process. When problems are reported by customers during use, diagnosis testing can be used to recreate and diagnose these problems. Controlled product release and operation use by limited customers is beta testing, a testing sub-phase. Acceptance testing is attached to the end of system testing, performed right after system testing to determine if the product should be released. Unit testing. Regression testing, which is used when adding new components. Unit testing tests small software units one at a time. It focuses on the implementation details and uses white box testing techniques. It focuses on executable statements and related control and data elements.

  • Component Testing: Tests a software component at a time.
  • Integration Testing: Deals with the integration of different product components to work together.
  • Acceptance Testing & Product Release:
    • What are the expectations of product reliability in customer settings if the product is released now?
    • What is the appropriate level of post-release product support?

Chart for sub-phase testing (Fig 10 209).

Component Testing Types

  • Type I: Loosely grouped components.
  • Type II: Tightly packed components.

Specialized Test Tasks and Techniques

  • Defect Diagnosis Testing: When problems are reported by customers during operation, diagnosis testing is used to help with problem diagnosis by recreating the problem.
  • Defect-Based Testing: Based on discovered defects or potential defects. Strategies include: defect risk-based testing, defect injection testing, and mutation testing.

Other Types of Testing

  • Performance testing
  • Stress testing
  • Usability testing

Simulation is an important technique that simulates the behavior of a system. Prototyping is similar. Timing and sequencing analyses, event tree analysis.

Web Testing Techniques

: HTML syntax checking, Link checking, Form testing, Verification of end to end transactions, java component testing, load testing, usability testing, browser rendering (TAR) = top access report, lists frequenctly accessed services or web pages, CPR = call pair report lists call pairs and the associated frequency. Heigherarchy of web testing, || Top Level: TAR/// Middle Level: UMM for groups of TAR entries// Bottom Level Detailed UMM or other models