ST Module 1 Notes

Soft Softwa warre Testi esting ng-1 -15I 5IS6 S63 3 I

Dept Dept., ., of ISE ISE

Modu Module le -

Module1- BASIS !" S!"T#A$E S!"T#A$E TESTI%& BASI DE"I%ITI!%S' Error' When people make mistakes while coding, we call these mistakes bugs. Errors tend to propagate; a requirements error may be magnified during du ring design and amplified still more during coding. "ault' a fault is the result of an error. It is more precise to say that a fault is the representation of an error, and can be represented as narrative text, data flow diagrams, hierarchy charts, source code, and so on. Defe(t synonym for fault fault,, as is )ug. When a designer makes an error of omission, the Defe(t is a good synonym resulting fault is that something is missing that should be present in the representation. T*pes of "ault' 



Fault of Omission: Faults of omission occur when we fail to enter correct information. Of these two types, faults of omission are more difficult to detect and resole.

faultt of comm commis issi sion on occu occurs rs when when we ente enterr some someth thin ing g into into a Fault Fault of Commis Commission sion:: ! faul representation that is incorrect. E" subtracting a figure that should have been added

"ailure' a failure occurs when a fault e"ecutes and is usually represented to be source code, or loaded ob#ect. In(ident' when a failure occurs, it may or may not be readily apparent to the user $or customer or tester%. !n incident is the symptom associated with a failure that alerts the user to the occurrence of a failure. Test' testing is obiously concerned with errors, faults, failures, and incidents. ! test is the act of e"ercis e"ercising ing softwa software re with with test test cases. cases. ! test test has two disti distinct nct goals& goals& to find failures or to demonstrate correct execution. Test ase' test case has an identity and is associated with a program behaior. ! test case also has a set of inputs and a list of e"pected outputs. Software Testing' Definition' !ccording to the definition gien by 'ae (elperin and William ). *et+el  -oftware testing can be stated as the process of alidating and erifying that a software&  eets the requirements that guided its design and deelopment.  Works as e"pected.  -atisfies the needs of stakeholders. -oftware -oftware testing is the process of analy+ing analy+ing a software software item to detect the differences differences between e"isting and required conditions $that is, bugs% and to ealuate the features of the software item. One of the test Krupa K S, Asst. Professor

1 Global Academy of Technology echnolog y



Modu Module le -

techniques includes the process of e"ecuting a program or application with the intent of finding software bugs $errors or other defects%.

%e(essit*'     

/o determine a set of test cases. /o improe the quality of software. /o build confidence in the software /o demonstrate to the deeloper and the customer that the software meets its requirements. /o discoer faults or defects in the software.

Software Testing +ife *(le'

-ignificance of testing life cycle is that it depicts how and when bugs are introduced, tested and fi"ed in a software deelopment life cycle $-'0)%. ases putting Bugs I%' 





1equirement specification& Errors are made during requirement collection and analysis. 'esign& /esters work with deelopers in determining what aspects of a design are testable and under what parameter parameter those testers testers work. Errors Errors introduced in preious preious phase results results into faults faults and more errors are made. )oding& 'esign is implemented. Faults and errors introduced in preious phases propagate and more errors are made.

ase "I%DI%& Bugs' 

/esting& /est strategy or planning is done to generate test cases or scenarios, which are e"ecuted to find bugs or errors. 2ugs found are reported in error logs.

ases getting Bugs !T' 

Fault classification& 1eported faults are assigned to different seerity classes like catastrophic, serious, mild etc.



Fault isolation& )auses and location of different faults are pinpointed.



Fault resolution& 3atch work is done to fi" the faults which gie another opportunity for error and new faults to be made.

Krupa K S, Asst. Professor




Modu Module le -

S!"T#A$E /A+IT0 '-

-oftware quality is a multidimensional quantity and is measurable. /ualit* Attri)utes' /here e"ist seeral measures of software quality. /hese can be diided into stati( and d*nai( quality attributes. 

-tatic quality attributes refer to the actual code and related documentation.



'ynamic quality attributes relate to the behavior of the application while in use.

Stati( 2ualit* attri)utes include structured, maintainable, testable code along with aailability of correct and complete documentation. Krupa K S, Asst. Professor




Modu Module le -

4ou might hae come across complaints such as 53roduct 6 is e"cellent, I like the features it offers, but its user manual stinks78 In this case, the user manual brings down do wn the oerall product quality. !s a part of correctie maintenance on any application code, we need to understand portions of the code before we make any changes to it. /his is where attributes such as code documentation, understandability, understandability, and structure come into play. ! poorly structured and poorly documented piece of code will be harder to understand, modify and difficult to test.

include software software reliabili reliability ty,, correctnes correctness, s, completeness completeness,, consistency consistency,, D*nai( D*nai( 2ualit* 2ualit* attri)ut attri)utes es include usabil usability ity,, and perfor performan mance. ce. 'ynami 'ynamicc quality quality attrib attribute utess are general generally ly determ determine ined d through through multip multiple le e"ecutions of a program.  











$elia)ilit* refers to the probability of failurefree operation. orre(tness refers to the correct operation of an application and is always with reference to some artifact. For a tester, correctness is w.r.t requirements; for a user, it is often with respect to a user manual. opleteness refers to the aailability of all features listed in the requirements, or user manual. !n incomplete software is one that does not fully implement all features required. )ompleteness implements features that might itself be a subset of a larger set of features that are to be implemented in some future ersion of the application. /herefore eery piece of software that is correct is also complete with respect to some feature set. onsisten(* refers to adherence to a common set of conentions and assumptions. For e"ample, to follow a common color coding conention in a user interface. !n e"ample of inconsistency is to display date of birth of a person in different formats, without any regard for the user9s preferences. sa)ilit* refers to the ease with which an application can be used. :sability testing refers to the testing testing of a product by its potential potential users. :sers in turn test test for ease of use, functionalit functionality y as e"pected, performance, safety, and security. :sers thus sere as an important source of tests that deelo deeloper perss or tester testerss within within the organ organi+a i+atio tion n might might not hae concei conceied ed.. :sabil :sability ity testi testing ng is sometimes referred to as usercentric testing.

refers rs to the the time time the the appl applic icat atio ion n take takess to perfo perform rm a requ reques este ted d task task and and is erforan(e refe considered as a nonfunctional requirement. It is specified in terms such as 5/his task must be performed at the rate of 6 units of actiity in one second on a machine running at speed 4, 4, haing  gigaby gigabytes tes of memory memory.8 .8 For example, the performance requirement requirement for a compiler might be stated in terms of the minimum average time to compile of a set of numerical numerical applications. $elia)ilit* can be considered as a statistical measure of correctness.



Software Testing-15IS63 I

Dept., of ISE

Module -

Software reliability is the probability of failure-free operation of software over a given time interval and under given conditions . /his definition requires the user operational profile, which is difficult or impossible to estimate accurately. *oweer, if an operational profile can be estimated for a gien class of users, then an accurate estimate of the reliability can be found for this class of users. Software reliability is the probability of failure-free operation of software in its intended environment . /he term 5enironment8 refers to the software and hardware elements needed to e"ecute the application, such as operating system, hardware requirements, and any other applications needed for communication.

$E/I$EME%TS, BEA4I!$, A%D !$$ET%ESS'

3roducts, software in particular, are designed in response to requirements.



1equirements specify the functions that a product is e"pected to perform.



Once the product is ready, it is the requirements that determine the e"pected b ehaior.



'uring deelopment phase, the requirements might get changed from what was stated originally.



/esters test the e"pected behaior by understanding the requirements.

Eaple *ere are the two requirements, each of which leads to a different program.

1equirement <&

It is required to write a program that inputs two integers and outputs the ma"imum of these.

1equirement =&

It is required to write a program that inputs a sequence of integers and outputs the sorted ersion of this sequence.

/he 1equirement < as stated aboe fails to proide an answer if the tester wants to know if the two integers are to be input to the program on one line followed by a carriage return, or on two separate lines with a carriage return typed in after each number. /his e"ample illustrates the incompleteness of 1equirement <. /he second requirement in the aboe e"ample is ambiguous. It is not clear from this requirement whether the input sequence is to be sorted in ascending or descending order. /he behaior of the sort program, written to satisfy this requirement, will depend on the decision taken by the programmer while writing sort. Krupa K S, Asst. Professor

5 Global Academy of Technology


Dept., of ISE

Module -



/esters are often faced with incomplete and>or ambiguous requirements.



In such situations, a tester may resort to a ariety of ways to determine what behaior to e"pect from the program under test.



1egardless of the nature of the requirements, testing requires the determination of the expected behaior of the program under test.



/he observed behaior of the program is compared with the e"pected behaior to determine if the program functions as desired.

Input Doain and rogra orre(tness 

! program is considered correct if it behaes as desired on all possible test inputs.



/esting a program on all possible inputs is known as exhaustive testing.



If the requirements are complete and unambiguous, it should be possible to determine the set of all possible inputs.



/he set of all possible inputs to a program ? is known as the input domain, or input space, of .



/he set of all possible inputs is too large for the program to be e"ecuted on each input. For e"ample, suppose that the max program for 1equirement < is to be tested for integers range from @A=,BCD to A=,BCB, then a total of =A= e"ecutions of max is required.







For 1equirement< the input domain of ma" to be the set of all pair of integers where each elements in the pair is in the range @A=,BCD to A=,BCB. 1equirement = can be modified with input domain as 5 character !9 to represent ascending and '9 to represent descending order of sorting , followed by a sequence of +ero or more integers ending with a period 8 For e"ample, following are three elements in the input domain of sort: < A -3 15 12 55.>




Dept., of ISE

Module -

< D 23 78.> < A .>

orre(tness ' A program is considered correct if it behaves as expected on each element of its input domain. 4alid and inalid inputs'

In the e"amples aboe, the input domains are deried from the requirements. *oweer, the requirements are incomplete. /he requirement mentions that the request characters can be 5!8 or 5'8, but it fails to answer the question 5What if the user types a different character 8 other than 5!8 or 5'8 which is considered as inalid input to sort. /he requirement for sort does not specify what action it should take when an inalid input is encountered. Identifying the set of inalid inputs and testing the program against these inputs is an important part of the testing actiity. /esting a program against inalid inputs might reeal errors in the program. Eaple 1& -uppose that we are testing the sort program against the input& G H B < . J. !s input character E is not in the input domain, the sort program enters into an infinite loop and neither asks the user for any input nor responds to anything typed by the user. /his obsered behaior points to a possible error in sort.

Example 2 the requirements for the sort program do not specify how the program should behae if the sequence of integers to be sorted consists of any character, such as 5K8instead of integers. /hen the program should inform the user that the input is inalid. 2ut this e"pected behaior from sort needs to be tested. /his suggests that the input domain for sort should be modified. In cases where the input to a program is not guaranteed to be correct, it is conenient to partition the input domain into two subdomains. One subdomain consists of inputs that are alid and the other consists of inputs that are inalid. ! tester can then test the program on selected inputs from each subdomain. !$$ET%ESS 4E$SS $E+IABI+IT0' 



/hough correctness of a program is desirable, it is almost neer the ob#ectie of testing. /o establish correctness ia testing on all elements in the input domain is impossible to accomplish. /hus, correctness is established ia mathematical proofs of programs. ! proof uses the formal specification of requirements and the program te"t to proe or disproe that the program will behae as intended.



Software Testing-15IS63 I 

Dept., of ISE

Module -

While correctness attempts to establish that the program is error free, testing attempts to find if there are any errors in it.



1emoal of errors from the program usually improes the chances, or the probability, of the program e"ecuting without any failure.



/esting, debugging, and the error remoal processes together increase our confidence in the correct functioning of the program under test.

Example 1.11 This example illustrates why the probability of program failure might not change upon error removal. Consider the following program that inputs two integers x and y and prints the value of f ( x, y) or g( x, y) depending on the condition x < y.

The above program uses two functions f and g not defined here. Let us suppose that function fproduces an incorrect result whenever it is invoked with x  y and that f (x! y) " g(x! y)! x  y. #n its present form! the program fails when tested with e$ual input values because function g is invoked instead of function f. %hen the error is removed by changing the condition x < y to x & y! the program fails again when the input values are the same. The latter failure is due to the error in function f. #n this program! when the error in f is also removed! the program will be correct assuming that all other code is correct.

Reliability  5/he reliability of a program Ρ is the probability of its successful e"ecution on a randomly

selected element from its input domain.8  1eliability is a statistical attribute. !ccording to one iew, it is the probability of failure free

e"ecution of a program under gien conditions.  ! comparison of program correctness and reliability reeals that while correctness is a binary

metric, reliability is a continuous metric oer a scale from L to <. ! program can be either correct or incorrect; its reliability can be anywhere between L and <.  E"ample & )onsider a program ? that takes a pair of integers as input. /he input domain of this

program is the set of all pairs of integers. -uppose now that in actual use there are only three pairs that will be input to 3. /hese are


'<(! ) (l! l) (l! l)*+



Dept., of ISE

Module -

/he aboe set of three pairs is a subset of the input domain of ? and is deried from a knowledge of the actual use of 3, and not solely from its requirements.  -uppose also that each pair in the aboe set is equally likely to occur in practice. If it is known

that ? fails on e"actly one of the three possible input pairs then the frequency with which ? will function correctly is

/his number is an estimate of the probability of the successful operation

of ? and hence is the reliability of 3.

Operaonal profles  



!n operational profile is a statistical summary of how a program would be used in practice. !n operational profile is a numerical description of how a program is used. In accordance with the aboe definition, a program might hae seeral operational profiles depending on its users.

Eaple ' )onsider a sort program that, on any gien e"ecution, allows any one of two types of input sequences. One sequence consists of numbers only and the other consists of alphanumeric strings. One operational profile for sort is specified as follows&

!nother operational profile for sort is specified as follows&

/he two operational profiles aboe suggest significantly different uses of sort. In one case, it is used mostly for sorting sequences of numbers, and in the other case, it is used mostly for sorting alphanumeric strings.




Dept., of ISE

Module -

TESTI%& A%D DEB&&I%&'





/esting is the process of determining if a program behaes as e"pected during which errors may be discoered in the program under test. /he process used to determine the cause of this error and remoe it is known as debugging. !s illustrated in Figure below, testing and debugging are often used as two related actiities in a cyclic manner.

Figure : ,


test and debug cycle.



Dept., of ISE

Module -

 Preparing a test plan : 





! test cycle is often guided by a test plan. When relatiely small programs are being tested, a test plan is usually informal and in the tester9s mind, or there may be no plan at all. ! sample test plan considers items such as the method used for testing, method for ealuating the adequacy of test cases, and method to determine if a program has failed or not. ! sample test plan for testing the sort program is shown below

 Construcng Test Data 

! test case is a pair of input data and the corresponding program output.



/he test data are a set of alues& one for each input ariable.



! test set is a collection of test cases.



! test set> test data is sometimes referred to as a test suite.



Software Testing-15IS63 I 





Dept., of ISE

Module -

3rogram requirements and the test plan help in the construction of test data. E"ecution of the program on test data might begin after all or a few test cases hae been constructed. While testing, relatiely small programs testers often generate a few test cases and e"ecute the program against these. 2ased on the results obtained, the tester decides whether to continue the construction of additional test cases or to enter the debugging phase. Example /he following test cases are generated for the sort program using the test plan aboe

/est cases < and = are deried in response to item < in the test plan; A and M are in response to item =. Notice also that the requirements for the sort program do not indicate what should be the output of sort when there is nothing to be sorted. We therefore took an arbitrary decision while composing the 5E"pected output8 for an input that has no numbers to be sorted. /est cases  and C are in response to item A in the test plan.

 Execung the program 

E"ecution of a program under test is the ne"t significant step in testing.








Dept., of ISE

Module -

/he comple"ity of actual program e"ecution is dependent on the program itself. /ester constructs a test harness to aid in program e"ecution. /he harness initiali+es any global ariables, inputs a test case, and e"ecutes the program. /he output generated by the program may be saed in a file for subsequent e"amination by a tester. /he ne"t e"ample illustrates a simple test harness.

Eaple ' /he test harness in Figure below reads an input sequence, checks for its correctness, and then calls sort. /he sorted array sorted_sequence returned by sort is printed using the

print_sequence procedure. /est cases are assumed to be aailable in the /est pool file shown in the figure. In some cases, the tests might be generated from within the harness.

Figure : , simple test harness to test the sort program.

In preparing this test harness we assume that &  sort is coded as a procedure,  the get_input procedure reads the request character and the sequence to be sorted into ariables request_char, num_items, and in_numbers, and  the input is checked prior to calling sort by the check_input procedure.



/he test_setup procedure is usually inoked first to set up the test $ i.e identifying and opening the file containing tests%.



/he check_output procedure seres as the oracle that checks if the program under test behaes correctly. /he report_failure procedure is inoked in case the output from sort is incorrect.



! failure might be reported ia a message on the screen or saed in a test report file. /he print_sequence procedure prints the sequence generated by the sort program.




Dept., of ISE

Module -

/he output generated by print_sequence can also be piped into a file for subsequent e"amination.

TEST ASES'Test case is a scenario made up of a sequence of steps and conditions or variables, where test inputs are provided and the program is run using those inputs, to see how it performs

!n e"pected result is outlined and it is compared with result obtained. )ertain, working conditions present are also considered in the test cases so as to check how the program handles such conditions. 



/est cases are required to measure how program handles tricky situations or errors, and is e"pected to showcase the hidden logical errors by different cases of input. ! typical test case contains the following information&

Test (ase (ontains ' 

Inputs of = types& 3reconditions $prior to test case e"ecution% and the actual inputs that were identified by some testing methods.




Dept., of ISE

Module -



E"pected outputs @ = types & postconditions and actual outputs.



!ny test case should hae an identity and a reason for being written.



1equired to record the e"ecution history of a test case, including when and by whom it was run, the pass>fail result of each e"ecution, ersion of software on which it was run.



/est cases are as aluable as source code.



/est cases need to be deeloped, reiewed, used, managed and saed.

I%SI&T "$!M 4E%% DIA&$AM'-

/esting is concerned with behaiour, and behaiour is orthogonal to the structural iew common to software deelopers.

)onsider a unierse of program behaiour. (ien the program and its specification, consider the set - of specified behaiors and set 3 of program behaiors. Figure aboe shows the relationship among our unierse of discourse as well as the specified and programmed behaiour. Of all the program behaiors, the specified ones are in the circle labeled -, and all those behaiors actually programmed are in 3. Figure below shows, specified, implemented and tested behaiors. /he new circle in the below figure is for test cases.




Dept., of ISE

Module -

Region ! "pecified behaviors that are programmed and tested. Region #! "pecified behaviors that are programmed but not tested. Region $! %nspecified behaviors that are programmed and tested. Region &! "pecified behaviors that are tested but not programmed. Region '! "pecified behaviors that are neither programmed nor tested. Region (! %nspecified behaviors that are programmed but not tested. Region )! %nspecified behaviors that are tested but not programmed. Region *! +ehaviors that are neither specified and tested nor programmed.

IDE%TI"0I%& TE TEST ASES'-

/here are two different types of testing& <. Functional /esting =. -tructural /esting

1. Bla(7 )o8"un(tional testing'

2lack bo" testing is a method of software testing that e"amines the functionality of an application without peering into its internal structures or workings. -ome functional testing methods are 2oundary Palue /esting, Equialence )lass /esting and 'ecision /able2ased testing. Adantages' 

-ame /est )ases can be used in case of any changes in implementation i.e. Independent of -oftware Implementation.



/est case deelopment can be done in parallel with the implementation, *ence reducing the oerall pro#ect deelopment interal




Dept., of ISE

Module -

Disadantages' 

-ignificant 1edundancies among test cases i.e. test cases similar to each other may repeat multiple times.



3ossibility of gaps of untested software. /his may result in a faulty product being deliered to the customer.

9. #ite )o8Stru(tural8lear )o testing'

White bo" testing $also known as clear bo" testing, glass bo" testing, and transparent bo" testing and structural testing% is a method of testing software that tests internal structures or workings of an application, as opposed to its functionality. -ome structural testing methods are 3ath testing and dataflow testing.

TEST- &E%E$ATI!% ST$ATE&IES'

/est case generation  one of the key tasks in any software test actiity.








Dept., of ISE

Module -

/he program under test is e"ecuted against the test cases to determine whether or not it conforms to the requirements. /est generation uses a source document. In informal test methods, the source document resides in the mind of the tester who generates tests based on knowledge of the requirements. In some test processes, requirements document sere as a source for the deelopment of formal models used for test generation.

Figure below summari+es seeral strategies for test generation. 



/he top row in this figure captures informal techniques that are applied directly to the requirements. It assigns alues to input ariables without the use of any rigorous or formal methods. /hese techniques identify input ariables, capture the relationship amongst these ariables, and use formal techniques for test generation such as random test generation and cause@effect graphing.

Figure: 1equirements, models, and test generation algorithms. Krupa K S, Asst. Professor



Dept., of ISE

Module -



!odel based test generation strategies require that a subset of the requirements be modeled using a formal notation. -uch a model is also known as a specification of a subset of requirements. /he tests are then generated with the specification sering as the source.



Finite state machines, statecharts, 3etri nets, and timed input@output automata are some of the



wellknown and used formal notations for modeling arious subsets of requirements. 

/he graphical notations such as sequence and actiity diagrams in :0 are used as models of subsets of requirements.



0anguages based on predicate logic as well as algebraic languages are also used to e"press subsets of requirements in a formal manner.



code-based test generation techniques generate tests directly from the code and are useful when

enhancing e"isting tests based on test adequacy criteria. 

/hese techniques are also used during regression testing when there is often a need to reduce the si+e of the test suite, or prioriti+e tests, against which a regression test is to be p erformed.



-uch techniques take four inputs, the program to be regression tested -, the program Ρ from which - has been deried by making changes, an e"isting test suite  for , and some run time information obtained by e"ecuting Ρ against /. /his run time information may include items such as statement and branch coerage. /he test generation algorithm then uses this information to select tests from  that must be e"ecuted to test those parts of - that hae changed or are affected by the changes made to . /he resulting test suite is usually a subset of .

TEST MET$IS '

/he term 5metric8 refers to a standard of measurement which measures some aspect of the test process.






Dept., of ISE

Module -

In software testing, there e"ist a ariety of metrics. Figure shows a classification of arious types of metrics. etrics can be computed at the organi+ational, process, pro#ect, and product leels. Each set of measurements has its alue in monitoring, planning, and control.

"igure' /ypes of metrics used in software testing and their relationships. 



1egardless of the leel at which metrics are defined and collected, there e"ist four general core areas that assist in the design of metrics. /hese are schedule, quality, resources, and si+e. -chedulerelated metrics measure actual completion times of arious actiities and compare these with estimated time to completion.



Qualityrelated metrics measure the quality of a product or a process.



1esourcerelated metrics measure items such as cost in dollars, manpower, and tests e"ecuted.



-i+erelated metrics measure the si+e of arious ob#ects such as the source code and the number of tests in a test suite.

Organizaonal metrics : 







Organi+ational metrics are useful in oerall pro#ect planning and management. /hey are obtained by aggregating compatible metrics across multiple pro#ects. For e"ample, the number of defects reported after product release, aeraged oer a set of products deeloped and marketed by an organi+ation, is a useful metric of product quality at the organi+ational leel. Quality in organi+ation is ensured by )omputing this metric at regular interals and oerall products released oer a gien duration. Organi+ational leel metrics allow senior management to monitor the oerall strength of the organi+ation and points to areas of weaknesses.




Dept., of ISE

Module -

/hese metrics help the senior management in setting new goals and plan for resources needed to reali+e these goals. 0xample !/he average defect density across all software pro1ects in a company is .)$ defects per 2345. "enior management has found that for the next generation of software products, which they plan to bid, they need to show that product density can be reduced to 6. defects per 2345. /he management thus sets a new goal. 7iven the time frame from now until the time to bid, the management needs to do a feasibility analysis to determine whether or not this goal can be met. 8f a preliminary analysis shows that it can be met, then a detailed plan needs to be worked out and put into action. For example, management might decide to train its employees in the use of new tools and techniques for defect prevention and detection using sophisticated static analysis techniques.

Project etrics : 









3ro#ect metrics relate to a specific pro#ect, for e"ample, the I>O deice testing pro#ect or a compiler pro#ect. /hese are useful in the monitoring and control of a specific pro#ect. /he ratio of actual to planned system test effort is one pro#ect metric. /est effort could be measured in terms of the testermanmonths. !t the start of the system test phase, for e"ample, the pro#ect manager estimates the total system test effort. /he ratio of actual to estimated effort is +ero prior to the system test phase. /his ratio builds up oer time. /racking the ratio assists the pro#ect manager in allocating testing resources. !nother pro#ect metric is the ratio of the number of successful tests to the total number of tests in the system test phase. !t any time during the pro#ect, the eolution of this ratio from the start of the pro#ect could be used to estimate the time remaining to complete the system test process.

Process etrics : 







/he purpose of a process metric is to assess the 5goodness8 of a process. Eery pro#ect uses some test process. E"& 5big bang8 approach is a process used in relatiely small single person pro#ects. 'uring the test process phases like unit test, integration test, system test etc, we can measure how many defects were found in each phase. 'efects found at later stage will be costlier to fi".




Dept., of ISE

Module -

*ence, a metric that classifies defects according to the phase in which they are found assists in ealuating the process itself. 0xample! 8n one software development pro1ect, it was found that '9 of the total defects were reported by customers, ''9 of the defects prior to shipping were found during system test, ##9 during integration test, and the remaining during unit test. /he large number of defects found during the system test phase indicates a possibly weak integration and unit test process. /he management might also want to reduce the fraction of defects reported by customers.

Pro!uct etrics: generic 









3roduct metrics relate to a specific product E" &a compiler for a programming language. /hese are useful in making decisions related to the product, for e", 5-hould this product be released for use by the customer 8 /here are = types of 3roduct metrics a% te *(loati( (opleit* and b% te alstead etri(s . /he cyclomatic comple"ity proposed by /homas c)abe in <BC is based on the control flow of a program. (ien the )F( ( of program ? containing R nodes, H edges, and p connected procedures, the cyclomatic comple"ity P$(% is computed as follows& P$(% S E @ N T =p



It is a measure of program comple"ity; higher alues imply higher comple"ity. /his is a product metric .





P$(% is the comple"ity of a )F( ( that corresponds to a procedure p reachable from the main procedure. !lso, P$(% is not the comple"ity of the entire program, instead it is the comple"ity of a procedure in program ? that corresponds to (. 0arger alues of P$(% tend to imply higher program comple"ity and hence a program more difficult to understand and test than one with a smaller alues. P$(% is  or less are recommended.

 alstead (opleit* measures

were published by late 3rofessor aurice



/able below lists some of the *alstead comple"ity metrics. :sing program si+e $-% and effort $E%, the following estimator has been proposed for the number of errors $2% found during a software deelopment effort&



2 S B.CEL.CCB-L.AAA




Dept., of ISE

Module -

!n adantage of using an estimator such as U is that it allows the management to plan for testing resources. For e"ample, a larger alue of the number of e"pected errors will lead to a larger number of testers and testing resources to complete the test process o er a gien duration. Ta)le ' *alstead measures of program comple"ity and effort.

Pro!uct metrics: OO so"#are : 







/able below lists a sample of product metrics for ob#ectoriented and other applications. 3roduct reliability is a quality metric and refers to the probability of product failure for a gien operational profile. 3roduct reliability of software truly measures the probability of generating a failure causing test input. If the probability is L for a gien operational profile, then the program is perfectly reliable despite the possible presence of errors. /he OO metrics measure program or design comple"ity. ! product with a comple" design will require more test effort to obtain a gien leel of defect density than a product with less comple"ity. Table : A Sample of Product Metrics




Dept., of ISE

Module -

Progress monitoring an! tren!s: 





etrics are often used for monitoring progress where measurements are made on a regular basis oer time. For e"ample, suppose that a browser has been coded, unit tested, and its components integrated. It is now in the system testing phase. We can plot the measure of cumulatie number of defects found oer time. -uch a plot will rise oer time. Eentually, it will likely show a saturation indicating that the product is reaching a stability stage.

"igure' ! sample plot of cumulatie count of defects found oer seen consecutie months in a software pro#ect.




Dept., of ISE

Module -

$tac an! Dynamic etrics : 









3roduct metrics could be classified as static or dynamic. Stati( etri(s are those computed without haing to e"ecute the product. E"& Number of testable entities in an application. D*nai( etri(s require code e"ecution, e". the number of testable entities actually coered by a test suite is a dynamic product metric.

In an organi+ation and pro#ect, the aerage number of testers working on a pro#ect is a static pro#ect metric. Number of defects remaining to be fi"ed could be treated as a dynamic metric as it can be computed accurately only after a code change has been made and the product retested.

Testability : 



















/estability is the 5degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria hae been met.8 = ways to measure testability of a product& Stati( testa)ilit* etri(s . E" -oftware comple"ity ./he more comple" an application, the lower the testability, higher the effort required to test it. D*nai( testa)ilit* etri(s& e" codebased coerage criteria. When it is difficult to generate tests that satisfy the statement coerage criterion is said to hae low testability.

*igh testability is a desirable goal and is achieed by knowing what needs to be tested and how, well in adance. Features to be tested and how they are to be tested must be identified during the requirements stage, further it is updated during the design phase and coding phase. ! testability requirement can be met by writing additional code to a class, or by using special hardware and probes. /estability is a concern in both hardware and software designs. In hardware design, testability focuses on erifying the correctness $no faults% of a finished product. /estability in software focuses on the erification of design and implementation.




Dept., of ISE

Module -

E$$!$ A%D "A+T TA:!%!MIES' Basi( definitions of ro(ess and produ(t' 3rocess refers to how we do something, and product is end result of process.

"oftware quality :ssurance ";: tries to improe the product by improing the 3rocess, and is concerned with reducing errors endemic in the deelopment process. /esting is more concerned with discoering faults in a product.

Faults can be classified in seeral ways& deelopment phase of fault occurance, consequences of corresponding failure, difficulty to resole, risk etc. "ault (lassifi(ation )* seerit*'

"arious error and fault ta#onomies based upon a$%nput & Output Faults b$ 'ogic Faults c$ Computation Faults d$ %nterface Faults e$ (ata Faults Input'  

 

)orrect input not accepted Incorrect input !ccepted 'escription wrong or missing 3arameters wrong or missing

!utput' Wrong Format 



Software Testing-15IS63 I     

Dept., of ISE

Module -

Wrong 1esult )orrect result at wrong time Incomplete or missing result -purious result -pelling > (rammar

+ogi(al faults' issing )ase $s% 'uplicate )ase $s% E"treme )ondition Neglected isinterpretation issing )ondition E"traneous )ondition $s% /est of wrong Pariable Incorrect loop iteration Wrong operator $e.g., G instead of GS%         

oputation "aults' Incorrect !lgorithms issing )omputations Incorrect Operand Incorrect Operation 3arenthesis error Insufficient 3recision $ 1oundoff , /runcation% Wrong 2uiltin function       

Interfa(e "aults' Incorrect interrupt handling I>O /iming )all to wrong 3rocedure )all to None"istent 3rocedure 3arameter mismatch $type, number% Incompatible types -uperfluous types       

Data "aults' Incorrect initiali+ation Incorrect storage > access Wrong flag > inde" alue Incorrect packing > unpacking Wrong ariable used Wrong data reference -caling or unit errors       



Software Testing-15IS63 I    

Dept., of ISE

Module -

Incorrect data dimension Incorrect -ubscript , Incorrect type Incorrect data scope -ensor data out of limits

+E4E+S !" TESTI%&'-

Figure below shows the leel of abstraction and testing in the waterfall model.

Te leels are as follows' ; $e2uireent spe(ifi(ation' /his step inoles taking down requirements from the customer and making a complete sense of the specified requirements. It corresponds to -ystem /esting. ; reliinar* design' ! rough estimate $replica or sketch% of the product required by the customer is made in this phase taking into account the requirements specified in the preious leel. It corresponds to Integration /esting ; Detailed design' ! complete detailed design of the product is made in this phase taking into account all the requirements specified in the first phase and the sketch made in the second phase. . It corresponds to :nit /esting. ; oding' /he detailed design deeloped in the preious phase $i.e. detail design% is ne"t coded and a product is deeloped $in segments or subunits%. ; nit Testing' /he deeloped subunits or segments are then tested for their specific functionality. -tructural testing is more appropriate at unit leel testing.




Dept., of ISE

Module -

; Integration Testing' /he deeloped subunits in the coding phase are then integrated into a specific module which is again tested to check the interactions between the subunits that are integrated. Functional testing is more appropriate at Integration testing. ; S*ste Testing' /he integrated modules are then finally assembled into a finished product which is tested again to check whether the product performs to the required standards as mentioned by the customer. Functional testing is more appropriate at system leel testing.

TESTI%& A%D 4E$I"IATI!% ' 













/esting aims at uncoering errors in a program. 3rogram erification aims at proing the correctness of programs by showing that it contains no errors. !lso erification aims at showing that a gien program works for all possible inputs that satisfy a set of conditions, 3rogram erification and testing are best considered as complementary techniques. In the deelopment of critical applications, such as smart cards, or control of nuclear plants, one often makes use of erification techniques to proe the correctness of some artifact created during the deelopment cycle, not necessarily the complete program. 1egardless of such proofs, testing is used inariably to obtain confidence in the correctness of the application. Perification might appear to be a perfect process as it promises to erify that a program is free from errors. *oweer, a close look at erification reeals that it has its own weaknesses. /he person wh o erified a program might hae made mistakes in the erification process; there might be an incorrect assumption on the input conditions; /hus, neither erification nor testing are perfect techniques for proing the correctness of programs.

STATI TESTI%&' 



-tatic testing is carried out without e"ecuting the application under test. In contrast, dynamic testing requires one or more e"ecutions of the application under test. -tatic testing is useful in that it may lead to the discoery of faults in the application, as well as ambiguities and errors in requirements and other application relation documents, at a relatiely low cost.




Dept., of ISE



/his is especially so when dynamic testing is e"pensie.



-tatic testing is complementary to dynamic testing.







Module -

Organi+ations often sacrifice static testing in faor of dynamic testing though this is not considered a good practice. -tatic testing is best carried out by an indiidual who did not write the code, or by a team of indiiduals. ! sample process of static testing is illustrated in Figure. /he test team responsible for static testing has access to requirements documents, application, and all associated documents such as design document and user manuals. /he team also has access to one or more static testing tools. ! static testing tool takes the application code as input and generates a ariety of data useful in the test process.

Figure : -lements of static testing.

1.11.1 Walkthroughs 

Walkthroughs and inspections are an integral part of static testing.



Walkthrough is an informal process to reiew any applicationrelated document. For e"ample, requirements are reiewed using a process termed requirements walkthrough.



)ode walkthrough, also known as peer code reiew, is used to reiew code and may be considered as a static testing technique



! walkthrough begins with a reiew plan agreed upon by all members of the team. Each item of the document, for e"ample, a source code module, is reiewed with clearly stated ob#ecties in iew. ! detailed report is generated that lists items of concern regarding the document reied.




Dept., of ISE

Module -

In requirements walkthrough, the test team must reiew the requirements document to ensure that the requirements match user needs, and are free from ambiguities and inconsistencies.

1.11.2 Inspecons  



 

Inspection is a more formally defined process than a walkthrough. -eeral organi+ations consider formal code inspections as a tool to improe code quality at a lower cost than incurred when dynamic testing is used. Organi+ations hae reported significant increases in productiity and software quality due to the use of code inspections.

)ode inspection is a rigorous process for assessing the quality of code. )ode inspection is carried out by a team which works according to an inspection plan that consists of the following elements& $a% statement of purpose, $b% work product to be inspected, this includes code and associated documents needed for inspection, $c% team formation, roles, and tasks to be performed, $d% rate at which the inspection task is to be completed, and $e% data collection forms where the team will record its findings such as defects discoered, coding standard iolations, and time spent in each task.



embers of the inspection team are assigned roles of moderator, reader, recorder, and author.



/he moderator is in charge of the process and leads the reiew.



!ctual code is read by the reader, perhaps with the help of a code browser and with monitors for all in the team to iew the code.



/he recorder records any errors discoered or issues to be looked into.



/he author is the actual deeloper whose primary task is to help others to understand the code.

1.11.3 Sofware complexity and stac tesng  



Often a team must decide which of the seeral modules should be inspected first. -eeral parameters enter this decisionmaking processVone of these being module comple"ity. ! more comple" module is likely to hae more errors and must be accorded higher priority for inspection than a module with lower comple"ity.




Dept., of ISE

Module -

-tatic analysis tools often compute comple"ity metrics using one or more comple"ity metrics, could be used as a parameter in deciding which modules to inspect first. )ertainly, the criticality of the function a module seres in an application could oerride the comple"ity metric while prioriti+ing modules.

$!B+EM STATEME%TS'

&E%E$A+I
3seudo code proides a natural way to e"press program source code.




Dept., of ISE

Module -

Te Triangle ro)le'

 /he triangle program accepts A integers& a,b Xc as sides of a triangle  It outputs the type of the triangle from the length of the A sides @ Equilateral $all sides are equal% @ Isosceles $= sides are equal% @ -calene $all sides are not equal% @ Not a triangle $if the sum of any = sides GS the third%  /he triangle property& @ /he sum of any pair of sides must be strictly greater than the third side ..$J% not $JS% aGbTc bGaTc c G a T b  a, b,X c are integers from < to =LL "low art for traditional triangle progra ipleentation is as follows'




Dept., of ISE

Module -

&enerali=ed seudo (ode'

/1I!N(0E31O20E& 1E!' P!0:E- !, 2, !N' ) -O1/ P!0:E- !, 2, !N' ) $!-)EN'IN(% /O (IPE !!, 22, !N' )) IF !! T 22 J)) & IF !! S 22 S )) & 31IN/ YEQ:I0!/E1!0 /1I!N(0EY E0-E & IF !! S 22 && 31IN/ YI-O-)E0E- /1I!N(0EY & E0-E && 31IN/ Y-)!0ENE /1I!N(0EY & EN'IF & EN'IF E0-E 31IN/ Y/1I!N(0E I3O--I20EY EN'IF

Stru(tured Ipleentation'

Figure shows the dataflow diagram description of triangle program. We could implement it as a main program with the three indicated procedures.




Dept., of ISE

Module -

%ote' "or stru(tured progra' $efer +a) progra (ode.

%et date "un(tion' ro)le stateent' Ne"t'ate is a function of three ariables& month, day and year. It returns the date of the day after the input date. /he month, day and year ariables hae integer alues sub#ect to these conditions&

)<.
point of iew, only ersion < is sufficient. Interested readers can study ersion = also.%





Dept., of ISE

Module -




Dept., of ISE

Module -




Dept., of ISE

Module -



Dept., of ISE

Module -

oission pro)le' ro)le Stateent' ! rifle salesperson in the former !ri+ona /erritory sold rifle locks, stocks, and barrels made by a gunsmith in issouri. 0ocks cost ZM, stocks cost ZAL, and barrels cost Z=. /he salesperson had to sell at least one complete rifle per month, and production limits were such that the most the salesperson could sell in a month was BL locks, DL stocks, and L barrels. !fter each town isit, the salesperson sent a telegram to the issouri gunsmith with the number of locks, stocks, and barrels sold in that town. !t the end of a month, the salesperson sent a ery short telegram showing @< lock sold. /he gunsmith then knew the sales for the month were complete and computed the salesperson9s commission as follows&
/he commission program produced a monthly sales report on&  /he total number of locks,  /he total number of stocks,  /he total number of barrels sold,  /he salesperson9s total dollar sales, and,  Finally, the commission for the total sales.

E"ample& 0et us consider& /otal number of locks soldS
/otal number of stocks soldS
/otal

We can calculate the sales using the formula& -alesS $number of locksKMTnumber of stocksKALTnumber of barrelsK=% /herefore in our e"ample sales would be
3rogram commission $IN3:/, O:/3:/% 'im lock3rice, stock3rice, barrel3rice !s 1eal Krupa K S, Asst. Professor



Dept., of ISE

Module -

'im locks, stocks, barrels !s Integer 'im total0ocks, total-tocks !s Integer 'im total2arrels !s Integer 'im lock-ales, stock-ales !s 1eal 'im barrel-ales !s 1eal 'im sales, commission !s 1eal lock3rice S M.L, stock3rice S AL.L, barrel3rice S =.L total0ocks S L, total-tocks S L, total2arrels S L Input $locks% While NO/ $locks S <% Input deice uses < to indicate end of data Input $stocks, barrels% total0ocks S total0ocks T locks total-tocks S total-tocks T stocks total2arrels S total2arrels T barrels Input $locks% EndWhile Output $Y0ocks sold& Y, total0ocks% Output $Y-tocks sold& Y, total-tocks% Output $Y2arrels sold& Y, total2arrels% -ales S lock3riceKtotal0ocks T -tock3riceKtotal-tocks T barrel3rice K total2arrels Output $Y/otal sales& Y, sales% If $sales J
/he below figure shows -!/ terminal




Dept., of ISE

Module -

-!/ system is actually used for communicating with bank customers. -!/ customers can select any of three transaction types& 

deposits



withdrawals



saings

When a bank customer arries at an -!/ station, screen < is displayed. /he bank customer accesses the -!/ system with a plastic card encoded with a 3!N, which is a key to an internal customer account file containing ,among other things ,the customer name and account information. If customers 3!N matches, the screen = will be displayed to the customer else screen M will be displayed. -!/ communicate with bank customers ia < screens as shown in below figure&




Dept., of ISE

Module -

/hus -!/ is #ust the day to day !/ serices which we use in daily life. !nd if there is any error then cancel the transaction and e"it from the screen. !t screen =, customer is asked to enter the 3IN. If 3IN is correct then screen  will be displayed, else screen A will be displayed and user is gien A chances before the card is blocked. On screen , the information proided is customers account, current date, and increment to the number of !/ session. )ustomer selects the transaction to be processed. If balance is requested then screen
Te urren(* onerter

!n eentdrien program with a graphical user interface $(:I%, shown below



/he currency conersion program is another eent drien program that emphasi+es code associated with (:I. /he application conerts :.- dollars to any of four currencies& 2ra+ilian reals, )anadian dollars, European euros and \apanese yen.



)urrency selection is goerned by the radio buttons which are mutually e"clusie. When a country is selected, the system responds by completing the label. If )anadian button is clicked, a




Dept., of ISE

Module -

small )anadian flag appears ne"t to output position and the conerted currency amount is displayed. Either before or after currency selection, the user inputs an amount in :- dollars. Once tasks are accomplished, the user can click on the computer button, the clear button, or the quit button. )licking the computer button results in conersion of the :- dollar amount to the equialent amount in the selected currency. )licking on the clear button resets the currency selection, the :- dollar amount, and the equialent currency amount and the associated label. )licking on the quit button ends the application. Saturn #indsield #iper ontroller

 /he windshield wiper on some -aturn cars is controlled by a leer with a dial  /he leer has M positions& OFF, IN/$intermittent%, 0OW and *I(*  /he dial has three positions& <,=,A @ /he dial positions indicate three intermittent speeds @ It is releant only when the leer is at the IN/ position  /he decision table below shows the windshield wiper speeds in wipes per minute for the leer and dial positions.

/uestion )an7'

<. 'efine the following & a% Error b% Fault c% Failure d% Incident e% /est f% /est case $C% \uly =L<, \an =L<, \uly =L
$C%

\an=L<, \uly =L
A.What is software testing Why it is so important in -'0)

$%\an =L
M. E"plain the triangle problem statement along with flow chart for traditional implementation. $B% \uly =L<, \an =L

$% \uly =L<, \uly =L

ST Module 1 Notes

Recommend Documents