SwTOi - Software Testing Ontology Integrated A test design specification refines the test approach and identifies the features to be covered by the design and its associated tests. It also identifies the test cases and test procedures, if any, required to accomplish the testing and specifies the feature pass/fail criteria. test design specification Software testing is usually performed at different levels along the development and maintenace process. niveis de teste O teste de software e geralmente realizado em diferentes niveis ao longo do processo de desenvolvimento. test levels A program under test can be assessed by counting and classifying the discovered faults by their types. For each fault class, fault density is meadured as the ratio between the number of faults found and the size of the program. In cases where software is built to serve different users, configuration testing analyzes the software under the various specified configurations. configuration testing test selection criteria A test selection criterion is a means of deciding what a suitable set of test cases should be. A selection criterion can be used for selecting the test cases of for checking whether a selected test suite is adequate, that is, to decide whether the testing can be stopped. 1 Agent's veridict Agent's veridict test team Formalization of the test process may involve formalizing the test team organization as well. The test team cam be composed of internal members, of external members, in the hope of bringing in an unbiased, independent perspective, or, finally, of both internal and external members. Considerations of costs, schedule, maturity levels of the involved organizations, and criticality of the application may determine the decision. mutation testing A mutant is a slightly modified version of the program under test, differing form it by a small, syntactic change. 1 test case generation Generation of test cases is based on the level of testing to be performed and the particular testing techniques. Test cases should be under the control of software configuration management and include the expected results for each test. Um dos principais objetivos do teste e identificar o maior numero de falhas possiveis no software e para isso, varias tecnicas sao utilizadas. As tecnicas de teste sao utilizadas como mecanismos para que os objetivos de teste sejam concretizados. One of the aims of testing is to reveal as much potential for failure as possible, and many techniques have been developed to do this. The leading principle underlying such techniques is to be as systematic as possible in identifying a representative set of program behaviors; for instance, considering subclasses of the input domain, scenarios, states and dataflow. Sometimes, test techniques are confused with test objectives. Test techniques are to be viewed as aids which help to ensure the achievement of test objectives. test techniques tecnicas de teste Recovey testing is aimed at verifying software restart capabilities after a "desaster". recovery testing 1 A test case specification documents the actual values used for input along with the anticipated out-puts. A test case also identifies constraints on the test procedures resulting from use of that specific test case. Test cases are separated from test designs to allow for use in more than one design and to allow for reuse in other situations. test case test procedure specification A test procedure specification identifies all steps required to operate the system and exercise the specified test cases in order to implement the associated test design. Test procedures are separated from test design specifications as they are intended to be followed step by step and should not have extraneous detail. ferramentas de teste test tool O processo de teste pode contar com o apoio de varias ferramentas. Geralmente essas ferramentas contribuem para minimizar tempo, esforco e custo associado. unit test Unit testing verifies the functioning isolation of software pieces which are separately testable. Depending on the context, these could be the individual subprograms or a larger component made of tightly related unit. Evaluation of the program under test. Measures based on program size (for example, source line of code or function points) or on program structure (like complexity) are used to guide testing measurement. documentacao de teste test documentation Documentation is an integral part of the formalization of the test process. Test documentats may include, among others, Test Plan, Test Design Specifiation, Test Procedures Specification, Test Case Specification, Test Log, and Test Incident or Problem Report. The software under test is documented as the Test Item. Test documentation should be produced and continually updated, to the same level of quality as other types of documentation in software engineering. A documentacao e uma parte integral do processo de formalizacao do teste. Os documentos de teste incluem, entre outros, plano de teste, especificacao do projeto de teste, especificacao do procedimentos de teste, especificacao dos casos de teste, logs de teste e relatorio de problemas encontrados. O software que passa por um teste e documentado como um item de teste. A documentacao deve ser produzida e continuamente mantida com o mesmo nível de qualidade proposto pela engenharia de software para outros tipos de documentacoes. test results evaluation The results of testing must be evaluated to determine whether or not the test has been successful. In most cases, "successful" means that software performed as expected and did not have any major unexpected outcomes. Not all unexpected outcomes are necessarily faults however, but could be judged to be simply noise. Before a failure can be removed, an analysis and debugging effort is needed to isolate, identify, and describe it. When test results are particularly important, a formal review board may be convened to evaluate them. In testing for reliability evaluation, the test environment must reproduce the operational environment of the software as closely as prossible. The idea is to infer, from the observed test results, the future reliability of the software when in actual use. To do this, inputs are assigned a probability distribution, or profile, according to their occurrence in actual operation. Some faults are artificially introduced into the program before test. When the tests are executed, some of these seeded faults will be revealed, and possibly some faults which were already there will be as well. In theory, depending on which of the artificial faults are discovered, and how many, testing effectiveness can be evaluated, and the remaining number of genuine faults can be estimated. A fase de teste pode ser direcionada por varios propositos, por exemplo, no teste baseado em risco, onde o risco do produto ajuda a priorizar e definir a estrategia do teste. Um outro exemplo e o teste baseado em cenarios, onde os casos de teste sao definicos com base em cenarios especificos do software The testing phases could be guided by various aims, for example: in risk-based testing, which uses the product risks to prioritize and focus the test strategy; or in scenario-based testing, in which test cases are defined based on specified software scenarios. test guides obtained value Conceitos associados ao dominio de teste de software Software Test Domain Concept software test domain concept error Discrepancy between a computed, observed or measured value or condition and the true, specified or theoretically correct value or condition. Test cases are chosen on and near the boundaries of the input domain of variables, with the underlying rationale that many faults tend to concentrate near the extreme values of inputs. Failures observed during testing are most often due to faults or defects in the software. Such defects can be analyzed to determine when they were introduced into the software, what kind of error caused them to be created (poorly defined requirements, incorrect variable declaration, memory leak, programming sysntax error, for example), and when they could have been first observed in the software. Defect-tracking information is used to determine what aspects of software engineering need improvemented and how effective previous analyses and testing have been. defect tracking test related measures Prove uma avaliacao do programa baseada nos resultados do teste. A avaliacao pode ser usada para otimizar o planejamento e a excucao dos testes metodo de avaliacao do teste Provide an evaluation of the program under test based on the observed test outputs, and those which evaluate the thoroughness of the test set. Measurement may also be used to optimize the planning and execution of the tests. Acceptance testing checks the system behavior agains the customer's requirements, however these may have been expressed; the customers undertake, or specify, typical tasks to check that their requirements have been met or that the organization has identified these for the target market for the software. This testing activity may or may not involve the developers of the system. acceptance/qualification testing The input domain is subdivided into a collection of subsets, or equivalent classes, which are deemed equivalent according to a specified relation, and a representative set of tests (some times only one) is taken from each class. system testing System testing is concerned with the behavior of a whole system. The majority of functional failures should already have been identified during unit and integration testing. System testing is usually considered appropriate for comparing the system to the non-functional system requirements, such as security, speed, accuracy, and reliability. External interfaces to other applications, utilities, hardware devices or the operating environment are also evaluated at this level. software testing process processo de teste de software O teste de software consiste em uma verificacao dinamica do programa, levando em consideracao um conjunto finito de casos de teste extraido geralmente de um dominio de execucao infinito. Conceitos de teste, estrategias, tecnicas e mensuracoes precisam ser integradas em um processo definido e integrado que possa ser executado por pessoas. O processo de teste comporta uma série de atividades e oferece diretrizes de como essas atividades podem ser executadas pela equipe de teste. Software testing consists of the dynamic verification of the behavior of a program on a finite set of test cases, suitably selected from the usually infinite executions domain, against the expected behavior. Testing concepts, strategies, techniques, and measures need to be integrated into a defined and controlled process which is run by people. The test process supports testing activities and provides guidance to testing teams, from test planning to test output evaluation, in such a way as to provide justified assurance that the test objectives will be met cost-effectively. By modeling a program as a finite state machine, tests can be selected in order to cover states and transitions on it. Integration testing is the process of verifying the interaction between software componets. Modern systematic integration strategies are rather architecture-driven, which implies integrating the software components or subsystems based on identified functional threads. integration testing The environment used for testing should be compatible with the software engineering tools. It should facilitate development and control of test cases, as well as logging and recovery of expected results, scripts, and other testing materials. test environment development fault based techniques Fault-based testing techniques devise test cases specifically aimed at revealing categories of likely or predefined faults. alpha testing Before the software is released, it is sometimes given to a small, representative set of potential users for trial use, either in-house (alpha testing) TODO: Tem que ter uma Release It covers the basic definitions in the field of software testing the basic terminology and key issues, and its relationship with other activities. fundamentos de teste test fundamentals Representa a terminologia basica de teste de software software test Testing is conducted in view of a specific objective, which is stated more or less explicitly, and with varying degrees of precision. Testing objectives considers testing for specific conditions or properties. Testing objective varies with the test target; in general, different purposes being addressed at a different level of testing. Not all types of testing apply to every software product, not has every possible type has been listed. testing objectives failure/defect The inability of a system to perform its required functions within specified performance requirements. defeito internal team The test team can be composed of internal members that is, on the project team, involved or not in software construction. Decision tables represent logical relationships between conditions (roughly, inputs) and actions(roughly, outputs). Test cases are systematically derived by considering every possible combination of conditions and actions. stress testing Stress testing exercises software at the maximum design load, as well as beyond it. test log Testing activities can be entered into a test log to identify when a test was conducted, who performed the test, what software configuration was the basis fro testing, and other relevant identification information. Unexpected or incorrect test results can be recorded in a problem-reporting system, the data of which form the basis for later debugging and for fixing the problems that were observed as failures during testing. Also, anomalies not classified as faults could be documented in cas they later turn out to be more serious than first thought. Test reports are also an input to the change management request process strategy alvo de teste target The strategy or target of the test is a single module, a group of such modules (related by purpose, use, beharior, or structure), or a whole system. Three big test stages can be conceptually distinguished, namely Unit, Integration and System. In ad hoc, the tests are derived relying on the software engineer's skill, intuition, and experience with similar programs. Ad hoc testing might be useful for identifying special tests, those not easily captured by formalized techniques. Regression testing is the "selective retesting of a system or component to verify that modifications have not caused unintended effects". In practice, the idea is to show that software which previously passed the tests still does. regression testing An agent is an autonomous entity, that takes alone decisions without the interference of a system or another one entity. An agent should be capable of interact with the environment and with others agents, choosing trusted and cooperate or not with them. agent To evaluate the thoroughness of the executed tests, testers can monitor the elements covered, so that they can dynamically measure the ratio between covered elements and their total number. For example, it is possible to measure the percentage of covered branches in the program flowgraph, or that of the functional requirements exercised among those listed in the specifications document. code based techniques Techniques based on the software engineer's intuition and experience. intuition and experience This is specifically aimed at verifying that the software meets the specified performance requirements, for instance, capacity and response time. performance testing expected value value Execution of tests should embody a basic principle of scientific experimentation: everything done during testing should be performed and documented clearly enough that another person could replicate the results. Hence, testing should be performed in accordance with documented procedures using a clearly defined version of the software under test. execution Correct value or condition correct A statistic estimate of software reliability, which can be obtained by reliability achievement and evaluation, can be used to evaluate a product and decide whether or not testing can be stopped. external team In the hope of bringing in an unbiased, independent perspective. incorrect Many terms are used in the software engineering literature to describe a malfunction. It is essential to clearly distinguish between the cause of a malfunction. TODO: Definir melhor as subclasses In error guessing, test cases are specifically designed by software engineers trying to figure out the most plausible faults in a given program. A good source of information is the history of faults discovered in earlier projects, as well as the software engineer's expertise. error guessing usability testing This process evaluates how easy it is to end-users to use and learn the software, including user documentation; how effectively the software functions in supporting user tasks; and, finally, its ability to recover form user errors. Exploratory testing is defined as simultaneous learning, test design, and test execution; that is, the tests are not deined in advance in an established test plan, but are dynamically designed, executed and modified. The effectiveness of exploratory testing relies on the software engineer's knowledge, which can be derived from various sources: observed product behavior during testing, familiarity with the application, the platform, the failure process, the type of possible faults and failures. planning Like any other aspect of project management, testing activities must be planned. Key aspects of test planning include coordination of personnel, management of available test facilities and equipment (which may include magnetic media, test plans and procedures), and planning for possible undesirable outcomes. Planning for testing should start with the early stages of the requirement process, and test plans and procedures must be systematically and continuously developed, and possibly refined, as development proceeds. test patterns A proposta dos padroes de teste e a organizacao e o reuso sistematico em projetos similares. Geralmente os artefatos de teste são controlados por um gerenciador de configuracao. A ideia e que as modificacoes feitas nos requisitos ou no projeto do software possam refletir mais facilmente no escopo dos testes. padroes de teste To carry out testing or maintenance in an organized and cost-effective way, the means used to test each part of the software should be reused systematically. This repository of test materials must be under the control of software configuration management, so that changes to software requirements or design can be reflected in changes to the scope of the tests conducted. The test solution adopted for testing some application types under certain circumstances, with the motivations behind the decisions taken, form a test pattern which can itself be documented for later reuse in similar projects. installation testing Usually after completion of software and acceptance testing, the software can be verified upon installation in the target environment. Inatallation testing can be viewed as system testing conducted once again according to hardware configuration requirements. Intallation produces may also be verified. glass box falha fault Anormal condition that may cause a reduction in, or loss of, the capability of a functional unit to perform a required function. As falhas podem causar defeitos. O testes podem apontar defeitos mas são as falhas que efetivamente devem ser removidas. Conformance testing/Functional testing/Correctness testing is aimed at validating whether or not the observed beharvior of the tested software conforms to its specifications. conformance/functional/correctness testing beta testing TODO: Tem que ter uma release Before the software is released, it is sometimes given to a small, representative set of potential users for trial use, either external team (beta testing). In data-flow-based testing the control flowgraph is annotated with information about how the program variables are defined, used, and killed (undefined). data flow based criteria Is a testing method encompassing the whole development process, whereby testing is designed and guided by reliability objectives and expected relative usage and criticality of different functions in the field. Tests are generated purely at random. This form of testing falls under the heading of the specification-base entry, since at least the input domain must be known, to be able to pick random points within it. black box The test plan prescribes the scope, approach, resources, and schedule of the testing activities. It identifies the items to be tested, the features to be tested, the testing tasks to be performed, the personnel responsible for each task, and the risks associated with the plan. test plan test activity atividades de teste Control-flow-based coverage criteria is aimed at covering all the statements or blocks of statements in a program, or specified combinations of them. control flow based criteria Reliability growth models provide a prediction of reliability based on the failures observed under reliability achievement and evaluation. They assume, in general, that the faults that caused the observed failures have been fixed (although some models also accept imperfect fixes), and thus, on average, the product's reliability exhibits an increasing trend. The test objective varies with the test target; in general, different purposes being addressed at a different level of testing. The many proposed test techniques differ essentially in how they select the test set, and software engineers must be aware that different selection criteria may yield vastly the most suitable selection creterion under given conditions is a very complex problem; in practice, risk analysis techniques and test engineerint expertise are applied. isVeredictOf List of variables and their possible values used in the test case. The test case should be performed once for each combination of values. These values are written in set notation, one per line. E.g.: loginID = {Valid loginID, invalid loginID, valid email, invalid email, empty} password = {valid, invalid, empty} Assumptions that must be met before the test case can be run. Short sentence or two about the aspect of the system is being tested. If this gets too long, break the test case up or put more information into the feature descriptions. Steps to carry out the test. 1. visit LoginPage 2. enter userID 3. enter password 4. click login 5. see the terms of use page 6. click agree radio button at page bottom 7. click submit button 8. see PersonalPage 9. verify that welcome message is correct username Scope refers to the extent to which a requirement affects the software and software components. Some requirements, particularly certain nonfunctional ones, have a global scope in that their satisfaction cannot be allocated to a discrete component. Software Requirement Version Software requirement risk All kind of software requirement costs, like a material, people, etc. mandatory highly desirable desirable optional In general, the higher the priority, the more essential the requirement is for meeting the overall goals of the software. Oftem classified on a fixed-poit scale such as mandatory, highly desirable, desirable or optional, the priority often has to be balanced against the cost of development and implementation.