Basics
Testing can give confidence in the system if it finds no bugs.
- Finding Defects
- Gaining confidence about level of quality
- Providing information for decision-making
- Preventing defects
7 Testing Principles
(but can't prove there are no bugs).
2. Exhaustive Testing is Impossible
except in trivial cases. Use the level of risk to determine how much testing to do.
3. Early Testing
Start testing as early as possible (as soon as entry criteria are met).
4. Defect Clustering: Over time the areas where the new tests are created should be proportional to where bugs are being found.
5. Pesticide Paradox: Get's it name from the fact that if you spray pesticide on plants over time it stops working and doesn't detract bugs. In software, they are saying if you write code that has no tests associated with it then your current testing suite isn't covering that, and so in that area you're not getting the benefits of testing because you're not really doing it.
6. Testing is Context Dependent
safety-critical software is tested differently from e-commerce site
7. Absence of Errors Fallacy
Key concept is that the software needs to provide real value to users. Just because there are no errors / bugs doesn't necessarily mean it will be successful.
the Test Process
- Test planning and control
- Test analysis and design
- Test implementation and execution
- Evaluating exit criteria and reporting
- Test closure activities
Evaluating Exit Criteria
Assessing if more tests are needed or if the exit criteria should be changed,
Write a test summary report
The V-Model
- Component (unit) testing
- Integration Testing
- System Testing
- Acceptance Testing
Testing within a lifecycle
- For every development activity, a corresponding testing activity
- Each test level has test objectives specific to that level
- analysis and design of tests for a given test level should begin during corresponding development activity
- testsers should be involved in reviewing documents as soon as drafts are available in the development life cycle
Component Testing
Integration Testing
- Component integration testing- testing interactions between components within the client-side application
- System integration testing- testing the connections between your app and an external backend, db, etc.
System Testing
- Requirements
- use cases
- functional specifications
- risk analysis
- System, user, and operation manuals
- system configuration and configuration data
Acceptance Tests
- user requirements
- system requirements
- use cases
- business processes
- risk analysis
- business processes on fully integrated system
- operational and maintainance processes
- user procesdures
- forms
- reports
- configuration data
Black Box Testing
Functional Testing
Nonfunctional Testing
Regression Testing and Re-Testing
Maintainance Testing
Activities is a Formal Test Review
- Planning
- Kick-off
- Individual Preparation
- Examination/evaluation/recording results
- Rework
- Folow-up
Types of Reviews
- Informal Review
- Walkthrough
- Technical Review
- Inspection
Static Analysis
Black-Box Techniques
- Equivalence Partioning
- Boundary Value Analysis
- Decision Table Testing
White-Box Testing Techniques
- Statement Testing and Coverage
- Decision Testing and Coverage
- Other Structure-Based Techniques
Experience-Based Testing
Test Leader
Tasks are:
- Coordinate test strategy
- review test strategy
- plan
- allocate resources
- decide what needs to be done
- decide on tools to use
- review, contrivute to plan
- specify tests
- execute tests
- bring the test manager coffee (jk)
Entry Criteria
- Test environment availability and readiness
- Test tool readiness in the test environment
- Testable code availability
- Test data availability
Exit Criteria
- Thorough measures, coverage tools
- Estimates of defect density or reliability measures
- Residual risk, defects that have not been fixed
- Schedule based on time to market
Project Risks
- Organizational factors:
- Skill, training, staff shortages
- Personal issues
- Political Issues
- Improter attitude towards expectations of testing
- Technical issues:
- Problems in defining the right requirementds
- extent to which requirements cannot be met given existing constraints
- test environment not ready on time
- late data conversion, migration planning / data not available
- low quality of code
- Supplier Issues:
- Failure of third party
- Contractual issues
Product Risks
- Failure-Prone Software
- potential that the software/hardware could cause harm
- poor software characteristcs (functionality, reliability, usability, performance)
- poor data integrity and quality
- software deoes not perform intended function
Incident Management
- provide developers and other parties with feedback about problem
- means for tracking the bugs
- ideas for test process improvement
- date of issue, author
- expected and actual resulrs
- impact
Introducing a New Tool
- Don't do it all at once; roll it out incrementally
- Provide coaching and mentoring
- Monitoring usage
- Gather lessons learned