Even though I consider myself more of a developer than a tester, I've realized that the best creators of perfect software of those who can build a balanced suite of automated tests with some code implemented to make them work. Although the ISTQB (International Software Testing Qualifications Board) exam has to do with all testing including just manual testing. Also, as the inventor of Triplex Testing Theory I wanted to make sure I wasn't missing anything. They have some weird vocabulary, but overall studying for this this exam can give you a great awareness of all areas of software testing without being a programming language specific tutorial on it. I don't have much time left before I go to the testing center, but my last method of studying is copying my notes here in a blog post. If you're taking the exam soon, I hope this helps!
Basics
Testing is meant to detect bugs, not prove that there are no bugs.
Testing can give confidence in the system if it finds no bugs.
Testing can give confidence in the system if it finds no bugs.
Testing has the folowing objectives:
- Finding Defects
- Gaining confidence about level of quality
- Providing information for decision-making
- Preventing defects
7 Testing Principles
1. Testings Shows the Presence of Defects
(but can't prove there are no bugs).
2. Exhaustive Testing is Impossible
except in trivial cases. Use the level of risk to determine how much testing to do.
3. Early Testing
Start testing as early as possible (as soon as entry criteria are met).
4. Defect Clustering: Over time the areas where the new tests are created should be proportional to where bugs are being found.
5. Pesticide Paradox: Get's it name from the fact that if you spray pesticide on plants over time it stops working and doesn't detract bugs. In software, they are saying if you write code that has no tests associated with it then your current testing suite isn't covering that, and so in that area you're not getting the benefits of testing because you're not really doing it.
6. Testing is Context Dependent
safety-critical software is tested differently from e-commerce site
7. Absence of Errors Fallacy
Key concept is that the software needs to provide real value to users. Just because there are no errors / bugs doesn't necessarily mean it will be successful.
(but can't prove there are no bugs).
2. Exhaustive Testing is Impossible
except in trivial cases. Use the level of risk to determine how much testing to do.
3. Early Testing
Start testing as early as possible (as soon as entry criteria are met).
4. Defect Clustering: Over time the areas where the new tests are created should be proportional to where bugs are being found.
5. Pesticide Paradox: Get's it name from the fact that if you spray pesticide on plants over time it stops working and doesn't detract bugs. In software, they are saying if you write code that has no tests associated with it then your current testing suite isn't covering that, and so in that area you're not getting the benefits of testing because you're not really doing it.
6. Testing is Context Dependent
safety-critical software is tested differently from e-commerce site
7. Absence of Errors Fallacy
Key concept is that the software needs to provide real value to users. Just because there are no errors / bugs doesn't necessarily mean it will be successful.
the Test Process
- Test planning and control
- Test analysis and design
- Test implementation and execution
- Evaluating exit criteria and reporting
- Test closure activities
Evaluating Exit Criteria
Checking test logs against exit criteria
Assessing if more tests are needed or if the exit criteria should be changed,
Write a test summary report
Assessing if more tests are needed or if the exit criteria should be changed,
Write a test summary report
The V-Model
- Component (unit) testing
- Integration Testing
- System Testing
- Acceptance Testing
Testing within a lifecycle
- For every development activity, a corresponding testing activity
- Each test level has test objectives specific to that level
- analysis and design of tests for a given test level should begin during corresponding development activity
- testsers should be involved in reviewing documents as soon as drafts are available in the development life cycle
Component Testing
This is what we think of as unit testing aka white-box testing aka structural testing. Think Jasmine running on karma, testing individual methods of the code in isolation.
Integration Testing
Two types:
- Component integration testing- testing interactions between components within the client-side application
- System integration testing- testing the connections between your app and an external backend, db, etc.
System Testing
Tests created from:
- Requirements
- use cases
- functional specifications
- risk analysis
Typically, these tests check:
- System, user, and operation manuals
- system configuration and configuration data
Acceptance Tests
Goal for these is to establish confidence in the system.
Tests Created from:
- user requirements
- system requirements
- use cases
- business processes
- risk analysis
Typically, these check:
- business processes on fully integrated system
- operational and maintainance processes
- user procesdures
- forms
- reports
- configuration data
Black Box Testing
Testing that doesn't look at the internals of the code- think Protractor / Selenium testing.
Functional Testing
Describes the "what" of the system. These tests describe and check that the system or component is doing the function that it was meant to perform. To help remember this, next time you stub you toe say, "ah what the functional testing" to help connect the words functions and what.
Nonfunctional Testing
Describes the "how" of the system. These testing include stress testing, load testing, usability testing, maintainability testing, reliability testing, portability testing
Regression Testing and Re-Testing
Regression testing is the act of running tests over and over again even though they are passing, just to make sure they are still passing (and be immediately alerted if they aren't!). Re-testing is the act of testing against after the developer fixes a bug to make sure it has been fixed.
Maintainance Testing
Not building something new, adding features, or fixing bugs; just maintaining. Think running tests after updating npm dependencies.
Activities is a Formal Test Review
- Planning
- Kick-off
- Individual Preparation
- Examination/evaluation/recording results
- Rework
- Folow-up
Types of Reviews
- Informal Review
- Walkthrough
- Technical Review
- Inspection
Static Analysis
Can refer to just manual review of code with another person, tools like IDE hinting and a JavaScript linter, or even Minko Gechev's Angular 2 static analysis tool.
Black-Box Techniques
- Equivalence Partioning
- Boundary Value Analysis
- Decision Table Testing
White-Box Testing Techniques
- Statement Testing and Coverage
- Decision Testing and Coverage
- Other Structure-Based Techniques
Experience-Based Testing
Refers to the fact that some testers have an intuition for what types of things frequently cause bugs and can test in ways that may find bugs that are otherwise difficult to uncover.
Test Leader
aka test coordinator or test manager.
Tasks are:
Tasks are:
- Coordinate test strategy
- review test strategy
- plan
- allocate resources
- decide what needs to be done
- decide on tools to use
tasks of test taker:
- review, contrivute to plan
- specify tests
- execute tests
- bring the test manager coffee (jk)
Entry Criteria
- Test environment availability and readiness
- Test tool readiness in the test environment
- Testable code availability
- Test data availability
Exit Criteria
- Thorough measures, coverage tools
- Estimates of defect density or reliability measures
- Residual risk, defects that have not been fixed
- Schedule based on time to market
Project Risks
- Organizational factors:
- Skill, training, staff shortages
- Personal issues
- Political Issues
- Improter attitude towards expectations of testing
- Technical issues:
- Problems in defining the right requirementds
- extent to which requirements cannot be met given existing constraints
- test environment not ready on time
- late data conversion, migration planning / data not available
- low quality of code
- Supplier Issues:
- Failure of third party
- Contractual issues
Product Risks
- Failure-Prone Software
- potential that the software/hardware could cause harm
- poor software characteristcs (functionality, reliability, usability, performance)
- poor data integrity and quality
- software deoes not perform intended function
Incident Management
Objectives:
- provide developers and other parties with feedback about problem
- means for tracking the bugs
- ideas for test process improvement
Report should include:
- date of issue, author
- expected and actual resulrs
- impact
Introducing a New Tool
- Don't do it all at once; roll it out incrementally
- Provide coaching and mentoring
- Monitoring usage
- Gather lessons learned
Don't allow teams to do their own things and implement whatever tools they want. Provide guidance and oversight.
Update: Welp, I took the exam, and I passed! I was a little annoyed though. I got thought I aced it, but I only got an 80%. There was a question about two-value boundary analysis that I had never heard of. Anyway, I've taking these concepts and hopefully won't forget them as I go on in the future to develop software and the Triplex Testing philosophy.
RSS Feed