Your Complete Guide to Software Testing- Part 1

Reading Time: 6 minutes

Introduction

Over the past 25 years, testing has gained prominence as a critical phase that determines the success of any project or program. Testing as a discipline/science is approximately a US$ 65–75 billion industry with an estimate of 9 to 15 % year-on-year growth (Ovum and other sources).

Given the stature of the testing industry, a crucial ingredient to make testing work is the people who deliver testing – testers. Like any major discipline, skills are of paramount importance in successful testing. Testing skills are crucial in aligning the test approach with the requirements of end users, making the outcome more relevant to the business context. Customer experience and stakeholder management are vital.

As the velocity of change in technology and the associated delivery methods have increased over the last 30 years, testing has kept pace. This combination of criticality (to the delivery and the business outcome) and the pace of change make testing itself the zenith of technology delivery.

As far as technical evolution and need for advanced testing skills needed for AI, Machine Learning, Big Data, Multi-Channel, Multi-Device Testing and DevOps drove innovation, I shall write another article. This article focuses more on the general aspect of testing skills.

Testing involves deep thinking, hard work, and meticulous planning. Testing can be quicker with smart work in the form of continuous improvements and clever planning. A tester’s reward is the identification of genuine, showstopper-critical defects (early in the life cycle) that results in changes to the software, application, program, project requirements, design, or code. Testing is an assimilation of features delivered by the requirements, design, and development phases, and determines the product quality. In the context of quality management, testing is the cornerstone of quality and provides motivation for quality improvements upstream in the lifecycle. Rephrasing a JRD Tata quote2, a tester must forever strive for excellence, or even perfection, in any test, however, small and never be satisfied with the second best result or performance.

This article touches on the key skills a tester should have to succeed in the era of the next generation of testing. Some of the key testing skills (this is not a comprehensive or exclusive list) are:

  • Capability areas – skills including testing knowledge, technical knowledge, analytical and logical reasoning (ability to map Requirements for testing and ability to validate and verify results), end-to-end business, functional and domain knowledge (this is for specialists/specialised phases of testing and is not mandatory for all testers).
  • Soft skills including clear communication (written and verbal), skills in stakeholder management, negotiation, diplomacy, team bonding with global multi-site multi-vendor teams, and empathy with the clients (i.e. both testing and communicating in a manner sympathetic to the end user requirements and demands). Soft skills are very important for a tester to be successful.
  • Other skills including observation, transparency, focused drive towards success, curiosity, exploration, inquisitiveness, critical analysis, creativity, ability to toggle between purism and pragmatism, cognitive psychology with an ability to think in other people’s shoes, attention to detail, knowledge of applied statistics, ability to detect patterns, knowledge gathering skills, ability to withstand pressure, good memory for reconstruction.

Core Skills Needed by Testers

  • First things first – a tester should be familiar with testing concepts. At a bare minimum, an entry-level tester should have a theoretical understanding of the testing concepts, test phases, SDLC, requirements, as well as basic industry, functional, and product knowledge. Like any other mature discipline, testing has solid literature, tools, and standards available. A tester can learn in an iterative structured manner based on their level of knowledge and skills by making use of the numerous learning methods and tools available (online, classroom, simulations) in the market. However, theoretical technical testing alone will not be sufficient, as the testers need to develop an ability to strike a good balance between perfectionism and pragmatism that comes with experience.
  • A professional tester should have a basic understanding of the following testing terms and concepts, depending on geography and industry. Some of the nice-to-learn topics are:
  1. Testing terms and concepts (TMMi, BS 7925-1, NIST 500-234 or similar industry/global standard)
  2. Test types, test phases, test prioritization, risk-based testing, regression test selection, test automation, test traceability to requirements, shift-left testing, model-based standards, application-based standards, application/product-specific functional standards, stakeholder management, requirements integrity levels, to name but a few
  3. Test policies (of testing organization, client firm)
  4. Test strategies (IEEE 1012), approaches and documentation (IEEE 829, ISTQB)
  5. Software engineering standards (ISO 12207, 15026, 15288, IEEE 1028, 1044, CMMi)
  6. IEEE Computer Society –SWEBOK focusing on software testing knowledge area and quality management, software (cognitive) ergonomics, and systems engineering discipline.

In addition to the basic knowledge, professional testers will be required to acquire knowledge in many areas within a program or project. These include:

  1. Vision, mission, goals of the program/project
  2. Customer experience and usability expectations
  3. Scope of the project/program/test phase
  4. Program or project test plan/test approaches
  5. Test phase plan (For system, domain/continuous integration/assembly/link, systems integration testing, user acceptance testing, operational acceptance testing, usability testing, etc.)
  6. Test process (generic or specific to a test phase)
  7. Defect management ( as per test policy or industry standard)
  8. Environment, data and support management, and test documentation
  9. Test tools and the skills to use them
  10. Test process and tester productivity improvement measures using tools, processes, and templates
  11. Lessons learned, dos and don’ts from past experiences in the firm, project, or earlier phases
  12. Challenge the constraints a team faces
  13. Navigate and understand the goals and mission of the project/program/test type
  14. Collaborate meticulously with the various teams/partners/vendors involved
  15. Manage risks and issues proactively with the project team(s)
  16. Provide excellent observation, recording, and documentation
  17. Be a cautious judge with critical thinking and the ability to be pragmatic when needed
  18. Be flexible enough to adapt to challenging situations without deviating from the original test strategy
  19. Handle ambiguity with a continuous learning mentality
  20. Take accountability for testing the relevant task in hand
  21. Treat clients with a ‘client first’ mentality

It is important for testers to understand the importance of their work – there are major examples of testing-related defects causing newspaper headlines, such as an aircraft being grounded due to battery issues, a financial institution facing IT integration issues, a floating point bug in a processor, a space shuttle failure, an airport baggage handling system going berserk, to name but a few.

From these categories, tests need to have three key inputs to succeed:

  1.  Project/program/solution context
  2. Technical/functional knowledge
  3. Personal experience and knowledge gathered These three key essentials give the testers a grip on:
  4. Stakeholder context
  5. Project understanding from a technical, functional, and business architecture perspective
  6. Testing methodology and good practices to follow
  7.  Soft skills needed to succeed
  8.  Pragmatism oriented Shortcuts and quick wins for effective testing

Once mastered, testers graduate to possess the skills (as a testing or subject matter expert) that allow them to:

  • Challenge the constraints a team faces
  • Navigate and understand the goals and mission of the project/program/test type
  • Collaborate meticulously with the various teams/partners/vendors involved
  • Manage risks and issues proactively with the project team(s)
  • Provide excellent observation, recording, and documentation
  • Be a cautious judge with critical thinking and the ability to be pragmatic when needed
  • Be flexible enough to adapt to challenging situations without deviating from the original test strategy
  • Handle ambiguity with a continuous learning mentality
  • Take accountability for testing the relevant task in hand
  • Treat clients with a ‘client first’ mentality

Requirement and Development Skills Mapped to the Tester’s Skills

Requirements determine the key testing scope of a project/program/ system/solution. Requirements are an integral pointer to the success or failure of a project or program. A quality business requirement will result in a high-quality development and a quality product/solution that meets the end user’s expectations. As per industry studies, a requirement- related bug found in production could cost between 10 and 100 times more than a defect found earlier in the life-cycle and as much as 10 times one found during the test phase.

Similarly, a design or architectural miss can cost 25 to 100 times the initial cost4 if found in production, and approximately 10 to 15 times more if found during the build/test phases. Based on information on the Internet and various studies, the highest number of delivered defects is requirements-related5 and better value is delivered by improving the efficiency of defect removal during the requirements phase.

This is largely due to a lack of a disciplined requirements management process. A disciplined, tools-managed process will help companies and programs prevent budget overruns by reducing the cost of poor quality. The tester needs to work with the key business analysts (BA), business transformation leads, and designers who have the business context and knowledge and help shape the final end user deliverables.

The quality of the requirements determines the quality of the end product and, hence, proper scrutiny of the requirements using static testing techniques such as reviews, walkthroughs, and inspection, visual validation (all manual), or using automated tools for requirements analysis for compliance is a must. Currently, standard tools are available in the market to convert requirements to test models and create test scenarios and conditions in an automated manner – covering boundary value conditions and negative scenarios. While it is good for testers to understand the fundamentals, it is equally important to check the output of the automated tools for consistency and accuracy. Experienced testers who understand the business function or industry very well can carry this out.

This article was originally published here by TapChief Expert, Kalilur Rahman.
To read the second part, click here.