Imagine a story. A man has been ignoring his health for years. Instead of doing regular check-ups, eating healthily and exercising, he opted for the cheapest self-medication he could find. After ignoring all the symptoms of poor health for a long time, he was finally hospitalized with serious problems that needed expensive treatment with little to no hope for full recovery. Sound familiar? Unfortunately, this is the story of the majority of businesses today. For years companies have been growing technical debt and cutting testing budgets with no clear plan of how to make up for these conscious poor judgments in the long run. With wide adoption of cloud and mobile a few years back, those bad habits became even clearer, making some of the platforms so bad that they simply could not be migrated. Additionally, some bad habits were pushed into a new era of development without companies understanding that today the cost of quality assurance and control is higher than ever, so such negligence is no longer tolerated by customers.
Now with tons of services and apps rat-racing in the same business, there is no room for mistakes and expectation on zero defects has become the reality companies simply have to live up to.
That's why year by year companies are growing their testing budgets to numbers not seen before: up to 26% of the entire budget, according to the World Quality Report 2014-2015 from Cap Gemini. Unfortunately, this rise comes with a lot of frustration since these investments are not always resulting in higher quality. Here are several reasons for that.
Bad Habits Die Hard
It’s not easy to start eat healthily and exercise after years of developing poor daily choices. If companies are used to cutting on test cycles, not documenting any test artifacts, or simply doing ad-hoc testing with no formal process executed by a hand-full of testers, then transition with no proper guidance or training would be close to impossible.
There Is No Silver Bullet
Simply adopting Agile cannot fix all the production problems. Simply automating all the regression testing cannot ensure higher productivity. Simply increasing testing staff cannot increase the level of quality assurance and control. It is important to adopt all the best practices with a proper understanding of what you want to achieve and how a specific tool can help you with that. There are no fit-all best practices; there are best practices for specific cases.
You Can't Improve What You Don't Measure
Calculating return on any investment, especially the one into quality assurance and control, should be properly evaluated. A lot of frustration comes when companies invest in a testing practice, yet seem to get little to no improvement, when the testing practice knows for a fact that the money was well spent. Test leaders need to learn how to justify their spending for short and long term wins. This needs to be done by showing measures of product and process quality that makes sense to the whole company and is correlated with customer feedback.
Transition Is an Era, Not a Phase
We are used to loud promises of quick fix and immediate results, but do they ever actually bring quality results to sustain? When you make decision to change organizational behavior, it takes time for your team to achieve impressive results. Do you want to get skinny fast, or do you want to be healthy for life? The point is that you can become healthy only by clearly defining what that means for your company, as defined in point three above. Otherwise you will see your teams “improving” for years with no visible results.
In its turn, 2015 will be a very exciting but challenging year for test teams with the following trends to keep in mind:
TREND #1. Test Center of Excellence
Last year most consulting companies were coming up with services related to “Quality Assurance Consulting" or “Test Center of Excellence”. Our company wasn't an exception, so here’s what we’ve learned so far. Test Centers of Excellence promise consistency, governance, and structure to improve efficiency from project to project by gathering all resources, knowledge and skills in one place, thus allowing you to manage all your resources more efficiently to ensure high quality assurance and control across applications. It sounds rather clear and straightforward, however many companies struggle to adopt this best practice due to the following reasons:
Organizational structure
Not all the companies are structured in a way that allows full centralization of competence. Especially companies that come from a history of mergers and acquisitions, accompanied by a zoo of internal tools and approaches, will face a great challenge on defining the best way of structuring their centers of excellence. In this case, it makes more sense to analyze the discrepancies first, define a desired “to-be” process, and only then proceed to centralization in order to avoid unnecessary conflicts.
Misuse of talent
In order to effectively manage resources across the portfolio and achieve efficiency and quality, look for proper people to start with. What we sometimes see is a crazy disproportion some companies have in their resourcing. Keeping too many testers (more testers than developers) doesn't bring more quality to your product, but in fact the opposite – it mainly brings tons of duplicated work in regression testing and procrastination. On the other hand, a more common case is underinvestment in either quantity (very few testers who are focused on too many projects at the same time), or quality (untrained or immature people being responsible for test strategy). Another wide spread situation is misuse of testers’ talent. Many companies think testing doesn’t require much work and so have testers work part-time as BAs or Configuration Managers, which affects a professional growth of testers and thus the quality of their work.
Over-engineering and fast results expectations
It’s a fact, we all tend to over-analyze and over-engineer when it's not needed. Test Centers of Excellence may seem very complicated at first, but they are also highly customizable to a specific need. There is no necessity to copy-paste a case study from the Harvard Business Review, it’s better to define your “as-is” and “to-be” state as clearly as possible and transformation will no longer seem such a hard and unreachable thing. Also you should understand that a Test Center of Excellence is a long run investment, the results of which can be seen only after all the components are stabilized and agreed.
TREND #2. Test Automation Myths
Test Automation was and still will remain a big thing in 2015. Well defined test automation frameworks, which provide great productivity and efficiency increase with a minimum of effort, are a very tempting promise for any business. Yet, after so many years, we still hear companies striving to get into test automation for all the wrong reasons: 100% test automation and close to zero manual testing efforts. Many companies will still need to learn to set clear reachable goals for test automation, in order to use it as a great ally for manual test teams. Many will learn that in cases where you do not have a strong test automation competency in the company, it is better to get this expertise somewhere outside, at least for the critical initial stages of framework definition, in order to obtain strong and tangible ROI.
Another open question in 2014 was if manual testers can automate. Well, it is both true and untrue: true because indeed, anyone can automate after good hands-on training (don’t waste time on general training or conferences, it’s better to hire an expert to train your employees to automate existing test cases with a selected tool); untrue because everyone automates based on their competence – a junior test automation engineer cannot build a solid test automation framework.
TREND #3. Making it Work in Agile
Migration to Agile is still a hot topic, as well as how to switch testing from the usual waterfall model into agile thinking. The main focus is usually test automation and communication, but I think we will have more innovation in the test planning and test design area in future, including next year, in order to get better coverage, faster and easier to maintain. Personally I think, mind maps and solid checklists are under-utilized by test teams who tend to either use high-level scenarios or too detailed test cases.
TREND #4. Quality Should be Quantitative
The cost of mistake could not be higher than now. In order to make firm business decisions, companies want to know the real quality level of the product at any time. Everyone is looking for dashboards that will help answer the ultimate question: how good/bad is my product today? Yet, it's still not easy to build a metrics program because of a range of common challenges that companies will be solving this year:
- Set of metrics to use
- Baseline definition
- Technology selection for tracking and monitoring (the more people “touch” data, the less reliable and unbiased it becomes)
- Lack of formal training to analyze metrics and their correlation
- Wrong behavior model for decision-making based on metrics outputs
- Building a governance model that ensures and motivates people to gather, track and use metrics
- Do we buy or build a metrics dashboard to fit our needs?
TREND #5. Maturing Process for Test Teams (to realize how to test non-functional requirements such as security and performance)
Non-functional requirements testing is no longer a full responsibility of an engineering team. Performance testing, as well as security testing, turned out to mean much more than was initially intended. Not only do they require testers to obtain new professional knowledge, but also in some cases a formal certification (e.g., security analyst). Companies need to face the truth that formal performance and security testing is way more extensive and complex than it was considered to be for many years. In 2015, some companies will need to get additional training for their teams, while others should acquire this skill altogether.
TREND #6. Prepare for Big Data and Analytics
Testers need to expand their knowledge in Big Data and Advanced Analytics. Currently, these technologies are a playground mainly for data analysts, data scientists and architects in R&D sections of any company. This trend is so huge that sooner rather than later test teams will need to verify and validate how well these models work in the real world.
Conclusions
With quality assurance and control being a mounting concern, testers need to start taking more ownership and leadership in business. Many transformations that companies are undergoing today will need a professional, fast and clear support from testers to know exactly how well quality assurance and control are conducted.
