ML, Big data, QA strategy and organization in 2019: the interview with an expert

171 1

A few days ago, I went to visit the mortgages ING team from IOVIO, our partner in QA consultancy. That was my chance to interview Herminio Vazquez, a Principal Consultant. He is now leading a mortgages data store project within the Mortgages NL tribe.

Mortgages ING team from IOVIO
Mortgages ING team from IOVIO, with Herminio Vazquez in the lower left corner

I met Herminio the first time last year and all I know about him so far is:

  1. He’s passionate about big data.
  2. He’s excelling in using very high impact similitude (keep reading if you want to understand what I mean).
  3. He speaks Italian, as he once held a project there.
  4. He’s a great dancer – not much a surprise for a Mexican.

Finally, I had the opportunity to interview him about the QA and test automation trends 2019. That was a nice chat. So nice that we decided to leave you below the recorded full interview you can listen to.

Otherwise, you can keep reading and I will guide you through the major takeaways of this conversation.




MARTINA – Since I started working as a marketer in the world of QA, last year, I’ve always wondered why some people could choose Quality Assurance as a career. So here comes my very first question. How did you end up in the world of QA? Was that your childhood dream?

HERMINIO – (laughing) No! When I was a child, I dreamed of becoming a Marine, then a lawyer, then an archaeologist and even a mathematician. Then my father said, “But you also like to eat, right?” (not many jobs at the time for Quantitative Degrees… I guess) and decided to study Computer Science.

I started my career as an architect: I spent my time doing code review. I grew up in Mexico and moved to the U.S. in 2000, helped by the fear of the collapse of the world due to the change of the century. A phenomenon for the IT museum known as the Y2K.

Then I moved to General Electric Nuclear Energy and that’s where I guess my passion for quality started: working with system critical applications pushes people to work more diligently on quality controls. In this industry understanding the implication of software development becomes critical, and the level of testing has to be very high, there’s no compromise!

After that I completed many consulting assignments in a variety of industries around the world. During this experience I had the chance to meet, and work together with very talented professionals that have enriched my career and helped me to be where I am today.

MARTINA – Means you chose QA because you are a perfectionist?

HERMINIO – I wouldn’t say so. A lot of people have the feeling that everything has to be perfect. I think it has to be a trade-off: you can just aim for software with less failure. And that is what testing does. Whoever says that you can get the system out of defects, he probably hasn’t seen enough systems. They just need more observation.

MARTINA – So “less failure” for you is the role of QA nowadays?

HERMINIO – It is one of the roles of Quality Assurance, indeed. But most important our mission is to spread even more awareness about the benefits of working in a high-quality environment.

Software quality is too often left to the end, still. And it’s hard sometimes to quantify how quality can help you to save money and resources. The better we can prove how quality influences a company reputation and helps you gain a competitive advantage, the more value QA will bring to an organization.

Systems nowadays are also getting very complex and interconnected: another role of QA is to reduce complexity.

MARTINA: What is the key to success for a testing strategy, to you?

HERMINIO – The success of any quality and testing strategy relies on 3 pillars:

  1. Test Cases: The ability to reproduce realistic scenarios. Those that may influence the business in many dimensions, including Revenue, Quality, Reputation, Image, Time, etc.
  2. A Reliable Test Environment: you need an environment able to reproduce those scenarios close to reality. Meaning, a solid strategy to manage and adapt to change in environments. Environment management is one of the most susceptible areas to compromise your test results. With containerization this dilemma is being translated to a new paradigm that sees environments like cattle, not like pets.
  3. Good Data: Data-driven testing that cares about the data lineage and governed in a data store. I have countless experiences about not having the right inputs to produce the right outputs, and that hasn’t changed over the years. New challenges like Data Science or Machine Learning have strengthened the importance to have a solid data foundation first. To me, data practices help organizations to put more diligence and controls on testing. That is the reason why I lean toward the embedding of data practices alongside testing strategies: so that the testing industry can benefit and prevent the common biases of pure feelings.

If you cannot put together these 3 factors for your testing strategy I think you are wasting your time. These 3 things are pillars for you to come out with good results.

MARTINA – I guess implementing this 3-factor strategy comes with many challenges. What is the biggest challenge of Quality Assurance?

HERMINIO – The number one challenge is to step too much into one of the 3 areas above. We see companies that spend months, or even years, working on an automation framework, or the adoption of a new tool. Without controlling their data, or putting effort on managing their environments or having a better configuration management strategy.

We need to integrate the 3 pillars together, see them as one thing only. Thinking, for example, that the environment is not something you should worry about can lead to wrong assumptions. Well, you should worry about it. It is like when you want to lose weight and you say: “I want to go to the gym but I will carry on eating pizza”.

Another challenge, nowadays. is data dependencies. When you work in a complex environment with multiple systems, where more people and areas are involved, you need to make sure that your change doesn’t collide the change of anyone else. That creates data dependencies.

10 years ago the testing industry was focusing more on virtualizing and creating mocks for this data. Nowadays the real challenge is the orchestration of data across systems so that if something changes in a particular area, it propagates across the entire landscape. In this way, you can follow up those changes propagating across all other systems.

MARTINA – QA organizations have also changed a lot lately to keep pace with the new trends. What do you think is the ideal team?

HERMINIO – The structure of a QA organization has changed drastically in the last years. In the past there used to be something called the Center of Excellence: every industry wanted to have a group of super cool guys who knew all the tools and testing requirements. One team will serve all.

But the CoEs were not agile: what we saw is a dismantling of this structure towards the squad, a smaller team. More quality driven people were embedded in these squads. Testers started to collaborate with developers on the development cycles of the new features. And that’s where the battle of “it works in my machine”, “well, it doesn’t work in mine” came to an end.

The team that I see as the future of QA industries is the following:

  1. A DevOps engineer.
  2. A feature engineer, or a developer.
  3. A quality-driven person.
  4. A data engineer.
  5. A product owner who makes sure you don’t derail too much from the objectives of the business when developing new features.

MARTINA – What are the key skills required to succeed in the world of Quality Assurance?

HERMINIO – Two skills are fundamental to me:

  1. Communication: communicate in an effective manner.
  2. Contribute by thinking different: you cannot expect different results by doing the same all the time. People start to understand that the attitude of “everything fails” is wrong. I’m going to gain more by saying “how can we fix this?” or “let’s change this, let’s contribute that way”.

MARTINA – What is your number one recommendation to QA managers, to keep their team and work agile?

HERMINIO – Always try to balance your decisions and evolve!

There are 3 types of managers:

  1. The pragmatic. The one who goes for less complex things to meet the goal and make sure that the system works. This group tends to see immediate solutions, sometimes overlooking the impact in the long term.
  2. The situational. This leans towards pragmatism and bureaucracy arbitrarily, it changes or persuades people depending on the temperature of the conversation. Is hard to understand them at times, because is depending on the situation how they react.
  3. The purist: the one going through the theory part, playing by the book. Is the typical manager that wants to reinforce methods or processes, without accounting the context of the problem, they tend to fall into the “hammer purchase” – when you own a hammer everything has the face of a nail.

The question is: “What kind of manager do I want to be?”. You don’t have to be only one of these, you can be in any of the buckets at any time. It’s more about when to be in the right bucket that makes the difference. It’s to play all these cards in balance what, I think, is the key for good management.

More importantly, don’t always play the same game, try to change it! There’s a very nice book about the football coach of Barcelona, Pep Guardiola. He had a winning team and, as everyone knows in sport, if you have a winning team, you don’t change it: you don’t sell the winning horse!

He, instead, came with a different approach: “If I keep doing the same I’m predictable and everyone will know our techniques and strategies, or where to find the players. I’m winning but I also need to evolve”. Winning is not the difficulty, stay winning is the real challenge!

MARTINA – We hear a lot these days about Machine Learning? What is all about, and how it is influencing the testing strategies?

HERMINIO – I think there’s a lot of misunderstanding on what machine learning can do for an organization: the whole idea of how algorithms can help us with decisions is absolutely fabulous. With the amount of data that we produce this is a game changer: to start making machines do the things that we are not good at doing.

How we employ ML and the ethical aspect behind it is still something companies are not fully aware: don’t make algorithms that you don’t understand. If you are going to give a mortgage, for instance, and you have an algorithm that decides how to calculate the risk, don’t make it ethnicity based. How do you avoid this? Making sure there’s always traceability and transparency because sometimes when you have these complex mathematical abstracts, it is not easily reversible.

In terms of how ML is influencing testing, you start hearing concepts like cognitive testing or robotic process automation. All these new buzz words are coming and going like waves. Once the high wave will go, we will stabilize on the profitability bar and we will continue consolidating good use cases that prove good value for your testing strategy. I think for ML we are not there yet.

MARTINA – One of the 3 pillars you mentioned above is data: what are the benefits and challenges of implementing a data-driven strategy?

HERMINIO – There’s absolutely no question about the benefits of operating with a large quantity of data: the more people use data for making decisions, the better it is. Delegating those decisions to machines that can identify what we can’t, is a big revolution in the industry.

Still, companies are not fully aware of the implication of using personal identifying information: what is that, how do we store it, how we handle and process it?

Another challenge is the lack of power in computer processing for a huge amount of information. Just as an example, I recently collaborated with the polytechnics Zurich on bioinformatics. These guys are geniuses, they work with microscopes and in super sophisticated labs. They take 400 pictures of a cell per second. And each picture is 16 megabytes, so it’s definitely a huge amount of data. The fun part is that they don’t have the computing power to process all these images so they have to look at them, one by one, to recognize fluorescence within the cells. And these guys are working on resolving problems like the cancer spread in the world!

Managing data in the proper way is still a big challenge across the industries, which are not fully aware of the benefits and don’t have enough processing capabilities.

MARTINA – To me, another huge risk we have when relying on data is misunderstanding it: how can I make sure data is actually helping? How can I be confident enough to know that data is not misleading our decisions?

HERMINIO – First of all, you need to know what you would like the data to help you with: answering is not the difficulty, asking the right question is.

Then it’s a matter of balancing the right mix of skills within a team which is collecting, elaborating and interpreting data. Do I prefer people with domain expertise or people without knowledge? It’s always good to have people who don’t know about the domain: they will ask questions and will help you to confirm or reconfirm your knowledge and the things you gave for granted.

Still, you need people with domain knowledge who can make a good interpretation of the data. And then you also need someone who can make the magic happen with data: they can slice it, cut it, dice it and process it by creating algorithms.

Last, but not least, you need people who can make the data look pretty. Here is where visualization and info graphics come in hand: they leave you with an interpretation, not with cognitive dissonance. That is the real trick: to convert data into information. And this transition of turning data into information requires a lot of skills, including artistic talents.

So, again, you need the right mix of ingredients within your team. The better the combination of all those ingredients, the better you will feel the data is not misleading you.

MARTINA – And how about you, as an expert of big data: what role are you playing in this mix of ingredients?

HERMINIO – (laughing) Depending on the situation, I play in different buckets. Just like as I said for the purist, situational and pragmatic.


And that’s where the interview came to an end. I’m blasted by the great takeaways I got from this conversation. The most important one to me is no matter the type of industry you are working in and your area of expertise, knowledge is nothing without the right approach and skills.

Communicate effectively, be flexible, think out of the box (which box?) and you will make your knowledge valuable!

1 comment

  1. Awesome!

Leave a Reply