Why I Love Software Testing
I spent the first few warm days of summer in a cold, empty, and dark computer lab. Worse than just being nerdy, I was stressed. Our senior project had us partnered with Lockheed to design a magical computer chip — one that detects errors in messages and that can even correct those errors on the receiving end without asking for a re-transmit. This is super important when the message can be ‘scrambled’ by interstellar radiation or other noise, and it can take minutes to reach the other end (Spaceships) and or you don’t have time to resend the message (Missiles). As we turned in our completed project, our advisor mumbled something about him taking this back to the company to be used in things like missiles and satellite communication. I freaked out.
What could go wrong? Um, everything. I suddenly had visions of satellites running into each other and crashing through the sky. Watching a mars rover take the wrong turn into a ditch. Rockets told to call off their attack only to move to a new target. Lives, treasure, and global progress could be at risk if our little chip had a bug, a flaw, a mistake we didn’t know about. Even if no one else knew, I would know. Worse, I might never really know if it was my fault. So, I did what every engineer who thinks about the real-world, customer, users, impact, correctness, risk does — I started thinking about testing the chip.
I realized our little team’s testing to date had been little more than simple probing. The chip worked using esoteric math called Galois Field Arithmetic to produce what is called a Reed-Solomon encoder. This algo is now used in common flash chips and memory so. you never lose a bit. We spent most of our project learning the concepts, then like most computer projects, rushed it through design and production. Our mindset was typical of most software engineers, testing little pieces as went along, trying some edge cases, but primarily focused on two things: Hoping it worked and getting to all the other work in other classes.
Just as I was realizing the constant toil of University was over, and looking to take a few days off before my first job at Microsoft, I suddenly felt dread realizing all the tests we hadn’t performed. Think of all the pending disasters. I knew I had a few days, maybe a week, to see how bad the situation was. I did what I had to do and headed to the lab. Over the course of several days, I intuitively taught myself the basics of a test framework, generating coverage, and risk analysis. I spent days generating tens of thousands of different messages with varying errors and corrections. It all worked. No bugs. Whew! The relief I felt was physical. Testers don’t believe something works until they know it works. No one else ever knew I tested the chip exhaustively — but I did. The endorphins from the risk of the chip failing, and to the rush of knowing it would work, helped form my love for software testing.
The tester’s mindset is different than the developer’s mindset, testers are folks that like to find problems in software — they are a different personality. The Testing profession attracts creative folks that like to question assumptions, competitively try to find bugs before end-users. Testers like to play games — at testing conferences, they are playing new games and even creating new ones. Testers are generally fun, ego-free, and honest folks that are great to know and hang out with. The profession attracts the type of people I like to be around.
A geeky aspect of being a tester that I love is getting early access to the future — seeing the early versions of software while it is still vulnerable and often secret. I remember working on the early Google ChromeBooks, and having a meeting where a key early partner showed us their prototype hardware. The device (yes, a laptop) was literally brought in with one person handcuffed to the case, and two security folks. They locked us in the room, held the device aloft for a few seconds, placed it back in the case, then quickly exited the room. That’s fun drama. Even more fun was realizing the 7th prototype was in my backseat while picking up a friend from Microsoft — driving through MSFT campus with a legit Windows competitor from their Rival Google, I felt a little bit like a British spy. Crushingly, that prototype was soon stolen when someone broke into my car a few weeks later. Most likely two teenagers had the least useful laptop in the world but I could still imagine it was part of a larger plot. Today, my kids use those ChromeBooksk to do all their schoolwork. Access to early hardware and software that will soon change the world is another reason I love testing.
I remember being at one of those fancy evening dinners with startup founders and investors circling, most wearing the types of clothes I don't even want to have in my closet. The bait to get all the founders to show up wasn’t the food — it was a few of the cool guests that were also mingling. One one of them was the head of DARPA (Defense Advanced Research Projects), who focuses on problems so difficult that researchers don’t even work on the them without incentives. On the deck overlooking the city, I heard she was hoping I’d show up, because the hardest of the hard problems for the DARPA team was worrying about was the ‘testing’ of all these new AI-based systems. Testing really is the most difficult problem in Computer Science and I love working on challenges that difficult and important.
Loving hard problems, especially problems at scale is compelling. Think about it, with some heuristics (aka PageRank, et al), you can build a search engine. But…how do you know it is working correctly? If you had a testing system that could produce the correct answers to a user's query — why wouldn’t you just ship that system as the search engine? Testing search is a huge problem and a whole field in itself. I gravitated to working on both Bing and Google search engines because testing those systems is a hard problem. Funny note to give credence to my test-loving and the test-centric universe, when I landed at Google I realized that all the search engineer’s titles weren't ‘Software Engineers’ like they were at Microsoft, they were called ‘Search Quality Engineers’ — they were testing and focusing on quality all day. Testing is the main activity when it comes to search engines, and search engines are arguably one of the most difficult problems at scale today, but few realize it is all about testing. Perhaps even larger in scope is testing web browsers. I had the opportunity to test Internet Explorer 4 DHTML back in the day, and later work on the Chrome Browser/WebKit. What a wonderfully large problem — verify that all the HTML and code on the internet works and will render in the browser! What a great, endless testing problem. You’ll never be done as the web keeps growing and the browsers keep changing. Gotta love testing — perhaps the largest job in the world.
Many people leave the software testing world at some point in their careers. They either start as a junior engineer and relegated to simplistic testing and get bored, or they aspire to be software engineers or product managers. A buddy of mine got so sick of testing he left the industry to be a park ranger and live almost completely off the grid. What continues to motivate me to focus on testing, and love it, is the impact we have on society. Every year, more of our lives are online and dependant on apps; food, travel, medical care, entertainment, even remote work is all powered by software. We have all experienced times when that software didn’t work correctly. Better software means better lives for real people. The best thing is that a software tester’s work scales. Testing a physical good for defects might save a single user from an issue. Testing a single app with say 1 Million users means a tester finding a single bug and getting it fixed might help tens of thousands of people. Software Testing can literally be massively rewarding.
Shhh, don’t tell anyone, but testers actually have some of the best lifestyles too. Not only do you get to play on computers all day, but developers have a schedule and a deadline to complete adding that new button or image to an application with project managers asking them for status every day. Testers on the other hand — decide how many tests are enough, how fast to execute them, and decide when they are done. The key is no one actually knows how many bugs are in the product, or how many tests there should be, so if a tester doesn't find a bug that day — maybe there just wasn’t a bug in the area they were testing or maybe they spent a bit more time learning. Testers have lots of wiggle room to think, be creative, and explore on a daily basis. This flexibility has enabled me to experiment and learn new skills over the years — another reason I love testing.
The testing community can also be diverse, even small teams are often international teams. In fact, the best test team needs to be diverse to understand and ensure the software meets the world’s needs. The testing community is a global one, and software crosses cultural and language barriers — even the software we test sometimes helps break those barriers. I love being a software tester because I get to meet people with different perspectives than I would in many other related fields.
I love being a software tester, but don’t share all the reasons, lets keep it our little secret. The product managers can stress about schedules and resourcing and design. The developers can stress about adding more buttons to web pages and fixing bugs. All the while, Software Testers sit in a sweet spot of flexibility where we can learn, live on the cutting edge of technology, and work with generally interesting and nice people. Oh, and the salaries aren’t that bad either!
— Jason Arbon
CTO/Founder, test.ai
