Why waste time testing, developers can deliver 0 bugs…
While I have had this conversation at nearly every company I have worked at, I do not believe that testing is a waste of time nor do I believe that you can deliver 100% bug free code ever. I recently saw this great post which got me thinking: Top ten most infamous software bugs of all time.
- Yes, you can deliver software, even Enterpise software with no QA. QA is all about risk management, helping you to better understand what you are about to ship, where the risk areas are and gives you a certain sense of comfort. You can go without this if money is tight but there is major risk with this approach.
- Developers should just write code with 0 bugs. This is ideal of course, and, perhaps even feasible when you look at the world in the micro level of individual functions. Unit testing, of course, should help provide a solid level of comfort here but it is only a piece of the puzzle. The bugs that will bite you hardest come from the interplay of functions, from incomplete understanding of requirements, from poor performance. Individual developers focused on individual features will not catch these.
- Unit testing is the answer, get 100% code coverage and dump the QA team. Same basic problems as noted above. Good unit testing takes time and you are often spending higher $$s on the developer resources than on the QA resources to get the same level of coverage. There is definitely an argument that automation is worthwhile, which I completely agree with. However, it comes down to managing expenses and in most startup environments it’s cheaper and nearly as cost efficient to have a good QA team on board than to have your developers writing large amount of unit tests.
Thankfully, I have not had this discussion at my current company. We have a good QA team and developers that are investing a proper amount of time on their unit testing. We are also striving to live by my Engineering Tenants which keeps us focused on doing the right things.
Let me know what you think. Have you ever worked in an organization successfully delivering software that did not have a QA team?
-John
[...] testing is in fact waste, what does that mean for us poor testers! It can’t be true, or not always or not everybody will do the right [...]
Tester’s Dilemma | Techdoer Times
May 15, 2009
John
You have made good points in your post.
Keep in mind that defects in the use of
software may results from any of the following:
1. Application defects
2. Defects in the data
3. Defects in the underlying system or
components/ services used by the application
One would expect the developers to be
primarily concerned only about the application
defects (# 1 above) since the other types of
defects fall outside their purview.
Unit testing alone is not sufficient. Unit
testing is not suited to discover all
important defects in the following:
1. System performance
2. System security
3. System usability
In order to answer your question, an
organization that successfully delivers
working software does need QA. Building
software is a complex affair with many steps,
interactions, dependencies and individuals
(with different styles of thinking, skill
levels and knowledge levels) involved in the
process. Even if there is not a dedicated QA
team, some practioners have to take on the
role of QA to assure quality in the software.
It all starts with getting the requirements
right and clear and being disciplined
throughout the entire software development
life cycle.
Inder P Singh
Inder P Singh
April 15, 2009
[...] right schedule, at the right quality. This team believed in the nonsense that I discussed in this post. As I recall, the team never shipped a new product (after more than 3 years) and the team’s [...]
Should you “throw it over the wall” or implement QA hand-offs? « John Moore’s Weblog
March 19, 2009
Thanks Chris, I agree with you. Each member of the team has responsibility for product quality and use different approaches to find different types of issues.
-John
John Moore
March 2, 2009
Bug free code is only possible for the most simple of applications, and what customer would be paying for such a simple piece of code? Even if I could imagine bug-free code for software of any usable complexity, there are still the issues of interoperability as well as the other “ilities” (usability, reliability, scalability etc). Unit testing is not going to find these types of issues.
I see QA as complementary and additive to developer testing efforts, not as a replacement for them.
Chris
workorganized
March 2, 2009
[...] Why waste time testing, developers can deliver 0 bugs… [...]
Top posts for the month of February « John Moore’s Weblog
February 27, 2009
LinkedIn shifted this to the job area so I have copied/pasted the information here below:
Comments (16)
Ketan Trivedi
Sr. Programmer Analyst at iFuturistics Inc
I think that developers are writing bug-free code nowadays, because there is no job opportunity for QA this time.
Posted 15 hours ago | Reply Privately
Bashir Magomedov
Software Engineer at KLA-Tencor
I think that all developers should write test for their code, better prior writing code, and run all tests each time the intend to change it. But anyway QA are also required. They should perform manual or script-based testing, maintain errors database, write scripts and reports… all that is too time-consuming to be done by developers
Posted 13 hours ago | Reply Privately
Aamir Ansari
Software Programmer at Centrum Direct Limited
I think Developers can write bug free code,but there is a need for QA to test thoroughly because as a developer we think like a developer, but the user can’t think like a developer, so some one has to be there to test the program like a naive user do. So user can make a mistake that Developers don’t.
Posted 9 hours ago | Reply Privately
John Moore
VP of Engineering
This is one of those conversations that will always drive heated debate. Keep the conversation flowing here, on my blog, or just within your companies.
-John
Posted 8 hours ago | Delete comment
Guillaume Imbert
Software Engineer
A lot of people are confusing unit test, integration test and “poo” acceptance testing. Again a proof of this.
Posted 7 hours ago | Reply Privately
Radhakrishna Banavalikar
Consultant – Microsoft Technologies, MCP, MCTS
QA is more of a part of philosophy based on THE fact that: No programmers can call their baby ugly. It also serves a notion of professional approach to customers (in case of consulting and mainly product development firms).
Posted 7 hours ago | Reply Privately
James Knowles
Owner, Unwind Software Ltd
You will always need QA because developers will never ever be able to see what a user will actually do with there applications in practice. Classics of this are that a User will push buttons outside of sequence of what the developer unit tested for. For example push the View button first rather than the search button first then View. Not all testing can also be befored by unit testing some of the testing is just not possible and you need a range of people views in QA to make sure a product comes out tested fully. Qualtity products comes from a mixed team with different sets of skills.
Posted 3 hours ago | Reply Privately
Wiebe Tijsma
Software Architect and Engineer
Who needs a VP of engineering when your developers could easily lead the company?
Posted 3 hours ago | Reply Privately
Steven Lowe
IT Consultant, Architect, and Coach
quality assurance should be built into the development process; in the “waterfall” methodologies this was the function of the QA team or department, but trying to instill quality after the software was written is like trying to put toothpaste back into the tube – it’s possible, but tedious and wasteful. Much better to build quality into the process from day one, using test-driven development (TDD) et al.
Posted 3 hours ago | Reply Privately
Lance Robinson
Sr. Systems Analyst at Frontier Communications
QA is a good safety net to make up for the operational gaps, or inherent ‘blindness’, that exists between different depts/phases.
While my unit may perfectly fit the requirements presented me by the architect, who’s to say that he/she did not miss something from the req analysts? QA is less about individual developers/components and more about assuring the results of an entire project. It ‘tests’ all phases of a project from conception to delivery and is not ‘targeted’ at developers, even though they are usually the ones that must clean up any ‘mess’ found during QA. I am curious to see how many defects or issues are usually found via QA that are NOT attributed to coding errors (or developers), as opposed to the number that are?
Please don’t confuse this with unit testing.
Posted 3 hours ago | Reply Privately
Wesley Wilson
Development Manager at ChanneLinx, Inc
John, as you say in your blog post, totally bug-free code is impossible in an enterprise application. Unit testing with good code coverage eliminates the majority of bugs before they ever get checked into source control. And automation of those tests catches most that do get checked in.
I think automated unit tests are far more cost effective than having a QA team deal with the bugs that tests can easily catch. Much of the ROI of unit testing comes from the better design that unit tests encourage.
No unit test can determine if the UI is intuitive or if the feature being tested meets the customer’s expectations. In my opinion, automated unit testing frees up a QA team to focus on real-world testing with non-trivial data and usage of the application, as well as preliminary acceptance testing.
Posted 3 hours ago | Reply Privately
John Meyer
Sr. Software Engineer at GolfChannel.com
Unit testing/TDD helps the developers ensure their code does what they want it to do and that they can refactor confidently because tests will fail if they break that functionality. QA’s job is to ensure the product does what the specs require and to be another layer of integration testing.
Posted 3 hours ago | Reply Privately
Christopher J Tallos
Senior Software Engineer at National Marrow Donor Program
Quality of a project starts with good database and OO design coupled with unit testing and followed up by Quality Assurance testing. I’ve worked at too many companies that spend time on a roller-coaster ride going round in circles on projects. The amount of time and money wasted on Development and QA to try and fix things after the due date was enormous. If you ever see yourself in this situation you need to take a close look at the architecture of your application and question your architects and developers and never question if QA is necessary, because it is in too many situations.
Posted 3 hours ago | Reply Privately
John Moore
VP of Engineering
I am glad that I’ve been able to put out a question leading to this much conversation, keep it flowing. If you have a chance to leave comments on my original blog posting please also do so as I’d to have my other readers benefit from this conversation.
http://johnfmoore.wordpress.com/2009/02/24/why-waste-time-testing-developers-can-deliver-0-bugs/
-John
Posted 2 hours ago | Delete comment
Brian Owens
Technical Recruiter at Softworld Inc
There aren’t any QA opportunities?? Could have fooled me.
Posted 2 hours ago | Reply Privately
George Hassan III
Sr Software Engineer at iConnect
Trying to define yourself is like trying to bite your own teeth.
-Alan Watts
Trying to QA your own code is like trying to Define yourself.
-Me.
Really developers know way too much about how it was designed to work. You just can’t undo that properly and apply how it will work / be used.
Plus all the other reasons listed above.
I’ve seen some that can do both jobs, but oh so rarely. And never at the same time.
Posted 58 minutes ago | Reply Privately
John Moore
February 25, 2009
QA is important. Developers would code the application but would not do justice to the requirements while Unit Testing. They would be too code-concentrated rather than business concentrated. Though as technology advances and Test Scripts and cases get automated through automation tools we would see a decline of the tribe.
The business users are more confident on an Application when it is QA approved.
Mustafa Balsara
February 25, 2009
QA is a good safety net to make up for the operational gaps, or inherent ‘blindness’, that exists between different depts/phases.
While my unit may perfectly fit the requirements presented me by the architect, who’s to say that he/she did not miss something from the req analysts? QA is less about individual developers/components and more about assuring the results of an entire project. It ‘tests’ all phases of a project from conception to delivery and is not ‘targeted’ at developers, even though they are usually the ones that must clean up any ‘mess’ found during QA. I am curious to see how many defects or issues are usually found via QA that are NOT attributed to coding errors (or developers), as opposed to the number that are?
Please don’t confuse this with unit testing.
Lance R.
February 25, 2009